The couple - who stepped down as senior royals earlier this year - had decided to move out of Los Angeles and into the quieter neighbourhood, but locals are said to be displeased about the move as paparazzi rush to snap a photo of the royals.
It is believed there are helicopters flying over four times a day to try and get the perfect shot of the couple and their 15-month-old son Archie while paparazzi are also on the ground, nearby their home and a local shopping centre.
And the new residents are also said to be attracting tourists to the area, with local shop owners being asked if they've seen the Duke and Duchess of Sussex, TMZ reported.
Meanwhile, a source previously claimed that Prince Harry "hated" Los Angeles, and is much happier since relocating out of town to Santa Barbara.
An insider said: "Meghan visited Montecito in her teens and fell in love with the picturesque scenery and stunning architecture. Moving there was always an option, but to begin with, she and Harry wanted to give Los Angeles a shot.
“Unfortunately, Harry absolutely hated it - the timing was so wrong amid the [coronavirus] pandemic and they lacked privacy.
An added bonus is that Montecito is only just over an hour's drive from LA, which is where a majority of their work is based, yet far enough away to escape the crowds, paparazzi and tourism in Hollywood."
The Duke and Duchess of Sussex were warned to be wary of mudslides hitting their new nine-bedroom abode after former owner Terry Cunningham - who bought the land in 1999 and spent four years building the house - revealed that just two years ago, there was devastating mudslides that came within just 200ft of the estate.
Terry - who sold the house in 2009 - said: "We sure as hell didn't think about it [mudslides] when we were building but certainly it's something I would be thinking about now.