Raised by her mum after her parents split up when she was a small child, she later spent time with family friends, foster cares and in children's homes when her mum Anne, then battling a serious illness, was too unwell to look after her.
Money was always very tight, too.
Despite all that, Ferguson fought for a better life. Her dreams of becoming a singer were put on hold, though, when she became pregnant in her teens and then a single mum-of-two after her relationship with the father broke down.
She qualified as a legal secretary, but her lifelong dream of being a star bubbled on and she made several attempts to get on various talent shows - one of which saw her whole family saving to fund a trip to the US.
The determination paid off: in 2010 she finally got on to The X Factor and, despite finishing as runner-up, a hugely successful album soon followed.
It wasn't just the chance of releasing a record she was so grateful for, but the opportunity to pay back her relatives and give her children a better childhood than her own. "I pay my mum's rent and 'leccy bills now," she says proudly, in her soft Liverpudlian accent.
Unfortunately, The X Factor and subsequent No 1 album, Heaven, was only the start of another set of problems.
"I started attracting really bad people," she says. "I came off The X Factor and had all these clingers-on, trying be mates with me. But because I was so trusting and shy, when people said they wanted to be my friend I went along with it, and they ended up completely using me, really."
Ferguson remains remarkably calm when talking about them. "They attempted to ruin me, basically," she says. "I'd get threatening mail, to the point where I was petrified in my own house."
Ferguson's now rid of these "strange characters" and her former management company who - she claimed via Twitter last year - overworked her to the point that she didn't see her children.
She's locked in a legal battle with her ex-managers. When they're mentioned now, she trots out a line she's clearly had to learn, parrot fashion, to avoid speaking about the issue with the same honesty she does everything else: "For legal reasons I cannot talk about that. I am bound and can't say a word."
She was going through all of these ordeals while writing her new album, Freedom. The title is perhaps a hint at how she felt previously - and she decided to work through her problems by heading to the studio.
"It was just about working out who was who in my life, and deciding that I wasn't going to let these negative people beat me."
She says it was her naivety that got her into trouble in the first place, and admits that she had no idea just how successful she was when Heaven was selling so well. "I thought everyone sold a million records," she says, although her main crime was not realising the record industry was a business, and sometimes a fickle one.
"I thought everyone loved everyone, and I quickly learned that wasn't the case," she says. "It's all about making money and being well-connected. It's cut-throat. I didn't know that because I'd come from singing in my bedroom."
Ferguson's keen not to let these negative experiences change her, although she admits that now when she meets people, she tries to figure out their motives and decide, within an hour, whether they're for or against her.
Freedom deals with many aspects of Ferguson's life, in particular being unlucky in love. The song Fake Smile, particularly, is directed at some of the men she's dated.
"You know when you think something's serious, but you're just a bit of fluff to them? Well that song's about loving a boy that doesn't love you back," she says, before adding she's now in a relationship that's made her happier than ever.
"He's lovely and he looks after the kids," she says, visibly swooning. "We've been together a year now and I've learned a lot, mainly that love and happiness can be different, and there are two types of love: the love where you get infatuated with each other and you get butterflies, which is always a bit unhealthy, and there's a practical love, where you care about each other deeply, have the same outlook and see yourself marrying them.
"I think I've found that with my partner now, and I'm over the moon."
n Rebecca Ferguson's second album Freedom is out now. She will perform at the Usher Hall in Edinburgh on March 22.