The US president called the late South African president "a giant of history".
And he spoke of the "heroic" life of the anti-apartheid leader describing him as the "last great liberator of the 20th Century".
Mr Obama was addressing the thousands who had gathered for the memorial service to Mr Mandela in the rain-soaked FNB Stadium in Soweto.
He compared Mr Mandela's actions to those of Gandhi, Abraham Lincoln and US civil rights leader Martin Luther King.
But he also warned against viewing Mr Mandela as detached from normal life.
"He was not a bust made of marble, he was a man made of flesh and blood," Mr Obama told the crowds in the stadium, including leaders from more than 90 countries.
"Around the world today we still see children suffering from hunger and disease and we still still see run down schools and we see young people without prospects for the future," he said. "Around the world today men and women are still imprisoned for their political beliefs and are still persecuted for what they look like and how they worship and who they are.
"That is happening today.
"There are too many people who happily embrace Madiba's (Mandela's) legacy of racial reconciliation but passionately resist even modest reforms that would challenge chronic poverty and growing inequality.
Mr Obama added: "It was precisely because he could admit to imperfection, because he was so full of good humour, that we loved him so."
"He was a son and a husband, a father and a friend, that is why we have learned so much from him and that is why we can learn from him still.
"Nothing he achieved was inevitable, in the arc of his life, we see a man who earned his place in history through struggle and shrewdness and persistence and faith."
Mr Obama's speech came after he shook hands in the stadium with Cuban president Raul Castro, the first such greeting in public involving a president of the US since the 1959 Cuban revolution.
During the service in the two-thirds filled stadium, boos were heard for the current South African president Jacob Zuma and the former US president George W Bush.
But there were cheers for Mr Obama - and for the Zimbabwean president Robert Mugabe.
Those attending included Prime Minister David Cameron and his three surviving predecessors, Sir John Major, Tony Blair and Gordon Brown.
Also at the service were supermodel Naomi Campbell, rock star Bono, former South African president FW De Klerk, Bill Clinton and his wife ex-US Secretary of State Hillary Clinton, former French president Nicolas Sarkozy and the current French president Francois Hollande.
The service heard from Andrew Mlangeni, a former prisoner on Robben Island with Mr Mandela, who said "Madiba is looking down on us."