He had been receiving intense home-based medical care for a lung infection after three months in hospital.
A statement on South African national TV, Jacob Zuma said Mr Mandela had "departed" and was at peace.
"Our nation has lost its greatest son," Mr Zuma said.
He said Mr Mandela would receive a full state funeral, and flags would be flown at half-mast.
BBC correspondents say Mr Mandela's body will be moved to a mortuary in Pretoria, and the funeral is likely to take place next Saturday.
A crowd has gathered outside the house where Mr Mandela died. Some are flying South African flags and wearing the shirts of the governing African National Congress, which Mr Mandela once led.
The Nobel Peace Prize laureate was one of the world's most revered statesmen after preaching reconciliation despite being imprisoned for 27 years.
He had rarely been seen in public since officially retiring in 2004. He made his last public appearance in 2010, at the football World Cup in South Africa.
His fellow campaigner against apartheid, Archbishop Desmond Tutu said he was not only an amazing gift to humankind, he made South Africans and Africans feel good about being who we are. He made us walk tall. God be praised."
What made Nelson Mandela great was precisely what made him human. We saw in him what we seek in ourselves," Mr Zuma said.
"Fellow South Africans, Nelson Mandela brought us together and it is together that we will bid him farewell."
Tributes have come in from around the world. UN Secretary General Ban Ki-moon said he was "a giant for justice and a down-to-earth human inspiration".
"Many around the world were greatly influenced by his selfless struggle for human dignity, equality and freedom. He touched our lives in deeply personal ways."
US President Barack Obama said Mr Mandela achieved more than could be expected of any man.
"He no longer belongs to us - he belongs to the ages," Mr Obama said, saying Mr Mandela "took history in his hands and bent the arc of the moral universe towards justice".
Mr Obama, the first black president of the United States, said he was one of the millions who drew inspiration from Mr Mandela's life.
FW de Klerk, who as South Africa's last white president ordered Mr Mandela's release, called him a "unifier" and said he had "a remarkable lack of bitterness".
He told the BBC: "I think his greatest legacy... is that we are basically at peace with each other notwithstanding our great diversity, that we will be taking hands once again now around his death and around our common sadness and mourning."
UK Prime Minister David Cameron also paid tribute, saying "a great light has gone out in the world".
Earlier this year, Mr Mandela spent nearly three months in hospital with a recurring lung infection.
He was moved to his home in the Houghton suburb of Johannesburg in September, where he continued to receive intensive care. (Source: BBC)
1918 Born in the Eastern Cape
1943 Joined African National Congress
1956 Charged with high treason, but charges dropped after a four-year trial
1962 Arrested, convicted of incitement and leaving country without a passport, sentenced to five years in prison
1964 Charged with sabotage, sentenced to life
1990 Freed from prison
1993 Wins Nobel Peace Prize
1994 Elected first black president
1999 Steps down as leader
2001 Diagnosed with prostate cancer
2004 Retires from public life
2005 Announces his son has died of an HIV/Aids-related illness