President Salva Kiir witnessed the moment at a ceremony at the State House in the capital, Juba.
It is hoped that the new unity government will bring an end to the conflict that has killed about 400,000 people and displaced millions.
Saturday’s ceremony took place just before the deadline for an agreement expired.
“I do hereby swear that I shall be faithful and bear diligence to the Republic of South Sudan,” Machar said in his oath.
He then embraced and shook hands with President Kiir. Also present at the ceremony was the leader of Sudan, General Abdel Fattah al-Burhan.
Under the agreement, the current cabinet has been dissolved to make way for more opposition members.
Correspondents say some issues remain unresolved including power-sharing and the integration of rebel fighters, but the two sides have agreed to form a government and address other matters later.
The deal was announced hours after the UN released a damning report accusing both sides of deliberately starving civilians during their struggle for power.
President Kiir has expressed hope that the transitional three-year period will pave the way for refugees and internally displaced people to return to their homes.
In addition to those killed or displaced, many others have been pushed to the brink of starvation and faced untold suffering.
If the deal holds, it could herald a fresh start in the world’s newest country.
South Sudan became an independent state from Sudan in 2011, marking the end of a long-running civil war. But it did not take long for the promise of peace to crumble.
Just two years after independence, the country returned to violent conflict after President Kiir sacked Machar, then the deputy president in December 2013.
President Kiir had accused Machar of plotting a coup to overthrow him, which Machar denied.
While the war had political origins, it also has ethnic undertones and is based on power dynamics.
The Dinka and Nuer, South Sudan’s two largest ethnic groups, which the two leaders belong to, have been accused of targeting each other in the war, with atrocities committed by both sides.
Parties had been unable or unwilling to agree on the terms for the formation of a transitional government, in line with the revitalised peace agreement of 2018.
The deal was supposed to have been finalised by May 2019 but was postponed twice – the latest deadline being 22 February.
The conflict has pushed the country into a catastrophic humanitarian crisis.
Despite the situation, it has been difficult for the parties to reach and maintain a peace deal that could stabilise the country.
The two main leaders have a mutual distrust of each other and there has not been a cordial working relationship since President Kiir fired Machar in 2013.
Machar has never returned permanently to the capital, Juba, fearing for his safety. He fled the country when his forces were engaged in fierce clashes with government troops as the 2016 peace agreement collapsed.
South Sudan also has one of the lowest literacy rates in the world at 34.5%, according to Unesco (2018).
Globally, South Sudan has the fourth-lowest human development indices despite its huge natural resource potential, such as fertile agricultural land, gold, diamonds and petroleum. (UNDP’s Human Development Index measures the average achievements in human development: a long and healthy life, knowledge and a decent standard of living.)
As of 2019, more than half of the population required humanitarian assistance, with extreme levels of acute food insecurity across the country, according to the World Bank.
The country is almost solely dependent on oil revenues and there is very little investment in other sectors such as agriculture and infrastructure.