Haitham bin Tariq al-Said took the oath of office on Saturday after a meeting of the Royal Family Council, the BBC quoted the government as saying.
The former Minister is a cousin of Qaboos, the last Sultan of the Middle East who had no heir or a designated successor.
The official Omani agency ONA announced the news early Saturday in a short message without providing details of the causes of the demise of the Sultan, who travelled to Belgium last month for a medical check-up, reports Efe news.
"With great sorrow and deep sadness... the royal court mourns His Majesty Sultan Qaboos bin Said, who passed away on Friday," it said.
Oman has declared three days of mourning and shutdown of offices in both the public and private sectors following the loss of its leader. Flags will fly at half-mast for the next 40 days.
Images showed a crowed of men gathered outside the Sultan Qaboos Grand Mosque in the capital, Muscat, where the casket had been taken and prayers were being held.
Qaboos was born in the southern city of Salalah, the then-capital of the kingdom, and was the only son of Said bin Taimur and Princess Mazoon al-Mashani.
He overthrew his father in 1970 in a bloodless coup with the help of the British.
Venerated in Oman, he is remembered as wise, righteous and the chief mediator in one of the most conflict-ridden regions of the world. He established ties with neighbouring Iran and even Israel.
After ascending the throne in 1970, and heavily influenced by the West and Zanzibar - his main country of reference - he sought to turn around the fortunes of Oman and its people.
He also placed much emphasis on education. By 1975, there were already 214 schools and in 1982, the first university, named Qaboos, was founded.
In addition, he set up a free, modern healthcare system - that went from having 150 doctors in 1975 to more than 3,500 in the present day - which significantly improved life expectancy and infant mortality.
The Sultan is the paramount decision-maker in Oman and also holds the positions of prime minister, supreme commander of the armed forces, minister of defence, minister of finance and minister of foreign affairs.
( With inputs from IANS )