The roof of the Burrell Collection in Pollok Country Park has suffered from leaks for more than a decade.
In 1999, the city council promised the roof would be replaced within five years at an estimated cost of £2 million.
It was said to be the number one priority after the refurbishment of Kelvingrove Art Gallery and Museum.
The roof replacement has never been carried out, but this month work will begin on £70,000 of repairs which will be carried out over 12 weeks – weather permitting.
A spokesman for Glasgow Life, which runs the museum, said: "Next year, the Burrell will celebrate its 30th birthday and this work will help to preserve not only an architectural beauty but some of the treasures it is home to."
The gallery will remain open while repairs are being carried out, but some objects may be moved for their own safety if work is taking place above them.
Glasgow-born shipping magnate Sir William Burrell gifted his art collection to the city in 1944.
It is one of the greatest art collections ever created and includes more than 8000 items.
They include works by Rodin, Degas and Cezanne as well as important examples of Medieval, Chinese and Islamic art.
The collection moves to its present home in Pollok Park in 1983, when the building was opened by the Queen.
The structure was named Scotland's second greatest post-war building in a poll of architects in 2005.
However, the roof of the Burrell has been springing leaks for many years and, on occasions, staff have been forced to put down buckets to catch the drips.
In 2009, Neil Baxter, secretary of the Royal incorporation of Architects in Scotland, warned the flat roof of the building needed constant and substantial maintenance.
He said: "The roof has always been problematic, partly due to original building defects rather than its design."