Castle Grant, in Grantown on Spey, was repossessed by the Bank of Scotland last week over mortgage arrears.
It has a ballroom, cinema room, billiards room, a library, four bedrooms and a guest wing. The four-storey
castle also has a landscaped loch and 35 acres of private land.
Most of Mr Whyte's belongings were removed from the property last week.
Estate agent Kevin Maley said: "It is clear that a number of improvements have been made, however a large part of the property remains undeveloped and a significant amount of investment will be required to finish this off.
"Castle Grant is a wonderful property which is absolutely ripe for development which could lead to it being one of the premier private houses in the UK."
Mr Whyte bought Rangers in 2011 while a tax liability was in dispute. The club went into administration less than a year later.