Mr Swinney agreed to a request from Labour to mitigate fully the impact of the 'bedroom tax' in Scotland estimated to be at £50m.
The Finance Secretary revealed the plan in his statement before the budget was passed and was supported by Labour and Scottish Green MSPs.
The Tories, however, opposed the plan and said any available cash should have been spent instead on measures to boost the economy.
Mr Swinney will increase the amount the Scottish Government gives in Discretionary Housing Payments to almost £23m, the legal maximum allowed by Westminster.
He said there is still a shortfall of £12m and if the UK Government rejects the call to allow him to put that in as well, he will set up two funds to give the cash to councils and housing associations to cover any arrears purely from the 'bedroom tax', or spare room subsidy.
He said it will ensure no one is evicted in Scotland as a result of benefit cuts from the so-called 'tax'.
Mr Swinney said: "We can never be able to mitigate all of the damage of welfare reforms."
He said the Chancellor was planning another £4.5bn of welfare cuts in the next two years.
He added: "We are spending £244m on alleviating the impact of welfare reforms. That is resources we could have spent on services in Scotland.
"The 'bedroom tax' is iniquitous and damaging."
He said he will take some cash from rail budget and use some Barnett consequential cash form Westminster to fund the scheme.
He added: "None of this is the solution to the 'bedroom tax'. The only solution is to scrap it altogether."
Labour said the 'bedroom tax' move was enough to allow it to back Mr Swinney's budget, the first time since the SNP came to power in 2007.
Iain Gray, finance spokesman, said he did not agree with all the measures in the budget that didn't mean they couldn't support it.
He said: "There are many provisions in this budget not to our liking."
He mentioned the council tax freeze, college places and said free school meals was the wrong priority providing benefits to well off families.
However, he added: "This budget is an opportunity for something we, the SNP and the Greens see as desirable, the end of the impact of the 'bedroom tax' in Scotland."
He said it was a "sink the 'bedroom tax' to the bottom of the sea" budget.
The Greens said the budget was not perfect but backed the 'bedroom tax' plan, and childcare and free school meals
Patrick Harvie, Green MSP for Glasgow said: "This backlash budget sees a united front from most of Scotland's political landscape against the Tories' 'bedroom tax' attack on tenants.
"The Scottish Government is right to show that we can make very different choices from Westminster even within the current constraints of devolution.
"Greens welcome new money to extend childcare and school meals, which will have a positive impact on equality and opportunity."
The Tories opposed the extra cash to alleviate the impact of the 'bedroom tax' and wanted to see more done to promote economic growth.
Gavin Brown, finance spokesman, said: "The economy barely got a mention in the cabinet secretary's speech.
"It should've been about individual measures that would not win votes but grow the economy."
After Mr Swinney said he had found funds from the budget to be used to prevent eviction of tenants who had seen their benefits cut from the 'bedroom tax', Mr Brown said that was not the best use of resources.
He said: "We think any available funds should be invested in the economy."
The 'bedroom tax' move was welcomed by housing campaigners.
Shelter Scotland said it was common sense. Director Graeme Brown, said "Today's statement recognises the importance of both getting money into the hands of tenants struggling to pay their rent and also protecting the future of Scotland's social housing by ensuring social landlords don't go out of business.
"Ultimately the only way to banish the 'bedroom tax' for good is to scrap the policy. Until that is possible, these moves to help those affected should be seen not only as a victory for common sense, but also for social justice."