He was quizzed by MSPs on the Scottish Parliament Health and Sport Committee on poverty and welfare reforms.
He disputed claims from a Child Poverty Action Group (CPAG) report that 60,000 children in Scotland would be pushed into poverty as a result of benefit changes and instead said Universal Credit would see 300,000 people in the UK go into work.
Glasgow SNP MSP, Bob Doris told him about a constituent who was sanctioned and lost benefits for not attending a job centre interview because she was on a work placement, as directed by staff.
Mr Doris said there was a "gung-ho attitude to take benefits off the most vulnerable."
He added there was an increase in the number of people going to food banks.
The MSP said there had been more than 50,000 cases of benefit sanctions and called for a review of the system.
Mr Carmichael was asked if there was a link between sanctions and an increase in people going to food banks for help.
He said: "I think there may well be, because sanctions are normally the result of the conduct of the claimant themselves.
"I wouldn't be at all surprised if that was the case."
After the meeting he was presented with figures from a Trussell Trust food bank in Govanhill, where in one month 335 people asked for help - 222 as a result of benefit changes.
He said: "Anybody with any social conscience at all will look at the growth of food banks and they will want to know what that is about.
"But to draw a link between benefit sanctions and the growth of food banks, I suspect, is highly simplistic."
The Scottish Secretary admitted he didn't know how many people would be pushed into poverty as a result of Coalition welfare reforms.
Asked by SNP MSP Aileen Macleod, how many children would be living in poverty by 2020, Mr Carmichael said: "The honest answer, I think, is that nobody really can know.
"The CPAG assumption does not take into account efforts to get people off benefits and into work.
"That is very, very difficult to predict outcomes from."
After the meeting Ms Macleod said: "What Alistair Carmichael today highlighted was how totally uninterested Westminster is with child poverty and the needs of people in Scotland."