The Reds were hardly in the game during the first half and trailed 2-1 at the break.
And although the arrival of captain Steven Gerrard just before the hour-mark changed the momentum, he could not orchastrate a recovery.
Rodgers left top stars like Gerrard, Jamie Carragher, Lucas and Stewart Downing on the bench with only Martin Skrtel, Luis Suarez and keeper Brad Jones aged over 23 – but when the result went against them it was the youngsters he criticised.
"We are trying to give them experience and let them see what it is like to play for Liverpool," he said. "We were just on the back foot too much early on, you really have to be aggressive.
"It is great to put young players in but you have to have personality and the young players disappointed me."
In the other shock of the day, Neil Warnock's Leeds United beat Spurs 2-1 at Elland Road, with goals from Luke Varney and Ross McCormack sending the home fans wild before Clint Dempsey got one back.
Warnock said: "It's a great day for us. The atmosphere was good, two cracking goals.
"There were a lot of heroes out there for us. It was a typical FA Cup tie."
Fifth round draw: Huddersfield Town/Leicester City v Wigan Athletic; Milton Keynes Dons v Barnsley; Oldham Athletic v Everton; Luton Town v Millwall; Arsenal v Blackburn; Manchester City v Leeds United; Manchester United v Reading; Middlesbrough v Brentford/Chelsea.
Ties will take place on February 16 and 17.