The 2006 world champion, who also reached the world final two years ago, was well below his best in going down 6-2 against Shaun Murphy.
When it was all over, Dott cut a disconsolate figure as he was left to ponder if he will ever return to the form that saw him claim the world crown six years ago. He said: "I've not been happy with how I have been playing for a while.
"I just don't think that I'm the same player I was, I haven't been anywhere near the way I played when I got to the world final in 2010 or that I won in 2006.
"If anything I have probably got worse. I'm playing so many really bad matches these days.
"I don't think there's anyone else in the top 16 that can play as badly as I can and it just isn't good enough.
"I hope it's not just because of my age. I'm only 35 but maybe I have had it, maybe I'm finished.
"I probably played worse in the first round but I have never had so many bad matches, it never used to be like this."
Fellow Scot Stephen Maguire was also disappointed, beaten by in-form Stuart Bingham.
Maguire had his chances and after trailing 2-0 early on tied things up at 4-4 before Bingham sealed the last two frames in impressive style.
Bingham's opponent in the quarters will be Ali Carter who ended the hopes of giant-killer Mark Joyce.
Conqueror of world No.1 and defending champion Judd Trump in the previous round, Joyce found Carter too hot to handle and went down 6-4.
Carter insisted he was still to play his best snooker this week. "I have not played well yet but it's about winning rather than playing well," he said. "But it's coming together."
lBritish Eurosport is the Home of Snooker, showing 19 tournaments this season including the World Championship, UK Championship, the Masters and all ranking events, available on Sky (410/411) and Virgin Media (521/525) or online/on mobile via eurosportplayer.co.uk