Walker was jailed for 12 months last September after being found guilty of 23 assaults and one breach of the peace against three former wives and his stepdaughter which spanned decades.
Judges at the Appeal Court in Edinburgh refused his appeal against two assaults on his first wife, Maureen Traquair, in 1967 and 1969.
They rejected a submission which argued that these offences were not sufficient in terms of "time, character and circumstance" in order for mutual corroboration to be applied.
The other offences took place from 1978 onwards, meaning there was a "very large gap" between these offences and later ones, the Appeal Court heard.
The Lord Justice Clerk Lord Carloway, Lady Dorrian and Lord Bracadale took a short adjournment to consider their decision.
Lady Dorrian said: "The only issue which arises in this appeal is whether there was sufficient circumstances of similarity between evidence in charges one and two for the doctrine of mutual corroboration to be applied.
"Leave to appeal on other charges was refused at the first sift."
Lady Dorrian said the judges took the view it was "entirely artificial to seek to compartmentalise evidence" in relation to the two earliest offences.
Sheriff Kathrine Mackie, who heard the two-week trial at Edinburgh Sheriff Court last year, found Walker guilty of assaulting Ms Traquair on three separate occasions in the 1960s and 1980s.
Walker, 72, was released from Dumfries Prison last month after serving six months of his sentence. He was not in court for yesterday's hearing.
The former SNP MSP said he was "disappointed" by the decision.
He added: "Unfortunately, I simply cannot afford to pursue these further through legal channels."