(CNN) - Two years ago, House GOP members made Holder the first sitting Cabinet member to be sanctioned with Contempt of Congress, in a still ongoing dispute over documents related to a botched federal gun probe known as "Fast and Furious."
Since, Holder has shown less patience for the punches lawmakers tend to throw his way when he travels down Pennsylvania Avenue to provide public testimony, particularly in the rambunctious House.
At a House Judiciary Committee hearing on Tuesday, Holder and Rep. Louie Gohmert tangled over whether the nation's top law enforcement officer was being sufficiently responsive to lawmaker requests.
Gohmert quickly got under Holder's skin by raising the House contempt sanction and saying it appeared to not be "a big deal" to Holder.
"You don't want to go there, buddy," Holder replied sharply. "You don't want to go there, OK?"
Holder complained that the contempt vote was a political ploy to appease pro-gun rights groups, a charge he has made before.
Visibly angry, Holder pointed his finger at the seated lawmakers: "You should not assume that is not a big deal to me. I think it was inappropriate. I think it was unjust. But never think that was not a big deal to me."
It was the third hearing in five days for the attorney general, most of which have gone relatively smoothly.
A Senate hearing last Thursday, led by Democrats, was quick and painless. Friday's hearing before a House appropriations panel began with some Republicans giving Holder an "A+" for some of his work, but soon went off script as lawmakers accused the attorney general of "selective" enforcement of federal laws, including marijuana and gay marriage.
The relationship appears to have little chance of mending, particularly as Republicans have pushed for articles of impeachment and regularly remind Holder that they want him to resign.
At a May hearing in the House, Holder offered a glimpse of the problem at the root of the dysfunctional relationship, telling one lawmaker: "I don't think I've been always treated with respect," he said. "You may not like me, but I am the attorney general."
The remark came near the end of an hours-long hearing during which Holder and Rep. Darrell Issa, who helped lead the confrontation over "Fast and Furious" and the contempt vote, shouted at each other.
As Issa tried to get Holder to stop speaking, he refused, saying: "It is inappropriate and too consistent with the way in which you conduct yourself as a member of Congress. It's unacceptable, and it's shameful."
That was also the hearing at which another tussle between Gohmert and Holder produced a now famous head-scratching moment, in which the flustered Texas Republican warned Holder: "the attorney general will not cast aspersions on my asparagus."
No one in the room seemed to know what to make of Gohmert's verbal slip.
At the end of Tuesday's verbal fisticuffs, Holder mocked Gohmert: "Good luck with your asparagus!"