The presenter said he bumped into the viewer during a trip to Cornwall and could tell straight away that the man "loathed" him.
Opening up on the How To Be 60 podcast, he said he passed the person as he walked to the shops and that the man was "laser-focused" on him.
Madeley made a friendly comment about what a lovely day it was and said the man just "growled" back.
Their paths crossed again as the star returned from buying food and the viewer approached him.
Madeley said: "He came right into my space, finger jabbing at my chest and he said, ‘I just want you to know that everything you’ve ever done, everything you’ve ever said, everything you have ever said or ever will say, I loathe and f****** despise, you c***.
"And then beamed with pleasure!"
Former This Morning presenter Madeley said that he "burst out laughing" after the rant.
"Because he was just ridiculous and it was funny, his passion was funny," he said.
The TV star, 67, said the man walked on, and he called out to tell him that he had given him a great story to tell.
"But honestly, I couldn’t give a f***," added Madeley, who is married to fellow TV star Judy Finnigan.
"I do Good Morning Britain a lot now and I discovered that I only have to cough or sneeze in the wrong direction to get extraordinary abuse on Twitter, I mean really amazing stuff."
Watch: Richard Madeley challenges Steve Barclay on migrant boats