Andy began his career with Barnsley (36 appearances), Doncaster Rovers (106 games) and Oldham Athletic. He came to Norwich in March 1996 as cover for Andy Marshall following Bryan Gunn's broken ankle suffered at Nottingham Forest on December 27th 1995. Rhodes made 8 appearances for the Reserves, keeping two clean sheets in London at Wimbledon and at QPR. He had also played on loan for Bolton in February 1995.

Andy later spent some time with Norwich's Suffolk rivals Ipswich Town as their goalkeeping coach. In October 2004, injuries to Ipswich reserve team keeper Shane Supple saw the club seek to register Andy as a player for their clash on 23rd October. In the end, they failed to do and entered the match with no substitute keeper. Andy left Ipswich in the summer of 2008.

Previously to joining Ipswich, Andy was with Barnsley as their goalkeeping coach. Prior to his return back to Oakwell in the summer of 2000, he was playing for Emley Town of the Unibond Northern League. Previously he was with GM Vauxhall Conference champions Halifax Town whom he joined in March 1998. Before that he was registered with Scottish First Division side Airdrie although at the end of November 1997, he joined Division Three side Scarborough on loan. He joined Airdrie from St Johnstone whom he had joined on 31st May 1992 in an exchange deal worth £300,000. St Johnstone were his second Scottish club having joined them from Dunfermline whom he made 89 appearances since joining from Oldham in July 1990 for £80,000. He made his debut for the Pars in a 3-1 defeat to Rangers on 25th August 1990 and his final game for them was against St Mirren on 25th May 1992, which the Pars also lost 3-1.

Andy joined Preston as part of their backroom coaching team. In January 2010, he joined Sheffield Wednesday as goalkeeping coach.

In June 2018 Rhodes joined Oldham as Assistant Manager to Frankie Bunn but Rhodes was sacked on 26th October 2018 with the Chairman citing financial reasons.

His son Jordan joined Norwich on loan in July 2018