Born and raised in Fulham, London, Simon Milton moved to Thetford in his early teens. A keen footballer, he had trials as a schoolboy with both Norwich and Cambridge United but was turned down for being too small. After playing as a junior for Thetford Rovers, Simon went on to initially play in the Jewson League for Thetford and Bury. Playing well against Ipswich Reserves and Colchester in pre-season friendlies led Ipswich to note his talent. He played a few games for the Ipswich second string before manager Bobby Ferguson offered him a contract for the following season.

However Ferguson lost his job leaving Milton in limbo. When John Duncan arrived as the Blues' manager, coach Charlie Woods and Bryan Klug persuaded Duncan to take a gamble on Milton. Ipswich paid £5000 to take Simon to Portman Road in 1987. During his first season at Ipswich, he had loan spells at Exeter and Torquay which helped to toughen Simon up and prepare him for league football. In all, he would spent 11 seasons at Portman Road and was voted player of the year in 1996 - his local roots endearing him to the fans.

Simon retired from playing in 1997 but has worked for Ipswich's academy since it opened in 1998, He also runs the popular Milton's Bar in Bury St Edmunds.