Kompany's shock declaration came hours after the Premier League champions thrashed Watford 6-0 at the Wembley in the FA Cup final to become the first men's side to complete the domestic treble in England.
"It doesn't feel real," Kompany wrote on Facebook.
"Countless of times have I imagined this day," he said. "After all, the end has felt nearby for so many years. Man City has given me everything. I've tried to give back as much as I possibly could."
Kompany, 33, has won four Premier League titles, two FA Cups and four League Cups during his 11-year stay at the club.
The Belgium international has also scored 20 goals in 360 games.
"The time has come for me to go now," he said.
"As overwhelming as it is, I feel nothing but gratefulness. I am grateful to all those who supported me on a special journey, at a very special club. I remember the first day, as clear as I see the last. I remember the boundless kindness I received from the people of Manchester.
"I will never forget how all Man City supporters remained loyal to me in good times and especially bad times. Against the odds, you have always backed me and inspired me to never give up," Kompany said.