During a taping of Thursday's show, Letterman said he has informed CBS that he will step down in 2015, when his current contract expires.
He announced no specific end date, telling his audience he expects his exit will be in "at least a year or so, but sometime in the not too distant future — 2015, for the love of God, [band leader] Paul [Shaffer] and I will be wrapping things up."
Referring to CBS chairman Leslie Moonves as "the man who owns this network," Letterman said, "I phoned him just before the program, and I said, 'Leslie, it's been great, you've been great, and the network has been great, but I'm retiring.'"
One of the show's guests, singer-songwriter Joseph Arthur, tweeted the announcement shortly after the show was taped.
Along with his network, Letterman thanked "all of the people who have worked here, all of the people in the theater, all the people on the staff, everybody at home, thank you very much.
"What this means now," he cracked, "is that Paul and I can be married."
Letterman turns 67 next week. He has the longest tenure of any late-night talk show host in U.S. television history, already marking 32 years since he created "Late Night" at NBC in 1982.
He jumped to CBS to start "Late Show" in 1993. Jay Leno, his rival at the time to host NBC's "Tonight Show," retired earlier this year, making way for Jimmy Fallon.
With the late-night landscape now settling at NBC, the succession plan at CBS becomes the new guessing game. In the wings as a possible successor: Craig Ferguson, host of "The Late Late Show," which follows Letterman.
"For 21 years, David Letterman has graced our network's air in late night with wit, gravitas and brilliance unique in the history of our medium," said Moonves. "It's going to be tough to say goodbye. Fortunately, we won't have to do that for another year or so. Until then, we look forward to celebrating Dave's remarkable show and incredible talents."
Leno held the ratings leadership for most of his two-decade duel with Letterman, but Dave remained the overwhelming critical favorite, pushing forward in a "Tonight Show" tradition forged by Johnny Carson and, before him, Jack Paar and Steve Allen.
"Late Show" won a prime-time Emmy in 1994. Letterman earned a Peabody Award in 1992 and was a Kennedy Center Honoree in 2012.
In an interview with Howard Stern in January, Letterman said that Leno's departure would have no impact on how much longer he might stay as host of "Late Show."
"I would do it forever if it were up to me," said Letterman, before adding a wry aside: "Sometimes, it isn't up to me."
This story has been updated.