99 Most Essential" series. Right now four of these MP3 collections are on sale for $1.99 each (Handel, Schubert, and Mendelssohn, plus a "relaxing classics" compilation). Even at the regular price ($5.99 to $7.99), these collections are a pretty cheap way to build up a collection of classical music. With the $1.99 sale collections, you're getting some 14 hours of music, a "boxed set" (9-12 CD's) for less than a cup of coffee in most places.
Of course there are caveats. If you are any sort of classical music and/or audio purist, you would not even think of collecting and listening to classical music in MP3 form. They use VBR recording, the sample rates are pretty high, and the recordings generally sound good to me. I'm almost always listening on my computer or iPod anyway, so having a higher quality CD in the basement usually doesn't help me (though I can rip my own CD's at higher quality if I want to). Second, the conductors, orchestras, and other performers are usually not the most famous superstars of classical, but with few exceptions, to my non-expert ears, they are all quite good. There actually are some big name performers, usually on older recordings, but still sounding very good. Third, the earlier collections (e.g., Mozart, Bach, Beethoven) often consisted of isolated movements rather than complete works. Some people like this "greatest hits" approach while others (including me) prefer to have all the movements of a particular symphony, string quartet, etc. together, in order to hear the piece as the composer intended. The "X5 Music Group" listened, and the collections released in recent months include mostly full-length works (with the exception of operas, which are not heavily represented in any case). Another thing some people don't like is that MP3's don't include full performance information (there are no "liner notes"), but they are getting better and the MP3 info generally includes the orchestra, conductor, and some soloists' names. Amazon customers sometimes research and fill in more details in their reviews.
Another point is that these offers may only be available in the U.S. - I'm not really sure, but I've seen reviewer comments for some Amazon MP3 recordings (not these) saying that they could not be downloaded in some other countries (I believe it was someone in the military in Iraq or Afghanistan). So your mileage may vary.
Right now I'm listening to Mendelssohn's "Italian" symphony, a wonderful piece that I must have owned on vinyl because I haven't heard it in forever. This particular recording is by the Bamberg Symphony Orchestra conducted by Hans Swarowsky and it sounds great.
The final problem is how to deal with this explosion of new music - two bucks, and bam, I've got 14 more hours of music to listen to (I've bought collections for Beethoven, Brahms, Chopin, Debussy, Handel, Mendelssohn, Schubert, Tchaikovsky, and Vivaldi, and maybe one or two more, almost all for $2 each). But hey, it's only two bucks, and if I only get to enjoy a handful of pieces from each collection, it's still a pretty good deal. There are worse problems to have.