Rebecca W. just alerted me recently to this fabulous Musopen website! Have you seen it yet? They have an enormous selection of music (audio) and sheet music that can be accessed for free, played from the site, downloaded, and embedded into other websites. In fact, as I type this, I am enjoying listening to a recording of the Dvorak Cello Concerto, Opus 104, I Allegro performed by the Davis High School Symphony Orchestra (pretty good for a high school group!).
The people behind Musopen are also working on a public domain music theory textbook. Sounds like a great concept and an interesting project! From the comments on this post, I also found out about a free music theory course by Catherine Schmidt-Jones that looks like a great resource!