Is there any site that just has a bunch of clips of languages, preferably organized into various categories and such?
No, no. I meant audio clips :|
search for "language samples" on google or something. also something similar: speech accent archive http://accent.gmu.edu/
About.com does a phrase of the day with audio and has searchable categorized archives...
It's a clunky site, a bit overloaded with advertisements, but www.travlang.com has phrasebook phrases for travelers in around 50 languages, from the very useful (Mandarin) to the highly obscure (Occitan). Most have sound clips.