I'm no expert, but as far as I understand it, OGG is actually a "container" format. AVI is also a container format. What this means, is that the file type may have nothing to do with that you'll need to decode it. For instance, OGM can contain Xvid video and audio ac3 audio, OR it could have h264 (or whatever it is) video and mp3 audio. (Just like AVIs can be Divx, Xvid, etc.)
Again, I may be talking out my a** here, but as far as I know, "Vorbis" is actually an audio compression format that is almost always contained in the OGG container format (hence the term "Ogg-Vorbis").
MKVs are another container format that's gaining popularity. They allow DVD style chapters, multiple audio tracks, and menus (I think). For more info on MKVs (or Matroska files) check out: matroska.org
If you're looking for a nice Matroska (and probably OGM also) player, check out The Core Media Player at corecoded.com
BTW, T.C.M.P. is based on ffdshow, which, one way or another, has/gets all the codecs it needs.