Once I had ogg123 -d alsa09 foo.ogg producing sound, reconfiguring gstreamer to use ALSA instead of OSS means that at least Rhythmbox (which I haven't really used in the past, but I dug up for the purposes of testing) works. XMMS, even with ALSA output, is still unreliable, but at one point I saw someone claim on a mailing list that it's some fundamental brokenness of XMMS.
Now if I can only figure out why it powers off instead of going to sleep, I'll have a computer that has technological innovation rivalling something out of the mid-nineties!