06:26 pm, 15 Sep 03
aggregation sucks
One of the many things I’m interested in is weblog aggregation. LJ is, of course, my favorite aggregator, even though when compared to a real RSS aggregator it’s terrible, because it perfectly aggregates pretty much everyone I want to read. But scalability (and relatedly, reliability) problems keep me from thinking it’s the end-all solution.
The Atom effort is well-intended, but to me they don’t really address any of the interesting questions. Sure, RSS is a mess and a well-defined spec ought to replace it, but taking a step back it’s clear to me that polling static feeds will simply not scale. (The Atom API question is independent, but from what I could tell it seemed to be mostly SOAP versus REST versus your favorite TLA wankery, again missing the forest for the trees.)
On top of that, trying to follow Atom (and in general, that whole contingent of the web) makes me unbelievably angry in a way I can’t quite put my finger on. It’s something to do with the self-congratulatory smugness/name-dropping/egos, and a little to do with the blindness to the more difficult technical issues that are so familiar to me from looking at something the size of LJ, that prevent any real progress from being made. (Or maybe it’s bitterness because I’m not involved in it anymore. :P)
With that said, I’m pretty sure I don’t have any good solutions handy, either. I think if even if I did magically come up with a good solution and tried to present it, it wouldn’t be adopted anyway (see also: NIH syndrome) and that would be frustrating. So maybe I should stop worrying about the problem at all and hope that other people solve it. :)
Syndication is a big mess, anyway, and I fear that ultimately even a well-designed format would still have people providing partial content, forcing us to fire up a web browser anyway. The other path to go is the
jwz route: everything sucks, so I might as well screen-scrape. (For context, note that jwz’s LJ client just posts to the web form, rather than use the LJ protocol.) It would be interesting to make tools to automate screen-scraping that interacted well with an aggregator...
A step back from all of this, though: getting away from worrying about any of this and from reading LJ would probably do me some good as well.
The Atom effort is well-intended, but to me they don’t really address any of the interesting questions. Sure, RSS is a mess and a well-defined spec ought to replace it, but taking a step back it’s clear to me that polling static feeds will simply not scale. (The Atom API question is independent, but from what I could tell it seemed to be mostly SOAP versus REST versus your favorite TLA wankery, again missing the forest for the trees.)
On top of that, trying to follow Atom (and in general, that whole contingent of the web) makes me unbelievably angry in a way I can’t quite put my finger on. It’s something to do with the self-congratulatory smugness/name-dropping/egos, and a little to do with the blindness to the more difficult technical issues that are so familiar to me from looking at something the size of LJ, that prevent any real progress from being made. (Or maybe it’s bitterness because I’m not involved in it anymore. :P)
With that said, I’m pretty sure I don’t have any good solutions handy, either. I think if even if I did magically come up with a good solution and tried to present it, it wouldn’t be adopted anyway (see also: NIH syndrome) and that would be frustrating. So maybe I should stop worrying about the problem at all and hope that other people solve it. :)
Syndication is a big mess, anyway, and I fear that ultimately even a well-designed format would still have people providing partial content, forcing us to fire up a web browser anyway. The other path to go is the
jwz route: everything sucks, so I might as well screen-scrape. (For context, note that jwz’s LJ client just posts to the web form, rather than use the LJ protocol.) It would be interesting to make tools to automate screen-scraping that interacted well with an aggregator...A step back from all of this, though: getting away from worrying about any of this and from reading LJ would probably do me some good as well.
What you said!
I agree.I've tried a few times to wander around the Atom wiki and figure out what's going on. It's rather difficult, like wandering through a twisty maze of conference rooms that were all vacated just before I got there, and trying to figure out what went on by reading the cryptic notes on the whiteboards. But I guess that means I Just Don't Get Wikis or something. *rolls eyes*
I also have this fear that I'll turn around from my whiteboard-entrail-reading and see the looming shadowy form of Dave Winer advancing on me...
Anyway, as far as I was able to figure out, the "aggregation API" is where the meat is, but all I could find on the site was a bunch of hair-splitting about what protocol to use. This reminded me depressingly of the old IETF instant messaging working group, which spent ages just arguing whether to use TCP or UDP.
Re: What you said!
Yes, that is what the Aggregator API discussion has become, essentially. It started off with good intentions, but now like most of the rest of Atom has become bogged down in trivialities and is spread over so much of the wiki that it's impossible to follow.
Worst. Pun. Ever.