Aggregators That Think For You

February 26th, 2004

Last year at our lunch break at MIT, Greg Notess and I were talking about aggregators. Greg mentioned that he wanted to use an aggregator that would not only provide content, but would give him only stuff that he wanted to read. Consider it readers advisory in the syndication world. Impossible, I thought. But interesting. My only fix for him was use customizable feeds from weblog or news engines (actually, at that time only one or two weblog search engines were available. Sigh. My life truly was empty before Feedster)

Tonight, however, I read this:

“For a while now I’ve wanted to write an RSS aggregator which told me whether or not each story was interesting; the idea is that I mark stories I find particularly interesting or particularly boring, and it learns what I want to read. This uses tokenization, stemming, stopwords, Bayesian (or other categorization algorithm) analysis, and other things which fit nicely into the remaining chapter of the book, the one on Natural Language Processing.

“So I took Richard Clamp’s lovely Timesink web-based aggregator, and integrated it with half of CPAN, and came up with this hack. Log in as demo/demo, and you too can influence whether or not I read the next story on Slashdot…”

Now, I have no idea how or if this works, but the idea is staggering. Imagine how much time would be saved if my aggregator would provide only that information that interested me. I could rate each post and my aggregator would learn my tastes. Hmm, it would think. Steven likes stories on libraries, but not necessarily about filtering. He enjoys reading about RSS, but doesn’t care about the development of the spec. Let’s not stop there. Try this. Hmmm, it would think. Steven really enjoys posts about the Grateful Dead. Let me check my database (powered by Feedster, Technorati, Pub Sub, etc) and find more feeds that Steven may like…Maybe shows to download, or setlists…

I think I’ve said this before, but I’ll mention it again. This is 2004. Don’t work for your aggregator. Let your aggregator work for you. The hack described above is a perfect example of this theory in action.

Last, take a look at Intelli-Aggie, which describes a similar concept.

Comments are closed.