SubscribeScrapi
Tuesday, February 28th, 2006One of my favorite things out of MashupCamp is the word scrAPI.
The biggest challange in designing co.mments is figuring out how to identify comments. We could use comment feeds, but most blogs don’t support them. We can invent a new API and sit around waiting for bloggers to support it … some day. There are other solutions, but the one I found out to be the simplest is to just scrape the page and look for comments.
So essentially, you can think of each blog as having an API. A scraping API. Not all of them have it, and some are better than others, but for the majority of blogs, it’s good enough.
And not just for comments. Over the next few weeks, you’ll see a few additions that are currently in the works. I’m looking at ways to add authors, so you can track all comments made by a given person, and a way to add tags so you can drill down into conversations other people are tracking.
It’s all a matter of adding more features to the existing scrAPI.
And more on scrAPIs, from Thor Muller of RubyRedLabs:
The most exciting thing for me about Mashup Camp was seeing clearly the contours of an emergent phenomenon now in its earliest stages. We have APIs for only a miniscule portion of the data providers out there, and this is unlikely to change anytime soon. But we are starting to see a new breed of home-brewed APIs built on top of the screen scrapers we’ve all been writing and maintaining for years—scrapers that pull crime stats from police blotters, address data from Craigslist apartment listings, mp3s from web sites.
And of course there are ways to improve on the scrAPI, with a touch of Microformats. For bloggers, that’s the easiest and quickest solution to make sure comments are easily tracked. I’ll get to that in a future post.
