Joey Hess mentions the Rawdog aggregator, which is for setting up Planet-style sites. I normally use spycyroll for this (that’s what Planet Wolves uses, for example) because the idea of requiring a database to do this is just brain dead. It is. Don’t deny it. However, I think I might start using rawdog for that purpose in the future. First, it looks maintained. Secondly, Adam Sampson, the hacker who built it, wrote a plugin for it to use Vellum templates! Cor. Every now and again I see Vellum pop up somewhere; I think its plugin design might have been influenced by Vellum’s, too, which is really cool. Nice work, Adam, anyway.
On I wrote Rawdog, on the subject of Web.
nordle
sil
nordle: that’s the point. Aggregators of this type are in the following three categories:
- Require a database
- spycyroll, which doesn’t
- rawdog, which doesn’t either
My notes above mean: “I chose spycyroll over the others, because it doesn’t require a database and because I didn’t know about rawdog. Now that I know about rawdog, I’ll choose that instead of spycyroll“.
Adam Sampson
Thanks! Your guess is quite right that rawdog’s plugin design was inspired by Vellum’s; I had a look around when I was implementing plugins, and Vellum’s approach seemed the most Pythonic.
rawdog does sort of have a database (the ~/.rawdog/state file, which is just a Python pickled data structure), but when used for planet-like sites you’d usually turn on “currentonly“, which stops it from keeping articles that aren’t in the current feeds. You do need to keep some sort of state in order to avoid fetching feeds that haven’t changed since the aggregator was last run.
sil
Adam: ah, when I say “database” I mean “a long-running process separate from the rawdog process, which probably is an RDBMS and lets clients get at the data from SQL“. Stashing some data away to keep state does not count as a database. :-)
Maybe I missunderstood, but Im using Rawdog without a database…