On Mar 16, 2005 I wrote Moving to del.icio.us, part 4, on the subject of Howtos.

And finally, we need to actually display links from del.icio.us on the front page. Fetch them from del.icio.us hourly and write out a document snippet, and then include that snippet in the page.

First, how to fetch them: a trivial Python script which uses the del.icio.us REST API to get recent posts:

import xmltramp,urllib2,cgi

def e(s):
  return cgi.escape(s).replace('"','&quot;')

authinfo = urllib2.HTTPBasicAuthHandler()
authinfo.add_password('del.icio.us API', 'http://del.icio.us',
  'USERNAME','PASSWORD')
opener = urllib2.build_opener(authinfo)
urllib2.install_opener(opener)
data = urllib2.urlopen('http://del.icio.us/api/posts/recent').read()
dom = xmltramp.parse(data)

out = []
for p in dom['post':]:
  try:
    ext = p('extended')
  except:
    ext = ''
  out.append('<a href="%s" title="%s">%s</a>' % \
  (e(p('href')),e(ext),e(p('description'))))
fp = open('/var/www/kryogenix.org/scripts/index.curlies.cached','w')
fp.write('\n'.join(out))
fp.close()

Then throw a line in crontab to actually run it, with crontab -e:

@hourly python /var/www/kryogenix.org/scripts/get-delicious-recent.py

And finally a brief snippet to include it in the index page, which is a Castalian page:

<?cas
CACHED_COPY = '../scripts/index.curlies.cached'
pfp = open(CACHED_COPY)
response.write(pfp.read())
pfp.close()
?>

and that’s it. Move complete. No more maintaining my own linklog. :)

(Updated: changed the crontab call and the script so that if there’s a failure it doesn’t just blank out index.curlies.cached!)

Gary Fleming

Welcome to the future :P

Need to take a look at storing via urls and names, because it’s the one thing that really bugs me just now. People deserve the credit.

Also, make sure you have a good get out strategy that you keep up to date. Though I’m sure del.icio.us wouldn’t screw over its users, no point in taking chances with our data.

deepak

Shouldn’t you be reusing your old fetched posts in case a call to del.icio.us returns a ‘service unavailable’ or something like that?

if len(out):
  fp = open('/var/www/kryogenix.org/scripts/index.curlies.cached','w')
  fp.write('\n'.join(out))
  fp.close()

sil

Deepak: yes. Actually, if it throws an error then it should be OK, because xmltramp.parse will die (as the input is not XML) and therefore we never get as far as writing the file. However, you are completely right, and cheers, except that I use “if out:” rather than “if len(out):“. Thanks!