Posts categorized “JavaScript and the DOM”.

Not blocking the UI in tight JavaScript loops

Everyone’s written a JavaScript loop that just loops over all the {LIs, links, divs} on a page*, and it’s pretty standard. Something like

var lis = document.getElementsByTagName("li");
for (var i=0; i<lis.length; i++) { // yes this could be more efficient, don't care
  // do something here to lis[i]
};

or, if you’re using jQuery:

$("li").each(function() {
  // do something here to this
});

This is problematic if there are, say, 2000 LI elements on the page, and what you’re doing in the loop is semi-intensive (imagine you’re creating a couple of extra elements to append to each of those LIs, or something like that). The reason this is a problem is that JavaScript is single-threaded. A tight loop like this hangs the browser until it’s finished, you get the “this script has been running for a long time” dialog, and the user interface doesn’t update while you’re in this kind of loop. You might think: aha, this will take a long time, so I’ll have some sort of a progress monitor thing:

var lis = document.getElementsByTagName("li");
for (var i=0; i<lis.length; i++) { // yes this could be more efficient, don't care
  // do something here to lis[i]
  progressMonitor.innerHTML = "processing list item " + i; // fail
};

but that doesn’t work. What happens is that the browser freezes until the loop finishes. Annoying, but there it is.

One approach to getting around this is with timeouts rather than a for loop.

var lis = document.getElementsByTagName("li");
var counter = 0;
function doWork() {
  // do something here to lis[i]
  counter += 1;
  progressMonitor.innerHTML = "processing list item " + counter;
  if (counter < lis.length) {
    setTimeout(doWork, 1);
  }
};
setTimeout(doWork, 1);

so you move the bit of work you need to do into a function, and that function re-schedules itself repeatedly, using setTimeout. This time, your user interface will indeed update, and your progress monitor will show where you’re up to. There are a couple of caveats with this: it’ll take a bit longer, and you’re no longer guaranteed to have things processed in the order you expect, but they’re minor issues.

For doing this in jQuery, a tiny plugin:

jQuery.eachCallback = function(arr, process, callback) {
    var cnt = 0;
    function work() {
        var item = arr[cnt];
        process.apply(item);
        callback.apply(item, [cnt]);
        cnt += 1;
        if (cnt < arr.length) {
            setTimeout(work, 1);
        }
    }
    setTimeout(work, 1);
};
jQuery.fn.eachCallback = function(process, callback) {
    var cnt = 0;
    var jq = this;
    function work() {
        var item = jq.get(cnt);
        process.apply(item);
        callback.apply(item, [cnt]);
        cnt += 1;
        if (cnt < jq.length) {
            setTimeout(work, 1);
        }
    }
    setTimeout(work, 1);
};

and now you can do

$.eachCallback(someArray, function() {
  // "this" is the array item, just like $.each
}, function(loopcount) {
  // here you get to do some UI updating
  // loopcount is how far into the loop you are
});

$("li").eachCallback(function() {
  // do something to this
}, function(loopcount) {
  // update the UI
});

Not always a useful technique, but when you need it, you need it.

Absolute beginners guide to Google Maps JavaScript

A mate of mine has been building a relatively complex website for Potton, the town he lives in. In six months he’s gone from knowing nothing about Django or JavaScript to building something pretty cool with lots of Google maps and so on, and he’s started a series writing up what he’s learned for other people in the same position. First essay, an Absolute beginners guide to Google Maps JavaScript.

Games in pure SVG

In the “stupid experiments” category…

If you’ve got nothing better to do for three hours in an evening, why not experiment a bit with SVG? That’s what I thought, earlier on this evening.

So: cave.svg, a game for people with no graphics criticism ability and only one finger. One single SVG file, with all the controlling JavaScript therein. Inspired by SFCave, a game I played a million jillion years ago on a Palm IIIx and which I was astounded to discover has a website and a Java version and everything.

I have discovered the following things about SVG this evening.

  1. It is dog slow. I mean, slo-o-o-o-o-o-o-ow. You’ve-finished-the-exam-and-there’s-still-an-hour-to-go slow. Sitting in a traffic jam for three hours and the kids keep asking for an ice cream slow.
  2. It works in Firefox and Opera. I’ve tested in Midori, which is a WebKit browser, and it seems to work there too except that the fonts display as black-on-black, which means that either (a) Midori misimplents the spec (10% chance) or (b) I’m doing something wrong (90% chance). It’s probably broken the same way in Safari too, but the game seems to work.
  3. You have to care about XML at inopportune moments. I spent twenty minutes trying to work out why createElement didn’t work before sighing and using createElementNS.
  4. You can specify all the sizes in percentages! So it works at any size at all and the browser handles it all for you! Resize the game while you’re playing and it all still carries on working and scales for you! Do that with canvas!
  5. The previous point appears to be the only thing where SVG scores over canvas. For everything else using SVG on the web seems rather like having your scrotum gently resting between a pit bull’s teeth. It makes everything slightly more awkward than it ought to be… and any moment now you know the pain is coming.
  6. Did I mention slow? Can somebody please tell me what kind of a world we live in where my dual-core 2×2GHz PC can’t render a screen made out of rectangles at more than 10fps without dropping keypress events? I mean, come on.
  7. It’s not very optimised code (but it shouldn’t need to be). It would probably be a lot faster if I actually did things an SVGish way, by which I mean use the transform attribute and so on, but there are so many things I’d rather do than matrix arithmetic that it’s not even funny. Up to and including eating a pound of fish fingers with broken glass in.

In my head for a while has been a slight disappointment that everyone writing games or graphics things using JavaScript has gone for canvas (which is one step away from being a plugin — it gives you a white box and you draw in it) rather than the more web-ish SVG (which works like HTML and can be intermixed and everything). I am no longer disappointed. People don’t avoid dynamic SVG on the web because they’re wrong. People avoid dynamic SVG on the web because it’s quite shit.

A WAI-ARIA “stylesheet”

Yesterday at the Multipack Presents event, Matt Machell spoke about WAI-ARIA, which is a way of marking up HTML with accessibility hints. (Think of it as an accessibility microformat, pretty much.) Anyway, the way it works is by adding loads of new attributes to HTML: to say “this div is actually being used as a slider”, you say <div id="myslider" aria-role="slider">. There are lots of similar new aria-* attributes: aria-describedby, aria-minvalue, and so on.

Anyway, I asked: why did loads of new attributes get invented? Why not have an “aria stylesheet”, where instead of <div id="myslider" aria-role="slider">, you put #myslider { role: slider } in a separate ARIA “stylesheet” file? And Matt said, hm, dunno, why don’t you write something to do that?

So here it is. This is in no way complete. Requires jQuery, too. What you do is add

<link rel="aria" href="first.aria">

to the head of the page, then include jQuery:

<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.1/jquery.min.js"></script>

and then finally have this snippet of code:

<script>
$(document).ready(function(){
  $("link[rel=aria]").each(function(){
    $.get(this.getAttribute("href"), function(data) {
      $.each(data.split("}"),function(idx, stanza){
        var parts = stanza.split("{");
        if (parts.length != 2) return;
        var sel = parts[0];
        $.each(parts[1].split(";"), function(idx, directive) {
          var bits = directive.split(":");
          if (bits.length != 2) return;
          $(sel).attr("aria-"+bits[0].replace(/^\s+|\s+$/g,''),
          bits[1].replace(/^\s+|\s+$/g,''));
        });
      });
    });
  });
});
</script>

which takes an “ARIA stylesheet” which looks like

#first {
  role: slider;
  described-by: #second;
  value-min: 10;
  value-max: 30;
}
p + p {
  role: main;
}

and applies the specified ARIA attributes to the specified elements, by CSS selector.

This is rambunctiously untested, and the parser for the “stylesheet” is pretty noddy and not at all error-tolerant. If people find this useful as a concept then I’ll try and trick it up into something a bit better.

Whizzy JavaScript stuff

Yesterday I did a talk at the inaugural Multipack Presents event in Birmingham, about some whizzy JavaScript stuff that people might not know about. It covered things that are nearly available for use everywhere (getElementsByClassName, Array.forEach) right up to things that only have an implementation at the moment in one browser (server-sent events (Opera), local SQL database storage (Webkit)). The idea was to say firstly “look at all this cool stuff that’s coming up” but more importantly to say “look at all this cool stuff you can use right now if you want to!”

Talk available for browsing and downloading over at the Whizzy JavaScript Stuff talk page, anyway.

Thanks to the Multipack for having me; I hope there are many more of these events! There are event photos on Flickr, and I’m told there may be (incomplete) video of my talk available at some point (when there is I’ll link to it).

YUI3 and Launchpad

For the last week I’ve been in Berlin with the Launchpad JavaScript team building a set of cool widgets using YUI3 that will gradually start to appear in Launchpad and make its UI nicer. The LP guys have started to write up the things that they’re doing over on the Launchpad blog. I’ve learned quite a lot about YUI3 in this week; I need to do a writeup (or more probably a set of individual writeups and tips about stuff I’ve learned).

Now is not the best time, though, since YUI is being incredibly frustrating with its widgets; they don’t clean up after themselves, so if your widget creates HTML then it’s your responsibility to remove that HTML from the document later. And my widget seems to be creating its HTML twice, which is less than excellent. Frustration abounds.

That aside, though, this has been a good week; got a lot done, and I want to see Canonical be part of the web community like we’re already part of the Gnome/Linux community. Building cool stuff and talking about it and releasing it is going to help, there.

Opera Web Standards Curriculum: the JavaScript version

The Opera guys have been putting together a Web Standards Curriculum — a sort of school textbook of modern web development. Anyway, after a certain amount of cajoling from Chris “Dark Sanatic” Mills I came through on my promise to actually write some articles for it. They’ve now been published in the JavaScript section, if you like that sort of thing: one on modifying the HTML DOM tree and one on simple animation. Enjoy.

(I am proud I was not as whiny as Nyman — if you saw excerpts of my emails to Mills & co they’d all say “oh, piss, yes, I did say I’d do that, didn’t I? I’ll get it done this week, promise.” Who’d be an editor, eh?)

Giving searchhi a bit of a makeover

A user of one of the scripts I wrote a while back, searchhi, dropped me a mail to say: it would be handy if I could take some action if no hits are found. After a bit of probing, it turned out that they were using it for an in-page search — what searchhi actually does is look at the referrer to see if you came here from a Google/Yahoo search and highlight the stuff you searched for — so you could fill in a search form and then search the words in the page and highlight them. I’m at a bit of a loss as to how the built-in browser search (which is Ctrl-F for everyone, I think, although Macs might be different, and I use / on Firefox) doesn’t satisfy this need, but needs must and all that. Hacking both of those things in (have a <form class="searchhi"> in the page and its first textbox will automatically be used as the search entry) was relatively quick.

The nice thing about this is that I took the opportunity to tidy the script up a bit; it was written some time ago, and created its functions in the global namespace, was careless about var, that sort of thing. I’m now a bit happier with it — it passes JSLint for a start, modulo that I disagree with Douglas about single-line if — by which I mean that I don’t cringe when I look at it.

(I haven’t documented the changes on the searchhi page itself, because the in-page search stuff is obviously JavaScript-dependent and that’s not how you should do it, so I don’t want to encourage people to use it.)

behave!

After watching @darrenf (a) tell an awful lot of people to behave on Twitter and (b) have to manually type it in every time, thus providing wear and tear on his hands which mean that he can’t fix bugs instead of me, I present: behave.user.js, a Greasemonkey script to tell people to behave on Twitter. Just hover over a name and click the ! to tell that person to behave. Winner.

Trying to understand YUI 3

I’ve been looking at YUI 3, because I ought to know about it and I currently don’t. I used YUI 2 for a few things and found myself perpetually frustrated with the documentation and the general feel of it, but I figured that maybe that was just me. (Disclosure: I’m a jQuery guy. Use it for everything. So I’m biased, right out of the gate.) Anyway, my frustration with the YUI documentation doesn’t seem to have been alleviated in version 3. This is a case study in how someone new to the YUI 3 library sees the documentation. (Well, it’s really a set of stream-of-consciousness rants, because that’s the best way I can think of getting my sense of confusion and discombobulation across.)

The underlying problem I always had with the YUI 2 documentation was that the examples show how to do one specific thing but don’t explain why to do it that specific way, and don’t seem very generalisable to do other similar things. So they’re hard to learn from. The reference manual, on the other hand, is useless. It gives you a textbox and says “Start typing to find a property/method/event/config”. If you already know what you’re looking for, if you know which function you want to use but can’t remember the order of the parameters that you need to pass to it, then that’s fine. It’s a dictionary. But you can’t learn a language from a dictionary, and you can’t learn the YUI from the reference manual. I hardly ever find myself thinking “I need to use the YUI.Get.script function but I can’t remember the name of the parameters”. Instead, I find myself thinking “I want to execute a JSON-P script; how do I do it?” Now, the jQuery documentation has a similar problem, to some extent, but critically it shows you all the functions in a list, so I can look through them and work out which is likely to be the one I want. The YUI does not. The jQuery docs also have inline executable examples with each function, so I can see working code and test it. The YUI reference manual does not.

But I shouldn’t start with the reference manual. The YUI docs specifically say to , so I did so.

The very first example is showing how to attach an event handler to a node. Fine as far as it goes (and I like the idea that you get passed a full-on event object to your function whether you’re in a browser that supports that or not), but…the code in the example just drifts in space. It’s not wrapped in an onDOMContentLoaded or anything. (I admit it’s the first example, which means you don’t want to load people down with infrastructure just so they can write a click function.) So, since there’s a working example in the page, I looked in the source myself, and we get

YUI({combine: true, timeout: 10000}).use("node",
function(Y) {
 ... code from the example...
});

Ah, OK, so that’s onDOMContentLoaded, is it? a new YUI function. I’m sure it’ll be explained later.

Moving on, the next example explains onAvailable, onContentReady, and event:ready — three different “is the thing ready for me to use” load events. (All of which make sense at different times, and are potentially useful.) That’s all good, but…the previous example, even after I’d looked in the code, didn’t use any of them! How did that work?

Ah, a feeling of doom creeps over me. More disclosure: I really, really don’t like the Yahoo practice of putting your JS inline at the bottom of the page. I think it means that my JS isn’t separated out into different files where it’s easy to edit, it makes refactoring the page hard because you have to care exactly where your script elements are and be careful if you move HTML elements around, and it’s a lot easier to teach people to make their JS unobtrusive if you can offer the rule “Put all your JavaScript in separate files and don’t have any anywhere in your HTML”. I know there are good reasons why Yahoo do it, and I accept those reasons. If I build a site which gets as many hits as they do then I’ll test both ways and the extra three milliseconds you get from having the JS in-page without DOMContentLoaded may make a big difference. For most of the stuff I write, though, I don’t like it and try to not use it. So, the first example was obviously reliant on that (because it’s not hooking any load event at all), but didn’t say so. Moving on, the next example I read, about nodes, does the same thing. As far as I can tell, nothing anywhere in these examples has said “make sure you put this example code after the things it’s referring to”. If you don’t do that (say, if you put it in the head because that’s where you’ve been taught that JavaScript should go) then it plain won’t work, and you won’t know why. (Maybe I missed the 72-point red writing which says this? I hope I did.)

Next example I read (event delegation, a thing I like very much): doesn’t show why the clicked items turn green (obviously CSS, and indeed it doesn’t take long to find the link in the source), but also…when I click on one of the items and it turns green, there’s a subtle animation effect to it, as if the green colour expands out from the centre of the item. It’s quite a nice effect. I can’t for the life of me find what’s doing it, though. (Doesn’t do it in Midori, a webkit browser, so maybe it’s Firefox?)

I spent ten minutes trying to work out why clicking on an item sets a class (”yui-pass”) on that item and then alters text in all non-yui-pass items, rather than just using e.target. Of course, the reason is that a second click on a different item should set that second item to green and not unset the first green item. Fine and dandy: a note about that in the example would have helped, though. Yes, the examples are there to demonstrate YUI, not to explain how to produce a specific effect, but…what people want is a specific effect. Explaining how it works would help with that. Explaining the YUI parts of an example but not putting them in the context of the rest of the code is like explaining how to build a house with a two-hour lecture on bricks without ever mentioning that you need mortar to stick them together. “The rest of the code” here is a couple of lines, but the example would make much more sense if it was all explained.

Once I’d got this far, I started thinking, hm, hasn’t changed, still the docs are written for someone who isn’t me and I don’t get them. So I started dotting around picking and choosing bits to look at rather than going through step-by-step. One of the features I like most about jQuery is that I can call a JSON-P script and still specify my callback function inline, rather than having to separate it out and give it a name and pass that name as callback=myCallback to the JSON-P script. (It gets given a name under-the-covers by jQuery, of course, so that the callback does have something to call, but I don’t have to think about it. I like inline functions.) It’s be useful to me if YUI 3 could do that, so I thought I’d have a look to see if it’s possible. I don’t think it is, but…I can’t tell. The docs don’t seem to mention the idea of loading scripts that call a callback function at all. Instead, I think you’re supposed to use YUI.Get.script and read the JSON out of the nodes array that you get passed. Perhaps. Here we come back to my problem with the YUI docs: I know what I want to do, but not how to do it, and there’s no way to answer my question. If there isn’t an example dealing with it, I am out of luck.

Someone tell me how I ought to be reading the documentation. Maybe I’m missing something. But it does rather feel to me like the docs are reference material: they’re there for people who already understand YUI. The examples are incomplete — focusing on just the part you’re trying to show is understandable, but if the example won’t work without the other parts then you have to show the other parts too! If your example only works if it’s wrapped with YUI().use("node"), function(Y) { ... }) then you have to show that. If it only works if it’s wrapped in DOMContentLoaded, or if it’s placed at the bottom of the page in a script element, then you have to say that. Yes, it’s a ballache to have to explain how events work and how loading works and not get to any real concrete examples until chapter 3 or 4 of your documentation. But people need to know that stuff or your examples will not work when someone uses them without knowing the context.

Video of “Secrets of JavaScript Closures” available

The Secrets of JavaScript Closures talk that I delivered at Fronteers 2008 was filmed, and the video is now available. You can watch it with streaming Flash video from the presentation page. (I’ve dropped a mail to ask whether I can re-encode it as downloadable Ogg Theora for people who don’t have Flash.)

An “up” button in the browser

At @media Ajax today, in the panel discussion session, someone said that they’d really like an “Up” button in the browser. (Apparently Seamonkey had it, or Opera did, or something.) For people who want it, here’s a bookmarklet: up

It looks for a <link rel="up" href="..."> tag and uses it if there is one; if there isn’t, it chops the last part off the URL.

Secrets of JavaScript Closures

Just come off stage after doing my presentation at Fronteers 2008 on the secrets of JavaScript closures. Go download it if you like that sort of thing.

(I’ve corrected the error that I got called on while I was on stage. Er. Oops.)

labelify: a jQuery plugin to add labels to your textboxes

A fairly common design pattern in web forms is to have some explanatory help text for a textfield appear inside the text field, and then remove it when the user clicks into that field. It has the benefit of putting the help precisely where the user’s looking.

Labelify is a jQuery plugin that does this for you, as simply as possible. It handles a number of corner cases, and it’s quite customisable if you need it to be, while still working as simply as possible out of the box.

Enjoy.

Fix embedded YouTube videos for Gnash users

I use Gnash instead of Adobe Flash, because it’s open source, and because I largely don’t miss Flash. The one thing I would miss, though, is YouTube. However, Gnash (in Ubuntu gutsy), while it works fine at displaying videos on YouTube itself, doesn’t work on embedded videos (where someone puts the YouTube video on their own site). GreaseMonkey to the rescue; this very short GreaseMonkey user script simply replaces embedded YouTube videos with a big link to the appropriate YouTube page, so I can click through and watch the video. Only useful for people who are as lunatic as me about freedom but still like YouTube, which might be an audience of one.

Clicking it should allow you to install it, if you’ve got GreaseMonkey installed already in Firefox; Epiphany users with GM enabled should right-click the link and say Install User Script.

youtubepopout.user.js

generated-toc: Generate a Table of Contents with the DOM

More excitement from the Stuart House Of JavaScript Stuff: generated-toc: Generate a Table of Contents with the DOM. A very easy way to get a table of contents onto your documents: generate a table of contents using JavaScript.

A big thanks goes out to the organisation who funded this work, who I can’t name. Getting stuff like this out into the world is a good thing, and them allowing me to open-source it is better still. Good work.

Capturing Caps Lock

Another JavaScript usability tweak from Stuart’s House Of JavaScript Weirdness: alert users if they’ve got Caps Lock turned on when they’re entering passwords into your web applications.

Capturing Caps Lock

Just install the script and go!

The other nice thing about this is that it’s part of 24 ways, the thing Drew McLellan runs every Advent showcasing 24 cool web things. I’m pretty pleased to have been invited. Drew is pretty pleased that this year I didn’t promise him something and then renege :)

Showing source in Firefox

Annoyed that Firefox pops up the source of a page in a separate window when you View Source? Annoyed that Epiphany shows the source of a page in Gedit? Help is at hand! Simply drag the following link to your bookmarklet bar:

Toggle source

Clicking it will replace the page you’re viewing with that page’s source, nicely syntax-highlighted, in the same browser tab. Clicking it again will go back to the page. That should keep Sean Middleditch happy and stop complaints about Epiphany using Gedit as an HTML viewer.

(Warning: I don’t know what happens if you do this on a page where you’ve just posted some data, like if you’ve just placed an Amazon order or something. Don’t do that.)

@media Ajax 2007

Finally returned from @media Ajax 2007, and I had a great time. I was a presenter, talking about How To Destroy The Web, which I thoroughly enjoyed doing. My slides are here:

(in-line presentation made with a bodged version of John Resig’s easy PDF sharing, but using libpoppler’s pdf2ppm because libpoppler can read my PDF and GhostScript can’t, for whatever reason. You can also get the presentation as a PDF or see it on SlideShare if you’re a freedom-hating Flash person.)

It was a great conference. Particular highlights for me were Derek Featherstone talking about accessibility (since I don’t know anything about it and I should do), being asked to be on the panel at the end (with Alex Russell (!) and Douglas Crockford (!!) and Brendan “inventor of JavaScript” Eich (!!!)), the coolness incarnate that is Firefox 3 and whizzy SVG stuff, having Chris Heilmann spend two days trying to convince me how nice London was even though it did nothing but piss rain solidly the whole time I was there, finally meeting John “Kelly Osbourne” Resig, and having a rather pregnant lady (who may have been this lady) tell me that my talk was so funny that she nearly gave birth during it. That’s not a compliment you hear every day, that one; I was pretty chuffed with that, I have to say. Thankyou!

The usual suspects have photos on Flickr (tag seems to be “atmediaajax”); the wifi didn’t work (but it never does, at any conference); we’re discussing what (if anything) to do with the WaSP’s DOM Scripting Task Force (set up after @media 2005: the question is, do we still need a “task force”? Hasn’t DOM scripting now become part of the toolkit? It does still need to be used properly, in moderation, and not be critical for it to be present, but the concept doesn’t need evangelising any more, I don’t think); I shall be back for @media 2008. Great work Patrick and your orange-shirted helpers. Cheers to a cool bunch of people: it was great to catch up with Jeremy again, and having a beer with Drew and Rachel and Chris and ppk and Bruce and Harry and Alex and oh, just everyone is a good way to spend an evening. Maybe even more than one evening.

hasAccount

Satisfaction, some new web service or other, has a neat wrinkle; on the sign-up page, you can specify your ID in one of a number of other services (flickr, technorati, last.fm, twitter, others) and it’ll retrieve your (publically available) information from there so you don’t have to fill it all in again. A good idea, I think. (It’s all done through the Magic Of Microformats.)

Something that would make it more useful, though, is if it highlighted which services it thinks you’re a member of. It’d be possible to do that if these online services provided a way to say “does the current user have an account here?” Imagine that Flickr, say, made available a URL http://flickr.com/services/hasAccount. When you call it as http://flickr.com/services/hasAccount?callback=myFunction, it looks to see if you, the user requesting the URL, are logged into Flickr. If you are, it returns this JavaScript:

myFunction(1);

otherwise (i.e., you don’t have a Flickr account, or you do and you’re not logged into it, or possibly you do and you are logged into it and you’ve ticked “hide my online status from other websites” in the Flickr account page) it returns

myFunction(0);

This would usefully let the Satisfaction signup page (and other services that want to offer the same sort of function) display which services it thinks you’re a member of: so for me, it would leave Upcoming and Flickr in the list and disable or grey-out or otherwise de-emphasise Cork’d and Twitter because I don’t have accounts there. This would be relatively trivial to do: Flickr just write somewhere “our hasAccount URL is http://flickr.com/services/hasAccount?callback=” and your code just calls that URL (by creating a <script> element with it in the src) and drops a callback-function name on the end of it so it can read the results. A nice usability enhancement, and it doesn’t particularly invade privacy (it’s a slight violation that site A can know that you’ve got an account at site B, but not a major one since they don’t get any details, and you allow people to opt-out if they choose).

For extra ease-of-use bonuses, although a quantum leap of privacy violation above that, have a hasAccount URL return

myFunction({”username”:”sil”});

instead, and then use hCard as above to automatically fetch the public information from that site. Obviously this is a lot less appealing from the point of view of keeping your information secret, though.

Apart from that it requires all the services to bother doing this, any reason why it shouldn’t happen?

DOMContentLoaded for IE, Safari, everything, without document.write

We have now had, for a while, a way of emulating the “DOMContentLoaded” event across all the main browsers. For those of you who don’t know what I’m talking about, a web browser fires an event, “load”, when a page finishes loading, and JavaScript can attach code to that event which will then run once the page has loaded. However, if there are big images or other slow-loading things in the page then the code gets delayed until after they’ve finished loading too; this is annoying, because most DOM scripting doesn’t actually need the images to be loaded, and so the delay is unnecessary. Opera and Mozilla fire a “DOMContentLoaded” event which runs after the page has loaded but before all the slow images and applets and Flash movies need to load, which is perfect. IE and Safari don’t fire such an event, but it’s been possible to do IE and Safari-specific things to make it happen. (In Safari you check document.readyState, in IE you document.write a <script> element with defer set. See Dean’s writeup.) If you’re using a JavaScript library of some sort (jQuery, Prototype, Mootools, whatever) then all this complexity gets wrapped up for you and you don’t have to worry about it, but if you’re writing stand-alone scripts (like my sorttable), which don’t depend on a library, then this is more of a problem; you have to include a big swathe of boilerplate code to emulate the DOMContentLoaded event.

Recently, Hedger Wang provided an alternative approach for IE that doesn’t rely on document.write, which is rather neat. So, it should be possible to combine the existing Safari method, Opera and Mozilla’s DOMContentLoaded support, and Hedger’s IE approach into a short bit of boilerplate code that can be dropped into your standalone script. And indeed it is. See a simple demo, and more importantly, here is the short version of the code:

(function(i) {var u =navigator.userAgent;var e=/*@cc_on!@*/false; var st =
setTimeout;if(/webkit/i.test(u)){st(function(){var dr=document.readyState;
if(dr=="loaded"||dr=="complete"){i()}else{st(arguments.callee,10);}},10);}
else if((/mozilla/i.test(u)&&!/(compati)/.test(u)) || (/opera/i.test(u))){
document.addEventListener("DOMContentLoaded",i,false); } else if(e){     (
function(){var t=document.createElement('doc:rdy');try{t.doScroll('left');
i();t=null;}catch(e){st(arguments.callee,0);}})();}else{window.onload=i;}})(init);

The init on the last line is the name of your “init” function; the one that you want to call on page load. (It could be an inline function if required.) Simply copy those 7 lines into your standalone script and change the name of the init function to be your init, and you’re good to go on all browsers. Remember that if you’re already using a JavaScript library then you don’t need this, but if you’re not then it’s simpler than the current swathes of code, and less heavy than depending on a library just to get access to cross-browser onload event handling.

Alternative DOMContentLoaded for IE

Interesting new JavaScript trick from Hedger Wang: poll the doScroll() method of an unattached DOM element until it stops throwing errors, and when it does, the DOM’s loaded. This gives you the equivalent of the DOMContentLoaded event for IE, and in a rather nicer way than the current-best method of document.write-ing a script tag with a defer attribute. I like this a lot, although I’d like to see some more detail about whether it works in all IEs before switching to it!

Tracking bugs for sorttable

I get lots of people mailing me for help with things I’ve written, and sorttable is way up at the top of the list. People find it useful, which is good, and they mail me to say thanks. I like that. On the other hand, they also mail me to say that there are a few things it doesn’t do. Now, they’re right; there are a couple of bugs that slipped through, and more importantly there are a couple of things that never crossed my mind until people started asking whether sorttable could handle them — loading tables with Ajax, tables with zebra stripes, that sort of thing. I’m now at the stage where in my “sorttable unprocessed” folder I’ve got over 150 emails, and an awful lot of them are about the same things.

So, a bug tracker seemed like a good idea. However, I don’t want one where everyone can submit bugs into it. I get little enough time to hack on projects as it is, without having to triage bugs as well. So, I wanted something which was basically read-only; a place where I could point people to say “look, here, the problem you have identified is a known issue, and I am planning on fixing it, and you can subscribe to this and get notified when I do fix it”.

So: the sorttable bug tracker.

It’s shamelessly borrowed from the PuTTY wishlist in almost all particulars, right down to the differing levels of priority. Simon Tatham actually pointed me, when I asked, at the code they use to run that wishlist. I had a couple of extra wrinkles I wanted to add, though, and their code is in Perl and I’m not much cop at Perl, so I built my own very similar thing. You can read my spec for a read-only bug tracker if you like; essentially, the bugs themselves are in a (secured) folder in my Subversion repository as RFC822-style files, and then there’s a little collection of scripts which know how to build a static HTML collection of bugs for the project. I’d like to make the scripts available so other people can do the same thing, but they’ve got passwords hardcoded into them (because the bug area in svn is secured, because there are bugs in projects that I can’t make public). If I can think of a way of making the code available, I’ll do it, but it’s relatively trivial anyway.

At the moment I’m only tracking sorttable bugs in here, but I’m gradually going to bring all my other projects that are me-develop-you-use things into it. It’s not suitable for something like Jackfield, which will eventually develop more of a community around it (and will therefore be a lot more collaborative); sorttable isn’t a big enough deal to be a community-maintained project, but I do need somewhere I can point people at when they say “why doesn’t it handle Ajax-created tables?”, and now I have that somewhere.

All busy on the Western front

Stone me I’m busy. Bah. Haven’t even had time to write some things up here. However, my collection of kept-unread things in Google Reader has reached high enough proportions that I have to do something about it, so this gets a few things off my list.

A post about the stuff I discussed at LugRadio Live — whether it might be reasonable to make “be permissive about IP” be the default for Linux distributions and make the Americans and others in oppressive IP regimes do the legwork to remove what they consider to be “patent-infringing”, rather than all of us non-Americans suffering for it — is stacked into the holding pattern and circulating over Heathrow, incidentally.

Recent interesting things

Google Reader finally lets you search past posts
Thank god for that. Finally. Finally.
ATI open graphics specs and open source the driver for video cards
This is a pretty big deal, this one. At first blush it looks as if two of the three leading graphics card manufacturers (ATI and Intel)* will now have good support in open source. I imagine someone at nVidia has at least noticed that this has happened. More openness of this sort cannot be a bad thing. Well done ATI, I say.
I should note a couple of small reservations: it’s only for their newer cards (i.e., not most of the ATI cards that are out there), and it’s not as simple as them just open sourcing their existing fglrx binary driver. They’re providing a library to access the BIOS on the card, but most importantly they’re providing specifications for the cards. It’s still down to the open source community to write the driver itself, but we’ve historically not had a problem with that (and ATI are funding the initial writing of this driver!). Phoronix has more detail on what ATI are actually doing — the important quotation is “The aim of this open-source driver is not to overtake the fglrx driver but rather is designed for those who just want a working desktop with 3D capabilities and basic video playback. This new driver is ideal for FOSS enthusiasts” — the point here is that if you want the very best performance from your ATI card you’ll still need to go proprietary, but there’s now enough information that the open source code will be able to provide the things that a lot of people are clamouring for — 3d acceleration, video acceleration, 3d desktop effects. Well done, ATI, I say. Obviously I’d like to see it all being Free, but I’m confident in the open source community’s ability to build a great driver (maybe even a better driver than the binary one?) based on having the information they’ve asked for, and now they’ve got it. Blizzard’s thoughts are informative.
@media Ajax 2007
I’m really looking forward to speaking at @media Ajax this year. The list of speakers is pretty cool, indeed, including some people I’ve never had the chance to meet. As usual, details of events I’m going to be at are on my events page.
Separated at birth
John “jQuery” Resig says that he’s “baffled” by why I referred to him as John “Kelly Osbourne” Resig in an earlier post. Well, the camera doesn’t lie.

John Resig and Kelly Osbourne

On the other hand, John did an excellent Google Tech Talk on building a JavaScript library, so don’t hold it against him.

Convert a physical Linux box into a VMWare virtual machine
Useful, although convoluted. Parallels lets Mac people virtualise from a bootable partition, so you can either boot into your image or run it as a VM inside another OS. Can we do that under Linux? It’d save my dad rebooting into Windows to make his scanner work.

Speaking at @media Ajax

The @media people have just posted a schedule for @media Ajax, in November this year. In addition to seeing me talk about how to destroy the Web*, there’s a veritable smorgasbord* of other speakers, like Brendan “I invented JavaScript” Eich, Douglas “I really understand JavaScript in a way that no-one else does” Crockford, Peter-Paul “quirksmode” Koch, John “Kelly Osbourne” Resig, Alex “Dojo is cleverer than you are” Russell, and others. It should be a good bash. I’m told that ticket sales are nearly closed, so if you want to come to London to see all these great people you need to get in quick. Get stuck in.

Chat to me

My contact page now allows you to chat to me in real-time without having to sign up to an IM service or anything. Steve Chipman, AOL hacker, took a break from painting good pictures of everyone who crosses his path to post about AIM’s new Wimzi widget; if you’ve got an AIM account, you can just go to the Wimzi site and get a widget for your website that allows people to talk to you even without their own AIM account. (It does require JavaScript, mind.) Good work. At the moment I’m just using the stock widget, but I’d like to customise it so that it fits better into the look of the site. Nonetheless, this is a pretty cool thing, especially when you think that this doesn’t have to be used for chat; it’s basically a generic way of throwing data around between websites and networks and computers over a checked network without signup required and easily. On the other hand, that network’s controlled by AOL, which might be a problem. On the third hand, AIM pretty much ranks as the best thing that AOL have done, even if the Pidgin project did have to change their name away from Gaim because AOL’s lawyers complained. I imagine it works with iChat names too, since they’re just AIM under the covers; so, let people chat to you! Open up the flow of discourse. Good work, Steve and the gang.

Yahoo Hackday is over

What a fun weekend.

I had a thoroughly enjoyable time at the Yahoo/BBC hack day this past weekend. Dave Neary says “lightning struck twice”, referring to how he was left out of the programme for the conference he was at. Pah. Lightning did actually strike twice; the building I was in was struck. By lightning. Twice.

Excuse me while I say that again. The building I was in was struck by lightning.

Now there’s a pretty unique experience. Although it’s hardly surprising since Alexandra Palace has a 40 metre metal mast on the roof which looks like the Eiffel Tower and probably weighs about ninety tons.

Rumours abound that Google’s new weather API is actually a writeable API. If it were, it’d probably look something like Simon’s Google Smite. To be honest, the bolt hit the building just as I and a couple of others were slagging off Microsoft Silverlight. Draw what conclusions you will.


The roof panels open after the lightning strike, allowing cool refreshing rain to land on everyone’s laptops

Aside from inadvertent electrical events, a lot, lot, lot else happened. There were many cool hacks done with the BBC and Yahoo APIs, seventy in total; I think my favourite was Fruitr, which identified pictures of fruit that you emailed it and then suggested mad recipes in which to use it. Other highlights:

  • Jeremy Keith, Nat Downe, and others actually working really hard for the entire weekend and coming up with Hackfight, Top Trumps where your scores are based on how many twitters you’ve done, how high your Technorati rating is, that sort of thing. So it’s a game where you can find out who’s the biggest gimp. Cool hack, mind. :-)
  • Me trying to make sense of Simon Willison’s Oxford Geeks code and build a Birmingham Geeks site around it in one hour. And then discovering that the git depended on all of Django just so he could use its template engine for one 10-line template file. Good one.
  • I am going to set up Birmingham Geeks, though, so if you’re in this area leave a comment or something and when I do it you can see how to get on there. This will finally give me an excuse to go and have a beer with Bruce Lawson, which I’ve been meaning to do for about six months.
  • Travelling down on the train with Matthew Somerville and finding out about all the cool stuff that MySociety are doing, including a wicked cool hack with the BBC Parliament videos that I’m really hoping to see happen.

  • The wi-fi being constantly shit for all of the first day, even before the whole building was zapped. Pretty coloured circles on the Cisco wi-fi access points, though. Has anyone ever done a conference where the wi-fi worked properly? (Yes, I know about the Pycon writeup.)
  • Having two people come up and ask me if I was me based on recognising my voice from LugRadio. Cool. LugRadio Live is in three weeks, remember, people!
  • Meeting up with some people I hadn’t seen for a while and meeting some for the first time
    Aral 'Flash hero' BalkanA chap from Hacker Voice Radio whose name I never caughtJacob 'Django' Kaplan-MossSimon Willison and Andy BuddDave Glass, new Yahoo recruitNorm
  • Getting hassle from Christian Heilmann again about moving to London and working for Yahoo. I’m not moving to London, dude. Open a Birmingham office! Embrace telecommuting! It is the 21st century! :-)
  • I got pretty pissed off with my laptop, mind, which is fine when sitting on my desk but, what with it being stuffed full of proprietary hardware (listen to this week’s LugRadio for more on that) doesn’t work very well on the wireless (having to reboot to make it re-detect it five or six times a day, grr), hibernate properly, or work with a projector without rebooting. I need a conference laptop which doesn’t have a load of undocumented Broadcom and ATI hardware in it and doesn’t weigh very much. I must try and find a very cheap second-hand one somewhere.


People hide their laptops from the rain with umbrellas inside. An unusual sort of sight.

It was great. I did think about putting some hacks together but, to be honest, I was enjoying chatting away and meeting back up with people so much that I didn’t really get around to much actual coding. Simon was kind enough to credit the London Geeks stuff he did to “The Oxford-Birmingham-Kansas alliance”, with me being the Birmingham third of that, but that’s about all. I do have a plan for making Jackfield support Yahoo widgets, mind. Oh, and I helped out a very small amount with Gerv Markham and Ewan Spence’s rocket launcher, which was entertaining in itself and provided more than one laugh on the day. My pics available of the weekend.

Good work, Yahoo and the BBC. I have no idea whether the weekend met any of the goals you had for it, but you’ve convinced five hundred people that (a) you’re good companies, (b) there are many APIs out there which need mashing up, (c) it is possible to live for two days on Twixes and nervous energy, and (d) the world needs more weekends like this. When’s the next one? Bring it on.

Ads and directories

The most popular thing I’ve ever written is the JavaScript table sort library, sorttable. Recently, I got into a discussion with the LugRadio chaps about advertising on websites, and whether it’s a good idea or not. I’ve toyed in the past with the idea of adding adverts to this site; it somehow feels a little distasteful to me, but during the discussion it became apparent that I don’t actually have any reason to back that up; it’s pure prejudice. Somehow it feels less pure to do something if you make money from it: the phrase “filthy lucre” exists for a reason, and the love of money is the root of all evil. On the other hand, I do love money since it lets me buy cool things. So. To ad or not to ad?

On the other hand, I got an email from the Dynamic Drive people asking if they could feature sorttable in their directory of JavaScripts. Now, one of the things that the JavaScript community have discussed (and this came up in this past weekend’s JavaScript pub meet — pics available) is trying to get good, well-coded, modern scripting into these huge galleries of download-and-use plug-n-play scripts. The idea here (and not everyone agrees with me on this, I feel bound to say) is that we are never going to convince some people to actually give a damn about the quality of scripts that they use or write. They’re just going to think, “I want some effect, I googled for that effect, and here’s a script that does it, so I follow the instructions to drop that script into my site, job done.” Since that’s going to happen, we should make sure that the scripts that people find are decent ones. That’s why I wrote sorttable, for example: to ensure that people have a simple drop-in decently-coded way of doing client-side table sorting. One good way of doing this is to talk to the current big repositories of scripts, like Dynamic Drive, and work with them to try and make sure that their scripts are good ones.

But…if I put the script on Dynamic Drive, then some people will get it there instead of from me (even though there’ll be a link to my site) and…that’ll hurt my ad revenue, if I have ads.

So, questions therefore, for those of you out there reading this, whether you’re a JavaScripter or a website owner or just a consumer:

  1. Would it be a good idea or a bad idea to have ads on some of my pages? (I’d probably do the browser code and other code pages, and old (not current) weblog posts.)
  2. Should I put the script on Dynamic Drive?
  3. Should I do both?

Your opinions greatly received in the comment box.

Nicetitles now handles image links

My nicetitles JavaScript library now handles links with images in them. Countless people have mailed me about this over the last four years; finally Vigdor mailed me and chased me down on instant messenger and I decided to get off my lazy arse and fix it. It now works. There is clearly a lesson here for those of you who want me to do stuff.

Sorttable v2: making your tables even more sortable

Sorttable, the JavaScript table sorting library, is pretty much the most popular thing I’ve ever written. (Top hit for “javascript sort table” on Google, too.) Over the four years or so since I first released it, there’s been a steady trickle of thanks, feature requests, and bug fixes to it. I’ve finally got around to releasing a new version of the script that hopefully covers everything that people were looking for. You can get it at the same place as it ever was, sorttable: Make all your tables sortable.

What’s changed in version 2

The most important thing that’s changed is that sorttable now has a much much clever automatic column typer. You still can just drop sorttable in place and not have to do any configuration, but it’s quite a bit cleverer about working out whether your columns are numeric or dates or currency or text. It skips blank lines, it can tell the difference between English and American format dates, it understands numbers with commas in and negative numbers. This is the number one feature request.

Secondly, you can now use custom sort keys for those columns that sorttable can’t work out for itself. See the page for details, but this should allow anyone to make anything work with sorttable v2.

It’s also a bit more compliant with HTML standards. To add a “totals” row you should add the row to a <tfoot> section in your table. Sorttable v1 used a class of “sortbottom” on totals rows; for backwards compatibility this is still supported, but you really should be using <tfoot> instead.

In a similar way, sorttable now uses <thead> as your column headers if you have it. If you do not have it then it will create a <thead> and put the first row of your table in it. This is a backwards incompatible change: to apply style to your table headers, you should now style table.sortable thead, not table.sortable a.sortheader as in v1.

Input elements are now supported in table cells! If your table cells contain input elements then sorttable will use the value in the input element.

Sorttable is now quite a lot faster than it used to be, owing to various little tweaks. It’s especially fast at resorting the table by the column you’ve already sorted by (but reversed), because now it just reverses the table rather than doing the sort.

Stable sorting is supported but commented out in the code (because the stable sort is implemented in JavaScript, not using the native list sorter, and so it’s a lot slower). You can change this by editing one line in sorttable.js if stable sorting is important; see the page for details.

Thanks

Lots of people (really, really lots) have contributed code suggestions, feature requests, bug reports, and compliments about sorttable. Here’s my chance to say thankyou. So: thankyou to…

Eric Chang, Mehmet Kut, Dan Synder, Jeff Bradley, Eric Holstege, Victor Kryukov, Nikola Ivanov, Geoff Clevenger, Mikko Suniala, Scott Reynen, Justin Meyer, Leonardo Kenji Shikida, Ben Ostrowsky, Daniel Berlin, Richard Snazell, Mark James, Eric Rizzo, Garve, Andreas Rehm, Scott Kingdon, Marcus Månsson, Bill Barry, Michael Rumpler, Avi Rogers, Kevin Wu, Espen Hovlands, Petronel Laviniu Malutan, John M. Pierre, opus27, Brendan Bowles, Marty O’Brien, Alex Rubin, Chris Gay, Jack Vanhan, Guy Rosen, John Dawson, Ryan Korczykowski, Carl Forde, Pete K, Glenn Haser, Richard Grassy, Olivier Briat, Bob Hanson, Alex Z, Narayana Rao Kankipati, Rey Hernandez, Eric Moriztz, Marco Valk, Marcus Daniel, David Dockhorn, Angela Weise, Aneel Nazareth, Jeremy Hulford, Thomas K. Weinstock, Edward, Matthew Egbers, Anton Berezin, Moustafa Elqabbany, Justin Williams, Daniel Chace, Inigo Surguy, Vladimir Kornea, John Mund, Pablo, David Golden, Brad Kemper, Brian Schott, Christian Robottom Reis, Justin Haygood, C. David Eagle, Simon Willison, and Dean Edwards.

Phew! All those people have contacted me asking for new features or have sent me patches or have provided code that I was able to reuse in v2 of sorttable. Thankyou: for those of you who wanted sorttable to do something new, I hope you like it.

DED|Chain

Dustin Diaz releases DED|Chain, a JQuery-a-like library based on the Yahoo UI library. Apart from winning this week’s “Most Ridiculous Name For A Software Project” award (previous winners include Ning), it looks pretty cool. Nice work, Dustin.

There is one concern, though. When I say it’s JQuery-a-like, I’m not kidding; it looks a lot, lot like JQuery. This isn’t necessarily a bad thing; I use JQuery all the time because I think it’s brilliant. However, it’s pretty clear to me when I’m looking at arbitrary JS code whether it’s JQuery code or not. Now, with D|C in the world, I’ll have to be pretty careful to know which bits are DED|Chain bits and which are JQuery. In fact, I’m inclined to not use D|C too much purely because I don’t want to pollute the bit of my brain which remembers JQuery things with parts of D|C.

Of course, it’s difficult to know precisely what D|C supports because the link to the docs (http://dedchain.dustindiaz.com/api/docs/, as mentioned in readme.txt) is 404, but I’m assuming that that’s just because we’re so early into release.

There are now sufficient libraries out there that one useful part of a new library’s documentation would be to show “translations” between one of the “headline” libraries (YUI, Prototype, JQuery, Dojo…what else should be in this list?) and your new library; it’ll help people who are already using libraries to understand why they might want to switch to yours, and it won’t hurt people who are approaching your library as their first. So, a few examples which say “to do this effect, you’d do this in YUI, and you’d do it this way in MyNewLib” would not go amiss for new JS library developers; what you’re trying to do with that is convince someone like, oh, say, me to move from their current library of choice to your new one. The advantage there is that people will go out and evangelise for you. Of course, getting someone who doesn’t use a library at all to be convinced by yours is equally a coup; you can’t dodge the need for proper documentation by just saying “read Prototype’s docs and then convert your code from Prototype to MyNewLib” :)

I was faintly surprised that D|C is an actual new library rather than just a “port” of the JQuery API to YUI (meaning that I could switch my JQuery apps to D|C without changing any code, and get the (debatable) benefit of the “reliability of Yahoo! UI”), but I guess that YUI’s asserted reliability isn’t enough on its own to motivate a port, and that’d stop Dustin doing anything innovative.

Library proliferation is, in theory at least, a bad thing, but I don’t think so. We’re so early in doing structured JS development in a web-wide, collaborative way (not just inside one company, say) that it’s worth exploring lots of different approaches to library development and JS development as a whole, and see in a year or so what’s emerged as the leading techniques.

Right, now I must go away and have a play with DED|Chain. Stupid name. I can’t get out of my head the image of a chained and dead pirate, swinging from the gibbet in eternal torment. An idea for a logo there perhaps, Dustin.

iTheora and Java support detection

Interesting project: iTheora, a PHP script handling in-browser display of Ogg Theora video files, using the Flumotion Cortado Java applet. The site’s originally in French but has also been translated into English by Bruno (nice one Bruno!).

I’d like to do this client-side with JavaScript. The big issue I find with Cortado, and indeed all Java applets, is: Java takes so long to start up. What I’d like to do is only load the Java applet if someone asks to see the video, and not show it at all if they don’t have Java installed. The problem is: how do you detect whether the punter can display Java applets, without starting Java? As far as I can tell, it isn’t possible. I’ve seen a few attempts at it, but none of them work across browsers. JavaScript people: how do you detect Java support without starting Java? I’m quite happy to write the script to do it if I can work out how, but all my experimentation and research have not led me to a solution. Lazyweb: help me out!

No highlighting in searchhi

A chap mailed me to say: is it possible to have your searchhi JavaScript library, which automatically highlights things that the user searched for if they came to your page from Google or similar, not highlight some bits of the page like the navigation section?

Good idea, I thought. So the library has now been updated to allow you to do precisely that: stop highlighting bits if they’ve got a class of nosearchhi. Thanks to Helmut Lorenz for the idea!

Shortest way to create an XMLHttpRequest object

This should work in everything, I think, to get an XMLHttpRequest object for JavaScript in your browser:

http = window.XMLHttpRequest ? new window.XMLHttpRequest : (window.ActiveXObject ? new ActiveXObject("MSXML2.XMLHTTP"): null);

Of course, this only applies if you’re not using a library to make all this stuff go away, which you maybe should be. But if you’re not, this is a very short way to get the thing you need for Ajax.

LugRadio season 4 episode 12

The LugRadio team proudly presents “The importance of being critical”, episode 12 of season 4 of LugRadio. Those of you who are already part of the LugRadio community will of course know about this, but those of you who come here for web stuff rather than Linux stuff may be interested in the big discussion in this show about web applications, desktop applications, why use one rather than the other, and me ranting about what “progressive enhancement” means. In particular, I’d be interested in comments on any full-on web apps that genuinely use progressive enhancement techniques, in that they work with simple or text-only or mobile browsers but provide a better experience to those people with modern ones, and they do something actually proper. I spoke about using progressive enhancement to improve existing apps a little, but was challenged to name a full-on app doing something complex like Google Spreadsheets that would work without Ajax, and couldn’t do so. Help appreciated.

Truprint Firefox extension

Truprint, a photo processing firm in the UK, have leapt into the digital world with both feet (as have all the other photo processors) by allowing you to upload digital photos to their website for printing as actual real photos, which they’ll then send back to you. We’ve always used Truprint, and last night we thought we’d try out this new upload-and-print-out thing, what with all our cameras now being digital.

I was surprised and pleased to discover that, to upload to their site, they provide a little tool called QuickUpload. But it’s not an ActiveX control; it’s not a Windows-specific application. It’s a Firefox extension!

They do have an ActiveX one for IE users, but they’re also supporting Firefox. Bloody well done Truprint. This sort of thing should be applauded. I have mailed them to tell them so.

I also mailed them to mention that the extension said it’d take 10 minutes to upload our photos and it actually took two hours, but hey, all software’s got bugs.

Following comment threads

It really massively annoys me when I post a comment to someone’s site and then have to keep checking back to see if anyone’s followed it up. Now, I’m aware of coComment but I don’t like it much; people have to sign up for it, for a start, and that’s just one more place to sign up for. (This might be helped with OpenID or something, but whatever.)

So all my comment forms now have a “Post comment and follow comments in your feed reader” button as well as the normal Post Comment button. Write a comment and click that and it will post your comment and then take you to a page where you can choose your feed reader; you’ll then have any further comments on that post appear in your reader when they happen. The thing that chooses your feed reader is Aquarion’s SubscribeMe, so I don’t have to do the heavy lifting in finding out how to throw you a feed in such a way as to make it appear in your reader.

It’s all done with JavaScript, not surprisingly. At the moment the script is pretty specific to my comments pages, but I plan to make it a bit more generic so that you can just pick it up and add it to your comments pages to get the same feature for your site. Make it easier for your commenters to follow along with a comment thread.

(For those of you who really want it now, it’s available. It requires JQuery and the JQuery form plugin at the moment, although the final version won’t. To adapt it for your site, you’ll need to change subsc.FEED_READER_URL to make it construct the URL for the RSS feed for comments for the post you’re on, and also change inps[i].value == 'Submit Comment' so that it finds your existing submit button. If that didn’t make a lot of sense to you then I advise you to wait until I release a more generic version.)

Let me know if there’s a problem with it. You might also want to let Aquarion know if your feed reader isn’t supported, or you can tell me if you prefer.

The fog of libraries

It’s pretty foggy at the moment, here in the Midlands in the UK. It’s been like it for two days. Stepping out of work the other day, after dark, with each street light at the centre of a glowing aura, made the street I work on — in the centre of Birmingham — look like something Victorian. Like a woodcut from a Sherlock Holmes book. I wouldn’t have been at all surprised to see a hansom cab pull up and a man with a sword-stick get out.

Of course, I wouldn’t have been surprised to see Jack the Ripper hiding in a doorway either.

It made it feel like it was nearly Christmas, which it is. It was really rather difficult to remember that it’s still the same old modern street underneath.

So now I’m going to talk about JavaScript libraries. You’ll see how the fog is relevant in a bit.

There’s been quite a bit of discussion of libraries recently. Jeremy Keith writes:

There’s a lot of talk about JavaScript libraries, including a lot of hype and cheerleading, but I think that maybe the discussion is disproportionate to the amount of people actually using libraries. Personally, I’m somewhat mistrustful of using other people’s code (I’m a bit of a control freak) and I thought I was in the minority, but having spoken to people like Stuart and PPK who share my feelings, now I’m not so sure.

PPK himself says:

a quick and non-scientific poll @media showed that only a minority of JavaScript developers use a library; the others just code by hand. But JavaScript libraries are cool nowadays and will remain so until the Ajax hype blows over.

Chris Heilmann complains mightily about libraries in general, although about specific details rather than about the concept of using libraries as a whole.

And Roger lays into people who use them, too:

Overuse of JavaScript frameworks/libraries. Back to the 90’s, baby, except they were called DHTML libraries that time around. What is it with people learning a JavaScript library instead of learning to write JavaScript? It’s like learning Frontpage instead of HTML. Yes, script libraries can be great. But not when people use them because they can instead of because they should.

Lots of people are getting into the whole idea of using a JS library. Among the cognoscenti, though, there’s starting to be a bit of a backlash. “Putting guns in the hands of children” is how the discussion was phrased during our JS panel at @media 2006. The idea behind this discussion is that libraries give developers access to lots of cool powerful stuff — hiding and showing areas of the page, Ajax requests, animation — without them having to really demonstrate that they know what they’re doing. It’s an argument with a certain amount of truth in it; it’s used a lot by gun-control advocates as well, because if you have to go through five years of training to become a karate black belt then you’re much less likely to kill someone after leaving the pub, and much less likely to accidentally kill a member of your family, than someone who’s bought a Saturday night special and hasn’t had any training.

On the other hand, it’s a pretty elitist viewpoint. Apparently, you shouldn’t be allowed to use JavaScript unless you’re an expert. New developers, people just looking to get their job done, someone who wants to apply a cool effect: these people are not wanted in our glorious new revolution. Dammit, programming ought to be hard: it keeps out the idiots. This attitude comes up a lot, a lot, in the Linux world, and it’s one of the reasons why Linux is perceived as being harder to use than, say, a Mac: I’m surprised that there are Mac people out there who are highly happy* with the ease of use of their computers and don’t have to understand one iota about how they work under the covers, but still believe that JavaScript developers should be held to a higher standard.

What drives adoption of technology is real ordinary people, people who come in at 9am and clock off at 5pm and don’t think about work outside those hours, being able to use those technologies. JavaScript libraries bridge that gap. I’ve changed my opinions on this quite a bit; I used to hand-code all my JavaScript myself, and now, for quite a few of my projects, I use jQuery. I just got sick and tired of thinking: how do I hook up events in IE5.5? Is it different to 6.0? Where’s that cut-and-pasted code for creating and using an XMLHttpRequest object in every browser? I’d ended up building a tiny library for myself of bits of code I cut-and-pasted into every project, and my library was inferior to others because I don’t do as much testing in every browser. So, why not use a library? I couldn’t think of a reason. Now cross-browser testing is John Resig’s problem, which is handy.

The fog hid the nastiness of Birmingham. Libraries hide the nastiness of cross-browser scripting. While it’s easy to say that you should have to know about the nastiness, this is the season of goodwill. It’s nice to step out of your office into a foggy lit street that looks like a scene from It’s A Wonderful Life. Let’s not take that away from people.

This doesn’t answer Chris Heilmann’s objections, though, and that’s because they’re a damned good set of objections. The libraries we have are in general not well documented; a list of functions that the API provides does not help if you know what you want to do but not how to do it, because you have to guess at what the library might have named this function. Making the documentation available only online doesn’t help either, although this is something that really could be fixed by someone other than the main library developers — most documentation issues, including a proper decent set of examples of how your library can progressively enhance a page, with a case study or two, can be fixed by people outside the core team. So if you really hate this issue, jump in and get involved by building some docs for the library you like the most.

None of these problems are insoluble, though. Part of the issue is that JavaScript developers divide into:

  1. People who aren’t hugely experienced with JS; these are at least some of the target market for a lot of libraries
  2. People who are experienced and are writing a library
  3. People who are experienced and think that you shouldn’t use libraries unless you have the experience to not use them
  4. People who are experienced and willing to help

Of those, category 1 people aren’t going to help write documentation very much, or case studies, or browser support matrices, because they’re consumers of that information rather than creators. (Those of you who think “yes! we’ll have a huge army of people contributing!”, take a tip from the Linux world, where we’ve been using that model for code since the beginning; it happens, but not all that often.) Category 2 people are too busy writing a library to build supporting stuff around it. Too many of the remaining: people who could be helping to make the libraries we have better, and better documented, and more appropriate, and less stuffed to the gills with flashy but bad effects, are too busy repudiating the entire concept of libraries. They’re in category 3. If we can collectively, as a community, move from category 3 to category 4, and make the libraries we have great, then there won’t need to be any kind of objection to them. Let’s try and help.

Speaking at Skycon

As mentioned a few days ago, the LugRadio team are going to be at SkyCon in Limerick in February 2007. Since we’re there anyway, they asked me if I’d like to do a talk. I said I was happy to, and they asked me to choose a subject.

Now, those of you who read this a lot will have noticed that the stuff I write about tends to be a mix of Linux and the web, and a mix of hard technical information and windy philosophising. So, that suggests something of a framework. If I were speaking somewhere and you were attending, what would you like to see me talk about? It strikes me that the talk could be about Linux or the web, and be either technical of philosophical. Your combinations therefore break down as something like:

The web + technical
This would likely be heavily DOM Scripting focused; JavaScript’s a growth area and it’s What I Do to a large extent. Think of this as something along the lines of bits of DHTML Utopia in talk format.
The web + philosophical
Where’s the web going? What’s the best way to use all this Ajax technology? What’s on the horizon? Why are we wasting web technology? That sort of thing.
Linux + technical
This’d likely focus on some bit of Linux software I write, like Jackfield or Jokosher; technical detail again of how we did the Extension API in Jokosher, for example, or a demo of Jackfield and what’s coming up for it, or some other bit of software I participate in.
Linux + philosophical
This is the sort of thing we talk about on LugRadio a lot. How’s the Linux desktop shaping up? Where does it need to go to be better? What’s the vision for the future? Why should you care?

What would you like to see me talk about? Maybe it’s something completely different to everything above. Post your comments now!

AOL AIM in JavaScript

Steve Chipman and the gang begin to unveil JavaScript bindings for AOL Instant Messenger. This is going to be pretty cool; we’ll be able to tie AIM (and its two hundred trillion users) into websites with the greatest of ease. Bring it on.

Google Reader is not good

I’m starting to get really hacked off with Google Reader. I’ve mentioned this already, but I thought I might fill in a few more reasons. You might be thinking: why haven’t these been submitted as bugs to the Reader team? Well, most of them aren’t really susceptible to that. I’ll come back to this point later.

The old version kept dying all the time: it popped up a “loading” screen and it never went away. Forced refresh sometimes brought it back, and sometimes didn’t. When the all-new exciting version came out I thought that this problem might be fixed: in fact, it’s actually got worse. I find Reader hangs probably once every three or four times I use it.

There’s no point in doing Ajax things if you have to wait 20 seconds for it to load. I’ve already ranted about this, though.

As far as I can tell, you can no longer skip to the next item by pressing space. Maybe this works for everyone else, because it ties into…

If you have All Items shown, and some new items have arrived since you clicked it, I can’t find a way to refresh the list of All Items. I have to switch to Home and then switch back to All Items again, with two long waits with a loading screen. Apparently “r” is meant to refresh, but that doesn’t do anything. I’ve tried “r” and alt+r and neither seem to help.

It gets confused about whether articles have unread items or not. See here:

The left hand pane shows Python News with a (1) for one unread item, the header shows

I could not get Reader to show me this mysterious unread item, or to acknowledge that it wasn’t there. I tried a full refresh (just loading Reader again), I tried switching to Home, back to All Items, I tried showing the feed individually (which is what the screenshot shows). This also happened with other feeds; the only way I could get the supposed new items alert (the “1 new items” and the (1) in the left pane) to go away was to “View all items” (all of which appeared to be read) and then say “Mark all read”. Now, though, I’m left with the lingering suspicion that there really was an unread item and now I won’t ever get to read it.

(Plus: 1 new items? Hm.)

In terms of filing bugs: what should I do about that last one? I have no idea how to replicate it, I can’t give instructions. I could say “look at my Reader page and you’ll see it”, but then I basically can’t use Reader until I get an acknowledgement that the Reader team have diagnosed the problem, which I don’t want to do.

Anyway, as soon as I’ve got my customisations done to Rawdog, I’m switching. Sorry, Google team. I have to say that Gmail is increasingly getting confused and having to be full-refreshed or just hanging on the “Sending…” bit when trying to send a mail (it’s still an Ajax loading screen, even if it’s just in the top right hand corner). Makes me sad. If Google can’t make this stuff work, who can? Maybe I should look harder at the Yahoo stuff instead? They seem to be coming on strong recently.

Ajax loading screens

Right, here’s the thing.

Using Ajax technologies is pretty cool. They can help your web application really feel smooth and whippy and quick. Like a desktop application. However:

There is no point in using Ajax if you have to pop up a “Loading” screen and wait ten seconds when a link is clicked.

Read that again. There’s no point. There is no difference in the UI between “click a link, see a ‘loading’ image pop up, wait ten seconds, the page is rebuilt with the DOM” and “click a link, wait ten seconds, see a new page load from the server”. None. Please, please, stop doing this. If it’s going to take ten seconds to load a page, just load it from the server. That way, the browser takes care of the work, your pages are more likely to be bookmarkable, and it very rarely dies half way through. Picking on Google Reader for a moment, about 10% of the time I click the “All items” link and get the loading div pop up, that loading div never goes away and I have to refresh the whole page using the browser Refresh button. This user experience is no better than using an “old-fashioned” no-JavaScript web app. It really isn’t. Browser manufacturers have spent ten years thinking about what to do when a page load fails, and perhaps how to do some recovery, and users have spent ten years learning how to cope with a ropy and abortive internet. Please, just stop doing this.

Another DHTML Utopia review

Nate Klaiber reviews DHTML Utopia and is rather nice about it. Thanks, Nate! Did I mention that you should buy my books?

JavaScript Libraries Friend or Foe podcast from @media 2006 published

I was a speaker on the JavaScript panel at the @media 2006 conference this year, and they’ve just published the audio from that JavaScript panel for everyone to listen to. Go thou and listen!

Dungeon

James builds an entire Wolfenstein-esque interactive dungeon out of CSS borders. Nutter. Nice one.

GMail random signatures

There are a few scripts around to do random signature cycling in GMail, but they all are limited from my perspective; they let you specify the signatures in the GMail interface, for example, but save those signatures on your local computer. Since I share stuff between home and work, I’ve got an updated script. Gmail: Random Signature Remote goes and fetches a fortune file from a URL and chooses signatures from that. You’ll need to edit the script once you’ve installed it and change the url entry in changeSignature (this isn’t an easy user process, because I just hacked this script together for me) to point at a fortune file. Fortune files have each signature separated by a % on a blank line.

Build Your Own AJAX Web Applications

A new book from SitePoint: Build Your Own AJAX Web Applications. Matthew Eernisse has done a good job here, and I’m not just saying that because I was tech editor. The more good books we have from decent publishers like SitePoint and Friends of Ed and O’Reilly, the fewer people will hopefully buy “Learn To Write 1999-era IE-Specific Crappy JavaScript In Twelve Hours And Then Inflict It On The Rest Of Us” and then decide that DOM scripting is rubbish.

Scripting Linux desktop applications

Hubert Figuere writes about scripting Linux desktop applications:

All the dirty work would be done within the scripting engine. It is the interface between the language and the IPC mechanism, and it implements the scripting protocol. The scripting interface will arbitrate, allowing to control several applications from one single script.

What he’s describing there, broadly, is what the Windows Scripting Host does, I’d say.

I don’t think it should be JavaScript. Now, I’m a JavaScript guy, and I’ve written a book on it, so I like the language a lot. But it doesn’t have any primitives for doing things like writing a file, which means you have to provide all that. It makes it awkward to write scripts in it on Windows, and it makes it awkward to write scripts in it in Mozilla, because essentially the fact that you’re using JS is irrelevant; what you have to program to is the API you provide. The JavaScript gets lots of extra objects made available to it, and everyone has to learn about those objects, which is way harder than just learning JS itself.

Writing a file using JavaScript and the Windows Scripting Host looks something like this:

var fso = new ActiveXObject("Scripting.FileSystemObject");
var fp = fso.CreateTextFile(savefile);
fp.WriteLine("Blah blah blah");
fp.Close();

Writing a file using JavaScript inside Mozilla (which uses XPCOM) looks something like this:

var file = Components.classes["@mozilla.org/file/local;1"]
		.createInstance(Components.interfaces.nsILocalFile);
	file.initWithPath( savefile );
	if ( file.exists() == false ) {
		alert( "Creating file... " );
		file.create( Components.interfaces.nsIFile.NORMAL_FILE_TYPE, 420 );
	}
	var outputStream = Components.classes["@mozilla.org/network/file-output-stream;1"]
		.createInstance( Components.interfaces.nsIFileOutputStream );
	outputStream.init( file, 0x04 | 0x08 | 0x20, 420, 0 );
	var output = document.getElementById('blog').value;
	var result = outputStream.write( output, output.length );
	outputStream.close();

To be honest with you, I think that the AppleScript approach is a better one than the MS COM/Mozilla XPCOM approach. In COM, each app exports an API, and you talk to that API. In AppleScript you talk to the app as if you’re a user of it. Personally, I’d like to see someone bash on Dogtail (http://people.redhat.com/zcerza/dogtail/) until it’s a generic scripting thing. The advantage you get with using Dogtail, which goes through the X accessibility APIs, is that apps do not have to provide a programming API explicitly. Every application on Mac OS, as I understand it, is automateable through AppleScript. Most applications on Windows are not programmatically controllable. Microsoft apps normally are, but third-party apps aren’t. If you have to provide a separate programming API for your app then (a) that’s extra work for the hackers, and (b) that’s extra documentation required. Even more so than just using an app, an app’s programming interface is useless without documentation, and documentation is one thing that the open source community does not do well. Let’s take Dogtail, or something like it, and make it do the work for us. If Dogtail can do most of the stuff we need, then adding JavaScript bindings for it would be pretty easy by comparison.

S3 is onlinedrive

Amazon’s new S3 online storage service is what I was talking about when I spoke of “online drives” a little while back, a place to store your data. There’s a clone of S3 called Park Place, so the “run your own servers” part is now handled. What’s missing is integration into applications. That’s something that someone should get onto with the greatest of speed; take your favourite web app that creates things and build some integration so it can save data onto your own personal S3 space (whether S3 itself or your own Park Place at an arbitrary URL). Then submit your patch to the upstream people so they incorporate it, or release a GreaseMonkey script to do it in the interim.

Truncated text

A minor little web experiment: truncated text. Resize the window and it wraps the text in a div so that only the first line shows.