Writing a Jokosher extension: a rambling essay

Jokosher is an extensible bit of software. We knew that we’d never be able to think of all the stuff that people would want to do with an audio editor, so we’ve set up Jokosher so that people can write their own extensions to it. So if you want to do something particular, something very specific, something we don’t like, or something we just haven’t got around to yet, you can roll up your sleeves and do it yourself. This is the story of how I built the Freesound extension, which lets you easily browse the comprehensive library of sampleable Creative Commons licenced stuff at the Freesound system and then easily drop the samples you like into your Jokosher project. Jokosher extensions are, basically, tiny separate programs that can talk to Jokosher itself in certain specific ways. So, the Freesound extension isn’t really an extension so much as it’s a graphical browser for Freesound, which happens to know how to talk to Jokosher a bit. For the moment, let’s pretend it’s a separate program without any Jokosher bits at all. It works like this:

  1. The program starts up
  2. It checks to see if it knows what your Freesound username and password are
  3. If it doesn’t, it pops up a dialog for you to enter them, and then saves them away for next time when you do type them in
  4. It shows a window containing a search box and a Go button
  5. When you type something to search for in the search box, it runs a query at Freesound using your username and password, and the thing you searched for, and gets back some XML describing matches
  6. It puts a list of all the matches in the window, each with a description (that it gets in the XML), and a little waveform image (that it gets in the XML), and a play button (that it creates itself, but that plays the file directly from a URL that it gets in the XML)

…and that’s it. The integration with Jokosher only comes in two parts; firstly, starting it up, and secondly, allowing you to add samples that it’s displaying to a Jokosher project. The first part’s pretty easy. Jokosher allows an extension to add itself to Jokosher’s Extensions menu. The way you do this is pretty simple. To write a Jokosher extension, you need to first create a Python file. In order for that Python file to *work* as a Jokosher extension, it needs to define certain things. Those things are: Three variables, EXTENSION_NAME, EXTENSION_DESCRIPTION, and EXTENSION_VERSION Two functions, startup() and shutdown() So, to make a Python file into an extension, you add something like this to the bottom of it:

def startup(API):

    pass



def shutdown():

    pass



EXTENSION_NAME = "Freesound search"

EXTENSION_DESCRIPTION = "Searches the Freesound library of " +

  "freely licenceable and useable sound clips"

EXTENSION_VERSION = "0.01"

The variables are needed so that Jokosher knows what your extension is called, and can display a useful description to the person using it. When Jokosher starts up, and loads your extension, it calls your extension’s startup() function, and it passes it one object: above, we’ve set the startup() function to call that passed parameter “API”. The API object is how extensions can talk to Jokosher. Now, it’s important to note that you don’t make your extension do all its work in the startup() function. That gets called as Jokosher itself starts up. Instead, we want the Freesound extension to add a menu item to Jokosher’s Extensions menu, and then when the user clicks that menu item we start doing the work. The API object has a function called, amazingly, add_menu_item() to do exactly that. So, our startup function should actually be:

def startup(API):  
    menu_item = API.add_menu_item("Search Freesound", callback_function)

What this will do is add a menu item “Search Freesound”, and when the user clicks on it it’ll call callback_function(), which is defined in our extension somewhere. Another minor thing is that anything you do in the startup() function has to be un-done in the shutdown() function. The reason for this is that shutdown() gets run if a user disables your extension. In our case, we need to remove that menu item, which is done using the menu item’s destroy() method. So, change startup() so that menu_item is a global variable (so it’s available to shutdown() as well), and change shutdown() so it removes it:

def startup(API):

    global menu_item

    menu_item = API.add_menu_item("Search Freesound", mainwindow.menu_cb)



def shutdown():

    menu_item.destroy()

That takes care of hooking the extension into Jokosher; the user can now start up and use anything that the extension can do. (Since this little discussion is about how to write Jokosher extensions, we won’t discuss how exactly the extension talks to Freesound. Just take it as read.) The part that’s left is: how do you get samples from the extension into your project itself? The way the user does it is to drag a sample from the extension window onto an instrument in Jokosher. Now, Jokosher understands dragged-and-dropped files; you can drag a music clip from Nautilus or from Firefox or from any other application straight onto a Jokosher instrument and it’ll work. So the Freesound extension just has to know how to allow users to drag things from it. Technically, this is called being a drag source. How to do this in Python is described in the PyGtk Tutorial — basically, you use code something like this:

e.drag_source_set(gtk.gdk.BUTTON1_MASK, [('text/plain',0,88)],gtk.gdk.ACTION_COPY)

e.connect("drag_data_get", self.returndata, sample)


...

def returndata(self, widget, drag_context, selection_data, info, time, sample):

    selection_data.set (selection_data.target,8, sample.previewURL)

where “e” in the first snippet is the widget displaying the sample, and “sample” is our actual Sample object which we’ve created from the FreeSound XML. In essence, the Freesound extension works like Firefox; when you drag a sample to a Jokosher instrument, Jokosher says “what’s this thing you’ve dragged to me?” and the extension says “it is a URL at Freesound”; Jokosher then thinks “aha, I know how to load URLs”, and loads the sample from that URL. This is a good example of how working with the Jokosher team can be important for extensions. Jokosher didn’t, when I started writing it, know how to load a sample from a URL (it only knew how to load one from your hard drive). To make it work, that needed to be added to Jokosher. (In this case, I added it myself, because I’m a Jokosher developer, but if I wasn’t then the team would have happily added it for me.) The Extension API, the way that extensions talk to Jokosher, deliberately isn’t complete; we’ve put the infrastructure in place but we haven’t tried to think up everything that everyone would want to do. Instead, if you’re writing an extension and you realise that you need Jokosher to be able to do something that it currently can’t for your extension to work, talk to us: we’ll add that extra part to Jokosher so that your extension works. This has been a relatively brief summary of writing a Jokosher extension, but hopefully it’ll give you some ideas. Now you can get extending!

More in the discussion (powered by webmentions)

  • (no mentions, yet.)