Making a better Ubuntu app scope (not very successfully)

I was rather excited to discover the idea that one could change the icons used on an Android phone’s home screen. (As Sam Hewitt put it, I have “become a themer”.) I didn’t know this was doable, and I went from not knowing this to having it done in the space of about half an hour. It is rare that something just works this nicely; I’ve been living my life a long time and I know how it works. So far, the other shoe hasn’t dropped, which is encouraging.

At first, there was Android

Anyway, my Android home screen now looks like this:

I’m pretty pleased with that. (The lock screen is even more minimal and pretty, but we’re not talking about lock screens today: see the G+ post for what that looks like.)

There is a UI concern here — in general, I don’t like using icons without text to indicate a thing, because it means you’re spending cognitive time remembering what an icon stands for rather than what it’s similar to. That is: icons are quite good for, say, files where you want to see that all of these are .txt files, but they’re not good for actions where you have to remember what a “heart” or some obscure little symbol means. I’ve lost count of the number of times when I’ve used a new app which only has icons and not text labels on its buttons and I have no idea what half the icons do; a few things are enshrined by everyone using them (a Share icon, a Save icon, an Open icon, etc) but most of the time using an icon alone to indicate an action or an app is a bad idea. However, this is an exception to that rule. I am very familiar with my home screen; the set of apps on it do not change very much at all. So there’s minimal cognitive load in remembering that this icon is for Twitter and that one is for Moon Reader. I do a reasonable proportion of that by spatial arrangement; the icons don’t move around, and I am familiar and trained in where they are. You wouldn’t want to ship a phone which looks like this to a new user, but it’s a perfectly good experience for me, with my own personal home screen that I know well.

How is this beautiful miracle achieved? Well. The default Android home screen doesn’t let you change the icons used for apps. So when I discovered the (free!) Lines icon set in the Play store my immediate thought was, cor, how does that work? And the answer is: use a different home screen app (in Android parlance, they’re called launchers). So I installed Nova Launcher. (This also had the rather nice benefit of allowing me to remove the hated Google search bar; I never, ever, ever used that because it shows results in some Google app rather than the web, meaning that you can’t long-press on images to save them or on links to open in a new tab. Pack it in, Google. Open in the browser. It’s good at that.) A little bit of setup with Nova Launcher and Lines and, lo, I had the nice minimal icons and pretty dark wallpaper (which came with Lines). Nova Launcher doesn’t let you resize the icons, and by default they’re a bit small, but you can buy the Nova Launcher Prime add-on to enable that, and it was fifty pence so that’s a no-brainer. The Lines icon set is both worryingly comprehensive (they claim to have over two thousand icons!) and equally worryingly incomplete (with a million apps in the Play store they’re never going to carry them all, but not having Telegram? Seriously?) and so I added two more similar-looking icon sets (Min and Glyphsy) to give me a wider choice. A bunch of tweaking of icon sizes, icon choices, margin sizes, and so on later and I have the screen above, and every time I look at it I like it a little more. Ultra-happy with it.

And then there was Ubuntu

I thought to myself, hang on, it ought to be possible to do the same thing with Ubuntu phone, right? One can write a custom scope which looks like that and shows all one’s apps, and add it to the Dash as the only scope, and then you’ve got a cool-looking app-launcher home screen. Sweet. That’s what I want. And it’ll give me an excuse to play with all the new scopes stuff and learn something about it.

Not so fast, grasshopper.

Here’s my basic idea. The existing apps scope is annoyingly slow; I’m sure it does a whole bunch of stuff and that’s why, but it’s slow to load, slow to scroll, and slow to react to things changing. This is obviously being worked on by the Canonical team and I’m sure it’ll get better, but currently it’s pretty frustrating. So I thought: what I’ll do is this. I’ll add a click hook so when apps get installed and removed I can update a sqlite database, and then my scope will just hit that database rather than inspecting the filesystem and loading loads of files. That database can also hold the locations of icons in my custom “wireframe” icon theme, and it’ll use those icons for the apps. So it’ll be fast and have custom icons, and tapping an icon will launch that app via an application:/// url. Job’s a good’un, etc. Off down the pub.

It was not as easy as I had hoped.

There are a few things that made the task difficult and which are basically my fault and for which no blame attaches to the Ubuntu team. Firstly, I’m still running Ubuntu 14.04. This is because it’s an long-term-support release, and LTSes don’t offer to upgrade to the next LTS release (16.04, which came out in April) until the first point release (16.04.1, which isn’t out at time of writing). This is so the new LTS gets a bit of testing before it hits all the people who desire stability, and is a good idea, and I am entirely happy with it. Secondly, I wanted to write the scope in JavaScript; one can write Unity 8 scopes in C++ (which I can’t do), Go (which I can’t do), and JS via node (which I am completely comfortable with). But the JS scopes stuff is new, and so doesn’t exist in 14.04. This meant that I had to do a great deal of faffing around in lxc containers to get a working version of a JS scope. However, that faffery did actually work out in the end, and it’s hard to see the need for it as due to anything more than my own desire to not upgrade away from the latest stable release LTS. So, no blame there, as mentioned. After a bit of poking, I managed to get a JavaScript scope, and some discussions with Brian Douglass and one tiny bash deploy script later (which basically does click build . and then adb pushes the .click file to my Bq E4.5 and pkcon install-locals it), I had a scope that I’d written running on the phone. Hooray! Now to make that scope actually do and look as I want it to.

This is where we run into problems. The JavaScript scopes stuff is, admittedly, new, but it had two major issues that caused me a great deal of grief. The first is that making async fs.readdir calls early in a scope search() handler were making my scope crash, and sometimes crashing all of Unity. (LP#1598967) Oops. That took some time debugging before I eventually went for the synchronous fs.readdirSync, cursing all the while. Worse, though, is the activate problem. You see, a scope, by default, when a thing is tapped shows you the preview page for that thing. However, you can change things so that a tap on a thing does something directly; in this case, of course, I wanted a tap on one of my app icons to launch that app via the URL dispatcher. To do that, you call set_intercept_activation() on a scope icon when you create it, and then Unity will call your scope’s runtime_config.activate() method with the details when that icon is tapped; you can then handle it in whatever way you want, or return an ActivationNotHandled type of ActivationResponse to let Unity handle it for you, which in this case means “open the app with the url you provided”. (You can see a good example of how this works in Go in Brian Douglass’s Falcon alternative app scope.) However! If you do that in JavaScript, your scope ignores the tap and throws an error about how you returned nullptr from activate(). (LP#1598969.) As the bug says, “this, in practice, means it’s impossible to write a JS scope which allows tapping on a result item to do anything other than show the preview (for example, launching an app).”

(The observant will notice that the JS scopes documentation doesn’t actually mention ActivationResponse or even that activate() handlers are possible in scope.initialize(), which will hopefully get better as the docs get written.)

Now, that’s just a bug, and I’m sure it’ll get fixed (I’m sure they’ll all be fixed). It is frustrating (has nobody writing the JS scopes stuff ever tried set_intercept_activation()? Isn’t there lots of QA built into the Ubuntu process now?) But it’s a temporary thing, I’m sure, and in the same class of “frustrating and show-stopping but will be fixed eventually” things as how unity-js-scopes-tool doesn’t build node-pre-gyp packages (LP#1598971) or that unity-js-scopes-tool seems to actually be the node binary itself but that’s carefully kept secret for some reason, or that you have to remove and reinstall an app if you change the apparmor stuff otherwise the changes aren’t picked up (LP#1549369) which cost me an hour of banging my head against the desk (thanks mzanetti for helping). They’ll get fixed. Move on.

The things which actually stopped me building the scope I wanted weren’t these sorts of incidental bugs; they’re design decisions saying that what I want should not be possible. I’m hoping that the detail I’m going into here helps to explain why I want these things, and perhaps makes its way back to the team deciding on this stuff and helps to inform their decisions.

For example. You see how my Android screen above has four columns of app icons? Like the Ubuntu apps scope has? Well… my replacement scope can’t. Scopes are only allowed three columns of icons. The apps scope has been magically hardcoded into Unity to be allowed four columns; nobody else can do that. That’s pretty annoying.

Similarly, customisation of the look of a scope is superficially good but actually rather limited. A scope header can have a custom background image, but a scope itself can’t; you can pick a colour, but you can’t set an image as the background for your scope. (LP#1598973) Scopes put the “Ubuntu shape” around all your scope icons, but you can turn that off by settings ShapeImages=false in your scope ini file. Except… that doesn’t work; it’s ignored, and you always get the Ubuntu shape border on every image, whether you want it or not. (LP#1598933) I understand the idea behind shaping imagery, but it looks heart-stoppingly ugly when done to icons with transparent backgrounds, such as all the “wireframe” icons I planned to use in this project. You can have settings for your scope, but they can’t change the appearance, because that stuff’s not programmatic; it’s in the scope.ini file and can’t be changed by code. So I can’t allow people to have a dark scope with white icons (as above) or choose a light scope with black icons. Also, making a replacement apps scope is essentially fruitless because the default apps scope cannot be removed. You can’t turn it off; it must be present in your Dash. This to me seems strange; I’m fine with you not being able to delete it, but it will always be present in the bottom-edge menu. Fine, maybe making replacement apps scopes is an edge case, but it’s hard to get the feeling that my phone is mine when I’m obliged to keep the default apps scope around even if I’ve got a replacement.

Oh, and the idea of making the scope fast by populating a database? You can’t have that, either. Yes, there are click hooks. But you can’t install one. They must be stored in /usr/share/click/hooks, and that’s part of the system image, which is read-only. What this means is that there’s no way that a third-party app will ever be able to be informed of app installs and removals, sadly. I can put an app which requires extra permissions into the Open Store, but even such an app can’t install a click hook; I’d have to make it a deb, and even then a deb won’t install either unless you mark your image as read-write, which requires plugging it into a computer. Talking someone through that process is close to impossible unless they’re the hardest of the hardcore Ubuntu touch hackers. Perhaps click might look in some other folder as well, which is accessible to apps, and have a new apparmor permission protect that folder? Then it would be possible for an application to request that permission and keep a database of installed and removed apps up to date, which would allow us to compete on making a better app launching experience.

So all this means that the best I can do on Ubuntu is this. Ubuntu shapes around the transparent icons, no background imagery, three columns. And that’s just not pretty, I’m sad to say.

(I freely admit that some of this is down to the icons, which I drew quickly, and some more is down to the E4.5 having much worse screen resolution than my OnePlus X. But those aren’t the only things.)

Anyway. I still have my beautiful Android home screen. But I’ve reluctantly decided that I just can’t do that on Ubuntu. Maybe it’ll be possible later; maybe these bugs will be fixed and the design decisions revisited. I hope the above explains why I think that’s important and what app developers could maybe do if those restrictions are slightly relaxed. Fingers crossed.

More in the discussion (powered by webmentions)

  • Birmingham.IO responded at Stuart Langridge: Making a better Ubuntu app scope (not very successfully) by @sil #PlanetBirmingham
  • Jackie Moon responded at +++ OFFTOPIC +++I never ever could have imagined that I would write this,but you were missed on the BadVoltage show heavily.Don't be like +Bryan Lundu…
  • DustinRachael MacDonald responded at ... (
  • Sabrin Islam responded at ... (
  • Nekhelesh Ramananthan responded at ... (
  • Fabio Colella responded at ... (
  • Richard Somlói responded at ... (
  • Michał Prędotka responded at ... (
  • James Henstridge responded at I think part of the problem is that you're replacing the wrong component of the system. The Ubuntu equivalent of "replacing the Android launcher" isn…
  • Stuart Langridge responded at +James Henstridge I see your point. I think about it slightly differently, though. Agreed that installing a new android launcher is like installing a…
  • James Henstridge responded at While you might not have set out with the goal of replacing the Android launcher, my point was that getting what you wanted on that system involved ma…
  • Stuart Langridge responded at Totally agree -- I like the separation where I provide a data model and the Dash displays it. The issues that stop me doing the above screenshot on Ub…
  • James Henstridge responded at I'm not sure I agree that all those things should be controllable by a scope. We've definitely had user requests to be able to set the wallpaper on t…
  • Stuart Langridge responded at Ah. I pretty much don't want a way to customise the dash as a whole because it'd be almost impossible to write scopes that can cope with it; if someon…
  • James Henstridge responded at That's definitely something that'd need to be solved if we wanted customisable backgrounds. But it certainly is something that can be solved. Pretty…
  • Stuart Langridge responded at Yeah. You basically make the "default foreground colour" be "XOR" rather than "black", or by default apply a text shadow, or whatever. What's frustrat…