Sailfish OS
Sailfish OS is the operating system that powers the Jolla smartphone. It's an evolution of the Maemo/MeeGo Linux distribution that started life on the Nokia tablets back in 2005.
During its rather tumultuous life it's evolved from a trailblazing tablet OS to possibly one of the most open and interesting smartphone OSes. Applications are built using the venerable QT framework with gestures and dialogue stacks at the core of its approach to interaction. At the same time it's a true Linux distribution allowing existing applications to be rebuilt and redeployed easily (you can even fire up GCC on the phone to build them in situ or hack away at Python if that's your bag).
Personally I've been using Maemo/SailfishOS since buying an N810 around 2008, followed by an N900 and then a Jolla phone. On this page you can find out about some of the software I've developed for the OS, as well checking out my 3D printed The Other Half covers produced in agreement with Jolla.
Sailfishos
This is an especially big hit for me, because when it comes to Sailfish OS — the smartphone operating system I use in preference to iOS or Android for reasons of privacy and control — there is no official native BBC Sounds app. I can run it using App Support, but what I really want is to listen to the radio through one of my preferred, native, open source apps (such as Received). I've been happily doing so for a decade, and while I accept the BBC is entitled to make its content available (or not) in whatever way it wants, it's frustrating to have a perfectly good option taken away.
So where's the upside? It's counterintuitive, but not listening to the radio turns out to be a really good thing. Since dropping the service my mornings have become an oasis of calm. They're no longer filled with politicians, pundits and presenters verbally sparing with one another. I don't have to suffer the banality of the sports coverage (made worse by the lack of any actual live sports in the mornings). The treacly vacuity of Thought for the Day is blissfully absent from my life.
It was during the pandemic that my obsession with the radio really embedded. Like everyone else I was desperate to know whether we were going to survive the next 24 hours. Stuck in a small apartment on my own it became an important proxy for real human interaction.
Happily that time is over, both the pandemic and the obsessive listening. I now sit quietly, listening to the birds sing, the leaves rustling in the breeze, the occasional propeller plane humming overhead. It's soothing. My anxiety is reduced. And I'm now certain this BBC-enforced calm is a positive change after all.
Having said all this, I've not stopped listening to BBC Radio completely: hapilly all BBC radio content is still available using my own GetiPlay app. This doesn't support live streaming though, so it means I have to actively choose to listen, rather than blithely surrounding myself with whatever happens to be on.
s/Covid Warn App/Corona Warn App/g
We've already relinquished so many rights in the fight against COVID-19, it's important that privacy isn't another one, not least because the benefit of contact tracing increases with the number of people who use it, and if it violates privacy it'll rightly put people off.
So I'm generally positive about the specification. It seems to be a fair attempt to provide privacy and functionality. Not only that, it's providing a benchmark for privacy that it would be easy for governments to fall short of if the spec weren't already available. Essentially, any government who now provides less privacy than this, is either incompetent, or has alterior motives.
But what does the spec actually say? Apple and Google have provided a decent high-level summary in the form of a slide deck, from which the image below is taken. They've also published a (non-final) technical specification. However, for me the summary is too high-level (it explains what the system does, but not how it works) and the technical specs are too low-level (there's too much detail to get a quick understanding). So this is my attempt at a middle-ground.
There are three parts to the system. There's the OS part, which is what the specification covers; there's an app provided by your regional health authority; and there's a server run by your regional health authority (or more likely, a company the health authority subcontracted to). They all act together to provide the contact tracing service.
- Each day the user's device generates a random secret $k$, which stays on the user's device for the time being.
- The device then broadcasts BLE beacons containing $h = H(k, c)$ where $H$ is a one-way hash function and $c$ is a counter. Since $k$ can't be derived from $h$, and since no pair of beacons $h_1, h_2$ can be associated with one another, the beacons can't in theory be used for tracking. This assumes that the BLE subsystem provides a level of tracking-protection, for example through MAC randomisation. Such protections don't always work, but at least in theory the contact-tracing feature doesn't make it any worse.
- The device also listens for any beacons sent out by other users and stores any it captures locally in a list $b_1, b_2, \ldots$.
- If a user tests positive for COVID-19 they are asked to notify the regional health authority through the app. This involves the app uploading their secret $k$ for the day to a central database run by the regional health authority (or their subcontractor). From what I can tell, neither Apple nor Google need to be involved in the running of this part of the system, or to have direct access to the database. Note that only $k$ is uploaded. Neither the individual beacons $h_1, h_2, \ldots$ sent, nor the beacons $b_1, b_2, \ldots$ received, need to be uploaded. This keeps data quantities down.
- Each day the user's phone also downloads a list $k_1, k_2, \ldots, k_m$ of secrets associated with people who tested positive. This is the list collated each day in the central database. These keys were randomly generated on the user's phone and so are pseudonymous.
- The user's phone then goes through the list and checks whether one of the $k_i$ is associated with someone they interacted with. It does this by re-calculating the beacons that were derived from this secret: $H(k_i, 1), H(k_i, 2), \ldots, H(k_i, m)$, and compares each against every beacon it collected the same day.
- If there's a match $H(k_i, j) = b_l$, then the user is alerted that they likely interacted with someone who has subsequently tested positive. Because the phone also now knows the counter $j$ used to generate the match, it can also provided a time for when the interaction occurred.
This is a significant simplification of the protocol, but hopefully gives an idea of how it works. This is also my interpretation based on reading the specs, so liable to error. By all means criticise my summary, but please don't use this summary to criticise the original specification. If you want to do that, you should read the full specs.
Because of the way the specification is split between the OS and the app, the BLE beacons can be transmitted and received without the user having to install any app. It's only when the user tests positive and wants to notify their regional health authority, or when a user wants to be notified that they may have interacted with someone who tested positive, that they need to install the app. This is a nice feature as it means there's still a benefit even if users don't immediately install the app.
One of the big areas for privacy concern will be the behaviour of the apps provided by the regional health authorities. These have the ability to undermine the anonymity of the system, for example by uploading personal details alongside $k$, or by tracking the IP addresses as the upload takes place. I think these are valid concerns, especially given that governments are notorious data-hoarders, and that the system itself is unlikely to be built or run by a health authority. It would be a tragic missed opportunity if apps do undermine the privacy of the system in this way, but unfortunately it may also be difficult to know unless the sourcecode of the apps themselves is made available.
Apple and Google today announced a joint protocol to support contact tracing using BLE. You can read their respective posts about it on the Apple Newsroom and Google blog.
The posts offer some context, but the real meat can be found in a series of specification documents. The specs provide enough information about how the system will work to allow a decent understanding, albeit with some caveats.
With so much potential for misuse, and given that mistrust could lead to some people choosing not to use the system, it's great that Google and Apple are apparently taking privacy and interoperability so seriously. But I'm a natural sceptic, so whenever a company claims to be taking privacy seriously, I like to apply a few tests.
- Are the specs and implementation details (ideally sourcecode) freely and openly available?
- Is interoperability with other software and devices supported.
- Based on the information available, is there a more privacy-preserving approach that the company could have gone with, but chose not to?
The catch is that the API defined by the specs provides only half of a full implementation. Apple and Google are providing an API for generating and capturing BLE beacons. They don't say what should happen to those beacons once they've been captured. Presumably this is because they expect this part of the system to be implemented by a third-party, most likely a regional public health authority (or, even more likely, a company that a health authority has subcontracted to).
Again, this makes sense, since different regions may want to implement their own client and server software to do this. In fact, by delegating this part of the system, Google and Apple strengthen their claim that they're acting in good faith. They're essentially encouraging public health authorities and their subcontractors to live up to the same privacy standards.
Apart from the privacy issues, my other main interest is in having the same system work on operating systems other than iOS and Android. My specific interest is for Sailfish OS, but there are other smartphone operating systems that people use, and locking users of alternative operating systems out of something like this would be a terrible result both for the operating system and for all users.
Delegation of the server and app portions to health authorities unfortunately makes it highly unlikely that alternative operating systems will be able to hook into the system. For this to happen, the health authority servers would also need to provide a public API. Google and Apple leave this part completely open, and the likelihood that health authorities will provide an API is unfortunately very slim.
I'd urge any organisation planning to develop the client software and servers for a fully working system to prove me wrong. Otherwise alternative operating system users like me could be left unable to access the benefits of the system. This reduces its utility for those users to nill, but it also reduces the effectiveness of the system for all users, independent of which operating system they use, because it increases the false negative rate.
There's one other aspect of the specification that intrigues me. In the overview slide deck it states that "Alice’s phone periodically downloads the broadcast beacon keys of everyone who has tested positive for COVID-19 in her region." (my emphasis). This implies some form of region-locking that's not covered by the spec. Presumably this is because the servers will be run by regional health authorities and so the user will install an app that applies to their particular region. There are many reasons why this is a good idea, not least because otherwise the amount of data a user would have to download to their device each day would be prohibitive. But there is a downside too. It essentially means that users travelling across regions won't be protected. If they interact with someone from a different region who tests positive, this interaction won't be flagged up by the system.
The spec is still very new and no doubt more details will emerge over the coming days and weeks. I'll be interested to see how it pans out, and also interested to see whether this can be implemented on devices like my Sailfish OS phone.
Version 0.8-1 of GetiPlay is now available, refreshed with the latest version 3.22 of the amazing get_iplayer and updated perl modules. After some recent glitches, this will now work again with the iPlayer catch-up service for downloading BBC TV and radio programmes to your Sailfish OS device (probably UK-only I'm afraid). Install the binary to your phone from OpenRepos, or get the source from GitHub.
Sun Mar 24 2019 David Llewellyn-Jones <david@flypig.co.uk> 0.7-1
- Correct iterator errors when deleting media files and items from queue.
- Correctly trim logfile and prevent UI performance degradation over time.
- Correct an incorrect RPM configuration.
- Remove cyclic dependences in QML.
- Fix various other QML errors.
- Add scroll animation when clicking on tab to jump to the top of the page.
- Allow control using the lockscreen media (MPRIS) controls.
- Improve the button layout on the queue item info screen.
In my spare time I've been developing a QT app called GetiPlay. It's a simple app that allows you to download audio and video from BBC iPlayer, for use on Sailfish OS phones. The traditional approach on Linux devices would be to use get_iplayer
in a console, but for all of the progress that's been made on mobile devices in the last decade, console use still sucks. Given I spend so much time listening to or watching BBC content, slapping a simple UI over the command line get_iplayer was an obvious thing to do.
The app has been developing nicely, using the QT Creator for C++ and the UI written in QML. Historically I've not been a fan of QML, but as I grow more familiar with it, it's been growing on me. For all of the things that I find weird about it, it really does give great performance and helps build a consistent UI, as well as promoting loose coupling between the UI and underlying functional logic.
A big downside to QML is that there's no preview, so the development process follows a consistent cycle: adjust code, build code, deploy code, test, repeat. The build and deploy steps are loooong. This impacts things in three serious ways: it makes development slow, it makes me sleepy, and it incentivises against making minor tweaks or experimentation.
Nevertheless, there's always a trade-off between configuring and learning new technologies, and just getting things done using those you're already using. The ever-relevant XKCD has more than one pertinent comics covering this topic.
The UI for GetiPlay is straightforward, so I was quite content to use this lengthy, but (crucially) working approach until yesterday. What prompted me to change was a feature request that needed some more subtle UI work, with animated transitions between elements that I knew would take a couple of hundred cycles round that development loop to get right. Doing the maths using Randall Munroe's automation matrix, I needed to find a more efficient approach.
So this morning I started out using QML Live. This is a pretty simple tool with an unnecessarily bulky UI that nevertheless does a great job of making the QML design approach more efficient. You build and run the app as usual, then any QML changes are directly copied over to the device (or emulator) and appear in the app immediately. Previously a build cycle took between 40 and 100 seconds. Now it's too quick to notice: less than a second.
QML-Live certainly isn't perfect, but it's simple, neat and has made me far more likely to try out interesting and experimental UI designs. Time configuring it is time well spent, even if that extra 18 hours is just about the same amount of time I wasted dithering over the last two days!
So I finally got to adding it in. Here's the video player screen.
The QT framework really does make it easy to add media like this. It still took a good few days to code up of course, but it'd be a lot quicker for someone who knew what they were doing.
I'm also quite proud of the audio player, with the same, super-useful '10 seconds back' button. It also stays playing no matter where you move to in the app. Here it is, showing the controls at the bottom of the screen.
If you'd like to get these new features in your copy of GetiPlay, just download the latest version from OpenRepos, grab yourself the source from GitHub, or check out the GetiPlay page.
Get it from GitHub, OpenRepos, or this site.
Immediate info about every one of the thousands and thousands of TV and radio programmes is also now just a tap away.
Install yourself a copy from OpenRepos, grab the MIT-licensed source from GitHub or visit the GetiPlay page on this site.
Android also has the claim of being Linux, which in theory means it has access to the existing - incredibly mature - Linux software ecosystem. In practice for most people this is moot, since their focus is on the very different type of software available from the Play Store. For developers though, this can be important. For me the distinction is important partly because I'm already familiar with Linux, and partly as a matter of principal. In my world computing is very much about control. I love the idea of having a computer in my pocket not because it gives me access to software, or as a means of communication, but because it's a blank slate just waiting to perform the precise tasks I ask of it. That sounds authoritarian, but better to apply it to a computer than a person. I'm pretty strict about it too. Ever since being exposed to the wonder of OPL on a Psion 3a (way back in 1998), direct programmability has always been one of the main critiera when choosing a phone.
This weekend was the Easter Bank Holiday, meaning a lengthy train ride across the country to visit my family. I wanted to download some radio programmes and possibly some videos to watch en-route, but didn't get time before we set off. I'd managed to install the Android version of BBC iPlayer on my Jolla, but for some reason this doesn't cover BBC Radio, which has been split off into a separate application. Hence I embarked on a second journey while sitting on the train: installing get_iplayer entirely using my phone. This meant no use of a laptop with the Sailfish IDE, and building things completely from source as required.
The experience was enlightening: during the course of the weekend I was able to install everything from source straight on my phone. This included the rtmp streaming library and ffmpeg audio/video converter all obtained direct from their git repositories, all just using my phone.
Why would anyone want to do this when you can download the BBC radio app from the store? You wouldn't, but I still think it's very cool that you can.
Here's how it happened.
get_iplayer is kind-of underground software. It shouldn't really exist, and the BBC barely tolerates it.
It's written in Perl and is currently available from http://www.infradead.org/get_iplayer. Getting it is just a matter of running the following command in the shell:
git clone git://git.infradead.org/get_iplayer.git
Perl is already installed on Sailfish OS by default (or at least was on my phone and is in the repositories otherwise). There were some other Perl libraries that needed installing, but which were also in the repositories. I was able to add them like this:
pkcon install perl-libwww-perl pkcon install perl-URI
Because it's Perl, there's no need to build anything, and at this point get_iplayer will happily query the BBC listing index and search for programmes. However, trying to download a programme generates an error about rtmpdump being missing.
The rtmpdump library isn't in the Sailfish repositories, but can be built from source really easily. You can get it from http://rtmpdump.mplayerhq.hu, and I was able to clone the source from the git repository:
git clone git://git.ffmpeg.org/rtmpdump
Building from source requires the open-ssl development libraries, which are in the repositories:
pkcon install openssl-devel
After this it can be built (although note developer mode is needed to complete the install):
cd rtmpdump make devel-su make install cd ..
As part of this build the librtmp library will be created, which needs to be added to the library path.
echo /usr/local/lib > /etc/ld.so.conf.d/librtmp.conf ldconfig
This should be enough to allow programmes to be downloaded in flv format. However, Sailfish won't be comfortable playing these unless you happen to have installed something to play them with. get_iplayer will convert them automatically as long as you have ffmpeg installed, so getting this up and running was the next step. Once again, the ffmpeg source can be cloned directly from its git repository:
git clone git://source.ffmpeg.org/ffmpeg.git
The ffmpeg developers have done an astonishing job of managing ffmpeg's dependencies. It allows many extras to be baked into it, but even without any of the other dependencies it'll use the autoconfig tools to allow a minimal build to be created:
pkcon install autotools cd ffmpeg ./configure make make install cd ..
ffmpeg is no small application, and compiling it on my phone took over an hour and a half. I know this because we watched an entire episode of Inspector Montalbano in the meantime, which get_iplayer helpfully tells me is 6000 seconds long!
Nonetheless, once completed the puzzle is complete, and get_iplayer will download and convert audio and video to formats that can be listened to or viewed on the Sailfish media player.
For me there's something beautiful about the ability to build, install and run these applications directly on the phone. get_iplayer is command-line, so lacks the polished GUIs of the official applications, but it's still very efficient and usable. I get that this makes me sound like Maddox, but that only makes me more right.
Three, my mobile carrier, insists I'm using tethering and cuts my connection whenever I try to download files using get_iplayer. It's annoying to say the least, but highlights the narrow gap between GNU/Linux on a laptop and GNU/Linux on a Sailfish OS phone.
No, this is to do with my Jolla phone. Back in the day, before smartphones were ubiquitous, many phone manufacturers tried to lure in the punters by offering interchangeable fascias or backplates. Not very subtle, or high-tech, but presumably effective.
Well, Jolla have decided to return to this, while taking the opportunity to update it for the 21st Century. Each Jolla smartphone appears to be built in two halves, split parallel to the screen and with the back half ("The Other Half") replaceable to provide not just different styles, but also additional functionality. The extra functionality is provided by cleverly using NFC-detection of different covers, along with the ability for covers to draw power from and communicate with the main phone via a selection of pins on the back.
At the moment there are only four official Other Halves that I'm aware of: Snow White (the one that comes as standard), Keira Black, Aloe and Poppy Red (the preorder-only cover). They use the NFC capability to change the styling of the phone theme as the cover is changed, but in the future there's a hope that new covers might provide things light wireless charging, solar charging, pull-out keyboard, etc.
For me, the interesting thing about the phone has always been the Sailfish OS that powers it. As anyone who's ever set eyes on me will attest, I've never been particularly fashion conscious, so the prospect of switching my phone cover to match my outfit has never offered much appeal. However, since the good sailors at Jolla have released a development kit for The Other Half, and since it seemed like an ideal challenge to test out the true potential of future manufacturing - by which I mean 3D printing - this was not an opportunity I could not miss.
Rather brilliantly, the development kit includes a 3D model which loads directly into Blender.
From there it's possible to export it in a suitable format for upload directly to the Shapeways site. The model is quite intricate, since it has various hooks and tabs to ensure it'll fit cleanly on to the back of the phone. Sadly this means that most of the usual materials offered by Shapeways are unavailable without making more edits to the model (sadly, it will take a bit more work before it can be printed in sterling silver or ceramic!). My attempt to print in polished Strong & Flexible failed, and eventually I had to go with Frosted Ultra Detail. Not a problem from a design perspective, but a bit more expensive.
The result was immaculate. All of the detail retained, a perfect fit on the phone and a curious transparent effect that allows the battery, sim and SD card to be seen through the plastic.
Although a perfect print, it wasn't a good look. Being able to see the innards of the phone is interesting in an industrial kind of way, but the contouring on the inside results in a fussy appearance.
The good news is that all of the undulations causing this really are on the inside. The outer face is slightly curved but otherwise smooth. The printing process results in a very slight wood-grain effect, which I wasn't anticipating, but in hindsight makes sense. The solution to all of this was therefore to sand the outside down and then add some colour.
The colour I chose was a pastel blue, or to give its full title according to the aerosol it came in, Tranquil Blue. Irrespective of the paint company's choice of name, the result was very pleasing, as you can see from the photos below. The 3D-printed surface isn't quite as nicely textured as the original Other Half cover that came with the phone, but I believe most people would be hard-pressed to identify it as a 3D-printed cover. It looks as good as you might expect from mass-produced commercial plasticware.
With the design coming straight from the developer kit, I can't claim to have made any real input to the process. And that's an amazing thing. Anyone can now generate their own 3D printed Other Half direct from Shapeways with just a few clicks (and some liberal unburdening of cash, of course!). A brand-new or updated design can be uploaded and tested out just as easily.
It's genuinely exciting to see how 3D printing can produce both practical and unique results. The next step will be to add in the NFC chip (it turns out they're very cheap and easy to source), so that the phone can identify when the cover is attached.
There's no shortage of places to get Apps from for a Jolla phone: the Jolla Store, the Yandex Store and the OpenRepos Warehouse being just a few. But even with this smörgåsbord of stores there are still obvious gaps. For example, I wanted to connect my phone through my home VPN, so that I can access things like SMB shares and ssh into my machines.
The iPhone has an OpenVPN client, but the frustrating file management on the iPhone meant I never got it up and running. Unsurprisingly Android has good OpenVPN support which combines well with the broad range of other good network tools for the platform.
In contrast the various SailfishOS stores are sadly bereft of OpenVPN solutions. However, a quick search using pkcon showed the command line openvpn client available in the Jolla repositories. I was astonished when, after a few commands to transfer the relevant client certificates and install the tool, it was able to connect to my VPN first time.
This is what I'm loving about SailfishOS. It speaks the same language as my other machines and runs the same software. Getting it to talk to my VPN server was really easy, even though you won't find this advertised in the headline features list.
Still, having a command line tool isn't the same as having a nicely integrated GUI App, so this seemed like a great opportunity to try out Jolla's Qt development tools. I've not done any Qt development in the past so started by working through the examples on the Sailfish site.
Qt seems to be a nice toolkit and it's set up well for the phone, but Qt Quick and QML in particular require a shift in approach compared to what I'm used to. Qt Quick obfuscates the boundary between the QML and C++ code. It's effective, but I find it a bit confusing.
Still, after a weekend of learning and coding, I've been able to knock together a simple but effective front-end for controlling OpenVPN connections from my phone.
As well as providing a simple fullscreen interface, you can also control the connection directly from the home screen using the clever SailfishOS multi-tasking cover gestures: pull the application thumbnail left or right to connect to or disconnect from the server.
What I think this demonstrates is how quick and easy it is to get a useful application up and running. The strength is the combination of the existing powerful Linux command line tools, and the ability to develop well-integrated SailfishOS user interfaces using Qt. I'm really pleased with the result given the relatively small amount of effort required.
If I get time, there's plenty more to be done. Currently the configuration runs directly from the openvpn script, but allowing this to be configured from the front-end would be an obvious and simple improvement. After this, mounting SMB shares will be next.
Download
- FOSDEM23
Version 0.7.1 (1 Feb 2023) for Sailfish OS.
Schedule your time at FOSDEM wisely. The Sailfish OS FOSDEM23 app allows you to browse the tracks, peruse the talks in each track, read their descriptions and then favourite the ones you want to attend. It'll then show you which talk is next on your schedule. The ideal conference companion.
Download: binary, source, screenshot. - GetiPlay
Version 1.1-1 (12 Sep 2022) for SailfishOS.
GetiPlay is a simple user interface for the get_iplayer command line utility for Sailfish OS devices. It allows TV and radio programmes to be downloaded from the BBC iPlayer listings.
Download: binary, source, screenshot. - PwdHash
Version 1.3.1 (19 Sep 2021) for SailfishOS.
PwdHash allows you to manage a different password for each website you use, without the need to use a password manager. It hashes the website name with your master password, generating a strong password that's unique for each site. It both the original Stanford algorithm and the stronger Cambridge algorithm.
Download: binary, source, screenshot. - Scintillon
Version 0.4.1 (4 Jul 2021) for SailfishOS.
Scintillon is a Philips Hue compatible smart home app that lets you control your Hue lighting using your Sailfish OS phone. Install the binary from the Jolla Store or OpenRepos. More info...
Download: binary, source, screenshot. - Contrac
Version 0.7.7 (16 Feb 2021) for SailfishOS.
A (nearly) full reimplementation of Germany's Corona-Warn-App for Sailfish OS. It supports BLE beacon send/receive, diagnosis key upload/downlaod and exposure checking. More info...
Download: binary, source, screenshot. - Pedalo
Version 0.21 (18 Jul 2018) for SailfishOS.
Measures your relative pedalo performance with others in a privacy-preserving way. Although originally designed for waterways, it could be used just as well to measure your performance in other cycling situations, such as when riding a bike. Install the binary from the Jolla Store. More info...
Download: source, screenshot. - ownNotes
Version 1.86 (22 Sep 2016) for SailfishOS.
ownNotes is a note-taking application for Sailfish OS. It supports some basic markdown feature and notes can be syncronized with ownCloud or any other WebDav server. More info...
Download: binary, source, screenshot. - OpenVPN-Rig
Version 0.00 (30 Mar 2014) for SailfishOS.
OpenVPN-Rig is an OpenVPN client configuration interface. It provides a simple interface for configuring your Sailfish OS phone and connecting as a client securely to an OpenVPN server. More info...
Download: binary, source, screenshot.
Comments
Uncover Disqus comments