Flatpak — the evolution of packaging software and apps

The problem with shipping software

What application authors need, and have been desperately asking for, is a way to untangle the release cycle of applications from that of distributions. When GNOME introduced predictable, time-based releases every six months, it revolutionized the industry, but these long release cycles don’t work well for every application. Consider a developer who misses the window to get into a new distribution by a few days, or a developer who needs to get fixes out the same day they change their server software — in both instances, a long release cycle just isn’t ideal.

Separating application releases from distributions is a difficult problem to solve because distributions provide such a great amount of value. They often have the infrastructure to deliver software to millions of users, and offer things like build systems, software updates, bug reports, and many other integration points. Unfortunately, each distribution also has its own configurations with their own runtime environment. Developers often aren't willing to support all the various configurations necessary to run their software on multiple distributions, so distributions have to patch software to work on their system and are constantly battling the addition of new software, security backports, and component integration. There is plenty of opportunity for something unexpected to break along the way.

We can drastically simplify the process of creating and testing software. Instead of having to test each distribution individually – a costly and unsustainable practice – application authors and distributions can meet in the middle. Together, we can choose the stability guarantee that allows us both to flourish with great new software.

Flatpak? [1] — a solution

GNOME developers have talked about the software distribution process for many years: how can we simplify the process of getting high-quality, stable software safely onto users' computers? Alexander Larsson has thought about this problem a lot. Drawing from his previous experience with Glick and his work on Docker file systems, this time he and many other contributors have brought us Flatpak.

Flatpak provides all the tooling necessary to create predictable runtimes and SDKs. To help developers build software for those runtimes, it provides tooling to make it simple. To build a safer and more trustworthy experience for users, it comes with sandboxing and a new concept called Portals, which help users safely give applications minimal access to personal information. Flatpak also provides an incredibly fast update mechanism based on OSTree, which could be described as "git for binaries". And it wouldn't be a GNOME project without a simple, beautiful interface to manage it, built into GNOME Software.

Best of all, Flatpak works across most modern Linux distributions.

Making life easier for developers

When developing an application, the developer chooses a target runtime. For example, GNOME can provide a "GNOME 3.20 runtime" which includes all of the core components of such version of the system. Application authors can target this runtime and their application will run everywhere it’s installed.

When building their software, developers can use the flatpak-builder tool to build their application and any necessary dependencies not part of the runtime, whether they’re a favorite EXIF library or a node.js module. The Builder IDE is also gaining a system allowing a developer to start from nothing to a Flatpak App in a matter of minutes, further simplifying the experience.

When it comes time to ship, all that is needed is a web server to host the application. The web server doesn't require any special features and the application can be hosted as a single file or as a content addressed repository, similar to git.

Creating a larger app ecosystem for GNOME users

We've been updating GNOME Software to support Flatpak natively so it will be seamless for users to update both the distribution and applications no matter where the software comes from.

Because Flatpak uses OSTree, application updates are minimal deltas between versions. This means less bandwidth, faster updates, and less frustration about the excessive bandwidth usage often seen with mobile applications.

Another side-effect of Flatpak is improving the longevity of software. Many of us remember using some niche game or application and wish we could relive that experience. With how much our systems change over the years, reliving the experience is increasingly difficult. Flatpak helps running our software much longer into the future by providing a consistent environment even as our underlying operating system changes.

Ensuring safety through sandboxing

Historically, we've put our trust in distributions to audit the source and build applications for us, — authors that aren't from our known distributions aren’t as trusted. To address security concerns and allow more authors onto the playing field, Flatpak includes a sandboxing functionality that is implemented using cutting edge Linux kernel features, some of which have been pushed forward by the Flatpak development.

One area of the Flatpak sandbox that we expect to play a big role going forward, is the concept of Portals, a way for a sandboxed application to interact with the host operating system. For example, it's unlikely that a user will want to grant a sandboxed application unrestricted access to her video cameras. However, she might want to grant it access to take a picture once for an avatar or for a video conference. Using a camera-mediator portal, the application can request access to a video stream and the user can be asked for authorization before proceeding, with the application never gaining full access to the camera device outside of the API.

Flatpak is the future!

GNOME is creating a technology where developers, users, and distributions all stand to win. In the process we make everyone’s lives a little bit easier, the software better, and the experience safer. That leaves me very excited about the future of the Free Software desktop.

Footnote:

• [1] Flatpak used to be called xdg-app during the development stage. The new name was announced as part of the formal launch of Flatpak as ready-for-use on May 18, 2016. You can find out more at flatpak.org.

Images

Comments

• As of May 2016, XDG App was renamed to Flatpak. The article needs to be reworked to take this into account and provide the historic transition. — Jeff
• The article is missing a global title. — Jeff
• "However, a six-month release cycle does not work for every application. Imagine if a developer misses the window to get their application into a new distribution — users might not see it until it is a year old!" .. wouldn't the app be 6 months old? - Nuritzi
• This paragraph is a bit unclear: "When building their software, developers can use the flatpak-builder tool to build their application and any necessary dependencies not part of the runtime, whether they’re a favorite EXIF library or a node.js module. The Builder IDE is also gaining a system that allows a developer to start from nothing to a Flatpak App in a matter of minutes, further simplifying the experience." - Nuritzi
• This paragraph needs work too: "Interestingly, a lot of the busywork in supporting various distributions is due to the lack of a runtime environment that the developer can rely on. A runtime environment includes a set of shared libraries at known versions with stability guarantees. This is sometimes called API and ABI stability. Any update to the runtime, especially security updates, will not break this contract. Certain important files and directories should be in known locations, and not vary per distribution. Access to drivers and codecs should be predictable, notably support for OpenGL and GStreamer, respectively. No matter what distribution the application is running on, the environment the application sees during execution should be identical." - Nuritzi
• what are "bandwidth hogs"? ... "Because Flatpak uses OSTree, application updates are minimal deltas between versions. This means less bandwidth, faster updates, and less frustration about the excessive bandwidth hogs often seen with mobile applications." - Nuritzi - RESOLVED, changed "hogs" to "usage"
• I'm not sure about the section titles. It'd be great to have some kind of standardization where it's all questions, or maybe nouns? - Nuritzi - RESOLVED

Reviewer(s) notes

• 1st edit done by CosimoCecchi and AdeliaRahim on 11-May-16

• 2nd edit done by RosannaYuen on 18-May-16. Needs at least one more edit.

• Made global change of "xdg-app" to "Flatpak" and included a footnote to provide brief background on name change - AdeliaRahim

Reviewed by Nuritzi on May 31, 2016

4th edit by Nuritzi on June 13, 2016