In recent weeks, I’ve been working on adding key rotation support to flatpak, so that repositories which sign their commits and summary files with a key with an expiry date have a way to push updates to that key to all the clients which use them. Currently that’s not possible without each client manually having its config updated to use the new key data (even if the change in key data is just to bump the expiry date).

In the process I’ve learned a few more things about GPG keys, subkeys, signatures, UIDs, etc., which I thought I might dump here in case it’s useful for someone else (or me, in the future). I don’t claim to be an expert, so I may still be misunderstanding some bits. GPG is complex. One thing which has helped ground things is finding the documentation in RFC 4880 (and related) which defines the GPG packet format.

One thing which keeps flatpak’s use of GPG simple is that it doesn’t use any of the web-of-trust features or trust-on-first-use (TOFU). Its use of GPG is limited to the keyring format (essentially pubring.gpg ), for storing and publishing public and private keys, subkeys, signatures, revocations, UIDs; and using them to sign and verify OSTree commits and various repository files ( summary , summary.idx , etc.).

Quick primer on the parts of GPG

GPG has keyrings: collections of primary keys. A primary key has public and private/secret parts. The way flatpak uses it, the secret part of a key always stays on the server, and is used to generate signatures. The public part is published by the server and configured in every client using that repository, used to verify the commit signatures. Each client has one keyring per remote it has configured. Typically this contains one primary key, but it could contain several, any of which can be used to verify signatures from that remote.

A key is identified using a fingerprint, a 40-character hex string. It can also be identified using a key ID, which is a substring of the fingerprint. There’s also keygrips, but let’s ignore those.

You can see fingerprints for keys using gpg --list-keys --fingerprint .

A primary key has one or more subkeys (at least in flatpak’s usage). All of these keys have public and private/secret parts as above. Each key has a usage which indicates what GPG will let you use it for, such as certifying, signing, authenticating, encrypting. The primary key is typically used to certify its subkeys by creating cross-certification signatures which bind them to the primary key, forming a short chain of trust — anyone who trusts the primary key should trust a subkey which it cross-certifies.

You can see subkeys as the sub lines using gpg --list-keys --with-subkey-fingerprint . The [SCE] , [S] , [E] (etc.) fields show the usage flags for a key.

The other usage we care about is signing (flatpak doesn’t use authentication or encryption usages). GPG separates keys by usage to prevent attacks where a key is used for a purpose it’s not intended for, and because keys used for different purposes often need to be treated with different levels of care.

In particular, separating by usage means the private/secret part of the primary key can be kept completely offline, and only brought out in a special key signing ceremony when a new subkey needs to be generated and cross-certified. This reduces the risk of the very valuable primary key, which is the root of every client’s trust in the repository, being leaked.

So, we use a signing subkey for day-to-day signing of OSTree commits. For a big flatpak repository, the private/secret part of this subkey might be kept in a hardware security module, so it can’t be exfiltrated from the server if the server were compromised. But there’s still the risk of a compromised server being used to sign things it shouldn’t (such as malicious apps).

That’s a matter for server security, but we can somewhat mitigate against the possibility of the signing subkey being leaked by setting an expiration date on it. Clients might choose not to trust signatures made by it after that date; and gpg certainly wouldn’t allow it to be used to create new signatures.

The expiry date of a key is shown as an expires field in the gpg --list-keys --with-subkey-fingerprint output.

What happens when the subkey expires? By that point, the administrators should have generated another subkey, cross-certified by the primary key in a key signing ceremony (I assume the ceremony involves cake). The private/secret part of the new subkey needs to stay secret, as before; but the public part needs to be distributed to every client’s keyring, along with the new cross-certification signature from the primary key, so the clients know they can trust signatures made by that subkey.

That’s the bit which flatpak is currently lacking.

So in summary: GPG has keyrings. Keyrings have primary keys. Primary keys have one or more subkeys and cross-certification signatures from the primary key on those subkeys. Each subkey has a usage, but flatpak only uses certify (for the primary key) and sign (for the subkeys). Keys can have expiration dates.

And if you want to see the full contents of a keyring, run gpg --list-keys --with-colons . It’ll output everything (no filtering) in a machine readable format described here (best reference I’ve been able to find), which is sometimes easier to use than remembering which --with-blah option to pass to GPG to get it to show the information you want.

What else does GPG have?

Quite a few things. We’ll ignore the big things which are not relevant to flatpak.

Each primary key also has one or more UIDs. These are like subkeys in that they are cross-certified by the primary key. Each UID is a user identity — typically a name and email address. If you were using GPG in a web of trust, the binding between the primary key and a UID is what you sign that you trust when you sign someone’s key in a key signing party.

The UIDs are listed below each primary key in gpg --list-keys .

Flatpak doesn’t need UIDs, but they are an unavoidable part of GPG — each primary key must have at least one. A flatpak repository will typically put a server contact email address in the UID and then everyone will ignore it.

UIDs can be revoked; for example if someone loses control of the email address in it and wants their friends to no longer trust emails from it. Flatpak currently doesn’t use this.

What else can be revoked? The cross-certification signatures! You may have heard of a GPG revocation certificate. This is a way of revoking an entire primary key. But there’s also a way of revoking a particular cross-certification signature, meaning that the primary key is still valid/trusted, but the owner of the primary key has lost control of one of the subkeys, and that subkey should no longer be trusted. This is different from key expiration, as it’s a statement that something has explicitly gone wrong.

Because of how GPG is built up as a series of packets of different types, a signature revocation is actually a revocation packet appended to the primary key. This means you can re-cross-certify a subkey after revoking it, by appending another cross-certification packet. And even revoke it again after that. Not sure if there’s a use case for this or if it’s just a consequence of the packet format, but this behaviour does play havoc with working out whether to trust a subkey.

Cross-certification signatures can also have an expiration date built into them, separate from the expiration date of the subkey. I’m not sure of the use case for this either, but there must be one.

Some notes on running GPG on the command line

GPG is historically famously hard to use. I feel this has got better in recent years, particularly for scripting it. In particular it’s added a whole load of --quick-blah commands to generate keys, set expiries, etc. from scripts.

One thing which repeatedly tripped me up before I stopped trying to fight it was its concept of a ’homedir’. GPG needs to look for its keyring (and trust database, and various other files) somewhere, and will not run without them, so you always need to pass it a ‘homedir’ to look for them in. By default, this will be ~/.gnupg , so it’s very easy to accidentally end up operating on your personal GPG keyring when you’re trying to do something in a project.

If using GPG as a tool or in a script, I think you should always create a temporary homedir, pass it as gpg --homedir=/path/to/temp and explicitly import whatever keys or context you need into this homedir before doing whatever operation you need.

This is necessary even if ‘all’ you want to do is view a downloaded .gpg keyring, because what GPG displays may be affected by the trust database in its homedir. So to view a downloaded keyring you should still do something like mkdir temp; gpg --homedir=./temp ./path/to/download.gpg .

If you are trying to sign something, you will typically pass the fingerprint or key ID of a primary key to GPG; for example as gpg --local-user 0xfingerprint --sign ./path/to/file . GPG will helpfully use the usage flags of the subkey of that primary key to choose which subkey to sign with. If you want to sign with a specific subkey, you need to suffix the fingerprint with an exclamation mark ( ! ) otherwise GPG will still choose what it thinks is the most appropriate subkey, which might not align with the subkey you carefully chose. This ! suffix format is common throughout the GPG command line interface for when you want to specify a specific subkey.

Sorry

That was more of a braindump than I imagined when I set out to write this. I hope some of it is useful; feedback welcome if I’ve got anything wrong. If any GPG experts fancy reviewing key rotation support in flatpak, the draft implementation is here .

Hello,

Here’s some thoughts on software for August.

Bear with me on these broad categories but I think you can group most software projects into one of these groups: art, infrastructure, and activism.

Art is primarily to communicate experiences and feelings to others. Making video games is art. Making digital musical and instruments and visual effects is art. The drawing you did as a child that’s stuck on somebody’s fridge is art. The 3rd year computing student’s university coursework, uploaded to Github without comment and abandoned forever… that’s art, or at least, it’s a sketchbook. The thorny entry to the IOCCC , the optimized inner loop deep in some graphics toolkit, that only a handful of people will ever look at, but all of them will agree: that’s art .

People make art for fun, learning and practice.

Infrastructure is stuff that is needed and expected for the world to function. The definition of infrastructure changes over time, as societies adopt and depend on new technologies. Roads, milk delivery, electricity, bakeries, aqueducts, supermarkets, the internet, trams, the lift in your apartment building, and so on. Many societies depend on software projects now. Web browsers, phone cells, operating systems, social networks, software developer tools, power grid management, Google Maps, and so on.

People make infrastructure for money, or perhaps out of a sense of duty.

Activism is a desire to bring about a particular social or political goal or change. I don’t know when the first activist software project started, but it was no later than the 1980s when the GNU project began its stated mission to make proprietary software unviable via copyleft , and many free software projects followed along. The design of the GNU C compiler was shaped by the mission. Tor began in the 2000s with the goal of ensuring private, uncensored internet access. Bitcoin began with the stated aim of destroying the financial system, the wake of the 2008 bank crisis. Although in many countries it’s now regulated financial infrastructure, which shows you that projects can move between these categories over time.

People spend our energy on activism based on our beliefs, usually a sense of wanting to make the world a better place for everyone, or at least for ourselves, and our friends and family.

Not everything fits into these categories (the biggest gap I can see is experiments and research) but let’s keep this short, I want to use them to look at the conversations I keep seeing in the open source world this year.

When one person sees a project as activism and another sees it as infrastructure, you see some genuinely confused conversations happening.

Codeberg banned projects with largely AI generated code from the site, and Sourcehut is considering doing the same . If you see these projects as Git hosting infrastructure similar to Gitlab and Github, then that decision makes very little sense. Why would they want to host fewer projects? However if you see the project as a group of activists trying to reduce the power and influence of US tech firms, and limit the harms of rapid of adoption of AI, then it makes a lot of sense.

Linux decided to allow some LLM use and not put too many limits, as long as its making the project better . If you see Linux as a bunch of engineers building infrastructure then it’s a very logical decision. But if you thought your contributions were part of some activist movement then that might be a disappointing decision.

Every so often I hear someone say things like “please keep politics out of software” . You can infer that they’re probably talking about infrastructure software — and it’s as misguided as if they said it about any other infrastructure. Is an aqueduct political? If one country is seen as stealing another country’s water then… yes. Can a shipping lane be political? Yes, see numerous examples, including the ongoing US-Iran conflict. Can a road be political? Yes, if it crosses a border, especially if it’s in dispute. Ask someone old from Berlin or the north of Ireland about whether roads can be political.

Maintaining software takes a lot of effort. If someone is putting in that effort without being paid, they have some other motivation for doing it. Humans rarely do difficult, laborious work for no reason. Our motivation might be to learn, to show off, to have fun, to meet people, to collaborate with friends, or it might be to work towards some kind of political or societal change. I’ve probably contributed to open source for all of these reasons at different times.

We are increasingly referring to open source software as “digital infrastructure” and funding some of the maintenance work via corporate money and public money. This is great, but it requires the project to frame itself as infrastructure rather than activism. You are unlikely to get funding from Microsoft if you openly state that your goal is to destroy Microsoft. You are unlikely to get funding from a government if your stated goal is to destroy the modern financial system or prevent all forms of censorship.

Activists using open source licenses have something of a problem to deal with. If your goal is to bring about world peace, you hardly want people building drones and missiles using the software you develop. Yet the open source movement have made it clear that if you try to limit who can use your software, it’s no longer open source. And, many software engineers have made it clear they don’t give a shit about software licensing anyway and they’ll use your code however they want without even reading the license.

You can control access to software infrastructure, of course, just like you can put soldiers and passport controls on a bridge or a road. But you can’t call it open source any more.

The GNOME desktop project is art, infrastructure and activism. The discussion on Reddit is mostly people who design and post desktops and themes for fun. Several corporations build products with GNOME, and treat it as infrastructure. And then there are contributors who want to bring about change, perhaps weakening the power of Big Tech by spreading an ethical alternative to Android and iOS.

GNOME hasn’t made a statement on AI use, and I don’t think we’ve tried very hard to discuss it so far. I wonder if we’re putting it off because we’d have to also discuss whether our goal is to maintain some infrastructure, or build something cool-looking, or bring about meaningful societal change?

Over the summer of 2026, I worked towards bringing the option of playing Vocab-Style puzzles to GNOME Crosswords as part of Google Summer of Code . This entailed adding support to the puzzle library, and writing the backend of the algorithm responsible for grid generation. Jonathan Blandford provides a thorough rundown on the ins and outs of Crosswords with these slides .

The first few weeks of the start of GSoC were spent by adding a drop-down calendar widget to Crosswords Editor, and storing GDate data as ISO8601. We decided to integrate this into my design despite it not being directly related to the proposed project because it still contributed to the Crosswords app.

For the new puzzle type, I started with adding support to the puzzle library, libipuz , which is responsible for formatting and representing puzzles styled as ‘paper-and-pencil’ crosswords. My primary mentor and I bounced ideas back and forth for some time before we started designing and writing the algorithm. We landed on an idea and I created an initial design document for the plan of action to follow for the summer. We decided that aiming for both the backend and frontend in one summer may be more work than we initially thought, so we concluded on focusing exclusively on getting a working algorithm to build from.

The bulk of this summer consisted of writing and reconfiguring the depth-first search backtracking algorithm.

GitLab Links to Code:

An overview of my GitLab profile can be found here.

• Implementation of New Vocab Puzzles
  • This is the bulk of the algorithm and unit tests. The user inputs a word, the word gets analyzed in a recursive function to check for crossings and constraints, we save the state of the board, backtrack if needed, and present a grid.
• Vocab Ipuz
  • Adding new class for IpuzVocab to support vocab puzzles in the puzzle library (libipuz).
• Check for Island Words
  • There may be a word in a list that may not share any intersection points with any others no matter how many backtracks are done. Consequently, this affects grid creation in a timely manner and may prevent grid generation at all by stating it false. We can compare the bitmask of words in the list before we even activate the algorithm to detect a word that would potentially conflict with the others.

• Date Validation
  • Previously, the Crosswords Editor had a free-for-all AdwEntryRow. However its purpose is to present a legible date to the user, and store the date in ISO8601 format. I used GDateTime, GDate, and a Gtk Calendar widget to add a drop-down calendar in the date box. The chosen date presented in Gregorian style to the user, and stored as ISO8601 to the backend.
• Added Dispose to Shapebg
  • Releases references to GObjects that Shapebg owns and frees Shapebg’s remaining memory.

Design Docs:

• Updated Design Doc
• Vocab Design Doc

Blog Post Links (Most Recent -> Oldest):

• GSoC 2026 Final Report – Laureen Caliman
• Vocab-style Crosswords Update | Final Stretch
• Intern Experience at GUADEC
• Update on Crosswords Backtracking Algorithm
• Pick a Word, Any Word
• Extending Libipuz
• GSoC Project Start
• Introduction Post

I also gave a lightning talk at the 2026 GNOME Users and Developers European Conference here . Thank you to the GNOME Travel Committee for making that opportunity possible.

I still have some work to do for both Libipuz and Crosswords: finish the island-checking function detailed above, open a new MR to choose the most compact/square grid out of 500 options and present that to the user directly rather than them cherry-pick through a large selection,  create a new MR to add photos of the puzzle in libipuz using gi-docgen, convert the 500-generated grid code to a PuzzleTask, and incorporate the frontend to make this a fully-functioning part of the game.

I would like to thank my primary mentor, Jonathan Blandford, and my other mentor, Federico Mena Quintero, for their guidance, feedback, patience, and teaching. This program was exactly what I needed to become better at development and serves as my rock to open-source contributions. I learned a lot of valuable skills such as document reading, how much to push in a commit, how to slow down trying to get a lot done at once, but simultaneously how to speed up my progression on the parts that actually matter, code with brevity, and a whole lot of dealing with nasty version control!

I intend to continue contributing to GNOME Crosswords as well as the overall GNOME Foundation. I look forward to collaborating with more people involved in the Foundation!

Overview

My project was to write the Pitivi Timeline Ruler in Rust using GTK4 and create GObject bindings for it. The project goals overall went well, and I was able to complete the widget successfully along with adding a layout manager structure to the ruler for child widgets to be added from the Pitivi/Python side. My standalone repository contains a working Python demo to verify successful FFI functionality.

Integration into the Pitivi application is not complete. The GTK4 port branch is still a work in progress and is not quite ready yet for deployment.

My commit adding the ruler to the WIP GTK4 branch can be found here .

My repository containing all of my commits and the standalone widget with demos can be found here

Plan Going Forward

I will stay actively involved in the GTK4 porting effort, apply for GNOME membership and stay involved in the Pitivi project. The GNOME community has been very welcoming, and I plan on continuing to contribute both to Pitivi and the broader GNOME ecosystem for the foreseeable future.

Design

General Architecture

A major structural change that has been made after mentor feedback was to move application orchestration out of the widget itself, and keep more of the logic on the app side. The design is a "dumb widget, smart app" framework, where the ruler does not own editor policy.

The app provides the logic for how to handle gestures and what to do when the user interacts with the widget. One example of this is that project_duration was removed as a property entirely, and logic around bounds is now handled entirely on the app side. This allows the widget to be used in a variety of contexts, and allows the app to handle bounds in whatever way is appropriate for the context.

The rendering code also received a fairly large cleanup. Previously, the widget stored several pieces of drawing state separately, including adjustments, cached Pango layouts, and font descriptions. These have now been grouped into a single DrawingState struct behind one RefCell . This made ownership easier to follow, and allows the code to release it's mutable state borrow before connecting or disconnecting signals.

I also introduced a labels_dirty flag so timeline labels are only recalculated when they're actually needed during the snapshot phase, rather than repopulating the cache with every setter or adjustment call.

The widget follows font and color settings from the users GTK4 theme, allowing user theme changes seamlessly.

Time Markers

The ruler utilizes a BTreeMap to cache the text markers that are visible at the current window size plus a half a screen size buffer over either edge. I used a BTreeMap because it offers high efficiency insertion, deletion, and lookup for this case where our markers are sorted.

The ruler recalculates the spacing needed between labels by calculating an intentionally wide timecode using the current font and some padding. This measurement is cached until a change in frame-rate or font invalidates it.

Labels include frames when zoomed in below 1 second, and drop the frame count when zoomed out past that point.

Major Ticks and Intervals

Major divisions are calculated using a two-mode strategy. First, the pixel density is calculated to determine whether a one-second interval can meet the minimum label spacing. If so, frame-aligned intervals are used, with the smallest interval meeting the minimum label spacing selected.

If one second is too narrow for a label, frame alignment is abandoned in favor of whole-second intervals, selecting the smallest interval that meets the minimum label spacing from a pre-defined list. This list can be tuned later to meet Pitivi's UI needs.

Minor Ticks

After discussing ticks with some video editors, I realized that frame alignment is more important to the video editing community than clean time divisions. Because of this, I made the decision to implement a reverse modulo loop to consider possible intervals from largest to smallest that accurately divide frames. The resulting spacing is also checked against the minimum tick spacing to make sure we don't end up with a block of ticks that are too close together. The result is that our minor ticks are asymmetrical and are not even as the user zooms in and out, but they do always accurately divide by frames in the major interval.

Layout Manager & Layout Child

We needed some mechanism to position arbitrary external widgets (play-head, markers, loop-brackets) on the timeline. The timeline uses nanosecond timestamps, so the parent ruler determines x-cords for the children based on the zoom and horizontal scrolling position.

I introduced two objects (public C-wrapper and Engine):

1. PitiviTimelineLayoutChild
   • Subclasses gtk::LayoutChild .
   • Metadata wrapper for generic widgets dropped onto the timeline.
   • Adds a custom GObject property for timestamp .
2. PitiviTimelineLayoutManager
   • Subclasses gtk::LayoutManager and is installed on PitiviTimelineRuler .
   • Creates the custom layout child for each child widget.
   • overrides measure() and allocate() .

During allocation, the manager reads the ruler's ns_per_pixel , horizontal adjustment, and the child's time stamp to determine the child's x-coordinate. The child is centered on it's time stamp and scrolls with the ruler. The Python demo shows an example using ruler.add_marker(playhead, 0) to add a play-head at the start of the timeline.

Bindings and Build

The FFI layer in ruler.h and capi.rs expose the following small API:

• pitivi_timeline_ruler_new() creates a ruler as a GtkWidget .
• pitivi_timeline_ruler_clocktime_from_pos() converts an x coordinate to a nanosecond time stamp.
• pitivi_timeline_ruler_add_marker() adds a child widget at a time stamp and returns a layout-child object.
• The properties of the ruler are exposed as GObject properties, and can be set and retrieved using standard GObject property accessors.

The Process

Challenges

The FFI bindings were a major challenge for me. I ran into significant challenges fixing bugs and understanding conceptually how the Rust bindings actually worked. My initial FFI attempt did compile, but I had to work through significant GTK initialization and headless CI issues. The fixes show up in my commit history as changes of a couple of small lines of code, but the time spent understanding the issues was significant.

I also came into this with very limited knowledge about Flatpak and Meson. Getting myself to a point where I understood what the build systems were doing took a significant amount of effort. I think I spent about the same amount of time reading about GIR, GObject, Flatpak and Meson as I did writing the code. For a very small percentage of the actual code, those tools required the most attention. I view this learning as really valuable for future work in the GNOME space, and I tried to take as much time as I could afford to do my best to genuinely deepen my understanding.

The rendering logic, while similar to previous GTK4 widgets I had built, ended up having numerous rounds of refactoring and learning. I wrote about 4 different strategies for scaling, multiple designs for splitting time and frames and multiple minor tick rules. Even once I was settled on a design, I had multiple waves of finding cache inefficiencies, clearing stale entries, and removing precision and allocation bugs. The ruler is visually small, but the underlying math and rendering logic took quite a bit of work to really get to a professional standard. I would not be surprised if the logic changes again in the future as I continue to work on the Pitivi GTK4 port and get feedback from the community.

Major Milestones

• May 3: Built the initial window render as a standalone GTK4 application.

• May 8: Added the initial GObject getter/setter structure, a GtkScrolledWindow test, and zoom bounds.

• May 9-13: Drew the initial major and minor ticks, changed the scale to nanoseconds per pixel, and added Pango labels to test scale behavior.

• May 14-28: Reworked time stamp math, wrote code to extract frame-rate from video's GES timeline which was later scrapped, and refactored rendering of ticks to draw a single interval and then paint it repeatedly across the ruler.

• May 30-June 4: Wrote the dynamic label-width measurement, minimum tick spacing, and frame-oriented major/minor interval selection structure. Added gtk::Scrollable interface to the ruler.

• June 6-26: Refined cache eviction for scrolling. I also addressed zoom drift, font and DPI changes, adjustment-signal cleanup, and precision around the playhead coordinates. I wrote the unit tests for subdivision and timecode math in this period as well.

• June 30-July 3: Removed redundant APIs, moved gesture handling to the app side, refactored types throughout the codebase, and removed the widget's project-state ownership.

• July 5-25: Created the initial FFI bindings and the layout manager and layout child, and then attended GUADEC in A Coruña, Spain. I managed to resolve the GTK initialization CI issues and successfully exposed the ruler and layout child.

• July 26-29: Added dirty-label cache invalidation and consolidated the rendering state into DrawingState struct. These optimizations simplified borrow management and reduced thrashing the Pango cache.

• August 1-12 - Added the Python demo to python/test.py , fixed allocation updates for children widgets near the start of the ruler during zoom changes, added Meson build, added the Meson test target that runs the Rust tests.

• August 12-Present: I am currently battling through adding my ruler to the Pitivi GTK4 port branch.

Special Thanks

I would like to send a massive thank you to my mentors, Yatin and Alex Băluț. I was going pretty significantly off track a few times throughout the project and I got nudged in the right direction at some critical moments.

I would also like to thank the GNOME travel committee. Getting to attend GUADEC was a really incredible experience. Sergey Bugaev took a lot of time to sit and work through some of my bugs with me and help provide some guidance. As a long time GNOME daily user, getting to spend time and meet maintainers and Federico was a really exciting opportunity.

GSoC has been a great experience, and I am very grateful for the opportunity to work on this project. I am looking forward to continuing to contribute to the GNOME ecosystem.

One of the larger bits of work I’ve been doing crosses many core projects. I’m motivated to switch to GNOME OS across multiple form factors, but to do that, homed and related tooling need quite a few functional gaps fixed.

TL;DR

• github.com/chergert/linux
• github.com/chergert/cryptsetup
• github.com/chergert/qemu
• github.com/chergert/systemd
• github.com/chergert/edk2

I really wanted a rather simple storage stack. I try to run XFS or ext4 in most places because they continue to serve me well. I also want to run GNOME OS on phones, where we’ll need dm-inlinecrypt for better performance and to avoid loading raw storage encryption
keys into system memory.

Where I prefer something like thin provisioning is a multi-user setup, where I want encryption, integrity, layering, and reliable accounting without dedicating a fixed partition to every user.

Getting all of those properties at once required work from the filesystem down through device-mapper, the block layer, UFS, QEMU, and cryptsetup.

Keeping the storage key out of memory

Traditional full-disk encryption requires the raw volume key to enter kernel memory. That key can unlock the entire device, so extracting it from a running system is particularly valuable to an attacker.

Hardware-wrapped keys change this arrangement. The long-term key is stored as an opaque, device-bound blob. During activation, it is converted into a boot-scoped ephemeral blob and handed to the storage hardware. The hardware derives and programs the AES-XTS key without disclosing it to software.

There is still a separate 32-byte software secret for integrity and other cryptographic operations which cannot be offloaded. Knowing that secret does not reveal the inline-encryption key.

This reduces the opportunity to extract a reusable storage key, but it is not magic. It does not protect plaintext already present in memory, nor does it defeat an attacker who fully controls the running system.

The practical problem with hardware-wrapped keys is that they are difficult to develop and test without the relevant hardware. Even when hardware is available, failures across the complete stack can be difficult to reproduce and inspect.

So I started building the hardware I needed in QEMU.

A virtual UFS inline-crypto engine

The QEMU work adds an optional UFSHCI 4.1 inline-crypto profile. It supports AES-256-XTS, 512- and 4096-byte data units, 32 keyslots, 64-bit data-unit numbers, and both legacy and MCQ request formats.

It models more than just the encryption operation. Keyslots are programmed in stages before being activated, can be evicted, and are zeroized during reset. Requests take their own snapshots of key state so that concurrent eviction or reprogramming has deterministic behavior.
Crypto failures are reported as storage errors rather than returning corrupted data.

There is also a test-only wrapped-key mailbox. It can import or generate a key, prepare a boot-scoped version, derive the associated software secret, program a keyslot, and evict it. The long-term and ephemeral representations use authenticated envelopes so tests can also exercise damaged or substituted blobs.

This mailbox models the API and lifecycle that the guest needs, but it is not a trusted execution environment. QEMU necessarily has access to its root secret.

Following an encrypted write

With the hardware model available, a write can be followed through the entire Linux stack.

dm-inlinecrypt attaches an encryption context to the I/O, including the key and data-unit number. The block inline-crypto layer programs a UFS keyslot, then UFS submits the request to QEMU. Only encrypted bytes reach the backing image. Reads take the reverse path.

This makes dm-inlinecrypt a useful full-device target. Device-mapper describes which blocks should be encrypted, while the actual transform stays in inline-encryption hardware.

The target distinguishes raw keys from hardware-wrapped keys. A long-term wrapped blob is prepared into a fresh ephemeral blob during activation. Only the ephemeral form is placed in an active device-mapper table.

Key replacement also makes secure suspend useful. Userspace can suspend the device and wipe the active key. Resume is refused until a replacement has been installed, at which point a new ephemeral key is prepared and programmed.

Integrity without losing the hierarchy

Encryption by itself does not prevent undetected modification, so the protected configuration places exported dm-integrity above dm-inlinecrypt .

That ordering means integrity authenticates the plaintext seen by the filesystem. Inline encryption protects the filesystem data as well as the integrity tags, superblock, and journal when they reach physical storage.

The HMAC-SHA256 integrity key is derived from the hardware-provided software secret using HKDF-SHA256. The binary LUKS UUID is used as the salt, along with a fixed domain-separation string. This keeps integrity separate from the hardware-only inline-encryption key.

I’ve added a fixed profile at 4096-byte integrity blocks, 32-byte HMAC tags, colocated metadata, and a 32 MiB journal. This is fairly intuition based so it needs more testing.

Suspend and resume follow the layering. Suspend wipes integrity first then inline encryption. Restoration is reversed. This is really hard to test with real hardware, so Qemu again really comes in handy.

Provisioning blocks before publishing them

Thin provisioning adds another problem. A filesystem may publish a logical allocation before the thin pool has assigned physical storage. Failure from lack of capacity is then deferred until too late such as when writing data, an integrity tag, or the integrity journal. All of those can be catastrophic.

I added REQ_OP_PROVISION based on earlier ideas on LKML to make persistent allocation a block-layer op. It is different from a write and is effectively the opposite of discard: it asks the storage stack to ensure that a range is physically backed.

Provisioning is carried through the block core, loop devices, device-mapper, thin volumes, dm-integrity , and ext4. Thin volumes allocate, zero, and commit their mappings. Integrity provisions every corresponding data, metadata, and journal region. Ext4 provisions new data and metadata extents before exposing mappings.

The initial ext4 support is conservative. The provision mount option implies nodelalloc , requires 4 KiB extents without bigalloc , rejects unsupported stacks, and disables online resize.

Turning it into a LUKS2 workflow

The cryptsetup work ties these pieces together.

A platform provisioner can generate or import a wrapped key, derive the optional software secret, format a LUKS2 device for hardware-wrapped encryption, and add ordinary LUKS2 keyslots using the opaque blob as the volume key.

Hardware-wrapped segments have an explicit key_type and mandatory requirements. The integrity configuration is also fixed and marked as dependent on hardware-wrapped-key integrity support. That should make older implementations reject the device.

During activation, cryptsetup retrieves the long-term blob, prepares an ephemeral one, creates dm-inlinecrypt , derives the integrity key, and finally creates dm-integrity above it.

For now, this interface is library-only which is how I’m using it from homed.

Testing all of this in a custom GNOME OS build resulted in me finding some issues in tianocore as well (edk2) which I’ve fixed in my tree to allow booting off PCI-UFS over SCSI.

A laboratory for the whole stack

The important result is that this can now be tested without specialized storage hardware.

We can run AES-XTS known-answer tests and independently inspect ciphertext in QEMU’s backing image. We can test both data-unit sizes, legacy and MCQ queues, fragmented requests, concurrency, reset, cancellation, rekeying, damaged envelopes, and storage errors.

We can also exercise thin-volume provisioning through integrity and inline encryption, reject malformed or downgraded LUKS2 metadata, and verify secure suspend, resume, and key replacement.

Each layer has tests which I tried to keep working and improve along the way.

There is plenty left to do. Namely, I’m not really interested in doing LKML type stuff while unemployed living abroad. So if this is something other people want, they’ll need to encourage their respective teams to pick up the work.

Either way, I now have something useful which is a virtual test lab for a security feature which requires each of these layers to work together.

Earlier this year as I drift abroad in France, I made a new abstraction over Avahi and systemd-resolved. It is called librebonjour and I wrote about it here .

It’s nice in that I no longer need to build Avahi to get GObject bindings to essentially call a D-Bus interface. It’s also nice to not have to care as an application developer if the system is configured with Avahi or systemd-resolved. Though, the systemd-resolved abstraction was lacking a bit compared to Avahi due to missing features.

When you are browsing for services using Avahi, you can be notified automatically of changes. This doesn’t quite work the same in systemd-resolved. Librebonjour had to set a timer and poll occasionally for updates and compare old-to-new sets to notify the application. Not very ideal.

When looking at a recent systemd checkout, I noticed that it already had support for the notification over its varlink interface. A handfull of commits later to hoist a few things and handle client disconnections/isolation properly and I can have the feature for librebonjour too.

One more dependency I can cut out of my system (there will be many more coming, I assure you, as GNOME is heavy with cruft).

I’ve been working on a bunch of things across the Linux puzzle for a product I want to build. Here is an overview of a few of those things.

LibMKS at 120hz

I wanted to get my virtual machines to 120hz so that I can start testing product features inside of VMs. In fact, I actually like doing development with virtual machines over say, trying to shove all your development tooling in a sysext which, at least to me, feels like square-peg/round-hole territory.

To get this working, a few things needed improvements.

Qemu

Qemu has a dbus display backend where it can send you DMABUF FD. But it doesn’t really handle any sort of sync and that becomes a problem as you crank up the frame rate. Additionally, it just defaulted to 75hz with no mechanism to override it.

So I have some patches which provide a new D-Bus interface which can be implemented by LibMKS. It provides something more like a Vulkan swap-chain as well as API to set the refresh rate. While this isn’t a mapping 1:1 of what a wayland protocol might do for frame rate, it does match more what the emulated graphics device expects, so it is probably fine for now and clearly an huge improvement.

A big change in the new API is that we will register all the DMABUF up front, and then tell the client just to switch to another DMABUF along with damage rectangles. Of course, I also had to make Qemu start collecting damage rectangles correctly.

• DMABUF3 Listener API for D-Bus
• Allow Clients to set Frame Rate

Linux

With those changes in place, I kept seeing damage being full-frame. The next part of the stack that can break is thus the Linux kernel virtio graphics driver. Damage rectangles come in as properties on the drm plane being submitted. So it turns out that in two places some short circuiting was preventing that from working right.

After fixing all that (and the corresponding LibMKS side) I have decent graphics performance in a VM.

Since I continue to be floating precariously abroad, this is my notice of such patches. If you are interested in seeing these upstream and work in either of those communities, feel free to crib them, improve them, and submit them upstream. I’m happy locally patching my software given the copious amount of free time I have so there is little incentive for me to collaborate with corporations.

• Virtio fix 1
• Virtio Fix 2

Combined with the LibMKS merge request !53 I can have both minimal damage rectangles all the way to host GPU scanout as well as drag windows around in the guest quite fast.