GSoC 2026 has come to a close, so here’s where things stand with GPU reset recovery in Mutter . If you’re catching up, the short version is in my intro post : a GPU reset invalidates the EGL context and wipes all GPU memory, and until now Mutter had no way to come back from that. My project is a recovery mechanism so a reset doesn’t take the whole session down. Here is what things look like now:

Mutter graphics recovery demo

In the video: After a period of normal operation, a reset is triggered. Mutter recovers successfully and is back to normal operation instantly. Everything is restored, from the background to windows and cursors. Then a loop is executed to trigger a reset every second while various activities are performed within the session. Through it all, Mutter remains unfazed and the session responsive.

Where We Left Off

In my last progress update , the compositor survived a reset, but with two open gaps:

• The display stayed blank until the creation of a new framebuffer was manually forced (I triggered it with a window maximize keyboard shortcut in that demo).
• The desktop background came back with garbled/wrong textures.

Both of those are now fixed, along with a lot more that wasn’t even in scope for that post yet.

A Well-Defined Recovery Order

My previous update touched on this problem: recovery involves a lot of GPU-associated state scattered across Cogl , Clutter and Meta , and a lot of it depends on being torn down and rebuilt in a specific order. The font renderer needs the stage unrealized first; the stage needs stage views rebuilt before it can be realized again; and so on. The earlier approach made use of a single signal on ClutterBackend , and relied on G_CONNECT_AFTER and GLib’s signal connect order for ordering, which worked but was fragile and implicit. That’s now been replaced with a dedicated ClutterGraphicsRecoveryContext type encapsulating all the core recovery logic, and whose signals are emitted in a fixed, well-documented sequence:

Anything in the compositor that owns GPU-associated state now hooks into one of these instead of guessing at ordering. Only recreate-context can actually fail; if it does, recovery aborts there and recovery-failed is emitted instead of continuing on to recreate graphics resources.

What Else Recovers Now

Framebuffer and background: The display now comes back on its own, no manual nudge required, and the pause is virtually unnoticeable. There is no blackout, at least from the compositor’s point of view. To the user, however, monitors driven by a GPU that resets may briefly go dark. The desktop background restores correctly and immediately: when a background image is set, loading it from disk can take a moment, so a plain colour is shown in the meantime. One caveat worth flagging for anyone testing this: background image loading was recently moved out of Mutter in !4980 . That means reloading the image and reuploading its texture after a reset is no longer something Mutter can do on its own; it now falls on downstream consumers (like Shell) to handle it.

Window and surface content: Wayland surfaces restore to their last rendered frame right after recovery, instead of going blank until the next commit. When a client commits a new buffer in the narrow window between when a reset occurs and when it finishes, and the buffer attach fails because the context is already lost, the surface gets a dummy texture so nothing crashes, and the real content replaces it when we re-attach the buffer during restoration.

Cursors: Both Xcursor-based cursors and Wayland client cursors restore their textures on reset.

Actor effects: All ClutterOffscreenEffect s (blur, desaturate, deform, shader, etc) recreate their GPU-side pipelines and textures on reset instead of quietly holding onto invalid ones.

Stream sources: Screencast and remote-desktop pipewire streams stop and restart cleanly around a reset, recreating the necessary GPU-side resources instead of ending up in a broken state.

Overlays: MetaOverlay s (used for things like cursor sprite compositing) recreate their pipeline and texture too.

Along the way, a decent amount of the codebase moved from static and class-level CoglPipeline s to named pipelines owned by the CoglContext , specifically so they’d get recreated automatically instead of leaking or going stale across a reset.

Tales From The War Front

The context that died too soon and then never died

Historically, objects didn’t take references to CoglContext ; they simply held bare pointers to it. The reason was that there was only a single context for an entire session, which got destroyed at exit after all the objects associated with it had been destroyed.

Since the context is now recreated during recovery, this introduced a whole new problem: some objects now outlived their associated CoglContext , resulting in use-after-frees and segfaults. Specifically, some objects can not be destroyed until they’re replaced by new ones (after we have recreated the context) because they hold on to crucial states which get destroyed by lower levels of the stack as soon as they become inactive. There’s also the case of garbage collection, e.g in Shell, where the garbage collector may keep objects alive after we’ve destroyed their associated context.

To solve this, every object holding a pointer to CoglContext now takes a reference, and the context is explicitly disposed of and marked as defunct during recovery. The use of a defunct context is restricted so associated objects don’t make use of invalid data and silently corrupt memory. The context is finalized when the last reference to it is dropped. Also, with this change, we can more easily spot objects that needed to be restored after a reset.

With this in place, defunct CoglContext objects were, for a while, never finalized after a recovery cycle; they’d plateau at a fixed refcount and stick around. Tracking it down took multiple long GDB sessions. The actual root cause, once Jonas helped dig further, turned out to be leaked CoglPipeline s and pipeline cache entries. The stencil and current pipelines weren’t being unrefed by the context, which in turn kept the default pipeline (their parent) alive too. There was also a cyclic dependency in CoglPipelineCache that only surfaced as a use-after-free once those pipelines were finally unrefed. Two separate bugs stacked on top of each other, and the second one was hidden by the first.

The popup that broke hell loose

During recovery, the whole actor tree is unrealized and re-realized, which started by unmapping everything from the stage down. With a popup open when reset occurs, this crashed at an internal invariant check. A thousand steps (in GDB) later… It turned out a ClutterInputOnlyActor , owned by the popup’s ClutterGrab , was getting destroyed mid-unmap. A grab doesn’t hold its own reference to an actor it owns. So when the actor got unmapped, the grab was detached, which in turn disposed of the actor, removing it from the tree. So, the unmap loop lost track of the next sibling and left the rest of the tree mapped.

The apparent fix was to have ClutterGrab take a proper reference to the actor it owns and pre-fetch the next sibling before unmapping so the loop doesn’t depend on an actor that might disappear underneath it. It worked, in the sense that the crash went away, but it turned out that removing a grab actor during recovery broke other things further down the line in ways that were harder to pin down. The real problem wasn’t how the unmap loop handled a disappearing actor; it was unmapping the actor tree at all as part of recovery.

The actual fix was to stop unmapping the stage during recovery entirely. Two new private methods, _clutter_actor_realize_mapped() and _clutter_actor_unrealize_mapped() , unrealize and realize the actor tree recursively while leaving everything mapped, with a flag on ClutterActorPrivate carving out an explicit exception to the established unmap-before-unrealize / realize-before-map invariant. Recovery now uses these directly without unmapping and mapping the stage, so the popup, its grab actor, and everything else stay exactly where they were.

The time-travelling touch

For a while, single resets were sometimes turning into two or three in a row, with no clear pattern. The reset trigger mechanism (used for testing without messing with real hardware) works by touching a file that llvmpipe watches for a changed mtime. That file lived in a virtiofs-mounted directory shared between my main machine and the test VM, and the mtime it ended up with was consistently a bit ahead of the VM’s and even my main machine’s clock at the time of touching the file, sometimes enough that by the time one recovery finished, the file still looked “new” and triggered another reset. Not a bug in the recovery logic at all or the manual reset implementation, just a quirk of the shared filesystem layer. Moving the trigger file to the VM’s own local filesystem made it go away.

Here’s a sample from one of my debug sessions (timestamps are the giveaway):

$ date -Ins && touch ~/llvmpipe_reset && date -Ins
2026-07-31T15:53:34,157134442+01:00
2026-07-31T15:53:34,160985080+01:00

$ stat ~/llvmpipe_reset
...
Modify: 2026-07-31 15:53:34.255579148 +0100
...

And the corresponding recovery logs:

Clutter-Message: 15:53:34.164: [RECOVERY]: Graphics reset detected
Clutter-Message: 15:53:34.251: [RECOVERY]: Graphics recovery successful
Clutter-Message: 15:53:34.251: [RECOVERY]: Graphics reset detected
Clutter-Message: 15:53:34.321: [RECOVERY]: Graphics recovery successful

Where Things Stand

The recovery mechanism itself is solid: Mutter survives GPU resets, the session stays alive and responsive, and the visible state (background, windows, cursors, text) is restored correctly, automatically and instantly. There are still a couple of edge cases I’m chasing down, mostly around rapid resets, but nothing that looks architecturally hard, just more debugging.

The implementation has now been submitted upstream for review. By the way, that MR is a clean recommit of the whole branch. The real development history, which can be found at my fork , was a lot messier: a lot of iteration, backtracking, and reordering as the design of the recovery cycle itself evolved. Once the approach stabilised, it made more sense to rebuild the commit history cleanly from the current state than to untangle months of exploratory commits.

Real hardware testing has also been trickier than expected. On the AMD GPU I tested with, the default reset method used by the kernel driver (MODE2) turns out not to invalidate EGL contexts, which means it can’t exercise the code path this project implements. The other reset modes either weren’t supported by the driver or failed outright. So testing thus far has mostly stayed in the VM, using the llvmpipe reset simulation Robert implemented before GSoC started.

Honest Note On Scope

Going by the goals set out at the start, this isn’t finished. A few things from the original plan are still ahead:

• GNOME Shell doesn’t fully recover yet: After a reset, Shell recovers successfully, but not completely. On-screen framebuffers, windows, cursors and text are restored. However, the Shell UI (the chrome, overview, widgets, effects, and icons) and desktop background image are not restored. Here’s a recording showing the current state of the UI:

GNOME Shell UI recovery state
Note: This was recorded with a modified branch; with the unmodified branch, Shell currently crashes, as expected.

• The llvmpipe reset simulation is not yet wired into tests or CI.
• Real hardware testing is still an open problem, for the reasons stated above.

I’ll keep working on these after GSoC.

What’s Next

• Chase down the remaining edge cases
• Get GNOME Shell recovering reliably; a few known to-do items include:
  • Clear the background cache and reload the image (will restore the desktop background)
  • Clear the texture cache (should restore icons)
  • Clear the theme node cache (should restore widgets and effects)
• Add test coverage using the llvmpipe reset simulation, and get it into CI
• Find a real hardware reset setup that actually exercises context invalidation
• Push the MR through upstream review

Reflection (A Personal Note)

I still remember opening gsoc.gnome.org on that fateful day “just to see what GNOME’s doing this year” and scrolling down just to see this project, and I was immediately drawn to it. I had known graphics was the path I wanted to take, and this project checked so many boxes. At the same time, I couldn’t deny how daunting it seemed. Yes, I have done some graphics work in the past (my GSoC project last year, in Uni and personally), but nothing of this scale. Anyway, I decided to take up the challenge; after all, why do it if it isn’t challenging?

Here are two notable challenges I faced, from which I also learned a lot:

• The sheer mass of the codebase: Man, Mutter is huge !! Coupled with the fact that my project cut across every layer and almost every aspect of it. I wasn’t building a compositor, but I had to understand how a lot of it worked. This wasn’t the kind of project that dealt with a single subsystem.
• Debugging, debugging and debugging: Logs are good and have their place, but also their limits. This project required using the debugger a lot. I’ve become so much more comfortable with sifting through logs and stack frames, and stepping through thousands of lines of code.

And here are three notable lessons I learned (more of):

• There’s always a simpler solution to any problem (thanks, Jonas).
• Keep digging, never give up: Some of the “bugs” I encountered just kept on giving. Some were like playing whack-a-mole, others like the hydra. Life is all about resilience.
• Community matters more: It’s good to get the job done, but the people we meet along the way are more important.

Throughout the course of this project, I got to do things I never imagined I would (at least, not this soon), like chatting on a kernel dev IRC . I could go on forever, but let’s call it a wrap here for now.

Thanks

A huge thank you to my mentors, Jonas Ådahl , Robert Mader , and Carlos Garnacho , for believing in me and for the guidance and hands-on help throughout. Thanks to Google and the GNOME Foundation for this awesome opportunity. Finally, thanks to the entire GNOME community and everyone who followed along this summer, asked questions, or gave feedback. GSoC has ended, but this isn’t done yet, and neither am I – more soon!