WPE WebKit and WebKitGTK 2.54 have been released with a bunch of improvements and new APIs as usual, but there’s one point that kept the Igalia WebKit graphics team busy for the whole cycle: the new Skia-based compositor. The replacement of Cairo with Skia for content rendering has been a success and it’s already well integrated and optimized. We thought we could try to use Skia for the composition too and replace TextureMapper with Skia. TextureMapper was introduced in 2010 for the Qt port and later adopted by other ports. It uses the OpenGL ES API and maintains a collection of shader programs to paint different content. Nowadays TextureMapper is mostly the same code and shader programs, and it’s unmaintained and missing features. However, the performance was good and it has served us really well all these years. So, this time the goal was not to get better results in benchmarks, but to modernize the implementation, reduce the amount of code to maintain ourselves (like all shader programs) and make it easier to implement the missing features and fix existing bugs. This post is a summary of all the work we have done this cycle to implement the new Skia compositor.
SkiaCompositingLayer
The first step was adding an SkiaCompositingLayer class to replace TextureMapperLayer and adapt all the code to use one or the other depending on an environment variable. The initial implementation was based on the TextureMapper one for the things that are common like iterating the layer tree, computing transformations, etc. The way layers produced their contents didn’t change, so we were receiving textures for tiled content, video buffers, WebGL, accelerated 2D canvas, etc. SkiaCompositingLayer created a Ganesh Skia surface to draw those textures using SkCanvas::drawImageRect(). This initial implementation was enough to run the default MotionMark test suite, since it doesn’t use other composition features. Even though performance was not the goal, we had to make sure we didn’t regress. This initial implementation was neutral in MotionMark. We needed tests to implement those features and measure performance at the same time, so we decided to add a new set of tests to MotionMark, just extending the existing tests to require composition, which makes sure that filters, masks, path clipping, transformations, etc. were done by the compositor.
Filters
We first tried implementing filters using an intermediate surface like TextureMapper does. It worked, but the MotionMark score in the filters test was much worse. We realized that with Skia we could implement most of the filters without using an intermediate surface. All filter types except blur and drop shadow can be simplified to an SkColorFilter with SkImageFilter::asAColorFilter() which can be implemented without an intermediate surface, just by setting the color filter in the SkPaint we pass to SkCanvas::drawImageRect(). This not only fixed the performance regression, but also gave better results than TextureMapper, which always needs an intermediate surface.
Masks
There are two different kinds of masks: image mask, where the source mask is an image already, and clip path, where the mask is represented by a path to be clipped. In TextureMapper both are implemented the same way using intermediate surfaces. The mask is painted into a surface and then the masked layer creates an intermediate surface where its contents are first painted and then the mask contents on top using DstIn blend mode. Skia has APIs that allowed us to implement both cases in a much simpler and more efficient way. In the case of image masks, where we already have an image, we paint the mask contents once and keep it cached, and then the masked layer creates an SkShader for the image mask that is passed to SkCanvas::clipShader() without having to paint into an intermediate surface. Clip path masks are even easier, because we can just take the path we get and build an SkPath we can pass to SkCanvas::clipPath(), without having to paint the mask as an image at all or use any other intermediate surface. Once again, masks were not only easier to implement but they ended up being more performant too.
MotionMark composition suite, WPE with GPU rendering on a Raspberry Pi 4, comparing TextureMapper (312400@main) with the Skia compositor (313600@main). TextureMapper never implemented blend modes, so its high score on bouncing blend circles is the score for not doing the work.
3D contexts
The implementation of 3D layer contexts is fairly independent of TextureMapper and OpenGL, so we could just take it almost as it was, using SkPath to build the clips and a few other adaptations. We could also fix existing bugs like the z-ordering that has always been broken in TextureMapper.
The same page rendered by TextureMapper (left) and by the Skia compositor (right), WPE on the same build. The red box intersects the rotated green plane. TextureMapper draws the box flat against the plane, so the intersection is lost; the Skia compositor splits it, drawing the part in front of the plane and hiding the part behind it.
Blend modes
TextureMapper never supported blend modes and they were easy to implement with Skia just using the SkPaint property for it. This made several layout tests start passing.
Batched painting
After implementing all the features we were at a point in which we had the same or better performance in all tests except for three MotionMark compositing tests that were giving much worse results. Those tests use small layers and give a high result which means we end up adding a lot of layers to the scene before we start skipping frames. The root cause was the large number of layers filling the command queue of Ganesh. Skia Ganesh queues the GL drawing operations instead of sending them to the GPU right away. When the surface is flushed for whatever reason, the queued GL drawing operations are then processed and sent to the GPU. This allows Skia to apply nice optimizations like merging several tasks and reducing the amount of draw operations we end up sending to the GPU. In those tests where a lot of layers are created and painted to the compositor Skia surface the internal command queue ends up being huge too. Processing and analyzing such a long queue to optimize what we send to the GPU required more CPU work than what we save by optimizing the GL draw operations. Skia provides an API that allows us to do the batching ourselves. Since the compositor already has information to decide what operations could be merged together, we could reduce the internal queue size in many cases. We can merge SkCanvas::drawImageRect() operations as long as they share the same color filter, blend modes and sampling options. In the best case scenario we could reduce the whole internal queue to just one operation. This time the change improved the results of those tests getting them to about 93% of the TextureMapper score, but still a bit behind.
Promise images
The Skia Ganesh backend requires that an SkImage backed by a texture is created for the current thread GrDirectContext, even if it’s borrowing an existing texture. In WebKit all textures are created with a sharing GL context so that they can be accessed and destroyed from different threads with the same sharing GL context. So, for a layer whose content is an image we had to create a texture in the compositing thread to upload the pixels if the image was not accelerated, or for accelerated images get the texture identifier of the image, and then create another SkImage from the compositing thread borrowing the texture for the current GrDirectContext. The Skia Ganesh backend provides an API to create promise images, which can be created from any thread but targeting a specific thread, providing a fulfill callback that will be called on the target thread when the SkImage is first used to retrieve the wrapped texture. This way we can create the SkImage from the main thread for the compositing thread without using OpenGL at creation time. For non-accelerated images we realized we don’t need to manually create the texture and upload the pixels in the compositor, we can just pass the unaccelerated SkImage to the compositor SkCanvas and Skia will handle it internally much more efficiently than we did. And this change improved those compositing tests much further than we expected. The reason turned out to be the batching from the previous section: Skia merges the entries of an image set by comparing texture proxy pointers, and until now we were wrapping the texture in a new SkImage on every frame for every layer, so hundreds of layers drawing the very same image produced hundreds of different proxies that Skia could not merge. Passing the same SkImage every time collapses all of them into a single draw operation, which is the best case we described above. With batched painting and promise images together we could beat TextureMapper significantly.
MotionMark composition suite, leaves subtests, WPE with GPU rendering. Score per revision; higher is better. The same two steps appear with CPU rendering.
Deferred Display Lists (DDL)
When we switched to Skia for painting, we kept the threaded rendering model, just using a separate smaller queue for GPU rendering workers. The GPU workers created their own GrDirectContext to paint the layer tiles. The resulting textures were re-wrapped in the compositing thread for the compositor GrDirectContext using fences for the proper synchronization. We knew this was not the recommended way to use Skia Ganesh from multiple threads, but with TextureMapper we had no other option. However, with the Skia compositor we can do it the recommended way by using a single GrDirectContext in the compositing thread and use Deferred Display Lists (DDL) and promise images to paint the tiles. With DDL, GPU workers no longer use GL at all and they don’t need a GrDirectContext, they paint tiles into a display list that records the GL drawing operations, but without touching GL. For image drawing operations recorded into the DDL, promise images are used too. Since this is now all CPU work we can remove the smaller GPU worker queue and use a single queue with more workers. The compositor replays the DDL into an SkSurface that is then passed to the compositor SkCanvas.
This change fixed rendering glitches on Android and was performance neutral for the whole composition suite and for most of the MotionMark tests, but in MotionMark 1.3 at 15fps it cost 29% in Suits and 14% in Leaves, while improving Images by 9%. Correctness and the other benefits of DDL made us accept those regressions.
MotionMark 1.3 at 15fps, suits subtest, WPE with GPU rendering on a Raspberry Pi 4. Shaded regions are where deferred display lists were enabled by default. CPU rendering moves less than 1% at all three switches, since it has no GPU worker threads for DDL to change.
Damage
TextureMapper already supported using damage information to optimize the painting while compositing, but it has always been disabled at run time because there were issues we never managed to fix. With the Skia compositor we decided to start from scratch and properly handle the damage information while compositing to render only the parts of the frame that actually changed. I’m not going to go into detail here because Nikolas Zimmermann has written an amazing blog post about it with all the details.
Current situation
The Skia compositor is finished and enabled by default in 2.54. Even though it was not the main goal, it performs better than TextureMapper in most of the benchmarks we run: the composition suite we added is 45% faster, and MotionMark 1.3.1 is 35% faster. The exception is MotionMark 1.3 at 15fps with GPU rendering, which comes out flat, because the Suits and Leaves tests are still about 26% and 10% behind due to the deferred display lists trade-off described above.
We are already working on fixing existing issues in composition that we never fixed in TextureMapper. In the main branch TextureMapper is now disabled by default at build time, and support will be removed soon for the GTK and WPE ports. In 2.54 it’s still a run-time decision so if you find any issue with 2.54, you can check if it’s a Skia compositor regression by trying TextureMapper with WEBKIT_USE_SKIA_FOR_COMPOSITION=0 environment variable.
Overall (geometric mean) score, WPE on a Raspberry Pi 4: 320000@main and later against the TextureMapper baseline at 312400-313296@main. Bars start at the baseline. Part of the gain in the MotionMark suites is Skia rendering work rather than the compositor.
WPE on a Raspberry Pi 4, change from the TextureMapper baseline (312400-313296@main) to 320000@main and later. Suits and leaves are the deferred display lists trade-off, not the compositor switch, which was neutral in this suite.
Future plans
We are already working on further improvements like using promise images for all external textures we have to pass to the compositor. We will explore the possibility of using Vulkan with the Ganesh backend instead of GL and eventually try the new Graphite backend. And of course we will continue fixing any existing issues related to the compositor.
