-
chevron_right
Michael Calabrese: Pitivi Timeline Ruler
news.movim.eu / PlanetGnome • 0:00 • 8 minutes
Overview
My project was to write the Pitivi Timeline Ruler in Rust using GTK4 and create GObject bindings for it. The project goals overall went well, and I was able to complete the widget successfully along with adding a layout manager structure to the ruler for child widgets to be added from the Pitivi/Python side. My standalone repository contains a working Python demo to verify successful FFI functionality.
Integration into the Pitivi application is not complete. The GTK4 port branch is still a work in progress and is not quite ready yet for deployment.
My commit adding the ruler to the WIP GTK4 branch can be found here .
My repository containing all of my commits and the standalone widget with demos can be found here
Plan Going Forward
I will stay actively involved in the GTK4 porting effort, apply for GNOME membership and stay involved in the Pitivi project. The GNOME community has been very welcoming, and I plan on continuing to contribute both to Pitivi and the broader GNOME ecosystem for the foreseeable future.
Design
General Architecture
A major structural change that has been made after mentor feedback was to move application orchestration out of the widget itself, and keep more of the logic on the app side. The design is a "dumb widget, smart app" framework, where the ruler does not own editor policy.
The app provides the logic for how to handle gestures and what to do when the user interacts with the widget. One example of this is that
project_duration
was removed as a property entirely, and logic around bounds is now handled entirely on the app side. This allows the widget to be used in a variety of contexts, and allows the app to handle bounds in whatever way is appropriate for the context.
The rendering code also received a fairly large cleanup. Previously, the widget stored several pieces of drawing state separately, including adjustments, cached Pango layouts, and font descriptions. These have now been grouped into a single
DrawingState
struct behind one
RefCell
. This made ownership easier to follow, and allows the code to release it's mutable state borrow before connecting or disconnecting signals.
I also introduced a
labels_dirty
flag so timeline labels are only recalculated when they're actually needed during the snapshot phase, rather than repopulating the cache with every setter or adjustment call.
The widget follows font and color settings from the users GTK4 theme, allowing user theme changes seamlessly.
Time Markers
The ruler utilizes a
BTreeMap
to cache the text markers that are visible at the current window size plus a half a screen size buffer over either edge. I used a
BTreeMap
because it offers high efficiency insertion, deletion, and lookup for this case where our markers are sorted.
The ruler recalculates the spacing needed between labels by calculating an intentionally wide timecode using the current font and some padding. This measurement is cached until a change in frame-rate or font invalidates it.
Labels include frames when zoomed in below 1 second, and drop the frame count when zoomed out past that point.
Major Ticks and Intervals
Major divisions are calculated using a two-mode strategy. First, the pixel density is calculated to determine whether a one-second interval can meet the minimum label spacing. If so, frame-aligned intervals are used, with the smallest interval meeting the minimum label spacing selected.
If one second is too narrow for a label, frame alignment is abandoned in favor of whole-second intervals, selecting the smallest interval that meets the minimum label spacing from a pre-defined list. This list can be tuned later to meet Pitivi's UI needs.
Minor Ticks
After discussing ticks with some video editors, I realized that frame alignment is more important to the video editing community than clean time divisions. Because of this, I made the decision to implement a reverse modulo loop to consider possible intervals from largest to smallest that accurately divide frames. The resulting spacing is also checked against the minimum tick spacing to make sure we don't end up with a block of ticks that are too close together. The result is that our minor ticks are asymmetrical and are not even as the user zooms in and out, but they do always accurately divide by frames in the major interval.
Layout Manager & Layout Child
We needed some mechanism to position arbitrary external widgets (play-head, markers, loop-brackets) on the timeline. The timeline uses nanosecond timestamps, so the parent ruler determines x-cords for the children based on the zoom and horizontal scrolling position.
I introduced two objects (public C-wrapper and Engine):
-
PitiviTimelineLayoutChild
-
Subclasses
gtk::LayoutChild. - Metadata wrapper for generic widgets dropped onto the timeline.
-
Adds a custom GObject property for
timestamp.
-
Subclasses
-
PitiviTimelineLayoutManager
-
Subclasses
gtk::LayoutManagerand is installed onPitiviTimelineRuler. - Creates the custom layout child for each child widget.
-
overrides
measure()andallocate().
-
Subclasses
During allocation, the manager reads the ruler's
ns_per_pixel
, horizontal adjustment, and the child's time stamp to determine the child's x-coordinate. The child is centered on it's time stamp and scrolls with the ruler. The Python demo shows an example using
ruler.add_marker(playhead, 0)
to add a play-head at the start of the timeline.
Bindings and Build
The FFI layer in
ruler.h
and
capi.rs
expose the following small API:
-
pitivi_timeline_ruler_new()creates a ruler as aGtkWidget. -
pitivi_timeline_ruler_clocktime_from_pos()converts an x coordinate to a nanosecond time stamp. -
pitivi_timeline_ruler_add_marker()adds a child widget at a time stamp and returns a layout-child object. - The properties of the ruler are exposed as GObject properties, and can be set and retrieved using standard GObject property accessors.
The Process
Challenges
The FFI bindings were a major challenge for me. I ran into significant challenges fixing bugs and understanding conceptually how the Rust bindings actually worked. My initial FFI attempt did compile, but I had to work through significant GTK initialization and headless CI issues. The fixes show up in my commit history as changes of a couple of small lines of code, but the time spent understanding the issues was significant.
I also came into this with very limited knowledge about Flatpak and Meson. Getting myself to a point where I understood what the build systems were doing took a significant amount of effort. I think I spent about the same amount of time reading about GIR, GObject, Flatpak and Meson as I did writing the code. For a very small percentage of the actual code, those tools required the most attention. I view this learning as really valuable for future work in the GNOME space, and I tried to take as much time as I could afford to do my best to genuinely deepen my understanding.
The rendering logic, while similar to previous GTK4 widgets I had built, ended up having numerous rounds of refactoring and learning. I wrote about 4 different strategies for scaling, multiple designs for splitting time and frames and multiple minor tick rules. Even once I was settled on a design, I had multiple waves of finding cache inefficiencies, clearing stale entries, and removing precision and allocation bugs. The ruler is visually small, but the underlying math and rendering logic took quite a bit of work to really get to a professional standard. I would not be surprised if the logic changes again in the future as I continue to work on the Pitivi GTK4 port and get feedback from the community.
Major Milestones
-
May 3: Built the initial window render as a standalone GTK4 application.
-
May 8: Added the initial GObject getter/setter structure, a
GtkScrolledWindowtest, and zoom bounds. -
May 9-13: Drew the initial major and minor ticks, changed the scale to nanoseconds per pixel, and added Pango labels to test scale behavior.
-
May 14-28: Reworked time stamp math, wrote code to extract frame-rate from video's GES timeline which was later scrapped, and refactored rendering of ticks to draw a single interval and then paint it repeatedly across the ruler.
-
May 30-June 4: Wrote the dynamic label-width measurement, minimum tick spacing, and frame-oriented major/minor interval selection structure. Added
gtk::Scrollableinterface to the ruler. -
June 6-26: Refined cache eviction for scrolling. I also addressed zoom drift, font and DPI changes, adjustment-signal cleanup, and precision around the playhead coordinates. I wrote the unit tests for subdivision and timecode math in this period as well.
-
June 30-July 3: Removed redundant APIs, moved gesture handling to the app side, refactored types throughout the codebase, and removed the widget's project-state ownership.
-
July 5-25: Created the initial FFI bindings and the layout manager and layout child, and then attended GUADEC in A Coruña, Spain. I managed to resolve the GTK initialization CI issues and successfully exposed the ruler and layout child.
-
July 26-29: Added dirty-label cache invalidation and consolidated the rendering state into
DrawingStatestruct. These optimizations simplified borrow management and reduced thrashing the Pango cache. -
August 1-12 - Added the Python demo to
python/test.py, fixed allocation updates for children widgets near the start of the ruler during zoom changes, added Meson build, added the Meson test target that runs the Rust tests. -
August 12-Present: I am currently battling through adding my ruler to the Pitivi GTK4 port branch.
Special Thanks
I would like to send a massive thank you to my mentors, Yatin and Alex Băluț. I was going pretty significantly off track a few times throughout the project and I got nudged in the right direction at some critical moments.
I would also like to thank the GNOME travel committee. Getting to attend GUADEC was a really incredible experience. Sergey Bugaev took a lot of time to sit and work through some of my bugs with me and help provide some guidance. As a long time GNOME daily user, getting to spend time and meet maintainers and Federico was a really exciting opportunity.
GSoC has been a great experience, and I am very grateful for the opportunity to work on this project. I am looking forward to continuing to contribute to the GNOME ecosystem.
. Kudos to Adrian and everyone else who’s helping to push this forward!