Gom
is a very old object mapper I wrote to bridge
GObject
to
SQLite
. It made a lot of assumptions about the world based on when it was prototyped.
The past couple years had me using it again for the documentation search in
Manuals
. Typically, I would have just built Manuals to parse all the XML files on disk and hold them in memory. That’s how both Devhelp and Builder always did things. Once we started supporting Flatpak SDKs that was no longer realistic. You could have numerous SDKs all with copies of the overlapping data and it just became easier to have a query model.
One of the more performance critical limitations was the locking model. When
gom-1.0
was written, it was not common for distributions to compile SQLite with locking support. So you just created a single thread and did your work over there.
Bolting fulltext search and many other missing features onto the old ABI just wasn’t realistic. Especially when I’ve wanted to make the thing properly async for years. One of my other projects,
Libdex
is just right over there and perfect for this sort of problem.
The landscape changed and so do our horizons.
A new informed ABI
In the years after Gom was prototyped, I worked at a commercial database company and learned a great deal about implementing the internals of both that database and more traditional RDBMS. That left a certain cringe on my mouth whenever looking at my code predating it. Knowing how things get done inside the database allows for building better APIs to interact with it.
This time everything is async. Queries are modeled like you do with a compiler. Lowered into the back-end specific implementation. There can be an entity map and real transactions which allows you to read back the same instance despite which query inflated it.
The Center
Your early stage objects are the
GomRepository
,
GomDriver
, and
GomRegistry
.
The registry describes the entities that can exist within the repository. This is handy because it allows us to pre-compile information into a model that is both immutable and fast at runtime. Compare that to methods like
g_object_class_list_properties()
which is a performance bottleneck of its own.
The driver is very obvious. It is our abstraction layer for the database engines. Currently we have support for
SQLite
and
PostgreSQL
.
The repository is the center of the center. It is how you query, insert, update, delete, transact, and more. It is likely your application instance owns one of these unless of course you use Gom as your file format in which case you’ll have one per “document”.
Two Access Models
This new version of Gom can support either the entity mapping you’re used to or; optionally, raw access to relations/projections via the
GomCursor
.
As the cursor moves through the resulting rows you will have access to all the projections requested in the query. Though it holds enough information to allow you to
gom_cursor_materialize()
the row into an
GomEntity
subclass.
If you want a snapshot of that cursor row without materializing, you can use
GomRecord
which can also conveniently be used in
GListModel
for integration into GTK applications.
Most of the time, you’ll use materialization. And even then it is likely to happen through automated collections rather than with a cursor directly. More on that later.
Sessions
As I mentioned, there was no concept of transactions previously.
In this iteration we have
GomSession
. It is your standard identity-map layer with transaction-scoping. If you perform multiple queries for the same record, the session will ensure you get the same instance back. hat is essential when you do local mutations on an instance and what to see that reflected in followup queries.
Additionally, it makes it nice to have multiple views of an object with an editor or listview and needing them to stay in sync.
Relationship Modeling
Support for relationships was adhoc previously. We had some functions named in ways that made you think you could, but I assure you, they were not well tested.
This time around you can model your
GomEntity
with 1:1, 1:M, M:M, inverse, self-referencing, all while handling proper delete rules. Combing this with the session support mentioned previously is crucial.
So now you should be able to show related models easily in
GtkListView
while keeping the paginated-and-lazy model beneath it transactional.
Migrations
In the previous version migrations were dynamic, but largely controlled by Gom itself. Very inflexible.
This time around we have things broken down into
Migrator
and
Migration
.
You can use built-in implementations like the
EntityMigrator
or implement your own.
CustomMigrator
makes that easy. Especially since you can inject your own migrations at just the right point.
Internally, libgom-2 can snapshot your
GomRegistry
at specific versions based on the provided metadata. Then it performs a diff between two versions of the registry to determine what migration work must be done.
You can just as easily use a
SqlMigration
with custom SQL scripts. This stuff is all highly composable now to get exactly what you need.
Live List Models
I’ve written many ways to get live SQLite results into GTK over the past two decades. I think one of the first was a GtkTreeModel implementation for GTK 2 which could do it. With that in mind, it was still rather annoying when making Manuals so I set off to make that convenient.
We have
GomRecordListModel
,
GomEntityListModel
,
GomRelatedListModel
,
GomQueryModel
all of which have practical uses based on application needs.
But in short, most of those are lazy and support transaction-backed stable identities for entities. Very useful when you have a list of items and an editor loaded in another frame, both of which must reflect the same data.
Expression Trees
This time around I implemented proper expression trees. They model the query, relations, and projections in a manner that allows the driver to lower into a query much more accurately.
You can model things like function-calls cleanly all of which required writing manual SQL before. If you did anything outside of what gom could generate previously, it became madness to maintain.
Vectors
This version of libgom embeds the
vec1
extension for SQLite. That means we can store vectors in your records and query them.
GomVector
makes that easier to manage as a property within your application entity.
I can think of a few things this will be useful for, maybe
you can too
.
Profiling
This version of libgom has profiling support with another project of mine,
Sysprof
. The whole library emits profiler marks about what is going on so that it is easy for you to figure out why something might be slow in your application.
Since we’ve already done the integration of Sysprof into GLib/GObject, GTK, Pango, Libdex, and GNOME Shell/Mutter you can very quickly get an idea with details of what is going on in your application. Click record, select the problem area, zoom, and it is often pretty clear. You can have flamegraphs, callgraphs, and timing marks all in one place.
Local First with Sync Coordination
One of my personal motivations for this is around building a native sync protocol for applications I’m building. I wrote numerous SQLite-based sync protocols for the now defuct
catch.com
before they were acquired by apple. That means I know multiple wrong ways to do it.
This time around, I want to put it right in the data-mapper at the point where you have the most insight. So libgom has the right abstractions in place to build that. The
GomSyncCoordinator
manages the process and
GomSyncTransport
is the abstraction-point for service integration.
You work with
GomDelta
at this layer. The application can provide you with a
GomMergePolicy
to help make decisions which allow for contextually doing the right thing.
This part is still very new. I’m still building the other side of it but landing the shape early allows me to mock and test things comprehensively before committing to the ABI.
My goal is building a practical, robust, and correct implementation for personal local first features.
A small personal note:
as I wrote in
my recent update from France
, I am no longer employed by Red Hat. Work like this is currently self-funded, out of pocket, while my family and I settle into a new chapter. If you find it useful, a note of encouragement or a contribution means a lot right now. It helps make it possible to keep improving the free software infrastructure many of us rely on.
