The C++ standard library (also know as the STL) is, without a doubt, an astounding piece of work. Its scope, performance and incredible backwards compatibility have taken decades of work by many of the world's best programmers. My hat's off to all those people who have contributed to it.
All of that is not to say that it is not without its problems. The biggest one being the absolutely abysmal compile times but unreadability, and certain unoptimalities caused by strict backwards compatibility are also at the top of the list. In fact, it could be argued that most of the things people really dislike about C++ are features of the STL rather than the language itself. Fortunately, using the STL is not mandatory. If you are crazy enough, you can disable it completely and build your own standard library in the best Bender style.
One of the main advantages of being an unemployed-by-choice open source developer is that you can do all of that if you wish. There are no incompetent middle damagers hovering over your shoulder to ensure you are "producing immediate customer value" rather than "wasting time on useless polishing that does not produce immediate customer value".
It's
my
time, and I'll waste it if I want to!
What's in it?
The biggest design questions of a standard library are scope and the "feel" of the API. Rather than spending time on design, we steal it. Thus, when in doubt, read the Python stdlib documentation and replicate it. Thus the name of the library is
pystd
.
The test app
To keep the scope meaningful, we start by writing only enough of stdlib to build an app that reads a text file, validates it as UTF-8, splits the contents into words, counts how many time each word appears in the file and prints all words and how many times it appears sorted by decreasing count.
This requires, at least:
-
File handling
-
Strings
-
UTF8 validation
-
A hash map
-
A vector
-
Sorting
The training wheels come off
The code is available in
this Github repo
for those who want to follow along at home.
Disabling the STL is fairly easy (with Linux+GCC at least) and requires only these two Meson statements:
add_global_arguments('-nostdinc++', language: 'cpp')
add_global_link_arguments('-nostdlib++', '-lsupc++', language: 'cpp')
The
supc++
library is (according to stackoverflow) a support library GCC needs to implement core language features. Now the stdlib is off and it is time to implement everything with sticks, stones and duct tape.
The outcome
Once you have implemented everything discussed above and auxiliary stuff like a hashing framework the main application looks like this.
The end result is both Valgrind and Asan clean. There is one chunk of unreleased memory, but that comes from
supc++
. There is probably UB in the implementation. But it should be the good kind of UB that, if it would actually not work, would break the entire Linux userspace because
everything
depends on it working "as expected".
All of this took fewer than 1000 lines of code in the library itself (including a regex implementation that is not actually used). For comparison merely including
vector
from the STL brings in 27 thousand lines of code.
Comparison to an STL version
Converting this code to use the STL is fairly simple and only requires changing some types and fine tuning the API. The main difference is that the STL version does not validate that the input is UTF-8 as there is no builtin function for that. Now we can compare the two.
Runtime for both is 0.001 to 0.002 seconds on the small test file I used. Pystd is not noticeably slower than the STL version, which is enough for our purposes. It almost certainly scales worse because there has been zero performance work on it.
Compiling the pystd version with
-O2
takes 0.3 seconds whereas the STL version takes 1.2 seconds. The measurements were done on a Ryzen 7 3700X processor.
The executable's unstripped size is 349k for STL and 309k for pystd. The stripped sizes are 23k and 135k. Approximately 100 k of the pystd executable comes from
supc++
. In the STL version that probably comes dynamically from
libstdc++
(which, on this machine, takes 2.5 MB).
Perfect ABI stability
Designing a standard library is exceedingly difficult because you can't ever really change it. Someone, somewhere, is depending on every misfeature in it so they can never be changed.
Pystd has been designed to both support perfect ABI stability
and
make it possible to change it in arbitrary ways in the future. If you start from scratch this turned out to be fairly simple.
The sample code above used the
pystd
namespace. It does not actually exist. Instead it is defined like this in the cpp file:
#include <pystd2025.hpp>
namespace pystd = pystd2025;
In pystd all code is in a namespace with a year and is stored in a header file with the same year. The idea is, then, that every year you create a new release. This involves copying all stdlib header files to a file with the new year and regexping the namespace declarations to match. The old code is now frozen forever (except for bug fixes) whereas the new code can be changed at will because there are
zero existing lines of code that depend on it
.
End users now have the choice of when to update their code to use newer pystd versions. Even better, if there is an old library that can not be updated, any of the old versions can be used in parallel. For example:
pystd2030::SomeType foo;
pystd2025::SomeType bar(foo.something(), foo.something_else());
Thus if no code is ever updated, everything keeps working. If all code is updated at once, everything works. If only parts of the code are updated, things can still be made to work with some glue code. This puts the maintenance burden on the people whose projects can not be updated as opposed to every other developer in the world. This is as it should be, and also would motivate people with broken deps to spend some more effort to get them fixed.
chevron_right
Select a mark to filter by in the Marks view
Samples will be filtered by that mark
The GNOME release team
