No sample library — just equations, PCM, and a small OS-specific playback bridge.

Early game consoles had little memory or storage to spare for recorded sound. Their music relied on a small number of voices and a limited vocabulary: simple waveforms, noise, and envelopes. Composers turned those restrictions into instantly recognizable melodies.

Chiptune grew out of these constraints. The term covers a broad family of music and music-making practices rooted in programmable sound chips.

It is often called “8-bit music,” although the label is imprecise: not every chiptune machine was 8-bit, and not every modern track with a retro timbre runs on vintage hardware. The important idea for us is the working method. Instead of asking an audio player to reproduce a recorded instrument, we describe a small signal generator and tell it which frequencies to produce over time. The hardware limitations helped shape the art; today we can choose similar limitations because they are fun.

This project builds a chiptune-inspired synthesizer directly in Erlang. It is not an emulator for a particular console or sound chip. It borrows the useful ingredients—elementary waveforms, noise, short envelopes, and a small set of instruments—and renders them as ordinary digital audio. A bass note is not a WAV file. A kick drum is not a hidden sample. Every instrument is a mathematical function evaluated while the song plays.

The project was inspired in part by the 2020 Erlang Solutions article “The sound of Erlang: How to use Erlang as an instrument”. That article develops sound from first principles, writes raw audio to a file, and plays it with ffplay. We will keep the first-principles spirit but take a different route: multiple simultaneous tracks, several synthesized instruments, chunked rendering, and live playback through the operating system’s audio API.

By the end, you will know how a value such as a4 becomes 440 Hz, how Erlang turns that frequency into signed 16-bit samples, and how those samples reach the speakers.

First, hear the small hand-written demo we will use for the walkthrough. This clip records the synthesizer’s output:

You need Erlang/OTP on PATH, including erl.exe and escript.exe. The playback bridge on Windows uses the powershell.exe and built-in C# compiler available on modern Windows. No FFmpeg, external synthesizer, or audio library is required. Open PowerShell in the repository root and play the small hand-written demo:

.\run-windows.bat demo_song

On macOS, Erlang’s erl and escript commands must be on PATH. The launcher also needs the Apple Command Line Tools to compile the bundled Core Audio bridge. If they are missing, install them with xcode-select --install. Run the equivalent command in Terminal:

./run-macos.sh demo_song

Both launchers compile the application, start a non-interactive Erlang VM, synthesize the song from its Erlang data, and stream it to the default audio output device. They accept any compiled song module with the interface we will examine below.

Before isolating tracks, ask the module what it contains:

On Windows:

.\run-windows.bat demo_song tracks

On macOS:

./run-macos.sh demo_song tracks

The output will be:

[{1, kick}, {2, snare}, {3, closed_hat}, {4, bass}, {5, pad}, {6, lead}]

Now we can listen to the lead by itself or remove the percussion from the mix:

On Windows:

.\run-windows.bat demo_song solo 6
.\run-windows.bat demo_song mute 1,2,3

On macOS:

./run-macos.sh demo_song solo 6
./run-macos.sh demo_song mute 1,2,3

These controls are intentionally small. They are enough to make the synthesizer explorable without turning a blog-sized project into a workstation.

Map of the Project

The playback path is compact enough to follow from beginning to end:

There are three useful boundaries here. Song modules describe what to play. music_synth calculates what the waveform is. The player and its platform bridge decide how bytes reach an audio device.

A Song Is Erlang Data

A song module exports just two functions. This excerpt shows the tempo and percussion tracks from demo_song:

-module(demo_song).
-export([bpm/0, tracks/0]).

bpm() -> 120.

tracks() ->
  [{kick, lists:seq(0, 30, 2)},
   {snare, lists:seq(1, 31, 2)},
   {closed_hat, [B / 2 || B <- lists:seq(0, 63)]},
   ...].

Try it on Windows or macOS

“`

Each track is {Preset, Events}. A percussion event is simply a beat number. At 120 beats per minute, the kick events above land at beats 0, 2, 4, and so on, while the snare occupies the alternating beats. The closed hat list uses half-beats, which creates the faster pulse running across the demo.

Pitched instruments use {Beat, Duration, Notes}:

{bass, [{0, 0.45, c2}, {1, 0.45, c2}, {2, 0.45, g2}]}

{pad, [{0, 4, {c4, e4, g4}},
       {4, 4, {a3, c4, e4}}]}

“`

Notes may be one note atom or a non-empty tuple. A tuple schedules its notes together, giving us a chord without a second data structure. Both Beat and Duration are measured in beats and can have whole or fractional values.

The generated song_01_slay_the_evil through song_18_infinite_darkness modules follow exactly the same contract. Their scores are much larger, but they are still ordinary data returned by functions.

The public music module keeps callers away from the internal details:

music:play(demo_song).
music:play(demo_song, {solo, 6}).
music:play(demo_song, {mute, [1, 2, 3]}).
music:tracks(demo_song).

“`

Track selection happens before synthesis. music_synth:prepare/2 either keeps all tracks, selects one numbered track, or removes the requested track numbers. It then expands the remaining events into voices: one for each sounding note or percussion hit. A chord creates several voices with the same start time.

Walking Through the Player

music_player.erl is intentionally small. Its two public functions first ask the synthesizer to prepare the whole score, then pass the resulting state to open/1:

play(Song) ->
    open(music_synth:prepare(Song)).

play(Song, Selection) ->
    case music_synth:prepare(Song, Selection) of
        {error, _} = Error -> Error;
        Synth -> open(Synth)
    end.

“`

Once prepared, open/1 selects the platform bridge and starts playback. The bridge requests chunks as its audio buffers become available. Sample indexes determine when notes belong in the score; the audio device paces those samples in real time. We will follow the synthesis steps first, then return to the bridge protocol.

From a Note Name to a Frequency

The note atom a4 becomes MIDI note 69, which corresponds to 440 Hz. In twelve-tone equal temperament, moving one semitone multiplies a frequency by the twelfth root of two. Using A4 as the reference, any MIDI note number Midi can be converted with:

frequency = 440 × 2^((Midi - 69) / 12)

“`

The synthesizer parses atoms such as c4, fs4, and a4, converts the letter, optional s for sharp, and octave into a MIDI number, and applies that formula:

Midi = (Octave + 1) * 12 + Base + Sharp,
440.0 * math:pow(2.0, (Midi - 69) / 12).

“`

For example, c4 becomes MIDI note 60 and approximately 261.63 Hz.

To turn that frequency into audio, we need samples. Sound is changing air pressure; digital audio represents that change as a sequence of numbers. At 44,100 samples per second, sample index SampleI occurs at time T in seconds:

T = SampleI / 44100

“`

A sine wave of frequency F can be sampled with:

sin(2 × pi × F × T)

“`

The result moves between -1 and 1. Repeating the calculation at consecutive values of T produces the waveform. Higher frequencies complete more cycles per second and sound higher.

A note becomes audio data. Every point in the final waveform is calculated for its position in time.

Instruments Are Functions

The center of the project is the tone/6 family in music_synth.erl. Its clauses are our instruments. Each receives the time since the voice began, its gate duration, its frequency, and where useful the absolute sample index and a deterministic seed. The gate duration is the time the note is held before its release begins; both it and the elapsed time are measured in seconds here.

The melodic presets start with elementary periodic waveforms:

tone(bass, T, G, F, _, _) ->
  0.32 * square(?PI2 * F * T) * envelope(T, G, 0.005, 0.04);
tone(lead, T, G, F, _, _) ->
  0.22 * saw(?PI2 * F * T) * envelope(T, G, 0.01, 0.12);
tone(pad, T, G, F, _, _) ->
  0.12 * triangle(?PI2 * F * T) * envelope(T, G, 0.18, 0.50).

“`

The ?PI2 macro represents 2 × pi. A square wave flips between two levels and gives the bass a hollow, buzzy sound. A sawtooth ramps and jumps, producing the brighter lead. A triangle wave changes more gently and suits the softer pad.

The piano, organ, and strings presets combine basic waveforms. “Piano” adds sine waves at the fundamental, twice the frequency, and three times the frequency, then applies exponential decay. “Organ” uses a related harmonic mixture without the same decay. Strings combine a saw and a slightly detuned triangle. These are impressions, not physical models of real instruments, and their simplicity is part of the chiptune character.

For percussion, the kick feeds a rapidly falling phase into sin and damps it with exp(-9 × T). The tom uses a gentler downward sweep. The snare adds a short 180 Hz body to deterministic pseudo-random noise; the closed hat is an even shorter burst of noise. The cymbal combines noise with a high square wave. A handful of arithmetic expressions becomes a recognizable drum kit.

One more function keeps notes from clicking abruptly at their edges:

envelope(T, _Gate, Attack, _Release) when T < Attack -> T / Attack;
envelope(T, Gate, _Attack, _Release) when T < Gate -> 1.0;
envelope(T, Gate, _Attack, Release) ->
  max(0.0, 1.0 - (T - Gate) / Release).

“`

This is a compact attack–sustain–release envelope. During attack, amplitude rises from zero. It remains at full level while the note is gated, then falls after release begins. Different attack and release values make the same oscillator feel percussive, plucked, or slow and pad-like.

Scheduling, Mixing, and PCM

Tempo determines when each voice begins. beat_sample/2 converts a beat position to an absolute sample index:

beat_sample(Beat, Bpm) -> round(Beat * 60 * ?RATE / Bpm).

“`

Here ?RATE is 44,100. At 120 BPM, one beat lasts half a second, or 22,050 samples; beat 4 begins at sample 88,200.

A pitched event records three positions: Start, the first sample of the note; Gate, the end of its written duration; and End, the end of its release tail. The renderer advances a sample cursor instead of calling timer:sleep/1 for each note. This keeps note positions fixed even if rendering pauses briefly, although playback still needs a steady supply of audio to avoid gaps.

Multiple tracks do not require separate Erlang processes. prepare_tracks/2 gathers all selected tracks into one list of voices. Notes and chords can overlap because voices whose sample ranges intersect are active together.

Preparation turns events into voice tuples containing the preset, start, gate, end, frequency, and seed. Percussion receives a frequency of zero because its formula does not need the pitch argument. Every preset also has a short tail, allowing its release or decay to continue after the gate closes.

The renderer does not allocate the whole song as one giant list. It works in chunks of 4,410 samples—one tenth of a second at 44.1 kHz:

Count = min(?CHUNK, Total - Pos),
Last = Pos + Count,
Active = [V || V = {_, Start, _, End, _, _} <- Voices,
          Start < Last, End > Pos].

“`

Only voices overlapping the current chunk are considered. For each sample index, the renderer evaluates those voices and sums their amplitudes. The sum is scaled and clamped to the valid range from -1 to 1. Clamping prevents an overflowing mix from wrapping into a radically different value, although a heavily clipped mix can still sound distorted.

Finally, each floating-point sample becomes a signed 16-bit little-endian integer:

pcm(X) -> <<(round(X * 32767)):16/little-signed>>.

“`

The resulting binary is mono linear PCM: 44,100 samples per second, two bytes per sample, 88,200 bytes per second. It contains no file header because we are sending it to a device configured with the matching format, not saving a WAV file.

Playback on Windows and macOS

Erlang can calculate the audio portably, but it still needs an operating-system API to make speakers move. music_player.erl chooses a deliberately small adapter: wave_out.ps1 on Windows or audio_out.c on macOS.

The synthesizer is platform-neutral; this diagram shows the Windows branch of the final bridge.

The player first calls music_synth:prepare/1 or prepare/2. It then checks the operating system and starts a TCP listener bound to 127.0.0.1 on a temporary port. Binding to loopback keeps this private protocol on the local machine.

Next, open_port/2 launches the selected bridge with the temporary port number. Here an Erlang port manages the external process; the PCM itself travels over the loopback TCP socket. The socket uses Erlang’s {packet, 4} mode, so each message receives a four-byte length prefix and arrives as one binary frame.

The PowerShell script uses Add-Type to compile a small embedded C# class. That class connects back to Erlang and opens the default waveform-audio output device through waveOutOpen. Its declared format matches the renderer: PCM, one channel, 44,100 Hz, 16 bits.

On macOS, run-macos.sh compiles the bundled C bridge with Apple Clang into the ignored _build directory. The bridge connects back to Erlang and creates an output queue with Core Audio’s Audio Queue Services. It declares the same mono, 44,100 Hz, signed 16-bit PCM format, so the Erlang renderer sends identical binaries on both systems.

The macOS bridge uses the same PCM format and request protocol as the Windows bridge.

Four native buffers act as playback slots. For every free slot, the bridge sends an R (“ready”) frame. Erlang responds with A followed by one PCM chunk. When the native API makes a buffer reusable, the bridge requests another. This credit-based exchange prevents Erlang from sending audio faster than the fixed buffer pool can accept it.

When the renderer reaches the end, Erlang sends E. The bridge lets all active buffers drain, replies with D, and closes. An X frame carries a bridge error back to Erlang. The protocol is tiny, but it gives the two runtimes explicit flow control and a clean ending. The macOS bridge asks Audio Queue Services to stop non-immediately and waits for its running-state notification before sending D, so buffered audio is not cut off.

The macOS launcher rebuilds the native bridge when its C source changes. The platform boundary remains music_synth:render/1, which returns either {PCM, NextState} or done. A future Linux adapter could consume those same chunks, handle buffering and errors, and drain at the end without changing the score or synthesizer.

MIDI Is an Authoring Tool, Not the Synthesizer

Writing demo_song.erl by hand is useful for learning, but a complete arrangement can contain thousands of note events. import/import.py converts Standard MIDI files into modules with the same bpm/0 and tracks/0 interface.

The larger song_*.erl modules were generated from MIDI files in HydroGene’s free chiptune collection on itch.io, which includes rendered music and its source MIDI files.

Here is the synthesizer playing an imported arrangement, song_01_slay_the_evil. It uses the same instruments as the hand-written demo, with a larger score:

The importer uses only Python’s standard library. It reads MIDI formats 0 and 1, tracks tempo, note-on and note-off events, program changes, chords, and percussion on MIDI channel 10. General MIDI programs are mapped onto the small preset collection rather than reproduced as samples.

You can validate the files in import/input without writing anything:

python3 import/import.py --check

To generate compilable Erlang modules in src, use:

python3 import/import.py --output src

On Windows, use python if that is the name of your Python 3 command.

Existing destinations are skipped unless —overwrite is supplied. Importing is an optional composition workflow; neither Python nor a MIDI parser participates when a song plays. Once generated, a song is simply Erlang source data, and the same mathematical instruments render it.

Equations Become Music

Follow one note and every transformation is visible: a4 becomes MIDI 69, MIDI 69 becomes 440 Hz, 440 Hz drives an oscillator, an envelope shapes its amplitude, voices add together, floats become little-endian integers, and buffered PCM reaches the speakers.

Chiptune began in a world where small sound vocabularies were unavoidable. Recreating that economy in Erlang is a reminder that a musical instrument does not have to be a large framework. Sometimes it can be a few functions, a clock, and 44,100 samples per second.

To hear how much one function matters, change the lead’s saw call to square in music_synth.erl, then run .\run-windows.bat demo_song solo 6 on Windows or ./run-macos.sh demo_song solo 6 on macOS. Keep the notes the same and listen to how the instrument changes. Next, try a longer attack or a shorter release: the score stays familiar while the sound becomes your own.

References and Further Reading

Erlang Solutions, “The sound of Erlang: How to use Erlang as an instrument”, 2020.

Kenneth B. McAlpine, Bits and Pieces: A History of Chiptunes (book), Oxford University Press, 2018.

The post A Live Chiptune Synthesizer in Erlang appeared first on Erlang Solutions.

The Ignite Realtime community is proud to announce that Smack 4.5.0 has been released.

This is a new major release after 4.4 nearly two years ago, with many bug fixes and improvements. Smack’s modular connection architecture is now deemed stable and users of the legacy XMPPTCPConnection should switch over to a modern architecture. The modular connection architecture abstracts the underlying connection mechanism, for example TCP, BOSH, and WebSockets, transporting the raw bytes of the XMPP stream, from the higher level XML layer. This allows, among other things, for a connection to transparently switch between TCP and WebSocket connections.

As always, this new Smack release is available on Maven Central.

1 post - 1 participant

Read full topic

On mobile (and worse), baseline XMPP is a little more painful than it should be. It has a lot of round-trips, where the client sends a request and waits - patiently - for the answer before it can continue.

Baseline XMPP actually has 9 of these before you can send and receive messages, and while Openfire has dropped some of these for a while (we’ve supported Direct TLS , for clients, for ever), others haven’t yet made it here. Silly, as I wrote SASL2 some time ago, which is a framework for amortising some of the start-up requests into the authentication.

Others are Bind2 - which pulls resource binding into SASL2 and also allows multiple other “connection setup” things to work - and FAST , which hands out tokens to allow reauthentication which is, well, FAST. This gets us down to 4 round-trips - halving connection time.

Thanks to a supporting grant from the NLNet Foundation, these are now all in our “main” branch, and undergoing their final testing before we make a release, so will be in our nightly builds from now on. I’ve been lucky enough to spend three weeks doing (sometimes literal!) field testing over slow links, so I’ve seen first hand that fortune really favours the bold with these extensions.

We’ve also improved security, with Channel Bindings and Downgrade Protection - also in the nightlies. We even found another round-trip to save, with Initial Authentication Pipelining .

There is also supporting work to make all this happen - like improvements to message archiving, and improvements to our sister project, the XMPP Interop Testing framework, so we can do live testing. This is based on Smack , so that, too now has SASL2, Bind2, and FAST support in its latest Alpha release, 4.6.0-alpha1.

We’d welcome people trying this code out - it’s a lot of exciting new features, and should be highly beneficial to most users of Openfire, and indeed client developers using Smack. It’s still nightlies, not releases, but we’re keen to move this forward as, erm, FAST as we can.

Yeah, I’m not even sorry.

For other release announcements and news follow us on Mastodon or X

1 post - 1 participant

Read full topic

Earlier today, version 1.8.0 has been released. This release adds tests for exciting new XMPP functionality!

Classic XMPP connection setup takes a lot of round-trips: authenticate, bind a resource, enable carbons, kick off stream management, sync the archive. Each one is its own request. Fine on wifi, annoying on a shaky mobile connection.

XMPP offers various XEPs to fix that. SASL2 (XEP-0388) turns authentication into a single extensible envelope instead of a fixed sequence of steps, and everything else here builds on it. Bind 2 (XEP-0386) folds resource binding and feature enablement into that same envelope, so a client can walk away with a fully set-up session in (almost) one shot. And FAST (XEP-0484) lets a client swap its password for a short-lived, rotating token and reconnect to the server, authenticated, in a single round-trip, no SCRAM handshake required.

Put together, a client can go from opening a socket to fully authenticated and ready to send in essentially one round-trip. That’s also exactly the kind of multi-XEP choreography where servers tend to disagree on the details, so we’re glad to have it covered.

Give the new tests a try, and let us know what you find!

Splash image courtesy of Conny Schneider, Unsplash

XMPP Newsletter Banner

Welcome to the XMPP Newsletter, great to have you here again! This issue covers the month of August 2026.

The XMPP Newsletter is brought to you by the XSF Communication Team and contributors of the XMPP community.

Just like any other product or project by the XSF, the Newsletter is the result of the voluntary work of its members and contributors. If you are happy with the services and software you may be using, please consider saying thanks or help these projects!

Interested in contributing to the XSF Communication Team? Read more at the bottom.

XSF Announcements

XMPP Events

• The di.day takes place every first Sunday of the month where the XMPP community also promotes their solutions! di.day is a mainly German initiative to help people switch to open-source and privacy-friendly solutions.
• There will be an XMPP stand at the LinuxDays conference in Prague during the first weekend of October. You can also vote for XMPP-related lectures during the first week of September.
• XMPP stand at OmniOpenCon in Bucharest, Romania, too. From 16th - 17th October 2026.

Videos and Talks

• Quicksy ist aus den App Stores verschwunden… was nun?, by eversten.net for XMPP Tutorials DE. [DE]
• Von Quicksy zu Conversations (Schnellanleitung), by eversten.net for XMPP Tutorials DE. [DE]

XMPP Articles

• Coming home to Jabber, by TabTwo for the Sprawl blog.
• Admin Benachrichtigungen via XMPP, from Rainer’s Blogseiten. [DE]
• Una alternativa fiable a Whastapp para chatear con privacidad, by David Palacios for Lente Tisquesusa. [ES]
• Teaching an Old Dog New Tricks: Forgejo + XMPP, from マリウス.com.
• Eu quero acreditar, by isadora for the isaCloud diario-de-bordo. [PT_BR]
• Slidge got a security audit, from slidge.im blog
• Jabber/XMPP: 25 Years of Digital Independence, by Daniel Gultsch.
• XMPP ha 25 anni: perché la messaggistica aperta non è mai davvero morta, by Michele Nasi for ilsoftware.it. [IT]
• Libervia on building an all-in-one communication platform grounded in the right to privacy, by Xavier Coadic for Association for Progressive Communications (APC).
• Tubercule, a collective of hackers and hackerettes from the Rennes region (France) that wants to be inclusive and accessible, announced the availability of their newly implemented XMPP instance for those of you in Rennes or around! The announcement came along with a really handy guide (which will evolve over time), that also adds plain and simple instructions on how to get in touch with them and get an account. Have fun! [FR]

Software news

Clients and applications

• aTalk has released version 6.5.0 of its encrypted instant messaging with video call and GPS features for Android. This release brings a lot of improvements, quite a few fixes and some really heavy work ‘under the hood’. Please refer to the release notes for all the details.
• Gajim has released version 2.6.0 of its free and fully featured chat app for XMPP. Gajim now adapts responsively to window size changes. This release comes with many small improvements and bugfixes. Thank you for all your contributions!
• Introducing Livewire: an alpha-quality native XMPP client for Ubuntu Touch, written in Rust on top of tokio-xmpp and xmpp-parsers.
• Introducing Mynah: an XMPP client using Python, GTK, and slixmpp. It has a long-term vision of being maximally customisable, user-friendly, and cross-platform (Linux, Windows, macOS).
• Monocles has released version 2.3 of its chat client for Android. This is a bigger release that brings in a lot of new features like multi file messages, a new global search, group calls (still in beta stage), several UI/UX improvements and some ground breaking ones as Post Quantum OMEMO2, along with updated translations, a whole lot of fixes and more! Note: due to some delays with the Play Store updates, this release is only available through F-droid for the time being.

Servers

• MongooseIM has released versions 6.8.0 and 6.8.1 of their enterprise instant messaging solution. You can read all the details in the changelog.
• The Ignite Realtime community is pleased to announce the release of Openfire 5.1.2, a maintenance update to the open-source XMPP real-time communication server. Head over to the full changelog for all the details!
• The Prosody App for Yunohost has been updated to provide a configuration panel, to give an easy access to the most relevant parameters from the web interface!
• Enthusiasts from New Zealand announced a new local XMPP server for “Kiwis”: xmpp.nz

Libraries & Tools

• Introducing xmpp-translate-bot, a translation bot for XMPP (MUC) that makes use of a self-hosted LibreTranslate instance as a backend for language detection and translation. [PT_BR]
• jabber.el, the XMPP client for Emacs, versions 0.13.0, 0.13.1 and [0.13.2])(https://git.thanosapollo.org/emacs-jabber/tag/?h=0.13.2) have been released. Full details on the changelog.
• libomemo.js, a TypeScript implementation of the OMEMO Multi-End Message and Object Encryption protocol for XMPP, version 3.0.0 has been released. Once again, the list of changes is larger and has more details and notices on breaking changes that what could mention in here. Please refer to the full changelog for all the details.
• matteridge, the Mattermost to XMPP gateway based on Slidge and mattermost-api-reference-client, version 0.4.1 has been released. You can check the changelog from 0.4.0 to 0.4.1 for all the details.
• python-nbxmpp, a Python library that provides a way for Python applications to use the XMPP network, version 7.4.0 has been released. Full details on the changelog.
• slidcord, the Discord to XMPP gateway based on Slidge and discord.py-self, version 0.4.1, has been released. You can check the intermediate changelog from 0.4.0 to 0.4.1 for all the details.
• slidgram, the Telegram to XMPP gateway based on Slidge and Pyrofork, version 0.4.1, has been released. You can check the changelog from 0.4.0 to 0.4.1 for all the details.
• xmpp-dns, a CLI tool to check XMPP SRV records, version 0.6.4 has been released. Full details on the changelog.

Extensions and specifications

The XMPP Standards Foundation develops extensions to XMPP in its XEP series in addition to XMPP RFCs. Developers and other standards experts from around the world collaborate on these extensions, developing new specifications for emerging practices, and refining existing ways of doing things. Proposed by anybody, the particularly successful ones end up as Final or Active - depending on their type - while others are carefully archived as Deferred. This life cycle is described in XEP-0001, which contains the formal and canonical definitions for the types, states, and processes. Read more about the standards process. Communication around Standards and Extensions happens in the Standards Mailing List (online archive).

Proposed

The XEP development process starts by writing up an idea and submitting it to the XMPP Editor. Within two weeks, the Council decides whether to accept this proposal as an Experimental XEP.

• No proposed XEPs this month.

New

• Version 0.1.0 of XEP-0518 (Payment Required)
  • Accepted as Experimental by council vote on 2026-07-07 (XEP Editor(dg))

Deferred

If an experimental XEP is not updated for more than twelve months, it will be moved off Experimental to Deferred. If there is another update, it will put the XEP back onto Experimental.

• No XEPs deferred this month.

Updated

• Version 1.6 of XEP-0066 (Out of Band Data)
  • Twenty years later, strike section 6’s SI suggestion (dwd)
• Version 1.1.5 of XEP-0084 (User Avatar)
  • Fix inconsistent hash in the example with multiple data sources for the same file. (nc)
• Version 1.1.0 of XEP-0490 (Message Displayed Synchronization)
  • Add security consideration for sender verification of PEP notifications. (dg)
• Version 0.1.1 of XEP-0515 (TLS Channel-Binding Downgrade Protection)
  • Fixed typo in Bind 2 namespace (XEP Editor (dg))

Last Call

Last calls are issued once everyone seems satisfied with the current XEP status. After the Council decides whether the XEP seems ready, the XMPP Editor issues a Last Call for comments. The feedback gathered during the Last Call can help improve the XEP before returning it to the Council for advancement to Stable.

• No XEPs last calls this month.

Stable

• No stable XEPs this month.

Deprecated

• No XEPs deprecated this month.

Rejected

• No XEPs rejected this month.

Spread the news

Please share the news on other networks:

• Mastodon
• Movim
• Bluesky
• Reddit
• LinkedIn
• YouTube
• Lemmy instance (unofficial)
• XMPP Facebook page (unofficial)

Also check out our RSS Feed!

Looking for job offers or want to hire a professional consultant for your XMPP project? Visit our XMPP job board.

Newsletter Contributors & Translations

This is a community effort, and we would like to thank translators for their contributions. Volunteers and more languages are welcome! Translations of the XMPP Newsletter will be released here (with some delay):

• Contributors:

  • To this issue: cal0pteryx, emus, Gonzalo Raúl Nemmi, Ludovic Bocquet, poVoq, XSF iTeam

• Translations:

  • French: Adrien Bourmault (neox), alkino, anubis, Arkem, Benoît Sibaud, mathieui, nyco, Pierre Jarillon, Ppjet6, seveso, Ysabeau
  • Italian: Mario Sabatino, Roberto Resoli

Help us to build the newsletter

This XMPP Newsletter is produced collaboratively by the XMPP community. Each month’s newsletter issue is drafted in this simple pad. At the end of each month, the pad’s content is merged into the XSF GitHub repository. We are always happy to welcome contributors. Do not hesitate to join the discussion in our XSF Communications Team group chat (MUC) and thereby help us sustain this as a community effort. You have a project and want to spread the news? Please consider sharing your news or events here, and promote it to a large audience.

Tasks we do on a regular basis:

• gathering news in the XMPP universe
• short summaries of news and events
• summary of the monthly communication on extensions (XEPs)
• review of the newsletter draft
• preparation of media images
• translations
• communication via media accounts

Unsubscribe from the XMPP Newsletter

For this newsletter either log in here and unsubscribe or simply send an email to newsletter-leave@xmpp.org. (If you have not previously logged in, you may need to set up an account with the appropriate email address.)

License

This newsletter is published under CC BY-SA license.

The largest single piece of that work is where you stopped reading, and how Fluux shares that position with your other clients. The rest is spread thin on purpose: badges that would not clear, a "New messages" line that kept coming back, a conversation that moved while you were reading it, failures that never said why.

Where you left off, on every client

Every XMPP client on your account should agree on where you stopped reading. Fluux publishes that position with XEP-0490, and this release closes the cases where the publication used to stall.

• Your read position now leaves the device in the cases where it used to get stuck : after you reply in a one-to-one chat, in a conversation with nothing currently loaded, and after a first attempt that did not go through. Your other clients stop showing a stale "New messages" line and an inflated badge.
• A position arriving from another client is no longer dropped when Fluux cannot yet place it in its own history. It is kept and applied the moment you open the conversation.
• A conversation whose oldest messages have aged out of the server archive no longer freezes your position. Fluux could stop publishing that conversation altogether, which left your other clients months behind.
• A group chat you read elsewhere now shows what your other clients show , instead of sitting at zero unread with no read marker while they display both.

One unread count, everywhere it appears

There used to be more than one way to answer "how many messages are unread here", and the answers could disagree. Now there is one: the messages that sit after your last read position. Every surface that shows a number reads that same number.

• The sidebar badge, the "New messages" divider, the floating pill and the scroll-to-bottom badge always agree. A conversation left in the background no longer under-counts, and opening a conversation no longer zeroes its badge before you have read anything.
• The divider stays where it is while you are looking at it. It no longer reappears every time you reopen a conversation you have already read to the end.
• A badge that reading could not clear is gone. Your read position also stops moving backwards when the last message you read is missing from the loaded history.
• Rooms say more at a glance : a room row now tells you how many messages are unread, and the unread dot and the mention badge moved after the timestamp so they line up down the list.

A conversation that stays where you put it

Scrolling is how you navigate a conversation. It should never happen on its own.

• Closing an image or a dialog with Escape keeps your place. A late-arriving older message no longer pushes a scrolled-up reader backwards in time either.
• Returning to a room lands where you left it. Opening a room whose read position predates the loaded history no longer strands you at the oldest message, and Home works immediately after opening a conversation.
• The view stays on the newest message when a late link preview, reaction or attachment makes the last message taller, and when the composer collapses back to one line after you send.
• The typing indicator no longer covers messages. Its label wraps to a second line instead of being cut off.

When something fails, Fluux tells you

Silence is the worst answer an app can give.

• Older messages that could not be loaded now show a marker, right above the button that tries again.
• A bookmarked room that could not be rejoined says why : the room wants a password, or your nickname is already taken by somebody else in it, or it only admits members. Joining a password-protected room asks for the password wherever you join from, the sidebar, an invitation or Browse Rooms, and remembers it for next time.
• A message that cannot be read explains which problem it hit : a key this device does not have, an invalid signature, or content that could not be parsed, instead of always blaming a missing key.
• Opening a conversation shows a loading indicator and lets you go back while history loads, and an update that has to upgrade your local history shows a progress bar while it does.

Faster catch-up

• Launching after a long absence and opening a conversation are both faster. Fluux writes far less to local storage while it catches up on history, and upgrading local history after an update is faster too, on large archives especially.
• Group chat history no longer arrives in two visible phases on mobile and in the browser. After a reconnect, a joined room no longer sits inert in the sidebar with no preview and no timestamp.
• A busy room you join no longer announces its recent history as new arrivals.

Encryption, desktop, and the rest

• OpenPGP interoperability : keys generated by Fluux can now be imported by Gajim, existing keys are repaired when you unlock them, and Fluux no longer removes your other clients&apos keys from your published key list. Saving an image from the lightbox no longer writes the raw encrypted file to disk, and re-publishing an unchanged key no longer locks encryption behind a warning.
• A system tray on Windows and Linux : a new setting decides whether closing the window quits Fluux or leaves it running in the tray, wherever the desktop provides one. Clicking a notification restores the window and opens the conversation and the message it came from, and on Linux the notification settings button now opens the right panel on Cinnamon, KDE, XFCE, Budgie and LXQt.
• Connections that used to be refused : Fluux now asks your server where to connect before falling back to its own list of known addresses, so an account on a server whose address changed is no longer locked out. A password with accented or non-Latin characters is sent exactly as you typed it, so accounts that were rejected with "invalid username or password" can sign in. If you were affected and typed an address into the advanced server field, clear it once and Fluux will find the server on its own.
• Korean, strikethrough, and the small things : Korean joins the interface languages, contributed by the community. Markdown strikethrough renders alongside the XEP-0393 form. A deleted message stays deleted everywhere Fluux summarises a conversation, relative dates update after midnight even when the app stays open overnight, and copying several messages at once no longer drops polls and attachments.

Get it

Fluux Messenger 0.17.3 is available for macOS, Windows and Linux, or directly in your browser, from the Fluux Messenger page . If you upgrade and something still feels off, tell us: bug reports and feature requests both go to GitHub Issues .

Distributed systems need async message delivery across nodes. Phoenix provides Phoenix PubSub for this, with pluggable adapters for different backends — officially PG2 and Redis.

This post walks through implementing a Phoenix PubSub adapter backed by EventStore , an Elixir event sourcing library that persists events to PostgreSQL as an append-only log.

Using EventStore as a PubSub backend has a few advantages over the default PG2 adapter:

• No Erlang distribution required : nodes communicate through the shared database rather than through the Erlang cluster, so you can run multiple nodes without configuring Erlang node connectivity.
• Persistence : every broadcast is stored and can be replayed or audited later.

The tradeoffs are the need for storage and the additional latency of a database round-trip per broadcast, making it best suited for lower-throughput messaging where persistence and cross-node decoupling matter more than raw speed. This implementation is a proof of concept — no load tests were performed.

A full implementation of the adapter can be found on Github .

Phoenix.PubSub.Adapter in a nutshell

A Phoenix PubSub adapter must implement a few callbacks specified in Phoenix.PubSub.Adapter :

node_name(adapter_name)

Returns the node name as an atom or binary. Used mainly by Phoenix.Tracker. In most cases:

def node_name(nil), do: node()
def node_name(configured_name), do: configured_name

child_spec(keyword)

Generates the child spec for the adapter. GenServer provides a default; this rarely needs overriding.

broadcast(adapter_name, topic, message, dispatcher)

Called when a message is broadcast through Phoenix.PubSub.broadcast . The adapter_name is the PubSub name with .Adapter appended (e.g. MyApp.PubSub → MyApp.PubSub.Adapter ). The dispatcher module handles local delivery via dispatch/3 .

direct_broadcast(adapter_name, node_name, topic, message, dispatcher)

Same as broadcast/4 with an additional node_name — the message should only reach subscribers on that node.

The EventStore adapter

This section walks through a possible implementation of a Phoenix PubSub adapter that uses EventStore to distribute messages between nodes. This gives a solution that does not depend on Erlang/Elixir distribution, and an event log is stored in case further analysis is needed.

How Phoenix.PubSub works

Phoenix.PubSub uses Elixir’s Registry for subscriptions — each subscribe call registers an entry under the topic key. When broadcast is called, the framework invokes the adapter callback to distribute the message, then handles local dispatch.

The adapter’s job is to get the message to other nodes. For direct_broadcast , only subscribers on the target node should receive it.

The implementation

The adapter is a GenServer that joins the PubSub supervision tree. An eventstore option selects which EventStore module to use (in case you have multiple):

{Phoenix.PubSub,
  [name: MyApp.PubSub,
   adapter: Phoenix.PubSub.EventStore,
   eventstore: MyApp.EventStore]
}

The GenServer stores the EventStore module and the PubSub name in state — both are needed later:

defmodule Phoenix.PubSub.EventStore do
  @behaviour Phoenix.PubSub.Adapter
  use GenServer

  def start_link(opts) do
    GenServer.start_link(__MODULE__, opts, name: opts[:adapter_name])
  end

  def init(opts) do
    {:ok,
     %{
       eventstore: opts[:eventstore],
       pubsub_name: opts[:name]
     }}
  end
  #... implementation will come here ...#
end

Note the difference between opts[:name] and opts[:adapter_name] . The former is the name of the PubSub as a whole and is reserved for the Registry. Publishers use it when broadcasting messages. opts[:adapter_name] can be used as the name of the GenServer.

Distributing a message as an event

The GenServer appends a new event to the EventStore when broadcast is called:

def broadcast(server, topic, message, dispatcher, metadata \\ %{}) do
  metadata = Map.put(metadata, :dispatcher, dispatcher)
  GenServer.call(server, {:broadcast, topic, message, metadata})
end

def handle_call(
      {:broadcast, topic, message, metadata},
      _from_pid,
      %{id: id, eventstore: eventstore, serializer: serializer, pubsub_name: pubsub_name} = state
    ) do
  event = %EventStore.EventData{
    # ... constructed below
  }

  res = eventstore.append_to_stream(topic, :any_version, [event])

  # For direct_broadcast targeting the current node, the framework does not
  # call local dispatch, so the adapter must do it. For regular broadcast,
  # the framework handles local dispatch after adapter.broadcast returns :ok.
  current_node = to_string(node())
  destination_node = Map.get(metadata, :destination_node)

  if destination_node == current_node do
    dispatcher = Map.get(metadata, :dispatcher, Phoenix.PubSub)
    Phoenix.PubSub.local_broadcast(pubsub_name, topic, message, dispatcher)
  end

  {:reply, res, state}
end

direct_broadcast/5 is a thin wrapper that sets destination_node in the metadata before delegating to broadcast/5 :

def direct_broadcast(server, node_name, topic, message, dispatcher) do
  metadata = %{
    destination_node: to_string(node_name),
    source_node: to_string(node())
  }
  broadcast(server, topic, message, dispatcher, metadata)
end

source_node is stored in the event metadata for auditing. Routing is handled downstream by comparing destination_node against the current node.

The key decision is how to wrap the message inside %EventStore.EventData{} . Serialization is handled by a pluggable module (defaulting to Phoenix.PubSub.EventStore.Serializer.Base64 ) so the adapter is not tied to a specific encoding. The default serializer base64-encodes :erlang.term_to_binary/ 1 output — this is necessary because EventStore stores data as JSON and raw binaries would be invalid, and because JSON cannot distinguish atoms from strings so a round-trip through term serialization preserves type fidelity.

event = %EventStore.EventData{
  event_type: to_string(serializer),
  data: serializer.serialize(message)
}

A custom serializer can be provided via the serializer option as long as it implements serialize/1 and deserialize/1 .

Handling events, local distribution

Now that events are in the event store, any subscribed process will receive them. The GenServer must subscribe to all topics ( "$all" ). If the event store is also used for another purpose, it’s best to have a separate one for PubSub. The subscription is set up via handle_continue/2 , which runs immediately after init/1 completes, before any other messages can be processed.

def handle_continue(:subscribe, %{eventstore: eventstore} = state) do
  eventstore.subscribe("$all")

  {:noreply, state}
end

def handle_info({:subscribed, _subscription}, state), do: {:noreply, state}

A transient subscription is used since previous messages are not needed. The event store replies with a { :subscribed, subscription } message, which must also be handled. After this, the server will start receiving { :events, events } messages.

To avoid dispatching a local message twice (once from broadcast and once when the event arrives back from EventStore), a unique ID is added to the process state:

def init(opts) do
  {:ok,
   %{
     id: generate_unique_id(opts),
     eventstore: opts[:eventstore],
     pubsub_name: opts[:name],
     serializer: opts[:serializer] || Phoenix.PubSub.EventStore.Serializer.Base64
   }, {:continue, :subscribe}}
end

defp generate_unique_id(opts) do
  unique_id_fn = opts[:unique_id_fn] || fn _name -> UUID.uuid4() end
  unique_id_fn.(opts[:name])
end

A custom ID generator can be provided via unique_id_fn — a function that receives the PubSub name and returns a unique string. Useful when UUID is unavailable or when a deterministic ID is needed for testing.

The id is added to the event’s metadata field as source_id , keeping it separate from the message data. Serialization is delegated to the configurable serializer module. The handle_call for :broadcast becomes:

event = %EventStore.EventData{
  event_type: to_string(serializer),
  data: serializer.serialize(message),
  metadata: Map.put(metadata, :source_id, id)
}

Where the value of id and serializer come from the state, and metadata already contains dispatcher and any destination_node for direct broadcasts. When an event arrives back, source_id identifies the origin node so duplicates can be skipped:

def handle_info({:events, events}, state) do
  Enum.each(events, &local_broadcast_event(&1, state))

  {:noreply, state}
end

defp local_broadcast_event(
       %EventStore.RecordedEvent{
         data: data,
         metadata: metadata,
         stream_uuid: topic,
         eventbies_type: event_type
       },
       %{id: id, serializer: serializer, pubsub_name: pubsub_name} = _state
     ) do
  current_node = to_string(node())

  %{source_id: source_id, destination_node: destination_node, dispatcher: dispatcher} =
    convert_metadata_keys_to_atoms(metadata)

  is_destination? = is_nil(destination_node) or destination_node == current_node

  if not is_nil(dispatcher) and is_destination? and source_id != id and
       event_type == to_string(serializer) do
    Phoenix.PubSub.local_broadcast(
      pubsub_name,
      topic,
      serializer.deserialize(data),
      maybe_convert_to_existing_atom(dispatcher)
    )
  end
end

That’s it — a complete implementation of Phoenix PubSub using EventStore, including support for direct_broadcast via the destination_node metadata field and pluggable serialization.

The complete implementation can be found at esl/phoenix_pubsub_eventstore .
Need help building reliable distributed systems with Elixir? Get in touch with our team .

The post Implementing a Phoenix PubSub Adapter with EventStore appeared first on Erlang Solutions .

When I need to configure something in a complicated way, I find myself reviewing the embedded language that provided the server to create a flexible configuration. In Redis, you can improve the performance of requests, in Nginx, you can improve the handling of incoming requests, FreeSwitch offers alternatives for performing the same tasks using different embedded languages. Even in a software like TheGimp, you can add your own code to edit images.

Among the embedded languages, JavaScript and Lua are the most commonly used languages. JavaScript is very well known to the Erlang community because it was integrated (as a port, it is not implemented on top of Erlang) in popular products such as CouchDB and Riak. But I think the more exciting option, raised by Erlang Co-Creator, Robert Virding, is to implement Lua on top of Erlang, which can be used as an embedded language.

Why? Let’s take a look.

Complex Configuration

Usually, when tasked with fitting the definition of a behaviour we would like to configure, we would create an algorithm in a simple language such as Lua. This saves us from performing activities like:

1. Defining the configuration to fit all of the cases.
2. Reading and transmitting that information to be prepared for use.
3. Writing specific code to handle that standardised information.

This kind of implementation is used frequently. It is easy to think of examples you’re likely to come across in day-to-day life. For example, supermarket offers which have multiple dependencies, commissions for salespeople which might feature variable ranges and percentages based on the type of sale, amount of sale or tax brackets, even SMS, emails or HTTP requests could be considered examples.

To demonstrate this, let’s look at an example of developing a load balancer. This is a simple project using cowboy, and luerl as dependencies.Dependingdependencies., and  depending on the headers and other information from the HTTP request, we can send it to the different web servers we have available and configured.

Based on the above premise we can write the configuration as follows:

{load_balancer, [
    {servers, [
        {odin, "1.1.1.1", [
            {in, method, [post]},
            {'>', <<"content-length">>, 10000},
            {in, <<"accept">>, [<<"json">>]}
        ]},
        {thor, "1.1.1.2", [
            {in, method, [get, post]},
            {'==', http_version, <<"2">>}
        ]},
        {balder, "1.1.1.3", [
            {in, method, [get, post]}
        ]}
    ]}
]}.

As you can see, we have to define a 3-tuple system for the rules with the operation in the first element and the two operators as the following elements inside of the tuple. In addition, we are occasionally handling the second element as a header name (if it is a binary), but at other times it’s the method we use to perform the request (using the atom “method”) and other times still, the HTTP version is used to gather the information.

The problem is that we have no closed specifications. We could add more elements or even change the meaning of them. What if we want to use logical modifiers like “and” and “or” to join the checks instead of assuming they are always using “and”? This change will add more complexity to our configuration and more complexity means more possibilities for making mistakes.

At the moment, if the configuration is wrong or adds something that is not granted, it is up to us to trigger the corresponding error and point to where it is to make it easier to fix. As you can imagine, that is not an easy thing to do if you are handling it during runtime.

Lua saves the day!

It’s not unconventional to think about configuration in terms of a specific code. At this point, Lua code could be put in charge of the definition because it is based on Lua semantics.

We only need the information for the configuration and running of the snippet to give us the desired behaviour we want to plug into the correct place. For example, the previous configuration could be written as:

local odin = "1.1.1.1"
local thor = "1.1.1.2"
local balder = "1.1.1.3"

local method = http.method()
local size = tonumber(http.header("content-length")) or 0
local accept = config.split(http.header("accept") or "", ", ")
local httpver = http.version()

if method == "post" and size > 10000 and config.member("json", accept) then
    return odin
elseif config.member(method, {"get", "post"}) and httpver == "2" then
    return thor
elseif config.member(method, {"get", "post"}) then
    return balder
end

As you can see, we are able to optimise and fix the code to suit our needs, it is shorter and clearer than the original configuration and, most importantly, we can now test and check to be sure it is compiling correctly.

The important thing to keep in mind is that the configuration code must include the functions which are going to be needed to handle the request. In the example above, we are using functions like http_version() , http_header("...") or even split(...) and member(...) . These functions should be provided to the interpreter.

Of course, the interpreter also has other functions available, we only need to provide the specific functions that are required for our business logic.

In addition to improving the performance, using these functions we are also improving the security because it has not been able to access functions which are not used. You can check the file luerl_sandbox.erl where it is removing the access to the functions which use the underlying operating system.

Where the code dwells?

Inserting the Lua code into the configuration can be a little tricky. To avoid this, I recommend putting these scripts into the priv directory as a normal Lua file (using the extension .lua) this could even be done inside of a database if we are handling the configuration in an automated way using a key/value storage configuration such as etcd.

The most important thing to keep in mind before running that code is to have a specific task which helps you to parse it and ensure the code is correct. One solution is to conduct a testing phase to ensure that the configuration is not breaking or negatively impacting other parts of the system.

For example, in the previous code, we would write a couple of libraries, one called utils for the functions needed for strings and tables and another called http needed for the HTTP functions. An example would be:

-module(luerl_lib_http).
-export([load/1, install/1, put_request/2]).

-include_lib("luerl/include/luerl.hrl").

-define(REQUEST, http_request).

load(St) ->
    luerl:load_module([<<"http">>], luerl_lib_http, St).

install(St) ->
    luerl_heap:alloc_table(table(), St).

put_request(Request, St) ->
    luerl:put_private(?REQUEST, Request, St).

table() ->
  [
    {<<"method">>,  #erl_func{code = fun method/2}},
    {<<"version">>, #erl_func{code = fun version/2}},
    {<<"header">>,  #erl_func{code = fun header/2}}
  ].

method(_Args, St) ->
  #{method := Method} = request(St),
  {[Method], St}.

version(_Args, St) ->
  #{version := Version} = request(St),
  {[Version], St}.

header([Name|_], St) when is_binary(Name) ->
  #{headers := Headers} = request(St),
  {[maps:get(Name, Headers, nil)], St};
header(Args, St) ->
  luerl_lib:badarg_error(<<"header">>, Args, St).

request(St) -> luerl:get_private(?REQUEST, St).

defmodule LuerlLib.Http do
  require Record

  Record.defrecord(:erl_func, Record.extract(:erl_func, from_lib: "luerl/include/luerl.hrl"))

  @private_key :http_request

  def load(state), do: :luerl.load_module(["http"], __MODULE__, state)

  def install(state), do: :luerl_heap.alloc_table(exports(), state)

  def put_request(state, request), do: :luerl.put_private(@private_key, request, state)

  defp exports do
    [
      {"method", erl_func(code: &method/2)},
      {"version", erl_func(code: &version/2)},
      {"header", erl_func(code: &header/2)}
    ]
  end

  defp method(_args, state), do: {[request(state).method], state}

  defp version(_args, state), do: {[request(state).version], state}

  defp header([name | _], state) when is_binary(name),
    do: {[Map.get(request(state).headers, name)], state}

  defp header(args, state), do: :luerl_lib.badarg_error("header", args, state)

  defp request(state), do: :luerl.get_private(@private_key, state)
end

-module(luerl_lib_config).
-export([load/1, install/1]).

-include_lib("luerl/include/luerl.hrl").

load(St) ->
    luerl:load_module([<<"config">>], luerl_lib_config, St).

install(St) ->
    luerl_heap:alloc_table(table(), St).

table() ->
  [
    {<<"split">>, #erl_func{code = fun split/2}},
    {<<"member">>, #erl_func{code = fun member/2}}
  ].

member([Entry, #tref{}=Table], St) ->
  #table{a = Array} = luerl_heap:get_table(Table, St),
  Result = array:foldl(fun
    (_, V, false) when V =:= Entry -> true;
    (_, _, Acc) -> Acc
  end, false, Array),
  {[Result], St};
member(Args, St) ->
  luerl_lib:badarg_error(<<"member">>, Args, St).

split([String, Sep], St) when is_binary(String), is_binary(Sep) ->
  {Tref, St1} = luerl:encode(string:split(String, Sep, all), St),
  {[Tref], St1};
split(Args, St) ->
  luerl_lib:badarg_error(<<"split">>, Args, St).

defmodule LuerlLib.Config do
  require Record

  Record.defrecord(:erl_func, Record.extract(:erl_func, from_lib: "luerl/include/luerl.hrl"))
  Record.defrecord(:table, Record.extract(:table, from_lib: "luerl/include/luerl.hrl"))

  def load(state), do: :luerl.load_module(["config"], __MODULE__, state)

  def install(state), do: :luerl_heap.alloc_table(exports(), state)

  defp exports do
    [
      {"split", erl_func(code: &split/2)},
      {"member", erl_func(code: &member/2)}
    ]
  end

  defp split([string, separator | _], state) when is_binary(string) and is_binary(separator) do
    {table_ref, state} = :luerl.encode(String.split(string, separator), state)
    {[table_ref], state}
  end

  defp split(args, state), do: :luerl_lib.badarg_error("split", args, state)

  defp member([entry, table_ref | _], state) when Record.is_record(table_ref, :tref) do
    table(a: array) = :luerl_heap.get_table(table_ref, state)
    found = :array.sparse_foldl(fn _index, value, acc -> acc or value === entry end, false, array)
    {[found], state}
  end

  defp member(args, state), do: :luerl_lib.badarg_error("member", args, state)
end

As you can see, we are implementing the functions we need and making them available to our Lua interface under the config and http packages. To load these functions, we have to run the load function which is exported in both modules. NOTE: install is used as a callback function in load_module

A great benefit of doing things this way is the “compile once, and ready many” approach. The config does not have to be loaded for every request. If we put it all together in our application, it would something  like like the following

Add a build_request function and its helpers to your luerl_lib_http and LuerlLib.Http modules.

-export([load/1, install/1, put_request/2, build_request/1]). % add export

build_request(CowboyReq) ->
    #{method => string:lowercase(cowboy_req:method(CowboyReq)),
      version => version_to_binary(cowboy_req:version(CowboyReq)),
      headers => cowboy_req:headers(CowboyReq)}.

version_to_binary('HTTP/1.0') -> <<"1.0">>;
version_to_binary('HTTP/1.1') -> <<"1.1">>;
version_to_binary('HTTP/2') -> <<"2">>.

  def build_request(cowboy_req) do
    %{
      method: cowboy_req |> :cowboy_req.method() |> String.downcase(),
      version: version_to_binary(:cowboy_req.version(cowboy_req)),
      headers: :cowboy_req.headers(cowboy_req)
    }
  end

  defp version_to_binary(:"HTTP/1.0"), do: "1.0"
  defp version_to_binary(:"HTTP/1.1"), do: "1.1"
  defp version_to_binary(:"HTTP/2"), do: "2"

Add handler modules for handling the request

-module(luerl_lib_handler).
-export([init/2]).

init(CowboyReq, {Form, St}) ->
    LuaReq = luerl_lib_http:build_request(CowboyReq),
    St1 = luerl_lib_http:put_request(LuaReq, St),
    Reply = case luerl:call_chunk(Form, St1) of
        {ok, [], _St2} ->
            cowboy_req:reply(404, #{}, <<"no matching route">>, CowboyReq);
        {ok, Rets, St2} ->
            [Backend] = luerl:decode_list(Rets, St2),
            cowboy_req:reply(200, #{}, Backend, CowboyReq);
        {lua_error, _Reason, _St2} ->
            cowboy_req:reply(500, #{}, <<"routing error">>, CowboyReq)
    end,
    {ok, Reply, {Form, St}}.

defmodule LuerlLib.Handler do
  def init(cowboy_req, {form, state}) do
    lua_req = LuerlLib.Http.build_request(cowboy_req)
    state1 = LuerlLib.Http.put_request(state, lua_req)

    reply =
      case :luerl.call_chunk(form, state1) do
        {:ok, [], _state2} ->
          :cowboy_req.reply(404, %{}, "no matching route", cowboy_req)

        {:ok, returns, state2} ->
          [backend] = :luerl.decode_list(returns, state2)
          :cowboy_req.reply(200, %{}, backend, cowboy_req)

        {:lua_error, _reason, _state2} ->
          :cowboy_req.reply(500, %{}, "routing error", cowboy_req)
      end

    {:ok, reply, {form, state}}
  end
end

With all of that defined, we can now call the relevant functions from our application startup:

-module(luerl_lib_app).
-behaviour(application).
-export([start/2, stop/1]).

start(_StartType, _StartArgs) ->
    {ok, Form, St} = build_lua_state(),
    Dispatch = cowboy_router:compile([{'_', [{'_', luerl_lib_handler, {Form, St}}]}]),
    {ok, _} = cowboy:start_clear(luerl_lib_listener, [{port, 8080}], #{env => #{dispatch => Dispatch}}),
    luerl_lib_sup:start_link().

stop(_State) ->
    ok = cowboy:stop_listener(luerl_lib_listener).

build_lua_state() ->
    St0 = luerl_lib_http:load(luerl_lib_config:load(luerl_sandbox:init())),
    luerl:loadfile("priv/config.lua", St0).

Supervisor

-module(luerl_lib_sup).
-behaviour(supervisor).
-export([start_link/0, init/1]).

start_link() ->
    supervisor:start_link({local, ?MODULE}, ?MODULE, []).

init([]) ->
    {ok, {#{strategy => one_for_one, intensity => 1, period => 5}, []}}.

defmodule LuerlLib.Application do
  use Application

  def start(_type, _args) do
    state =
      :luerl_sandbox.init()
      |> LuerlLib.Config.load()
      |> LuerlLib.Http.load()

    {:ok, form, state} = :luerl.loadfile(~c"priv/config.lua", state)
    dispatch = :cowboy_router.compile([{:_, [{:_, LuerlLib.Handler, {form, state}}]}])

    {:ok, _} =
      :cowboy.start_clear(:luerl_lib_listener, [port: 8080], %{env: %{dispatch: dispatch}})

    LuerlLib.Supervisor.start_link()
  end
end

Notice that we load the file once (with the lines below) and pass it down to the handler, it helps us obtain the forms that we need and the handler will always execute based on that initially loaded configuration.

{ok, Form, St1} = luerl:loadfile("priv/config.lua", St0).

{:ok, form, state} = :luerl.loadfile(String.to_charlist(path), state)

Scaling up

Lua scales up because it is built on top of Erlang. This means Lua is using the same processes as Erlang does, it also means we are not using ports to communicate with the Lua interpreter, we have the Lua interpreter running on Erlang.

This makes a great difference in comparison to JavaScript because if you are handling millions of requests and all of them require a JavaScript snippet, this can cause a bottleneck very quickly if you have to limit the number of ports or requests.

On the other hand, Lua is using native Erlang functions when it calls the functions we want to provide for the interpreter. That makes a clear improvement, saving us from performing data serialisation or transformation.

Conclusion

Using a language for flexible configuration gives us the possibility to create an easy interface to provide configurations, reduce the amount of code we need to write, and improve the maintenance without jeopardising the performance of the system. At the moment, you can use Lua as we have explained during the article or you can jump into PHP if you need to process text or templates on top of Erlang or Elixir.

Alternatively you can join the community and provide other solutions which help us build better fit-for-purpose software. Get in touch if you need help building your system with one of these solutions.

The post How to use Lua for flexible configurations in Erlang and Elixir appeared first on Erlang Solutions .

Members of the community will operate an XMPP-themed stand at the upcoming LinuxDays conference in Prague – if you are around, you are welcome to come visit. Similarly, if you know of someone who might be interested in XMPP or whom you would like to introduce to it, Czech-speaking or not, please pass on the information to them.

Two XMPP talks have also been submitted for the conference, you can support them in a vote which takes place over the first week of September.

The conference takes place over the first weekend of October, 3rd and 4th, at the Faculty of Information Technology of the Czech Technical University and is free to attend (participants are asked to register, if possible). You can find more information at the conference’s website (in Czech).