gen_statem and protocols

This blog post is a deep dive into some of the concepts discussed in my recent conference talk at FOSDEM . The presentation explored some basic theoretical concepts of Finite State Machines, and some special powers of Erlang’s gen_statem in the context of protocols and event-driven development, and building upon this insight, this post delves into harnessing the capabilities of the gen_statem behaviour. Let’s jump straight into it!

Protocols

The word protocol comes from the Greek “πρωτόκολλον”, from πρῶτος (prôtos, “first”) + κόλλα (kólla, “glue”), used in Byzantine greek as the first sheet of a papyrus-roll, bearing the official authentication and date of manufacture of the papyrus ¹ . Over time, the word describing the first page became a synecdoche for the entire document.

The word protocol was then used primarily to refer to diplomatic or political treaties until the field of Information Technology overloaded the word to describe “treaties” too, but between machines, which as in diplomacy, governs the manner of communication between two entities . As the entities communicate, a given entity receives messages describing the interactions that peers are establishing with it, creating a model where an entity reacts to events .
In this field of Technology, so much of the job of a programmer is implementing such communication protocol , which reacts to events . The protocol defines the valid messages and the valid order, and any side effects an event might have. You know many such protocols: TCP, TLS, HTTP, or XMPP, just to name some good old classics.

The event queue

As a BEAM programmer, implementing such an event-driven program is an archetypical paradigm you’re well familiar with: you have a process, which has a mailbox, and the process reacts to these messages one by one. It is the actor model in a nutshell: an actor can, in response to a message it receives:

• send a finite number of messages to other Actors;
• create a finite number of new Actors;
• designate the behaviour to be used for the next message it receives.

It is ubiquitous to implement such actors as a gen_server, but, pay attention to the last point: designate the behaviour to be used for the next message it receives . When a given event (a message) implies information about how the next event should be processed, there is implicitly a transformation of the process state. What you have is a State Machine in disguise.

Finite State Machines

Finite State Machines (FSM for short) are a function 𝛿 of an input state and an input event, to an output state where the function can be applied again. This is the idea of the actor receiving a message and designating the behaviour for the next: it chooses the state that will be input together with the next event.

FSMs can also define output, in such cases they are called Finite State Transducers (FST for short, often simplified to FSMs too), and their definition adds another alphabet for output symbols, and the function 𝛿 that defines the machine does return the next state together with the next output symbol for the current input.

gen_statem

When the function’s input is the current state and an input symbol, and the output is a new state and a new output symbol, we have a Mealy machine ² . And when the output alphabet of one machine is the input alphabet of another, we can then intuitively compose them. This is the pattern that gen_statem implements.
gen_statem has three important features that are easily overlooked, taking the best of pure Erlang programming and state machine modelling: it can simulate selective receives, offers an extended mailbox, and allows for complex data structures as the FSM state.

Selective receives

Imagine the archetypical example of an FSM, a light switch. The switch is for example digital and translates requests to a fancy light-set using an analogous cable protocol. The code you’ll need to implement will look something like the following:

handle_call(on, _From, {off, Light}) ->
	on = request(on, Light),
	{reply, on, {on, Light}};
handle_call(off, _From, {on, Light}) ->
	off = request(off, Light),
	{reply, off, {off, Light}};
handle_call(on, _From, {on, Light}) ->
	{reply, on, {on, Light}};
handle_call(off, _From, {off, Light}) ->
	{reply, off, {off, Light}}.

But now imagine the light request was to be asynchronous, now your code would look like the following:

handle_call(on, From, {off, undefined, Light}) ->
	Ref = request(on, Light),
	{noreply, {off, {on, Ref, From}, Light}};
handle_call(off, From, {on, undefined, Light}) ->
	Ref = request(off, Light),
	{noreply, {on, {off, Ref, From}, Light}};

handle_call(off, _From, {on, {off, _, _}, Light} = State) ->
	{reply, turning_off, State};  %% ???
handle_call(on, _From, {off, {on, _, _}, Light} = State) ->
	{reply, turning_on, State}; %% ???
handle_call(off, _From, {off, {on, _, _}, Light} = State) ->
	{reply, turning_on_wait, State};  %% ???
handle_call(on, _From, {on, {off, _, _}, Light} = State) ->
	{reply, turning_off_wait, State}; %% ???

handle_info(Ref, {State, {Request, Ref, From}, Light}) ->
	gen_server:reply(From, Request),
	{noreply, {Request, undefined, Light}}.

The problem is, that now the order of events is not defined, and reorderings of the user requesting a switch and the light system announcing finalising the request are possible, so you need to handle these cases. When the switch and the light system had only two states each, you had to design and write four new cases: the number of new cases grows by multiplying the number of cases on each side. And each case is a computation of the previous cases, effectively creating a user-level callstack.

So we now try migrating the code to a properly explicit state machine, as follows:

off({call, From}, off, {undefined, Light}) ->
	{keep_state_and_data, [{reply, From, off}]};
off({call, From}, on, {undefined, Light}) ->
	Ref = request(on, Light),
	{keep_state, {{Ref, From}, Light}, []};
off({call, From}, _, _) ->
	{keep_state_and_data, [postpone]};
off(info, {Ref, Response}, {{Ref, From}, Light}) ->
	{next_state, Response, {undefined, Light}, [{reply, From, Response}]}.

on({call, From}, on, {undefined, Light}) ->
	{keep_state_and_data, [{reply, From, on}]};
on({call, From}, off, {undefined, Light}) ->
	Ref = request(off, Light),
	{keep_state, {{Ref, From}, Light}, []};
on({call, From}, _, _) ->
	{keep_state_and_data, [postpone]};
on(info, {Ref, Response}, {{Ref, From}, Light}) ->
	{next_state, Response, {undefined, Light}, [{reply, From, Response}]}.

Now the key lies in postponing requests: this is akin to Erlang’s selective receive clauses, where the mailbox is explored until a matching message is found. Events that arrive out of order can this way be treated when the order is right.

This is an important difference between how we learn to program in pure Erlang, with the power of selective receives where we chose which message to handle, and how we learn to program in OTP, where generic behaviours like gen_server force us to handle the first message always, but in different clauses depending on the semantics of the message ( handle_cast , handle_call and handle_info) . With the power to postpone a message, we effectively choose which message to handle without being constrained with the code location.

This section is inspired really by Ulf Wiger’s fantastic talk, Death by Accidental Complexity ³ , so if you’ve known of the challenge he explained, this section hopefully serves as a solution to you.

Complex Data Structures

This was much explained in the previous blog on state machines ⁴ : by using gen_statem ’s handle_event_function callback, apart from all the advantages explained in the aforementioned blog, we can also reduce the implementation of 𝛿 to a single function called handle_event , which makes the previous code take advantage of a lot of code reuse, see the following equivalent state machine:

handle_event({call, From}, State, State, {undefined, Light}) ->
	{keep_state_and_data, [{reply, From, State}]};
handle_event({call, From}, Request, State, {undefined, Light}) ->
	Ref = request(Request, Light),
	{keep_state, {{Ref, From}, Light}, []};
handle_event({call, _}, _, _, _) ->
	{keep_state_and_data, [postpone]};
handle_event(info, {Ref, Response}, State, {{Ref, From}, Light}) ->
	{next_state, Response, {undefined, Light}, [{reply, From, Response}]}.

This section was extensively described in the previous blog post so to learn more about it, please enjoy your read!

An extended mailbox

We saw that the function 𝛿 of the FSM in question is called when a new event is triggered. In implementing a protocol, this is modelled by messages to the actor’s mailbox. In a pure FSM, a message that has no meaning within a state would crash the process, but in practice, while the order of messages is not defined, it might be a valid computation to postpone them and process them when we reach the right state.

This is what a selective receive would do, by exploring the mailbox and looking for the right message to handle for the current state. In OTP, the general practice is to leave the lower-level communication abstractions to the underlying language features, and code in a higher and more sequential style as defined by the generic behaviours: in gen_statem , we have an extended view of the FSM’s event queue.

There are two more things to notice we can do with gen_statem actions: one is to insert ad-hoc events with the construct { next_event , EventType , EventContent }, and the other is to insert timeouts, which can be restarted automatically on any new event, any state change, or not at all. These seem like different event queues for our eventful state machine, together with the process’s mailbox, but really it is only one queue we can see as an extended mailbox.

The mental picture is as follows: There is only one event queue, which is an extension of the process mailbox, and this queue has got three pointers:

• A head pointing at the oldest event;
• A current pointing at the next event to be processed.
• A tail pointing at the youngest event;

This model is meant to be practically identical to how the process mailbox is perceived ⁵ .

• postpone causes the current position to move to its next younger event, so the previous current position is still in the queue reachable from head .
• Not postponing an event i.e consuming it causes the event to be removed from the queue and current position to move to its next younger event.
• NewState =/= State causes the current position to be set to head i.e the oldest event.
• next_event inserts event(s) at the current position i.e as just older than the previous current position.
• { timeout , 0 , Msg } inserts a timeout , Msg event after tail i.e as the new youngest received event.

Let’s see the event queue in pictures:

handle_event(Type1, Content1, State1, Data) -> {keep_state_and_data, [postpone]};		When the first event to process is 1 , after any necessary logic we might decide to postpone it. In such case, the event remains in the queue, reachable from HEAD , but Current is moved to the next event in the queue, event 2 .
handle_event(Type1, Content1, State1, Data) -> {keep_state_and_data, [postpone]}; ... handle_event(Type2, Content2, State1, Data) -> {next_state, State2};		When handling event 2 , after any necessary logic, we decide to transition to a new state. In this case, 2 is removed from the queue, as it has been processed, and Current is moved to HEAD , which points again to 1 , as the state is now a new one.
handle_event(Type1, Content1, State1, Data) -> {keep_state_and_data, [postpone]}; ... handle_event(Type2, Content2, State1, Data) -> {next_state, State2}; ... handle_event(Type1, Content1, State2, Data) -> {keep_state_and_data, [{next_event, TypeA, ContentA}]};		After any necessary handling for 1 , we now decide to insert a next_event called A . Then 1 is dropped from the queue, and A is inserted at the point where Current was pointing. HEAD is also updated to the next event after 1 , which in this case is now A .
handle_event(Type1, Content1, State1, Data) -> {keep_state_and_data, [postpone]}; ... handle_event(Type2, Content2, State1, Data) -> {next_state, State2}; ... handle_event(Type1, Content1, State2, Data) -> {keep_state_and_data, [{next_event, TypeA, ContentA}]}; ... handle_event(TypeA, ContentA, State2, Data) -> {keep_state_and_data, [postpone]};		Now we decide to postpone A, so Current is moved to the next event in the queue, 3 .
handle_event(Type1, Content1, State1, Data) -> {keep_state_and_data, [postpone]}; ... handle_event(Type2, Content2, State1, Data) -> {next_state, State2}; ... handle_event(Type1, Content1, State2, Data) -> {keep_state_and_data, [{next_event, TypeA, ContentA}]}; ... handle_event(TypeA, ContentA, State2, Data) -> {keep_state_and_data, [postpone]}; ... handle_event(Type3, Content3, State2, Data) -> keep_state_and_data;		3 is processed normally, and then dropped from the queue. No other event is inserted nor postponed, so Current is simply moved to the next, 4 .
handle_event(Type1, Content1, State1, Data) -> {keep_state_and_data, [postpone]}; ... handle_event(Type2, Content2, State1, Data) -> {next_state, State2}; ... handle_event(Type1, Content1, State2, Data) -> {keep_state_and_data, [{next_event, TypeA, ContentA}]}; ... handle_event(TypeA, ContentA, State2, Data) -> {keep_state_and_data, [postpone]}; ... handle_event(Type3, Content3, State2, Data) -> keep_state_and_data; ... handle_event(Type4, Content4, State2, Data) -> {keep_state_and_data, [postpone, {next_event, TypeB, ContentB}]};		And 4 is now postponed, and a new event B is inserted, so while HEAD still remains pointing at A, 4 is kept in the queue and Current will now point to the newly inserted event B .

This section is in turn inspired by this comment on GitHub ⁶ .

Conclusions

We’ve seen how protocols are governances over the manners of communication between two entities and that these governances define how messages are transmitted, processed, and how they relate to each other. We’ve seen how the third clause of the actor model dictates that an actor can designate the behaviour to be used for the next message it receives and how this essentially defines the 𝛿 function of a state machine and that Erlang’s gen_statem behaviour is an FSM engine with a lot of power over the event queue and the state data structures.

Do you have protocol implementations that have suffered from extensibility problems? Have you had to handle an exploding number of cases to implement when the event order might reorder in any possible way? If you’ve suffered from death by accidental complexity, or your code has suffered from state machines in disguise, or your testing isn’t comprehensive enough by default, the tricks and points of view of this post should help you get started, and we can always help you keep moving forward!

1. πρωτόκολλον — Wiktionary
2. Mealy Machine — Wikipedia
3. https://www.infoq.com/presentations/Death-by-Accidental-Complexity/
4. https://github.com/erlang/otp/pull/960#issuecomment-198141456
5. https://github.com/erlang/otp/pull/960#issuecomment-198141456
6. Reimplementing technical debt with state machines

The post gen_statem Unveiled appeared first on Erlang Solutions .

As of version 24.02 , ejabberd is shipped with a Matrix gateway and can participate in the Matrix
federation . This means that an XMPP client can exchange messages with Matrix users or rooms.

Let’s see how to configure your ejabberd to enable this gateway.

Configuration in ejabberd

HTTPS listener

First, add an HTTP handler , as Matrix uses HTTPS for Server-Server API.

In the listen section of your ejabberd.yml configuration file, add a handler on Matrix port 8448 for path /_matrix that calls the mod_matrix_gw module. You must enable TLS on this port to accept HTTPS connections (unless a proxy already handles HTTPS in front of ejabberd) and provide a valid certificate for your Matrix domain (see matrix_domain below). You can set this certificate using the certfile option of the listener, like in the example below, or listing it in the certfiles top level option .

Example :

listen:
  -
    port: 5222
    module: ejabberd_c2s
  -
    port: 8448 # Matrix federation
    module: ejabberd_http
    tls: true
    certfile: "/opt/ejabberd/conf/matrix.pem"
    request_handlers:
      "/_matrix": mod_matrix_gw

If you want to use a non-standard port instead of 8448 , you must serve a /.well-known/matrix/server on your Matrix domain (see below).

Server-to-Server

You must enable s2s (Server-to-Server federation) by setting an access rule all or allow on s2s_access top level option:

Example :

s2s_access: s2s

access_rules:
  local:
    - allow: local
  c2s:
    - deny: blocked
    - allow
  s2s:
    - allow # to allow Matrix federation

Matrix gateway module

Finally, add mod_matrix_gw module in the modules list.

Example :

modules:
  mod_matrix_gw:
    matrix_domain: "matrixdomain.com"
    key_name: "key1"
    key: "SU4mu/j8b8A1i1EdyxIcKlFlrp+eSRBIlZwGyHP7Mfo="

matrix_domain

Replace matrixdomain.com with your Matrix domain. That domain must resolve to your ejabberd server or serve a file https://matrixdomain.com/.well-known/matrix/server that contains a JSON file with the address and Matrix port (as defined by the Matrix HTTPS handler, see above) of your ejabberd server:

Example :

{
   "m.server": "ejabberddomain.com:8448"
}

key_name & key

The key_name is arbitrary. The key value is your base64-encoded ed25519 Matrix signing key . It can be generated by Matrix tools or in an Erlang shell using the command base64:encode(element(2, crypto:generate_key(eddsa, ed25519))). :

Example :

$ erl
Erlang/OTP 24 [erts-12.3.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [dtrace]

Eshell V12.3.1 (abort with ^G)
1> base64:encode(element(2, crypto:generate_key(eddsa, ed25519))).
<<"SU4mu/j8b8A1i1EdyxIcKlFlrp+eSRBIlZwGyHP7Mfo=">>
2> q().
ok

Once your configuration is ready, you can restart ejabberd.

Testing

To check if your setup is correct, go to the following page and enter your Matrix domain (as set by the matrix_domain option):
https://federationtester.matrix.org/

This page should list any problem related to Matrix on your ejabberd installation.

Routing

What messages are routed to an external Matrix server?

Implicit routing

Let’s say an XMPP client connected to your ejabberd server sends a message to a JID user1@domain1.com . If domain1.com is defined by the hosts parameter of your ejabberd server (i.e. it’s one of your XMPP domains), the message will be routed locally. If it’s not, ejabberd will try to establish an XMPP Server-to-Server connection to a remote domain1.com XMPP server. If this fails (i.e. there is no such external domain1.com XMPP domain), then ejabberd will try on the Matrix federation, transforming the user1@domain1.com JID into the Matrix ID @user1:domain1.com and will try to open a connection to a remote domain1.com Matrix domain.

Explicit routing

It is also possible to route messages explicitly to the Matrix federation by setting the option matrix_id_as_jid in the mod_matrix_gw module to true :

Example :

modules:
  mod_matrix_gw:
    host: "matrix.@HOST@"
    matrix_domain: "matrixdomain.com"
    key_name: "key1"
    key: "SU4mu/j8b8A1i1EdyxIcKlFlrp+eSRBIlZwGyHP7Mfo="
    matrix_id_as_jid: true

In this case, the automatic fallback to Matrix when XMPP s2s fails is disabled and messages must be explicitly sent to the matrix gateway service Jabber ID to be routed to a remote Matrix server.

To send a message to the Matrix user @user:remotedomain.com , the XMPP client must send a message to the JID user%remotedomain.com@matrix.xmppdomain.com , where matrix.xmppdomain.com is the JID of the gateway service as set by the host option of the mod_matrix_gw module (the keyword @HOST@ is replaced with the XMPP domain of the server). If host is not set, the Matrix gateway JID is your XMPP domain with the matrix. prefix added.

The default value for matrix_id_as_jid is false , so the implicit routing will be used if this option is not set.

The post Matrix gateway setup with ejabberd first appeared on ProcessOne .

The Ignite Realtime Community is pleased to announce the release of Openfire 4.8.1. This release addresses a number of issues found with the major 4.8.0 release a few months back.

Interested in getting started? You can download installers of Openfire here . Our documentation contains an upgrade guide that helps you update from an older version.

sha256sum checksum values for the release artefacts are as follows

2ff28c5d7ff97305b2d6572e60b02f3708e86750d959459d7c5d6e17d4f9f932  openfire-4.8.1-1.noarch.rpm
f622719e4dbd43aadc9434ba4ebc0d8c65ec30dd25a7d2e99c7de33006a24f56  openfire_4.8.1_all.deb
3507b5d64c961daf526a52a73baaac7c84af12eb0115b961c2f95039255aec57  openfire_4_8_1.dmg
141f6eaf374dfb7c4cca345e1b598fed5ce3af9c70062a8cc0d9571e15c29c7d  openfire_4_8_1.exe
c6f0cf25a2d10acd6c02239ad59ab5954da5a4b541bc19949bd381fefb856da1  openfire_4_8_1.tar.gz
bec5b03ed56146fec2f84593c7e7b269ee5c32b3a0d5f9e175bd41f28a853abe  openfire_4_8_1_x64.exe
7403113b701aaf8a37dcd2d7e22fbb133161d322ad74505c95e54eaf6533f183  openfire_4_8_1.zip

For other release announcements and news follow us on Mastodon or X

1 post - 1 participant

Read full topic

Overview

This release adds new functionality and features to Cobalt, our web based role and user provisioning tool. You can find out more about Cobalt here .

Multiple Cobalt Servers

This enhancement enables multiple Cobalt servers to be run against a single directory. There are two reasons for this.

1. In a distributed environment it is useful to have multiple Cobalt servers at different locations, each connected to the local node of a multi-master directory.
2. Where a read only directory is replicated, for example using Sodium Sync to a Mobile Unit, it is useful to run Cobalt (read only) against the replica, to allow local administrators to conveniently view the configuration using Cobalt.

Password Management and Password Policy

This update includes a number of enhancements relating to password management:

1. Cobalt is now aware of password policy. A key change is that after administrator creation or change of password, when password policy requires user change, Cobalt will mark the password as requiring user change. To be useful in deployment, the applications used also need to be password policy aware.
2. Cobalt added a user UI to enable password change/reset, to complement Administrator password change.
3. Administrator option to email new password to user.

Security Management

1. Directory Access Rights Management. M-Vault Directory Groups enable specification of user rights, to directory and messaging configuration in the directory. This can be configured by Cobalt by domain administrators.
2. Certificate expiry checking. When managing a directory holding many certificates, it is important to keep them up to date. Cobalt provides a tool which can be run at intervals to determine certificates which have expired and certificates which will expire soon.

User Directory Viewer

Cobalt’s primary purpose is directory administration. This update adds a complementary tool which enables users to access information in the directory managed by Cobalt. This uses anonymous access for user convenience.

Miscellaneous

1. Flexible Search. Cobalt administrators have the option to configure search fields available for users. Configuration is per-domain.
2. Users, Roles and mailing list members now sorted alphabetically.
3. Base DN can be specified for users for a domain. If specified, Cobalt allows browsing users under this DIT (entry) using subtree search. Add user operation is disabled if this is specified. This allows Cobalt to:
   1. Utilize User provision by other means, for reference from within Cobalt managed components.
   2. To modify the entries, but does not allow addition of new entries.

🚀 Introducing ejabberd 24.02: A Huge Release!

ejabberd 24.02 has just been release and well, this is a huge release with 200 commits and more in the libraries. We’ve packed this update with a plethora of new features, significant improvements, and essential bug fixes, all designed to supercharge your messaging infrastructure.


– 🌐 Matrix Federation Unleashed: Imagine seamlessly connecting with Matrix servers – it’s now possible! ejabberd breaks new ground in cross-platform communication, fostering a more interconnected messaging universe. We have still some ground to cover and for that we are waiting for your feedback.
– 🔐 Cutting-Edge Security with TLS 1.3 & SASL2 : In an era where security is paramount, ejabberd steps up its game. With support for TLS 1.3 and advanced SASL2 protocols, we increase the overall security for all platform users.
– 🚀 Performance Enhancements with Bind 2: Faster connection times, especially crucial for mobile network users, thanks to Bind 2 and other performance optimizations.
– 🔄 User gains better control over on their messages: The new support for XEP-0424 : Message Retraction allows users to manage their message history and remove something they posted by mistake.
– 🔧 Optimized server pings by relying on an existing mechanism coming from XEP-0198
– 📈 Streamlined API Versioning: Our refined API versioning means smoother, more flexible integration for your applications.
– 🧩 Enhanced Elixir, Mix and Rebar3 Support

If you upgrade ejabberd from a previous release, please review those changes:

• Update the SQL schema
• Update API commands as explained below, or use API versioning
• Mix or Rebar3 used by default instead of Rebar to compile ejabberd
• Authentication workaround for Converse.js and Strophe.js

A more detailed explanation of those topics and other features:

Matrix federation

ejabberd is now able to federate with Matrix servers. Detailed instructions to setup Matrix federation with ejabberd will be detailed in another post.

Here is a quick summary of the configuration steps:

First, s2s must be enabled on ejabberd. Then define a listener that uses mod_matrix_gw :

listen:
  -
    port: 8448
    module: ejabberd_http
    tls: true
    certfile: "/opt/ejabberd/conf/server.pem"
    request_handlers:
      "/_matrix": mod_matrix_gw

And add mod_matrix_gw in your modules:

modules:
  mod_matrix_gw:
    matrix_domain: "domain.com"
    key_name: "somename"
    key: "yourkeyinbase64"

Support TLS 1.3, Bind 2, SASL2

• RFC 9266 Channel Bindings for TLS 1.3 : This enhances security and reliability.
• XEP-0386: Bind 2 : This is going to reduce the connection time for clients. This is especially important if you are using XMPP to connect from a mobile network.
• XEP-0388: Extensible SASL Profile – SASL2
• XEP-0440: SASL Channel-Binding Type Capability : These updates are aimed at bolstering our authentication mechanisms.
• XEP-0474: SASL SCRAM Downgrade Protection

Support for XEP-0424 Message Retraction

With the new support for XEP-0424: Message Retraction , users of MAM message archiving can control their message archiving, with the ability to ask for deletion.

Support for XEP-0198 pings

If stream management is enabled, let mod_ping trigger XEP-0198 <r/>equests rather than sending XEP-0199 pings. This avoids the overhead of the ping IQ stanzas, which, if stream management is enabled, are accompanied by XEP-0198 elements anyway.

Update the SQL schema

The table archive has a text column named origin_id (see commit 975681 ). You have two methods to update the SQL schema of your existing database:

If using MySQL or PosgreSQL, you can enable the option update_sql_schema and ejabberd will take care to update the SQL schema when needed: add in your ejabberd configuration file the line update_sql_schema: true

If you are using other database, or prefer to update manually the SQL schema:

• MySQL default schema:

ALTER TABLE archive ADD COLUMN origin_id text NOT NULL DEFAULT '';
ALTER TABLE archive ALTER COLUMN origin_id DROP DEFAULT;
CREATE INDEX i_archive_username_origin_id USING BTREE ON archive(username(191), origin_id(191));

• MySQL new schema:

ALTER TABLE archive ADD COLUMN origin_id text NOT NULL DEFAULT '';
ALTER TABLE archive ALTER COLUMN origin_id DROP DEFAULT;
CREATE INDEX i_archive_sh_username_origin_id USING BTREE ON archive(server_host(191), username(191), origin_id(191))

• PostgreSQL default schema:

ALTER TABLE archive ADD COLUMN origin_id text NOT NULL DEFAULT '';
ALTER TABLE archive ALTER COLUMN origin_id DROP DEFAULT;
CREATE INDEX i_archive_username_origin_id ON archive USING btree (username, origin_id);

• PostgreSQL new schema:

ALTER TABLE archive ADD COLUMN origin_id text NOT NULL DEFAULT '';
ALTER TABLE archive ALTER COLUMN origin_id DROP DEFAULT;
CREATE INDEX i_archive_sh_username_origin_id ON archive USING btree (server_host, username, origin_id);

• MSSQL default schema:

ALTER TABLE [dbo].[archive] ADD [origin_id] VARCHAR (250) NOT NULL;
CREATE INDEX [archive_username_origin_id] ON [archive] (username, origin_id)
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON);

• MSSQL new schema:

ALTER TABLE [dbo].[archive] ADD [origin_id] VARCHAR (250) NOT NULL;
CREATE INDEX [archive_sh_username_origin_id] ON [archive] (server_host, username, origin_id)
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON);

• SQLite default schema:

ALTER TABLE archive ADD COLUMN origin_id text NOT NULL DEFAULT '';
CREATE INDEX i_archive_username_origin_id ON archive (username, origin_id);

• SQLite new schema:

ALTER TABLE archive ADD COLUMN origin_id text NOT NULL DEFAULT '';
CREATE INDEX i_archive_sh_username_origin_id ON archive (server_host, username, origin_id);

Authentication workaround for Converse.js and Strophe.js

This ejabberd release includes support for XEP-0474: SASL SCRAM Downgrade Protection , and some clients may not support it correctly yet.

If you are using Converse.js 10.1.6 or older, Movim 0.23 Kojima or older, or any other client based in Strophe.js v1.6.2 or older, you may notice that they cannot authenticate correctly to ejabberd.

To solve that problem, either update to newer versions of those programs (if they exist), or you can enable temporarily the option disable_sasl_scram_downgrade_protection in the ejabberd configuration file ejabberd.yml like this:

disable_sasl_scram_downgrade_protection: true

Support for API versioning

Until now, when a new ejabberd release changed some API command (an argument renamed, a result in a different format…), then you had to update your API client to the new API at the same time that you updated ejabberd.

Now the ejabberd API commands can have different versions, by default the most recent one is used, and the API client can specify the API version it supports.

In fact, this feature was implemented seven years ago , included in ejabberd 16.04 , documented in ejabberd Docs: API Versioning … but it was never actually used!

This ejabberd release includes many fixes to get API versioning up to date, and it starts being used by several commands.

Let’s say that ejabberd 23.10 implemented API version 0, and this ejabberd 24.02 adds API version 1. You may want to update your API client to use the new API version 1… or you can continue using API version 0 and delay API update a few weeks or months.

To continue using API version 0:
– if using ejabberdctl, use the switch --version 0 . For example: ejabberdctl --version 0 get_roster admin localhost
– if using mod_http_api, in ejabberd configuration file add v0 to the request_handlers path. For example: /api/v0: mod_http_api

Check the details in ejabberd Docs: API Versioning .

ejabberd commands API version 1

When you want to update your API client to support ejabberd API version 1, those are the changes to take into account:
– Commands with list arguments
– mod_http_api does not name integer and string results
– ejabberdctl with list arguments
– ejabberdctl list results

All those changes are described in the next sections.

Commands with list arguments

Several commands now use list argument instead of a string with separators (different commands used different separators: ; : \\n , ).

The commands improved in API version 1:
– add_rosteritem
– oauth_issue_token
– send_direct_invitation
– srg_create
– subscribe_room
– subscribe_room_many

For example, srg_create in API version 0 took as arguments:

{"group": "group3",
 "host": "myserver.com",
 "label": "Group3",
 "description": "Third group",
 "display": "group1\\ngroup2"}

now in API version 1 the command expects as arguments:

{"group": "group3",
 "host": "myserver.com",
 "label": "Group3",
 "description": "Third group",
 "display": ["group1", "group2"]}

mod_http_api not named results

There was an incoherence in mod_http_api results when they were integer/string and when they were list/tuple/rescode…: the result contained the name, for example:

$ curl -k -X POST -H "Content-type: application/json" -d '{}' "http://localhost:5280/api/get_loglevel/v0"
{"levelatom":"info"}

Staring in API version 1, when result is an integer or a string, it will not contain the result name. This is now coherent with the other result formats (list, tuple, …) which don’t contain the result name either.

Some examples with API version 0 and API version 1:

$ curl -k -X POST -H "Content-type: application/json" -d '{}' "http://localhost:5280/api/get_loglevel/v0"
{"levelatom":"info"}

$ curl -k -X POST -H "Content-type: application/json" -d '{}' "http://localhost:5280/api/get_loglevel"
"info"

$ curl -k -X POST -H "Content-type: application/json" -d '{"name": "registeredusers"}' "http://localhost:5280/api/stats/v0"
{"stat":2}

$ curl -k -X POST -H "Content-type: application/json" -d '{"name": "registeredusers"}' "http://localhost:5280/api/stats"
2

$ curl -k -X POST -H "Content-type: application/json" -d '{"host": "localhost"}' "http://localhost:5280/api/registered_users/v0"
["admin","user1"]

$ curl -k -X POST -H "Content-type: application/json" -d '{"host": "localhost"}' "http://localhost:5280/api/registered_users"
["admin","user1"]

ejabberdctl with list arguments

ejabberdctl now supports list and tuple arguments, like mod_http_api and ejabberd_xmlrpc. This allows ejabberdctl to execute all the existing commands, even some that were impossible until now like create_room_with_opts and set_vcard2_multi .

List elements are separated with , and tuple elements are separated with : .

Relevant commands:
– add_rosteritem
– create_room_with_opts
– oauth_issue_token
– send_direct_invitation
– set_vcard2_multi
– srg_create
– subscribe_room
– subscribe_room_many

Some example uses:

ejabberdctl add_rosteritem user1 localhost testuser7 localhost NickUser77l gr1,gr2,gr3 both
ejabberdctl create_room_with_opts room1 conference.localhost localhost public:false,persistent:true
ejabberdctl subscribe_room_many user1@localhost:User1,admin@localhost:Admin room1@conference.localhost urn:xmpp:mucsub:nodes:messages,u

ejabberdctl list results

Until now, ejabberdctl returned list elements separated with ; . Now in API version 1 list elements are separated with , .

For example, in ejabberd 23.10:

$ ejabberdctl get_roster admin localhost
jan@localhost jan   none    subscribe       group1;group2
tom@localhost tom   none    subscribe       group3

Since this ejabberd release, using API version 1:

$ ejabberdctl get_roster admin localhost
jan@localhost jan   none    subscribe       group1,group2
tom@localhost tom   none    subscribe       group3

it is still possible to get the results in the old syntax, using API version 0:

$ ejabberdctl --version 0 get_roster admin localhost
jan@localhost jan   none    subscribe       group1;group2
tom@localhost tom   none    subscribe       group3

ejabberdctl help improved

ejabberd supports around 200 administrative commands, and probably you consult them in the ejabberd Docs -> API Reference page, where all the commands documentation is perfectly displayed…

The ejabberdctl command-line script already allowed to consult the commands documentation, consulting in real-time your ejabberd server to show you exactly the commands that are available. But it lacked some details about the commands. That has been improved, and now ejabberdctl shows all the information, including arguments description, examples and version notes.

For example, the connected_users_vhost command documentation as seen in the ejabberd Docs site is equivalently visible using ejabberdctl :

$ ejabberdctl help connected_users_vhost
  Command Name: connected_users_vhost

  Arguments: host::binary : Server name

  Result: connected_users_vhost::[ sessions::string ]

  Example: ejabberdctl connected_users_vhost "myexample.com"
           user1@myserver.com/tka
           user2@localhost/tka

  Tags: session

  Module: mod_admin_extra

  Description: Get the list of established sessions in a vhost

Experimental support for Erlang/OTP 27

Erlang/OTP 27.0-rc1 was recently released, and ejabberd can be compiled with it. If you are developing or experimenting with ejabberd, it would be great if you can use Erlang/OTP 27 and report any problems you find. For production servers, it’s recommended to stick with Erlang/OTP 26.2 or any previous version.

In this sense, the rebar and rebar3 binaries included with ejabberd are also updated: now they support from Erlang 24 to Erlang 27. If you want to use older Erlang versions from 20 to 23, there are compatible binaries available in git: rebar from ejabberd 21.12 and rebar3 from ejabberd 21.12 .

Of course, if you have rebar or rebar3 already installed in your system, it’s preferable if you use those ones, because probably they will be perfectly compatible with whatever erlang version you have installed.

Installers and ejabberd container image

The binary installers now include the recent and stable Erlang/OTP 26.2.2 and Elixir 1.16.1. Many other dependencies were updated in the installers, the most notable is OpenSSL that has jumped to version 3.2.1.

The ejabberd container image has gotten all those version updates, and also Alpine is updated to 3.19.

By the way, this container image already had support to run commands when the container starts … And now you can setup the commands to allow them fail, by prepending the character ! .

Summary of compilation methods

When compiling ejabberd from source code, you may have noticed there are a lot of possibilities. Let’s take an overview before digging in the new improvements:

• Tools to manage the dependencies and compilation:
  • Rebar : it is nowadays very obsolete, but still does the job of compiling ejabberd
  • Rebar3 : the successor of Rebar, with many improvements and plugins, supports hex.pm and Elixir compilation
  • Mix : included with the Elixir programming language , supports hex.pm, and erlang compilation
• Installation methods:
  • make install : copies the files to the system
  • make prod : prepares a self-contained OTP production release in _build/prod/ , and generates a tar.gz file. This was previously named make rel
  • make dev : prepares quickly an OTP development release in _build/dev/
  • make relive : prepares the barely minimum in _build/relive/ to run ejabberd and starts it
• Start scripts and alternatives:
  • ejabberdctl with erlang shell: start / foreground / live
  • ejabberdctl with elixir shell: iexlive
  • ejabberd console / start (this script is generated by rebar3 or mix, and does not support ejabberdctl configurable options)

For example:
– the CI dynamic tests use rebar3 , and Runtime tries to test all the possible combinations
– ejabberd binary installers are built using: mix + make prod
– container images are built using: mix + make prod too, and started with ejabberdctl foreground

Several combinations didn’t work correctly until now and have been fixed, for example:
– mix + make relive
– mix + make prod/dev + ejabberdctl iexlive
– mix + make install + ejabberdctl start/foregorund/live
– make uninstall buggy has an experimental alternative: make uninstall-rel
– rebar + make prod with Erlang 26

Use Mix or Rebar3 by default instead of Rebar to compile ejabberd

ejabberd uses Rebar to manage dependencies and compilation since ejabberd 13.10 4d8f770 . However, that tool is obsolete and unmaintained since years ago, because there is a complete replacement:

Rebar3 is supported by ejabberd since 20.12 0fc1aea . Among other benefits, this allows to download dependencies from hex.pm and cache them in your system instead of downloading them from git every time, and allows to compile Elixir files and Elixir dependencies.

In fact, ejabberd can be compiled using mix (a tool included with the Elixir programming language ) since ejabberd 15.04 ea8db99 (with improvements in ejabberd 21.07 4c5641a )

For those reasons, the tool selection performed by ./configure will now be:
– If --with-rebar=rebar3 but Rebar3 not found installed in the system, use the rebar3 binary included with ejabberd
– Use the program specified in option: --with-rebar=/path/to/bin
– If none is specified, use the system mix
– If Elixir not found, use the system rebar3
– If Rebar3 not found, use the rebar3 binary included with ejabberd

Removed Elixir support in Rebar

Support for Elixir 1.1 was added as a dependency in commit 01e1f67 to ejabberd 15.02 . This allowed to compile Elixir files. But since Elixir 1.4.5 (released Jun 22, 2017) it isn’t possible to get Elixir as a dependency… it’s nowadays a standalone program. For that reason, support to download old Elixir 1.4.4 as a dependency has been removed.

When Elixir support is required, better simply install Elixir and use mix as build tool:

./configure --with-rebar=mix

Or install Elixir and use the experimental Rebar3 support to compile Elixir files and dependencies:

./configure --with-rebar=rebar3 --enable-elixir

Added Elixir support in Rebar3

It is now possible to compile ejabberd using Rebar3 and support Elixir compilation. This compiles the Elixir files included in ejabberd’s lib/ path. There’s also support to get dependencies written in Elixir, and it’s possible to build OTP releases including Elixir support.

It is necessary to have Elixir installed in the system, and configure the compilation using --enable-elixir . For example:

apt-get install erlang erlang-dev elixir
git clone https://github.com/processone/ejabberd.git ejabberd
cd ejabberd
./autogen.sh
./configure --with-rebar=rebar3 --enable-elixir
make
make dev
_build/dev/rel/ejabberd/bin/ejabberdctl iexlive

Elixir versions supported

Elixir 1.10.3 is the minimum supported, but:
– Elixir 1.10.3 or higher is required to build an OTP release with make prod or make dev
– Elixir 1.11.4 or higher is required to build an OTP release if using Erlang/OTP 24 or higher
– Elixir 1.11.0 or higher is required to use make relive
– Elixir 1.13.4 with Erlang/OTP 23.0 are the lowest versions tested by Runtime

For all those reasons, if you want to use Elixir, it is highly recommended to use Elixir 1.13.4 or higher with Erlang/OTP 23.0 or higher.

make rel is renamed to make prod

When ejabberd started to use Rebar2 build tool, that tool could create an OTP release, and the target in Makefile.in was conveniently named make rel .

However, newer tools like Rebar3 and Elixir’s Mix support creating different types of releases: production, development, … In this sense, our make rel target is nowadays more properly named make prod .

For backwards compatibility, make rel redirects to make prod .

New make install-rel and make uninstall-rel

This is an alternative method to install ejabberd in the system, based in the OTP release process. It should produce exactly the same results than the existing make install .

The benefits of make install-rel over the existing method:
– this uses OTP release code from rebar/rebar3/mix, and consequently requires less code in our Makefile.in
– make uninstall-rel correctly deletes all the library files

This is still experimental, and it would be great if you are able to test it and report any problem; eventually this method could replace the existing one.

Just for curiosity:
– ejabberd 13.03-beta1 got support for make uninstall was added
– ejabberd 13.10 introduced Rebar build tool and code got more modular
– ejabberd 15.10 started to use the OTP directory structure for ‘make install’ , and this broke make uninstall

Acknowledgments

We would like to thank the contributions to the source code, documentation, and translation provided for this release by:

• Holger Weiß for the support XEP-0198 pings and XEP-0425 fixes
• Mr. EddX , updated the Bulgarian translation
• Sketch6580 , updated the Chinese translation
• Jan Aschenbrenner , updated the Czech translation
• Ranforingus , updated the Dutch translation
• Ermete Melchiorre , updated the Italian translation
• Mako N , updated the Japanese translation
• Silvério Santos , updated the Portuguese translation
• Олександр Кревський , updated the Ukrainian translation

And also to all the people contributing in the ejabberd chatroom, issue tracker…

Improvements in ejabberd Business Edition

Customers of the ejabberd Business Edition , in addition to all those improvements and bugfixes, also get:

Push

• Fix clock issue when signing Apple push JWT tokens
• Share Apple push JWT tokens between nodes in cluster
• Increase allowed certificates chain depth in GCM requests
• Use x:oob data as source for image delivered in pushes
• Process only https urls in oob as images in pushes
• Fix jid in disable push iq generated by GCM and Webhook service
• Add better logging for TooManyProviderTokenUpdated error
• Make get_push_logs command generate better error if mod_push_logger not available
• Add command get_push_logs that can be used to retrieve info about recent pushes and errors reported by push services
• Add support for webpush protocol for sending pushes to safari/chrome/firefox browsers

MAM

• Expand mod_mam_http_access API to also accept range of messages

MUC

• Update mod_muc_state_query to fix subject_author room state field
• Fix encoding of config xdata in mod_muc_state_query

PubSub

• Allow pubsub node owner to overwrite items published by other persons (p1db)

ChangeLog

This is a more detailed list of changes in this ejabberd release:

Core

• Added Matrix gateway in mod_matrix_gw
• Support SASL2 and Bind2
• Support tls-server-end-point channel binding and sasl2 codec
• Support tls-exporter channel binding
• Support XEP-0474: SASL SCRAM Downgrade Protection
• Fix presenting features and returning results of inline bind2 elements
• disable_sasl_scram_downgrade_protection : New option to disable XEP-0474
• negotiation_timeout : Increase default value from 30s to 2m
• mod_carboncopy: Teach how to interact with bind2 inline requests

Other

• ejabberdctl: Fix startup problem when having set EJABBERD_OPTS and logger options
• ejabberdctl: Set EJABBERD_OPTS back to "" , and use previous flags as example
• eldap: Change logic for eldap tls_verify=soft and false
• eldap: Don’t set fail_if_no_peer_cert for eldap ssl client connections
• Ignore hints when checking for chat states
• mod_mam: Support XEP-0424 Message Retraction
• mod_mam: Fix XEP-0425: Message Moderation with SQL storage
• mod_ping: Support XEP-0198 pings when stream management is enabled
• mod_pubsub: Normalize pubsub max_items node options on read
• mod_pubsub: PEP nodetree: Fix reversed logic in node fixup function
• mod_pubsub: Only care about PEP bookmarks options when creating node from scratch

SQL

• MySQL: Support sha256_password auth plugin
• ejabberd_sql_schema: Use the first unique index as a primary key
• Update SQL schema files for MAM’s XEP-0424
• New option sql_flags : right now only useful to enable mysql_alternative_upsert

Installers and Container

• Container: Add ability to ignore failures in execution of CTL_ON_* commands
• Container: Update to Erlang/OTP 26.2, Elixir 1.16.1 and Alpine 3.19
• Container: Update this custom ejabberdctl to match the main one
• make-binaries: Bump OpenSSL 3.2.1, Erlang/OTP 26.2.2, Elixir 1.16.1
• make-binaries: Bump many dependency versions

Commands API

• print_sql_schema : New command available in ejabberdctl command-line script
• ejabberdctl: Rework temporary node name generation
• ejabberdctl: Print argument description, examples and note in help
• ejabberdctl: Document exclusive ejabberdctl commands like all the others
• Commands: Add a new muc_sub tag to all the relevant commands
• Commands: Improve syntax of many commands documentation
• Commands: Use list arguments in many commands that used separators
• Commands: set_presence : switch priority argument from string to integer
• ejabberd_commands: Add the command API version as a tag vX
• ejabberd_ctl: Add support for list and tuple arguments
• ejabberd_xmlrpc: Fix support for restuple error response
• mod_http_api: When no specific API version is requested, use the latest

Compilation with Rebar3/Elixir/Mix

• Fix compilation with Erlang/OTP 27: don’t use the reserved word ‘maybe’
• configure: Fix explanation of --enable-group option ( #4135 )
• Add observer and runtime_tools in releases when --enable-tools
• Update “make translations” to reduce build requirements
• Use Luerl 1.0 for Erlang 20, 1.1.1 for 21-26, and temporary fork for 27
• Makefile: Add install-rel and uninstall-rel
• Makefile: Rename make rel to make prod
• Makefile: Update make edoc to use ExDoc, requires mix
• Makefile: No need to use escript to run rebar|rebar3|mix
• configure: If --with-rebar=rebar3 but rebar3 not system-installed, use local one
• configure: Use Mix or Rebar3 by default instead of Rebar2 to compile ejabberd
• ejabberdctl: Detect problem running iex or etop and show explanation
• Rebar3: Include Elixir files when making a release
• Rebar3: Workaround to fix protocol consolidation
• Rebar3: Add support to compile Elixir dependencies
• Rebar3: Compile explicitly our Elixir files when --enable-elixir
• Rebar3: Provide proper path to iex
• Rebar/Rebar3: Update binaries to work with Erlang/OTP 24-27
• Rebar/Rebar3: Remove Elixir as a rebar dependency
• Rebar3/Mix: If dev profile/environment, enable tools automatically
• Elixir: Fix compiling ejabberd as a dependency ( #4128 )
• Elixir: Fix ejabberdctl start/live when installed
• Elixir: Fix: FORMATTER ERROR: bad return value ( #4087 )
• Elixir: Fix: Couldn’t find file Elixir Hex API
• Mix: Enable stun by default when vars.config not found
• Mix: New option vars_config_path to set path to vars.config ( #4128 )
• Mix: Fix ejabberdctl iexlive problem locating iex in an OTP release

Full Changelog

https://github.com/processone/ejabberd/compare/23.10…24.02

ejabberd 24.02 download & feedback

As usual, the release is tagged in the Git source code repository on GitHub .

The source package and installers are available in ejabberd Downloads page. To check the *.asc signature files, see How to verify ProcessOne downloads integrity .

For convenience, there are alternative download locations like the ejabberd DEB/RPM Packages Repository and the GitHub Release / Tags .

The ecs container image is available in docker.io/ejabberd/ecs and ghcr.io/processone/ecs . The alternative ejabberd container image is available in ghcr.io/processone/ejabberd .

If you consider that you’ve found a bug, please search or fill a bug report on GitHub Issues .

The post ejabberd 24.02 first appeared on ProcessOne .

What is Elixir: Exploring its Functional Programming Essence

In our latest post, we’ll be exploring Elixir, a robust programming language known for its concurrency and fault-tolerance capabilities. We’ll look at some of Elixir’s syntax, and core features, as well as the Elixir community some resources for beginners and enthusiasts alike.

The birth of Elixir

As the brainchild of José Valim, Elixir is rooted in Valim’s experiences with Ruby on Rails and Erlang. Elixir aimed to tackle the challenges of building scalable and fault-tolerant applications.

Harnessing the power of the Erlang Virtual Machine (VM), Elixir inherits its renowned traits of low latency, distributed computing, and fault tolerance. This foundation empowers developers to create robust systems capable of handling demanding workloads across diverse industries.

Erlang VM and Virtual Machine Process

Elixir’s versatility extends far beyond its roots. With its powerful tooling and ecosystem, Elixir facilitates productivity in various domains, including web development, embedded software, machine learning, data pipelines, and multimedia processing. Its flexibility and efficiency make it an ideal choice for tackling an array of challenges in today’s tech landscape.

Elixir and Erlang: A powerful duo

Elixir, a robust programming language, collaborates closely with Erlang, renowned for building fault-tolerant and distributed systems. Developed by Ericsson in the late 1980s, Erlang initially targeted telecommunications applications, prioritising reliability and uninterrupted service.

A key element driving the synergy between Elixir and Erlang is the BEAM, a virtual environment proficient in executing code written in both languages. Elixir, uniquely, is constructed directly atop the BEAM, inheriting its capacity for highly concurrent and fault-tolerant runtime operations. This integration fosters seamless interoperability between Elixir and Erlang applications, ensuring optimal performance and reliability

Elixir on BEAM

Elixir benefits significantly from Erlang’s robust framework, leveraging its scalability, fault tolerance, and distributed processing capabilities. This makes Elixir a preferred choice in industries where system uptime is paramount. Additionally, Elixir developers gain access to Erlang’s established ecosystem and libraries, simplifying the development of resilient and scalable systems.

It also introduces contemporary syntax and language features, enhancing developer productivity and code expressiveness. This modernisation, combined with Erlang’s robust runtime, empowers developers to navigate the complexities of today’s software landscape confidently, delivering efficient and reliable solutions.

Furthermore, Elixir introduces contemporary syntax and language features, enhancing developer productivity and code expressiveness. This modernisation, combined with Erlang’s robust runtime, empowers developers to navigate today’s complex software landscape with confidence, delivering efficient and reliable solutions.

Elixir and Erlang are a formidable duo, Complementing each other’s strengths to empower developers in crafting dependable, scalable, and fault-tolerant systems with ease and effectiveness.

Elixir’s syntax and language features

Elixir boasts a clean and expressive syntax inspired by Ruby, with a focus on developer productivity and readability. Its language features are designed to promote conciseness and clarity, making it an ideal choice for both beginners and experienced developers alike.

• Concurrency with Erlang processes: Elixir utilises lightweight Erlang processes for concurrency. These processes communicate via message passing, facilitating highly concurrent and fault-tolerant systems.

• Immutable data: Elixir promotes immutability, ensuring that once data is created, it cannot be changed. This simplifies code reasoning and mitigates unexpected side effects.

• Pattern matching : A core feature, pattern matching allows developers to destructure data and match it against predefined patterns, leading to concise and elegant code.

• Functions as first-class entities: Functions can be assigned to variables, passed as arguments, and returned from other functions, enabling powerful abstractions and composition.

• Metaprogramming with macros: Elixir offers metaprogramming capabilities through macros, empowering developers to generate and manipulate code at compile time. This facilitates the creation of domain-specific languages and powerful abstractions.

• Fault Tolerance via Supervision Trees: Elixir adopts Erlang’s “let it crash” philosophy, isolating processes and containing failures. Supervision trees structure and manage process supervision, ensuring robust fault tolerance.

• OTP for scalability and reliability: Elixir includes OTP, providing libraries and best practices for building scalable, fault-tolerant, and distributed systems. OTP features such as gen_servers and supervisors enhance system reliability.

• Comprehensive tooling and documentation: Elixir offers a rich set of tools for development, testing, and deployment. Mix, the build tool, manages dependencies and runs tests, while ExDoc simplifies documentation creation and maintenance.

Understanding functional programming in Elixir

In Elixir, functional programming prioritises pure functions, immutable data, and higher-order functions. It encourages writing code clearly and expressively, treating functions as primary elements that can be passed as arguments or returned as results.

Elixir’s functional programming paradigm supports the development of robust, scalable, and fault-tolerant systems, which makes it an excellent option for creating distributed and concurrent applications.

Robustness :

• Elixir’s functional approach reduces bugs and promotes code clarity.
• Problems are solved in smaller, testable units.

Scalability:

• Elixir’s lightweight processes enable easy concurrency.
• Systems can scale across multiple cores or nodes effortlessly.

Fault Tolerance:

• Elixir’s supervision tree ensures system resilience.
• Failures are isolated and managed, keeping the system running.

Concurrency:

• Elixir’s processes communicate asynchronously.
• Concurrent operations are efficient and responsive.

Distribution:

• Elixir applications can easily scale across multiple nodes.
• Distributed computing is simplified, enabling high availability.

Practical applications of Elixir

Elixir’s versatility and robust features make it a powerful language for developing a wide range of applications across different domains. From web development to distributed systems and embedded devices, Elixir’s concurrency, fault tolerance, and scalability enable developers to build resilient and efficient solutions. Here are some practical applications where Elixir shines:

Web development with Phoenix Framework

Phoenix, Elixir’s web framework, offers high-performance solutions for modern web apps. Leveraging Elixir’s concurrency and fault tolerance, Phoenix scales effortlessly to handle concurrent connections and real-time features.

Elixir Phoenix

Distributed systems and microservices

Elixir’s lightweight processes and distribution support make it ideal for distributed systems and microservices. Its fault-tolerant supervision trees ensure system reliability and scalability across multiple nodes.

Embedded systems and IoT

Elixir’s small footprint and low-latency performance suit embedded systems and IoT. With Nerves, developers can deploy Elixir to devices like Raspberry Pi, ensuring fault tolerance and resilience.

Real-time messaging and chat applications

Elixir’s concurrency and real-time support make it perfect for messaging apps. Libraries like Phoenix Channels enable scalable and fault-tolerant chat systems, handling numerous concurrent users seamlessly.

Financial and e-commerce systems

Elixir’s reliability and scalability are beneficial for financial and e-commerce platforms. Its fault-tolerant supervision ensures uninterrupted processing of transactions. Frameworks like Broadway facilitate scalable data processing for large transaction volumes.

What is Elixir used for: Real-world examples

Elixir, with its powerful features and versatile ecosystem, finds applications in a host of real-world scenarios across different industries:

Discord: Community-driven communication

Discord, serving over 150 million monthly active users, relies on Elixir for seamless voice and text chat experiences in gaming and educational communities.

Pinterest: Scalable backend services

Pinterest, with over 450 million monthly active users, employs Elixir for its backend services, handling millions of user interactions and content updates daily.

Deliveroo: Reliable food delivery services

Deliveroo, operating in over 800 cities globally, employs Elixir for its backend systems, ensuring reliable food delivery services for millions of customers worldwide.

Bleacher Report: Real-time sports updates

Bleacher Report delivers real-time sports updates and news to over 40 million monthly active users, leveraging Elixir for efficient data processing and content delivery.

PepsiCo: Supply chain optimisation

PepsiCo, one of the world’s largest food and beverage companies, uses Elixir to optimise its supply chain operations, ensuring efficient distribution of products across its global network.

Scalable web applications with Elixir

When you’re making mobile or web apps, scalability is key. If your app can’t handle more users, you might lose them. Plus, you could miss out on chances for growth.

Scalability also matters financially. If your app can’t grow smoothly, you’ll end up spending more on infrastructure. That’s where Elixir comes in. It’s a powerful language for building apps that can handle lots of users. Elixir is used in different areas such as gaming and e-commerce.

It’s like combining the best of OCaml and Haskel languages. With strong tools and a helpful community, Elixir is perfect for making apps that can grow with businesses.

Elixir’s ecosystem and community

Elixir’s success isn’t just attributed to its language features, but also to its vibrant ecosystem and supportive community. With a growing collection of libraries, tools, and resources, Elixir’s ecosystem continues to expand, making it easier for developers to build and maintain Elixir applications.

Tools and libraries enhancing Elixir development

Elixir boasts a rich ecosystem of libraries and tools that cover a wide range of functionalities, from web development and database integration to concurrency and distributed computing. The Phoenix web framework, for instance, provides a robust foundation for building scalable and real-time web applications, while Ecto offers a powerful database abstraction layer for interacting with databases in Elixir applications. Other notable libraries include Broadway for building concurrent and fault-tolerant data processing pipelines, and Nerves for developing embedded systems and IoT applications.

The growing community and learning resources

One of Elixir’s greatest strengths is its supportive and inclusive community. From online forums and chat rooms to local meetups and conferences, Elixir enthusiasts have numerous avenues to connect, learn, and collaborate with fellow developers.
The Elixir Forum and the Elixir Slack community are popular online hubs where developers can seek help, share knowledge, and discuss best practices.

Additionally, ElixirConf, the annual conference dedicated to Elixir and Erlang, provides a platform for developers to network, attend talks and workshops, and stay up-to-date with the latest developments in the Elixir ecosystem.

Conclusion

Elixir’s unique blend of concurrency, fault tolerance, and scalability makes it a powerful language for modern application development. Its clean syntax and functional programming principles enhance developer productivity and code maintainability.

Looking ahead, Elixir’s future in software development is promising. With the growing demand for distributed systems and data-intensive applications, Elixir’s strengths position it well for continued growth and innovation.

Supported by an active community and ongoing contributions, Elixir is set to play a significant role in shaping the future of software development. Whether you’re a seasoned developer or new to Elixir, its potential is boundless in the dynamic landscape of software engineering.

Further reading and resources

Books on Elixir

• “ Programming Elixir” by Dave Thomas : This book offers a comprehensive introduction to Elixir, covering its syntax, features, and best practices for building robust applications.
• “Elixir in Action” by Saša Jurić: A practical guide to Elixir programming, covering topics such as concurrency, distributed computing, and building scalable applications.
• “The Little Elixir & OTP Guidebook” by Benjamin Tan Wei Hao: This book provides a gentle introduction to Elixir’s concurrency model and the OTP framework, essential for building fault-tolerant and distributed systems.

Online Tutorials on Elixir

• Elixir School: A free online resource offering interactive lessons and tutorials on Elixir programming, suitable for beginners and experienced developers alike.

The Complete Elixir and Phoenix Bootcamp

Master Functional Programming techniques with Elixir and Phoenix while learning to build compelling web applications!

Community forums and support for Elixir learners

• Elixir Forum : A vibrant online community for Elixir enthusiasts to ask questions, share knowledge, and discuss topics related to Elixir programming and ecosystem.
• Elixir Slack Community: (https://elixir-slackin.herokuapp.com/): Join the Elixir Slack community to connect with fellow developers, ask for help, and engage in discussions on all things Elixir.
• Reddit r/elixir: The Elixir subreddit is a place to share news, articles, and questions about the Elixir programming language and ecosystem.

These resources provide a solid foundation for learning the Elixir programming language and engaging with the vibrant Elixir community. Whether you’re just starting your journey with Elixir or looking to deepen your knowledge, these books, tutorials, and forums offer valuable insights and support along the way.

The post What is Elixir? appeared first on Erlang Solutions .