There are a lot of things that go into building a chat system, such as client, server, and protocol.  Even for only making a client there are lots of areas of focus, such as user experience, features, and performance.  To keep this post a manageable size, we will just be building a client and will use an existing server and protocol (accessing Jabber network services using the XMPP protocol).  We’ll make a practical GUI so we can test things, but not spend too much time on polish, and look at getting to a useful baseline of features.

You can find all the code for this post in git .  All code licensed AGPL3+ .

Use a Library

As with most large programming tasks, if we wanted to do every single thing ourselves we would spend a lot more time, so we should find some good libraries.  There is another reason to use a library: any improvements we make to the library benefits others.  While releasing our code might help someone else if they choose to read it, a library improvement can be picked up by users of that library right away.

We need to speak the XMPP protocol so let’s choose Blather .  We need a GUI so we can see this working, but don’t really want to futz with it much so let’s choose Glimmer .  The code here will use these libraries and be written in the Ruby programming language, but these ideas are general purpose to the task and hopefully we won’t get too bogged down in syntax specifics.

One little language-specific thing you will need to create is a description of which ruby packages are being used, so let’s make that file (named Gemfile ):

Gemfile

source "https://rubygems.org"

gem "blather", git: "https://github.com/adhearsion/blather", branch: "develop"
gem "glimmer-dsl-libui", "~> 0.5.24"

Run this to get the packages installed:

bundle install --path=.gems

Let’s get the bare minimum: a connection to a Jabber service and a window.

client.rb

require "glimmer-dsl-libui"
require "blather/client"

BLATHER = self
include Glimmer

Thread.new do
	window("Contacts") {
		on_destroy {
			BLATHER.shutdown
		}
	}
end

When required in this way, Blather will automatically set up a connection with event processing on the main thread, and will process command line arguments to get connection details.  So we put the GUI on a second thread to not have them block each other.  When the window is closed ( on_destroy ), be sure to disconnect from the server too.  You can run this barely-a-client like this:

bundle exec ruby client.rb user@example.com password

The arguments are a Jabber ID (which you can get from many existing services ), and the associated password.

You should get a blank window and no errors in your terminal.  If you wanted to you could even look in another client and confirm that it is connected to the account by seeing it come online.

Show a Contact List

Let’s fetch the user’s contacts from the server and show them in the window (if you use this with a new, blank test account there won’t be any contacts yet of course, but still).

$roster = [["", ""]]

Thread.new do
	window("Contacts") {
		vertical_box {
			table {
				button_column("Contact") {
				}
				editable false
				cell_rows $roster
			}
		}

		on_destroy {
			BLATHER.shutdown
		}
    }.show
end

after(:roster) do
	LibUI.queue_main do
		$roster.clear
		my_roster.each do |item|
			$roster << [item.name || item.jid, item.jid]
		end
	end
end

In a real app you would probably want some kind of singleton object to represent the contacts window and the contact list (“roster”) etc.  For simplicity here we just use a global variable for the roster, starting with some dummy data so that the GUI framework knows what it will look like, etc.

We fill out the window from before a little bit to have a table with a column of buttons, one for each contact.  The button_column is the first (and in this case, only) column definition so it will source data from the first element of each item in cell_rows .  It’s not an editable table, and it gets its data from the global variable.

We then add an event handler to our XMPP connection to say that once the roster has been loaded from the server, we hand control over to the GUI thread and there we clear out the global variable and fill it up with the roster as we now see it.  The first item in each row is the name that will be shown on the button (either item.name or item.jid if there is no name set), the second item is the Jabber ID which won’t be shown because we didn’t define that column when we made the window.  Any updates to the global variable will be automatically painted into the GUI so we’re done.

One Window Per Conversation

For simplicity, let’s say we want to show one window per conversation, like so:

$conversations = {}

class Conversation
	include Glimmer

	def self.open(jid, m=nil)
		return if $conversations[jid]

		($conversations[jid] = new(jid, m)).launch
	end

	def initialize(jid, m=nil)
		@jid = jid
		@messages = [["", ""]]
		new_message(m) if m
	end

	def launch
		window("Conversation With #{@jid}") {
			vertical_box {
				table {
					text_column("Sender")
					text_column("Message")
					editable false
					cell_rows @messages
					@messages.clear
				}

				horizontal_box {
					stretchy false

					@message_entry = entry
					button("Send") {
						stretchy false

						on_clicked do
							BLATHER.say(@jid, @message_entry.text)
							@messages << [ARGV[0], @message_entry.text]
							@message_entry.text = ""
						end
					}
				}
			}

			on_closing do
				$conversations.delete(@jid)
			end
		}.show
	end

	def format_sender(jid)
		BLATHER.my_roster[jid]&.name || jid
	end

	def message_row(m)
		[
			format_sender(m.from&.stripped || BLATHER.jid.stripped),
			m.body
		]
	end

	def new_message(m)
		@messages << message_row(m)
	end
end

message :body do |m|
	LibUI.queue_main do
		conversation = $conversations[m.from.stripped.to_s]
		if conversation
			conversation.new_message(m)
		else
			Conversation.open(m.from.stripped.to_s, m)
		end
	end
end

Most of this is the window definition again, with a table of the messages in this conversation sourced from an instance variable @messages .  At the bottom of the window is an entry box to type in text and a button to trigger sending it as a message.  When the button is clicked, send that message to the contact this conversation is with, add it to the list of messages so that it shows up in the GUI, and make the entry box empty again.  When the window closes ( on_closing this time because it’s not the “main” window) delete the object from the global set of open conversations.

This object also has a helper to open a conversation window if there isn’t already one with a given Jabber ID (jid), some helpers to format message objects into table rows by extracting the sender and body (including format_sender which gets the roster item if there is one, uses &.name to get the name if there was a roster item or else nil , and if there was no roster item or no name just show jid ) and a helper that adds new messages into the GUI.

Finally we add a new XMPP event handler for incoming messages that have a body.  Any such incoming message we look up in the global if there is a conversation open already, if so we pass the new message there to have it appended to the GUI table, otherwise we open the conversation with this message as the first thing it will show.

Getting from the Contact List to a Conversation

Now we wire up the contact list to the conversation view:

button_column("Contact") {
	on_clicked do |row|
		Conversation.open($roster[row][1].to_s)
	end
}

When a contact button is clicked, grab the Jabber ID from the hidden end of the table row that we had stashed there, and open the conversation.

horizontal_box {
	stretchy false

	jid_entry = entry {
		label("Jabber ID")
	}

	button("Start Conversation") {
		stretchy false

		on_clicked do
			Conversation.open(jid_entry.text)
		end
	}
}

And let’s provide a way to start a new conversation with an address that isn’t a contact too.  An entry to type in a Jabber ID and a button that opens the conversation.

Adding a Contact

Might as well add a button to the main window that re-uses that entry box to allow adding a contact as well:

button("Add Contact") {
	stretchy false

	on_clicked do
		BLATHER.my_roster << jid_entry.text
	end
}

Handling Multiple Devices

In many chat protocols, it is common to have multiple devices or apps connected simultaneously. It is often desirable to show messages sent to or from one device on all the others as well.  So let’s implement that.  First, a helper for creating XML structures we may need:

def xml_child(parent, name, namespace)
	child = Niceogiri::XML::Node.new(name, parent.document, namespace)
	parent << child
	child
end

We need to tell the server that we support this feature:

when_ready do
	self << Blather::Stanza::Iq.new(:set).tap { |iq|
		xml_child(iq, :enable, "urn:xmpp:carbons:2")
	}
end

We will be handling live messages from multiple event handlers so let’s pull the live message handling out into a helper:

def handle_live_message(m, counterpart: m.from.stripped.to_s)
	LibUI.queue_main do
		conversation = $conversations[counterpart]
		if conversation
			conversation.new_message(m)
		else
			Conversation.open(counterpart, m)
		end
	end
end

And the helper that will handle messages from other devices of ours:

def handle_carbons(fwd, counterpart:)
	fwd = fwd.first if fwd.is_a?(Nokogiri::XML::NodeSet)
	return unless fwd

	m = Blather::XMPPNode.import(fwd)
	return unless m.is_a?(Blather::Stanza::Message) && m.body.present?

	handle_live_message(m, counterpart: counterpart.call(m))
end

This takes in the forwarded XML object (allowing for it to be a set of which we take the first one) and imports it with Blather’s logic to become hopefully a Message object.  If it’s not a Message or has no body, we don’t really care so we stop there. Otherwise we can handle this extracted message as though we had received it ourselves.

And then wire up the event handlers:

message(
	"./carbon:received/fwd:forwarded/*[1]",
	carbon: "urn:xmpp:carbons:2",
	fwd: "urn:xmpp:forward:0"
) do |_, fwd|
	handle_carbons(fwd, counterpart: ->(m) { m.from.stripped.to_s })
end

Because XMPP is just XML, we can use regular XPath stuff to extract from incoming messages.  Here we say that if the message contains a forwarded element inside a carbons received element, then we should handle this with the carbons handler instead of just the live messages handler.  The XML that matches our XPath comes in as the second argument and that is what we pass to the handler to get converted into a Message object.

message(
	"./carbon:sent/fwd:forwarded/*[1]",
	carbon: "urn:xmpp:carbons:2",
	fwd: "urn:xmpp:forward:0"
) do |_, fwd|
	handle_carbons(fwd, counterpart: ->(m) { m.to.stripped.to_s })
end

This handler is for messages sent by other devices instead of received by other devices.  It is pretty much the same, except that we know the “other side of the conversation” (here called counterpart) is in the to not the from.

message :body do |m|
	handle_live_message(m)
end

And our old message-with-body handler now just needs to call the helper.

History

So far our client only processes and displays live messages.  If you close the app, or even close a conversation window, the history is gone.  If you chat with another client or device, you can’t see that when you re-open this one.  To fix that we’ll need to store messages persistently, and also fetch any history from while we were disconnected from the server.  We will need a few more lines in our Gemfile first:

gem "sqlite3"
gem "xdg"

And then to set up a basic database schema:

require "securerandom"
require "sqlite3"
require "xdg"

DATA_DIR = XDG::Data.new.home + "jabber-client-demo"
DATA_DIR.mkpath
DB = SQLite3::Database.new(DATA_DIR + "db.sqlite3")

if DB.user_version < 1
	DB.execute(<<~SQL)
		CREATE TABLE messages (
			mam_id TEXT PRIMARY KEY,
			stanza_id TEXT NOT NULL,
			conversation TEXT NOT NULL,
			created_at INTEGER NOT NULL,
			stanza TEXT NOT NULL
		)
	SQL
	DB.execute("CREATE TABLE data (key TEXT PRIMARY KEY, value TEXT)")
	DB.user_version = 1
end

user_version is a SQLite feature that allows storing a simple integer alongside the database.  It starts at 0 if never set, and so here we use it to check if our schema has been created or not.  We store the database in a new directory created according to the XDG Base Directory specification .  There are two relevant IDs for most XMPP operations: the MAM ID (the ID in the server’s archive) and the Stanza ID (which was usually selected by the original sender).  We also create a data table for storing basic key-value stuff, which we’ll use in a minute to remember where we have sync’d up to so far.  Let’s edit the Conversation object to store messages as we send them, updating the send button on_clicked handler:

def message
	Blather::Stanza::Message.new(@jid, @message_entry.text, :chat).tap { |m|
		m.id = SecureRandom.uuid
	}
end

…

on_clicked do
	m = message
	EM.defer do
		BLATHER << m
		DB.execute(<<~SQL, [nil, m.id, @jid, m.to_s])
			INSERT INTO messages
			(mam_id, stanza_id, conversation, created_at, stanza)
			VALUES (?,?,?,unixepoch(),?)
		SQL
	end
	@messages << message_row(m)
	@message_entry.text = ""
end

When we send a message we don’t yet know the server’s archive ID, so we set that to nil for now.  We set mam_id to be the primary key, but SQLite allows multiple rows to have NULL in there so this will work.  We don’t want to block the GUI thread while doing database work so we use EM.defer to move this to a worker pool.  We also want to store messages when we receive them live, so add this to the start of handle_live_message :

mam_id = m.xpath("./ns:stanza-id", ns: "urn:xmpp:sid:0").find { |el|
	el["by"] == jid.stripped.to_s
}&.[]("id")
delay = m.delay&.stamp&.to_i || Time.now.to_i
DB.execute(<<~SQL, [mam_id, m.id, counterpart, delay, m.to_s])
	INSERT INTO messages (mam_id, stanza_id, conversation, created_at, stanza)
    VALUES (?,?,?,?,?)
SQL

Here we extract the server archive’s ID for the message (added by the server in a stanza-id with by="Your Jabber ID" ) and figure out what time the message was originally sent (usually this is just right now for a live message, but if it is coming from offline storage because every client was offline or similar, then there can be a “delay” set on it which we can use).  Now that we have stored the history of message we received we need to load them into the GUI when we start up a Conversation so add this at the end of initialize:

EM.defer do
	mam_messages = []
	query = <<~SQL
		SELECT stanza
		FROM messages
		WHERE conversation=?
		ORDER BY created_at
	SQL
	DB.execute(query, [@jid]) do |row|
		m = Blather::XMPPNode.import(
			Nokogiri::XML.parse(row[0]).root
		)
		mam_messages << m
	end

	LibUI.queue_main do
		mam_messages.map! { |m| message_row(m) }
		@messages.replace(mam_messages + @messages)
	end
end

In the worker pool we load up all the stored messages for the current conversation in order, then we take the XML stored as a string in the database and parse it into a Blather Message object.  Once we’ve done as much of the work as we can in we worker pool we use queue_main to switch back to the GUI thread and actually build the rows for the table and replace them into the GUI.

With these changes, we are now storing all messages we see while connected and displaying them in the conversation.  But what about messages sent or received by other devices or clients while we were not connected?  For that we need to sync with the server’s archive, fetching messages at a reasonable page size from whatever we already have until the end.

def sync_mam(last_id)
	start_mam = Blather::Stanza::Iq.new(:set).tap { |iq|
		xml_child(iq, :query, "urn:xmpp:mam:2").tap do |query|
			xml_child(query, :set, "http://jabber.org/protocol/rsm").tap do |rsm|
				xml_child(rsm, :max, "http://jabber.org/protocol/rsm").tap do |max|
					max.content = (EM.threadpool_size * 5).to_s
				end
				next unless last_id

				xml_child(rsm, :after, "http://jabber.org/protocol/rsm").tap do |after|
					after.content = last_id
				end
			end
		end
	}

	client.write_with_handler(start_mam) do |reply|
		next if reply.error?

		fin = reply.find_first("./ns:fin", ns: "urn:xmpp:mam:2")
		next unless fin

		handle_rsm_reply_when_idle(fin)
	end
end

The first half of this creates the XML stanza to request a page from the server’s archive. We create a query with a max page size based on the size of our worker threadpool, and ask for messages only after the last known id (if we have one, which we won’t on first run). Then we use write_with_handler to send this request to the server and wait for a reply. The reply is sent after all messages have been sent down (sent seperately, not returned in this reply, see below), but we may still be processing some of them in the worker pool so we next create a helper to wait for the worker pool to be done:

def handle_rsm_reply_when_idle(fin)
	unless EM.defers_finished?
		EM.add_timer(0.1) { handle_rsm_reply_when_idle(fin) }
		return
	end

	last = fin.find_first(
		"./ns:set/ns:last",
		ns: "http://jabber.org/protocol/rsm"
	)&.content

	if last
		DB.execute(<<~SQL, [last, last])
			INSERT INTO data VALUES ('last_mam_id', ?)
			ON CONFLICT(key) DO UPDATE SET value=? WHERE key='last_mam_id'
		SQL
	end
	return if fin["complete"].to_s == "true"

	sync_mam(last)
end

Poll with a timer until the worker pool is all done so that we aren’t fetching new pages before we have handled the last one.  Get the value of the last archive ID that was part of the page just processed and store it in the database for next time we start up.  If this was the last page (that is, complete="true" ) then we’re all done, otherwise get the next page.  We need to make sure we actually start this sync process inside the when_ready handler:

last_mam_id = DB.execute(<<~SQL)[0]&.first
	SELECT value FROM data WHERE key='last_mam_id' LIMIT 1
SQL
sync_mam(last_mam_id)

And also, we need to actually handle the messages as they come down from the server archive:

message "./ns:result", ns: "urn:xmpp:mam:2" do |_, result|
	fwd = result.xpath("./ns:forwarded", ns: "urn:xmpp:forward:0").first
	fwd = fwd.find_first("./ns:message", ns: "jabber:client")
	m = Blather::XMPPNode.import(fwd)
	next unless m.is_a?(Blather::Stanza::Message) && m.body.present?

	mam_id = result.first["id"]&.to_s
	# Can't really race because we're checking for something from the past
	# Any new message inserted isn't the one we're looking for here anyway
	sent = DB.execute(<<~SQL, [m.id])[0][0]
		SELECT count(*) FROM messages WHERE stanza_id=? AND mam_id IS NULL
	SQL
	if sent < 1
		counterpart = if m.from.stripped.to_s == jid.stripped.to_s
			m.to.stripped.to_s
		else
			m.from.stripped.to_s
		end
		delay =
			fwd.find_first("./ns:delay", ns: "urn:xmpp:delay")
			&.[]("stamp")&.then(Time.method(:parse))
		delay = delay&.to_i || m.delay&.stamp&.to_i || Time.now.to_i
		DB.execute(<<~SQL, [mam_id, m.id, counterpart, delay, m.to_s])
			INSERT OR IGNORE INTO messages
			(mam_id, stanza_id, conversation, created_at, stanza)
			VALUES (?,?,?,?,?)
		SQL
	else
		DB.execute(<<~SQL, [mam_id, m.id])
			UPDATE messages SET mam_id=? WHERE stanza_id=?
		SQL
	end
end

Any message which contains a MAM (server archive) result will get handled here.  Just like with carbons we extract the forwarded message and import, making sure it ends up as a Blather Message object with a body.

Remember how when we stored a sent message we didn’t know the archive ID yet?  Here we check if there is anything in our database already with this stanza ID and no archive ID, if no we will insert it as a new message, but otherwise we can update the row we already have to store the server archive ID on it, which we now know.

And with that, our client now stores and syncs all history with the server, to give the user a full view of their conversation no matter where or when it happened.

Display Names

If a user is added to the contact list with a name, we already show that name instead of their address in conversations.  What if a user is not a contact yet, or we haven’t set a name for them?  It might be useful to be able to fetch any display name they advertise for themselves and show that.  First we add a simple helper to expose write_with_handler outside of the main object:

public def write_with_handler(stanza, &block)
	client.write_with_handler(stanza, &block)
end

We need an attribute on the Conversation to hold the nickname:

attr_accessor :nickname

And then we can use this in Conversation#initialize to fetch the other side’s nickname if they advertise one and we don’t have one for them yet:

self.nickname = BLATHER.my_roster[jid]&.name || jid
return unless nickname.to_s == jid.to_s

BLATHER.write_with_handler(
	Blather::Stanza::PubSub::Items.new(:get).tap { |iq|
		iq.node = "http://jabber.org/protocol/nick"
		iq.to = jid
	}
) do |reply|
	self.nickname = reply.items.first.payload_node.text rescue self.nickname
end

Inside the window declaration we can use this as the window title:

title <=> [self, :nickname]

and in format_sender we can use this as well:

return nickname if jid.to_s == @jid.to_s

Avatars

Names are nice, but what about pictures?  Can we have nice avatar images that go with each user?  What should we display if they don’t have an avatar set?  Well not only is there a protocol to get an avatar, but a specification that allows all clients to use the same colours to represent things, so we can use a block of that if there is no avatar set.  Let’s generate the colour blocks first.  Add this to Gemfile :

gem "hsluv"

Require the library at the top:

require "hsluv"

$avatars = {}

And a method on Conversation to use this:

def default_avatar(string)
	hue = (Digest::SHA1.digest(string).unpack1("v").to_f / 65536) * 360
	rgb = Hsluv.rgb_prepare(Hsluv.hsluv_to_rgb(hue, 100, 50))
	rgba = rgb.pack("CCC") + "xff".b
	image { image_part(rgba * 32 * 32, 32, 32, 4) }
end

This takes the SHA-1 of a string, unpacks the first two bytes as a 16-bit little-endian integer, converts the range from 0 to MAX_SHORT into the range from 0 to 360 for hue degrees, then passes to the library we added to convert from HSV to RGB colour formats.  The GUI library expects images as a byte string where every 4 bytes are 0 to 255 for red, then green, then blue, then transparency.  Because we want a square of all one colour, we can create the byte string for one pixel and then multiply the string by the width and height (multiplying a string by a number in Ruby make a new string with that many copies repeated) to get the whole image.

In Conversation#initialize we can use this to make a default avatar on the dummy message row then the window first opens:

@messages = [[default_avatar(""), "", ""]]

And we will need to add a new column definition to be beginning of the table { block:

image_column("Avatar")

And actually add the image to message_row :

def message_row(m)
	from = m.from&.stripped || BLATHER.jid.stripped
	[
		$avatars[from.to_s] || default_avatar(from.to_s),
		format_sender(from),
		m.body
	]
end

If you run this you should now see a coloured square next to each message.  We would now like to get actual avatars, so add this somewhere at the top level to advertise support for this:

set_caps(
	"https://git.singpolyma.net/jabber-client-demo",
	[],
	["urn:xmpp:avatar:metadata+notify"]
)

Then in the when_ready block make sure to send it to the server:

send_caps

And handle the avatars as they come in:

pubsub_event(
	"//ns:items[@node='urn:xmpp:avatar:metadata']",
	ns: "http://jabber.org/protocol/pubsub#event"
) do |m|
	id = m.items.first&.payload_node&.children&.first&.[]("id")
	next $avatars.delete(m.from.stripped.to_s) unless id

	path = DATA_DIR + id.to_s
	key = m.from.stripped.to_s
	if path.exist?
		LibUI.queue_main { $avatars[key] = image(path.to_s, 32, 32) rescue nil }
	else
		write_with_handler(
			Blather::Stanza::PubSub::Items.new(:get).tap { |iq|
				iq.node = "urn:xmpp:avatar:data"
				iq.to = m.from
			}
		) do |reply|
			next if reply.error?

			data = Base64.decode64(reply.items.first&.payload_node&.text.to_s)
			path.write(data)
			LibUI.queue_main { $avatars[key] = image(path.to_s, 32, 32) rescue nil }
		end
	end
end

When an avatar metadata event comes in, we check what it is advertising as the ID of the avatar for this user.  If there is none, that means they don’t have an avatar anymore so delete anything we may have in the global cache for them, otherwise create a file path in the same folder as the database based on this ID.  If that file exists already, then no need to fetch it again, create the image from that path on the GUI thread and set it into our global in-memory cache.  If the file does not exist, then use write_with_handler to request their avatar data.  It comes back Base64 encoded, so decode it and then write it to the file.

If you run this you should now see avatars next to messages for anyone who has one set.

Delivery Receipts

The Internet is a wild place, and sometimes things don’t work out how you’d hope.  Sometimes something goes wrong, or perhaps just all of a user’s devices are turned off.  Whatever the reason, it can be useful to see if a message has been delivered to at least one of the intended user’s devices yet or not.  We’ll need a new database column to store that status, add after the end of the DB.user_version < 1 if block:

if DB.user_version < 2
	DB.execute(<<~SQL)
		ALTER TABLE messages ADD COLUMN delivered INTEGER NOT NULL DEFAULT 0
	SQL
	DB.user_version = 2
end

Let’s advertise support for the feature:

set_caps(
	"https://git.singpolyma.net/jabber-client-demo",
	[],
	["urn:xmpp:avatar:metadata+notify", "urn:xmpp:receipts"]
)

We need to add delivery status and stanza id to the dummy row for the messages table:

@messages = [[default_avatar(""), "", "", false, nil]]

And make sure we select the status out of the database when loading up messages:

SELECT stanza,delivered FROM messages WHERE conversation=? ORDER BY created_at

And pass that through when building the message rows

mam_messages << [m, row[1]]

…

mam_messages.map! { |args| message_row(*args) }

Update the messages table to expect the new data model:

table {
	image_column("Avatar")
	text_column("Sender")
	text_column("Message")
	checkbox_column("Delivered")
	editable false
	cell_rows @messages
	@messages.clear if @messages.length == 1 && @messages.first.last.nil?
}

And update the row builder to include this new data:

def message_row(m, delivered=false)
	from = m.from&.stripped || BLATHER.jid.stripped
	[
		$avatars[from.to_s] || default_avatar(from.to_s),
		format_sender(from),
		m.body,
		delivered,
		m.id
	]
end

Inbound messages are always considered delivered, since we have them:

def new_message(m)
	@messages << message_row(m, true)
end

And a method to allow signalling that a delivery receipt should be displayed, using the fact that we now hide the stanza id off the end of the rows in the table to find the relevant message to update:

def delivered_message(id)
	row = @messages.find_index { |r| r.last == id }
	return unless row

	@messages[row] = @messages[row][0..-3] + [true, id]
end

In the Send button’s on_clicked handler we need to actually request that others send us receipts:

m = message
xml_child(m, :request, "urn:xmpp:receipts")

And we need to handle the receipts when they arrive:

message "./ns:received", ns: "urn:xmpp:receipts" do |m, received|
	DB.execute(<<~SQL, [received.first["id"].to_s])
		UPDATE messages SET delivered=1 WHERE stanza_id=?
	SQL

	conversation = $conversations[m.from.stripped.to_s]
	return unless conversation

	LibUI.queue_main do
		conversation.delivered_message(received.first["id"].to_s)
	end
end

When we get a received receipt, we get the id attribute off of it, which represents a stanza ID that this receipt is for.  We update the database, and inform any open conversation window so the GUI can be updated.

Finally, if someone requests a receipt from us we should send it to them:

message :body do |m|
	handle_live_message(m)

	if m.id && m.at("./ns:request", ns: "urn:xmpp:receipts")
		self << m.reply(remove_children: true).tap { |receipt|
			xml_child(receipt, :received, "urn:xmpp:receipts").tap { |received|
				received["id"] = m.id
			}
		}
	end
end

If the stanza has an id and a receipt request, we construct a reply that contains just the received receipt and send it.

Message Correction

Sometimes people send a message with a mistake in it and want to send another to fix it.  It is convenvient for the GUI to support this and render only the new version of the message.  So let’s implement that.  First we add it to the list of things we advertise support for:

set_caps(
	"https://git.singpolyma.net/jabber-client-demo",
	[],
	[
		"urn:xmpp:avatar:metadata+notify",
		"urn:xmpp:receipts",
		"urn:xmpp:message-correct:0"
	]
)

Then we need a method on Conversation to process incoming corrections and update the GUI:

def new_correction(replace_id, m)
	row = @messages.find_index { |r| r.last == replace_id }
	return unless row

	@messages[row] = message_row(m, true)
end

We look up the message row on the stanza id, just as we did for delivery receipts, and just completely replace it with a row based on the new incoming message.  That’s it for the GUI.  Corrections may come from live messages, from carbons, or even from the server archive if they happened while we were disconnected, so we create a new insert_message helper to handle any case we previously did the SQL INSERT for an incoming message:

def insert_message(
	m,
	mam_id:,
	counterpart: m.from.stripped.to_s,
	delay: m.delay&.stamp&.to_i
)
	if (replace = m.at("./ns:replace", ns: "urn:xmpp:message-correct:0"))
		DB.execute(<<~SQL, [m.to_s, counterpart, replace["id"].to_s])
			UPDATE messages SET stanza=? WHERE conversation=? AND stanza_id=?
		SQL
	else
		delay ||= Time.now.to_i
		DB.execute(<<~SQL, [mam_id, m.id, counterpart, delay, m.to_s])
			INSERT OR IGNORE INTO messages
			(mam_id, stanza_id, conversation, created_at, stanza, delivered)
			VALUES (?,?,?,?,?,1)
		SQL
	end
end

The else case here is the same as the INSERTs we’ve been using up to this point, but we also check first for an element that signals this as a replacement and if that is the case we issue an UPDATE instead to correct our internal archive to the new version.

Then in handle_live_message we also signal the possibly-open GUI:

if (replace = m.at("./ns:replace", ns: "urn:xmpp:message-correct:0"))
	conversation.new_correction(replace["id"].to_s, m)
else
	conversation.new_message(m)
end

We can now display incoming corrections, but it would also be nice to be able to send them.  Add a second button after the Send button in Conversation that can re-use the @message_entry box to correct the most recently sent message:

button("Correct") {
	stretchy false

	on_clicked do
		replace_row = @messages.rindex { |message|
			message[1] == format_sender(BLATHER.jid.stripped)
		}
		next unless replace_row

		m = message
		m << xml_child(m, :replace, "urn:xmpp:message-correct:0").tap { |replace|
			replace["id"] = @messages[replace_row].last
		}
		EM.defer do
			BLATHER << m
			DB.execute(<<~SQL, [m.to_s, @jid, @messages[replace_row].last])
				UPDATE messages SET stanza=? WHERE conversation=? AND stanza_id=?
			SQL
		end
		@messages[replace_row] = message_row(m, @messages[replace_row][-2])
		@message_entry.text = ""
	end
}

When the button is clicked we find the row for the most recently sent message, construct a message to send just as in the Send case but add the message correction replace child with the id matching the stanza id of the most recently sent message.  We send that message and also update our own local copy of the stanza both in the database and in the memory model rendered in the GUI.

Conclusion

There are a lot more features that a chat system can implement, but hopefully this gives you a useful taste of how each one can be incrementally layered in, and what the considerations might be for a wide variety of different kinds of features.  All the code for the working application developed in this article is available in git under AGPLv3+, with commits that corrospond to the path we took here .

Hello, it's time for a long overdue progress note.

I'll talk here about the work made on ActivityPub (AP) gateway and on end-to-end encryption around pubsub.

Oh, and if everything goes well, this blog post should be accessible from XMPP and ActivityPub (and HTTP and ATOM feed), using the same identifier goffi@goffi.org .

Forewords

The work made on the AP gateway has been possible thanks to a NLnet/NGI0 grant (with financial support from the European Commission's Next Generation Internet programme).

I especially appreciated that the team was really there to help bring the ideas to life, and not once did they get in the way: little paperwork, no unnecessary pressure, caring, contacts when help was needed, etc.

I wish there were more organizations like this one that really help develop libre projects for the common good.

So once again I want to thank them for all that.

XMPP ⬌ ActivityPub Gateway

There is probably no need to explain here what is ActivityPub , we can simply write that it is an open protocol that allows to do things that XMPP also allows doing, and that until now these 2 protocols could not communicate together. The work on the ActivityPub gateway aims to allow software implementing one of these 2 protocols to communicate as easily as possible. I firmly believe that all open protocols should be able to communicate which each other, to avoid creating more silos, proprietary software is already good enough at that.

To be useful, a gateway must use the full potential of both protocols. A simple bot transcribing messages as we see too often, using unsuitable features (such as instant messaging for blog posts), or using a very limited set of features to ensure compatibility are flaws that I have tried to avoid. Building a good gateway is a difficult and time-consuming task. If done right, the gateway should be as invisible as possible to the end user.

XMPP is featuring blogging since long before AP, however the set of features is not exactly the same. Current use of AP is clearly inspired from commercial "social" networks, and metadata such as subscribers/subscribed nodes (or followers/following in AP terms) are highlighted, feature such as like/favourite were missing in XMPP, and some implementation such as Pleroma do implement reactions. To integrate that in the gateway, I've been working on new specifications:

• Pubsub Public Subscriptions : a way to publicly announce subscriptions, in an opt-in way. With this it's possible to implement followers/following features in a way respectful of privacy.

• Pubsub Attachments : a generic way to attach any kind of data to a pubsub item. It's notably used to implements noticed/favourite button (see here and reactions.

• Calendar Events : handling of events and all the RSVP mechanism. Libervia was handling events for years, but it was an experimental implementation, this specification is a next step in the effort to make it a standard.

Note that this XEP and the others linked below have been accepted but are not yet visible in official list.

You may wonder why there is a specification for Calendar Events… It's because the AP gateway also handles them, making it compatible with Mobilizon . The gateway may evolve in the future to support other non (micro)blogging use cases.

The gateway is now finished in terms of functionalities, however the code is clearly of an alpha quality for the moment. Now the goal in the coming months will be to stabilize and possibly implement other features if there is a demand for it.

Early adopters are encouraged to try and test it as long as they keep in mind that it's not stable. So if you do try it, I recommend keeping a separate ActivityPub account in whatever stable implementation you use at the moment, this way you can check if messages or media are missing, if there is any inconsistency or other bugs, and report them to me. If you test it, please join the XMPP room libervia@chat.jabberfr.org ( click here to connect from your browser ) for help and feedback. Stabilization will probably take weeks, but I hope to have it done by early 2023.

Installation instructions and details on how the conversion between protocols is done is available in the documentation and notably here

A question I've been asked a lot: yes, you can use the same identifier for XMPP (JID) and AP ( WebFinger actor handle) as long as you use "simple" characters (i.e. alphanumeric ASCII chars, _ , . and - ). If you use something more complicated, you'll have to use the escaping mechanism explained in the doc (this is due to constraints with some AP implementations).

As for blogs on pubsub nodes (what Movim calls "communities"), I made it simple: you can use directly the name of the node that holds the blog in the local part (i.e. before the "@") of your actor handle: a blog named community_bog at the XMPP pubsub service pubsub.example.org can thus be addressed with the AP actor community_blog@pubsub.example.org . This way you can use a rather user-friendly identifier to share your blog with people who are only on ActivityPub.

This gateway should work with any XMPP server, and any client that implement blogging features (Only Libervia itself and Movim implement it for now, but I have heard that other clients are planning support for it). To enjoy the whole feature set of the gateway, the new specifications need to be implemented by the clients, so you can start to fill feature requests…

With this gateway, the door is open to have a client able to talk to the ActivityPub network, while having the feature of XMPP, including e2e encrypted private messages (e2e encrypted only if you communicate with an XMPP account, not with an AP one).

Oh, and please update your graphics, drawing and other texts to include XMPP in the fediverse ;)

End-to-End Encryption

Much effort has also gone into end-to-end encryption.

OMEMO implementation has been updated (OMEMO:2 is now used), including Stanza Content Encryption which allows encrypting arbitrary elements instead of only the \<body/> of the message, I believe that Libervia is the first XMPP client to implement it. OpenPGP for XMPP (or "OX") has also been implemented, all that thanks to the work of Tim Henkes "Syndace", the author of python-omemo .

Beside instant messaging, end-to-end encryption has also been introduced to pubsub. I've made specifications for two methods:

• An OpenPGP profile for pubsub which is thought to encrypt a whole node, with a system of secret sharing/rotation/revocation. With it, it is easy to give access to new entities after publication, and to retrieve old items for newcomers. This specification can be used to encrypt any pubsub based features: (micro)blogging, calendar events, lists, etc.

• Pubsub Targeted Encryption which is a way to apply the same cryptographic system used in instant messaging to pubsub. This way, OMEMO can be used with its forward secrecy property. It is not a good option to use this specification to encrypt a whole node, as archive is then not accessible to newcomers, and to add access to a new entity you have to re-encrypt all items, but it's an interesting option to encrypt an element occasionally, for instance to restrict access of a specific post in an otherwise public blog.

Specifications have also been written to sign a pubsub item in a backward compatible way (client which don't implement those specifications can still work normally):

• Pubsub Signing

• Pubsub Signing: OpenPGP Profile

All those specifications are already implemented in Libervia, but they are only usable from CLI frontend at the moment. All you have to do is to use the --encrypt and/or --sign options from pubsub or blog commands (check documentation for details).

Uploaded files were already encrypted with OMEMO Media Sharing which is what is commonly used these days, but this method has not been accepted as a standard as it was a workaround for limitation of legacy OMEMO implementation. The proper way is now specified with Stateless File Sharing and is encrypted with Encryption For Stateless File Sharing . Those methods are currently only usable when OMEMO:2 is implemented in the peer client, and with them metadata on the shared file can be attached, including thumbnails.

Encryption has also been implemented for Jingle ( XEP-0391 and XEP-0396 ), which is notably used for Jingle File Transfer (specially useful for large files transfers).

So to summarize, nearly everything (instant messaging, files uploaded, large file transfers, all pubsub related features) can now be e2e encrypted with Libervia.

Possible Future

With the AP gateway permitting to reach the whole AP network, all the new features implemented, and the work done on e2e encryption, Libervia has everything to be a solid option for communication. After the recent events regarding a famous commercial network, we see a breakthrough of ActivityPub that will hopefully last over time. We can now access AP from XMPP, while having the possibility to have e2e encrypted private conversations or even blogs or calendar events.

As far as I know this is, so far, something unique for a Libre decentralized software. However, there is still work to do on stabilization on UI/UX update before this is really usable.

Those feature were planned for very long (years), but the lack of resources made them slow to come. The grant has made it possible to greatly accelerate the pace of development, and I doubt that it would have been possible to have all that without it.

Regarding how large the project is, and my family life, it's not possible any more to develop seriously this project on my free time alone (and I would like to do other things, sometimes, of my free time).

In other words, I need to find a way to sustain the development of Libervia for the years to come, so I can work full-time on it, and with some luck, build a team. I'm thinking very seriously about it these days, I'll probably write on this topic in a little while. If you are willing to help in any way, please contact me (on the Libervia room linked above for instance).

That's all for this progress note. I'm now working on stabilization and UI/UX update on the web frontend.