formless and empty
Criteria and comparison of XMPP-based chat systems for Merveilles
This document is a working draft exploring options for a chat system for Merveilles. It's not a final decision, the idea is to make the constraints / trade offs explicit and to document what actually fits or does not fit into those constraints.
We want a private, secure, self-hosted chat system that a small group of maintainers can realistically operate long-term.
The goal isn't to pick a winner up front, but to be explicit about what we care about so it's clear how each option does or doesn't fit.
The idea is that identity comes from Mastodon, with authentication handled through Keycloak. If you have a Mastodon account, you should be able to use chat. Same handle, same profile, permissions synced where it makes sense. People should still be able to tweak their display name or profile on whatever platform they're using without it becoming a headache.
Merveilles runs the server. The server has rooms or channels. There are direct messages. Roles and permissions exist, both at the server level and per room.
Federation is opt-in and controlled by Merveilles admins. It happens per room and should not be global by default. If we want to share a specific room with another server, we can, but that's a deliberate choice, similar to how Mastodon instances decide when and how to federate.
End-to-end encryption should be the default. Admins can choose to keep some rooms plaintext if there's a reason to. The threat model here is pretty normal: we care about keeping conversations private from remote observers, but we're not pretending compromised devices don't exist.
Image and small file uploads should work. Large file hosting is out of scope. Retention should be finite. Attachments should get pruned after some number of months, and message history shouldn't be infinite.
Voice and video aren't required. If people want that, it's probably better handled by external tools rather than bolting it directly onto the chat system.
Costs should stay low. Ideally this runs comfortably under $50 a month.
The likely starting point is probably a single Ubuntu LTS server. Firewalls would be configured both at the provider level and on the OS itself. Block storage would be attached for anything that needs to persist.
There should be a clear separation between system administrators, who have sudo access to maintain the server, and chat service administrators, who manage the application itself.
nginx would be used to proxy one or more domains to the service.
Before inviting anyone in, we should spend time actually using the system and feeling out authentication, module choices, federation behavior, retention enforcement, and what day-to-day admin work looks like in practice.
A few things worth keeping in mind while reading this:
XMPP federation is server-to-server by design. What looks like "federating a room" is really access control and policy.
OMEMO encrypts message content and files, not metadata. Servers still see who's talking to who and when.
Erland Jabber Daemon, an XMPP server designed for very large-scale. Commonly used as an enterprise solution and is built for high concurrency across clustered environments.
ejabberd is written in Erlang and runs on the Erlang Virtual Machine. Modern versions primarily use YAML for configuration, but still support the legacy Erlang configuration.
https://www.process-one.net/ejabberd-features/
Scales to millions, used by WhatsApp, Riot Games, and Nintendo.
commonly referenced benchmark shows over two million concurrent users on a single node:
https://www.process-one.net/blog/ejabberd-massive-scalability-1node-2-million-concurrent-users/
Modern versions use YAML for configuration, though the legacy Erlang format is still supported.
https://www.process-one.net/ejabberd-features/
While primarily an XMPP server, it also supports MQTT broker functionality, Websockets, BOSH for web clients. Native clustering/Node discovery/
Typically has a higher baseline memory footprint than lighter XMPP servers. Resting memory usage is reported in the range of approximately 120MB to 200MB. This behavior is largely due to the Erlang VM, which pre-allocates memory to support concurrency/fault tolerance.
https://dustycloud.org/blog/prosody/
https://www.process-one.net/blog/movim-migrates-its-official-server-to-ejabberd/
Counter point: A common benchmark cited in the community (and mentioned by ProcessOne developers in various forums) is that while Prosody starts much smaller, ejabberd's memory usage per user is extremely stable and often more efficient once you reach the hundreds of thousands of users. (vs ya know, a community of around 500.)
It goes without saying OMEMO (E2EE) is supported out of the box. It also has a bunch of enterprise level security features.
Mercurial | Github is a mirror
Lightweight server written in Lua. It's an XML router with a focus on event-driven design. It is the engine inside Snikket and provides signaling for Jitsi Meet.
Every feature is a module. If you don't need a feature, don't load it. Obviously this keeps configuration fairly small and simple and resource usage low.
Supports a ton of modern XMPP extensions (XEPs) for mobile, web, and desktop clients. As well as HTTP File uploads.
https://prosody.im/doc/xeplist (full list of extensions)
https://modules.prosody.im/xeps.html (Community modules)
Prosody is extremely light on "resting" RAM. Typically sits between 15MB and 50MB for small/medium instances.
https://dustycloud.org/blog/prosody/ "Prosody only takes about 15 megabytes of memory"
https://prosody.im/doc/how_many_users
Prosody is capable of supporting 500 or more active users when configured correctly. At this scale, a SQL-based storage backend such as PostgreSQL is recommended to maintain good performance.
https://prosody.im/doc/modules/mod_storage_internal (Archives scale linearly and are limited to 10 000 items. Larger archives may cause more memory usage and poor performance. Very large archives may even fail to be read.)
https://prosody.im/doc/storage
https://prosody.im/doc/modules/mod_storage_sql
https://news.ycombinator.com/item?id=44729195
have no experience with ejabberd. As for Prosody, it does everything I/we want it to do, is very easy to configure and runs incredibly light - the host is a single-core virtual machine running OpenBSD on 256 MiB of RAM, and we've yet to face resource shortages.
Prosody's module API is straightforward and event-based. Custom logic can hook into message delivery, authentication, presence updates, ect.
-- mod_hello.lua
local st = require "util.stanza"; -- Import Prosody's stanza API into 'st'
function on_message(event)
module:log("info", "Received a message! Look: %s", tostring(event.stanza));
-- Check the type of the incoming stanza to avoid loops:
if event.stanza.attr.type == "error" then
return; -- We do not want to reply to these, so leave.
end
-- Create a clone of the received stanza to use for our reply:
local reply_stanza = st.clone(event.stanza);
-- This is a neat trick in Lua to swap two variables, without needing a third:
reply_stanza.attr.to, reply_stanza.attr.from = reply_stanza.attr.from, reply_stanza.attr.to;
-- Now send!
module:send(reply_stanza);
-- We're done! Now, let's halt event propagation
return true
end
module:hook("message/bare", on_message);
Event references: https://prosody.im/doc/developers/events
https://modules.prosody.im/mod_auth_oauth_external (external auth)
https://prosody.im/doc/developers/modules
Supports E2EE via modules. Great resource: https://we.riseup.net/tech-autonomy+infrastructure/prosody but also outdated.
You can use mod_e2ee_policy to require encryption. If a message comes in as plaintext, Prosody can be configured to reject it entirely, enforcing a "dark" server.
https://www.youtube.com/watch?v=FwKp8Nta0VA
Scaling beyond small or medium deployments requires careful storage configuration. Internal storage backend isn't suited for large message archives.
Great info here
Snikket is a preconfigured distribution of Prosody. It takes an opinionated approach to XMPP by bundling the server with a web portal and dedicated mobile apps. Provides an out-of-the-box experience that feels closer to Signal or WhatsApp than to regular XMPP deployments.
Accounts are created through invitation links rather than manual registration. A user clicks the invite, installs the app, and their account, server settings, and initial contacts are configured automatically on first launch. Users never need to think about server addresses, ports, or account configuration.
Official Android and iOS clients are provided and maintained as part of the project, which keeps client behavior predictable and avoids the usual client/feature fragmentation problems seen across the XMPP (and Matrix) ecosystems.
Snikket intentionally focuses on personal and small-group messaging rather than large, shared chat rooms.
One of the Snikket developers summarized this on Reddit:
Q:
Specifically group chats, how do they compare to discord regarding picture embeds, markdown text, etc? I'm trying to find how group chat would look like, but I literally can't find any screenshots or videos of how the group chat functions.
A:
Snikket dev here. The answer is no - Snikket is not what you're looking for if you want a replacement for Discord. We explicitly are not trying to build replacements for those kinds of platforms (what we call "team chat" - see [https://snikket.org/about/goals/](https://snikket.org/about/goals/) ). We are rather working on personal/mobile messaging (think Signal, Telegram, WhatsApp, etc.).
If you want something Snikket-compatible (i.e. also based on XMPP) then you could have a look at [https://Prose.org](https://Prose.org) . It's a relatively young project, but is definitely in the "team chat" space. Alternatively there are a few non-XMPP solutions that aim to be a bit closer to Discord than we are trying to be.
Of these, Element is the closest-aligned project, as it is also based on an open federated protocol like Snikket is (Matrix). Whether it meets your requirements for a Discord replacement is something you would have to decide for yourself.
Finally there are projects with the explicit goal to not just enable "team chat", but to fully replicate Discord in an open-source/self-hostable manner. These don't use open federated protocols, which limits your ability to interact across servers, but they may still fit your needs better. Take a look at [https://revolt.chat/](https://revolt.chat/) and [https://spacebar.chat/](https://spacebar.chat/) .
Hope this helps!
https://snikket.org/blog/snikket-server-sept-2024-release/
https://xmpp.org/extensions/xep-0401.html
Distributed primarily as a fully containerized service with all dependencies included. This makes deployment straightforward and keeps maintenance very low.
The Docker setup runs multiple containers providing the XMPP server, the web invitation and account management portal, and a TURN server for voice and video reliability.
TLS certificates via Let's Encrypt and database configuration are handled automatically as part of the deployment.
https://snikket.org/service/help/setup/upgrading/
https://compliance.conversations.im/server/snikket.de
Resource usage is pretty modest. The full containerized service typically uses around 100-150 MB of RAM with minimal CPU usage.
**A:** We recommend a machine with at least 1GB RAM to set up and run Snikket comfortably. The server software supports Linux on 64-bit (i.e. amd64), and ARM devices of ARMv7 or ARMv8. This includes Raspberry Pi 2/3/4 devices.
The server memory usage scales with the number of active users. Expect to fit 10-15 users comfortably in as little as 128MB RAM, but this may vary based on activity (video calls, media sharing, etc.). If you're not already running docker on your system then you also need to account for resource usage by the docker service. As with most things, spare RAM is always good to have :)
CPU requirements are generally minimal. A slow CPU may slow down some operations, but this is generally not noticeable unless you have a significant number of users.
https://github.com/snikket-im/snikket-web-portal
Snikket treats end to end encryption as mandatory rather than optional. The system assumes modern encrypted clients and does not try to support insecure fallbacks.
The default configuration aligns with current XMPP security recommendations and passes public compliance checks without additional hardening.
TLS certificates via Let's Encrypt and database configuration are handled automatically as part of the deployment.
Snikket scales by keeping chat spaces small. It is optimized for 1:1 and small-group messaging and does not aim to support large shared rooms or high fan-out broadcast chats.
Rather than concentrating more users into a single server or room, Snikket relies on federation. Small, independently operated servers can communicate with one another, allowing reach to grow without centralization.
As the project describes it:
Individual servers are great, but your small 5-person setup is of no comparison to the massive commercial networks of today. Snikket's secret is federation. Federation means that users of one Snikket service can easily communicate with users of another Snikket service.
Federation turns many small individually-owned servers from a collection of tiny silos into an extensive open global messaging network! What's more, since Snikket uses XMPP, there is already a large network of people and services that Snikket becomes a part of.
Revisions