Migrating trashserver.net from Prosody to Ejabberd

Yesterday I moved my old Prosody setup to a new Ejabberd-based XMPP server setup. I’d like to leave you a few notes on why and how I did that.

Motivation

There ware multiple reasons for my decision to give Ejabberd a chance. Most of them are based on flaws in Prosody that will probably not be solved too soon:

• Prosody consumes lots of system memory. I needed to restart Prosody about every 2-3 days to make it use less than 8 GB. The situation got a bit better with mod_external_upload and new versions of the PostgreSQL connectivity code, but is still not satisfactory.
• Prosody is only using one single CPU core. I was not happy with that on my 16 cores server.
• No support for clustering. In the future I’d like to provide a highly available XMPP setup.
• Sometimes not up to date with new XMPP standards required by Conversations Android app

Ejabberd has some advantages over Prosody:

• Developers have long time experience with XMPP - Ejabberd is very mature
• Erlang language is made exactly for this kind of application (very performant)
• Ejabberd uses all of the available CPU cores
• Clustering support
• Said to be very stable and robust - Big Enterprise use cases confirm that
• Supported by the ProcessOne company
• Conversation.im uses Ejabberd. When developing, the provider refers to it instead of Prosody

Don’t get me wrong: I still consider Prosody a good XMPP server - but maybe it’s just not made for big servers like mine. I’d suggest to use Prosody for home / private use, because usually it is easy to handle. For a public service, I feel better using Ejabberd as a more “enterprisy” solution.

General information on my server setup

Prosody was running with the PostgreSQL backend, because MySQL turned out to perform not good enough for that big amount of users (>8,000 registered, ~2,000 active). As I personally like PostgreSQL and consider it a very reliable and great performing database system, I decided to use it for my new Ejabberd instance, too.

How I did it

First set up a demo server on a different domain to check if Ejabberd satisfies my requirements (mainly supporting all Conversations features). Testing with https://compliance.conversations.im/

As soon as I was happy with the test results I checked how I could transfer Prosody user data to Ejabberd.

Converting data from PostgreSQL backend to file backend

There’s an Ejabberd feature import_prosody that takes Prosody file storage data and transfers it into Ejabberd’s data backend. To be able to use this tool, I first needed to convert my Prosody SQL backend to a file based backend.

Prosody provides the prosody-migrator tool for that. For Debian the package prosody-migrator-0.11 needed to be installed.

Then a created a configuration like this in /var/lib/prosody/migrator.cfg.lua:

input {
    type = "prosody_sql";
    driver = "PostgreSQL";
    database = "prosody";
    host = "localhost";
    port = 5432;
    username = "prosody";
    password = "mydbpassword";
}


output {
    type = "prosody_files";
    path = "/var/lib/prosody";
}

and started the “Migrator” tool:

prosody-migrator --config=/var/lib/prosody/migrator.cfg.lua

hg clone 'https://hg.prosody.im/0.11/' prosody-0.11
cd prosody-0.11/tools/migration/
lua prosody-migrator.lua --config=/var/lib/prosody/migrator.cfg.lua

Converting data from Prosody file backend to Ejabberd PostgreSQL backend

in the meantime trashserver.net has already been configured as a virtual host in Ejabberd. Just the PostgreSQL database was missing. As mentioned earlier, Ejabberd provides a tool, which reads Prosody’s file backend data and transfers it into the currently configured Ejabberd backend. That is PostgreSQL in my case.

I tared all the Prosody file storage directories (such as trashserver%2enet, conference%2etrashserver%2enet, and xmpp%2etrashserver%2enet) in /var/lib/prosody and extracted them in my new Ejabberd application container. All the directories were located at /var/lib/prosody - just as if I had been running Prosody on that container previously.

On the new XMPP system the erlang-luerl package needed to be installed. It enables Lua support for Erlang, so Ejabberd can read Prosody’s export format.

apt install erlang-luerl
systemctl restart ejabberd

During my first data import to Ejabberd I faced a timeout due to the sheer amount of data. The following setting disables the timeout (it will maybe be removed in future Ejabberd versions).

File /etc/ejabberd/ejabberdctl.cfg:

#
#  NO_TIMEOUT ([--no-timeout])
# Don't let ejabberdctl calls time out.
#
NO_TIMEOUT="--no-timeout"

Prepare PostgreSQL database

I created a new PostgeSQL user “ejabberd”:

su -c "psql" - postgres
create user ejabberd with password 'ejabberdpassword';

… and a new database including the Ejabberd Postgres schema:

create database ejabberd_trashserver;
alter database ejabberd_trashserver owner to ejabberd;
\q
su -c "psql ejabberd_trashserver < /usr/share/ejabberd/sql/pg.new.sql" - ejabberd

After that it was time to import all the data into Ejabberd:

ejabberdctl import_prosody /var/lib/prosody/

It is important to restart Ejabberd after the import, because otherwise it won’t pick up the changes to the conference component:

systemctl restart ejabberd

Some problems I faced …

There were compatibility issues with the mod_privacy module. It was part of Prosody until version 0.10, but is not relevant anymore for version 0.11. Still that mod_privacy data was present in /var/lib/prosody/trashserver%20enet/privacy and made the Ejabberd importer crash. I was able to resolve the issue by deleting the privacy data directory previous to the import.

Another issue was found with the MUC import: Ejabberd seems to choose a database / vHost for the import by alphabet. Just to be sure, you should uncomment all other vHosts in your config if you have defined multiple virtual hosts. Ejabberd will then import all MUC data to the proper vHost :-) (See https://github.com/processone/ejabberd/issues/2874#issuecomment-488936946)

The result

I was able to prepare and run a migration in few days of preparation - and while doing that, I fed the Ejabberd bug tracker a little bit ;-) Until now Ejabberd has been running very reliably and I am very happy with my new setup. Let’s hope it stays like that.

Oh, and by the way: RAM usage decreased drastically:

In case you are interested in details about my setup and its configuration, check out my Git repo: https://github.com/ThomasLeister/trashserver.net-xmpp

Big thanks to

• Matthew Wild (developer of Prosody) who always was happy to help with Prosody :-)
• Holger Weiss (developer of Ejabberd project), who supported me during Ejabberd evaluation and testing
• badlop who supported me dealing with my final “blocker issue”