Mailman3: "Cannot connect to SMTP server localhost on port 25"

Badly Misleading Error Messages

Most importantly, whatever you do, never call mailman as root. This will mess up permissions and lead to situations really hard to debug. In particular, the error message from the post's title:

Cannot connect to SMTP server localhost on port 25

apparently can have many reasons (or so the recipes you find on the net suggest), few of which have anything to do with SMTP, but one clearly is when mailman can't read or write to queue files or templates or whatever and bombs out while trying to submit mail.

Morale: Don't claim too much when writing error messages in your programs.

Unfortunately, I've fixed the thing accidentally, so I can't say what exactly broke. The take away still is that, in Debian (other installations' mailman users might be called something else) you run mailman like this:

sudo -u list mailman

However, I can now say how to go about debugging problems like these, at least when you can afford a bit of mailman unavailability. First, stop the mailman3 daemon, because you want to run the thing in the foreground. Then set a breakpoint in deliver.py by inserting, right after def deliver(mlist, msg, msgdata), something like:

import pdb; pdb.set_trace()

Assuming Debian packaging, you will find that file in /usr/lib/python3/dist-packages/mailman/mta.

Of course, you'll now need to talk to the debugger, so you'll have to run mailman in the foreground. To do that, call (perhaps adapting the path):

sudo -u list /usr/lib/mailman3/bin/master

From somewhere else, send the mail that should make it to the mail server, and you'll be dropped into the python debugger, where you can step until where the thing actually fails. Don't forget to remove the PDB call again, as it will itself cause funky errors when it triggers in the daemonised mailman. Of course, apt reinstall mailman3 will restore the original source, too.

Template Management Half-Broken

When I overrode the welcome message for a mailing list, the subscription notifications to the subscribing users came out empty.

This time, there was at least something halfway sensible in the log:

requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://localhost/postorius/api/templates/list/kal.sofo-hd.de/list:user:notice:welcome

Until you read up on the mailman3 system of managing templates (which, roughly, is: store URIs from where to pull them), it's a bit mystifying why mailman should even try this URI. Eventually, one can work out that when you configure these templates from Postorius, it will take the URI at which mailman should see it, Postorius, from POSTORIUS_TEMPLATE_BASE_URL in /etc/mailman/mailman-web.py. This is preconfigured to the localhost URI, which proabably only rarely is right.

To fix it, change that setting to:

POSTORIUS_TEMPLATE_BASE_URL = 'http://<your postorious vserver>/postorius/api/templates/'

Of course it'll still not work because the old, wrong, URI is still in mailman's configuration. So, you'll have to go back to the template configuration in Postorius and at least re-save the template. Interestingly, that didn't seem to fix it for me for reasons I've not bothered to fathom. What worked was deleting the template and re-adding it. Sigh.

As soon as you have more than one template, I expect it's faster to change the URIs directly in mailman's database, which isn't hard, as seen in the next section.

[Incidentally: does anyone know what the dire warnings in the docs about not using sqlite3 on “production” systems actually are about?]

Disable Emergency Moderation After Moving

Basically because I was hoping to get a more controlled migration, I had set one list on the old server to emergency moderation before pulling the config.pck. Don't do that, because at least as of now mailman3 has the notion of emergency moderation but makes it hard to switch it on or off. I eventually resorted to directly touching mailman's config database (if you've configured mailman to use something else than sqlite, the shell command is different, but the query should be the same):

$ sudo -u list sqlite3 /var/lib/mailman3/data/mailman.db
[on the sqlite prompt:]
update mailinglist set emergency=0 where list_id='<your list id>';

Note that <your list id> has a dot instead of the at, so if your list is mylist@example.org, its id is mylist.example.org.

Oh No, CSRF Token

The list I cared about most could be joined from an external web site, transparently posting to mailman2's cgi-bin/mailman/subscribe (oh! CGI! How am I missing you in the age of uwsgi and Django!). Looking at its counterpart for modern mailman3, the first thing I noted is that there's a CSRF token in it – if you've not encountered them before, it's a couple of bytes the originating server puts into a web form to prevent what Postorius' authors feels is Cross Site Request Forgery.

Of course, what I wanted was exactly that: Post to Postorius from a different web site. I don't feel that's forgery, very frankly.

I didn't see an obvious way to turn it off, and I was a bit curious about mailman3's own http API, so I wrote a few lines of code to do this; the API part itself was straightforward enough, something like:

result = requests.post(
  getConfig("mailmanAPI")+"/members", {
    'list_id': getConfig("mailmanListname"),
    'subscriber': toSubscribe,
    'pre_verified': False,
    'pre_confirmed': False,
    'pre_approved': True,},
  auth=(getConfig("mailmanAPIUser"),
    getConfig("mailmanAPIPassword")),
  timeout=1)

– but of course it sucks a bit that subscribing someone requires the same privilege level as, say, creating a mailing list or changing its description. And all that just to work around CSRF prevention. Sigh.

On top of that, I've tried K-SAT on the pre_X booleans to try and see if anything gives me the tried and tested workflow of “let folks enter a mail address, send a confirmation link there, subscribe them when it's being clicked“. No luck. Well, let's hope the pranksters don't hit this server until I figure out how to do this.

---

Hm. I think I'm a bit too locked in into mailman to migrate away, but I have to say I wish someone would port mailman2 to python3 and thus let mailman2 hang on essentially forever. It did all a mailing list manager needs to do as far as I am concerned, and while it wasn't pretty with the default browser stylesheets, even now, almost a decade into mailman3, it works a whole lot more smoothly.

Or perhaps there's a space for a new mailing list manager with a trivially deployable web interface not requiring two separate database connections? Perhaps such a thing exists already?

Well, summing up, the central migration advice again: Mind the sudo option in

sudo -u list mailman import21 my-list@example.org config.pck