Tag Mail

  • Magie im Spam

    Leider vermag mich halbwegs origineller Spam durchaus mal nerdzusnipen (ref; zuvor). Und so habe ich mich heute über ein Angebot für dieses Gerät gefreut:

    Werbebild von irgendeinem Zwischenstecker und seiner Verpackung

    Rechte: Tja, halt bei irgendeinem Spammer.

    Beschrieben wird das im begleitenden Text (genauer: der text/plain-Alternative) als:

    Fangen Sie mit E-Energy schon heute an zu sparen! E-Energy sprecher die erhaltene reaktive Elektroenergie und gibt die aktive Elektroenergie an das Gerät ab, welches sich im selben Stromkreis mit dem Stromsparer befindet.

    Funktionsweise von E-Energy >>

    Energieeinsparung -50% + Ihr Rabatt -50%!

    Das ist natürlich kompletter Unfug, der mit Physik ungefähr so viel zu tun hat wie in die Zukunft blickende Glaskugeln. Demgegenüber hat selbst der Technobabble der originalen Star-Trek-Serie noch jede Menge Plausibilität. Schön!

    Ebenfalls schön ist, dass stil- und genresicher das, was diese Mail erzeugt hat, nicht in der Lage war, Entities aus dem doofen HTML in ordentliches Unicode zu übersetzen. Dennoch sind die Spammer in der Hinsicht den Schurken von cleverreach voraus, denn immerhin steht etwas im Plain Text.

    Was dort allerdings völlig unklar bleibt: Ist das ein Versuch, Leute zum Klicken auf giftige Links zu verleiten oder will da wirklich wer was verkloppen? In der HTML-Alternative stehen Links drin, und zwar auf eine Domain dirabore.co.in. Habt Verständnis, dass ich keinen Link draufsetze; zwar habe ich nicht viel Pagerank zu vergeben, aber auch den sollen sie nicht haben.

    Das wiederum redirectet (mit einem selbst unterschriebenen Zertifikat) auf eine wirre Seite auf litbeurope.com (soll das Vertipper von libreurope fangen?) mit folgendem Teaser:

    Screenshot mit Text: „So bestrafte ein einfacher Dortmunder die E.ON für den BETRUG and der ganzen Bevölkerung“

    Im Weiteren behauptet der „Text“, eon habe den Einsatz des e-energy-Wunderdings gerichtlich verbieten lassen wollen. Interessanter sind die (vermutlichen) Kauf-Links auf der Seite, denn die haben ohne Javascript keine Ziele. Rauspopeln, welches Javascript die setzt oder diese Leute gar Javascript bei mir ausführen lassen – nun, so neugierig bin ich dann auch nicht.

    Die Frage, ob das Betrug mit funktionslosen Steckern oder nicht ausgeführten Lieferungen ist oder ob das doch nur ein elaborierter Versuch ist, Leuten Malware unterzuschieben: Das muss ich dann wohl offen lassen. Dennoch schönen Dank an die AutorInnen für unterhaltsamen Spam.

    Ergänzung 2022-11-25: Oh wow. Das ist mehr als ein wirres Randphänomen. Es geht offenbar gerade groß im Netz rum – jedenfalls warnt die Bundesnetzagentur vor Maschen dieses Typs (heise online dazu). Was für eine katastrophale Niederlage für unser Schulsystem…

  • Rekursives 419: Ein brillianter Betrugsversuch

    Screenshot: Mail mit 'I am a banker'

    Ich glaube, jedeR hat irgendwelche dunklen Marotten, die normalen Menschen eigentlich peinlich sind. Ich zum Beispiel bin ein großer Fan der Nigeria-Connection, etwas korrekter Vorschussbetrug oder ähnlich inkorrekt (weil auch auf Nigeria bezogen) 419er-Mails genannt. Ich verfüge, ich gestehe es, ohne rot zu werden, über eine 12 Megabyte umfassende Sammlung handverlesener einschlägiger Mails aus den letzten 20 Jahren, die ich bei Interesse gerne für wissenschaftliche oder andere Untersuchungen zur Verfügung stelle.

    Heute kam nun eine besondere Perle: Ein 419er-Scam, der Opfern anderer 419er-Scams Hilfe anempfiehlt, natürlich gegen eine kleine Gebühr. Fantastisch. Wer könnte ein dankbareres Opfer für diese Sorte von Betrug sein als jemand, der/die schon mal auf sowas reingefallen ist? In meiner Begeisterung über die selbstbezügliche Genialität dieses Kozepts missachte ich eventuelle Urheberrechte und reproduziere hier die ganze Mail:

    Date: Tue, 18 Oct 2022 10:32:57 +0100
    From: William Christian <williamsgeorge00051@gmail.com>
    To: undisclosed-recipients:
    Subject: I HAVE GOOD NEWS FOR YOU

    Hi my friend,

    I must apologize for this spontaneous email to you. I am aware of this is certainly not a conservative way of approaching you, but you will understand the need for my action thought It’s true we don’t know each other, but “I” think you need to hear this truth to protect yourself from being defrauded.

    I am Williams Christian, from , I was one of the Victims in Africa by some individuals whom I contacted to help me get my Inheritance Funds, but they all took advantage of me and left me a Bankrupt, I have tried in different ways to get my payment but all to know avail, I lost my life savings to different FAKE groups that claimed to be working in Banks like CBN Bank, Zenith bank, UBA, First bank etc. and many others thinking they were helping me, but rather, they were busy defrauding my hard earned money, I am very sorry telling my past, but I think there are many innocent people out there too that need this message.

    Honestly I wouldn't want anybody to experience what I went through in the hands of those crooks scammers, I lost all my friends because I refused to listen to their advice, simply because I thought I was on the right part without knowing I was dealing with scammers,

    But the good news today is that God has finally remembered me, few weeks ago I received a mail from a unknown person, a man in the state of Arizona advising me to contact one senior and professional attorney Mr Femi Falana, He is a Nigerian international lawyer and human rights activist whose law firm is made up of UK/Nigerian qualified solicitor and renders commercial legal services in due diligence, corporate investigation and trade secrets, immigration, intellectual property etc.

    He actually helped me in Nigeria in other west African countries to claim my lost funds and if I have not gotten mine I wouldn’t have bothered to advice another person, although at the beginning ,I thought it was another trick to scam me, but two weeks I decided to give it a try by contacting the attorney using email address provided, and to my greatest surprise it happened to be 100% real and I got my lost funds.

    Of course, I promised myself never to share my testimony/news until I receive my fund and then be convinced enough before letting people know, and Today I am now writing with joy to advise us all to STOP feeding scammers out there and contact this attorney on the email below if you haven’t claimed your lost money yet,

    Although, you may be wondering how I got my funds without paying a cent, yes I paid little money to the attorney which is normal to secure a some documents including permit certificate in my name of which I have the copy on file, but my advice to you is to STOP dealing with those scammers and contact this senior attorney to claim all your lost funds, his email address ID and phone number are(sexkepy@gmail.com) (+2348145880372)

    I encourage you to contact him for your own money and he will help you get it because I am a living testimony.

    Yours Sincerely
    William Christian

    Gut: Stilistisch reicht es jetzt nicht ganz zum Literaturnobelpreis. Aber William Christian sollte, finde ich, zumindest den Münchhausen-Preis der Stadt Bordenwerder erhalten, denn der soll „Personen mit besonderer Begabung in Darstellungs- und Redekunst, Fantasie und Satire“ auszeichnen. All of the above.

  • HTML and Word in mutt

    I read my mail using mutt, and even though I was severely tempted by astroid, mutt just works too nicely for me to make moving away an attractive proposition. And it is a fine piece of software. If you're still stuck with Thunderbird (let alone some webmail interface in the browser) and wonder what text-based software you might adopt, right after vim I'd point you to mutt.

    I'm saying all that because the other day I complained about a snooping mail marketing firm (in German) abusing MIME's multipart/alternative type to clickbait people reading plain text mails into their tracker-infested web pages, and I promised to give an account on how I configured my mutt to cope with HTML mails and similar calamities.

    The basic mechanism is ~/.mutt/mailcap. That's a file analogous to /etc/mailcap, for which there's a man page, mailcap (5)[1]. That explains how, in general, software uses this file to figure out which program to use to display (or print or compose) files of which types.

    Mutt reads system-wide mailcaps, too, but I've found I generally want to handle a solid number of media differently in mails than, say, in browsers or from the shell[2], and hence I'm keeping most of this configuration in mutt's private mailcap. For HTML mail, I've put into that file:

    text/html; w3m -dump -T %t -I utf-8 -cols 72 -o display_link_number=1 %s; copiousoutput
    

    This uses w3m to format HTML rather than te lynx that the mutt docs give. Lynx these days really is too basic for my taste (I'm not even sure whether it has learned utf-8). Still, this will not execute javascript or retrieve images, so most of the ugly aspects of HTML mails are sidestepped. The copiousoutput option makes mutt use its standard pager when showing the program's output, and thus HTML mail will look almost like sane mail.

    To make that really seamless, you need an extra setting in your ~/.mutt/muttrc:

    auto_view text/html
    

    This makes mutt automatically render HTML (which, contrary to the behaviour of gmail or thunderbird I consider relatively safe if it's parsimonious w3m that does the rendering). In addition, since I still believe in the good in humans, I believe that when there is both HTML and plain text in a mail, the plain text will be better suited for my text terminal, and so I tell mutt to prefer text/plain, which, again in the muttrc, translates into:

    alternative_order text/plain
    

    And that's it: If the villains at cleverreach (the marketing firm I complained about) didn't have their treacherous text/plain alternative, my w3m would render their snooping HTML without retrieving their tracking pixel and I could read whatever they send me without them ever knowng if and when. I'm still not sure if that's the reason they have the nasty clickbait text/plain alternative. In general, I support the principle that you should never explain with malice what you can just as well explain with stupidity. But then we're dealing with a marketing firm here…

    Anyway: The best part of this setup is that you can quote-reply to HTML mails, giving your replies inline as $DEITY wanted e-mail to work. That is something that also is nice when folks send around MS office files (I get the impression that still happens quite a lot outside of my bubble). To cater for that, I have in my mailcap:

    application/msword; antiword %s; copiousoutput
    application/vnd.openxmlformats-officedocument.wordprocessingml.document; docx2txt %s - ; copiousoutput
    

    and consequently in the muttrc:

    auto_view application/msword
    auto_view application/vnd.openxmlformats-officedocument.wordprocessingml.document
    

    I admit I actually enjoy commenting inline when replying to office documents, and I trust antiword (though perhaps docx2txt a bit less) to not do too many funny things, so that I think I can run the risk of auto-rendering MS office files. I've not had to regret this for the, what, 15 years that I've been doing this for (in the antiword case; according to my git history, I've only given in to autorendering nasty docx in 2019).

    I mention in passing that I have similar rules for libreoffice, but there I have a few lines of python to do the text rendering, and that is material for another post (also, folks decent enough to use libreoffice are usually decent enough to not send around office files, and hence auto-displaying ODT is much less of a use case).

    Two more remarks: This actually cooperates nicely with rules not using copiousoutput. So, for instance, I also have in my mailcap:

    text/html; x-www-browser file://%s
    

    With that, if need by I can still navigate to an HTML file in the attachments menu and then fire up a “normal” browser (with all the privacy implications).

    And: people indecent enough to mail around MS office files often are not even decent enough to configure their mail clients to produce proper media types. Therefore, mutt lets you edit these to sanity. Just hit v, go to the misdeclared attachment and then press ^E. Since the “Office Open XML“ (i.e., modern Microsoft Office) media types are so insanely long and unmemoralisable, I have made up a profane media type that I can quickly type and remember for that particular purpose:

    application/x-shit; libreoffice %s
    
    [1]In case you're not so at home in Unix, writing “mailcap (5)” means you should type man 5 mailcap (for “show me the man page for mailcap in the man section 5”) to read or skim the documentation on that particular thing. Explicitly specifying a section has a lot of sense for things like getopt (which exists in sections 1 and 3) and otherwise is just an indication that folks ought to have a look at the man page.
    [2]You can use programs like see (1) or even compose (1) to use the information in your non-mutt mailcaps.
  • Big Brother Award: Wieder nicht cleverreach

    Ich will nicht ganz verhehlen, dass mich die diesjährigen Big Brother Awards ein klein wenig enttäuscht haben. Gewiss waren das alles verdiente Empfänger, aber mein Favorit war wieder mal nicht dabei. Dabei gebe ich den Leuten vom FoeBuD (ich verweigere mich der Umbenennung in digitalcourage aus reiner Nostalgie), dass dessen Verfehlungen überschaubarer sind als die Bewegungsprofile von Lieferando und die garstigen Fintech-Teufeleien von Klarna.

    Dennoch sollte, so finde ich, cleverreach gelegentlich bedacht werden, idealerweise in einer Kategorie „Ausschnüffeln per DSGVO-FUD“. Diese wäre Einrichtungen vorbehalten, die anderen Einrichtungen Überwachungstechnik unterschieben, indem sie die von weiten Teilen der freien Presse geschürten Ängste vor der DSGVO ausnutzen.

    Dieses Thema wird schon durch das augenblickliche Cookie-Banner dieser Leute gesetzt, das modal (also: nichts geht, solange mensch nicht klickt) zunächst von „Privatsphäre respektieren“ redet und dann über allzu bekannte Dark Patterns versucht, den Leuten hart am Rande der Legalität Daten abzupressen:

    Gestauchter Screenshot des cleverreach-cookiebanners: Der Zustimmen-Button ist sehr einladend.

    (ich habe das Legalesisch rausgeschnitten, da es nichts zur Sache tut). Zum Mitschreiben: Ein Laden, der Datenschutz ernstnimmt, braucht keine Cookie-Banner und schon gar keine Dark Patterns. Immerhin, das will ich den Leuten lassen, funktionieren weite Bereiche der Webseite inzwischen (das war vor zwei Jahren noch anders) ohne Javascript. Tatsächlich werden ohne Javascript nicht mal Cookie-Banner ausgespielt (Disclaimer: ich habe nicht nachgesehen, ob dann auch wirklich keine unnötigen Cookies[1] verschickt werden).

    Etwas ungehaltener bin ich schon über die Sirenentöne zur DSGVO, mit denen cleverreach Unternehmen und, noch schlimmer, andere Einrichtungen verunsichert: „eine Mailingliste ist datenschutzmäßig total kompliziert, und wenn du eine betreibst, bist du schon halb im Knast“. Das ist natürlich Unfug, solange beim Abonnieren klar ist, was die Leute kriegen und der dazugehörige Dialog transparent gestaltet ist. Aber welcheR „EntscheiderIn“ – Technik- und Sachkenntnisse sind in solchen Positionen ja eher optional – könnte Sirenentönen schon widerstehen?

    Nun könnte ich mit so ein wenig DSGVO-FUD zur Not noch leben, selbst wenn er zu einer – nur in seltenen Fällen dem Datenschutz wirklich helfenden – Zentralisierung von EDV führt, hier nämlich von jeweils ein paar lokalen Mailinglisten auf jeder Menge voneinander isolierter Server zu einer Firma mit „310.000 Kunden“ mit entsprechend vielen Listen.

    Diese Kunden sind zum Beispiel die Bundesorganisation der GEW, die Ebert-Gedenkstätte in Heidelberg und das Landesmuseum für Arbeit und Technik in Mannheim (dessen aktuellen Namen „Technoseum“ verweigere ich mit gleicher Sturheit wie die „digitalcourage“). Stellt euch vor, ihr wisst, dass jemand von allen drei Läden Info-Mails abonniert hat, notabene freiwillig, was schon eine gewisse Identifikation mit den jeweiligen Zwecken vermuten lässt: Entsteht da nicht ganz von selbst ein Profil?

    Faustregel: „clever“ heißt im Internet so viel wie „fies”

    Aber ist denn die Bildung so eines Profils nicht gegen die DSGVO? Oh, mit hinreichend viel Skrupellosigkeit ist das kein Problem. Und damit komme ich zum wirklich verwerflichen Teil von cleverreachs Geschäft, der meines Erachtens Restzweifel im Hinblick auf die Skrupellosigkeit zuverlässig zerstreut.

    Dazu braucht es einen Blick in die von cleverreach verschickten Mails. Dies sehen typischerweise so aus:

    I     1        [multipa/alternativ, 7bit, 79K]
    I     2 ├─>    [text/plain, quoted, utf-8, 0.7K]
    I     3 └─>    [text/html, quoted, utf-8, 78K]
    

    – sie gibt also vor, dass mensch alternativ ordentlichen Text oder ein HTML-Dokument haben kann. Das Problem ist noch nicht mal, dass es da überhaupt HTML gibt (auch wenn anständige Menschen kein HTML in Mails packen). Das Problem deutet sich an darin, dass der Plain Text nur ein Hundertstel der Länge des HTML-Teils hat. Das ist auch beim Einrechnen des HTML-typischen Fluffs nicht mehr glaubhaft, und in der Tat ist der Plain-Text-Teil nur ein Köder, um Menschen zu cleverreachs Schnüffelseiten zu bringen:

    Ihr E-Mail Programm unterstützt leider keine HTML E-Mails.
    
    Hier finden Sie diesen Newsletter online:
    https://213989.seu2.cleverreach.com/m/dddddddd/dddddd-hhhhhhhhhhhhhhhhhhhhhhhhhh
    hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh
    

    (wo d Dezimalziffern und h Sedezimalziffern sind; mit Code im Umfang dieser URL könnten DemoprogrammiererInnen Bälle über den Bildschirm hüpfen lassen). Wer dieser ganz klar personalisierten URL folgt, bekommt immerhin keinen Cookie-Banner, und erstaunlicherweise wird auch kein Javascript ausgeliefert. In der Tat setzen die verschiedenen Ressourcen auch keine nennenswerten Cookies. Allein die URL:

    https://stats-eu2.crsend.com/stats/mc_dddddd_dddddddd_hhhhhhhhhhh-cccccc.gif
    

    (c steht jetzt für Kleinbuchstaben) setzt einen PHP-Session-Cookie – aber das ist vermutlich nur Gedankenlosigkeit. Das wirkliche Problem an dieser Ressource: das ist ein Tracking-Pixel, in schlechter Tradition als leeres GIF von 40 Bytes, das aber (was passiert da wohl im Hintergrund?) jetzt gerade 5 Sekunden für die Auslieferung gebraucht hat.

    Hiermit nimmt cleverreach auf, wer ihre Mails liest und wann sie das tun. Nennt mich paranoid, aber ob und wann ich Werbepost lese, das geht wirklich nur mich etwas an. Ich will auf keinen Fall, dass jemand merkt, wenn ich auf Clickbait hereinfalle.

    Wenig überraschenderweise findet sich dieser Trackversuch auch am Ende der ganz klassisch als Tabelle ausgezeichneten HTML-Alternative in der Mail:

      </center>
      <img src="https://stats-eu2.crsend.com/stats/<wie oben>"
        border="0" alt="" height="1" width="1"></body>
    </html>
    

    Bei HTML dieser Art (und überhaupt: GIFs) werde ich wieder jung: so haben wir Mitte der 1990er Webseiten geschrieben.

    Trotz dieses Schnüffelversuchts wäre es übrigens (wenn es jetzt schon HTML sein muss) datenschützerisch besser, wenn cleverreach den Plain Text-Teil nicht einbasteln würde, denn dann würde mein datensparsames eigenes HTML-Rendering aktiv werden und ich könne die Mail lesen, ohne (wie im normalen Browser fast unvermeidlich) getrackt zu werden. Wie das alles mit meinem Mail-Client zusammengeht, beschreibe ich demnächst mal; ich bestehe jedenfalls darauf, dass Plain Text-Alternativen, die nur Clickbait enthalten, unethischer sind als gar kein Plain Text.

    Entweder Internet-Normalbetrieb oder DSGVO.

    Zugegeben: das verschlagene Unterschieben von Trackingpixeln, um mitzukriegen, wann wer was liest und das dann den eigenen KundInnen als total wichtige Marketingmetrik verkaufen zu können: Das könnte in Zeiten von Google Analytics schon fast als lässliche Sünde durchgehen. Wer sich auf Schurkereien dieser Art einlässt, darf jedoch im Gegenzug nicht Gewerkschaften und SPD-nahe Stiftungen (samt ihrer im Datenschutz eher unbeholfenen HauspolitiologInnen für antisoziale Medien) mit Datenschutzraunen und -drohen von datensparsamen Verfahren in das eigene überwachungskapitalistische Geschäftsmodell ziehen.

    [1]Ich lasse die Frage, ob es wohl „notwendige Cookies“ überhaupt gibt, mal beiseite, denn Antworten auf diese bräuchten jede Menge Platz für haarige technische und soziale Betrachtungen sowie eine Großpackung Wenns und Abers.
  • A Feedback Form in Pelican

    I realise that the great days of discussions on blogs are over, as Sam Hartman blogged the other day – at least for now. Still, I'd like to make it somewhat more straightforward to send me feedback on the posts here than having to get the contact address and dropping me a mail. Hence, I've written a little Python script, feedback, that lets people comment from within their web browsers.

    Nachtrag (2022-10-07)

    Don't take it from here; rather, see https://codeberg.org/AnselmF/pelican-ext

    While the script itself is perfectly general and ought to work with any static blog engine, the form template I give in the module docstring is geared towards pelican and jinja, although only in very few places.

    To make it work, this needs to become a CGI (the template assumes it will show up in /bin/feedback according to the server configuration). The notes on deployment from my post on the search engine apply here, too, except that in addition the host has to be able to deliver mail. Most Unix boxes do locally, but whether anyone reads such mail is a different question.

    Is it ethical to check “ok to publish” by default?

    To configure where it sends mail to (by default, that's root, which may make sense if you have your own VM), you can set the CONTACT_ADDRESS environment variable (see the search engine post in case you're unsure how to do that for a web context). If your machine is set up to deliver mail to remote addresses – be it with a full mail server or using a package like nullmailer –, you could use your “normal” mail address here. In that case, you probably should inform people in your privacy policy that their comments will be sent by unencrypted mail, in particular if that “normal“ e-mail is handled by one of the usual rogues (Google still gets about a half of the mail I send – sigh).

    If you look below (or perhaps right if you run your browser full-screen), you will see that there is a checkbox “feel free to publish“ that, right now, I have checked by default. I had some doubts about that in terms of creepy antipatterns. Of course I am as annoyed by most contemporary cookie banners as anyone else, which in violation of the GDPR usually have practical defaults – sure: not what you get when you say “configure” – set at the maximum creep level the operators believe they can get away with. On the other hand, defaults should also be expectable, and I'd guess the expectation when someone fills out a reply form on a blog is that the response will be published with the article. If you disagree: well, the comment form is there for you.

    In terms of spam protection, I do an incredibly dumb textcha. Even if this script got deployed to a few dozen sites (which of course would be charming), I cannot see some spam engine bothering to figure it out; since it just sends a mail to the operator, there is basically nothing to be gained from spamming using the CGI. I fully expect this will be enough to keep out the dumb spambots that blindly send whatever forms they can find – it has worked on many similar services.

    Security Considerations

    The feedback script does at least two things that could be exploited:

    1. It enters remotely controlled values into rendered HTML and
    2. It calls a local binary with content controlled by the remote user.

    In case (1) – that's when I put the URI of the originating article into the reply message to send people back to where they came from –, this can potentially be exploited in cross-site attacks. Suppose you trust my site on only execute benign javascript (I give you that's close to an oxymoron these days), someone could trick you into clicking on a link that goes to my site but executes their, presumably adversarial, javascript.

    Bugs aside, the script is resilient against that, as it properly escapes any user input that gets copied into the output. That is thanks to my “micro templating“ that I keep around to paste into such one-script wonders. Have a look into the code if you're interested in how that works. And totally feel free to paste that into any Python code producing HTML or XML templated in any way – sure, it's not jinja or stan, but it has covered 80% of my templating needs at much less than 20% of the effort (counted in code lines of whatever dependency you'd pull in otherwise), which is a good deal in my book.

    Case (2) is probably a lot more interesting. In the evaluate_form function, I am doing:

    mail_text = MAIL_TEMPLATE.format(**locals())
    

    Code like this usually is reason for alarm, as far too many text formats can be used to execute code or cause other havoc – the cross-site thing I've discussed for HTML above being one example, the totally bizarre Excel CSV import exploit another (where I really cannot see how this doesn't immediately bring every Windows machine on this planet to a grinding halt). In this case, people could for example insert \ncc: victim@address into anything that gets into headers naively and turn the form into a spam engine.

    There are exactly 10000 lines if Python's email module in version 3.9.

    In addition, there is a concrete risk creating some way of locally executing code, as the template being filled out is then used as an input for a local program – in this case, whatever you use as sendmail. In theory, I'm pretty sure this is not a problem here, as no user-controlled input goes into the headers. If you change this, either sanitise the input, probably by clamping everything down to printable ASCII and normalising whitespace, or by parsing them yourself. The message content, on the other hand, gets properly MIME-encapsulated. In practice, I can't say I trust Python's email package too much, as by Python stdlib standards, it feels not terribly mature and is probably less widely used than one may think.

    But that's a risk I'm willing to take; even if someone spots a problem in the email module, shodan or a similar service still has no way to automatically figure out that it is in use in this form, and my page's insignificance makes it extremely unlikely that someone will do a targeted attack on day 0. Or even day 10.

    But then, perhaps this is a good occasion to read through email's source code? Fun fact: in python 3.9, a find . -name "*.py" | xargs wc -l gives exactly 10000 lines. And my instinct that headers are the trickiest part is probably right, too: 3003 of those are in _header_value_parser.py.

  • Mutt says: “error encrypting data: Unusable public key”

    Today, I replied to an encypted mail, and right after the last “yes, go ahead, send this stuff already”, my mail client mutt showed an error:

    error encrypting data: Unusable public key
    

    Hu? What would “unusable” mean here? The message when all PGP keys are expired looks quite a bit different. And indeed, the key in question was not expired at all:

    $ gpg --list-keys person@example.net
    pub   rsa4096/0xDEEEEEEEEEEEEEEE 2015-03-21 [SCA] [expires: 2023-02-01]
          FINGERPINTWITHHELDFINGERPRINTWITHHELDFIN
    uid                   [  full  ] Person <person@example.net>
    

    – this should do for another year or so. Or should it?

    Feeding the message to a search engine brings up quite a few posts, most of them from times when keyservers would mess up subkeys, i.e., the cryptographic material that is used to actually encrypt stuff (as opposed to the main key that usually just authenticates these subkeys).

    This obviously did not apply here, since keyservers have long been fixed in this respect. But subkeys were the right hint. If you compare the output above with what such a command will output for the feedback key for this blog:

    $ gpg --list-keys zuengeln@tfiu.de
    pub   rsa3072/0x6C4D6F3882AF70AD 2021-01-28 [SC]
          60505502FB15190B10DBF1436C4D6F3882AF70AD
    uid                   [ultimate] Das Engelszüngeln-Blog <zuengeln@tfiu.de>
    sub   rsa3072/0x3FCFC394D8DF7140 2021-01-28 [E]
    

    you'll notice that the Person's key above does not have a sub line, i.e., there are no subkeys.

    How can that happen? Gnupg won't create such a thing without serious amounts of coercion, and such a key is largely useless.

    Well, it turns out it doesn't happen. The subkeys are there, gnupg just hides them because that's what it does with expired subkeys by default. If you override that default, you'll get:

    $  gpg --list-options show-unusable-subkeys --list-keys person@example.net
    pub   rsa4096/0xDEEEEEEEEEEEEEEE 2015-03-21 [SCA] [expires: 2023-02-01]
          FINGERPINTWITHHELDFINGERPRINTWITHHELDFIN
    uid                   [  full  ] Person <person@example.net>
    sub   rsa4096/0xEEEEEEEEEEEEEEEE 2015-02-01 [E] [expired: 2020-01-31]
    sub   elg4096/0xEEEEEEEEEEEEEEEE 2020-02-01 [E] [expired: 2022-01-31]
    

    So, that's the actual meaning of the error message about „Unusable public key“: “No usable subkey”.

    What's a fix for that? Well, for all I know you cannot force gnupg to encrypt for an expired key, so the way to temporarily fix things (for instance, to tell people make their keys permanent[1]) is to turn the clock. There's the nice program faketime that just changes the time for whatever runs below it. That's great because on modern computers, changing the system time has all kinds of ugly side effects (not to mention you'd have to kill the ntpd that your computer quite likely runs to keep your computer's clock synchronised with the rest of the world).

    Since I'm using mutt as a mailer, I'd use faketime like this:

    faketime 2022-01-31 mutt
    

    I'm fairly confident this would work with, say, thunderbird as well, though it might be a problem if the times of an X server and client are dramatically different.

    But that's really no substitute for an updated key: In most people's mailboxes, such mails will be way down in the swamp of rotting mails from one month ago[2] And mail servers sometimes refuse to transport mail that's so far from the past.

    Then again, to my own surprise, everytime I had to go to such extremes because I didn't have a non-expired key, the recipients eventually noticed.

    [1]Let me again advertise non-expiring keys. The main arguments for these are that (a) essentially nobody directly attacks keys, so it really doesn't matter if a key is used for a decade or more, and (b) PGP is hard enough for muggles even without auto-destructing keys. The net effect of expiring keys on privacy is thus negative, because they keep people off using PGP and even trying to understand crypto. And you can always revoke keys, in particular when we have educated people to now and then sync their keyring with keywervers.
    [2]As a side note: While inbox zero sounds to much like one of those market-radical self-improvement fads to me, I've been religious about less-than-a-page inbox for the past decade or so and found it did improve a relevant part of my life.
  • Long live PGP

    The other day I ran into two rants against PGP, What's the matter with PGP?, which still is relatively reasonable, and then the raving PGP Problem by people running a security consulting shop called Latacora. It's this second diatribe that made me write this post, because the amount of badmouthing of PGP done there, on the blog of a company promising to teach startups “security“ on top, is not only unwarranted, it's also actively damaging to meaningful encryption.

    Let me start with what I think are the fundamental fallacies of the Latacora folks is: They seem to think that identity, and hence key management, is something that others can do for you, and that thinking about crypto is “bad user experience“. In contrast, I'd argue that crypto you don't notice isn't crypto at all (but instead somewhere on the obfuscation spectrum), and that identity management done by others is equivalent to them encrypting for you.

    You see, if you're encrypting something, you're encrypting it for someone. In public key encryption, this “someone“ has two aspects: a real-world entity (your friend, a bank, whatever), and a public key. Any crypto system that does not make this separation transparent and asks you the question of how well you think the two things match (and whether you care), is fundamentally broken. That's because that question plainly has to be answered. If it's not you who answers it, it's someone else. And hence that someone else is free to change the mapping from key to real-world entity, and hence they determine which real-world entity gets to read what you've encrypted.

    This, by the way, is how https is regularly subverted by businesses, anti-virus software, and occasionally state actors, who simply make your browser trust their word on who is what. At that moment, they can inspect everything your browser exchanges with the rest of the world, be it in “anti-virus portals” or bluntly in surveillance systems. This is also why very few users of “encrypted” messengers would even notice if the operating company snooped on them.

    The big advantage of PGP is what the Latacora people call „obnoxious UX“. Yes, you have to make up your mind on keys, yes, you have to explicitly manage them: but that is what you need to understand if you want meaningful encryption, and plastering abstractions on top of that only gives extra levels people have to understand – and that can break. No: if you want to do encryption, you'll have to understand key management, and PGP makes that as explicit and transparent as any crypto system I've ever seen. That's a feature, not a bug.

    Actually, this one thing is far more important than regular key rotation (as an aside: I'm not aware of anyone ever having broken a PGP secret key because too much material has been encrypted using it; it's certainly not a major reason for failing encryption) or the latest cryptographic primitives (even 1024 bit RSA keys still require serious investment to break, 20 years after they've been state of the art).

    More generally, the scenarios requiring frequent key rotation mostly imagine a targeted attack from a state actor, following you into hotel rooms to steal your secret key and install key loggers to skim your passphrase (or similar). Frankly: it's an illusion to believe a muggle (or even normal wizards doing their daily work) could withstand such a determined and comptentent state actor for a long time.

    But that's not necessary. It's already great progress if Google, the (normal) police, and the people at your employer's computation centre can't read the content of your mails unless they really try hard. Actually, most “adversaries” are not terribly determined and/or terribly competent. You hence don't need a perfect cryptosystem. It just needs to withstand the most basic of man-in-the-middle attacks (which fends off the not terribly competent adversaries), and it needs to require at least a bit of individual effort to break each person's crypto (which fends off the not terribly determined ones). Any system with centralised identity management fails at least on the side of the individual effort – get the central entity to collude, or break it, and you have it all. And given the central entity at least legally sits somewhere if it's commercial, at least for the state of residence it fails the withstand-most-basic criterion as well: The average state has no trouble obtaining court orders as soon as it moans “terrorism”.

    PGP, on the other hand, does a fairly good job on both counts once people have grokked it, at least a much better one than anything SSL-based I've ever seen. People can understand the basic operations of PGP if they want, something that is much harder with SSL and X.509. The problem is that few people want to understand that. But in that case, any kind of crypto is doomed. Hence, the problem isn't PGP, it is, as so often, education. Working on that is effort well spent, as once people have understood PGP to the level of confidently using it, they have a much better chance of being able to competently use other, less explicit crypto systems.

    Having said that: sure, PGP and its UIs could be improved. But you can't get around PGP's long-term keys for e-mail, and whatever alternative you'd come up with, you'll still need keyrings and reasonable UIs to mark up the trust in there. And, in particular, you'll still need open standards so you don't have to take a single company's word for what it does.

    That's basically where I think Latacora's arguments are harmful. But since most of the claims in the article are fairly outrageous, I can't resist commenting them, too:

    • “Absurd Complexity“ – that's not a terribly credible charge given the page comes over HTTPS, which in many ways is a lot more complex than OpenPGP, in particular because it contains the nightmare that is X.509. But really: Much as I am all for reducing complexity, I'm even more for maintaining backward compatibility. Being able to read encrypted mails I got in the 1990ies matters to me, and anything flexible enough to at least support modern crypto and deal with archive data will have to have a certain amount of complexity. Start something new, and in 10 years it'll look the same. Or worse. Only it won't be able to read archival data, and it'll have repeated the history of early bugs software simply has, in particular when you have to worry about side channel attacks.
    • “Swiss Army Knife“ – that would be more convincing if they said how exactly PGP signatures are “mediocre“ or the encryption is a “pretty bad job“. I accept the argument they make a bit down that people may want to have forward secrecy in IM and it's easy to have there, so going for PGP alternatives may be a good idea there. But then I can't remember the last time I used PGP over XMPP, so these alternatives have existed for a long time, without any need to kill PGP.
    • “Mired in Backwards Compatibility“ – well, as said above, backwards compatiblity is great if your systems live a long time. And OpenPGP is doing a reasonable job to have both backwards compatiblity and evolvability. That rolling out new features isn't instantaneous, in particular for a federated, largely volunteer effort, is useless criticism: Build another distributed, open, volunteer effort, and it'll go the same way. Oh, and just incidentally: HTTPS is from the nineties, too, and the X.509 standard was published in 1988.
    • ”Obnoxious UX“, “Long-Term Secrets“, “Incoherent Identity“ – these are the core of Latacora's fallacies; see above.
    • “Broken Authentication“ – I can't say I've thought through the problem they're pointing to here; if it is as bad as they claim, there's no reason not to fix it within OpenPGP rather than invent something entirely new.
    • “Leaks Metadata“ – sure. As long as there's SMTP, there's no way around having quite a bit of cleartext: intermediate mail servers will have an idea who's mailing whom (though of course there's still Mixmaster). Having e-mail, or at least something that doesn't require me to be always online, is so important that the metadata leaks are an almost negligible price to pay, at least compared to the fine-grained activity profile you leak when you leave your phone online all the time, or the loss of control when you (or people you trust) can't run the necessary infrastructure components (as in: a mail server) any more.
    • ”No Forward Secrecy“ – fair enough, but again that's hard to have in store-and-forward e-mail, in particular when you'd like to have mailing lists and the like. Of course, it's true that »[a]gainst serious adversaries and without forward secrecy, breaches are a question of “when”, not “if”.« But on the other hand, most serious adversaries lose interest in the cleartext hours, weeks, or at worst a few years after the communication, or at least stop being terribly serious. So, for these intents and purposes holding …
  • Fürchten lernen 3: Microsoft

    Nach dem zweiten Teil meiner Leidensgeschichte über den Betrieb eines Mailservers in der Postmoderne ist es ruhig geworden: Nachdem ich erstmal das DNS vertrauenswürdig gestaltet hatte, fanden eigentlich alle, bei denen mein Server Mails einliefern wollte, dessen Reputation reiche schon hin.

    Ich hegte also Hoffnung, dass das alles nicht so schlimm ist wie ich geglaubt hatte, dass SMTP noch nicht kaputt ist und mensch mit vernünftigem Aufwand selbst Mail verteilen kann. Bis heute morgen, als von meinen Mailserver das hier zurückkam:

    XXXXXXXXXXXXX@XXXXXXXXXX.de
       host XXXXXXXXXXXX01i.mail.protection.outlook.com [104.47.2.36]
       SMTP error from remote mail server after RCPT TO:<XXXXXXXXXXXXX@XXXXXXXXXX.de>:
       550 5.7.606 Access denied, banned sending IP [116.203.206.117]. To request
     removal from this list please visit https://sender.office.com/ and follow the
     directions. For more information please go to
     http://go.microsoft.com/fwlink/?LinkID=526655 AS(1430)
    

    Oh nein. Microsoft. Na, mal sehen.

    Ich schicke meinen Web-Browser zu sender.office.com, und es erscheint einen Dialog mit einem kreiselnden Wartedings, ein sicheres Zeichen, dass da mal wieder „Web-Programmier“ am Werk waren, die einfach so voraussetzen, dass sie Code auf allen Maschinen ausführen dürfen. Na super: Mails verschicken geht nicht, ohne Microsoft-Code laufen zu lassen.

    Aber egal, sollen sie halt auf meine CPU; das Javascript zieht ein Captcha ein (zum ersten Mal bin ich froh, dass das Microsoft ist, denn so ist es immerhin kein Google-Captcha und ich muss nicht deren „KI“ trainieren). Captcha gelöst, für ein paar Sekunden passiert nichts, dann eine Meldung:

    Step 1: Our messaging service has experienced a temporary issue,
    please resubmit your information below.
    

    Mach ich das mit dem resubmit, gleiche Reaktion. Hatte ich was anderes erwartet?

    Ok, kann ja sein, dass die einfach keine gute Fehlermeldung haben und das irgendsoein dämlicher CSRF-Schutz mit Referrern ist (auch wenn mir wirklich nicht klar wäre, wer auf diesem Zeug CSRF machen wollen könnte). Lasse ich meinen Browser also noch Referrer-Header schicken. Keine Änderung, immer noch kaputt.

    An der Stelle werde ich sauer und will irgendwem eine zornige Mail schicken. Aber: Keine Kontaktadresse, nirgends. Ich überlege, ob ich eine Telemediengesetz-Beschwerde auf den Weg schicken soll, denn zweifelsohne ist das ein kommerzielles Angebot unter deutschem Recht (der Empfänger ist eine hiesige Firma, für die Microsoft das bestimmt nicht umsonst macht). Vielleicht. Aber bevor ich mit einem Scheiß wie dem Telemediengesetz argumentiere, brauche ich noch mehr Verzweiflung.

    Aber gucken wir erstmal, was da „more information“ ist. Ich lasse meinen Browser also auf http://go.microsoft.com/fwlink/?LinkID=526655 los und bekomme ein Redirect auf:

    https://technet.microsoft.com/library/mt661881(v=exchg.150).aspx
    

    Sowohl auf luakit als auch per curl reagiert der Microsoft-Server darauf mit:

    Unable to process request
    

    Gratuliere. Was machen diese Leute eigentlich beruflich?

    Aber wurst, werfe ich halt meinen Browser für wüste, grob privatsphäreverletzende Seiten an – mein Leben ist zu kurz, um den Quatsch, den die Firma da auf die Menschheit loslässt, verstehen zu wollen. Mit dem Browser fürs Grobe kommt dann auch was: eine Seite mit viel Rede von „Defender”. Aber keine nützliche Information, und so probiere ich halt sender.office.com nochmal mit dem Kamikaze-Browser.

    Ergebnis: Our messaging service has experienced a temporary issue, please resubmit your information below.

    Immerhin hat ist auf der Info-Seite etwas, das nach einem Kontakt-Link aussieht. Aber nein, es ist eine Issue-Seite auf github, https://github.com/MicrosoftDocs/feedback/issues, 1630 Open Bugs. Jaklar, da schreibe ich meinen noch dazu. Was glauben diese Leute eigentlich?

    Stattdessen habe ich dann heute vormittag an postmaster@office.com geschrieben, und Mail ist nicht gleich zurückgewiesen worden. Entsprechend hatte ich da noch Hoffnung, Microsoft könnte sich immerhin an diesen Teil der Mail-RFCs halten. Jetzt, 18 Uhr, sieht es nicht danach aus, da kam genau keine Reaktion (im diesem Vergleich, es schmerzt mich, das zuzuzugeben, sieht die Telekom viel besser aus). Und die sender.office.com-Seite ist auch noch kaputt.

    Andererseits: Offensichtlich haben die Exchange-Leute gerade tatsächlich außergewöhnliche Probleme. Vielleicht habe ich einfach nur Pech gehabt und das ist sonst nicht so rekursiver Murks?

    Nachtrag (2021-03-12)

    (um 15:30) Immerhin weist Microsoft Mails an postmaster nicht so rüde ab wie andere Mails. Mein Mailserver schreibt mir gerade, dass er es 24 Stunden lang nicht geschafft hat, die Mail an postmaster@office.com auszuliefern, dass er es aber weiter probieren wird. Es bleibt spannend. Unterdessen findet aber yahoo.de, dass es mit meiner Reputation bergab geht. Oh je. Das sieht nach einem Alptraum mit Verzögerung aus.

    Nachtrag (2021-03-15)

    (mittags) Nee, natürlich ist nichts bei postmaster@office.com einzuliefern. Wo kämen wir da auch hin. Dafür macht inzwischen wenigstens die sender.office.com-Geschichte etwas, wenn auch von der Mail, die das verspricht, innerhalb von 10 Minuten nichts zu sehen ist.

    Nachtrag (2021-03-15)

    (abends) Nach noch einem Versuch mit der sender.office.com-Geschichte kam dann auch eine Mail mit einem Link, und dessen Derefenenzierung hat tatsächlich etwas produziert, das versprach, mein Server werde innerhalb von 30 Minuten von der Blacklist genommen. Schon frech, wie dieser Laden über die Zeit anderer Menschen verfügt. Auf der anderern Seite: jetzt will ich da gar niemandem mehr Mails schreiben. Pfft.

  • Mailman3: "Cannot connect to SMTP server localhost on port 25"

    I've been a fairly happy mailman user for about 20 years, and I ran mailman installations for about a decade in the 2000s.

    Over the last week or so, I've spent more time setting up a mailman3 list off and on than I've spent with mailman guts in all the years before, which includes recovery form one or two bad spam attacks. Welcome to the brave new world of frameworks and microservices.

    Perhaps the following words of warning can help other mailman3 deployers to not waste quite as much time.

    Badly Misleading Error Messages

    Most importantly, whatever you do, never call mailman as root. This will mess up permissions and lead to situations really hard to debug. In particular, the error message from the post's title:

    Cannot connect to SMTP server localhost on port 25
    

    apparently can have many reasons (or so the recipes you find on the net suggest), few of which have anything to do with SMTP, but one clearly is when mailman can't read or write to queue files or templates or whatever and bombs out while trying to submit mail.

    Morale: Don't claim too much when writing error messages in your programs.

    Unfortunately, I've fixed the thing accidentally, so I can't say what exactly broke. The take away still is that, in Debian (other installations' mailman users might be called something else) you run mailman like this:

    sudo -u list mailman
    

    However, I can now say how to go about debugging problems like these, at least when you can afford a bit of mailman unavailability. First, stop the mailman3 daemon, because you want to run the thing in the foreground. Then set a breakpoint in deliver.py by inserting, right after def deliver(mlist, msg, msgdata), something like:

    import pdb; pdb.set_trace()
    

    Assuming Debian packaging, you will find that file in /usr/lib/python3/dist-packages/mailman/mta.

    Of course, you'll now need to talk to the debugger, so you'll have to run mailman in the foreground. To do that, call (perhaps adapting the path):

    sudo -u list /usr/lib/mailman3/bin/master
    

    From somewhere else, send the mail that should make it to the mail server, and you'll be dropped into the python debugger, where you can step until where the thing actually fails. Don't forget to remove the PDB call again, as it will itself cause funky errors when it triggers in the daemonised mailman. Of course, apt reinstall mailman3 will restore the original source, too.

    Template Management Half-Broken

    When I overrode the welcome message for a mailing list, the subscription notifications to the subscribing users came out empty.

    This time, there was at least something halfway sensible in the log:

    requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://localhost/postorius/api/templates/list/kal.sofo-hd.de/list:user:notice:welcome
    

    Until you read up on the mailman3 system of managing templates (which, roughly, is: store URIs from where to pull them), it's a bit mystifying why mailman should even try this URI. Eventually, one can work out that when you configure these templates from Postorius, it will take the URI at which mailman should see it, Postorius, from POSTORIUS_TEMPLATE_BASE_URL in /etc/mailman/mailman-web.py. This is preconfigured to the localhost URI, which proabably only rarely is right.

    To fix it, change that setting to:

    POSTORIUS_TEMPLATE_BASE_URL = 'http://<your postorious vserver>/postorius/api/templates/'
    

    Of course it'll still not work because the old, wrong, URI is still in mailman's configuration. So, you'll have to go back to the template configuration in Postorius and at least re-save the template. Interestingly, that didn't seem to fix it for me for reasons I've not bothered to fathom. What worked was deleting the template and re-adding it. Sigh.

    As soon as you have more than one template, I expect it's faster to change the URIs directly in mailman's database, which isn't hard, as seen in the next section.

    [Incidentally: does anyone know what the dire warnings in the docs about not using sqlite3 on “production” systems actually are about?]

    Disable Emergency Moderation After Moving

    Basically because I was hoping to get a more controlled migration, I had set one list on the old server to emergency moderation before pulling the config.pck. Don't do that, because at least as of now mailman3 has the notion of emergency moderation but makes it hard to switch it on or off. I eventually resorted to directly touching mailman's config database (if you've configured mailman to use something else than sqlite, the shell command is different, but the query should be the same):

    $ sudo -u list sqlite3 /var/lib/mailman3/data/mailman.db
    [on the sqlite prompt:]
    update mailinglist set emergency=0 where list_id='<your list id>';
    

    Note that <your list id> has a dot instead of the at, so if your list is mylist@example.org, its id is mylist.example.org.

    Oh No, CSRF Token

    The list I cared about most could be joined from an external web site, transparently posting to mailman2's cgi-bin/mailman/subscribe (oh! CGI! How am I missing you in the age of uwsgi and Django!). Looking at its counterpart for modern mailman3, the first thing I noted is that there's a CSRF token in it – if you've not encountered them before, it's a couple of bytes the originating server puts into a web form to prevent what Postorius' authors feels is Cross Site Request Forgery.

    Of course, what I wanted was exactly that: Post to Postorius from a different web site. I don't feel that's forgery, very frankly.

    I didn't see an obvious way to turn it off, and I was a bit curious about mailman3's own http API, so I wrote a few lines of code to do this; the API part itself was straightforward enough, something like:

    result = requests.post(
      getConfig("mailmanAPI")+"/members", {
        'list_id': getConfig("mailmanListname"),
        'subscriber': toSubscribe,
        'pre_verified': False,
        'pre_confirmed': False,
        'pre_approved': True,},
      auth=(getConfig("mailmanAPIUser"),
        getConfig("mailmanAPIPassword")),
      timeout=1)
    

    – but of course it sucks a bit that subscribing someone requires the same privilege level as, say, creating a mailing list or changing its description. And all that just to work around CSRF prevention. Sigh.

    On top of that, I've tried K-SAT on the pre_X booleans to try and see if anything gives me the tried and tested workflow of “let folks enter a mail address, send a confirmation link there, subscribe them when it's being clicked“. No luck. Well, let's hope the pranksters don't hit this server until I figure out how to do this.


    Hm. I think I'm a bit too locked in into mailman to migrate away, but I have to say I wish someone would port mailman2 to python3 and thus let mailman2 hang on essentially forever. It did all a mailing list manager needs to do as far as I am concerned, and while it wasn't pretty with the default browser stylesheets, even now, almost a decade into mailman3, it works a whole lot more smoothly.

    Or perhaps there's a space for a new mailing list manager with a trivially deployable web interface not requiring two separate database connections? Perhaps such a thing exists already?

    Well, summing up, the central migration advice again: Mind the sudo option in

    sudo -u list mailman import21 my-list@example.org config.pck
    
  • A Mail Server on Debian

    After decades of (essentially) using other people's smarthosts to send mail, I have recently needed to run a full-blown, deliver-to-the-world mail server (cf. Das Fürchten lernen; it's in German, though).

    While I had expected this to be a major nightmare, it turns out it's not so bad at all. Therefore I thought I'd write up a little how-to-like thing – perhaps it will help someone to set up their own mail server. Which would be a good thing. Don't leave mail to the bigshots, it's too important for that.

    Preparation

    You'll want to at least skim the exim4 page on the Debian wiki as well as /usr/share/doc/exim4/README.Debian.gz on your box. Don't worry if any of that talks about things you've never heard about at this point and come back here.

    The most important thing to work out in advance is to get your DNS to look attractive to the various spam estimators; I didn't have that (mostly because I moved “secondary” domains first), which caused a lot of headache (that article again is in German).

    How do you look attractive? Well, in your DNS make sure the PTR for your IP is to mail.<your-domain>, and make sure mail.<your-domain> exists and resolves to that IP or a CNAME pointing there. Note that this means that you can forget about running a mail server on a dynamic IP. But then dynamic IPs are a pain anyway.

    Before doing anything else, wait until the TTL of any previous records of this kind has expired. Don't take this lightly, and if you don't unterstand what I've been saying here, read up on DNS in the meantime. You won't have much joy with your mail server without a reasonable grasp of reverse DNS, DNS caching, and MX records.

    Use the opportunity to set the TTL of the MX record(s) for your domain(s) to a few minutes perhaps. Once you have configured your mail system, you can then quickly change where other hosts will deliver their mail for your domain(s) and raise the TTLs again.

    Exim4

    Debian has come with the mail transfer agent (MTA; the mail server proper if you will) exim4 installed by default for a long, long time, and I've been using it on many boxes to feed the smart hosts for as long as I've been using Debian. So, I'll not migrate to something else just because my mail server will talk to the world now. Still, you'll have to dpkg-reconfigure exim4-config. Much of what's being asked by that is well explained in the help texts. Just a few hints:

    • “General type of mail configuration” would obviously be “internet site“.
    • Mail name ought to be <your domain>; if you have multiple domains, choose the one you'd like to see if someone mails without choosing any.
    • Keep the IP addresses to listen on empty – you want other hosts to deliver mail on port 25. Technically, it would be enough to listen only on the address your MX record points to, but that's a complication that's rarely useful.
    • Relaying mail for non-local domains is what you want if you want to be a smart host yourself. You'll pretty certainly want to keep this empty as it's easy to mess it up, and it's easy to configure authenticated SMTP even on clients (also see client connections on avoiding authenticated SMTP on top).
    • Exim also is a mail delivery agent (MDA), i.e., something that will put mail for domains it handles into people's mail boxes. I'll assume below that you select Maildir format in home directory as the delivery method. Maildir is so much cooler than the ancient mboxes, and whoever wants something else can still use .forward or procmail.
    • And do split your configuration into small files. Sure, you'll have to remember to run update-exim4.conf after your edits, but that litte effort will be totally worth it after your next dist-upgrade, when you won't have to merge the (large) exim4 config file manually and figure out what changes you did where.

    DNS Edits

    With this, you should be in business for receiving mail. Hence, make your MX record point to your new mail server. In an NSD zone file (and NSD is my choice for running my DNS server), this could look like:

    <your domain>.  IN MX 10 <your domain>.
    

    (as usual in such files: Don't forget the trailing dots).

    A couple of years ago, it was all the craze to play games with having multiple MX records to fend off spam. It's definitely not worth it any more.

    While I personally think SPF is a really bad idea, some spam filters will regard your mail more kindly if they find an SPF record. So, unless you have stronger reasons to not have one than just “SPF is a bad concept and breaks sane mailing list practices, .forward files and simple mail bouncing”, add a record like this:

    <your domain>.                3600    IN      TXT     "v=spf1" "+mx" "+a" "+ip4:127.0.0.1" "-all"
    

    – where you have to replace the 127.0.0.1 with your IP and perhaps add a similar ip6 clause. What this means: Mail coming from senders in <your domain> ought to originate at the IP(s) given, and when it comes from somewhere else it's fishy. Which is why this breaks good mailing list practices. But forunately most spam filters know that and don't interpret these SPF clauses to narrow-mindedly.

    SSL

    I'm not a huge fan of SSL as a base for cryptography – X.509 alone is scary and a poor defense against state actors –, but since it's 2021, having non-SSL services doesn't look good. Since it's important to look good so people accept your mail, add SSL to your exim installation.

    Unless you can get longer-living, generally-trusted SSL certificates from somewhere else, use letsencrypt certificates. Since (possibly among others) the folks from t-online.de want to see some declaration who is behind a mail server on such a web site, set up a web server for mail.<your-domain> and obtain letsencrypt SSL certificates for them in whatever way you do that.

    Then, in the post-update script of your letsencrypt updater, run something like:

    /bin/cp mail.crt mail.key /etc/exim4/ssl/
    /usr/sbin/service exim4 restart
    

    (which of course assumes that script runs as root or at least with sufficient privileges). /etc/exim4/ssl you'll have to create yourself, and to keep your key material at least a bit secret, do a:

    chown root:Debian-exim /etc/exim4/ssl
    chmod 750 /etc/exim4/ssl
    

    – that way, exim can read it even if it's already dropped its privileges, but ordinary users on your box cannot.

    Then tell exim about your keys. For that, use some file in /etc/exim4/conf.d/main; such files are the main way of configuring the exim4 package in non-trivial ways. I have 00_localmacros, which contains:

    MAIN_TLS_ENABLE = yes
    MAIN_TLS_CERTIFICATE = /etc/exim4/ssl/mail.crt
    MAIN_TLS_PRIVATEKEY = /etc/exim4/ssl/mail.key
    

    – that ought to work for you, too.

    Then, do the usual update-exim4.conf && service exim4 restart, and you should be able to speak SSL with your exim. The easiest way to test this is to install the swaks package (which will come in useful when you want to run authenticated SMTP or similar, too) and then run:

    swaks -a -tls -q HELO -s mail.<your domain> -au test -ap '<>'
    

    This will spit out the dialogue with your mail server and at some point say 220 TLS go ahead or so if things work, some more or less helpful error message if not.

    Aliases

    Exim comes with the most important aliases (e.g., postmaster) pre-configured in /etc/aliases. If you want to accept mail for people not in your /etc/passwd, add them there.

    The way this is set up, exim ignores domains; if you told exim to accept mails for domain1.de and domain2.fi, then mail to both user@domain1.de and user@domain2.fi will end up in /home/user/Maildir (or be rejected if user doesn't exist and there's no alias either). If you want to have domain-specific handling, add a file /etc/exim4/forwards that contains pairs like:

    drjekyll@example.org: mrhyde@otherexample.org
    

    The standard Debian configuration of Exim will not evaluate this file; to make it do that, drop a file wil something like:

    # Route using a global incoming -> outgoing alias file
    
    global_aliases:
      debug_print = "R: global_aliases for $local_part@$domain"
      driver = redirect
      domains = +local_domains
      allow_fail
      allow_defer
      data = ${lookup{$local_part@$domain}lsearch{/etc/exim4/forwards}}
    

    into (say) /etc/exim4/conf.d/router/450_other-aliases. After the usual update-exim4.conf, you should be good to go.

    Client Connections

    This setup only accepts mail for transport locally, and it will only deliver locally. That is: This isn't a smarthost setup yet.

    For delivery from remote systems, we're using ssh because pubkey auth is cool. This even works from an exim on the remote system …

  • OpenSSL: get_name: no start line?

    As part of my DIY mail server project, the other day I put a POP3 server on that box – solid-pop3d if you want to know –, and since that server doesn't have SSL built in, I configured stunnel to provide that, re-using a certificate I get for mail.tfiu.de's https server from letsencrypt. Trivial configuration:

    [spop3d]
    accept=995
    connect=110
    cert=/etc/stunnel/mail.pem
    

    And bang!, an error message from stunnel:

    [ ] Loading private key from file: /etc/stunnel/mail.pem
    [!] error queue: 140B0009: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib
    [!] SSL_CTX_use_PrivateKey_file: 909006C: error:0909006C:PEM routines:get_name:no start line
    

    One of my least favourite pastimes is figuring out cryptic OpenSSL error messages, and so I immediately fed this to $SEARCH_ENGINE. The responses were, let's say, lacking rigour, and so I thought I might use this blog to give future message googlers an explanation of what the problem was in my case.

    What OpenSSL was saying here simply was: there's no private key in the PEM.

    Where would the fun be if OpenSSL had said that itself?

    In case this doesn't immediately tell you how to fix things: “PEM files” in today's computing [1] are typically bundles of a “key” (that's the pair of public and secret key in sensible language), a “certificate” (that's a signed public key in sensible language), and possibly intermediate certificates that user agents may need to figure out that the signature on the certificate is any good, based on what certificate authorities they trust.

    All these things almost always come in base64 encoded ASCII these days (that's the actual meaning of “PEM“), which is nice because you can create your “PEM file” with cat if you've got the other parts. For instance, in my dealings with letsencrypt, I'm creating the key using:

    openssl genrsa 4096 > $SERVERNAME.key
    

    Then I build a certificiate signing request in some way that's immaterial here, and finally call the great acme-tiny something like:

    acme-tiny --account-key ./account.key --csr ./"$SERVERNAME".csr \
            --acme-dir /var/www/acme-challenge\
             > ./"$SERVERNAME".crt
    

    Letsencrypt also hands out the the intermediate certificates at a well-known URI, so I pull that, too:

    curl https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem \
            > intermediate.pem
    

    With that, all I have to do to make the “PEM file” is:

    cat $SERVERNAME.crt intermediate.pem > $SERVERNAME.pem  # not
    

    That was basically what I had in my certificate updating script, and that is what caused the error in my case. Spot it? Right, I failed to cat the key file in. I should have written:

    cat $SERVERNAME.key $SERVERNAME.crt intermediate.pem > $SERVERNAME.pem
    

    So – if you're seeing this error message, while I can't say why your key pair is missing in the PEM, I'd strongly suspect it is. Diagnosis: look for

    -----BEGIN RSA PRIVATE KEY-----

    somewhere in the file (and make sure all the dashes are present if you see something that looks like that and you're still seeing the odd OpenSSL message).

    [1]

    I've had to look that up myself: PEM actually has nothing to do with all kinds of cryptographic material cat-ed together into one file. Rather, it stands for Privacy-Enhanced Mail, something the IETF tried to establish in the early 1990ies where today (regrettably) S/MIME sits and what we could all mercifully forget if people finally just adopted PGP already.

    RFC 1421 – where a good deal of PEM is defined – was published in 1993 and still talks about BITNET! Oh wow. While this sort of PEM is dead, it did pioneer the ASCII-armoring of X.509 material. Of course, ASCII-armoring as such had been around for many years at that time – let me just mention uuencode, the cornerstone of software distribution on Usenet –, and PGP had even used base64 for crypto stuff, but all these (sensibly) steered clear of X.509.

    And ASCII-armored X.509 is PEM's legacy, as acknowledged by RFC 7468 (published in 2015, more than 20 years after the original PEM). Of course, RFC 7468 doesn't mention the .pem extension, let alone anything about the practice of assembling multiple kinds of cryptographic material in files with that extension.

  • Fürchten lernen 2: Ihre Reputation ist nicht verfügbar

    In Teil 2 der Saga von einem, der Mails verschicken will geht es um mailbox.org – an sich ja einer eher sympatische Firma, gerade im Vergleich zu Telekom und Vodafone. Aber, beim Einliefern:

    2021-02-08 05:52:09 1l8yWu-0006Ry-EM ** (Adresse tut nichts zur Sache)
    R=dnslookup T=remote_smtp H=mxext2.mailbox.org [80.241.60.215]
    X=TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256 CV=yes DN="C=DE,ST=Berlin,
    L=Berlin,O=Heinlein Support GmbH,OU=MBO,CN=*.mailbox.org": SMTP error
    from remote mail server after RCPT TO:<Adresse tut nichts zur Sache>:
    554 5.7.1 Service unavailable; Client host [116.203.206.117] blocked
    by RBL; http://www.barracudanetworks.com/reputation/?pr=1&ip=116.203.206.117
    

    Mit anderen Worten: Eine „Real Time Black List“ RBL fand, dass die IP nach Spam riecht, und drum darf ich nicht einliefern.

    Hu? Von dieser IP kam seit mindestens einem Jahr (also seit ich sie habe) nicht eine Mail an öffentliche SMTP-Server (von meinem Smarthost abgesehen), und schon gar nichts, was auch nur entfernt nach Spam aussah. Wie kommt die Kiste dann auf diese Blacklist? Schaun wir mal und dereferenzieren die URL. Resultat: Weiße Seite. Ach nee, soll ich denen Javascript erlauben? Gibt immer noch eine weiße Seite. Local Storage? Weiße Seite. Curl anwerfen. Nach kurzer Wartezeit kommen genau null Bytes zurück.

    Echte Profis, die barracudas. Vertrauenswürdige Treuhänder unserer Mail-Infrastruktur.

    Also gut, tippe ich halt mal barracudanetworks.com in den Browser ein (na gut, in Wirklichkeit habe ich in luakit 2gu gesagt; so viel nerd cred muss an der Stelle sein). Kommt ein animierter Security-Salespitch (yikes! Habe ich denen gerade Javascript erlaubt?), den sich niemand antun müssen sollte – „It's time to protect your business“ my ass. Und klar, google analytics ist auch gleich mit drauf.

    Wie finde ich denn jetzt raus, warum der Laden mich für einen Spammer hält? Auch mit etwas Rumklicken: nichts außer dämlichen Testimonials („The intuitive interface makes deployment simple and the bundled features combine to create a comprehensive email security solution.“).

    Und das alles nur, damit ich Mail einliefern darf?

    Tatsächlich kann mensch sich irgendwie zu barracudacentral.org durchklicken. Und da gibts dann einen Removal Request, der eigentlich aussieht wie ein Phishing-Versuch:

    Screenshot Formular

    Wozu wollen die jetzt eine Telefonnummer haben? Wozu eigentlich eine Mailadresse? Wie kann das bei der Prüfung eine Rolle spielen? Rufen die wirklich zurück? Und wenn ja: wärs nicht freundlich, wenigstens nach der Zeitzone zu fragen?

    Wenn ich den mailbox.org-Leuten nicht ziemlich arg trauen würde, wäre ich jetzt weg. Aber ich probiers mal. „Removal requests are typically investigated and processed within 12 hours of submission“ sagen die. Ich bin ja mal neugierig.

  • Das Fürchten lernen

    Nachdem ich in einer Zeit, als es üblich war, einfach von überall her Mails anzunehmen, tatsächlich mal ein ganz klassisches Sendmail betrieben habe, hatte ich seit den späten 90er Jahren nur noch Mailserver, die bei Smart Hosts eingeliefert haben, also kurzerhand alle ausgehende Mail an den gleichen anderen Server weitergereicht haben. – und umgekehrt auch alle Mails von dort bezogen.

    Im letzten Jahr allerdings hat das immer weniger funktioniert. Irgendwie scheint es, als hätten mit Corona die diversen Mail-Betreiber immer gewagtere Politiken ausgerollt, die vielleicht ein wenig Spam, vor allem aber die Zuverlässigkeit von Mail bekämpften; gleichzeitig nimmt die Bereitschaft erkennbar ab, die Weisheit dieser Politiken nochmal zu überdenken und im Fall von Fehlfunktionen zu klären, was eigentlich kaputt ist.

    Gegen die Politiken kann ich nichts tun, gegen das stille Verschlucken von Mails gibts eine Lösung: Ich muss wieder selbst einen richtigen, direkt ausliefernden Mailserver betreiben. Denn das ist ja eigentlich das Schöne an offenen Standards: JedeR kanns selbst machen.

    Allerdings: einfach einen Server aufsetzen und los gehts, das ist im kommerzialisierten Internet nur noch selten möglich (Mumble sei hier mal als löbliche Ausnahme erwähnt). Im Web zum Beispiel gehts ohne https und die damit zusammenhängenden Verrenkungen kaum noch, von den mittlerweile üblichen Labyrinthen aus Reverse Proxies und Containerfarmen ganz zu schweigen.

    Bei Mail, so scheint es, ist es noch viel schlimmer; ich hatte ja schon damit gerechnet, dass es da und dort etwas fummelig würde in Zeiten von SPF, DKIM, DANE und so fort.

    Dass es aber so schlimm ist, hätte ich nicht gedacht. Und so glaube ich, dass dieses Blog mutieren wird zu einer Geschichte von einem, der auszog, das Fürchten zu lernen – und insbesondere, warum immer weniger Leute eigene Infrastruktur betreiben und sich das Internet immer weiter zentralisiert.

    Kapitel 1: t-online.de

    Einliefern bei t-online.de:

    2021-02-07 11:27:26 1l8hHU-00087u-Mk H=mx00.t-online.de [194.25.134.8]:
    SMTP error from remote mail server after initial connection:
    554 IP=116.203.206.117 - A problem occurred. (Ask your postmaster
    for help or to contact tosa@rx.t-online.de to clarify.)
    

    Na klasse. „A problem occurred“. Auch mit richtig viel Mühe kann ich mir keine weniger Informative Fehlermeldung vorstellen.

    Duckduckgo berichtet, dass das halt einfach so ein whitelisting der Telekom ist und dass mensch den Laden freundlich bitten muss. Wie bitte? Was machen die eigentlich, wenn irgendwer aus Singapur einliefert? Erwarten die ernsthaft, dass auch so ein Laden bei ihnen anklopft?

    So kann mensch Standards natürlich auch aushöhlen.

    Aber immerhin: selbst am Samstag abend um acht antwortet jemand auf Kontaktmails – es sieht ganz so aus, als hätte die Telekom das an irgendwelche Kontraktoren ausgelagert. Und die wollen, dass an dem Mailserver ein Webserver hängt, auf dem, so sieht es aus, ein Impressum nach Medienstaatsvertrag liegen soll. Wozu? Keine Ahnung. Ich habe gefragt und keine sinnvolle Antwort bekommen. Und mit welchem Recht ein doch recht großer Laden einfach anderen Leuten Vorschriften machen will, welche Daten sie zu publizieren haben, bleibt natürlich offen.

    Tatsächlich bin ich in dem Punkt auch etwas empfindlich, denn die Hartleibigkeit, mit der die Exekutive im Medienstaatsvertrag den erklärten Willen des Bundestags aus dem Telemediengesetz auszuhebeln versucht... nun, das ist Thema für einen anderen Post.

    Na ja, zumindest für t-online.de sollte es mein alter Smarthost noch eine Weile tun; kriege ich halt weiter nicht mit, wenn da was kaputt ist.

    Mein MTA, exim4, hat auf Debian-Systemen die praktische Datei /etc/exim4/hubbed.hosts. Da steht t-online.de jetzt erstmal drin. Ich werde ein andermal über unnötige Komplexität jammern.

    Kapitel 2: arcor.de

    Einliefern bei arcor.de:

    2021-02-07 17:41:20 1l8n7d-0001gH-Qj ** <elided>@arcor.de R=dnslookup
    T=remote_smtp H=mx2.vodafonemail.de [2.207.150.241]: SMTP error
    from remote mail server after initial connection:
    554 fra1frontrelay13.vodafonemail.de ESMTP not accepting messages
    

    Was ist das jetzt schon wieder für eine Teufelei? Duckduckgo führt auf eine Forendiskussion die ein Vodafon-Mitarbeiter wie folgt beendet:

    wir befinden uns mitten in dem Umzug zu einem anderen Mailbetreiber. Die Arbeiten sind ca bis Ende Janaur [aus dem Kontext ist 2021 abzuleiten – A.] abgeschlossen.

    Ich geh davon aus, dass die Emailadresse Deiner Homepage auf einer Blacklist steht und daher von unserem Spamfilter aussortiert wird.

    Dafür kann ich zum jetzigen Stand kein technisches Ticket aufnehmen.

    Wie bitte? „Kein technisches Ticket“? Für eine doch recht drastische Fehlfunktion eines wirklich fundamentalen Internetdiensts, nämlich E-Mail?

    Au weia. Nun, kommt arcor.de halt auch erstmal in die Hubbed Hosts. Vielleicht probiere ich es Anfang März nochmal, kann ja sein, dass dann die Zeit ist für ein „technisches Ticket aufnehmen“.

    Es sieht nicht gut aus für offene Standards.

Seite 1 / 1

Letzte Ergänzungen