[vox-tech] Test/Upgrade

[Bill Broadley]

Rick we seem to be violently agreeing on most technical details.  I think a large part of the
problem is you have too much experience and too much history.

You can give a cursory glance at the 1700 line readme and have a good instinct for which pieces to
ignore, which pieces to use, and which pieces need some polish.  For someone who hasn't does this
before that doc is a minefield.  Sure there are good ideas, and bad ideas.  Current ideas, obsolete
ideas, and things missing.

Do you really think lugod's new mail server should be some weird conglomoration of upstream source,
and out of tree patches?

That's batshit insane in my book.

More details below.

On 1/9/19 6:35 AM, Rick Moen wrote:
> Quoting Bill Broadley (bill at broadley.org):
> 
>> I hear this line of thought, and am always puzzled by it.  My recipe is:
>> 1) run a current OS with a current spam assassin, like the newest ubuntu LTS or
>>    newest debian stable.
>> 2) run postfix and whatever glue you like (mail filter, amavis, and amavis-ng
>>    are popular)
>> 3) use greylisting
> [...]
> 
> I'm puzzled that you're puzzled.  

When you say "Architecting effective antispam for a modern mail server is an art form" it makes it
sound rather difficult and mysterious.  Not something as simple as install spam assassin, paste in a
few lines of recipient/client/helo restrictions, and enable grey listing.

Makes me think of someone actually spending many hours tuning, tinkering, doing, and redoing.
Working late into the night by a small lamp, hungry, sleep deprived, and pulling ones hair out
against the unstoppable spam onslaught.

Not install mailserver, a few daemons, fire, and forget.

> What I said was that a distro's out-of-the-box MTA configuration has
> basically zero spam-rejection

Your guide lists the anti-spam like SpamAssassin, SAS-Exim, and Exiscan as optional as well.

>, which must be created by local
> administrative effort. 

Indeed, an apt get, and 5 lines of conf file.  One for amavis, one for grey listing, and few for
smtpd/helo/client restrictions.  I supect my total is more like 20 lines, but that's just
preferences not just to get reliable mail delivery with minimal spam.

> The meritorious configuration you describe is
> not provided by any Linux distribution's out-of-the-box MTA
> configuration, but must be created by local administrative effort.

Indeed, but it's just a few lines of config and a few apt-get installs.

> So, you agree with, and restate slightly differently, what I said -- yet
> you claim to be puzzled by what I said?

Just the difference between easy, install a few daemons, and something so hard/challenging that's an
art form.

> This is solving the wrong problem.
> 
> If an SMTP host ceases to use alias redirectors (/etc/aliases and
> ~/.forward) for cross-domain purposes, then the problem of 'forwarding a
> malicious attachment' to a remote SMTP host basically goes away without
> the collateral damage of neutering the file-attachment mechanism.

Right, but requires not forwarding and doing things like publishing the email addresses of officers
so they don't use the mail server... right?  Not sending email through a server isn't a particularly
good work around for an email server.

Even without aliases and .forward, if a lugod member has an email on yahoo, gmail, or microsoft and
they get a malicious attachment the likely result is making the blocking of delivery from lugod.

There are replacements for /etc/aliases and .forwards that don't break SPF. I pipe to sendmail, or a
short procmail can fix it.

The more complicated solution is SRS, but is painful for various reasons.

>  (The
> other type of mail reflector, MLM software such as Mailman and Sympa,
> makes it easy to avoid being a malware reflector for reasons I won't
> spend time covering here.  It should suffice to note that LUG mailing
> lists are not a vector for same.)

I've heard several reports of large mail services blocking delivery from lugod, not sure what caused
it though.  Statistically speaking seems more likely to be mailing list related (a large volume of
email) then direct mail to officers... but of course it's hard to tell.

>> Heh, well using current software is much easier than engineering it from
>> scratch.
> 
> I'll note in passing that implementing some or all of the ideas in Boggis's 
> toolkit entails using current software, and not engineering any from
> scratch.
> 
> I would guess that you did not bother looking at Boggis's work before 
> commenting.

I read the http://www.jcdigita.com/eximconfig/, looked hugely complicated, and somewhat fragile.
Certainly with enough work I'm sure it works quite well.  But pretty much turn key solutions work
quite well without even needing to understanding much of what's on that page.

Maintaining that solution looks like a part time job all in itself.  Generally compiling sources
code for any internet facing service seems borderline insane... unless you manage the source code
yourself on a daily basis.

> And devoid of spam-rejection upon initial installation of just an MTA
> from a distro package.  So we agree -- and yet you are arguing anyway.
> Well, I guess that's what I get for being on the Internet.

Not arguing for the sake of arguing.  You made it sound like quite a daunting projects for experts,
er artists, only.  Instead of the common sense of approach of install anti-spam if you want anti-spam.

> The ideas Boggis collected in one place that were excellent in 2011 
> are sill excellent in 2019.  FYI, Michael Paoli recently used Boggis's 
> tarball and docs to create a new MTA configuration for Bay Area Linux
> User Group, and the only problem he found IIRC was the Perl SPF package
> reference being outdated.

There's an awful lot of complexity there.  Seems quite fragile.  It seems to mostly depend on a
bunch of ACLs, I clicked on a dozen or so links for the ACLs... they were all forbidden.  I looked
at the README, all 1700 lines, again quite complex.  My recommendation is dramatically simpler.
With so much of the complexity based on ancient unreachable ACLs it does make me wonder what is the
point is exactly.

Sure Tim/whoever could take that 1700 lines, build things from source, and make a special snowflake
server that's unique, undocumented, based on a 5 year old reference.  They could analyze the source
code, figure out what patches were included, which are obsolete, which have better replacements.
Then regularly revisit that every time the upstream source changes.  To get some value from that
work and contribute back to the community they could publish a new 1700 line readme that's actually
current as a trap for any other pour soul who is silly enough to implement it.

Or they could use apt.

So I recommend (and am willing to help implement) picking a mail server (like postfix), pick an
anti-spam (like SA), and add greylisting (via any number of daemons).  Customize 10-20 lines of
config files and call it done.  If you get too much spam revisit things and consider additional
measures.

The eximconfig readme mentions compiling things quite a bit, which is a huge timesink and a
significant security risk.  Most people don't have time to regularly revisit every upstream source
to check for important security updates.

Just this single paragraph (of 1700 lines) would make me recommend this for any normal/busy person
running a mail server:

  If you need to compile SA-Exim, obtain it from the URL above.  If you are
  compiling it as a module for use with local_scan_path, simply follow
  through the README and compile it separately from Exim.  Once compiled,
  copy the resulting sa-exim-N.N.so to bin/sa-exim.so in your EximConfig
  installation.

Use something more popular, common, and documented that "just works" and gets regular updates/patches.

Under requirements it says:
   Exim 4.2x or later MTA (See http://www.exim.org) preferably with TLS
   support and either the dl_local_scan patch applied or compiled with
   SA-Exim replacement local_scan.c

Generally if you are worrying about that, and building source not from the upstream generally I'd
say you are doing it wrong.  Sure I've done it, because I had to.  But postfix is quite functional
and doesn't require multiple patches and building from source.

Last thing we want is lugod.org compromised because someone missed an important security update
because we built from source instead of using the distro's package.

>> Greylisting + current SA works wonders + sane postfix checks works
>> wonders.  Keep in mind you can't just update the SA rules, you have to
>> update SA itself for the best protection.
> 
> There are a number of things that need to be done with SA, including
> avoiding using all of the default weightings, for the simple reason that
> any halfway competent spamhaus tests spamicity against a default spamd
> configuration before sending.  (Competent spammers are relatively rare,
> but there are enough of them to be a problem.)

There's certainly always tuning that can be done.  But even by default I find greylisting + current
SA quite effective.  When I upgrade SA and use the defaults I often find it dramatically more
effective than even a heavily tuned SA from 2 years ago.  I read that SA reanalyzes the a huge
current ham/spam corpus every 6 months or so and optimizes the rules and weights.  Spammers of
course use SA to benchmark themselves, but rarely stay as current as the newest version of SA.

> [problems with use of /etc/aliases or ~/.forward redirection for 
> interdomain reflection:]
> 
>> There are technical work around that are a pain in the ass.
> 
> You're probably thinking of software to implement Sender Rewriting
> Scheme, as invented by Meng Wong in 2003, in the form of a wrapper Perl 
> script intended to rewrite SMTP envelopes, semi-mimicking the way MLM
> software does that, using a variable envelope return path (VERP).  I
> found this to be not only a pain in the ass but also that the results
> are downright peculiar, and that it's far more practical to simply cease
> using /etc/aliases and ~/.forward entries for cross-domain mail
> reflection.  (As I said.)

Heh, yeah, we agree it's painful.
> I'm also skittish about trusting pipes to Perl scripts in /etc/aliases
> (etc.) entries.  There's a good (security) reason why the processing of
> those is now disabled by default in modern distros.

Indeed, and similar reasons are why I prefer sieve over procmail.

>> Sieve can do things like:
>>
>> if header :contains "To" "root at lugod.com" {
>>   fileinto "admin.root";
>>   redirect :copy "poor.soul at randomdomain.com";
>> }
> 
> If I understand this bit of Postfix plumbing correctly, this is

Sieve is it's own standard.  Dovecot's deliver is often paired postfix and is the middleman between
postfix and dovecot (for imap/pop).  The deliver agent implements a useful subset of the sieve
standard.  The piece missing last I checked was the ability to send notifications over XMPP.

> functionally exactly the same as having /etc/aliases entry
> root: poor.soul at randomdomain.com
> ...except I guess this would make Postfix write a new outbound SMTP
> envelope, making this a tiny improvement over the /etc/aliases
> equivalent -- but leaving in place the huge problem that lugod.org 
> would mindlessly reflect any received spam (including as a subcatetory
> spam with attached MS-Windows malware) that passes MTA checking, and
> putting lugod.org at war with the receiving MTA's spam defences.  That's
> exactly the problem LUGOD should want to avoid having.  Hence my
> recommendation to dispose of the 'role' reflectors as sadly impractical.

Sounds about right.  But it has to get past grey listing, ACLs (if you have any), blacklisting, rate
limiting (if you want it), and SA.  Much like emailing vox-tech at lugod.org  should be.

> I recommend a week _with entirely standard packaged software_ (as you
> would understand if you'd bothered to look at what I recommended) so that
> the admin fully understands what he/she is setting up and can run it
> properly.

Now I'm doubly confused. Your eximconfig readme is 1700 lines of how to customize a mail server
including notes about compiling things, linking to libraries, ACLs, etc.

That's about as far from "entirely standard packaged software" as you can get.

I'm recommending more along the lines of a few apt-gets and a 10s of lines of config changes.