Simple Mail Transfer Protocol harkens from another era, with all of the security acumen 1980 could muster. It’s a fundamentally broken protocol but it’s nearly impossible to wholesale replace, so we keep adding bandages to it in hopes we can fix an initially bad assumption.

Receiving Mail

Receiving email isn’t that bad. Many of us use a web portal which could be bypassing the standard protocols in favor of direct database access. However, third party clients usually connect using POP3 (essentially a polled interface that checks for new messages periodically) or IMAP (with a more persistent synchronization protocol). Both of these protocols have encrypted flavors and involve authentication to make sure you have access to the email.

Sending Mail

This is where things fall down. It’s not that sending mail is difficult, it’s more that it’s too easy. In order to prove to the mail server that you’re authorized to send an email as the user, you have to put it into the message and pinky-promise it’s the right one. There are legitimate reasons to be able to send an email with another address - most automated emails come from an address like [email protected] which wouldn’t be possible if you couldn’t specify the sender. But the open mechanism used today means spam is a constant problem.

We’re gradually introducing more and more solutions to try to fix the global spam issue, most of them relying on passing public information via DNS which can be validated against the information included in the email headers.

DNS Intro

DNS is the Domain Name System. It’s the infrastructure that converts useful hostnames into IP addresses via A (IPv4) or AAAA (IPv6) records. However, DNS is much more useful than just a way to convert a human-friendly hostname into an address the Internet can use to get the packet where it needs to go. In this case we care about two different record types.

MX records define the mail server for a particular domain. When an email is sent to the @ebower.com domain, the sender’s mail server looks up the MX record for @ebower.com using something like dig +short ebower.com mx to discover a few CloudFlare mail servers as the destination. Looking at other hostnames can tell you if they use Google Workspace, Office365, or some other mail server.

TXT records are just for passing aribtrary text-encoded data to the good people of the Internet, they’re a list of arbitrary messages associated with a domain or hostname usually to prove ownership of a domain or to communicate data as part of a defined protocol. Many sites use these messages for verification that you control the DNS of a domain, if you run dig +short ebower.com txt you’ll see a few with a service identifier and a code. There are a few custom records that are used to fix certain issues by publishing data in your DNS server, these are implemented as TXT records.

SPF

The first problem we’ll try to solve is making sure the email server is actually authorized to send the message. The protocol designed to do this is called the Sender Policy Framework (SPF) and, while flawed, was a valiant effort. This is a simple TXT record at the root of your domain, you’ll see a line for many email-sending organizations that looks something like this:

v=spf1 include:_spf.mx.cloudflare.net include:SPF.brevo.com include:_spf.google.com mx ~all

What this means is that I have authorized mail servers from CloudFlare, Brevo, and Google to send my email, an email sourced by another service will fail my SPF check. However, that doesn’t mean that the message wasn’t from me. Brevo, for example, processes mail in a manner that makes it incompatible with SPF and that entry is largely useless. Perhaps an email from me will trigger from HubSpot in the future, but it comes before the SPF entries are updated and propagated.

On the reverse side, anyone sending messages from CloudFlare, Brevo, or Google will pass my SPF check (at least, if Brevo changes their architecture to allow for it). And it’s necessarily easy to find the list of services I use.

So SPF gets a C for actually solving a real problem. Essentially, prior to this it was trivial to forge the sender’s email, now it may take a little work. Back in the days when people ran self-hosted mail servers MAYBE it could have caught a lot of fraudulent emails, but with cloud services using shared IPs this causes more headache than benefit. But it’s a checkbox that needs to be filled, even if not a panacea as was hoped.

DKIM

DKIM is a much better attempt. Your mail sending server will add a digital signature to your email signed with a DomainKeys Identified Mail (DKIM) private key. The receiver mail server can use that header message and the public key (published via DNS TXT records) to validate that the message came, not just from a valid mail server IP address, but was signed using a key generally associated with the specific customer. So my Google Workspace-sent messages are signed with a key associated with my specific Google Workspace account and were not sent by some random person able to associate my domain with a Gmail-based service.

Domain keys aren’t stored in standard locations, every provider likes to come up with something new. However, inside an email you can figure out where the key is stored, in Google’s case it’s in dig google._domainkey.ebower.com txt and looks like this:

google._domainkey.ebower.com. 300 IN TXT "v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAlrWO/IRwX7hbcmnPd/QQjUSQZgIgStcCB7a+j20H/HRN7gCxLqe2xh9JZE6j3248C38bJiqq5V+LE/wSlvOim831w4dFtDWcTnaZSfV04CtQSEGnXwf7BFLaEzC3xx6QSwWi3UPGM5l78KPbfsphoDWw64TASVJb49KaVbczLx+tIEuDgfpmijv78vePrdNmq" "LlkDaGREHo2GEzVkh3G2ulbZT8eFj6grT9ZME/e/hQCx4ci+n0wPsJIfkkQcdlSs/DNDwT3lnu9FNxeDoAQKOt4krIL+zg75zSUz4Al8TmAwmTujENtuOy+niTbW4NHLvHVWN1oZcu6lPRdJoND2wIDAQAB"

It looks like super secure information nobody should know, but they’re just public keys and anyone can look up how they’re provisioned - they need to in order to validate that an email came from a domain that the sender mail server could understand.

DKIM is a much better solution than SPF, but it’s still flawed. The metadata for the signing is stored in the header itself which can have a ton of issues. And it’s really easy to buy a temporary look-alike site. Swapping characters around or just adding something like “-support” to a well-known hostname lends more than enough credence. A valiant effort, and maybe with a few more tweaks it could be workable.

DMARC

Domain-based Message Authentication, Reporting, and Conformance (DMARC) helps bind together SPF and DKIM by specifying a policy to be applied as well as places to send reports to. By running dig _dmarc.ebower.com txt you get a record like this:

v=DMARC1; p=quarantine; sp=quarantine; aspf=s; rua=mailto:[email protected]; ruf=mailto:[email protected]; fo=0:1:d:s;

Essentially, this is an enforced policy where messages that fail to pass should be send to the Spam folder. I have two email addresses setup for DMARC monitoring and Forensic Monitoring. It's both telling mail servers that my SPF and DKIM are configured and should be working, as well as where to report to me if messages start failing.

MTA-STS

Mail Transfer Agent Strict Transport Security (MTA-STS) essentially ensures that all connections between the source mail server and the destination use both encryption and authentication to make sure there’s no server in the middle trying to modify the email in transit.

The configuration for MTA-STS takes a bit of a deviation. There is a record at dig _mta-sts.ebower.com txt with the following content:

v=STSv1;id=1727605999881;

This mainly just asks mail servers to check for a policy, but that policy is stored in a text file in a fixed location on a dedicated domain. You have to create a policy and publish it at a specific URL, for example https://mta-sts.ebower.com/.well-known/mta-sts.txt.

version: STSv1mode: testingmx: route3.mx.cloudflare.netmx: route2.mx.cloudflare.netmx: route1.mx.cloudflare.netmax_age: 604800

Finally, you need to check to make sure it’s working. That’s a DIFFERENT TXT message in a fixed location, dig _smtp._tls.ebower.com txt can view my own record which simply specifies an email to grab reports.

v=TLSRPTv1;rua=mailto:[email protected];

This is a pretty convoluted configuration, and I wish they’d created the .well-known file as a one-stop shop for SPF, DKIM, MTA-STS, etc. But it means that you have access to a mail server, the DNS system, as well as the ability to host a file on the company’s domain. It’s yet another factor in your mail system’s spam scoring to help you get through to your customer.

BIMI

BIMI is essentially registering your logo so it can be used by mail readers. In Gmail, for example, by default you just get a letter in a solid circle next to your name. With BIMI enabled, it will be replaced with your registered logo. It’s a cute idea, and maybe I’ll play with it eventually, but I don’t know that I actually want that. However, for a corporate email domain it's pretty powerful to have mail readers show your logo.

In essence you register your in SVG format and get it digitally signed. Then you host it on a webserver, publish a TXT record, and now properly signed messages can display your company logo in a way to reassure the user that your company has sent the message. This is much easier if your logo is a registered trademark, so a little more difficult to do for a personal domain.

Tools

As with most things, once and done doesn’t cut it. If something starts to break, you’ll want to know. And you’ll want regular updates to keep on the edge of new standards before it’s too late. You’ll also want to understand baseline values, there will be some failures in your emails in general - particularly when an email is forwarded it becomes a much trickier thing to figure out the original sender.

Many of these tools use email-based integrations - yes, email is the medium used to police the viability of email as a platform. Rather than including these custom emails, I point messages to a mailing list with a collection of tools. This allows me to easily update the list of tools without touching delicate DNS entries. However, many services will check to see if that custom email address is present and claim to be improperly configured.

CloudFlare

CloudFlare has a reasonable (and free!) DMARC reporting interface in beta. Essentially you just configure a special email address in the RUA field of your DMARC record. The reporting is pretty basic, but it tells me if my DMARC is passing and if it’s SPF or DKIM that’s failing.

Dmarcian

Dmarcian is much more purpose-built as a decent free tier. Again, I wish I had API and alerting access so I could more actively monitor things but they lock that behind the paywall. However, if I really wanted to I could always shunt the mailing list to a mailbox where I could pull the data and parse it myself.

With Dmarcian I get a bunch of domain health tooling, pretty graphs and reports, and even TLS Reporting for MTA-STS.

MXToolbox

Probably my favorite tool set, MXToolbox allows you to dig deep into your DNS and email configuration. You can also send an email to [email protected] and have it evaluated before you commit.

Their Domain Health Report allows a concise listing of most major email configurations. It will warn you about things that may be a bit aggressive (“SOA Serial Number Format is Invalid” refers to a standard that was abandoned and something that often cannot be fixed unless you run your own DNS server.

They have SPF and DMARC record generators to help you craft your own values.

Another tool worth playing with is their Blacklist monitor. A free account can set up a monitor for any one domain. The claim is that on a weekly basis you’ll get a report as to whether the domain appears on any spam blacklists, along with a link to help with remediation. In practice changes to be added to a blacklist seem to come in asynchronously instead of waiting for the next weekly message and I was immediately able to take action.

Sending Email from Your Personal Domain from Free Gmail

Sending email from Google Workspace is easy, it all just works and is tied to your domain. However, using your personal email with a Gmail backend is a bit more involved. Not only does the email need to go SOMEPLACE, but when you respond it needs to be to a server that can properly sign the message. This gets complicated.

It used to be easy to send email from your personal domain via Gmail, just prove to them that you own a specific email address and you can send from that email as though it were your own. SPF can still work, just add the Google records to your SPF record. But DKIM is a bit harder, either you get a personal signing key for every free GMail account out there or you just punt on the problem.

Now you need a few components. Of course, I assume you have a personal @gmail.com email address you want to use. Next you’ll have to get emails to that account to your Gmail account using an email routing system of some sort. Finally, you’ll need something to send emails from your domain.

Assuming you’ve got a GMail account already, let’s start with inbound emails. Here, I use CloudFlare Email Routing. It’s not perfect, but it works and let me migrate my registrar (which had a similar feature) to CloudFlare. Configuration is pretty easy, you simply go to Email Routing in the left-hand menu and first select Destination Emails so you can enter your @gmail.com address. This will result in an email to the address and you just click the Verify link to validate it. Repeat for any other destination emails you may have. Now setup routing rules; for example [email protected] could map to [email protected]. Set up as many as you need. Finally, under the Settings section they’ll walk you through setting up the MX and SPF records. WARNING: When you’re done with the DNS setup all emails will be routed according to your routing rules. This could lead to a loss of email access if you’ve done things improperly, so make sure to test inbound emails from multiple addresses.

At this point, incoming emails should be forwarded to your Gmail account. However, we need to configure something to send emails. Since I need something with an SMTP interface and I already used this for some newsletters, I went with Brevo (formerly SendInBlue). Their free tier handles 300 emails a day, more than enough for my family. Once you have an account, you’ll have to set up your domain which will walk you through changes to your Brevo authentication code, a DKIM record, and a change to your DMARC record. Finally, you’ll want to go to SMTP and API under your Brevo account and create a master password.

Finally, under Gmail settings open up All Settings and select Accounts and Import. Under “Send mail as” you’ll want to add your personal email. Not as an Alias, Using: smtp-relay.brevo.com on port 587 using TLS. Login using your Brevo master login email and use the Master Password under SMTP and API. You can set this as the default sender once you’ve tested it.

Finally, make some tests. Send an email from another account to your personal email? Did you get it? If not, the setup with CloudFlare may be at fault. Reply to that email, did it come from your Gmail or Personal email? Did it pass DKIM (for those who need it, “Show Original” while reading an email will tell you whether the message passed DKIM, SPF, and DMARC). Send a new email, does that get through? Email [email protected] and see what they have to say about it. Now, use the tools above to monitor the DMARC status of outgoing messages to make sure there are no anomalies.

Summary

Email is growing ever more complex, but having a properly configured (and monitored) email system allows you to stay out of the spam box. At a minimum, everyone should configure SPF, DKIM, and DMARC. MTA-STS is easy enough and can help against man-in-the-middle attacks. BIMI requires some effort and is useful for businesses, but may not be practical for a personal domain unless you want to have a real logo.

The various protocols you should configure are:

SPF - A way to track the IP addresses of authorized mail servers for your domain.

DKIM - A way for your mail server to sign a message in a way unique for your domain.

DMARC - A way to report information about your sent emails and to configure how SPF and DKIM should operate.

MTA-STS - Prevents man-in-the-middle attacks by requiring inter-mail server communications to be encrypted and authenticated.

BIMI - A nice way to get your brand front and center, but the primary intent is more for marketing than for personal emails.

So, with all of this why do spam messages get through? Because all this is leaps ahead of where we were but it’s still lipstick on a pig and an arms race between spammers/scammers and email providers. Fundamentally something should have replaced email long ago (Google Wave - we miss you!), and for most important communications this is happening. Email will never die off completely, but I can’t use it for alerting since I get so many - I use Slack to organize my alerts. Transferring files is particularly problematic, but again Slack has a great mechanism and sharing cloud-hosted documents means anything that can send simple URLs can be used. We never got the promise of federating all the various chat apps but whether you use Slack, WhatsApp, Telegram, even those saddled with Microsoft Teams, the use of these communications channels for most things can help prevent someone from sending a legitimate-looking email via an unexpected mechanism. One more layer of “why is this person sending me a PDF of their expense report instead of pinging me on Slack” helps to reduce the number of effective attacks on the people with access to your systems.