Email is a wonderful thing. Anyone can send an email to anyone and it all works beautifully. In the early stages of the Internet, we didn't have to worry about security and scammers. As emails started to play a more vital role in our life scammers started to con people to steal their money.
Not only anyone can send email to anyone but anyone can pretend to be anyone. I can send an email to you and the email will come to you as if it is sent by "Elon Musk" and the from address could be "elon@musk.com".
I can send an email to you pretending that the email is from your bank and you need to change your password. In one case a hacker sent an email to a company's finance team telling them that the company's bank account number has changed. The hacker walked away with millions of dollars.
Hackers can also do damage to one's domain reputation.
Let's assume that there is a business called "PrixPapers" and they have thousands of customers all over the country. Malicious folks can start sending emails to people pretending to be "PrixPapers". When people receive an email like this they might buy the item. The item could be "fake goods". Ideally, no one should be able to send any email pretending to be someone else.
Since these malicious folks don't have the actual list of the customers of PrixPapers will in general spam people. Some people receiving this email will mark these emails as "spam". As more and more people mark these emails as "spam" the domain reputation of "prixpapers.com" will go down. It means that when "prixpapers.com" will send out a legitimate newsletter to its customers some of these emails will be marked as "spam" because of the nefarious work done by the previous scammer.
The challenge presented to the Internet authorities is that they want to keep email simple but safe at the same time. They came up with certain policies to make emails safe and secure. In this blog, we will see how SPF helps in keeping email secure. However, first, we need to know what is "Return-path". We'll later see how it is used in blocking fake emails.
Return-path
Let's say that we are dealing with a company called neetoCal. They have the domain neetocal.com.
Let's say that brian@neetocal.com sent an email to peter@gmail.com. Let's pretend that for some reason Peter's mailbox is full and the email sent by Brian bounces. In that case, Brain will get an email that the message bounced. We all have seen these kinds of messages.
Now let's imagine that it's end of the year sale time and neetoCal decided to send an email campaign to all its customers giving them a 10% discount. This email will go out to their 5000 customers. Some of these emails are bound to bounce. If all the bounced emails come to Brain then Brain's inbox would be filled with such bounced emails.
To solve this problem email protocol allows us to set a hidden field called Return-path. This value can be set in the email header. Return-path set in the email header indicates how to process bounced emails. Anytime an email server detects that an email can't be delivered for reasons like "mailbox is full" or "the email doesn't exist" then that email server can send an email to the email address mentioned in Return-path. The business owner can look at Return-path emails to analyze how many emails are bouncing and why.
If you are sending an email using "Gmail", "outlook", "yahoo" etc then the Return-path is set as your email so that you get to know if there is a bounced email. Email service providers allow us to customize Return-path in case one is running a big marketing campaign. For example here is a document from SendGrid describing how to set Return-path.
Why are we talking about how bounced emails are processed when we are dealing with the subject of "email deliverability". That's because Return-path plays a dual role as we will see next.
What is SPF record
Before we get into SPF record let's take a simple real-world example of how it works.
Let's say that in the neetoCal office, there is a gatekeeper. This gatekeeper will only allow the people who work there. It means the gatekeeper has a list of approved people and for each person getting through the gate, the gatekeeper checks if the person is in the approved list or not.
SPF works similarly. SPF policy is a mechanism to tell the email server if the email is coming from a trusted source then accept the email or reject the email. A sender policy framework (SPF) record is a type of DNS TXT record that lists all the servers authorized to send emails from a particular domain.
Let's take a real-world example. The SPF record of neetocal.com looks like v=spf1 include:spf.messagingengine.com -all. We can see this data using mxtoolbox.
How the SPF record is used
Let's look at the following case.
Step 1. notifications@neetocal.com sends an email to elon@gmail.com.
Step 2. This email is sent to Fastmail since neetoCal is using the Fastmail services.
Step 3. The Fastmail email server receives this email as an outgoing email.
Step 4. Fastmail email server adds email header Return-path value to "notifiations@neetocal.com".
Step 5. The Fastmail email server sends this email to the gmail server.
Step 6. The Gmail server gets this email.
Step 7. The Gmail server extracts the Return-path key and finds that the domain is "neetocal.com".
Step 8. The Gmail server finds the TXT DNS records of "neetocal.com". This lists all the approved IP addresses.
Step 9. The Gmail server will check if the email server which sent the email is in the approved IP addresses or not.
Step 10. If the IP address is in the approved IP addresses list then the email is approved for further processing. Otherwise, the email is rejected.
We can go to https://dmarcian.com/spf-survey/ and enter "neetocal.com" here. They get the IP addresses published for "messagingengine.com" and then they show the list of the approved IPs.
Allowing the third party to send emails on your behalf
Let's say that Brian decides to use Mailerlite to send marketing emails. Now if Mailerlite sends an email then the receiving email server will notice that the IP address of Mailerlite is not in the approved list of ips and the email will be rejected.
A domain is allowed to have only one SPF record. So if neetoCal wants to use Mailerlite for marketing then the spf records need to be updated. If you signup for a domain in Mailerlite then Mailerlite will check if that domain has an existing SPF record or not. If there is no SPF record then Mailerlite won't do anything. However, if there is an existing SPF record then Mailerlite will insist that first you update the SPF record to include Mailerlite so that the emails from Mailerlite are not rejected.
BigBinary uses Mailerlite to email newly published blogs to the subscribers. Given below is what the SPF record of BigBinary looks like. We can also see this result online.
1v=spf1 include:_spf.mlsend.com include:_spf.google.com -all
BigBinary uses google workspace so the second include is for that reason. The first include is to ensure that Mailerlite is in the allowed ip list.
If we look at the result for "bigbinary.com" by visiting https://dmarcian.com/spf-survey/ then we will that all these includes are like regular programming language "imports". They allow the third party to include other third parties in the chain. The end result is that we have a finite list of approved IP addresses which can send email.
spf record standard
Let's take a look at the SPF record of neetoCal.
1v=spf1 include:spf.messagingengine.com -all
v=spf1 tells the server that this record contains an SPF record. Every SPF record must begin with this string.
include:spf.messagingengine.com tells the server what third-party organizations are authorized to send emails on behalf of the domain. This tag signals that the content of the SPF record for the included domain (messagingengine.com in this case) should be checked and the IP addresses it contains should also be considered authorized. Multiple domains can be included within an SPF record.
-all tells the server that addresses not listed in the SPF record is not authorized to send emails and should be rejected. Alternative options here include ~all, which states that unlisted emails will be marked as insecure or spam but still accepted. +all signifies that any server can send emails on behalf of the domain.
Here is another format of SPF record.
1v=spf1 ip4=192.0.2.0 ip4=192.0.2.1 include:messagingengine.email -all
In this example, the SPF record is telling the server that ip4=192.0.2.0 and ip4=192.0.2.1 are also authorized to send emails on behalf of the domain.