The Problem

Users find that an email sent to user@cs.toronto.edu and forwarded to Gmail is rejected by Gmail, commonly with the claim that This message does not pass authentication checks.

Gmail and "Send mail as"

Gmail provides an option called Send mail as which allows you to read and compose with Gmail but to send that email from a different email address. How Gmail does this has changed in the past few years, which we will address later.

If you are user@gmail.com and you also own user@somewhere.org, you could compose and send an email which would appear to have come from user@somewhere.org. If your recipient hit 'reply' on the email, the reply would be sent back to user@somewhere.org.

But a problem with the delivery of these emails is manifesting, the direct result of a policy change Gmail has made related to email verification. There is every reason to expect that this problem will propagate out to other mail systems as they too follow Gmail in implementing stricter email validation criteria.

Email validation

There are mechanisms in use today in order to identify forged emails.

Fabricating the 'from address' of an email (e.g. a mail is sent by user@abcd.com but the 'from address' of the email indicates user@wxyz.com) is a common strategy amongst spammers and phishers. It allows them to send email in such a manner that it appears to come from a known contact, thus playing on the trust of the recipient and thereby increasing the chance of opening a malicious email.

Email systems are trying to detect these forgeries by using various validation mechanisms such as SPF and DKIM.

The exact methods SPF and DKIM use are beyond the scope of this article, and there exist numerous resources online that explain them.

However, it should be obvious that altering the 'from address' of an email is likely to cause problems in email validation because a forged 'from address' is one of the exact scenarios that these mechanisms were designed to identify and prevent.

Changes at Gmail

Previously, if you configured your Gmail to send with a different 'from address', Gmail simply changed that field on outgoing mail. New configurations using this method are no longer supported, and instead it is done in a manner that works with email validation techniques such as DKIM. If you were to add a Send mail as configuration to your Gmail account in recent years, you would be asked for your mail credentials at the domain from which you wished to send email. That is to say, if you wished to use Gmail to send email as user@somewhere.org, you would have to give Gmail the account details (SMTP server, password) to allow Gmail to log in on your behalf to somewhere.org's mail servers to send the email. This results in the validation credentials for an email sent from somewhere.org to actually be generated by the mail servers of somewhere.org, ensuring successful validation at the recipient's end of things.

When Gmail changed their policy such that Send mail as required credentials and configuration to send via external mail servers, they did not retroactively enforce the changes on anyone who had set up their Gmail account in the former manner, where they simply altered the 'from address'. These 'legacy configurations' are still sending from Gmail servers with from addresses altered, and are now beginning to fail validation checks because they are effectively forged emails by the modern definition.

Gmail has implemented additional email headers to try to support validation in some scenarios even with an altered 'from address', such that those emails will at least be accepted by recipients at Gmail, but in practice validation of such an email will always fail if the email is forwarded through another mail server en route. Gmail can choose to apply different criteria for accepting or rejecting mail depending on source and validation, for example they may choose to accept mail from Microsoft mail servers regardless of SPF and DKIM validation - it is up to the mail provider to decide how to treat mail that has failed one or more validation tests.

Finally, Gmail itself is now becoming more strict about accepting emails that do not pass at least one type of email validation. Unfortunately, for those Gmail users with the old Send mail as mechanism still in place, if an email they have sent is forwarded through the CS mail system back to Gmail, it will be rejected by Gmail itself for failing validation.

Solutions

Fixing at the source

The optimal solution is for anyone that still maintains the old Gmail Send mail as feature, i.e. without using the SMTP servers responsible for the domain of their 'from address', to reconfigure their Gmail sending to use the current practice supported by Gmail. Email validation is becoming more critical in modern practice, and to ensure delivery it is essential that you use the mail servers with which your email 'from address' is associated.

Currently this process, using a cs.toronto.edu address as an example, is:

Using the gmail web client on a PC: 

- choose the cog (to go to settings). 
- select "See all settings" 
- select the "Accounts and Import" tab
- the 4th section of this tab is "Send mail as:"
- in "Send mail as:", click "add another email address"
  Add your name and the email address you wish to send as, in this example user@cs.toronto.edu
- click "next step"
- fill in the form as follows:
  SMTP Server: smtp.cs.toronto.edu
  Port: 587
  Username: user
  Password: [user's password]
  Select: Secured connection using TLS (recommended)
- click "Add Account".

When sending a message, the From field will be a drop-down menu, where you can choose which address you are sending your message from.

Please be aware that anyone configuring Gmail through this process is effectively storing their login credentials on servers run by Google.

If you are in a situation where a contact is sending email from Gmail via the old method, and you forward your email to Gmail or another strict email validation site where it is now being rejected for the reasons explained above, and they are unwilling to change their sending method, there may be a few workarounds you can implement to get their emails delivered to your Gmail account.

A workaround using mailing lists

The simplest will be to set up a .forward-nonspam which contains a mailing list, where you create the mailing list with your Gmail address as a recipient. This should allow the forwarded message to pass SPF tests, and it will be accepted by Gmail for as long as they are willing to accept email purely based on a successful SPF validation.

Using procmail to re-send as new email

It may be possible for you to use procmail to effectively 're-mail' their messages to you, rather than forwarding them. This results in a message that effectively comes from CS rather than the outside sender, and thus it would be DKIM signed by our mail servers, and validation checks should succeed. This is not a simple task, and setup and testing will likely not be without risk of misdirected email, and it may break message continuity for such things as contacts, conversation threading, replying, etc. Please consult your PoC if you choose to go this route.