As of 2023-07-26 FeedMail messages sent via AWS SES have an invalid feedmail.org DKIM signature. This may result in messages ending up in your spam folder. This is an ongoing issue and updates will be posted here as they are available.
2023-09-25 We have issue an update that should avoid SES re-formatting signed fields. This is a workaround but should result in valid signatures.
Who is affected?
This is unlikely to affect most of our users for the following reasons:
- 97% of our mail is sent by us directly, not via AWS.
- This mail still has a SPF-verified sending IP.
However this may still affect users because spam filters may consider these messages less "good" than they would have been with a valid DKIM signature.
The most notable Inbox Providers affected by this are Apple and Microsoft. FeedMail uses AWS SES for these providers as they reject all messages from our network provider. However other smaller providers may also have a portion of their messages sent via AWS SES.
What to do if you are affected?
If you are affected the best option is to add a rule to ensure that messages from feedmail.org are not set to spam. The process for doing this depends on your email provider.
If you have other concerns please contact FeedMail Support.
Technical Description
Background
FeedMail uses DKIM to apply a digital signature to outgoing email. This allows users who receive the email to know that the message is from FeedMail (not an impersonator) and hasn't been altered.
FeedMail uses a multi-pronged approach to delivering email. We have a "direct send" approach that we use for inbox providers that we have built a good reputation with. These providers trust feedmail.org and accept our email reliably. So FeedMail directly connects and delivers email to these providers. This provides maximum privacy, performance, control and lowest cost. However some email providers are much pickier. Most commonly they have blacklisted our IP range and reject every email even though it is coming from feedmail.org. In order to resolve this issue we proxy these messages via AWS SES. This allows us to use an IP address with a higher reputation for a small fee. The problem exists with these proxied emails.
Header Munging
The problem appears to be around a single header. FeedMail sends the following header in our outgoing messages:
Reply-To: "FeedMail Support" <contact-project+kevincox-feedmail-support@incoming.gitlab.com>
AWS SES has recently started changing this to:
Reply-To: FeedMail Support <contact-project+kevincox-feedmail-support@incoming.gitlab.com>
While this change is semantically unimportant it is enough to break our DKIM signature. Since the message has been altered it does not validate and can't be confirmed as coming from us.
Who is at fault?
We believe that AWS SES is behaving wrongly here. Their documentation clearly states which headers may be altered by SES and therefore can't be signed. Reply-To is not on their list so altering it is breaking their contract of preserving the DKIM signature.
Possible Solutions
The simplest workaround would be to skip signing the Reply-To header. However since this header may be used by users to contact support it could be a valuable phishing vector. So it would be best not to skip signing this header. However if SES remains broken we may need to stop signing it.
Another possible workaround is to try to match SES's format in hopes that they don't alter it if it is already in that format. this is possible but would require altering the libraries that we use to generate the email messages and would open up a game of chasing SES's header encoding format.
One last option is moving off of SES. This is a difficult option because SES is very inexpensive and pay-as-you-go. This is important to FeedMail as we only send a small volume of mail via SES and minimum monthly feeds would blow our budget. We aim to be low cost so keeping our mail sending cheap is important. Furthermore SES is one of the few providers that allow us to sign our messages before submission. This improves security by not allowing them to impersonate us with our DKIM key. Other than this bug they have been a very good provider and we would prefer not to have to move off. However we have experimented with various providers in the past and it is very easy for us to switch. So if necessary we will find an alternate provider.
What we are doing to resolve this issue?
Unfortunately AWS has no unpaid support, even for bugs in their services. We have tried to make them aware of this issue but lack any official channels to do so.
If there is no movement on their side we will explore some solutions that can be done from our side as described in the above section.
Comments
Post a Comment