Forward incoming mail using Amazon Services and Lambda

In this post I’ll try to explain how I used Amazon to forward our incoming mail to an e-mail address like gmail.

Update 25 jan 2017: I seem to have missed some comments on this post, because they ended up in my spam folder :-(. Personally I don’t use my own solution as described here anymore. I think we switched to namecheaps forwarding service after all. Anyways, I couldn’t help myself but googling a bit on this subject, and it seems somebody else came up with the idea of using lambda to forward e-mail. I haven’t tried it but this project has nice little green icons that says it has been tested, so it must be better than what I did here.

Update 16 feb: Updated the lambda script thanks to Chris in the comments :-)

Next day update: Forwarding mime coded mails didn’t show up too well in my mail client. Of course I was testing with plaintext first ;-). Also I found this github page, describing pretty much the same as this post, although they still use S3 objects which I find redundant. I updated my Lambda script. It now sends a raw message, copying the Content-Type and Content-Transfer-Encoding headers from the original headers. This seems to display much better :-)

Use case

Say you have a domain specifically for e-mail campaigns. To setup this domain as a whitelabel domain in Sendgrid, or whatever third party you use, you must probably verify the send address. To do so, you must be able to receive mail.

Maintaining a mailserver is a pain in the ass though, with all those evil spammers around. So we happily leave that to companies who are must better in that. On the other hand, we don’t want to pay for each campaign just to receive this stupid verification link. Forwarding e-mail seems like a nice solution.

However, we found that the services at namecheap, which included e-mail forwarding, isn’t always that reliable. So we switched to Amazon for our DNS services. Wouldn’t it be nice to use Amazon to receive and handle our e-mail as well? Could Amazon simply forward e-mail?

Well, turns out it can forward e-mail, but not with a one-click install. Here’s how I did it.

Services

I used SES for incoming e-mail (and eventually outgoing as well). SES pushes the mail to SNS. And SNS pushes the notification with a Lamda function to e-mail with a little javascript.

I found this post on the amazon forum, but this guy used a s3 bucket to pull the e-mail from. I don’t want to store the mail in a bucket; I want to forward it immediately.

So I changed the script to fetch the information from the SES message directly. After I had fixed the scripts, permissions, etc it turned out that SES doesn’t send the actual contents of the e-mail message to the lambda script. But if you push the message to SNS, then the SNS message does contain the contents. So therefore I push to SNS, and let SNS push to lambda.

Setup SES

SES is Amazons Simpel Email Service. It is able to send and receive mail. But you need to verify your domain first, as well as the Email addresses you are going to use. At first I thought I only needed to verify the e-mail address that I use in the FROM field, but if you are in sandbox mode, you will also need to add the TO address.

Just add it exactly the same way, by pushing the “Verify a new email address” button, and enter the e-mail address that will be receiving the forwarded mails. Of course you can also have your SES account un-sandboxed, but actually I like this extra layer of security.

After you verify your e-mail addresses, you have to setup the Email receiving part. You have to make sure that your domain is configured correctly to let Amazon receive the mail. If you use route53 (which I think you should), Amazon can configure this automatically for you.

When that’s all done, you can create a rule set to receive the mails. But first, we need to setup the lambda function, and then the SNS service.

Setup Lambda

The lambda function can be created in python, node.js and java. I already had a node.js script to start with, so I changed that a bit to fit my needs. Just go to the lambda section and follow directions on the screen until the part where you can ‘Create a Lambda function’.

In my case it first came up with a screen ‘Select blueprint’, but you can just skip that. Then fill in the name, description and runtime (Node.js). And then use this script, but change it a bit to fit your needs.

At handler I used index.handler, and at Role I used the basic execution role. You will need this role later on to add permissions.

Now you could run a test with SNS, but the example test data doesn’t contain an e-mail message, so this won’t work very well. It should not come up with any errors though.

The downside of this forward is that you cannot see the original “to” address in your e-mail client directly. It is sent in the X-Original-To header though, so you could use this to build a filter or something like that.

Setup SNS

Now go to the SNS service to create a Simple Notification Service. First you create a topic that you name ‘ForwardEmail’ or whatever you like.

Then on the navigation on the left you can click on “topics” which gives you an overview of the topics. Click on that geeky looking ARN name. Now you should be able to create a subscription for this topic.

Here you select AWS Lambda as protocol, and then select the endpoint you created in the previous step.

So now you have a notification service that forwards notifications to the lambda script. You should note that the lambda function is only suitable for e-mails though, so if you connect another Amazon service to this SNS things will probably break.

Setup SES again

Now back to the SES configuration again. Make sure both the forwardFrom and forwardTo address are verified at the “Email addresses” section.

Now create the rule set at Email Receiving.

I setup the first action as SNS, selecting my ForwardEmail SNS. Then as second action I added Stop Rule Set, but I’m note sure if this is mandatory.

Test your setup

When I was testing my Lambda script at the Lambda service section, I basically had two problems, after I weeded out all the bugs in my script. First I got an e-mail verification error, because I didn’t know I had to verify the receiving e-mail address as well.

Second, I received an error, something like this:

User arn:aws:sts::167718239101:assumed-role/lambda_basic_execution/awslambda_724_20160203104053770' is not authorized to perform ses:SendEmail' on resource `arn:aws:ses:us-west-2:167718239101:identity/forwarder@email-service-support.com'

Turns out that Amazon also has a very sophisticated permission management thing. To fix this, go to IAM under Services (Security & Identity). At the dashboard you can see that you have Roles, my dashboard says Roles: 1. Click on it and you will see an overview of your roles. Here you will also see lambda_basic_execution.

Click on it, and at Inline policies click on “Create Role Policy”. Then add this policy:

That’s it. At the service CloudWatch you should be able to see the logging of your lambda script. Also you should probably reduce the retention, because if you keep your logs forever I think Amazon will make you pay for it.

Hope this helps you. I am no Amazon expert whatsoever, so I’m afraid this is all I can tell you at the moment :-).

13 thoughts on “Forward incoming mail using Amazon Services and Lambda

  1. Good stuff, however the code didn’t work for me out of the box, I had to change both occurences of:
    headers += res[0];
    to
    headers += res[0]+”\r\n”;

    otherwise the mail header was corrupt and the mail wasn’t forwarded at all.

    Will try with multiple recipients next.

    • Thanks! I tested this by sending mail from a google-account mail, and this worked fine. But sending from Thunderbird failed. I updated the Lambda script and tested it with hotmail and yahoo as well. With a few more modifications it seems to work now, so far so good.

      • Hey rolandow!

        I have the same problems with Thunderbird.

        Though google mail is not working properly. The content of the message is not correct. I only get “Empty email” as the content. Cloudwatch tells me “Cannot read property ‘Message’ of undefined”

        Do you have an idea?

        • I’m sorry, I missed a few comment notifications because they ended up in my spam folder. I haven’t touched this script for a while, and we don’t use it anymore either. In the end parsing e-mail is more complicated than this little script I suppose, so there may still be situations where it doesn’t work properly.

          I think we switched to the namecheap e-mail forwarding service after all.

        • Hello,
          Thank you Rolandow for the code.

          I had the same problem of “empty email” from google. A solution would be commenting out the if statement which filters the spams (lines 9 ~ 12).

    • Hi Alex,

      You shoudln’t do this, because this will conflict with SPF records that are set for the original from. Your forwarded mail will then probably end up in the spam list. It’s also bad practise, so maybe the Amazon SMTP servers will even end up on the blacklist.

      The Reply-To header is set though, so when you use the reply function in your mail client, it should go right to the original sender.

  2. I could be wrong, but can’t you just skip the lambda step and just subscribe the target forwarder address to the SNS topic?

  3. Thank you so much.

    I tried the github post, and I had trouble getting it to work. I had also tried an email forwarding service, improvmx, but it recently had server problems. I was going to try namecheap, but that would have meant trying my luck also.

    This worked for me. I tried testing from different services, but then I dared to use my aws email for a professional setting (it’s @.com, just to be cool). It didn’t work. Maybe it had an attachment? I would have to find out.

    I should also note that, when verifying any email addresses under SES, you might have to set the DKIM settings. It made things work for me.

  4. I want to achieve the below flow. One of my applications need to send email to SES and i want to be able to receive the email and read the content of the email. There is a username tag which i want to extract and make a API call to third party system to find the user’s manager and forward the email to that manager email address. How can i achieve this.

    I have verified the domain name and from address. But i will have to generate the recipient dynamically using the above logic. Appreciate your help.

    • I don’t have experience with other Amazon services. I guess you’d have to figure out how to make a REST call to your third party system that will give you back the managers e-mail. Javascript isn’t my core programming language, so I’d have to google and get it working by trial on error either. Good luck!

Leave a Reply

Your email address will not be published. Required fields are marked *