Forward incoming mail using Amazon Services and Lambda

In this post I’ll try to explain how I used Amazon to forward our incoming mail to an e-mail address like gmail.

Update 25 jan 2017: I seem to have missed some comments on this post, because they ended up in my spam folder :-(. Personally I don’t use my own solution as described here anymore. I think we switched to namecheaps forwarding service after all. Anyways, I couldn’t help myself but googling a bit on this subject, and it seems somebody else came up with the idea of using lambda to forward e-mail. I haven’t tried it but this project has nice little green icons that says it has been tested, so it must be better than what I did here.

Update 16 feb: Updated the lambda script thanks to Chris in the comments :-)

Next day update: Forwarding mime coded mails didn’t show up too well in my mail client. Of course I was testing with plaintext first ;-). Also I found this github page, describing pretty much the same as this post, although they still use S3 objects which I find redundant. I updated my Lambda script. It now sends a raw message, copying the Content-Type and Content-Transfer-Encoding headers from the original headers. This seems to display much better :-)

Use case

Say you have a domain specifically for e-mail campaigns. To setup this domain as a whitelabel domain in Sendgrid, or whatever third party you use, you must probably verify the send address. To do so, you must be able to receive mail.

Maintaining a mailserver is a pain in the ass though, with all those evil spammers around. So we happily leave that to companies who are must better in that. On the other hand, we don’t want to pay for each campaign just to receive this stupid verification link. Forwarding e-mail seems like a nice solution.

However, we found that the services at namecheap, which included e-mail forwarding, isn’t always that reliable. So we switched to Amazon for our DNS services. Wouldn’t it be nice to use Amazon to receive and handle our e-mail as well? Could Amazon simply forward e-mail?

Well, turns out it can forward e-mail, but not with a one-click install. Here’s how I did it.

Services

I used SES for incoming e-mail (and eventually outgoing as well). SES pushes the mail to SNS. And SNS pushes the notification with a Lamda function to e-mail with a little javascript.

I found this post on the amazon forum, but this guy used a s3 bucket to pull the e-mail from. I don’t want to store the mail in a bucket; I want to forward it immediately.

So I changed the script to fetch the information from the SES message directly. After I had fixed the scripts, permissions, etc it turned out that SES doesn’t send the actual contents of the e-mail message to the lambda script. But if you push the message to SNS, then the SNS message does contain the contents. So therefore I push to SNS, and let SNS push to lambda.

Setup SES

SES is Amazons Simpel Email Service. It is able to send and receive mail. But you need to verify your domain first, as well as the Email addresses you are going to use. At first I thought I only needed to verify the e-mail address that I use in the FROM field, but if you are in sandbox mode, you will also need to add the TO address.

Just add it exactly the same way, by pushing the “Verify a new email address” button, and enter the e-mail address that will be receiving the forwarded mails. Of course you can also have your SES account un-sandboxed, but actually I like this extra layer of security.

After you verify your e-mail addresses, you have to setup the Email receiving part. You have to make sure that your domain is configured correctly to let Amazon receive the mail. If you use route53 (which I think you should), Amazon can configure this automatically for you.

When that’s all done, you can create a rule set to receive the mails. But first, we need to setup the lambda function, and then the SNS service.

Setup Lambda

The lambda function can be created in python, node.js and java. I already had a node.js script to start with, so I changed that a bit to fit my needs. Just go to the lambda section and follow directions on the screen until the part where you can ‘Create a Lambda function’.

In my case it first came up with a screen ‘Select blueprint’, but you can just skip that. Then fill in the name, description and runtime (Node.js). And then use this script, but change it a bit to fit your needs.

At handler I used index.handler, and at Role I used the basic execution role. You will need this role later on to add permissions.

Now you could run a test with SNS, but the example test data doesn’t contain an e-mail message, so this won’t work very well. It should not come up with any errors though.

The downside of this forward is that you cannot see the original “to” address in your e-mail client directly. It is sent in the X-Original-To header though, so you could use this to build a filter or something like that.

Setup SNS

Now go to the SNS service to create a Simple Notification Service. First you create a topic that you name ‘ForwardEmail’ or whatever you like.

Then on the navigation on the left you can click on “topics” which gives you an overview of the topics. Click on that geeky looking ARN name. Now you should be able to create a subscription for this topic.

Here you select AWS Lambda as protocol, and then select the endpoint you created in the previous step.

So now you have a notification service that forwards notifications to the lambda script. You should note that the lambda function is only suitable for e-mails though, so if you connect another Amazon service to this SNS things will probably break.

Setup SES again

Now back to the SES configuration again. Make sure both the forwardFrom and forwardTo address are verified at the “Email addresses” section.

Now create the rule set at Email Receiving.

I setup the first action as SNS, selecting my ForwardEmail SNS. Then as second action I added Stop Rule Set, but I’m note sure if this is mandatory.

Test your setup

When I was testing my Lambda script at the Lambda service section, I basically had two problems, after I weeded out all the bugs in my script. First I got an e-mail verification error, because I didn’t know I had to verify the receiving e-mail address as well.

Second, I received an error, something like this:

User arn:aws:sts::167718239101:assumed-role/lambda_basic_execution/awslambda_724_20160203104053770' is not authorized to perform ses:SendEmail' on resource `arn:aws:ses:us-west-2:167718239101:identity/forwarder@email-service-support.com'

Turns out that Amazon also has a very sophisticated permission management thing. To fix this, go to IAM under Services (Security & Identity). At the dashboard you can see that you have Roles, my dashboard says Roles: 1. Click on it and you will see an overview of your roles. Here you will also see lambda_basic_execution.

Click on it, and at Inline policies click on “Create Role Policy”. Then add this policy:

That’s it. At the service CloudWatch you should be able to see the logging of your lambda script. Also you should probably reduce the retention, because if you keep your logs forever I think Amazon will make you pay for it.

Hope this helps you. I am no Amazon expert whatsoever, so I’m afraid this is all I can tell you at the moment :-).