Goodbye own mail server

After almost a decade of hosting my mail server, an era is ending. My mail server is slowly drained and will be shutdown soon. This is a look back at the years, what troubles I faced, and why I finally gave up and will host my stuff with a hosting provider instead.

Mail is still an open concept, but it gets harder

Many companies have tried to replace email with proprietary systems, but email's status as an open protocol has persisted. However, it's no longer truly distributed. Unfortunately, creating a new, independent email node is no longer as easy as it used to be.

I have been self-hosting my email ever since I got my first domains almost a decade ago. It's been a journey to learn about setting up everything properly, and also a nice thing to say you truly host your stuff.  I felt like a first-class citizen of the Internet, and I learned so much.

Over the years, quite a set of standards have evolved to make email more secure. There are many policies you should implement, all to fight the increasing amount of malicious actors out there sending Spam, Phishing mails or simply attacking servers.

There are a ton of blacklist providers out there, many legitimate ones. But also some that use it to make some bucks putting IPs on their lists and forcing you to pay money to get unblocked. In addition, there are plenty of untransparent rules, where some just block you if you happen to host your server with a provider where one customer used to send spam once.

Providers like Microsoft tend to just block everyone and require them to unblock IPs manually, without a real verification. One might consider this a practice to simply increase efforts of hosting an own mail server.

Problems I faced

To give you an idea of what troubles I faced over the years, I tried to sum them up in a few sections.

Getting blocked by providers because the domain was to new

Until your domain has its first birthday, some providers don't see you as trustworthy. I kid you not, had to learn this the hard way. As these rules can be different for any provider, it could be even longer for some.

Microsoft randomly blocking my IP

I did the dance of unblocking my IP using literally a 10-minute mail and specifying my IP in the beginning. This worked, and the mail got unblocked. After almost a year of hosting my mail server, I suddenly got blocked again, no reason given. Did the same dance again, and it worked ever since.

Until this day, I keep an live.com mailbox account just to verify that my mails arrive and test occasionally, also especially important when a migration with a changing IP has to happen.

An MX record must resolve to an IP address

This is stated in RFC 2181 10.3, and not all mail providers are this restrictive. So let's give a concrete example, what would not work:

Here, the MX record does not directly resolve to an IP or an alias; therefore, the mail would not be delivered by some providers like GMX. The following is valid:

Getting shadow blocked

By some providers you get shadow blocked, your mail won't arrive, but you also won't be notified in any way. This sometimes happens because providers count mails by given IPs sometimes even domains and give you a gradual limit E.g., you can send 10 mails per day for one week, 100 for the next week and so on.

Marked as spam just because you are not a known provider

Had this happened to me more times than I would like to admit. The spam level was very low, still the mail appeared in the spam folder. Talking to support revealed the reason, simply because I am not one of the big providers out there, they don't trust you “out of principle”.

When and why did I give up?

Moving mail servers is hard

When you get a new mail server, the whole dance starts again:

• slowly get the IP out there to mail providers
• do all the manual bits of getting unblocked, doing stupid “preventive block” unblock forms
• ensure that not too many mails are sent to providers

It does not matter if your domain is clean, the only thing that matters is the IP. Once I was unlucky and got an IP from a hosting provider that was used before to send spam. While the IP range of the provider was clean, this particular IP was not. I had to cancel my subscription and get a new one hoping it would be better. In such cases, support from companies can be quite stubborn.

Another migration?

I did already four mail server migrations due to server upgrades over the years. Every time it was a bit of hassle, but every time I managed to get a good deliverability. You get an idea what needs to be done and follow it almost automatic.

Still, the process is quite time-consuming and annoying, while always being at risk that the move was to quick and mails won't arrive.

An offer I could not decline

This time it was different, netcup had a few perfect Black Friday deals enabling me to host my mails on servers with a good reputation and a whole team of engineers in the back. And that for not even half the price it costs me to host it myself. Given the struggle I would have to go through again, I finally gave up.

Most of my mail is already handled by netcup, with a few more mailboxes for projects to be migrated in the upcoming months.

The neat part is that they now do all the hard bits and I just get to sit back and let them do that with a higher availability. As I also already host my servers and domains there, it was also straightforward to migrate.

Should you still host your mail server?

Many people will tell you not to, and it definitely gets harder. While still possible, you will have to have a lot of patience and time on your hands. If I had to choose again, I am not sure if I would go that path again. But the learnings you will get on the way is undoubtedly worth it. This also helped me a lot in the job, where I also took care of mail infrastructure throughout the years.

If you really want to, here are a few tips

If you intend to get your mail infrastructure out there, I can only recommend to start slowly and take care of a few aspects

• Read a lot of material about specifications, ensure you get all the policies right from the very beginning.
• Take a good look at the provider you are picking, check their IP ranges, forums, reddit etc.
• Keep a few mailboxes handy at the big providers you intend to send to.
  For me, it was a good thing to keep Gmail, Outlook and GMX mailboxes to verify deliverability of mails. This applies for incoming and outgoing mails alike.
• Use tools like MXToolbox, imapsync, mail-tester.com to help you to verify stuff works.