Email is an open system, right? Anyone can send a message to anyone... unless they are on Gmail!
- vmailer1.vig.co.nz 69.48.142.156
- vmailer3.vig.co.nz 190.92.153.150
It goes without saying that our messages are not spam, and we do everything right to prevent them ending up in spam folders. Both email servers have PTR records set up, and SPF records only allow email emanating from these two servers. Every message is cryptographically signed with DKIM, and DMARC records tell receiving servers to apply all these restrictions strictly.
Neither server is a open relay, and both send only our messages - no-one else is sending spam from these servers.
All this effort pays off. We check our servers aren't on spam blacklists every day, and our Google Postmaster Tools account shows practically perfect scores on all metrics:
Can anyone hear a "but" coming?
But... Gmail is rate-limiting our messages daily.
Those blue dots show over 3,500 Gmail customers having the booking confirmation email they asked for delayed by up to 12 hours. Our support people get several calls a day asking about missing confirmation emails, and they wearily explain that Gmail is delaying delivery for no good reason.
Our system has detected an unusual rate of 421-4.7.28 unsolicited mail originating from your IP address. To protect our 421-4.7.28 users from spam, mail sent from your IP address has been temporarily 421-4.7.28 rate limited.
"An unusual rate of unsolicited mail"? We've established we don't send spam, and Google's own Postmaster Tools can't detect any spam, so it seems to me that Gmail's detection algorithm is broken.
And it has been broken for a while, because we've been rate-limited for months. We've done everything we can to alert Google to the problem, and this blog post is a last-ditch attempt to get the message across. If you know anyone at Google, please send them a link.
Wait, I think I hear another "but"!
BUT... Problems like this are killing independant email.
This rate-limiting issue is not the only thorn in the side of email server admins. If you search the forums, you'll come across hundreds of reports of the big three email providers (Google, Microsoft and Yahoo) making email delivery so difficult that independant email servers are becoming untenable - keeping messages flowing is just too hard.
And this is happening after SPF, DKIM and DMARC provided a solution to the spam problem.
Any mail system can remove practically all spam by insisting messages conform to those three standards, so Gmail could - and should - be accepting our squeaky-clean messages without restriction.
Instead, the big three are making independant email delivery so difficult that we give up and move our inboxes to their services, and use one of their partners (like MailChimp or SendGrid) to deliver our system-generated messages.
I really hope I can write a follow-up saying that fixing the rate-limiting problem proves that Gmail is still committed to the open email standard. The alternative is that email will be subsumed into giant corporation's proprietary systems, and the wonderfully open and extensible message service we've enjoyed since the dawn of the internet will be gone.
Well.... welcome to the club. I have been setup independent email server for people for the last 10 years and making EVERYTHING you mention to be out of the spam jail but Google will ALWAYS put message in the spam folder for no reason. Microsoft do the same btw.
ReplyDeleteAt least until web servers start refusing creating an account from all 3 MMP (mafia mail provider), this isn't going to change. Keep all the evidences that they acted in an unethical, not respecting standard and adding an obvious commercial burden on your standard email service, so when they'll fight back, you are covered.
ReplyDeleteThe message on your sign up page could be "[ ] I understand that Google GMail is unreliable with unexplained 12H offset for mail delivery. And I still want to use that rubbish email service to sign up. YOURSERVICE can not be claimed for any damage due to a third party service not respecting email's delivery standard."
I'm pretty sure that, as soon as theses messages contaminate the internet, Gmail will change its policy.
The saddest aspect of this situation is that the malicious behavior is not a targeted attack on independent mail servers, but rather a result of the ignorance and carelessness of the employees involved. This lack of knowledge and thoughtlessness is a direct consequence of belonging to an enormous organization with abundant resources and numerous clients. Regrettably, this issue seems nearly insurmountable unless there will be a way to run a free email service provider with an exceptional user interface.
ReplyDeleteIf you're EU resident please consider pursuing another Google fine for monopoly behaviour.
ReplyDeleteTake them to court...
ReplyDeleteI'm a System Engineer who has set up email systems for many years. These issues are just as you describe, and have been longstanding issues. The decision makers at google have no visibility or feedback mechanisms by design, about the only thing you can do is take google out of the equation, file FTC complaints, and reach out to congressional representatives. Seeing as its a school, I don't see a class action happening. Setup a server you control that has notifications after logging in, and have the parents check and confirm. Free email service has been made unreliable.
ReplyDeleteIt all reminds me of Embrace Extend Extinguish. Emails have been embraced by corporatocracy, and then turned into walled garden.
ReplyDeleteThis is interesting, but instead of drawing the conclusion that google is destroying an open standard by doing so, consider the amount of nefarious sh*t google has to deal with on a regular basis. A 12h delay on a small school email, compared to tools made to deal with bots from all over the world, is almost a miracle. When you’re equipped to deal with the same amount of nefarious traffic, you can still cant make the argument, because they havent changed the standards
ReplyDelete