A Spam Blacklist Shuts Down, And For Some Chaos Ensues
By Ken Magill
More than six months ago, Brielle Bruns announced she was shutting down the anti-spam blocklist Abusive Host Blocking List, or AHBL.
What has happened since can serve as instructional for marketers on how blocklists work. I found the whole business fascinating, at least.
Bruns made the shutdown announcement in several online forums and on the AHBL home page multiple times and gave the email administrators using the service until Jan. 1 to remove the blocklist from their spam-fighting formulas.
AHBL is a Domain Name Service Blocklist that helps email administrators identify sources of spam so they can treat incoming email appropriately.
As was expained quite well on email deliverability consultancy Word to the Wise’s blog post on this subject: “A DNSBL works like this, a mail server checks the sender’s IP address of every inbound email against a blacklist and the blacklist responses with either, yes that IP address is on the blacklist or no I did not find that IP address on the list. If an IP address is found on the list, the email administrator, based on the policies setup on their server, can take a number of actions such as rejecting the message, quarantining the message, or increasing the spam score of the email.”
Well-known spam fighting service SpamAssassin reportedly used the AHBL as part of its spam-scoring system.
SpamAssassin also reportedly removed the AHBL from its system as soon as Bruns announced she was shutting the blocklist down.
Many others apparently did not. And that’s where the problems started.
The trick to shutting down a popular blocklist is to get the thousands of email administrators using it to stop querying it and needlessly sucking up bandwidth. Bruns decided the best way to accomplish this was to blacklist the entire Internet so administrators would realize something was wrong.
“The administrators of AHBL have chosen to list the world as their shutdown strategy,” wrote Josh at Word to the Wise. “The DNSBL now answers ‘yes’ to every query. The theory behind this strategy is that users of the list will discover that their mail is all being blocked and stop querying the list causing this. In principle, this should work. But in practice it really does not because many people querying lists are not doing it as part of a pass/fail delivery system. Many lists are queried as part of a scoring system.”
As a result, many administrators may be erroneously giving some incoming email higher spam scoring than is appropriate and mishandling mails as a result. Some may not even know yet there is a problem.
I reached out to Bruns with an email saying blacklisting the entire Internet looked to this layman like an irresponsible way to decommission a blocklist. She was kind enough to provide a detailed answer. Here it is in full:
I've been running the AHBL for around 11 years now. Decided back in March/April 2014 to shut down the service and give people until Jan. 1st, 2015 to stop using the service. I made the announcement on several mailing lists frequented by mail admins and tech minded individuals, as well as on our website's main page. I also disabled and put up a warning on the DNSbl lookup tool.
Initially I changed the name server records for the zones to invalid ones, as to create noticeable delays and warnings in people's log files in the hope that people would discover it and remove the zones from their server software.
In late December, I made another warning prior to the new year. Again, the notice got posted to our website, on various mailing lists, and there was even lively discussion about what kind of impact this would have. I agreed to delay the wild-carding of the zone files until the 5th, so that network/system admins would have time to check their systems without the holidays getting in the way.
On the 6th, the wildcard was put in the various public DNSbl zones.
Because of the way DNSbl queries work, by default 'no response' or 'no such record' means 'This host is okay'. Removing the zones have no impact on the incoming queries, and will show no sign of anything being amiss. Server software will continue to happily query like they always have.
The only way to make it obvious that something needed to be fixed, was to answer each query as 'This host is NOT okay'. The downside being the situation some people are experiencing.
Many of the people having issues are because of:
* Unmaintained server software - for one reason or another, they aren't monitoring, updating, or keeping track of what’s going on with machines under their control.
* Hosted/outsourced services where the the use of DNSbl lists aren't documented and/or aren't being maintained by the people who are providing the services.
Software developers are partly to blame for this too - many packages are poorly documented, and don't always tell the server admins what they are doing or using. Quite a few packages are obsolete or abandoned, but were never replaced by end users.
Many of these software packages are commercial pieces of software, and their developers implemented support and sold their 'solution' without ever contacting us or even considering the load their software would be putting on us.
SpamAssassin, perhaps one of the best software packages out there for spam filtering, disabled support for the AHBL lists almost as soon as I announced they were going away, both in new versions of the software and in the automatic updates for rules in previous versions.
However, many people during setup of SA, either disabled the automatic updates, or never activated them in the first place, and are thus having mail issues due to outdated rules.
Message to Magill Report readers: It’s probably a good idea to ask your email administrators if your systems are querying AHBL as part of their spam-fighting formula. If they are, obviously they should stop. And if they use SpamAssassin, obviously it should be the latest, most up-to-date version.