SURBL

An Open Letter To Operators Of Redirection Sites

Dear Redirection Site Operator,
In our efforts to help identify unsolicited messages and related abuse, we have noticed the use of redirection sites in unsolicited messages. The effect is to "wash" the sites through redirectors presumably to defeat URI-matching techniques. In essence, abusers are using redirection sites as legitimate-looking camouflage to conceal their sites and bypass URI checks.

URI-checking mail filter programs such as SpamAssassin have been updated to filter out the redirected sites when a destination remains visible, for example as part of a path or in a CGI argument. However for those "opaque" redirector sites which hide, encode or key the destination so that it's not visible (after extraction or decoding) in the URI, the only option remaining for URI checkers is to follow the redirector through to see what site it leads to. Clearly this would be too resource-expensive for most mail filters, especially if a chain of multiple redirections were used.

Without a doubt abusers will figure out this loophole soon enough, and the abuse of redirectors in unsolicited messages will increase as a result.

A good solution to the issue of abusers washing their URIs through redirectors may be for the operators of redirector sites to deny services to sites listed on SURBLs. Several operators of redirection services are doing this currently, as you can see from the news below. Perhaps the worst solution would be to do nothing and let abuse of redirector services continue. This also increases the operating and abuse-handling costs of the redirector service, not to mention the costs to the community of unsolicited messages and related phishing and malware sites.

Therefore we appeal to operators of redirection sites to deny access to your services RBL listed sites. This is easy to do via a DNS query, as explained in the Implementation Guidelines. You may be able to do these queries using existing code and scripts on our Links Page and in our best practice recommendations below.

Best Practice Recommendations

• Check links against SURBL datasets using existing programs such as the below.
• In addition to checking links at creation time, also recheck periodically including when redirecting. But don't check on every redirection or too frequently. Rechecking once every 10 minutes may be often enough.
• Deny link creation from malicious IP addresses, for example using Spamhaus lists.
• Also check the resolved IP addresses of domain name and nameserver records against lists such as Spamhaus.
• Disallow redirection to other redirectors. In other words, don't shorten other shortener sites.
• Return an HTTP status code of 410 when an abused redirector has been removed. This makes it easier to determine that the abuse has been removed and no longer requires reporting. You can still monetize the page by providing a page that browsers can display as well.
• Register an abuse contact role account at abuse.net, which is widely used to find where to report abuse.

SURBL is now sending shortener abuse reports in ARF (RFC 5965) format.

Partial list of shorteners using our intelligence

Ask Bjørn Hansen of Metamark.net is now using our intelligence to deny services to abusers:

30 April 2004: Ask Bjørn Hansen of develooper.com is using our data to block abused sites in the Metamark Shorten™ Service URI shortening and redirection service. This is the first use of SURBL data to prevent abuse of a redirection site that we've heard of! Great going! Ask explains his motivation as: "I mostly did it to make it less likely that I'll have to deal with abusers of the service manually. Hopefully the other redirection services will realize that benefit soon as well."

SnipURL is another redirection service that is using SURBLs in a similar way:

23 July 2004: SnipURL is now using our intelligence to deny abusers access to their URL shortening and redirection service.

And TinyURL is also using our intelligence:

17 November 2005: Kevin Gilbertson reports that he has been using SURBL intelligence for "about a year now" to protect his popular redirection site TinyURL.com against abusers and phishers.

As is Notlong.com:

18 November 2005: Eric Hammond says that his Notlong.com redirection service "has been protected by SURBL intelligence since July 2004."

As is easyurl.net:

27 September 2006: Mark Jeftovic reports that they "are now checking destination URLs against SURBL intelligence refusing to shorten them via easyurl.net."

And YATUC.com:

21 February 2008: Daniel Flandorfer says: "Our redirection service YATUC (yet another tiny url creator) uses SURBL intelligence to check links added into the system. Every link will from now on be checked against SURBL intelligence and - if not passed - not be added into the system. [...] Additionally, we will periodically check all links in our system and if needed mark them so that they can't be used any longer. Today we already marked 858 links as spam !! We hope that we can help a bit to reduce the massive use of spam urls."

bit.ly too:

4 August 2008: Nathan Folkman has integrated SURBL intelligence checks into his bit.ly URL shortening service.

And is.gd:

4 August 2008: Richard West says, "I consider SURBL intelligence an extremely valuable tool to deny redirection to the majority of spam sites and greatly decrease the amount of abuse we experience."

And Delivr.com:

25 March 2009: David Harper says, "Delivr.com has integrated SURBL checks into its mobile-friendly sharing service."

As is safe.mn:

17 March 2011: Julien Sobrier says: "Safe.mn is a secure URL shortener. All links are checked against several blocklists, including SURBL intelligence. Unlike most URL shorteners, we also check the content (antivirus, malicious ActiveX, etc.) and we check that final long URL, after potential HTTP and JavaScript redirection, is checked. The links are then checked again regularly to ensure they are still safe."

And cort.as:

31 March 2011: "We are using SURBL intelligence in cort.as due to our main interest in security for our users. SPAM is one of the biggest problems for URL shorteners and we are strongly fighting spammer sites," says Raul Rivero, CTO of elpais.com.

Other shortener services not listed above are also using SURBL intelligence. At this point using SURBL intelligence has become nearly a standard practice against shortener abuse. Please see more references at: http://www.surbl.org/news.html.

Thank you for your attention and your hopeful consideration of using our data to stop the abuse of your services.