I THOUGHT THIS UPDATE FROM Pivotal Veracity from the Authentication and Online Trust Summit (AOTS) was quite valuable to share with the MediaPost audience. As background, Microsoft started AOTS in 2004 to foster adoption of Sender ID and promote email authentication through the industry. I will warn you, if you are a novice to deliverability and authentication, you will be completely lost in this article, so I recommend you read a few things before you dive into this. Authentication: http://www.deliverability.com/resources/emailAuthentication.php DKIM: http://www.dkim.org.

Thanks to Len Shneyder, director of partner relations & industry communications at Pivotal Veracity, for writing this very timely update.

AOL & AIM. Not much new to report except that they plan to incorporate Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM) into the reputation model in the coming months. By late summer they hope to be actively using both as a means of authentication and reputation checking for connecting IPs as an added data point to help them determine the final disposition of email. As is always the case, they advocate being responsible with your sending patterns and trying to limit your email to "active" customers who have purchase patterns or some form of response-based relationship.

Comcast. Ninety-one percent of all inbound email to Comcast, the largest broadband provider in the United States, is flagged as spam. Comcast monitors the quantity of hard bounces, specifically unknown users, sent to their domains. They use this as a measure of list quality and maintenance. By the end of the year, Comcast announced, they will adopt DKIM as an extension of their reputation check, in addition to launching a feedback loop.

MSFT, Hotmail, Windows Live Mail. Hotmail receives 4.5 billion emails a day, of which 90 % is flagged as spam. Statistics show that 35% of all spam today is "image spam," or messages comprised solely of one or multiple images. Microsoft advised mailers to use the hard fail flag in their SPF/SenderID records (-all), in addition to covering every sub domain with its own SPF/SenderID record. If you are lost at this point, contact your ESP or one of the delivery auditing companies and they can help you understand how to set this up. It is important to note that IPs sending a low volume of email at infrequent intervals will most likely be flagged as "suspect" by this filter as they have similar characteristics to spammers. Because of this, new IPs with no mailing history will have a harder time getting their email to even the junk mail folder. Microsoft is relying heavily on an IP's mailing history as one aspect of reputation in order to determine where email should be placed. Further insights from Microsoft personnel included information on mailing volumes: It's better to email smaller volumes more often rather than larger volumes at infrequent intervals. Consistent mailings of 5-10K emails per day have been documented as performing well.

Implications. ISPs use a variety of filters, including third-party filters, blacklists, whitelists, reputation metrics, volume filters, and content filters. There are many reasons your message may not make it to the recipient's inbox. Reputation metrics consist of one or all three of the following, depending on the ISP: spam complaints (the number of recipients complaining you are spamming them); unknown user bounce rates (bad addresses), and spam traps (emails you send to email addresses harvested from the Web or purchased). Managing feedback loops (spam complaints), good bounce management (removing bad addresses), and carefully governing your email acquisition methods will remain crucial to sustaining a good delivery reputation with the ISPs.

Authentication will become increasingly important as well. If you are not already authenticating with a combination of SPF, SenderID, and DKIM now is the time to get started. Whereas Microsoft adopted SenderID, many others adopted a combination of SPF + DKIM or SenderID + DKIM. Yahoo is even requiring DKIM in order to sign up for their feedback loop (spam complaint reporting).

================== www.emailexperience.org ======================================
Pivotal Veracity's Recap of the Authentication & Online Trust Association Summit
April 17 - 19, 2007
Boston, MA

Critical insights from this conference:

AOL, AIM

How does AOL's spam filtering work? AOL detailed their basic Anti-Spam architecture as a process flow and how it affects inbound email:

Blacklist Filters: Your email will be checked against AOL’s internal blacklist.

Reputation Filters: AOL will review reputation metrics in the form of “abuse data” (spam complaints and spam traps).

Whitelist Filters: AOL will check if your from-address is in the recipient’s address book or if your IP and Domain are on any of their white lists.

From this point down you may bypass some or all of these filters based upon your whitelisting with AOL and if you’ve achieved, or not, the enhanced whitelist.

Volume Filters: If you’re not on AOL’s whitelist, your email will go through AOL's volume filters.

Content Filters: AOL applies its proprietary content filtering and Bayesian filtering rules.

Recipient Filters: Your email may pass a domain block, recipients personal Bayesian (defined by previous actions) rules or recipient’s personal filters

Your email will then be delivered, placed in the spam folder or blocked.

AOL claimed that the best way to improve deliverability is to setup a Feedback Loop with them to reduce the number of complaints they receive from their end users.

Plans exist to incorporate SPF & DKIM into the reputation model in the coming months. By late summer they hope to be actively using both as a means of authentication and reputation checking for connecting IPs as an added data point to help them determine the final disposition of email.

AOL recommends keeping content relevant and fresh. They suggest that you email only active users (subscribers or customers who have either opened, clicked or have an existing purchase history) as a best practice to achieving better deliverability.

COMCAST

How does Comcast's spam filtering work? Comcast detailed their basic Anti-Spam architecture as a process flow and how it affects inbound email:

Your email arrives at Comcast gateway at which time IP routing and reputation classification is established.

3rd Party Filters & Blacklists: Next your email will pass through the Brightmail Spam Filter and be checked against a variety of 3rd party blacklists such as the DNSBL.

Volume Filters: Comcast checks your email volume against traffic pattern models.

Content & Reputation Filters: Your email will pass through Comcast’s content and reputation (spam complaints, unknown user rates) filters.

Your email will then be delivered, placed in the junk folder or blocked.

91% of all inbound email to Comcast domains is flagged as spam.

Comcast is the largest broadband provider in the United States which makes them an easy and eager target for spammers.

Comcast monitors the quantity of hard bounces, specifically user unknowns, sent to their domains. They use this as a measure of list quality and maintenance. Sending active names (subscribers who have shown signs of open, click or purchases) is critical to ensure deliverability.

By the end of the year Comcast will adopt DKIM (Domain Keys Identified Mail) as an extension of their reputation check.

In addition, a Feedback Loop will be launched by the end of this year.

MSFT, HOTMAIL, WINDOWS LIVE MAIL

Ever wonder how MSFT's spam filtering works? Microsoft detailed their basic Anti-Spam architecture as a process flow and how it affects inbound email:

Blacklists & Volume Filters: Your IP and Domain are checked against MSFT's internal and 3rd party block lists. Throttling and-or traffic shaping also happens at this level; connecting MTAs can be completely blocked at this level.

3rd Party Filters: Your email passes through Brightmail’s spam filters.

Content Filters: Your email is then passed through Smart Screen (MSFT's proprietary content spam filter).

Whitelist Filters: Your IP and Domain are checked against Safe-lists and Sender Score.

Authentication check (SenderID): MSFT verifies your domain has authorized the mail to be sent from the sending IP Address (this method of authentication is known as SenderID).

Recipient Filters: Recipients' address books are reviewed to determine if your from-address has been setup in them.

A score is generated at this point which determines the final disposition of your mail (Inbox, Junk or Trash).

Hotmail receives 4.5 billion emails a day of which 90% are flagged as spam.

Statistics show that 35% of all spam today is image spam or messages comprised solely of one or multiple images.

SPF and SenderID records should utilize the hard fail mechanism “–all”. e.g. “v=spf1 mx –all” as opposed to neutrals (~all or ?all).

Every sub domain should have its own SPF and SenderID record.

Microsoft’s attack detection is geared to detect distributed bot-net attacks. IP’s sending a low volume of email at infrequent intervals will most likely be flagged suspect by this filter.

New IPs with no mailing history will have a harder time achieving inbox placement. Microsoft is relying heavily on mailing history as 1 aspect of reputation in order to determine where email will be placed.

According to Microsoft personnel, it’s better to email smaller volumes more often rather than larger volumes at infrequent intervals. Consistent mailings of 5-10K emails per day have been documented as performing well.

Microsoft doesn’t factor in the user’s address book the way that AOL relies on it as a determining factor for email placement. The address book is referenced late in the process, email can be blocked by other filters even if the user has added the sender to their address book and safe listed the domain.

Implications..

ISPs use a variety of different types of filters including 3rd party filters, blacklists, whitelists, reputation metrics, volume filters, and content filters. There are many reasons your message may not make it to the recipient's inbox and focusing on just one aspect of this equation (e.g. blacklists) is insufficient.

Reputation is measurable and is important but it is *not* something you can buy from a 3rd party nor do you need a 3rd party to have one! Reputation metrics consist of one or all three of the following depending on the ISP: Spam Complaints (the number of recipients complaining you are spamming them), Unknown User Bounce Rates (bad addresses), and Spam Traps (emails you send to email addresses harvested from the web or purchased). Managing Feedback Loops (spam complaints), good bounce management (removing bad addresses), and carefully governing your email acquisition methods are absolutely critical.

Content Filters are not dead and should never be ignored. Despite some companies implying that content does not matter, it does indeed as evidenced by each of these ISPs' stated use of 3rd party spam filters such as Brightmail that analyze content, Bayesian or adaptive learning filters that analyze content, and other proprietary content filters.

Blacklists matter and can be monitored. Verifying your IP address and URL domains do not get flagged on highly important blacklists still matters as all ISPs use them to some degree.

Volume filters are also at work at all ISPs. At AOL if you are registered on their whitelist, you can avoid the volume filter. Msft recommends mailing in lower volumes (stagger your mail) although this may not always be feasible for all mailers.

Authentication will become increasingly important and if you are not already authenticating with a combination of SPF, SenderID, and DKIM now is the time to get started as it will put you in good stead prior to the Holiday season. Whereas MSFT espouses SenderID, many others espouse a combination of SPF + DKIM or SenderID + DKIM. Yahoo is even requiring DKIM in order to sign up for their feedback loop (spam complaint reporting).