Marketing’s Weekly Dose of the Truth

Ken Magill

About Us

Bam! Kapow! The Great Open-Rate Debate


So just where does the highly flawed open-rate metric fit in the email marketer’s measurement arsenal? Does it even have a place?

After all, an “open” is recorded when the receiving machine calls for graphics from the sender. So with graphics turned off by default in many email inboxes and graphics appearing in the preview panes of others, the “open rate” doesn’t really measure “opens” at all.

After a short back-and-forth on Twitter with a marketing professional who considers the open rate all but useless, I decided to pose the question to participants in Bill McCloskey’s Only Influencers discussion list of digital-marketing professionals.

Let’s just say the question turned out to be a little more complicated than I thought and the Influencers had A LOT to say on the subject. So much, in fact, that the debate will appear in this newsletter in two installments.

At times, things even got a little heated. [The Magill Report loves heated.]

At certain points, the discussion isn’t necessarily lightning-fast reading. But it is jam packed through and through with insights from some of the smartest digital-marketing professionals in the world.

Dela Quist, CEO, Alchemy Worx

Open rates are good for one thing and one thing only.
Misleading the people who use them as a metric.

Bob Frady, director, email marketing, ZeetoMedia

While I hesitate to disagree with Dela...I would say that "Misleading the people who use them as THE metric."

Are they useless? No. Are they super-powerful? No.

Open rates are just another ingredient in helping a marketer determine the value of their program.

Jordan Cohen, vice president of marketing, Movable Ink

"Show me an email marketer who thinks open rates are useless, and I'll show you an email marketer who is useless."
- Anonymous, early 21st century

Weird, possibly made-up quote aside, of course open rates are an important metric. Marketers just need to understand them in context, and also measure success using a host of additional metrics.

Understanding that the "open rate" is more accurately described as the "HTML Open Rate" or "Image Render Rate" is critical:
• In email clients the display images by default, it is easy to bank an "open" even if the recipient casually glances at the message. Therefore, it's important to further measure things like read length in those cases to ascertain whether it was a casual open or an engaged open. Advances in email tracking and analytics technology now make this possi ble.
• In email clients that suppress images by default, a marketer's open rates will generally be lower BUT those are higher intent opens -- the recipient has either changed their settings to always display images from the marketer, or has actively clicked to display the blocked images. This can indicate that the marketer is sending emails with great, provocative subject lines, and/or that, over time, the marketer is sending messages/content that recipients look forward to receiving.

John Caldwell, principal and founder, Red Pill Email

Doh! Plus one Jordan’s quote:

"Show me an email marketer who thinks open rates are useless, and I'll show you an email marketer who is useless."

- Anonymous, early 21st century

Blah blah blah Dela’s quote:

“Open rates are good for one thing and one thing only.

Misleading the people who use them as a metric.”

If a newsletter publisher sells ad space on CPM [cost per thousand] is the advertiser going to buy based on the 40mm records in the publisher’s “list” or the average 5% open rate? No matter how obtuse the buyer might be, they’re going to want an idea of how many people might see their ads.

Luke Glasner, principal, Glasner Consulting

John makes a good point here. For publishers or others doing ad supported newsletters. Opens or more specifically the Render Rate is the closest metric we have in email to ad impressions. Since most of the larger advertisers out there also use their own tracking saying we delivered your ad to 50k people is misleading. Telling them you got them 50k impressions and they won't pay for 50k impressions. You have to know your open rate to be able to forecast those impressions or you won't be able to deliver on your promises to advertisers.

Also if you are testing visual changes in, say, your buy-now button, restricting those test results to those that opened is a good control step for the test. If the person did not download images they did not see the creative changes such as imagery, color or other changes.

That's two ways the open rate and opens can be used effectively. They can also be used to mislead people, too. They are just another tool in the marketer’s tool belt. Anyone can buy a hammer, screw driver and nails, but it’s the skill in using them that sets apart a master carpenter from a kid build a tree house. Which would you want to hire to build your marketing home?

Dela Quist, CEO, Alchemy Worx

Hmmmm where to begin

1st of all I hope Ken does see all the comments, he is a pretty smart guy and I have no problem if he chooses to share Jordan’s (surprising) comments/joke maybe? about my competence as an email marketer or John Caldwell’s (unsurprising difference of opinion) with the wider world so be it. The one thing I do know is if Ken was as certain about the value of open rates as they seem to be he would NOT have posed the question to the list.

In a way I deserved it by making a slightly flippant and in hindsight too provocative comment on the fly (because I was in the middle of responding to Loren’s question (at the same time as running a $6m, 70 person, International EMAIL MARKETING agency). So rather than go to bed for a well-deserved rest after a long day and a very enjoyable OI meet-up here in Atlanta (Great to see you all folks) I will do the right thing and explain myself better.

My issue with open rates is and has always been that they are about the message and not the campaign or worse still the subscriber. For an industry that spends all its time talking about engagement and conversations that is unacceptable to me. Rather bizarrely my detractors also pride themselves as being master subscriber/consumer champions and engaging email conversationalists.

Let’s start with the subscriber: Assuming no list growth and no unsubs - you have 2 different weekly campaigns promoting identical products going to 2 different lists of 100k subscribers with identical properties, A has an average open rate of 50% and B 20%.

Now if you ignore the rate and focus on the subscriber you might find that after 3 months, 50k of list A subscribers have opened all 13 messages and 50k none. On the other hand all 100k subscribers to list B have opened a minimum of 2 messages. Which campaign would you say was the most engaging and most likely to generate the most revenue?

Now to the campaign: You send a weekly newsletter to the entire list 100k - open rate 15% and a weekly targeted message to the most engaged – a segment of 20k every week – open rate 20%; the less engaged get nothing. Is the open rate for the campaign 27.5% the average of averages [(13 X 15) +( 13 X 40)/26] or 15.8% the true average [(13 x 15000) + (13 x 4000)/156000]?

We use the true average for our clients, but how many people do you know calculate or have ESPs that make it easy to calculate the TRUE average open rate for a campaign i.e. Total Opens/Total Sent or Delivered for a month quarter or year? There is a huge tendency to calculate the average open rate of a campaign by summing all the open rates and dividing that number by the number of campaigns. WRONG and misleading

Now to subject lines.

Optimising for open rates tends to have a negative impact on clicks – we published a white paper describing this effect 5 years ago, since then our research has been replicated by MailerMailer and most recently in an excellent white paper by Adestra FREQUENCY: and what would a post by me be without a nod to that?
Sending more email WILL reduce your open RATE but it will also INCREASE the total number of people who open and click and transact as a result. As I have said many times before percentages are for the show, but numbers are for the dough – take your pick which you prefer.

Now to the publisher exception:

Publishing is a VERY different business model - how many of you reading this are publishers? and is generally an exception (as I have stated before never compare publishers/3rd party models with your own email program) to most email marketing rules and please note most of Moveable Ink/Jordan’s biggest customers are publishers; we have NO publishers as clients.

To John’s earlier post and I quote “If a newsletter publisher sells ad space on CPM is the advertiser going to buy based on the 40mm records in the publisher’s “list” or the average 5% open rate? No matter how obtuse the buyer might be, they’re going to want an idea of how many people might see their ads”. Now I started my career in advertising sales and spent 15 successful years selling ads in magazines newspapers and online before founding an EMAIL MARKETING agency and I can tell you this.

The reason media buyers focus on opens rather than list size, clicks rather than opens, leads rather than clicks and acquisition rather than leads, is because they are smarter than the publishers. The trick is to change that meme. If any publisher want to find out how to transform the way they sell ( or should I say give away) their inventory they can contact me offline. I might even tell them for free! If you are an email marketer working for a b2c brand don’t bother – I am a useless email marketer.

Jaffer Ali, CEO, PulseTV

I agree mostly with Dela. As a marketer, I view open rate more of a derivative metric. I have spoken about this with Dela privately when he was using a “cumulative open rate” metric for a while…and I thought that was derivative. The further away you get from the actual sale, the less value a metric has to the marketer. So click metrics are more valuable than open rate metrics… This may be a simplification, but we all need ways to simplify our lives…not complexify them.

I think in the end, people will measure open rate because they can. We use it for one purpose. If someone hasn’t opened for 6 months, we scrape them off the files for a different mailing cadence…different treatment. We do not pay ANY attention to daily open rates…do not optimize toward open rate. We think it is easy to optimize to the wrong metric and be distracted. Now, I am not sure anybody suggested that optimizing open rate should be a marketer’s goal. I have tried to get up to speed on the thread.

Publishers selling advertising have a different mindset completely. So the two types of email models should be distinguished…and Dela did a decent job of that.

Lastly, few people will be persuaded by any argument one way or the other. I have come to the conclusion that I have completely lost the ability to persuade anybody of anything. I think raising three sons convinced me of this unfortunate fact.

Jordan Cohen, vice president of marketing, Movable Ink

To set the record straight:
• First and foremost, my comments were not in response to Dela's, were not directed to Dela, were not about his competency, were not about him whatsover. I feel bad that he took my differing viewpoint on the utility of open rates (which many others on the list share) as some sort of personal affront on his competency as a result. They were not.
• This statement is inaccurate " most of Moveable Ink/Jordan’s biggest customers are publishers" - The majority of our customers are B2C marketers in retail, apparel, finance and travel:
• Per both clarifications, I had no dark agenda in posting my take on the utility of open rates, and certainly not a publisher driven one at all. If you review my post you'll see I actually addressed some of the limitations of the metric that marketers should account for when conducting campaign analysis (e.g., that in some instances they may want to dive deeper into open rates by reviewing read-length).
• There are widespread issues and inconsistencies with reporting in the industry across many metrics beyond the open rate -- it doesn't mean the metrics lack value, it just means that marketers need to ask questions about them and understand how and what they are measuring, and in relation to their business objectives.



Show: Newest | Oldest

Post a Comment
Your Name:
Please type the letters in the image above

Terms: Feel free to be as big a jerk as you want, but don't attack anyone other than me personally. And don't criticize people or companies other than me anonymously. Got something crappy to say? Say it under your real name. Anonymous potshots and personal attacks aimed at me, however, are fine.

Posted by: Kestrel Lemen, Bronto Software
Date: 2012-09-27 10:41:19
Subject: Can we look at the big picture here?

To preface, I work mostly with e-commerce retailers in the B2C space. So with that being said, Luke I like your reference to the open rate as being the 'just another tool in the marketers tool belt'. Let's face it the open rate is a piece of data, that isn't perfect, but it is a piece of data that we can utilize to form baselines, look for trends, and use to see a holistic view of an email marketing program. The key word here is holistic. No one piece of data is going to tell the whole story. Which is why we need to take a look at lots of data points to understand how a program is progressing. My clients always want to know 'what are industry standards for open rates' and while I understand their need to compare (as it's typically a top-down request to create business metrics). I make sure they understand how important it is to understand the strength of their own list and to develop their own baselines. At the end of the day, even if the metric is flawed, it's flawed consistently and we can form a basic understanding of how a program is performing by taking baselines (not just of open rates). Also, by using segmentation of inactive contacts I can show clients the value of engaged vs. unengaged contacts (yes, obviously purchase data plays into this as well). So at the end of the day, would I throw away a piece of data because I can't tell you it's 100% accurate, but it is consistent? Hell no.
Posted by: Dela
Date: 2012-09-26 13:16:00
Subject: Jordan & I are still friends :-)

when I wrote that I was unaware that Jordan had written his comment before he saw mine, our emails crossed in cyber -space. The coincidental timing made me assume his comment was aimed at me. Hence my tetchy response. I apologised, we kissed and made up and as often happens we remain in amiable disagreement