Marketing’s Weekly Dose of the Truth

Ken Magill

About Us

Bam! Kapow! Open-Rate Debate, Part Two


By Lots of People
Introduction by Ken Magill

Last week’s piece on the great open-rate debate was the single most popular article of the year for The Magill Report. It drew more readers than this newsletter has subscribers.

So apparently there’s an appetite for this information. And that’s good because there’s a lot more of it.

For those who didn’t see last week’s piece, the question was: “Do open rates have any place in the marketing-metrics arsenal?”

An “open” is recorded in email when the receiving machine calls for graphics from the sender. As such, the open rate is misnamed. Some marketers consider it useful. Some do not. And all have good reasons.

I put the question to Bill McCloskey’s online marketing discussion group Only Influencers to see what they had to say.

Not surprisingly, they had A LOT to say. They had so much to say, in fact, that I decided to publish the answers in two parts. And even in two parts, the pieces run long.

In any case, once again, we have a ton of informative opinions from some of the smartest marketers in the world. I encourage reading to the end.

Bill Kaplan, CEO, FreshAddress

While tracking Open rates is not an exact science due to the significant number of email clients and individuals that have images turned off, there’s certainly valuable information to be gleaned from Open rate statistics. While for most marketers, transactions and revenues are the most important data points to track, Open rate data will be highly correlated to these two critical measures of success, all things being equal.

The argument that Open rates are only useful if you’re sending the same thing over and over lacks any merit. Of course, if you’re sending the same thing over and over, then comparing Open rates across each campaign is a simpler exercise. However, that doesn’t discount the value of Open rate information if you’re analyzing rates across different types of campaigns - it just requires a little more work and perhaps some regression analysis to determine any valuable insights.

Finally, if visibility and branding are your end goals, then again Open rate information is valuable. If one deployment has an open rate of 50% and another deployment has an Open rate of 5%, it’s pretty easy to see which one generated more visibility.

Jeff Ginsberg, chief email officer, The eMail Company

I think Open Rates are a good metric to use to determine the effectiveness of a subject line.

Granted it's not perfect, but it is close.

(Insert angry debate here)

Christopher Donald, strategist, Inbox Group -- Indiemark

Okay so next time you do your metrics analysis, remove opens, don’t even look at them.

That’ll leave you with Clicks and Conversions.

Let’s say you sent 1-million emails, had a 4% click rate and a 2% conversion.

How did the creative/message do? How many people who saw the creative acted on it? You don’t know that number without a rendered open number.

If I see rendered opens at 25% and a click through of <2% then I know that my audience was engaged enough to open, the subject line was probably good, but the offer, creative, message didn’t do enough to move them to click. If I didn’t have the open rate, I couldn’t figure that out easily.

Derek Harding, CEO, Innovyx

- They aren't necessarily without any use, but I do think they're often over emphasized and, hence, misused.
- Open rates can be affected by a wide range of factors, some of which are entirely unrelated to the content.
- Unless you're comparing like with like comparative open rates can be misleading. But that's true in any comparative study -- you have to adjust for independent variables.
- Comparing open rates over time is often a mistake (unless you endeavor to correct for trends).
- Setting open rate goals may be ineffective (I know a company who killed an important list hygiene program because it was lowering their overall open rates).

Jose Cebrian, managing director, eastern region, Acxiom Digital Agency Services

I will recall the financial service company not to be named who had a marketing manager who kept putting in a subject line of “Important Account Information”. High open rate compared to other marketing managers in that company and in general for a PROMOTIONAL message, but also high opt-out rate. In that case the open rate was useless. Rates by their very nature can be manipulated. But if they can be correlated to higher sales then they are of value.

Or in the case of a welcome stream (not message) we generally see that open rates decrease by about half for each incremental message until they get to the norm for the program. In that case, the open rate is very useful as it helps us focus on getting the most important information into the first few messages and working to make that experience as great as possible to drive a brand promise (and lower call center calls, higher downloads, etc) for the program.

I would say that opens and open rates are useful to measure a complete program or type of message over time so you can see changes in the program. Many email marketers (and email tools) look at programs on a campaign by campaign basis. That too has usage, but for an open rate it is less important than watching it over time. A one time pop or decrease isn’t significant unless you know why it happened. It is important to understand if people are seeing your message more or less and what may be driving that – better targeting, changes in ISP behavior, list source, rise in the use of iPhone/iPad, better offers, etc

Nancy Weaver, manager, ebusiness, AAA Ohio

I use open rates as compared to other rates to determine what I need to do to improve my campaigns. For example, the open rate is a good measure of how compelling your subject line is. Then by comparing that to click rates, I can get a feel for how compelling the content is. If my subscribers opened, but did not click, then my content isn't up to par and I can go back to my marketing team & tell them to do better.

I do not, however, ever use the open rate by itself as a measure of success; it is always in context.

Doug Meierdiercks, senior manager, email marketing, Huddle

All the metrics are important and UNIQUE OPEN RATE is important. It says to me "This email address is engaged enough to open and look" - granted you get false positives, that can't be helped and it's why I completely ignore gross open rate. Two, it helps me determine the effectiveness of the subject line. Three - I like to look at the Unique Click-to- Unique Open Rate to help determine the effectiveness of my message (i.e. 7 of 10 people who opened also clicked on content).

Glad you asked as I'm currently building a new nurturing series and getting ready to test new subject lines to see if I can get a higher unique open rate (more eyeballs on message) and if I can maintain or increase CTR and Click to Open by testing some new downloadable content.

Karen Talavera, founder, Synchronicity Marketing

Lots of email doesn't directly drive revenue – it indirectly (and sometimes very loosely influences it -- and even attributing its influencing power is a problem. I've worked with several clients whose email never directly drives revenue (or if it does that can't be reliably tracked since they are non-e-commerce campaigns). Instead email is an essential part of their branding, customer relationship building, customer communication, and content marketing initiatives. A growth in email marketing/engagement may be able to be correlated in aggregate (say, over a quarter or half a year) with an uptick in revenue or new customers, but never tied together on a individual person-to-transaction basis.

So in cases where email is non-direct-revenue-driving and/or non-direct-revenue-attributable, I find open rates a worthwhile measure of subscriber enagagement, brand impressions, and sometimes (as John's blog post pointed out) pass-along and social influence. In these cases as well, next to click and unsubscribe rates opens are the only other direct action taken with an email which the marketer can measure. So of course they're not useless; technologically imperfect though they may be, they're a quite valuable in certain marketing contexts.

Loren McDonald, vice president of industry relations, Silverpop

The problem with open rates is not that they are inaccurate because of the way they are measured or that they don't tell you whether your email marketing program or message was successful or not. But rather that some marketers and executives use them as success metrics rather than a simple benchmark of subject line effectiveness and engagement over time. In the right hands and context, open rates can provide diagnostic insight on emails. In the wrong hands, however, they can inaccurately convey success or failure of a program.

John Caldwell, principal, Red Pill Email

The first thing one must understand is that a recognized sender with a compelling subject line causes a message to be opened. Nobody opens on just the sender alone. “So what I know you; what do you want?”

After a message has been opened relevant value proposition with a compelling call to action causes a link/button/whatever in the message to be clicked, and conversions happen on the landing page….

Knowing just those few things a good marketer should be able to see the value in the open metric if nothing else from an operational standpoint; in its simplest form the open metric will tell the marketer if they’re communicating the value of “opening” their messages.

Let’s say that you’re a publisher and sell ad space by CPM; what are you going to base your CPM on? It can’t be volume alone; the publisher wouldn’t need an active subscriber base, just a big one if it was all about list size, right? But if that publisher has a million subscribers and a 50% open rate then they have something that they can sell, right?

Let’s say that the marketer reports by domain. If they don’t see any opens to addresses that could be an indicator that their messages are being blocked or filtered by Yahoo. It might cause them to look at the next part of the equation and see if there have been any clicks from Yahoo addresses. No opens + no clicks = probable blocking….

I have a brand-name production client that sends significant email and quite often doesn’t get a lot of opens or clicks, but does see an up-tick in sales for about 36 hours after a promotional email has been sent. Not surprisingly the sales happen to be related to the subject line (this product, that discount, etc.); what does that mean? That the email was a waste of time? That no opens or clicks mean a failed campaign? Not hardly; they can attribute the sales to the email without an open or a click….

But we’re not talking about that, we’re talking about if there is any value in measuring open rates…. An open rate as a measurement has value if the person knows what they’re measuring for. If they’re just looking at it – or not - because some wrote about it on a blog somewhere said that they should – or shouldn’t – they’re missing the value or selling themselves short; you have to have a reason other than just checking off a box to say that you’ve done it….

Let’s say that you have a welcome program – you know; a series of welcome messages, not just an obligatory one…. There is no or minimal call to action in the message; its purpose is to set messaging expectations and to introduce the subscriber to the company/web site/whatever. Your first message gets, let’s say, a 67% open rate; your second message gets a 53% open rate, and the third message gets a 12% open rate, your new subscriber attrition has been about 0.02%.

Because you’re planning on jumping on the shiny “big data” bandwagon you decide to practice with small data that you already have and see that 80% of the people that of the people that subscribed in the first place have – or are – opening (and sometimes clicking on) other messages that you’re sending as part of their subscription and/or transactional messages.

What does that tell you? That maybe you don’t need a 3-deep Welcome program and that 2 messages in the Welcome series are enough and that the third message can be dropped from the stream….



Show: Newest | Oldest

Post a Comment
Your Name:
Please type the letters in the image above

Terms: Feel free to be as big a jerk as you want, but don't attack anyone other than me personally. And don't criticize people or companies other than me anonymously. Got something crappy to say? Say it under your real name. Anonymous potshots and personal attacks aimed at me, however, are fine.

Posted by: Jon Raney, ThomasNet News
Date: 2012-10-11 14:32:47
Subject: open rates

An interesting discussion with no simple answers. The problem with open rate (aka render rate) as has been mentioned is it's growing unreliability as a metric. The other problem for those in charge of reporting performance over time is that benchmarks die hard and if you report year of year month over month etc- unless you redesignate and inform people they will see the negative of the decline. This is apart from the click rate which also inevitably declines due to aggressive spam filters (heck your sign up confirmation email was filtered). The one thing I endeavor to show on my own is by compare the click bench mark analysis vs. the open bench mark analysis and how the open rate graph is skewed in comparison. After all you can't click something without having seen it. I think it's a reasonable assumption that your open rates should be proportional to your click rate over time. If your click rates are flat but your open rates are in the toilet it might be safe to assume that the issue is the metric itself. For me I've been forced to find some answers that work for me, and one is a formula recommended by my ESP, Xtenit company called the adjusted open rate (they just call it a read rate now). Without getting into the math it seek to restore the likely opens based on how many people clicked on it but don't show up on the open list. You take your overall ratio of open/clicks and project that new number for unrendered opens. Some may see this as fuzzy math. I don't. It does tend to work better if the sampling is large. very small samples could distort this. Litmus company also has an interesting pixel tracking campaigns that is able to capture more the issue of "reach" in particular how many people actually receive your email thru "organic forwards" (people that use their email client to forward not the built in forward) The one problem I have had with the CTOR metric (mentioned by Doug Meierdiercks)is that if you are reporting cTOR variances over time, a decline in open rate (the denominator in this case) could result in a positive variance which isn't quite the point. so you have to report both over open and delivered. For all the industry experts who do white papers on the subject, I find the info useful but it tends to be applied more towards direct marketing campaigns, it is less useful for newsletter news where the ROI is essentially whether people are interested in the information you are providing them. The "conversion" is a different animal and the incentives, etc that entice the user activity cannot be be compared. So everything is viewed with an * or caveat. btw my intentions of the company mentions was not to endorse, simply give people credit for something they (to my knowledge) originated.
Posted by: Bill McCloskey
Date: 2012-10-02 18:12:16
Subject: Only Influencers

Ken, Thanks for the mention. If others find this interesting, this is pretty typical of the discussions that go on every day on Only Influencers. I want to thank all the folks who contributed for being so open on a public forum. Bill
Posted by: Dude
Date: 2012-10-02 17:24:21
Subject: Read Rate

Panel Read Rates are the future ...