Marketing’s Weekly Dose of the Truth

Ken Magill

About Us

One More Take On Subject-Line Length

5/26/15
 
By Ken Magill
 
After I ran a piece on Return Path’s study on subject-line length claiming there is no correlation between length and performance, direct-marketing copywriting expert Bob Bly sent me a note calling BS.
 
“Their study is wrong,” he wrote. “Pinpointe, with whom I work, tracks millions of emails and we know for an absolute certainty that shorter subject lines (35 characters or less) pull best. We have tested it to a fare thee well.”
 
So I contacted Craig Stouffer, general manager at email marketing software provider Pinpointe On-Demand. He gave me permission to run his response here.
 
Here is what he had to say:
 
We reviewed subject line tests results from customers and from our own data - We do not have a customer report comparing data as Return Path did.
 
However - I'd say you and Bob are comparing apples and oranges.
 
Report says - "Taking a random sampling of a few billion emails – there’s no correlation between # characters and read rate."
 
Might be true - but what it doesn't say is:
What's the correlation between line length and engagement rate? [clicks for example]
 
If a user tests several subject lines, which will win - shorter or longer? That's a different question.
 
Bob is an experienced copywriter. He tests stuff. 
 
Bob's experience which is consistent with ours and with customer data is - if you test a set of subject lines to optimize results, shorter subject lines usually win. Seems inconsistent with the report. But:
 
Bob takes an un optimized subject from a customer. Likely that the subject isn't as on-point as it could be.
 
Then Bob optimizes and tests. Clarity often means fewer words.
 
Bob's versions of optimized subject lines will outperform the initial version. 
 
The edited version is almost always shorter.
 
This is 100% consistent with our internal tests and the results from multi-tests that customers have run.
 
On the other hand - we've also seen the following:
 
Customer starts with a well written, clear subject line that is 'long'.
 
Customer attempts to optimize by simply shortening the subject - and in doing so, the message becomes more vague, possibly off-point.
 
Result: The initial [longer] subject usually wins.
 
The takeaway in our book is:
 
Even in light of the RP data, it is always a good idea to test content and subjects with your audience.
 
Testing subject lines and the lead-in sentence is important.
 
The objective is not to get to the shortest subject line but rather to be as specific as possible in as few words as possible.
 
Hope that helps,
 
PS - I've seen recent articles drawing the same conclusion about 'reducing the spam score in your email' -- claiming that there is no correlation between reducing your spam score and campaign results.  That's completely opposite of our findings.
 
Comments

Show: Newest | Oldest

Post a Comment
Your Name:
Subject:
Comments:
Verification:
Please type the letters in the image above

Terms: Feel free to be as big a jerk as you want, but don't attack anyone other than me personally. And don't criticize people or companies other than me anonymously. Got something crappy to say? Say it under your real name. Anonymous potshots and personal attacks aimed at me, however, are fine.

Posted by: Dave
Date: 2015-06-03 16:59:22
Subject: In addtion..

How to Lie with Statistics - by Darrell Huff I get it -- a lot of marketers aren't mathematicians nor statisticians, but this is stats 101 stuff right here. Know the source of data that was analyzed, know the analysis method, try to identify other contributing factors that were not / could not be tested on. I'm in the boat with Parry -- these results we've been fed don't tell the general population much of anything.
Posted by: Parry Malm
Date: 2015-05-29 11:36:30
Subject: You cannot test subject line length

Hi Ken So, this is a never-ending argument but both points above are incorrect for one simple reason: YOU CAN'T TEST SUBJECT LINE LENGTH. For example let's take two lines: "Sale on now - save up to 50% off on sofas and beds" "Sale on now - save up to 50% off" Sure, the first is longer and the second is shorter. BUT THAT'S NOT NECESSARILY THE MAIN CAUSAL INFLUENCE. If people are testing long vs short, they are not testing long vs short. They are testing an entire chunk of information. This is an important point. Correlation IS NOT causality, and both quoted people above have confused this point. I covered this off on a blog post a while ago. http://phrasee.co/the-final-word-on-email-subject-line-length/ The point is this - anyone who tests subject line length is not actually testing subject line length. If they say they are, they don't understand how to design an experiment properly.

Xverify