Many of our readers have already signed up to the live seminar on split testing that we announced last week.

But even if you can’t make it, you’re probably interested in learning more about split testing now, right?

Fortunately, we happen to have a case study on hand that shows just the sort of information you can learn about your email marketing campaigns by conducting split tests.

Today, let’s look at a split test that we ran on our own blog newsletter to get more of you to come to the site and read the latest posts.

The Test

Last year, Marc and I were discussing how to increase clickthroughs on the emails we send to our blog subscribers.

One of the ideas that came up was to replace the text links that we had been using to drive people to the blog with a “button.”

Previous testing on the website had shown that in many cases, buttons make better calls to action than text links do. We thought the same might hold true for email.

So, Marc created a button-shaped image with the words “Read More” stamped on it:

We then created A/B split tests for our Blog Broadcasts, inserted this image into one version as the call to action (to read the full post on our blog) and continued to use text links in the other version as we had before.

The emails were otherwise identical — we kept subject lines, sending dates/times and templates the same for each version.

Measuring Our Success

Since we’re trying to get people to visit our site to read the full blog posts, we want to compare clickthrough rates.

We chose to use clicks-to-opens as our measure of success. Instead of dividing clickthroughs by the number of emails sent, we divided it by the number of emails opened.

That way, if one version of the message got an unusually high number of opens, it wouldn’t skew the results to make that version’s call to action look more effective than it really was.

Our Expectations

Going into this test, we expected the button to beat the text links handily.

Why?

It was physically larger than the text link

It contained a clear call to action — “Read More” — while a contextual link might be less obvious.

It was an image placed in a part of the email where readers hadn’t previously been shown images

Basically, we expected the button would grab people’s attention as they scanned through the email.

On the flipside, we knew that readers might have images disabled and wouldn’t see the button.

So we added the ALT text “Read More” to the button image.

Since with images disabled the text “Read More” would appear in place of the button, we felt that even for those readers with images disabled, the button should do at least approximately as well as the text link.

Initial Results

We started running the test on our Blog Broadcasts last year.

As we expected, the button grabbed readers’ attention and incented them to click through, much better than the text link did.

Clicks-to-opens for the button was repeatedly higher — a lot higher — than it was for the text link.

In our first five split tests, the button drew a clicks-to-opens rate that was on average 33.29% higher than the text link clicks-to-opens rate.

At this point, about 2 weeks into our test, it was tempting to say, “The button clearly draws more attention and clicks than text links. Let’s just start using buttons and move on to another test.”

…But, We Kept Going!

We could have stopped after those first few tests — and in many cases, one-time or short-time split tests are appropriate.

However, even in our initial few tests, the text had beaten the button once, and by a large margin.

I wanted to see whether the button was doing better because it was a more compelling call to action in general, or because of the “novelty” of it.

So we continued to split test our Blog Broadcasts…

Later Results

We ultimately ran the button-versus-text split test about 40 times, over the course of several months.

For a while, the button continued to beat the text links — but we noticed that it wasn’t doing so by as large a margin as it first had.

While over our first five tests, the button beat the text by over 33%, after 20 tests it was only winning by an average of 17.29%, and the text version was beginning to hold its own in the win column.

Final Outcome

With each new split test, the text asserted itself as the better call to action.

By the time we ended our experiment, text links were consistently outperforming our button, winning nearly two-thirds of the time, by double-digit margins as high as nearly 35%.

Conclusions: What Happened?

The button is effective in the short run, but after a while readers become “numb” to it and no longer respond at the same initial high rate.

Consider the following two stats from our split tests:

• Overall, text links outperformed buttons 53% of the time.
• After an initial period where the button was more effective, text links outperformed buttons 67% of the time.

That first stat doesn’t seem important in and of itself — 53% is barely over half the time.

However, for the first three weeks of the experiment, the button won two-thirds of our split tests. After that, the opposite became true — the button just stopped “working.”

Which brings us to conclusion #2:

Test results are not forever.

What works today may not work tomorrow.

Had we stopped our testing after one broadcast, or even one or two weeks, we would have concluded that buttons were better than text links.

It’s important to continually test your email campaigns to make sure that you know what works, rather than assuming you know what works.

Finally, one last point I feel obligated to make:

What works for someone else may not work for you.

The text links won out in our split test, but that doesn’t mean a button can’t be an effective call to action for you.

We send our blog newsletter often — 2 or 3 times a week. So we exposed subscribers to the button often, which may have increased the speed with which they started to ignore it.

If you send less frequently, or only use a button for emails where you have a particularly compelling offer, you may find it to be effective.

Plus, we tested a specific button. Perhaps another one, with different design or wording, may have been more effective.

Again, don’t just take our word for it. Find out for yourself through your own testing.

Learn More About Split Testing

Unfamiliar with split testing? Want to see how you can methodically raise response rates for your email marketing campaigns?

Join us for a free one-hour seminar on split testing tomorrow — Wednesday, March 26th:

Split Testing Emails and Web Forms

Wednesday, March 26, 2008

2:00 – 3:00PM ET

Convert to Your Time (New Window)

What Do You Think?

What are your thoughts on this case study?

Have you tested calls to action or other elements of your email newsletters? What were your findings?

Can you think of any other factors that may have influenced our results?

Share your reactions on the blog!

MarketingSherpa announced their 2008 Email Award winners recently.

These are always a great place to look for innovative ideas as well as proven/tested tactics that you can use to improve your own email marketing campaigns.

Out of all the award-winning emails they present (and there are a lot!), I was particularly drawn to the best automated/autoresponder series category.

Let’s take a look at what’s working well for companies running autoresponder series…

Successful Autoresponder Campaign Trends

In reading through MarketingSherpa’s writeup of each winner and looking at the included samples, I noticed a few patterns among the successful campaigns:

Simple Design

Simplicity was a recurring theme in many of the successful campaigns Sherpa profiled, whether on landing pages or in the emails themselves.

GMAC Mortgage had a particularly compelling design that, while simple (it looked to me like a personal letter sent by postal mail on company stationery) incorporated some nice, simple design elements such as their logo and a picture of the mortgage agent. A great example of “Light HTML.”

Short or Long Copy Can Work

Sherpa’s winners included both shorter, more image-heavy emails and longer, text-focused ones.

It seemed to me that short copy with more imagery worked especially well for direct promotions, while longer, text-heavy emails were more suited to educating and nurturing leads.

Freebies

Giving something away is still a great way to attract and engage subscribers.

Examples of winning uses of freebies included:

“Starter kit” with a series of whitepapers and a case study

Free trial/demo of a software

Sweepstakes entry in exchange for customer feedback on a recent purchase

Include Site Navigation In Emails

Many, if not most, of the winners included some sort of site navigation in the emails, such as links to a homepage, “My Account”/login page or useful tools on their sites.

This can help to raise response because even if someone isn’t interested in your primary call to action, s/he may still be interested in visiting your site for something else (example: I may not want to purchase a product yet, but I might want to read your blog or use your pricing calculator).

It also can help to build recognition of your emails by including design elements that readers will remember from your website.

Try Using Buttons For Your Calls To Action

Several of the winners used image-based “buttons” for their calls to action.

Those with longer, more text-heavy emails also included text links to supplement the button, rather than taking an “either/or” approach to their calls to action.

Preview Panes: Give People Something To Do

As Sherpa’s winners showed us, putting some kind of action near the top of your emails can yield positive results.

Examples of this include:

Site navigation links

Link to an online version of your email

Your primary call to action

A secondary call to action, such as driving the user to a helpful resource on your site.

See the Winning Campaigns For Yourself

To see all the results, head over to MarketingSherpa’s site.

The success of our email marketing campaigns does not rely entirely on the success of our email.

What I mean is, if the goal of our email is not only to drive traffic to our websites but ultimately to convert that traffic into sales, our websites need to do their jobs too.

Email tracking and analytics let us know how our emails are performing. In order to make sure our marketing campaigns are performing as we’d hope as a whole, we’ll also want to take a look at the web pages we send our subscribers to.

Make Sure Your Web Pages Work, Not Just the Links

Before we send our email messages, we should always test them. In doing so, one of the things we’re looking for is working links.

This entails clicking on each link in a message to make sure the correct web page opens properly in a browser. If they do, we tend to close the browser window and move on to the next step in our sending routines.

But wait! Here are 3 other things we should look for before we close out the window:

One or More Clear Calls to Action

Dead ends are always bad when it comes to conversions. Do your emails’ landing pages clearly prioritize where what the subscriber should do once they click through and browse through the first page they see? Or, do they have to really think through what options there are for action?

Something Valuable the Email Didn’t Provide

There’s absolutely nothing wrong with re-purposing content from our websites for email. But if our emails link to pages that don’t really provide anything new and valuable, our subscribers are more likely to close out the window than do anything else on the site..

Analytics Tracking

OK, so this is something subscribers won’t actually see, but if you’ve read to the third tip in this article, you’re clearly interested in the performance of your web pages. The best way to get actionable insight is to install an analytics software on your website.

If you have one and have not yet integrated it with your email marketing campaign, take the small amount of time it takes and do so.

More In-Depth Tips on Landing Page Optimization

Our web pages are an integral part of our email campaigns, even though they’re found in the web browser and not the inbox. They’re something we shouldn’t forget while we’re working on our campaigns.

For more tips on optimizing them, check out some of the free resources over at Marketing Experiments Journal. On their site you’ll find archived reports and opportunities to sit in on live seminars with experts on the topic.

Videos are a great marketing tool — they your prospects’ attention, they let you show (not just tell) about your products, and they get passed around (helping you to get more subscribers virally).

However, as many people have found out, creating "video emails" by embedding the video directly in an HTML email (like you would on a web page) doesn’t fly. It’s simply not reliable because most email programs disable or strip out the video.

But that doesn’t mean you can’t take advantage of video’s appeal in your emails!

You just have to get a bit creative…

Easily Send "Video Emails" Using Image Links

In a recent addition to our Knowledge Base, we’ve shown step-by-step how to drive your email subscribers to watch videos that you’ve posted to your site or video hosting sites like YouTube.

How Can I Add Video To My Messages?

A Couple Pointers

I recommend using an image to link to your video — you can use text, too, but the image will naturally draw the eye and increase your response.

That said, not all of your subscribers will have images enabled in their email programs.

So, remember to put appropriate ALT text for your image, so that if they have images turned off, they know there’s a video to click to.

Example:

Have You Used This Tactic?

Has linking to videos this way worked well for you? Have you learned anything along the way that you think our readers could benefit from when creating their own video emails?

Share your thoughts!

RSS Subscribers: Here’s a permalink to pass along to others you think would benefit from this tip!

Interesting conversation going on over at Copyblogger where Brian Clark is talking about calls to action.

He points to a MarketingSherpa test that suggests that using the word “click” — as in “click here” or “click to continue” — can boost response.

You can make a lot of arguments for or against using that specific wording (and people appear to be doing so in the comments).

But what stood out to me as the real lesson is…

Click to Find Out!