Better Product Managers, and Product Management

A couple weeks ago, I wanted to make some visual changes to this blog to see if I improve upon some basic engagement metrics – time spent on site, number of entries read, repeat visits, etc.  And the counsel that I frequently give to others immediately intoned:

THOU SHALT NOT MAKE CHANGES WITHOUT A/B TESTING.

OK. But this isn’t as straightforward as it sounds.   Google Web Optimizer will not help you test your WordPress blog.

• GWO is focused on single-action conversions, such as completing a signup or purchase.  I wanted to compare a series of metrics.
• It’s also not ideal for a database-driven site that uses common headers and footers.
• Around 25% of my traffic hits my homepage directly, but WordPress doesn’t have a simple way to create two homepages.
• I tweet a lot of single-article links, which means that every page needed an alternate.  But I wanted to avoid being penalized by Google for having duplicate content.
• I realized there was a lot of value in permanently having an “A” and “B” site so I could always be testing something.

So I decided to roll my own solution. It’s not particularly elegant.  But I’ve laid it out step-by-step so that anyone else with a self-hosted WordPress blog can try this themselves.

(And maybe, someone with better programming chops than myself will write a more streamlined plug-in version and make it available to the blogging community.)

You will need:

• A self-hosted WordPress blog (I don’t think this will work for a *.wordpress.com blog unless you can directly edit your header.php and footer.php files)
• Ability to back up your WordPress database (if something I wrote kills your site, I want you to be able to retrieve it)
• Ability to create new directories and edit files and permissions on your server
• Two Google Analytics accounts (one for the “A” site and one for the “B” site)
• Willingness to do some messy WordPress hacking

Pick what you want to test

You want to test a single variable (if you make multiple changes, you won’t know which one is driving changes!)  A few suggestions:

• Two different WordPress themes
• Minor CSS/color scheme changes
• Front page settings (i.e. showing 10 posts on homepage vs. showing 5 posts on homepage)
• Two different sidebar designs (that’s what this blog is testing)

Creating the “B” blog:

Step 1: Back Up

First back up your WordPress database.

Step 2: Manually Install “B” blog:

Create a sub-directory called bblog where your “B” blog will live.

Copy your entire original wordpress/ directory into the bblog/ directory.   This is effectively doing a manual install of WordPress for the “B” blog.  This ensures that all of your settings, themes, and plugins get copied over.

Step 3: Set Up “B” blog database:

Now you need to set up the MySQL database that will power the “B” blog.

You can run both copies of your blog off the same MySQL database. To do so, open up bblog/wp-config.php and change the $table_prefix from wp_ to wp2_ :

Step 4: Complete installation of your “B” blog:

Access your new WordPress dashboard by going to http://www.yourblogname.com/bblog/wp-admin/.  You will see the WordPress setup screen:

Enter your blog title and email address (both should be the same as for your original blog).

You may wish to uncheck the “Allow my blog to appear in search engines like Google and Technorati” checkbox while you’re setting this up.  You can always check it again later.

Click “Install WordPress” and write down your new Admin password.  You’ve installed the “B” blog!

Configuring the “B” blog

Step 5: Import your original blog entries to your “B” blog:

If you’ve logged into your “B” blog, you might be thinking: Where are my posts?!

Don’t worry – importing your posts is quite simple.

Open your original blog’s WordPress dashboard in a new browser tab.  Go to the Tools menu and select Export and click Download Export File. Your blog entries should immediately download as an XML file.

Now open your “B” blog’s WordPress dashboard.  Go to the Tools menu and select Import.  Choose WordPress as the import type and you’ll be prompted to upload a file.  (Note: WordPress says “Upload your WordPress eXtended RSS (WXR) file”, but the file is an XML file.  Just upload the XML file you just exported and it will work.)

Upload all authors and import your attachments as well and click Submit.

Step 6: Prevent duplicate content problems by adding canonical links:

Download the Canonical-B plug-in (this is a hacky modified version of Joost de Valk’s excellent Canonical plug-in.  It tells Google that the “real” link for the pages of your site is http://www.yourblogname.com/ not http://www.yourblogname.com/bblog/)

Add the Canonical-B plug-in to the plug-ins directory for your “B” blog.

(Also: I recommend you subscribe to Joost de Valk‘s awesome WordPress newsletter. Full of useful tips.)

Step 7: Finish configuring your “B” blog:

Go to the Appearances menu to choose the same theme that you are using on your original blog.  (Your “B” blog defaults to using the Default WordPress theme. )

Go to the Plugins menu to activate plugins for this new blog.  (You should see the newly-added Canonical-B plug-in along with whatever other plug-ins you had previously installed on your original blog.)

Go to the Users menu to replicate whatever users you had on your original blog.  (WordPress automatically creates a user called “admin”, but you may have created another user.  For example, I post as the “Cindy” user.)

Step 8: Update Google Analytics for your “B” blog:

If you haven’t done so already, log in to your Google Analytics account and create a new account (from the “Analytics Settings” page, click on the + Add New Account link and create a new account for http://www.yourblogname.com/bblog/).  Copy the javascript code snippet.

I’m assuming you put Google Analytics code in the wp-content/themes/THEME_NAME/footer.php file of your original blog.

So you’ll want to open up your bblog/wp-content/themes/THEME_NAME/footer.php file and replace the original Google Analytics code with the new code you just copied.

Now Google Analytics will separately track metrics for http://www.yourblogname.com and http://www.yourblogname.com/bblog.

Step 9: Set up the test differences for your “B” blog

Your “B” blog installation is set to the WordPress Default theme.  You’ll want it to change it so that it matches your original blog, except for the changes you are making for your test conditions.

Click through several posts on your “B” blog to ensure that it displays the way you want.  Once it looks good, now you’re ready to direct part of your traffic there!

Setting up your A/B Traffic Split

Step 10:  Set up canonical links and get the “B” versions of your links:

Download the Canonical-A plug-in (this is a second hacky modified version of Joost de Valk’s excellent Canonical plug-in.  It puts the canonical link in the header of your original blog, and gives you a variable for the “B” blog version.

Add the Canonical-A plug-in to the plug-ins directory for your original blog.

Step 11: Add the redirect script to your header to send X% of your traffic to the “B” versions of your blog:

Open wp-content/themes/THEME_NAME/header.php and add the following lines to the beginning of the file:

<?php

$url = yoast_guess_url($wp_query);

$bblogurl = str_replace('.com/', '.com/bblog/', $url);

$number=rand(1,100);

if ($number>=50)

  echo ' ';

else

  echo header('Location:' . $bblogurl);
?>

These lines must be at the very beginning of the file – no blank spaces or lines or code can come first.

This code will do a 50-50 split, sending half of your traffic to http://www.yourblogname.com/ and half to http://www.yourblogname.com/bblog/.

It also works for deep links, sending half your traffic to http://www.blogname.com/category/this-is-your-post and half to http://www.blogname.com/bblog/category/this-is-your-post.

If you want to test with a smaller percentage of your audience, change the line if ($number>=50). For example, to send only 10% of traffic to your “B” blog, you would change it to if ($number>=10).

You should not edit your “B” blog header.php.

Launch it! (And Measure)

Step 12: Begin the experiment:

Once you’ve saved this header.php change, your experiment has begun!

From now on, whether visitors come to your homepage or follow a deep-link, 50% of them will be sent to the “B” version.

(If you want to immediately stop the A/B test, just delete the code you added in Step 11 from your header.php file.)

Step 13: Some Constraints

Note that this 50-50 split happens each time a user loads the header – in other words, the user who visits multiple pages may be flip-flopped from “A” to “B” to “A” and so on.

This is not optimal, but since many blog readers aren’t reading multiple pages, I decided to go ahead and launch like this.  (I’m working on a cookied version that will maintain the “A” or “B” setting for an entire session – stay tuned.)

Also: the two blogs will not stay in sync.  After the initial export/import, you will need to publish new posts to your original blog, and then manually copy them over to the “B” blog.  Since I only post once a week, this is not too much overhead.

Step 14: Measuring

Once you’ve had both versions running for a few days, you can log into Google Analytics and compare the two accounts (be sure to set them to the same time period).   I’ve done this by opening the cindyalvarez-a and cindyalvarez-b reports in two browser tabs and flipping back and forth – crude, but it works.

Hope you enjoyed this and are able to put it to use.  Not everyone has access to an e-commerce site or a website funnel, but many of us have personal blogs and this is a good opportunity to dive into the details of your blog and try out A/B testing.

Popularity: 5% [?]

Popularity: 5% [?] blog comments powered by Disqus