Developing an observational hypothesis – Sometimes the best landing page test is when you do nothing at all

Once you start testing you get caught up with the testing bug, in fact you start to think of all the different areas you could test and sometimes this isn’t the best use of your time. After all if you are numbers led marketers (which by default you should be) then you should understand the life cycle of your test campaign and have a clear and defined goal. Sure you will stray off path as sometimes a test can open a new area you hadn’t previously thought of but that is often a good thing. I’m not saying you should test 41 shades of blue but if that’s an important part of your hypothesis then go for it.

Now the title is not suggesting that you kick back and let the tide of your website visitors wash over you but it is saying that the calm before the storm should be to sit and observe. For me this is one of the most exciting parts of the testing cycle, you get to view your visitors in their natural habitat whilst they go about their tasks and achieving (or trying to) their objectives.

I have been involved in tests where the designs were created out of a combination of what the in-house team thought were best practise and anecdotal feedback from users and work colleagues. Now don’t get me wrong this is often the catalyst for most in-house testing but with the wealth of data we have available at our fingertips we should be looking to develop a more robust testing hypothesis (but what better way to introduce landing page testing to your company by introducing an hypothesis and having someone disagree with it thus having to run a test to see who was right!).

Below is an example of some ways how you could go about doing this using analytics;

Website Analytics – often overlooked and rarely understood but a well configured analytics package is your lifeline to understanding how your website functions. BUT, it often only shows 2 dimensional data which is hard to draw conclusion from for landing page testing, where it can help though is;

• Understanding which pages to test – pull reports on high impact pages and ones that are at the beginning of your user’s journey (top landing pages). The higher the volume of traffic the quicker the test, the higher the revenue generated from that entry point the higher the return will be if the conversion rate is positively affected.
• Segmentation – you should already be segmenting all the traffic sources that come to your site, sure an aggregate view is good for the CEO but what if a significant percentage of the traffic source is from a one off campaign your running  or an email campaign drives traffic in a seasonal peak? Without this understanding you could find further testing a headache as you will need to start again, you can also gain marketing channel specific feedback if say one of your traffic sources has a disproportionately higher bounce rate than the others. Also be wary of new versus returning visitors as their motives are often different.
• Geo targeting – if your website is global (or targets more than one country) then make sure you understand where and how they are accessing your pages as any changes could be more detrimental than you think.
• Site overlay – can be very useful to understand how your segments are interacting with your page, the dynamics of this is important as it’s the landing to goal interaction ratio that you are looking to improve. Each page should have a goal, part of any micro conversion process is to break the main goal down into smaller goals thus making it easier to analyse and interpret the data.
• Entrance paths – these are very important, if many of your segments are visiting a second page before returning to the entrance page and interacting with your goal objective then you have already observed an area that could be improved
• Browsers – what is this 1998 I hear you say! No but in all seriousness what works in one browser won’t necessarily work in others, a simple changing of a button placement or movement of an image might seem ok in IE7 but if you haven’t tested it thoroughly you are leaving yourself open. Pull a report by browser, by page and check for bounce rates and average time on site, If anything is above the website average investigate it.

Ok so there are a few areas you should be reviewing whilst developing your observational hypothesis, also these are metrics that you should have readily available to you. If you want to dive deeper then there are other qualitative tools our there which will help further and which we’ll discuss in another post.