How we test fake sites on live traffic

A theory on why the &quot;ugly banking site&quot; did so well. Out of all of the designs, that was the most traditional, and probably most &quot;comfortable&quot; to the user. The site to me feels like it has depth, and it&#x27;s friendly to the user, encouraging them to explore. The others while more effective (conversion rate) felt like they were engineered to spoon feed me just the right data, and didn&#x27;t feel like &quot;complete&quot; sites open to browsing.<p>This of course is all anecdotal based on my years of being a web geek but sometimes the &quot;feel&quot; of the design speaks volumes where data and raw number crunching cannot.

&gt; The winning variation was Google Hover Clone<p>I was enjoying the article, but feel like it just abruptly ended. Why was the clone the winner? I would have loved to hear more about your process for judging the successful mockup.

I really enjoyed this article - the one question I have is if the &quot;winner&quot; has been rolled out to the masses (not only the SEM channel.) I&#x27;m a little scared for you guys because the test results show that the design works well for visitors from SEM, not necessarily from other channels. Have you tested this on traffic from other channels (organic, referral, etc)?

How long did you have the fake sites up? I would have thought you couldn&#x27;t leave them up for very long before frustrating users, but it seems like you would need a good chunk of data before you would know it was statistically significant. Or did you also have some sort of disclaimer for those users so they knew it was a beta test?

The keyword is &quot;Fake&quot;. The static HTML5 websites take much less time to produce thus reduce programming costs and shortens precious time-to-lauch.<p>In a nutshell: Moqups &gt; PSD &gt; PSD2HTML + Static data &gt; Fake sites

Getting user feedback on fake stuff is great. Tim Ferris didn&#x27;t know what to call the book that turned out to be <i>The 4 Hour Workweek</i>, so he spent money on AdWords to see which title more people clicked on. [1]<p>[1] <a href="http://futureperfectpublishing.com/2009/01/23/the-sticky-goodness-of-testing-book-titles-with-google-adwords/" rel="nofollow">http:&#x2F;&#x2F;futureperfectpublishing.com&#x2F;2009&#x2F;01&#x2F;23&#x2F;the-sticky-goo...</a>

Hover clone was the only one that put the locations on a map. This test seems to, more than anything else, have proven that when searching for office space _exact_ location is one of the primary criteria.<p>I wouldn&#x27;t be at all surprised if &quot;just show me where it is on a map&quot; is a feature most commercial real-estate search sites are missing.

You should really consider changing the title to something a bit more descriptive. Its a very good article about strategies for A&#x2F;B testing design ideas, but I didn&#x27;t get that from the title, and would it skipped it if not for the fact that I really enjoy 42floors blog posts.

Looking at a listing[1], I notice that you provide email addresses to users and presumably act like a craiglists for anonymous messaging...<p>I&#x27;m just curious though, don&#x27;t you find that you get spammed out if you do this?<p>I think providing email addresses to users on a marketplace is a great way of keeping discussions&#x2F;messages boxed up in your platform, but showing the emails on the site must result in way too many spam emails?<p>[1] : <a href="http://42floors.com/ny/new-york/25-w-39th-st/3171" rel="nofollow">http:&#x2F;&#x2F;42floors.com&#x2F;ny&#x2F;new-york&#x2F;25-w-39th-st&#x2F;3171</a>

The hovering map seems to be behaving oddly (Chrome&#x2F;Mac OS) for me and blocks off a large section of the top left with a beige colored background when I click on &quot;More Map&quot;.<p>Otherwise, very surprising to me that this was the winner. Did you see similar results across all browsers and form factors (especially interested in desktop vs. tablet).

Very interesting, thanks for sharing! What specific actions did you measure to determine a winner?<p><i>Noticed you answered the question above. </i>*Was just clawing my eyes out browsing craigslist for NYC spaces, remember hearing about you guys a little while back, thanks for refreshing my memory ;)

I kinda liked the map more. I thought moving to this new hover layout was a business decision (e.g: get some clients to pay for being featured on top of the list as the map makes all listings have equal look&#x2F;chance to be discovered). apparently you were just A&#x2F;B testing.

I&#x27;m not affiliated with the company, but Visual Website Optimizer offers the ability to do A&#x2F;B tests between different URLs - rather than just variations within a page (which it also does).

So didn&#x27;t users get pissed when they landed on the fake site?

The hover version exposed the most information in a usable and visually-understandable format. I am not surprised.

What is this gem called? I&#x27;ve seen it on boatbound and MyTime, now on 42floors. Same Rails based layout.

what did it mean &#x27;fake&#x27;? Weren&#x27;t all 8 fully working versions of the website?

Isn&#x27;t there a worry that &quot;fake&quot; sights would damage your reputation?