Insights in Optimisation: Q&A with Secret Escapes

Case study

Secret Escapes are doing something very right. A Forbes report on Secret Escapes tells me:

“[In the UK] the firm has seven million people signed up and 10 million worldwide. […] There are over 200 employees worldwide, and since launching in 2011 has registered 260 percent year-over-year growth.”

I got a chance to gain some insight into the optimisation strategy at Secret Escapes. Product owners, Thomas Evans and Rohit Gupta, kindly shared their time, knowledge and experiences with me.

Check out the interview below:

The team

Tell me a little about when and how you joined Secret Escapes?

Thomas Evans (TE):

“I joined Secret Escapes in 2011 as a web designer and front end developer (HTML & CSS).”

Rohit Gupta (RG):

“I’ve been at Secret Escapes about 10 months. I previously worked in sales [at a different company], pretty much taking the first job I could after University. But while I was there, a position was advertised internally for a product owner. One of the managers suggested that I apply.”

Secret Escapes customer on a sun lounger

How does the product team differ from other teams you’ve worked in?

RG:

“Most people land in product, it’s not a job of you dream of as a kid. There’s similarities between people who work on product. I’m fairly comfortable with numbers. I like being critical, analytical but I can also engage with the fluffy side.”

How is the team structured?

RG:

“Marianna (O’Hagan) heads up the team. Then there are the product managers. Everyone does a bit of everything but if you want to break it down to who has ownership of a particular area, I’m more on qualitative testing and Tom is more quantitative. We don’t tend to talk about it in that way though.

We each know what other people on the team are doing because we work closely. It’s divided by projects rather than products. Our similarities are that we’re all driven by metrics.”

Are there plans to grow the optimisation team?

TE:

“There’s no separate optimisation team as such, the whole product and business is involved in optimisation”

Optimisation culture

Has optimisation and testing always been Secret Escapes’ method?

TE:

“Yeah, when Secret Escapes started, optimisation was there from day one. We test and iterate, that’s how Secret Escapes has always worked. Almost everyone here has read the lean startup, and buys into many of those principles.

In the beginning we tested with a free A/B testing tool at the start, it was Google Content Experiments.”

How important would you say testing and optimisation are to future growth?

TE:

“Sometimes, some companies have trouble getting buy-in from senior management but Tom Valentine (COO) was already pro-optimisation. He led much of the early A/B testing, has always been a huge supporter of on-site optimisation.”

Signup page on Secret Escapes

Secret Escapes’ signup page circa 2011

Secret Escapes homepage and signup form

Secret Escapes’ signup page today after many iterations

Tom, you’ve published Secret Escapes’ CRO Stack (a list of the optimisation tools used by the team).

I’ve never heard of GoSquared before – is Secret Escapes as open to using small, maybe unheard of tools as much as the big ones like Optimizely?

TE:

“GoSquared aren’t new (they’ve been around since 2006) but I guess they are small. We’ll go with whatever is right. Whatever helps our team do their job better. Staff can bring tools to the team and we’ll trial them.

So with GoSquared, it tells us about problems quickly. GA (Google Analytics) is our primary tool but GoSquared is real time so can we find out how something is performing much quicker.
They also have a product called People Analytics. We are not yet using it. We tried it out but the tool is in its early days and we didn’t get that much from it. Maybe we’ll revisit it again in the future.”

So anything goes in terms of what tools you’ll use?

TE:

“If we spot something that we think will help us then we’ll try it out. We’ve been with Formisimo for over a year now but I think we were one of your biggest clients when we came on board (Editor: True) and we were happy to trial it and have been using it ever since.”

Tools and testing

So your list of tools on the CRO stack was true as of March this year. Are you ever hunting for new tools? Would you say you’ve got a plan to try to find tools over the next 12 months?

TE:

“If we see anything that we predict will shift the needle we’ll try it. We trial tools at the same time as trying new things on website”

What’s been the most surprising or unexpected result of testing?

TE:

“The most surprising thing is when we run a drastic test and it doesn’t see change –

It’s like mountains – for all you know you’re standing on top of the mountain but you need to make a big leap to get to the next peak.”

Sounds like a lot of prep work for every change you want to test.

TE:

“Not always. We do significant research and analysis before the big tests. Smaller tests are where we can try things out, ideas that we think will work.”

What’s been the most useful optimisation tool or approach for you guys? You’ve published a lot of the results you’ve got from your Optimizely tests…

TE:

“Optimizely is crucial to our A/B tests but, and I get laughed at in the office for saying this, Google sheets are one of my most useful and used tools. I use them for everything; whether to track tests or maintain a backend for a large test. They’re the most versatile thing I use and I do get laughed at for saying that.

Google sheets are so useful because they allow us to do things cheaply. We use Google forms to gather feedback from the website. It allows us to start off very cheaply – something we’ve stuck to from the beginning.

We try to work more on the front end than back end.”

A draft of a Secret Escapes' product page

A draft of a Secret Escapes’ product page

Qualitative data

Do you have an in-house testing lab?

RG:

“No, we don’t have a testing lab. We work on numbers first. We’re very much data led and we find that that tells us where somethings not right. We can then investigate further by going on to qual (qualitative testing).

We actually have a great customer service chain and we make the most of the feedback that they get. The sales team also pass on hotel feedback. It’s a great chain from customer to support to product team.

I’d like to do lab testing but time is a big factor for us. We actually watch back session recordings (screen recordings of users on the site) and those offer us a lot of insight.”

What do you think of remote user testing?

RG:

“There’s pros and cons. One benefit is that you’re not influencing the user when you’re doing remote testing or watching sessions back. A drawback is that you can’t ask follow up questions.”

Competitive edge

What makes Secret Escapes competitive from your perspective as product owner?

TE:

“We’re different primarily because we’re members only. Prices hidden and people have to actively join the mailing list.

All the hotels are handpicked. We perform price checking and find the right fit for the site, then add handwritten copy for each sale. Knowing our customers well and delivering a highly desirable product sets us apart. Basically we spend time optimising the product we offer.

People really want to look at our offers and open our emails, it’s like travel porn”

Booking.com and other travel sites are getting more known for their optimisation efforts.

Do you think doing testing and optimisation has helped beat the traditional marketing techniques of other travel companies?

TE:

“We like to see what Booking.com are doing, they’re really into optimisation but they’ve got a different [business] model to us.

In terms of traditional marketing, optimisation of our email is a pretty big thing for us. It’s the biggest win from optimising that we get.”

I read about the tests you ran to optimise the onboarding process for mobile. Interesting stuff. We have a general rule that shopping online shouldn’t require an account but you’re a good example of why there’s no ‘best practice’.

TE:

“Gated content is our selling point. Regular emails with lovely travel offers, inspirational images that people want to receive.”

Lady at a pool side dining table holding her phone

Optimisation for regional differences

You’ve set up different experiments in Formisimo for your regional sites, have you seen differences in behaviour for different geographies?

TE:

“Yes, we do see unique behaviours in different territories. And we always test and optimise separately for different sites. It would be difficult to maintain totally different websites for different demographics, so we can’t do that, but tests are run in isolation for different territories.”

Do you do research on cultural differences? Or is the target market already niche?

RG:

“Yes, and no. We’re in 11 countries so there are some differences but we predominantly let the numbers lead us.

When we run tests there can be some surprising results, that’s how we optimise for different countries. We get the local teams from the countries involved as well. They’re a brilliant resource to consult with during testing & optimisation.

We don’t have product teams in each country so the knowledge of the staff in our various locations is so valuable.”

TE:

“There are different legal requirements that we have to consider. For example, you have to use double opt-in in Germany. Germans refer to trust signals more. You really need to build trust with customers especially when launching in new territories.”

The future

What are your plans for the product? Where do you picture Secret Escapes in 5 years?

TE:

“In 5 years, hmmm. We haven’t got detailed plans for that far in advance.

I think it will always be discounted flash sales of luxury holidays, maybe down the road we’ll expand into other things too.

We have a roadmap for the next 3 months in advance. We’re not afraid to add something to the site, try it, take it off, add it again if they want to try again.

We’ve got ideas for what we’d like to do in 6 month or a 1 year but it’s not in the roadmap as such because we work more quickly than that, adapting to right now.”

Connect with Thomas on LinkedIn:

Connect with Rohit on LinkedIn:

image representing average form conversion rate uplift from using Formisimo From 0% to 24% Form Conversion Uplift Get the insight that increased form conversions by 24% (and rising)
Latest Insight
Improving Insurance Forms Reviewing the top 31 insurance forms