Thread: Day 28-30: Testing Landers and Offers

Finally - we get to learn how to test and optimize our way to profits!

Remember in the lesson "Intro to Testing & Optimization", I outlined this general testing and optimization approach in a nutshell?

1)Split-test landing pages and offers until finding a combination that turns enough traffic segments profitable, to result in a satisfactory amount of total profits across these green segments.

2)At the traffic source, either a)target the profitable traffic segments (an approach also referred to as "whitelisting") or b)exclude the unprofitable traffic segments (an approach also referred to as "blacklisting"). (The testing of bids is also a necessary part of this process - more on this in a future lesson.)

Basically, point 1) above focuses on TESTING, whereas point 2) focuses on OPTIMIZATION.

In this lesson, we'll cover the TESTING portion. In the next 3 lessons we'll cover the OPTIMIZATION portion.



***********************************

ACTION & EXPLANATION


General Offer+Lander Testing Strategy

In the last lesson, I mentioned the 3 pillars of pop traffic, i.e. Offer, Lander, and Traffic.

We talked about how all 3 pillars need to be strong in order for a campaign to get good conversion rate.

We talked about doing this testing at a traffic source with good quality traffic.

We also talked about the need to split-test offers and landers to find a good combination.


*****

Assuming we have multiple landers and offers (for the same geo + offer type - as the one we set up in the last lesson). To find the best offer and lander, we basically have three options.

Option 1: We treat EVERY offer+lander combination as a single split-test candidate. e.g. If we're testing 2 offers and 5 landers, we have 10 split-test candidates.

We'll use a statistical calculator to keep cutting inferior test candidates until we have a "last man standing" which would be the winner. (using this method - no need to go through that post right now - just giving you a sneak peek).

Option 2: We do what's called "multi-variate testing" and cut landers and offers at the same time.

Option 3: We pick an offer, use that to split-test landers to find a winning lander (using the stats calculator), then use that winning lander to split-test the offers to find a winning offer (using the stats calculator). And voila - we have our winning offer+lander!

Options Comparison:

Option 1 is the most straightforward and accurate, but it's also the most expensive as it needs the most data to complete the testing - the extra accuracy isn't worth it in my opinion, especially since pop campaigns tend to be relatively short-lived.

Option 2 is too complicated - I can't wrap my head around it so if you can figure out how to do it, and in a more efficient way than what I'm about to show you below, please enlighten me as I'd LOVE to know!

Option 3, to me, is the best option - decent level of accuracy, efficiency, and simplicity.

*****


The Catch 22 with Option 3:

Option 3 sounds deceptively simple at first glance: Just pick an offer and use it to split-test landers to find a winner, then use that to split-test offers to find a winner, and you have the winning offer+lander.

The problem? In order to split-test landers, we need to collect enough CONVERSIONS to tell us which lander is the best of the batch.

So - the offer we want to use to split-test landers: If it's a great offer, we won't need to spend much to generate the conversions necessary to split-test landers to find a winner! But if the offer is a dud, we could spend a ton of money in order to to collect enough conversions required to find the winning lander.

Here's the catch 22: In the beginning we don't have a PROVEN well-converting offer! (Yes of course your AM will tell you it's a proven offer, all the other affiliates are banking hard with it etc., but each affiliate can get different performance depending on the type of traffic they're running, how they're running etc. So an offer isn't TRULY "proven" until YOU can make it convert well.)

At this point you may ask, "how about picking a lander and using that to split-test offers instead?"

We'd be at the other half of the catch 22 situation: In the beginning, we don't have a PROVEN, well-converting lander EITHER.

So, how to get around this catch 22? Here's the lander+offer testing strategy I propose to you:


*****

My Lander+Offer Testing Strategy - in a Nutshell:

Step 1)Set Up a Campaign in the Tracker to Rotate the Offers and Landers. The rotation is crucial for split-testing.

Step 2)Set Up the Campaign at a Proven Traffic Source to Drive Traffic. PropellerAds would be a good choice.

Step 3)Identify a Good-Enough Offer for Cutting Landers.

Step 4)Use the Chosen Offer to Cut Landers Down to the Winner.

Step 5)Use Winning Lander to Test Offers.

Step 6)Decide Whether Winning Lander + Winning Offer is Good Enough to Optimize Further.

With this testing strategy you won't run into the catch 22 I described above.

Below, allow me to explain Steps 3-6 in detail. (Steps 1-2 you have already done.)



*****

Step 3: Identify a Good-Enough Offer For Cutting Landers

Action to Take: Run traffic, and pause each offer as it has received 10x payout's worth of traffic.

Condition to Check For: See whether any of the offers has made a total of 2 conversion or more after spending 10x payout in traffic cost.

Note: "10x payout in traffic cost" means "10 times the offer payout in traffic cost". For example, if the offer payout is $1, then 10x payout means $10 in spend.)

The first step is to find an offer that can give you the conversions you'll be needing to identify the best lander out of the ones you're split-testing. We'll go into how to split-test landers below.

For now, know that we'll need some conversions, so we can see how many conversions each lander makes, in order to know which lander is the best one.

AND - that offer can't convert too badly, or it would take too much spending in order to collect the required conversions.

After spending 10x payout on each offer, if an offer has made at least 2 conversions, we can move onto the next step: Continue to run that one offer to split-test the landers.

What if you have more than one offer that made 2 conversions or more? In that case, just pick the one that has made the most conversions. If they've both made the same number of conversions, just pick one arbitrarily.

What about offers that have made fewer than 2 conversions on the 10x payout in spend? We can consider them "dud" offers - ones that have little chance of making profits, even with further optimization

(You may be wondering, "why 2 conversions within 10x payout in spend?" It's just something that has worked for me. On the one hand, setting stricter criteria will increase the risk of cutting a promising offer. On the other hand, setting slacker criteria will increase the test budget required.

As you gain experience in your niche, feel free to set your own criteria - as long as it's a good balance between not cutting out too many offers by accident, while keeping the test budget required to test each offer reasonable.)

Further clarification: Know that our goal isn't to pick the BEST-performing offer out of the bunch at this point - we just want an offer that converts well enough, so that we don't break the bank while collecting conversions to identify the best lander.

If all of the offers you're testing now are eliminated by this rule, check Adplexity again to make sure you're using 3-5 popular landers that have received lots of traffic and that many affiliates are using, and confirm that you've optimized them fully - i.e. to load fast, function correctly, and display well.

And if you don't feel landers are the bottleneck, test another batch of offers. If you can't find more offers to test with the existing landers, start all over again - either select another sweepstakes prize or test another offer vertical or even another geo, and prepare a new set of landers and go from there.


Action Implementation:

To check how many conversions each offer has made: In your tracker, drill down into "Offer" stats for your campaign.

Remember to either update your tracker costs manually, or check the actual spend at the traffic source and figure out how much money each offer has spent. E.g. If actual spend is $12 and you're rotating 3 offers with equal weight, then each offer has spent $4.

Here's how to pause an offer in each respective tracker...

For Voluum: Go into the campaign's edit mode, find the offer and put the cursor on it, change the rotation weight from "100" to "0", and "Save".



For Binom: Go into campaign edit, find the offer and toggle it to grey, and click "Save".



For Funnelflux Pro: Open the funnel in the funnel editor, click on the offer group icon, change the weight to "0", and click "Save settings".





*****

Step 4: Use the Chosen Offer to Cut Landers Down to the Winner

ONLY do this AFTER you've found a good-enough offer to use for cutting landers - as described above.

Condition to Check For: Every time you check your stats, drill down to Offer > Landers and only consider lander stats for the chosen offer. Enter the lander stats into the stats calculator, and look for inferior lander(s) that have a "probability of being best" of below 10% when compared to the "current-best" lander.

Action to Take: Pause the lander(s) that fit the above condition, in the tracker.

Read this thread to learn how to enter lander stats into the stats calculator to cut inferior landers:

https://stmforum.com/forum/showthrea...Banners-Part-1

Perform this check regularly to cut inferior landers, until you're left with one winner.

If more than one lander keeps going "neck-to-neck", i.e. are performing around the same, instead of spending money on identifying a winner, just pick a winner at random.

IMPORTANT: When entering conversions into the stats calculator, you will need to use ONLY conversions that have resulted from the ONE offer you've chosen to use above (i.e. the one that made at least 2 conversions after 10x payout in spend).


Action Implementation:

For Voluum:

Drill down into campaign stats, first by "Offer" and then by "Lander". Then, expand lander stats for the chosen offer (the first one to reach 2 conversions), and enter those lander stats into the stats calculator as instructed in the tutorial linked to above, with "Visits" (i.e. impressions) as "Trials" and "Conversions" as "Successes".



(For more instructions on how to load campaign stats in Voluum, please see here.)

To pause an inferior lander, go into the campaign's edit mode, find the lander and put the cursor on it, change the rotation weight from "100" to "0", and "Save".




For Binom:

Drill down into campaign stats, first by ""Offers" and then by "Landers". Find the lander stats for the chosen offer (the first one to reach 2 conversions) and enter those into the stats calculator as instructed in the tutorial linked to above, with "Clicks" (i.e. impressions) as "Trials" and "Leads" as "Successes".



(For more instructions on how to load campaign stats in Binom, please see here.)

To pause an inferior lander, go into the campaign's edit mode, find the lander and and toggle it to grey, and click "Save".




For Funnelflux Pro:

Drill down into campaign stats, first by "Campaign", then by "Visitor Journey". (Instead of "Visitor Journey" you can also specify "Offer" and "Lander", but these would disappear and be replaced by "Visitor Journey" anyway.) Then, expand the layers of stats until you see lander stats for the chosen offer (the first one to reach 2 conversions), and enter those lander stats into the stats calculator as instructed in the tutorial linked to above, with "Visits" (i.e. impressions) as "Trials" and "Conv" as "Successes".



(For more instructions on how to load campaign stats in Funnelflux Pro, please see here.)

To pause an inferior lander, open up the funnel in funnel editor, click on the "Lander Group" icon to go into node settings, find the lander and and toggle it to grey, and click "Save settings".





*****

Step 5: Use Winning Lander to Test Offers

Action to Take: Start a new split-test for these offers, and use a stats calculator to cut down to a winner:

a)The "chosen offer" found in step 3.

b)Offers that were paused (if any) after the chosen offer was found in Step 3 - but only those that have made at least 2 conversions in the initial 10x payout in spend (across all landers).

c)Any new offers you can find that can be tested using the winner lander.

Also: If any offer failed to make at least 5 conversions per 10x payout in spend FOR THE WINNING LANDER, pause it!

Condition to Check For: Every time you check your stats, drill down to Landers > Offers, get offer stats for the winning lander and enter them into the stats calculator. Pause offers that have a "probability of being best" of below 10% when compared to the "current-best" offer. Also pause offers that fail to make at least 5 conversions per every 10x payout in spend for the winning lander.

In the last step, you were able to cut down to the last-man-standing, winning lander.

In this step, you're putting this winning lander to good use - by using it to test offers.

We'd want to include, in this offer split-test, the following offers:

-All offers that received 10x payout in spend in step 3, that made at least 2 conversions. This includes the "chosen" offer you ended up using for the lander split-test, as well as any other offer(s) that has/have made at least 2 conversions, that you paused.

You may be wondering: Why are we testing the same offers again? Answer - the first time we ran 10x payout to these offers, we did not run enough spend to reach statistical significance, so the "chosen offer" may not ACTUALLY be the winning offer in the batch! So we need to retest and use the stats calculator this time.

-Also include in this offer split-test, any new offers you can find in affiliate networks. This can be any additional offers your AM can provide - new offers that don't yet have track record, or old offers that have received very little traffic from other affiliates. Unlike in the beginning when you needed offers that were proven to convert, you can now test any offer - because you now have a proven lander (so can avoid the catch-22 situation previously described).

If you have trouble finding more offers to test with your winner lander, you may want to sign up to more affiliate networks. Spy on Adplexity, check Offervault.com, ask here in the forum for good offers (and sometimes aff network reps will recommend them) - to see which aff networks have these offers and sign up for them.

(Warning: If cashflow is an issue for you, try not to run with too many aff networks - having all your money stuck across many network accounts, with amounts that haven't reached minimum payment threshold, is not fun. So be selective and choose wisely! For example if you're planning on running sweeps for the next while, then join networks that focus on sweeps offers.)

The goal is to run this offers split-test and cut these offers down to a winner - just like what you did when split-testing landers!


Action Implementation:

First of all, you would add the new offers to the tracker (if they haven't been already), and then add them to the tracker's campaign settings (by going into campaign edit).

It's important to make a note of the date and hour you're starting this new split-test (use the campaign journal), because when assessing stats using the stat calculator, we will ONLY use stats that are collected AFTER the split-test starts. All previous data - including previous stats of the current offer and winner lander - will need to be excluded.

(When split-testing ANYTHING in general, all test candidates need to be exposed to the same set of conditions - thus they need to be run at the same time. Your campaign may receive different traffic quality from different placements etc. during different times, due to changes in competition levels etc. Therefore, you can't for example compare the performance of your current offer yesterday, to that of a new offer you started running today. That wouldn't be a fair comparison - say for example the current offer seemingly converts worse than the new offer, the reason could be that yesterday was a bad day for conversions - so we wouldn't be able to draw any conclusions.)

It may be easier to just put the campaign on pause for the rest of the day (i.e. pause it at the traffic source), and start the split-test the next day - just so you wouldn't need to specify the "hours" in addition to the date range when checking stats.

Next - remember we used a stats calculator to compare lander stats when doing a lander split-test? We need to do the same when comparing offer stats for this offer split-test, but because offers can have different payouts, we need to use another calculator. This thread contains all the instructions:

https://stmforum.com/forum/showthrea...211#post289211

Again: When fetching offer stats to put in the calculator, be sure to set the start of the date+hour range to the time you started the offers split-test.

Also: Don't forget to cut offers that are converting worse than 5x per every 10x payout in spend, for the winning lander.

Keep cutting until you're left with the last, winning offer. (Note: Occasionally, you'll have more than one offer in the end that are performing the same, i.e. "going neck-to-neck" on the number of conversions, and it doesn't look like statistical significance would be met anytime soon, then just pick the one with the higher revenue to continue.)



*****

Step 6: Decide Whether Winning Lander + Winning Offer is Good Enough to Optimize Further

If, by the end of the offer split-test in the last step, you still have an offer running, it means the winning offer+lander is converting at LEAST 5x on every 10x payout in spend, which translates to an ROI of -50% or better.

Remember when I pointed out in the lesson on cutting placements, that it's usually hard to optimize from less than -50% ROI all the way to profits? The same applies - whether it be a direct-linked campaign, or a campaign involving a winner lander + winner offer.

At any rate, now would be the time to stand back and decide on a next step:

1)Optimize the current lander+offer combo further (which is the subject of the next few lessons).

2)Do another round of offer split-testing (using the winner lander).

3)Do another round of lander split-testing (using the winner offer).

4)Start from scratch and test another type of offer, and/or another geo.

IMPORTANT: Once we have a winner offer or a winner lander, we'll be able to test other offers and landers - even ones that don't have good track-record - such as offers that are new to an affiliate network, or custom landers you've designed yourself from scratch.

You'll no longer run into the catch-22 situation mentioned above. Let's say you want to test a new offer - if you don't get conversions, you'll know that the new offer is probably a dud - because you're already using a proven winning lander. The same applies to new landers - because you're using an offer that has shown good performance, if a new lander performs badly, it's probably because the lander is not good.

The same also applies to traffic quality. Once you have a proven offer+lander combination, you can test different bids and different targeting options.

The idea of using proven factors at first to avoid the catch-22 situation is a very important one. I hope I've explained this concept clearly.



*****

I've just about covered the testing process! But please scroll down to the second post below - where there are a couple of details I'd like to mention.



Amy

*****

Can We Skip Step 5?

One question I can hear some of your asking is: If, after using the "chosen offer" to cut landers down to a winner, we already have a promising or even profitable lander+offer combo (doing better than -50% ROI), should we still go ahead and use the winner lander to split-test more offers?

It's up to you, but I would highly recommend to split-test more offers at this point.

Reason? A good offer can make ALL the difference to your campaign. You've already tested several landers and identified a winner, but as far as offers go, you've only really just used a single offer (not counting the ones you've paused in the beginning - because they probably didn't receive enough traffic to prove themselves either way).

Moreover, you now have a winner lander that's proven to convert, so testing more offers would be cheaper than before. You've nailed down a good lander, so the next step would be to nail down a good offer.

In other words: Even if your current offer is good enough to yield a green campaign, it would be a shame not to test more offers at this point, simply because the potential of drastically improving campaign performance greatly outweighs the time and money this testing will require.



*****

Cutting Placements

A lot of new affiliates have asked the question: Should I start cutting placements WHILE testing landers and offers?

Yes you can! Especially if you see one or more big placements just zapping your budget and not converting.

Remember that extensive lesson on cutting placements?

https://stmforum.com/forum/showthrea...ing-Placements

In the end, we're wanting to apply the SAME cutting method, but only to the winning lander + offer combination.

So, can we cut placements BEFORE this winning lander + offer combo emerges? i.e. While still split-testing offers and/or landers?

You absolutely can! But keep this in mind:

A placement can only convert well with a good offer+lander combo. So placements that perform "badly" while you're running multiple offers and/or landers, may actually perform better if you were to run JUST the winning lander+offer combo.

I would suggest that while you're still testing landers and/or offers, to cut placements that are in loss by 3-4x payout to curb your losses.

Then, once you have a winning lander+offer combo, view placement stats for that combo ONLY, and - if a placement isn't in loss by 2x payout (the criterion for cutting placements), unpause the placement to give it another chance - to see how it performs for the winning lander+offer combo.

So basically, pause first and retest later if applicable.



*****

Running in the Same Geos Repeatedly

I would like to point out, that there are advantages in running in the same geos again and again:

-You'd already know what the best and worst placements are, you can run on either a blacklist or whitelist, which will save you money when testing new landers and offers.

-You'd already have a proven lander (and maybe even a proven offer, if it's still active on the aff network) in one or more verticals, that you can use to quickly test new offers.

-If you've tested lander variations or custom landers, you'd understand what kinds of marketing tactics or psychological triggers the people in that geo responds well to. You may already know which hours of the day and/or days of the week typically convert the best in that geo and one or more verticals, and test new offers during those hours/days to save money.

In short: When you have some success with a certain geo, you should keep seeking out new offers to test there (same vertical as or different vertical from the initial camp you run there).

Make the initial investment, then reap the benefits repeatedly. Doing so will make you more efficient in terms of time and budget, and give you a leg-up on the competition.



*****

How Often to Check Stats

It may also help to talk about how often you need to be checking stats.

Basically: The faster the traffic comes in, the more frequently you should be checking your stats. In the beginning, try to check once per hour if you can - you'll soon know whether you're checking too frequently or not, and adjust accordingly.

If you're not able to check stats often, either:

1)Throttle/distribute your traffic at the traffic source so that traffic will flow more slowly (e.g. set a small daily budget and throttle traffic to reach this spend every 24 hours), or

2)Put the campaign on pause at the traffic source whenever you're too busy to check stats, and resume when you're free.

The less-often you check stats, the more money you can potentially waste, as you "overrun" offers, landers, and placements past the point where they can be identified as being unpromising and paused at the traffic source.

However, if you check stats too often, you'd be wasting valuable time you could be using to do more productive work (e.g. setting up new campaigns, scaling profitable campaigns). So you'll need to find a balance.

And for every major change you make to your campaign, don't forget to make a note in your Campaign Journal!



*****

Now that we have a promising lander+offer combination, let's optimize it to green!




Amy

Originally Posted by anthonyh

So if you have 3 offers:

Offer 1 - $2 payout x 10 = $20

Offer 2 - $1 payout x 10 = $10

Offer 3 - $1.25 payout x 10 = $12.50

$42.5 total spend for the offers

Is this correct?

Correct!



Amy