Thread: Day 32-35: a)Blacklisting and Whitelisting - Tips

As I've mentioned multiple times before, there are a million ways to optimize a campaign.

I can only start you off with MY way. And because I like to focus more on testing and scaling and less on optimization, I don't claim to be an expert by any means.

But it WILL be a good starting point for you to build on.

It's easy to look at a specific set of stats and give suggestions on how to optimize, but infinitely harder to come up with a generalized approach - and one that isn't a convoluted flowchart that would confuse everyone.

In the end, I settled for organizing this lesson into a series of optimization tips, plus go through a set of campaign stats in detail to illustrate some of those tips.

(You will need to read this lesson several times to understand it fully, because when I'm writing the TIPS, I refer to some of the EXAMPLE, and vice versa.)

In these lessons on optimization, I'll be using Voluum stats in the demonstration. If you're using Binom or Funnelflux Pro, I trust that by now you know how to check stats without a step-by-step.

This is going to be a long lesson - let's get started!


***********************************

General Optimization Method:

Here's an extremely rough outline to prime you for all the details to come:

1)Drill down to all targetable variables, to estimate daily profit. If estimated daily profit is acceptable, go on to next step. (Can quickly make sure ROI is acceptable as well.)

2)Identify blacklisting and whitelisting opportunities.

Good things to blacklist:
-Traffic segments with quite-negative ROI relative to rest of segments.

Good things to whitelist:
-Traffic segments with high ROI relative to rest of segments.

3)Come up with Plan of Attack consisting of blacklisting or whitelisting or a combination thereof, in order to maximize daily profits.


*****

Tip 1: Know Which Variables CAN be Optimized

When analyzing stats for a PropellerAds campaign, there's little point in considering mobile phone vs. tablet performance - because you can only choose to either target both or none.



(However, say you're seeing that tablet traffic is converting a lot better than phone traffic, you can still use this insight when scaling to a network that DOES allow you to target just tablet traffic.)

Also: Obviously, if a variable is not being tracked in your tracker, then you can't optimize by blacklisting/whitelisting them - because you wouldn't be able to judge its performance.

Therefore: When checking tracker stats to look for traffic segments to blacklist/whitelist, make sure you're able to exclude/include them at the traffic source.

Moreover, be aware that there are two types of tracking variables: Those that are detected by the tracker, and those that are passed by the traffic source.



In the screenshot above, the variables in the top part are detected by the tracker, whereas the ones at the bottom are passed to the tracker by the traffic source, which in this case is PopAds.

You can choose which variables to pass data back for, by editing the tracker's traffic source settings for the specific traffic source. e.g. For PopAds, you can find a list of available variables in PopAds' Advertiser's Knowledge Base.



*****

Tip 2: Estimating Daily Profit and ROI

Remember in the previous lesson, I stressed the importance of making sure that your best offer+lander has the potential of reaching profits?

If there's little hope of that happening, then you'd need to stop optimizing.

You could then test bids to see if you'd get better results, and/or test more offers/landers to make more of the total traffic profitable, before trying to optimize again.

But how do you evaluate whether your current offer+lander has the potential of reaching minimum profit + ROI?

I don't know of any accurate or fail-safe way of doing this. But let me introduce the concept of the Daily Profit Estimate:

The Daily Profit Estimate is an estimate of the final profits you could stand to make after optimizing a campaign.

Needless to say, when reviewing stats, we need to come up with an optimization strategy that will maximize the daily profit.

Only if this daily profits estimate is high enough, should we proceed with optimization.

Below are several ways to come up with this estimate.


Method 1)Drill into every non-placement variable that can be optimized and total up the profits of targetable green segments.

We'll covered how to estimate daily profits based on placement stats later. First, let's look at variables that are NOT placements.

Specifically, we're talking about variables that have one or more profitable, medium to large-sized segments, that can be whitelisted immediately to "lock into" profits, to theoretically yield a green campaign right away.


As was explained above, you'd want to look through campaign settings at the traffic source and cross-reference with tracker stats, to figure out which variables can be tracked and optimized (included/excluded).

For example, for popads, here are some of the variables that CAN be optimized:



If you scroll down the list there are more Voluum variables, plus variables that are tracked and passed back by PopAds. I'll leave you to figure out which other variables can be optimized (we'll be going over this in the EXAMPLES section later on as well).

Then, you'd drill into each variable, sort by decreasing profits, look for green segments, and total up the respective profits.

"Brands" is the first variable in the list:



In PopAds, we can cross-reference the "Devices" section:



If we do a cross-reference, we'll see that all the green items can be targeted on PopAds, except "T-Mobile". So we'll exclude that from our estimate. Also, I like to exclude items that have only 1 conversion, as those could just be "lottery" conversions (that are based on luck). For better accuracy you could even exclude items that have 2 conversions. At any rate, we're just trying to come up with a very rough estimate.

Therefore, the rough estimate of profits for "Brands" is:

$11.38 + $10.68 + $4.28 + $3.46 + $1.10 = approx. $30 (no need to use a calculator - just work out a rough estimate in your head)

We can work our way down the list of variables in the tracker stats filter and repeat this process. I'm not going to bore you by going through every single variable, but we'll do a couple more.

Drilling down to "OS Versions" (cross-reference at PopAds = "Operating Systems" section):



Total profits from targetable green segments = approx. $35.

Drilling down to "Model" (cross-reference at PopAds = "Devices" section):



Total profits from targetable green segments = approx. $58.

Again, you can drill into all targetable variables in this manner. Just quickly add up the profits in your head and jot each variable+total on a piece of paper.

IMPORTANT: Make sure you're looking at stats for your best offer+lander! If you've set the start of the tracker's date/time range to the time you made the last offer/lander cut (i.e. the time the final winner offer+lander emerged), then you can drill down to the various variables in the 1st level filter as shown above. However, if you've set the start of the tracker's date/time range to the time you started the campaign, including all the stats collected while testing offers and landers, then you'd need to use 3 filters when drilling down, i.e. offer > lander > [variable], and expand stats for the specific winning offer+lander.

Next: You would pick variable with the highest total profits and average/extrapolate to get daily profit and ROI estimate.

Let's say that after drilling into all targetable variables and adding up green totals, the "Model" variable with its approx. $58 of profits is the winner.

Next we need to estimate the corresponding daily profits based on the $58 and how long it took to collect these stats. Let's say you weren't throttling traffic, and it has been 50 hours since the winning offer+lander emerged. The Estimated Daily Profits would then be:

$60 / 50 hr * 24hr/day = approx. $30

Which is above the minimum daily profits we've discussed in the previous lesson.

We can also quickly verify the ROI, by totaling the costs of the same green segments:



ROI = Profit/Cost = $58/$59 (approx.) = around 100% ROI

Which is above the minimum ROI - so we're good.

What if you only had less than a day's worth of stats? You can extrapolate to 24 hours. Obviously, the more hours of data you have, the more accurate your estimate.

For example, let's say the same stats above were collected over a 10-hour period. The estimated daily profits would then be:

$58 / 10 hr * 24hr/day = approx. $140

So what if we HAD been throttling traffic when collecting those stats? In that case it would be practically impossible to estimate the daily profit+ROI. You'd need to run traffic unthrottled (i.e. at full speed or maximum traffic volume) for some time in order to estimate the total amount of traffic you'd get in a day. You can STILL include the stats you collected while running throttled, and assume the profit-per-impression ratio to be the same.

Let's do a quick example using the same "Model" stats. Let's say we've been throttling traffic when running this camp, and now we're wanting to come up with an estimate of daily profits. What we can do is run traffic unthrottled for a few hours - again, the more hours, the more accurate the resulting estimate. But let's say we run 3 hours of traffic, and receive a total of 1000 impressions. We could then estimate the daily traffic volume as follows:

2000 impressions / 3 hr * 24 hr/day = 16000 impressions/day

(The amount of traffic we receive from hour to hour will be different, but this is just a rough estimate.)

Next, we look at our "Model" stats again, and quickly add up the total impressions (i.e. "Visits") for the green segments - which comes to around 40000 impressions:



Based on our stats, these green segments made $58 profits from 38000 impressions. Therefore, for the estimate daily traffic volume of 16000 impressions/day, we can estimate the daily profit as:

$58 / 38000 impressions = ? / 16000 impressions

? = Estimated Daily Profit = $58 * 16000 impressions / 38000 impressions = approx. $24

Which is above the minimum daily profits we've set, so all good.

Accuracy of this method: Estimating the daily profits based on green segments of a non-placement variable is relatively accurate, as you'd be "locking into" profits by whitelisting placements. However, especially in the beginning of a campaign when you haven't cut a lot of placements yet, chances are you won't find so many green segments in non-placement variables.

This brings us to Method 2.


Method 2)Calculate EDP for placements.

We covered this in a previous lesson:

https://stmforum.com/forum/showthrea...ing-Placements

Quoting from that lesson, here's a summary on how to evaluate the EDP:

How to Use the EDP Numbers

Finally - here's how we use the above EDP numbers:

Our goal here is to cut enough Major Placements, so that the overall campaign will meet our profits goal (of at least $5/day).

So, a positive sign to look for would be this:

Major Placements EDP + Minor Placements EDP > $5

Again, placements EDP needs to stay above our minimum requirement of $5 in order for us to continue optimizing the campaign.



Method 3)Combination of Methods 1 & 2

Another optimization approach would be to combine Methods 1 and 2.

One thing we need to keep in mind, is that a lot of the variables are related, such that when you whitelist/blacklist segments in one variable, segments in other variables will be affected.

For example, when you blacklist an OS that has lots of traffic but doing very negative ROI compared to the overall campaign ROI, the ROI of segments of many of the other variables will increase - including placements.

Another example: As you blacklist more and more bad placements, the ROI of the other variables' segments will increase.

So what we can and should do, is look for non-placement segments that can POSSIBLY become profitable, when we blacklist bad placements and other very negative segments.

Example: Say that for a specific campaign, the Android OS is at -30% ROI but IOS OS is only at -70% ROI, and both are receiving significant amounts of traffic. If we blacklist IOS, and then keep running traffic to cut placements, the campaign has a chance of reaching green.

What we can do here to more-accurately estimate daily profit, is drill down to 2 levels - OS > Placements - and then only add up green placements for the Android OS.


Moreover, if you whitelist/blacklist segments in multiple variables, you can drill down similarly to multiple levels to perform a similar assessment.

There are too many possible cases for me to list, but the idea is that we'll always attempt to predict the profits resulting from blacklisting/whitelisting decisions BEFORE WE ACTUALLY IMPLEMENT THEM.

That way we can implement optimizations that can potentially result in the maximum amount of profits.

You'll see more examples in the EXAMPLE section of the lesson - hopefully after that, you'll get a good enough idea to be able to expand this method to specific cases you'll be encountering.



***********************************

To be continued in post below...

Thanks so much for bearing with my lengthy lessons!




Amy

*****

Tip 3: If Campaign's Daily Profits Estimate is Too Low

If after you've drilled down to the various variables, you don't see segments you can target to reach our daily profits goal, what then?

You can try one of these...

a)Gather more data. If, when you check stats, you find that most of the bigger traffic segments have NOT yet received enough traffic for you to tell whether they'd end up profitable or not, then you'll need to run more traffic to collect more data, then analyze stats again to estimate daily profits more accurately.

Worth noting: Even before you have enough stats to estimate daily profits, you can and should start blacklisting the worst placements, as discussed in a previous lesson.

b)Test bids. Another option would be to test bids. When you change the bid, you'll get traffic from a slightly different set of placements, which will give you different conversion rates than your current bid. You can bid lower to save on cost, or bid higher to potentially get traffic from better-converting placements.

However, if your "winning" offer+lander converts quite badly, then it may not even be worth testing bids for. Try this a few times and you'll get a better sense of when to test bids and when not to.

Bid testing will be covered in the next lesson.

c)Test more offers and/or landers. This is always an option, and can potentially improve your funnel to make more traffic segments profitable.



*****

Tip 4: Closely-Related Variables Can Be Examined Together

There are variables that are closely-related, that can be examined together to get a clearer picture on segment performance. Here are some of them:

Brands > Models

OS > OS Versions

Browser > Browser Versions

Connection Type > Mobile Carrier

Connection Type > ISP/Carrier

And you may discover more as you gain experience. As was mentioned before, many of the variables are related. It's really up to the individual to figure out their favorite approach to slicing and dicing the data, in order to optimize the campaign in an efficient and effective manner.

You may even want to drill down into 3 levels - e.g. Device Types > Brands > Models which I'll use as illustration below.


Application:

So in the pair, there's the first variable with fewer+big segments, and the second variable with more+small segments. One thing I like to do, is look at the first variable to identify the bigger and more-promising segments first, then for those segments drill down into the second variable to identify either some of the good segments to whitelist or some of the bad segments to blacklist.


Example:

When we examine stats, we can drill into Device Types > Brands > Models. For Device Type let's say we see this:



Mobile Phones are looking quite hopeful with its -30% ROI, and also the fact that it's a BIG segment. That would be a good opportunity for whitelisting. (Either that, or blacklist some/all of the other segments.)

Next we drill into Mobile Phones to see if we can cut very-negative segments to make Mobile Phones look better:



So there ARE some really negative brands that can be blacklisted to decrease the losses of "Mobile phone" (-$36.37 + $6.35 + $2.33 + $2.78 = approx. -$25; new ROI = -$25/$120.72 = -20% ROI). Motorola isn't hopeless at -43% ROI so we can keep targeting it and hope it will turn green when we cut other stuff such as placements - unless you want to optimize to green faster by cutting it now.

(By the way: You may be wondering what the cut-off ROI is when we choose to blacklist. There isn't a rule of thumb here, but I'd say anything without an ROI of close to -70% ROI or worse would be quite hopeless - more on that later.)

But wait: "Generic" brand isn't exactly a brand, so can't be included/excluded, at least not at PropellerAds or PopAds. (In fact, PropellerAds doesn't allow brand or model targeting - but let's assume this is a PopAds campaign.) So a better strategy would be to whitelist Samsung + Motorola + LG + Lenovo (to result in ROI = total profit / total cost = approx. -$20/$100 = -20%).

Next, we can drill further down into the Models for the 4 brands listed, to figure out what we can cut further to increase the ROI:



For Samsung there isn't much to cut - only some really small segments which may or may not be worth your trouble.



For Motorola there are a couple of segments we can cut:



We could do the same for LG and Lenovo but I'll skip them for now - the model sub-segments are too small to bother spending the time.

Action:

So what we can do is just whitelist "Smartphone" in PopAds:



And either whitelist the Brands & Models that looked promising (as we've seen above), or blacklist the ones that looked hopeless. I'll just show Samsung as an example:



Alternatively: We could have skipped Brands, and drilled to Models, pick out all the promising ones, and just target them at PopAds. It's a matter of personal preference + efficiency.

Lastly: Remember PopAds has some custom variables that they pass to the tracker as well? We could have used those instead of Voluum variables - which brings us to the next tip.



Tip 5: When Possible, Use Traffic Source Values Rather Than Tracker-Detected Values

When given a choice, I almost always use values that are passed by the traffic source, rather than values that are detected by the tracker.

This is because traffic sources may detect and categorize their data differently from the tracker. And because we need to whitelist/blacklist at the traffic source, it would be easier and more accurate to do so based on traffic source values.

In the PopAds example in the previous tip, we could have drilled down into "FORMFACTORNAME" instead of "Device Types", and "DEVICENAME" instead of "Models".



Because the items we see in the stats correspond 1-on-1 with the ones we see at the traffic source, optimizing based on these traffic source values will allow us to optimize more accurately.

For example - did you know that there's actually an "Unknown" device category in PopAds?





That's an extra $8 we could be saving just by using PopAds' DEVICENAME stats.



Tip 6: What ROI is Negative Enough to Blacklist?

Firstly - remember that in Tip 2 above, I mentioned that many of the variables are related, so that by blacklisting a negative-ROI segment in one variable, you can increase the ROI in the other variables?

This is why we aim to cut the really-negative segments first - because they are so negative, that no matter how many of the other segments we cut in the other variables, their ROI will not increase sufficiently to reach the minimum 10%+ ROI goal.

So the question is: How negative does a segment's ROI need to be, in order to justify cutting it?

It would be hard to come up with a one-size-fits-all rule, because every case is different. For example, if we're running in the US with a ton of placements, then we may be able to bring quite-negative ROIs to green by cutting a lot of placements.

Whereas if we were running in a small geo, we can't afford to cut a lot of placements, and it would therefore be difficult for segments with quite-negative ROIs to get green.

Having said that: I'd say anything without an ROI of close to -70% ROI or worse would be quite hopeless; -50% to -70% would be "not impossible"; -30% to -50% would be hopeful; and above -30% to be probable/promising.

You can use these guidelines as a crutch until you've analyzed enough stats to judge on a case-by-case basis.



Tip 7: When to Cut in Big Chunks vs. Small Chunks

If I haven't repeated this enough times, let me do so yet again because it's so important:

Mind the 80/20 rule! Focus mainly on whitelisting/blacklisting the bigger traffic segments. (Even for placements, don't spend too many days on waiting for the tiny placements to reach 0.5-1x payout or more - if that's what you have to do to get green, then you probably need to test more offers and landers before optimizing. )

Having said that: Even among the bigger traffic segments, there are relatively big segments and small segments. I want to talk about when to cut big segments and when to cut small ones.

In general:


1)Cut big segments when you want to optimize faster - but settle for less profits in the end.

Say we're running in a big geo for the first time, and although the estimated daily profit is pretty, we're losing a lot of money fast.

Or, if the overall campaign ROI is quite negative, and we don't want to take forever to see green (remember that with pop, speed is of essence!)

These cases would justify the need to optimize faster. The downside of course is that there would be more "wastage" - e.g. by doing some/all of the following:

-Cut larger segments.

-Cut less accurately and more aggressively based on less data (e.g. cutting placements at 0.5-1x payout without converting).

-Whitelist the biggest segment(s) with high/highest ROI(s) (relative to other big segments). Then continue to fine-tune by cutting placements and/or smaller segments.

(By the way, what you can do instead of whitelisting the high-ROI segments in the same campaign, is start a NEW campaign to whitelist these. Then, after the whitelist campaign has been running and cutting placements for a while, turn the original blacklist campaign back on. For this original campaign, blacklist the high-ROI segments being targeted by the new whitelist camp, cut the same placements that were cut in the whitelist camp, run a bit of traffic, then drill down to variables to see if there are more green segments you can transfer to the whitelist camp.)

Note: Once the campaign is green, you can choose to gradually turn back on some of what was blacklisted, as a retest. If you do this slowly, it won't cut into profits too much.


2)Cut smaller segments when you want to maximize profits - but optimize slower.

However, if the campaign isn't losing a ton of money fast, then you can afford to do smaller cuts in order to maximize profits in the end.

Or, if the campaign ROI is relatively close to breaking even or is even already profitable, then smaller cuts would do the job.

Or, when running in a small geo so need to cut selectively to avoid dwarfing total traffic.

-Cut in smaller chunks (for example, just cut placements, and only the bigger non-placement segments if they are very negative in ROI).

-Cut more accurately and less aggressively based on more data (e.g. cutting placements at 2x payout without converting).


Oftentimes you would use a combination of whitelisting/blacklisting bigger AND smaller segments to maximize profits. For example, if android is close to breaking even while IOS is quite negative, you could just target android and then cut placements (and/or other smaller segments) to reach green.


______________________

Sorry - still not done - please continue below...



Amy

*****

Tip 8: How to Blacklist/Whitelist Placements

We've covered that in detail in this lesson:

https://stmforum.com/forum/showthrea...ing-Placements



*****

Tip 9: Avoid Targeting "Too Narrow"

When you whitelist green segments, or blacklist red segments, you would naturally EXPECT to start earning the estimated daily profits as calculated in the post above.

Often though, this won't be the case. With pop camps there's just no telling what will actually happen when you tweak it. There's just so much volatility and uncertainty involved.

(This is also why, after you've run pop for a while, if you expand into other traffic types, they may even seem easier in a way - in terms of getting more predictable results. Pop is the easiest to learn how to set up campaigns for, but hard to learn how to achieve consistent profits with.)

One of the phenomena you should be aware of, is the fact that on some of the pop traffic networks, their algo is designed so that the broader you target, the more traffic you'll get, and the higher the quality of the traffic.

This means that when you whitelist, i.e. your targeting is narrower than the original campaign, you may not get the same traffic levels those green segments were originally getting (in the original, broad-targeting campaign).

And the traffic you do get may not convert as well as in the original campaign.

Same for blacklisting: If you cut a large percentage of the traffic, the remaining segments can send less traffic than they did in the original campaign.

Therefore, try not to over-optimize your camps, either by whitelisting overly-small segments, or blacklisting too many segments all at once. (Exception: If you're getting a ton of traffic and is bleeding to death with losses, blacklist hard and fast. But if you're getting that much traffic, then all the blacklisting should still leave a considerable percentage of traffic, to avoid running into the problem above.)

And overall speaking, blacklisting works better than whitelisting. Whitelisting can work, so try it if conditions are favorable for it (i.e. you've found some profitable segments with good profits), but if you observe the phenomenon I just described, you'd know to switch back to blacklisting.

With blacklisting, if after cutting one or multiple segments, you observe the above phenomenon, just reinstate that last batch of segments you've cut.



*****

Last Big Reminder...

One of the biggest keys to reaching profits, is the need to FIRST focus on testing offers and landers to find an offer+lander combination that can make a big-enough portion of the total traffic profitable, and THEN proceed to tweak campaign targeting - such as cutting the unprofitable traffic.

And what constitutes a "big-enough portion of the total traffic"? Basically something you can target at the traffic network - a group of profitable placements, a profitable OS, a profitable browser, etc. etc. - that can give you enough daily profits. (Otherwise, you may as well go work at McDonald's!)

Too many new affiliates try to skimp on the testing process - by taking a sub-par offer and/or lander and then brute-forcing their way to profits by focusing on heavy-cutting of placements etc.

That is NOT the way to go. In most cases, they end up spending lots of money in the cutting process, only to be left with tiny profits from the bits of traffic left over from the heavy cutting.

So, make this your mantra:

Test Landers and Offers until the Profitable Portion of the Traffic has potential of yielding your Target Daily Profits and ROI, and then Tweak Campaign Targeting to Optimize Further.

*****

Wow was that ever a long lesson!

In the next one we'll go through an example in detail so you can see optimization in action.




Amy