New IAB TCFv2.0 LiveRamp CMP live - revenue and CPM drops

[MattCharge]

Hi,

I'm new to github and have run out of ideas on why I've seen a significant drop in prebid revenue and cpm since we released LiveRamp's CMP on our site earlier this week.

We're running Prebid version 3.18.0
We released the Liveramp CMP across our site (cv-library.co.uk) at the beginning of the week. Since then I have seen Prebid revenue drop off a cliff and both Google AdX and Adsense CPM's have dropped by half.

When I debug Prebid on the site (e.g. https://www.cv-library.co.uk/accountancy-jobs?pbjs_debug=true) I can see the CMP load, when I consent I then see the consent string present.
I can also see that on the surface it appears we are providing the consent string in our Bid requests to partners (see attached sshot).
There are other prebid calls on the page (different versions) that relate to other partners loaded in our ad server which I'm not too concerned about for the time being.

It looks like both the GDPR Consent Module and GDPR Enforcement Module are working as expected.

What i can't explain is why our Prebid partners have stopped bidding or are bidding at a much lower rate.

If anyone can provide any suggestions I'm all ears.

First thoughts are:
Some sort of Timeout issue
CMP is interfering with the bid requests in some way...
There is an issue with the Prebid version we are using
Under Vendor data the 'cmpVersion' is set to '1' - should this not be '2'

I can see another thread related to Quantcast here that sounds similar to the issue we are having - #5462

I've contacted Prebid partners and am waiting to hear back as well as discussions with LiveRamp but nothing as yet has presented itself as the potential issue.

If anyone can take a look and advise at all I'd be extremely grateful!!

Many thanks

Matt

Test page

It's on the live site but ads and Prebid calls can be seen on all Job posting pages - e.g. https://www.cv-library.co.uk/accountancy-jobs

[MattCharge]

Ok I may have found a fix related to the purposeOneTreatment:

Prebid receives a response from the CMP that has the following value:
addtlConsent: "1~"
cmpId: 3
cmpStatus: "loading"
cmpVersion: 1
eventStatus: "tcloaded"
gdprApplies: true
isServiceSpecific: true
listenerId: 1
publisher: {consents: {…}, legitimateInterests: {…}}
publisherCC: "aa"
purpose: {consents: {…}, legitimateInterests: {…}}
purposeOneTreatment: false
specialFeatureOptins: {1: true, 2: true}
tcString: "CO25o5IO25o7vADABBENAvCsAP_AAH_AAAAAF0QIIAAgAFAAoACwAGgAQAAqABkADgAIAASAAqABaADIAGgAOgAewBEAEUAJkATwBQgDCAMQAaAA2AB4gEAAQIAhABFACOAEhAKQApQB-gElAVcAugC6ICgAAgAFgAVAAyABwAEAAKgAWgA0ACIAEwAJ4AYgA0ACEAEUAI6AUgBSgD9AXQAA.flAAAagAAAA"
tcfPolicyVersion: 2
useNonStandardStacks: false

If a user has consented should the CMP be sending the purposeOneTreatment as 'true'?

We have set the GDPR Enforcement Module within Prebid to 'Enforce that the user consents to DeviceAccess as an activity and consider their per-vendor selection.':

pbjs.setConfig({
consentManagement: {
gdpr: {
...
rules: [{
purpose: "storage",
enforcePurpose: true,
enforceVendor: true
}]
}
}
});

Does this mean that 'purposeOneTreatment: false' will be enforced and hence be false - this appears to be the case as I'm seeing this same value in the bid requests to partners.

We plan to change it to 'Turn off enforcement of Purpose 1: don’t enforce either the user’s DeviceAccess consent or their per-vendor selection.'

rules: [{
purpose: "storage",
enforcePurpose: false,
enforceVendor: false
}]

Will this allow 'purposeOneTreatment: true' to be passed into our bid requests if a user consents through our consent notice.

Does anyone think this would address the problem I've seen with drops in revenue and cpm?

Cheers

[patmmccann]

If I were you, I would retest after enforcement module behavior changes with 4.0 tomorrow, or merge these changes in

https://github.com/prebid/Prebid.js/compare/prebid-4.0

[MattCharge]

Thanks very much for the heads up! So do you think the problem I've described can be attributed to the Enforcement Module then?

Thanks

[patmmccann]

I'm not sure, but it is quite possible. With the enforcement module changing so much with 4.0, I think it makes sense to retest there.

[MattCharge]

OK thanks, this is helpful, rollback will commence shortly....

[andyblackwell]

The main issue with LiveRamp is that prebid doesn't really have a good working case for when gdprApplies=false, I think partly because the IAB spec isn't super clear on that either.
LiveRamp triggers the event listener to fire, but with a nameless event. So prebid keeps waiting for a proper signal until the timeout hits and it gives up.
Here's a screenshot from your site, with the CMP timing out on a USA request:

I've encountered multiple issues in trying to get a proper consent setup with OneTrust, Quantcast, and currently also setting up LiveRamp.

We had OneTrust before, but it turns out the old __cmp or __uspapi never worked right, at least for us. And checked their new TCF 2.0 free version, but it doesn't seem to implement the IAB apis __tcfapi or __uspapi.

Quantcast pre-TCF 2.0 was a breeze to work with, but they started migrating to the new version as we were implementing. We decided to pivot and make the switch too, but found several problems when doing so. Quantcast was a set of different issues up until their release on 7/9, I worked with them for over a month to get their main bugs fixed where they were setting gdprApplies wrong, and firing tcloaded and useractioncomplete all the time regardless of consent or locale. They still fire "tcloaded" in the US which isn't necessarily correct, but ticks the box for what prebid is listening for. [shrug]

LiveRamp in the US fires a nameless event, which I guess acts like ready/loaded? I was going to alter the prebid module to just work with it, but by default it will cause the consent module to wait until the timeout before continuing.
A plus for LiveRamp though is that you could also skip loading it if you know the user isn't from the EU, because GDPR/CCPA are separate scripts, and prebid would bypass the check because there's no stubbed __tcfapi at all.
With Quantcast, GDPR and CCPA are bundled so you always have to include both.

Here's a couple test pages I setup to see how they fire events:
https://andyblackwell.github.io/quantcast_choice_tcf2_debug/
https://andyblackwell.github.io/liveramp_consent_tcf2_debug/

The IAB core api doesn't seem to have a clear workflow for when gdprApplies=false, there should have been some event attached to addEventListener for ready/loaded in order to check if gdprApplies, but the only way I can tell from the spec is to just keep hitting the "ping" api until the cmp is loaded. Feel like, per the spec, that's what prebid would have to do in order to check if gdprApplies.

...sorry this was so long, I've just been working on this off and on for over a month and it's driving me a tad crazy. Looking forward to the day when consent is somehow just built straight into browsers.

[MattCharge]

Hi Andy,

Sincere thanks for this, it's extremely helpful. Can I ask if you are actively working with LiveRamp now and is this with their US team?
We're rolling back today - we'll retest with 4.0 and I'll come back with any findings and I'm sure some further questions...

Thanks

[andyblackwell]

We don't have anything live yet, but I'm actively working on integrating both Quantcast and LiveRamp so we can a/b test them. We have a US rep at LiveRamp, but have only really went through a demo with them, haven't had as much back-and-forth as we had with Quantcast.

[ian-lr]

@MattCharge @andyblackwell Hi both, LiveRamp here -- I have made my colleagues who work on the CMP aware of the behavior you're seeing. Let me know if I can help close the loop on some of those conversations and help get this resolved for you -- you can reach me directly over email at imeyers (at) liveramp.com.

[MattCharge]

Hi Ian,

Thanks for this. I should state that LiveRamp have been very good in this process so far. I'll PM you separately Ian.

Thanks

[andyblackwell]

Hey @ian-lr, similar to Matt, LiveRamp has been the easiest CMP to implement for us so far for TCF 2.0. I think the main problem is the IAB core api didn't set a clear standard workflow for when gdprApplies=false, which has just caused everybody to handle that differently. The real fix to this I believe is for the IAB to actually set a standard and/or prebid to more flexibly handle when gdprApplies=false.

[MattCharge]

Hi @ian-lr, for us the CMP appeared to work really well. 98% of our traffic is UK based and we were seeing consent rates of about 95% which is why the drop in revenue and CPM seemed so strange....
On further inspection this appears to be because our bidders are 'not allowed' to set their cookie on the browser even when the user has consented through the CMP. On our test page now we can the warning message - WARNING: User sync not allowed for 'bidder a', 'bidder b' etc...
This would explain the drop we have seen. Why we are seeing this behaviour we're not sure, which is why we thought it might be related our Purpose settings within the Enforcement Module.

Can someone tell me what the main changes are within Prebid 4.0.0 related to the Enforcement Module?

[ian-lr]

Hey @MattCharge checking into this on my end to see if there are any meaningful changes to the event handling you pointed out as the potential issue. I'm also going to pull in the folks from the CMP team to see if they can point me in the right direction.

[MattCharge]

Hey @ian-lr sincere thanks for this, that would be extremely helpful.

[Fawke]

Hi @MattCharge, I can help you understand the changes to the enforcementModule in Prebid 4. Here are the major changes:

• Purpose 2 ('basicAds') enforcement can be done now. (v3, you could only enforce Purpose 1 ('storage')).
• Purpose 1 and Purpose 2 enforcement is turned ON by default. What this means is that your gdpr.rules will look like this: (if you've included the gdprEnforcement module and haven't provided your own rules.)

The default:

gdpr: {
  rules: [{
  purpose: 'storage',
  enforcePurpose: true,
  enforceVendor: true,
  vendorExceptions: []
}, {
  purpose: 'basicAds',
  enforcePurpose: true,
  enforceVendor: true,
  vendorExceptions: []
}];
}

More details can be found in the docs on Prebid.org

[MattCharge]

Hey,

Many thanks for this, it's good to know what the default settings will be in 4.0. We're going to take a look at this internally and make a final decision on this in the next few days.

[ian-lr]

@MattCharge @andyblackwell Quick update here-- sounds like your hypothesis was correct regarding prebid processing the gdpr_applies=false event. We opened up a PR to address this behavior #5564

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[bretg]

Are we good here @MattCharge ?

[MattCharge]

Hi,

I'm still working through this with LRamp. It appears that the problem lies with Google running ad unit code before the CMP stub file. Hopefully this should be rectified shortly, thanks for checking!!!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.