dfpAdServerVideo module and GDPR

[bretg]

Type of issue

Enhancement

Description

Unless I'm misunderstanding Google's docs on this, it appears that we may have an issue with the dfpAdServerVideo module -- we should be adding additional parameters to videoUrls: gdpr, gdpr_consent, addtl_consent, and possibly "rdp" (restricted data processing).

Here's the background:

• dfpAdServerVideo is an example of a "tagless request' -- https://support.google.com/admanager/answer/2623168
• GDPR + tagless requests covered at https://support.google.com/admanager/answer/9805023?hl=en

publishers will need to pass the following signals manually: gdpr={0,1} and gdpr_consent={tc string}. You can also optionally pass addtl_consent={ac string}.

Proposed Solution

Enhance the dfpAdServerVideo module:

1. If bidRequest.gdprConsent.gdprApplies exists, add gdpr=VALUE to the video URL
2. if bidderRequest.gdprConsent.consentString exists, add gdpr_consent=VALUE to the video URL
3. if bidRequest.gdprConsent.addtlConsent exists, add addtl_consent=VALUE to the video URL

Note last comments for updated reqs

[spormeon]

i'd like to throw one thing in. In that I build a VMAP after the bids are back etc, if the consent string it lumped on the string, its going to make the string which is sent to the "VMAP Creator" huge and probably put it way over the limits of what can send on one string

[pm-harshad-mane]

We have added US-Privacy string in dfp.buildVideoUrl function, #6075

[spormeon]

what sort of time frame we looking at on a fix for the GDPR bit?

[bretg]

Since no one else picked it up, I assigned the GDPR part of this to be done in the next Magnite sprint. So early January.

going to make the string which is sent to the "VMAP Creator" huge

Please explain this some more @spormeon ? Don't know what a "VMAP Creator" is. Doesn't appear to be a GAM thing.

Do you want to propose a change to the suggestion solution here like making the addition of the consent string optional?

[spormeon]

i think we have got around that by "reducing our string", we basically send a string of variables to a server to spit out a vmap file, that built off of the variables in the string, we we're hitting the limits of what could be sent on one string, but we have managed to reduce it by over half, so i think we can now "take" a huge consent string being added to our string and sent to the "vmap creator"

[bretg]

removing CCPA since that was done in #6012

[spormeon]

I see the fix has gone out in 4.22.0 but how do you "trigger" it/ config it, nothing in the docs?
https://docs.prebid.org/dev-docs/modules/dfp_video.html

[spormeon]

oh i see, it auto adding the consent numbers and the consent string, holly molly, its huge! lol

only the consent string actually needs to be sent though, doesn't it? we dont need to send the huge list of numbers?

[spormeon]

ahhhh, looks like i completely missed/ forgot about your line "Do you want to propose a change to the suggestion solution here like making the addition of the consent string optional?"

I think there should be now, at least a
addgdpr: true/ false
addgdpr_consentadd: if addgdpr=true, send

maybe a even better way is to actually pick up if it needs to be sent, as otherwise its going to be sending this even outside of EU?

addtl_consent: true/ false

[bmjoju]

@bretg If player is using the Google IMA HTML5 SDK (https://developers.google.com/interactive-media-ads/docs/sdks/html5/client-side/history), it will be added by the SDK.

[spormeon]

@bmjoju but it doesn't get sent on the prebid bid request? and another problem is that GAM still reports 2.1A errors, so how over come those, if the SDK is sending them to GAM, why are they there? ( we are using IMA SDK)

1.2 errors, acceptable just people not "consented"
but there should be no reason for 2.1A errors, if SDK is sending?

[bmjoju]

@spormeon Now I don't know which player you are using and if it is using the Google IMA HTML5 SDK. Try and look in the network request when player is making request for GAM. If using the Google IMA HTML5 SDK it will add a lot parameters, like gdpr_consent, addtl_consent and also the us_privacy parameter. It will fetch the consent from the tcp api.

[spormeon]

@bmjoju yep, being sent so why 2.1A erros reported in GAM?

and still the question of "but it doesn't get sent on the prebid bid request?

[bmjoju]

@spormeon Seems that error is due to SDK cant pick up the consent, maybe cmp has not loaded yet in some situations?

Which prebid bid request are you referring? The bid requests has been made when prebid builds the url. Do you mean that the gdpr_consent is missing in the url for the player?

[spormeon]

@bmjoju request to appnexus, openx, pubmatic etc etc, buildVideoUrl doesn't/ didn't send the consent details before update #6143 ?

[bmjoju]

@spormeon buildVideoUrl is building the url for the player based on the bids coming back from requestBids. #6143 is adding the consent to the url for the player, which is not needed if using the Google IMA HTML5 SDK.

[spormeon]

@bmjoju ohhhhhh , might of dropped a ball there then lol still think a true/ false might be useful for the addtl_consent: true/ false for those that might need it. Still need to whittle these 2.1A errors down though some how, any suggestions?

[bretg]

If player is using the Google IMA HTML5 SDK

@bmjoju - please translate this for us.

Does it mean we should entirely yank this feature or make it configurable? It would be easy enough to add parameters to buildVideoUrl:

                add_consent: {
                    gdpr: true,
                    gdpr_consent: true,
                    attl_consent: true,
                    usp_consent: true
                },

@spormeon - please summarize for us what a "2.1a error" means and hypothesize what you think Prebid could do about them? I'll ask the video committee to take a look.

[spormeon]

i'll try to outline the problem im seeing. We use Quantcast CMP, we are using 1 dfp for display and 1 for video. In the video dfp we see 2.1a errors going up but in the display dfp we have hardly any 2.1a errors ( so small, they are acceptable), which doesn't make sense as we have 8 display ads per page and 1 video ad per page. We deliver display and video separately to page, with 2 prebids in effect, one global pbjs and another, lets call it pbjs2. Display uses client side, video uses s2s.
Ultimatly what im trying to find out/ fix, is why video 2.1a errors are high/ going up for video

this is all the info you get on the error, but this doesn't seem to be the reason why 2.1a errors are there for video. If the cmp hasnt loaded/ timed out etc, there should be more errors for display, which there is not.

At the moment im just having to go with " these are a loss" in effect, as luckily the bulk of what we do is USA but if im getting 2.1a errors for GDPR, then surely its prob happening for CCPA but luckily thats opt-out, not opt-in

[bretg]

@mike-chowla - please find someone on the video team to provide guidance on what we should do here?

[mike-chowla]

@bretg I'll take a look

[bmjoju]

@bretg It does not matter if the parameters are present or not in the url generated by Prebid, they will get overridden/set by the SDK. So it is not needed when using a player that have the Google IMA HTML5 SDK installed. If player is using eg. only vast client, the parameters will need to be added by Prebid.

One thing I noticed though, is that the video module looks for the GDPR data too early, before response from CMP. So it ends up being null.

[spormeon]

So it not completely useless than if work in something like following, might help others. Unfortunately for me I use Ima and ain't gonna sort my 2.1 a errors lol
UsingImaSdk = true/ false as well

[spormeon]

actually, what if you have vast tag "passback/ Fallbacks" in dfp, this is a pretty common setup, if prebid and or adx doesnt win, you have "fallbacks" in dfp which are usually vast tags, to try and "mop" up some otherwise zero impressions?

[spormeon]

any plan to strip these consent out now, or make them "optional", cant really "upgrade" from 4.21.0 with them in as they are so huge and break everything else we have going on

[bretg]

@mike-chowla - we discussed this in the Prebid.js committee meeting and propose to build additional buildVideoUrl config as as noted above, copied here for convenience:

pbjs.requestBids({
    bidsBackHandler: function(bids) {
        var videoUrl = pbjs.adServers.dfp.buildVideoUrl({
            adUnit: videoAdUnit,
            params: {
                iu: '/19968336/prebid_cache_video_adunit',
                cust_params: {
                    section: "blog",
                    anotherKey: "anotherValue"
                },
                add_consent: {
                    gdpr: true,
                    gdpr_consent: true,
                    addtl_consent: true,
                    usp_consent: true
                },
                hl: "en",
                output: "xml_vast2",
                url: "https://www.example.com",
            }
        });
        invokeVideoPlayer(videoUrl);
    }
});

We propose the default values for all of them would be true.

We're interested in confirmation from the video team on this proposal.

[patmmccann]

@mike-chowla it seems we discussed changing the defaults of #6143 to 'add consent = off' so that the default case is ima sdk adds it, fallback case is prebid adds it. In that case, is this a 5.0 breaking change?

since #6143 the double add consent problem might be affecting many users, needs research

[spormeon]

any news on being able to set these consent params to true/ false, this is stopping/ blocking us from rolling up to new versions

[bretg]

Discussed in the video committee again today. Approved the approach of adding the flags, not changing the defaults. We're looking for development volunteers.

[bretg]

A summary of the development work:

• this is in the dfpAdServerVideo.js file
• support params.add_consent
• instead of the module automatically adding the 4 attributes to the constructed URL, look at the booleans to determine whether to add them
• the default is 'true' -- i.e. add the attributes if params.add_consent doesn't specify a flag

[spormeon]

p.s we made a work around on this but would still be usefull to us at least, I completely forget this is still open/ in the pipeline. Did @mike-chowla fall aslep on the job, as this was over a year ago? lol

[patmmccann]

No one fell asleep on the job, we just couldn't identify any other users this [double consent strings post 6143] might be affecting

[fowler446]

Seems like this has been handled here:
https://github.com/prebid/Prebid.js/blob/master/modules/dfpAdServerVideo.js#L127

Closing.