I2I: Sirdata CMP integration with <amp-consent>

[tla-sirdata]

Summary

Sirdata CMP is a solution based on the IAB Europe Transparency & Consent Framework.

The purpose is to provide an AMP support with the <amp-consent> component by registering in "Supported Consent Management Platforms".

Motivation

We work with our publishers to help them in meeting requirements of GDPR related to the choice of the users with regards to the processing of their personal data.
Our CMP is deployed on several sites and we want to provide an AMP support for our publishers.

Additional context

It is possible to integrate our solution with the basic integration but we want to register it as a Supported Consent Management Platform as explained here : https://github.com/ampproject/amphtml/blob/master/extensions/amp-consent/integrating-consent.md

[zhouyx]

Hello @tla-sirdata. Please! Look forward to your PR : )

[zhouyx]

As the consent prompt is not shown again when there is an existant consent state, how long the consent information is stored ?

The consent info is stored for one year. However because information is stored by AMP viewer, it is not guaranteed. AMP viewer can remove stored information without notice.

In the window.name passed to the iframe, I can see two attributes consentState and consentStateValue with the same value ('accepted'/'rejected'/'unknown'). What's the difference between them ?

consentStateValue and consetnState are the same. Please use the consentStateValue as consentState is to be deprecated.

And what is the isDirty attribute for ?

isDirty attribute is set if the stored consent info is to be cleared. The value can by set by remote endpoint. If the isDirty bit is true, we think that the best practice is to not use the info. But we still pass the value for reference though.

In the checkConsentHref response I tried to add some key-value pairs with the sharedData field but I can't find how the client can read them.

Vendors should be able to get the value via #getConsentPolicySharedData() API.

Is it possible to add the promptUI button outside the amp-consent component ? Typically, a publisher could add the button in the footer.

Yes, that's possible.

In addition to IAB vendors, our CMP allows to collect consent for Google and publisher's purposes. How can we provide these information ?

The global consentState is designed for vendors that are not in the IAB vendor list. However granular support is not possible outside of IAB vendors. Let us know what's your requirement here.

I understand that you want the initial size to be small enough to not block users. However, we have to display a very significant amount of information on the initial layer so it's better for us to use the fullscreen mode in order to keep the layer readable. Moreover, our CMP do not block users as there is always a way to close it without giving consent. What do you think about this approach?

If I remember correctly, 60vh is the maximum height for init prompt. Is this not sufficient for you. I do think we can remove the restriction though as one can easily find workaround towards the restriction. A well known workaround is to request the full screen API before sending the ready signal. @jeffjose what do you think about us removing the maximum height restriction and allow consent to block the entire page? Thanks

[andresilveirah]

Hi @zhouyx from your answer above:

isDirty attribute is set if the stored consent info is to be cleared. The value can by set by remote endpoint. If the isDirty bit is true, we think that the best practice is to not use the info. But we still pass the value for reference though.

Does that mean that if isDirty is true the <amp-consent> will trigger the consent flow to happen again? Just to give you more context, that'd helpful in cases in which we need to ask consent again due to a change in the vendors list, for example.

[tla-sirdata]

Hi @zhouyx , thanks for the answers !

The consent info is stored for one year. However because information is stored by AMP viewer, it is not guaranteed. AMP viewer can remove stored information without notice.

So there's no way for the CMP to re-prompt the UI (in case of change in the vendor list) ?

Please use the consentStateValue as consentState is to be deprecated.

I'm not sure if we need the consentStateValue but maybe am I missing something ?

isDirty attribute is set if the stored consent info is to be cleared. The value can by set by remote endpoint. If the isDirty bit is true, we think that the best practice is to not use the info. But we still pass the value for reference though.

How can the CMP set the isDirty value by remote endpoint ?

The global consentState is designed for vendors that are not in the IAB vendor list. However granular support is not possible outside of IAB vendors. Let us know what's your requirement here.

As Google is not part of the IAB vendors, we want to send consent info related to this purpose like we do with our web solution (see https://cmp.sirdata.com/demo/index.html). In the same way, we want to send consent info related to publishers' purposes.
The general idea is to provide a way to send more info in the consent-response message once the user has validated his choices.

If I remember correctly, 60vh is the maximum height for init prompt. Is this not sufficient for you. I do think we can remove the restriction though as one can easily find workaround towards the restriction. A well known workaround is to request the full screen API before sending the ready signal.

To workaround the restriction, we send the ready signal then the enter-fullscreen signal and it works well. I agree that the restriction should be removed but if it's not I just want to make sure we are compliant with AMP rules.

You can see a demo page of what we've done so far here : https://cmp.sirdata.com/demo/amp.html
As we're not integrated in the AMP runtime yet, the amp-consent tag is configured in the basic way but the endpoints are the same.

[tla-sirdata]

Hi @zhouyx I would appreciate your help on my last comment please :) Thanks!

[zhouyx]

Hello @tla-sirdata, sorry for the delay in response. I was back from my vacation today : )

So there's no way for the CMP to re-prompt the UI (in case of change in the vendor list) ?
How can the CMP set the isDirty value by remote endpoint ?

I found these two questions to be quite related. AMP supports the dirtyBit so that the stored value will be erased the next visit. It serves the function to re-prompt the UI when the vendor list changes.
The value can be set by the checkConsentHref endpoint's response using the forcePromptOnNext field.

I'm not sure if we need the consentStateValue but maybe am I missing something ?

I agree that only consentString would be required in the ideal world. However considered there are still vendors that don't support consent string. AMP requires a consentStateValue as well. The value can be set to accept/reject/dismiss/unknown_not_required and is used by AMP to block/unblock components that don't understand consent string.
Note, consent string value will only be used when the consentStateValue is accept or reject.

The general idea is to provide a way to send more info in the consent-response message once the user has validated his choices.

As mentioned, AMP will blindly pass consentString value and also sharedData value from the CMP to the vendors. If there's agreement with ad vendors, you should still be able to use the consent string and the sharedData object to pass data to them.

To workaround the restriction, we send the ready signal then the enter-fullscreen signal and it works well. I agree that the restriction should be removed but if it's not I just want to make sure we are compliant with AMP rules.

SGTM. Thanks.

[tla-sirdata]

Hi @zhouyx, thanks again for the answers

I found these two questions to be quite related. AMP supports the dirtyBit so that the stored value will be erased the next visit. It serves the function to re-prompt the UI when the vendor list changes.
The value can be set by the checkConsentHref endpoint's response using the forcePromptOnNext field.

The forcePromptOnNext field is useful in case of an invalid consent string but sometimes we just want to re-prompt the UI while keeping the existing consent string. I suppose there's currently no way to do that.

As mentioned, AMP will blindly pass consentString value and also sharedData value from the CMP to the vendors. If there's agreement with ad vendors, you should still be able to use the consent string and the sharedData object to pass data to them.

sharedData is part of the checkConsentHref endpoint's response, which is retrieved before prompting the UI. But the additional info we want to transmit is not available at that time because it's collected by the UI, the same way the consent string is when the user saves his choices. So maybe we should have a way to update sharedData aside of checkConsentHref. What do you think ?

[zhouyx]

The forcePromptOnNext field is useful in case of an invalid consent string but sometimes we just want to re-prompt the UI while keeping the existing consent string. I suppose there's currently no way to do that.

forcePromptOnNext erases the stored value including consent state and consent string. But when we prompt the UI, the previously stored consent string will be passed to the UI iframe via name attribute. So you should be able to keep the existing consent string by asking the <amp-consent> to store that same value again.
Note: we are migrating the forcePromptOnNext field to expireCache. So please use the new field, thank you : )

sharedData is part of the checkConsentHref endpoint's response, which is retrieved before prompting the UI. But the additional info we want to transmit is not available at that time because it's collected by the UI, the same way the consent string is when the user saves his choices. So maybe we should have a way to update sharedData aside of checkConsentHref. What do you think ?

That's a good point. Could you please give me an example use case. What will be an example object that you want to share between the UI iframe and vendors. Thanks

[tla-sirdata]

forcePromptOnNext erases the stored value including consent state and consent string. But when we prompt the UI, the previously stored consent string will be passed to the UI iframe via name attribute. So you should be able to keep the existing consent string by asking the to store that same value again.

That's fine for the current visit but if the user dismisses the UI or leaves the page without interacting with the CMP, no consent-response message will be sent so the existing consent string will be lost on the next visit. On the other hand, if we try to send the message when the UI is prompted, the UI will be closed right after that, which is obviously not what we want.
I see here that the checkConsentHref response can contain the consentString value but AMP doesn't persist it in the client side storage before prompting the UI. Wouldn't it be the solution ?

That's a good point. Could you please give me an example use case. What will be an example object that you want to share between the UI iframe and vendors. Thanks

Our goal is to provide info about publisher's purposes as well as Google. You could extend the consent-response message to optionally include custom key/value pairs (see customInfo). I didn't think about it before but these pairs must be persisted and transmitted to the CMP the same way the consent string is (i.e. via checkConsentHref and name iframe attribute).
Additionally, sharedData will be updated after user has saved his choices to provide non-persisted info. Typically, with the IAB TCF, vendors shouldn't decode the consent string themselves but must read signals through the CMP, which is also more convenient to process. Moreover, publishers are not able to decode the publisher consent string. We could use sharedData to provide decoded data to both vendors and publishers.

Example :

{
  "type": "consent-response",
  "action": "reject",
  "info": "BOrMs0lOrMs6TBcAiCFRCyAAAAAtCAAA",
  "customInfo": {
    "publisher": "BOrTj6NOrTj6NBcApCFRCxABgAAACw",
    "google": true,
  }
  "sharedData": {
    "vendors": {
      "purposeConsents": { "1": true, "2": false, "3": true, "4": false, "5": true},
      "vendorConsents": { "1": true, "2": true, "3": true, "4": false, "5": false}
    },
    "publisher": {
      "standardConsents": { "1": true, "3": true}
    }
  }
}

Thanks

[zhouyx]

Thank you for the detailed explanation. Can I summarize them as two new feature requests.

1. Give CMP the choice to prompt regardless of stored consent information. Currently this is controlled by amp-consent.

This can either be achieved by introducing a forcePrompt in the checkConsentHref response. Or run CMP iframe in the background and provide an API for it to expand.
We need to be careful with this feature, as it provides the functionality to always prompt a dialog when user visit, and can be used to create bad user experience.

2. Allow CMP to pass additional structured information to vendors except consentState and consentString.

I can see this feature being useful since AMP won't interpret the consent string for vendors. Another possible solution is to keep the CMP iframe in the background and provide APIs for vendors to communicate with CMPs directly. That's the most flexible solution however with performance concerns.

I understand that we want to pass data to vendors, but could you please explain why passing shared data to publishers and how would publishers use the data? Thank you

Add @ampproject/wg-ads for ideas.

[tla-sirdata]

1. Give CMP the choice to prompt regardless of stored consent information. Currently this is controlled by amp-consent.

If there's a new vendor or purpose to disclose, we need to re-prompt the UI in order to collect consent for it. Since AMP doesn't interpret the consent string, I think it's up to the publisher and the CMP to decide. I agree that it can be used to create bad user experience so it would be good to find a way to prevent the UI to be prompted too often.

2. Allow CMP to pass additional structured information to vendors except consentState and consentString.

The TCF specifies a way to communicate with the CMP frame via postMessage (see here). If we keep the CMP iframe in the background, vendors will be able to communicate with it using the standard TCF API, which indeed is the most flexible for AMP.
I made a quick example of how to read vendor consent data, maybe there's an easier way to access the CMP iframe for vendors.

this.callId = this.callId || 0;
var ampConsents = document.getElementsByTagName('amp-consent');
if (ampConsents.length > 0) {
	var consentFrames = ampConsents[0].getElementsByTagName('iframe');
	if (consentFrames.length > 0) {
		consentFrames[0].contentWindow.postMessage({
			__cmpCall: {
				command: 'getVendorConsents',
				parameter: [1, 2],
				callId: ++this.callId
			}
		}, '*');
	}
};

I understand that we want to pass data to vendors, but could you please explain why passing shared data to publishers and how would publishers use the data? Thank you

The same way vendors need consent before processing, publishers may need consent to process on their own behalf (example: this publisher enables the 5 TCF purposes for his own) or for vendors not registered in the TCF (examples : Google, Facebook).
I don't know how AMP can provide APIs for publishers but they should be able to read this data before unblocking their components (example : analytics pixel).

[tla-sirdata]

Hi @zhouyx,
We've submitted our pull request to make our CMP available with the existing integration as most of our publishers will be fine with it.

I'm still interested into the evolutions we've discussed to improve the framework for vendors and publishers. Let me know if I can help.

Thanks

[tla-sirdata]

Hi @zhouyx,
We fixed our issue with the CLA. Is there anything else required from us to review the pull request?
Thanks

[zhouyx]

Thank you! Left comments to the PR.

Re: discussion here, we had recently created an issue to collect feedback from vendors and CMPs.
#26229

One of the proposal is to allow the CMP iframe to run in the background, which I believe would resolve the issue above. However that comes with performance concern, so it's still under discussion. Thanks.

[zhouyx]

Closed by #26040