user id module refresh ids when consent changes

[smenzer]

Type of change

• [ X ] Feature

Description of change

Per #5420, this is an attempt to force a refresh of IDs when consent changes. To handle this, the consent received when entering the userId module is stored in a cookie, which is then used to compare on the next call to the userId module. If the stored consent is different from the current consent, the getId method will be called, regardless of whether a refresh was needed or not. If there is nothing stored in the cookie, though, the getId call is not forced - the intention here is that we don't want to force a refresh on every user as soon as this code is rolled out. After the first call with the new code, the current consent is stored, so after that it can be properly compared.

[smenzer]

@jaiminpanchal27 this is ready for your review/merge whenever you get a chance

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[smenzer]

@robertrmartinez anything you need (besides time) to finish the review for this one? happy to provide more clarity offline if necessary. thanks!

[robertrmartinez]

This looks good.

Had to go look into getCoreStorageManager as I did not know there was a way to get around the GDPR checks. Cool stuff 👍

[muuki88]

We rolled out this change an received a lot of http status 431 from users as the cookie header grow beyond the normal limit of 8kbytes. The main driver is our vendorlist, which contains all currently listed IAB vendors (version 51). We know from other big publishers that they do the same as there is no sane or safe way to figure out which partners are actually required.

It would be nice to opt-out of this feature. Partners like id5 refresh their cookies all 8hours and users that change their consent preference are rare as well as vendorlist updates as they force all users to reconsent.

Or make it configurable if cookie or local storage should be used

[smenzer]

@muuki88 This feature only forces a refresh if consent changes. If consent does not change, then refreshes are handled based on the normal process of the userId module and the specifics of the submodule. If consent changes, it's important for identifiers to know that so they can update accordingly, since they may no longer have consent or may need to re-encrypt the ID based on the updated consent.

[muuki88]

Thanks for the fast repsonse @smenzer 😃

I understand what the module does and why it's important. The issue is how the consent data is being stored.

With a long vendorlist this additional cookie takes a lot of space in the cookie header, which causes quite a few users of our sites to receive 431 http status as the cookie header is too large.

So my suggestion would be to make the storage engine configurable (localstorage or cookie).

On a side note: Safari doesn't make use of this feature as 1st party client side cookies are also purge. At least this is how I understand the Intellligent tracking preventions.

[muuki88]

IMHO it would be nice to trigger this id refresh via an API. Most CMPs have hooks when the consent changes. So this behaviour can be enabled or triggered manually. Like the triggerUserSync() API. WDYT?

[smenzer]

Hi @muuki88 are you saying that on calls to your own servers (i.e. the first party domain), the cookies being stored by this PR are being included in the request, thus causing the 431's from your https responses (as opposed to responses from the identity providers or other ad tech platforms)?

perhaps instead of storing the full string, I could just create a hash (like md5 or something similar) and store that...would be a much smaller string and shouldn't cause problems. thoughts?

I hadn't looked into cmp hooks, but I suppose that's another option, although I don't see anything specific about changes in consent...can you point me to the specifics, please? I looked here: https://github.com/InteractiveAdvertisingBureau/GDPR-Transparency-and-Consent-Framework/blob/master/TCFv2/IAB%20Tech%20Lab%20-%20CMP%20API%20v2.md#addeventlistener

[muuki88]

Hi @smenzer

Indeed. The cookies are sent to the server.

I like the idea of an md5 hash 🤯👍. That should be way smaller and serve the same purpose.

Regarding the update hooks. Sourcepoint and LiveRamp both have the ability. And if I'm not mistaken the addEventListener should also work for this.

[smenzer]

I've put in a PR to correct this - #5641