Macro Support for FLEDGE creatives

[av-sherman]

We’d like to request a feature to be supported for FLEDGE Original Trials that allows modifying the rendering URL inside the Fenced Frame. This feature request is intended to cover the period of the Origin Trials only.

Example API: deprecatedURLToURN(originalUrn, newUrl) -> newUrn

Rationale:

Macros are a common mechanism for passing runtime data to creatives for use in rendering. An example flow is where an advertiser places text (ex: "%%HEIGHT%%") in their creative and the SSP substitutes this with relevant information (ex: the ad slot height) prior to rendering the creative. Google's ads products support various macros, with one set of examples found at https://support.google.com/admanager/answer/2376981 – usages include size, ad unit, URL, GDPR, and more. There are also commonly-used macros established by industry frameworks, such as ${GDPR}, ${GDPR_CONSENT_XXXXX}, and ${US_PRIVACY}.

Macro substitution fundamentally breaks with FLEDGE – the allowed creative mutations are very limited and fenced frame rendering means that creatives render without access to the publisher’s page or information.

The navigator.deprecatedURNToURL API supported by the FLEDGE OT spec allows the retrieval of the underlying URL for the winning FLEDGE creative, which would allow applying macro substitutions to the creative URL. With the current spec, however, this modified URL could only be rendered in an IFrame and not a Fenced Frame.

We propose that the browser support a debug API that allows loading the modified rendering URL inside the Fenced Frame (and in particular invoking the same reporting API as if deprecatedURNToURL had never been called). This would be a mechanism to continue to support macro substitution during the Origin Trials, while the ads industry adjusts to a new world where macro substitution is no longer possible.

[jkarlin]

Hi there, thanks for filing the report!

With the current spec, however, this modified URL could only be rendered in an IFrame and not a Fenced Frame.

Can you point me to where this is written? We need to update that. You can actually load a plaintext url in a Fenced Frame, and put it in opaque-ads mode. Which is what I recommend that you do. e.g., first call deprecatedURNToURL to get the plaintext URL, modify it as needed, and then load it in <fencedframe mode="opaque-ads" src=your_macro_substituted_url>.

[jeffkaufman]

@jkarlin Sorry, you're right that "this modified URL could only be rendered in an IFrame and not a Fenced Frame" is incorrect. The motivation for this proposal, though, is the "invoking the same reporting API as if deprecatedURNToURL had never been called" bit.

Specifically, rendering a plaintext url in a fencedframe won't allow the APIs in https://github.com/WICG/turtledove/blob/main/Fenced_Frames_Ads_Reporting.md (ex: window.fence.reportEvent), to work, because the browser will not know which FLEDGE auction they refer to.

[jkarlin]

Ah, I see. You lose the metadata associated with the oapque url when translating. Thanks for the clarification.

[JensenPaul]

This seems like a reasonable need as it enables experimentation with Fenced Frames and the reporting APIs therein as @jeffkaufman's comment mentions. This seems like something that’s permissible during the OT phase while converting URNs to URLs is already facilitated by the navigator.deprecatedURNToURL() API. Two questions:

1. Allowing replacement of the entire URL seems overly powerful; a bidder might be unpleasantly surprised if a seller replaced their winning ad creative URL with that of another ad creative URL. What if we limited this to only macro substitutions and only when the pattern to replace either started with “${“ and ended with “}” (and didn’t contain other “{“ or “}” instances), or started and ended with “%%” (and didn’t contain other “%%” instances)? For example the API might look like: navigator.deprecatedReplaceInURN(urn, {'${X}': '1', '%%Y%%': '2'})

2. How do you expect this to work with component ads? Perhaps it could perform the macro substitution on the component ad URLs also?

[av-sherman]

An API as you describe that formally supports macro substitution should work for this feature request. Also supporting these substitutions for component ads would be important as well.

Thanks.

[brusshamilton]

Do you need the API to support replacing "${"/"}" delimited sequences within URL paths (not query or fragment strings)? Supporting that case is more complicated due to how URL parsing works -- the URL parsing spec (https://url.spec.whatwg.org/#url-parsing) specifies that the '{' and '}' characters will be percent encoded inside paths but not inside query or fragment strings.

[av-sherman]

The spec for the ${GDPR_CONSENT_XXXXX} and similar macros indicates that they are used as URL parameters, though it's possible they are being used in other parts of the URL.

Note also that macro text isn't necessarily static and can include dynamic information -- the actual replacement logic that would be most useful would be closer to a regex. E.g., something that can handle &gdpr_consent=${GDPR_CONSENT_123}, &gdpr_consent=${GDPR_CONSENT_777}, etc.

[brusshamilton]

navigator.deprecatedReplaceInURN has now been implemented in Chrome 103.

[jeffkaufman]

Demo pages showing that it works: first https://www.trycontra.com/test/td/join-replace.html and then https://www.jefftk.com/test/td/auction-replace.html

[gtanzer]

Just so you all know, as of M112 deprecatedReplaceInURN and deprecatedURNToURL also work with fenced frame configs. (The method name is the same; you pass a config in the same slot instead of a urn.)

[timphsieh-google]

Do you know how long will deprecatedReplaceInURN() be supported?

[rahulkooverjee-google]

I’d like to follow up on @timphsieh's comment and ask if the Chrome team would be able to support deprecatedReplaceInURN until Fenced Frame enforcement in 2026.

Rationale

We believe that supporting macro substitution at 3PCD and through 2026 will help the industry better navigate the transition period, given there are a number of key advertising use cases that rely on macro substitution today, without materially impacting the privacy model.

Use cases for macro substitution
As @av-sherman noted, macro substitution is a common mechanism for passing data to creatives for use in rendering.

Perhaps the most significant use case is regulatory consent requirements, where macro substitution is commonly used within industry frameworks to pass user consent information to ad tech vendors (e.g. the ${GDPR_CONSENT_XXXXX} macro). Transitioning the entire industry to a different framework will take time, which may jeopardize ecosystem readiness for cookie deprecation.

There are other important use cases for macro substitution too - for example buyers may use information retrieved through macro substitution (such as ad size, or even contextual information) to optimize creative asset selection at render time. While there are alternative approaches that may enable this use case, it’d require that both sellers and buyers transition away from the existing, commonly used, and well optimized approach and it’s unclear what benefit that would bring (see below).

Why we believe 2026 should be okay
We appreciate that one of the privacy goals of the Protected Audience API and the requirement that ads render in a Fenced Frame is to prevent communication between the page and the creative. However, given that Fenced Frames will not be required until 2026, it will be the case that communication between the page and the creative is possible - for example via a postMessage() into an iFrame.

Therefore, it seems consistent to also allow for macro substitution via deprecatedReplaceInURN until Fenced Frame rendering is required.

[JensenPaul]

In May 2022 Chrome added support for deprecatedReplaceInURN to address a number of different transitional issues that arose as Protected Audiences was adopted. The original ask is detailed in this issue. Macro replacement in ad creative URLs was a practice that predates Protect Audiences. The ask for this API was originally only for the duration of the Protected Audience origin trial.

The API was also later found to help ease the transition to Protected Audience in other ways. For example, as adtechs work to adopt passing ad width and height into Protected Audience using the new ad sizes API, deprecatedReplaceInURN allowed prototyping this macro replacement before the new API was adopted, which helped avoid an extra network roundtrip.

We’ve been thinking more about when to deprecate this feature as it facilitates user identity flowing into the ad rendering frame, which is something that Fenced Frames seeks to avoid. However it turns out that deprecatedReplaceInURN aides in Fenced Frame adoption because it can facilitate some legacy instances where it’s useful to pass contextual information into the ad rendering frame. Without these legacy instances addressed by deprecatedReplaceInURN, adtech might prefer to render ads in iframes (thus delaying Fenced Frame adoption) and pass this information in via postMessage, which Fenced Frames don’t allow. So, while iframe rendering is allowed until 2026, it makes sense to allow deprecatedReplaceInURN to be used.

For the reasons above, we will continue to support deprecatedReplaceInURN until at least 2026.

[martinthomson]

Is there any intention to document this deprecatedXXX hole in the privacy story? These APIs aren't in the specification and it was not exactly obvious that the opaque URN was not effectively hiding anything from sites, given that an API to reverse the mapping exists (and will do for at least two more years).

[michaelkleber]

Hi @martinthomson: The various deprecatedXXX have different privacy leakage properties and different expected lifetimes, and you're quite right that we should do a better job of documenting and speccing the whole situation. (At one point we were hesitant to spec things with short intended lifespans, but that was probably shortsighted.)

At least sometimes, the place to look is the Chrome Developers Status of pending Protected Audience API capabilities page. For example, your comment sounds like it's about deprecatedURNToURL(), which does reverse the URN mapping, but which will not outlast third-party cookies (see here).

That's in contrast with deprecatedReplaceInURN(), which as noted in this issue will last until at least 2026, but which does not let the surrounding page reverse the URN mapping. Replacing things in the URN could certainly be used to insert a first-party identifier in the opaque otherwise-k-anonymous ad render URL — but that causes the same information leakage as event-level reporting, which is why this is seems like a sensible candidate for "until at least 2026".

[martinthomson]

That certainly helps. Discovering what exists is hard enough given the current scattered state of things, let alone what might exist in the future. I'm glad that you have some ability to map this out.

Separately, I'm still unclear on the deprecatedReplaceInURN function (a name which makes no real sense to me, as presumably the replacement happens on the underlying URL) as it relates to k-anonymity. Which is subject to k-anon constraints: the template or the output of the template transform? Your answer here suggests that it is the template.

I guess that leads me to conclude that the feature isn't worthwhile, given that bidding logic is able to apply similar templating logic using the signals provided by the auction runner, but maybe the entire point here is to circumvent those k-anonymity checks.

[michaelkleber]

The k-anonymity requirement is applied to the renderURL field, which comes from the Interest Group and which (if it wins) is passed out of the auction hidden behind the URN-to-URL mapping. It can indeed be a template which then gets transformed using deprecatedReplaceInURN. Those transformations cannot happen inside the bidding logic, though — the output renderURL must literally be the same k-anonymous string that was stored in the IG.

In general, the k-anonymity constraint is meant to apply to data carried across sites, while deprecatedReplaceInURN can only be used in the context of the publisher page to add in contextual data already known to the site where the ad will appear. As I mentioned, event-level win reports already include both the k-anonymous renderURL and arbitrary contextual information. And more generally, rendering the ad inside an iframe also means that postMessage between the ad and the publisher page (in either direction) could achieve the same goal.

[martinthomson]

(We're at risk of touch on off-topic subjects now, so don't let me continue too far.)

I don't follow why the URL that is attached to an IG needs to be reflected in the auction output verbatim. You aren't applying the k-anon check to the ads listed in IGs, but the output of the auction. I see no value in trying to restrict one input to the context that operates with elevated privileges unless you also prevent them from learning other things. That would be a completely different design.

Templating is inherently risky here. If a variable is replaced with arbitrary values from the current site, then any protection you have around the fetching of the ad is invalidated. Protections around fetching do not exist currently, but in theory you could build something (web bundles, anonymous proxies, etc...). A site can use the template to uniquely identify the page view on which the ad would be placed. Templating provides the means to exfiltrate information from the bidding system via a tagged channel. That tagging can greatly improve the efficiency of exfiltration.

Attack: Site A creates an IG to which I attach 2 templated ads, plus arbitrary supplementary information (for argument's sake, let's say that this is a user identifier on Site A). Each ad is basically identical, but they contain a single bit of difference in the URL. Site B works with Site A and sets up some auctions, ensuring that this IG wins consistently. The low entropy of the templates ensures that these sail through k-anon checks pretty quickly. But the choice of ad carries one bit of information from the Site A user identifier. Site B uses a template to inject its user identifier into the URL. Now you have a fetch that includes one bit of information from the user identifier on Site A and the entire user identifier from Site B. Site B can pass a bit index in its signals to the bidding logic, so that the bidding logic can select the correct bit to use when selecting an ad, plus include the same bit index in the template alongside its own user identifier. Include more ads in the IG, you get more bits.

(I started writing a PoC of this, but this API is astonishingly hard to get bootstrapped. I'm gaining new respect for those people who have managed to make it work.)

k-anonymity would partially defend against this sort of attack, if it applied after the URL is modified from the template. It would force Site B in the attack to use reduced entropy for its own identifier. It doesn't completely blunt the attack, but it reduces it from $O(N)$ to $O(N^2)$.

Good isolation of the ad could help, but as you note, that's not going to be possible any time soon.

I understand that you believe that this overall situation is acceptable on the basis that you don't have protections in place for communication initiated by the displayed ad, but I want to ensure that we're clear on the magnitude of the privacy risk here. I also understand that you believe that attestations help here; I'm happy to elaborate on why I believe those are a poor defense.

[michaelkleber]

I understand the attack that you're describing, and it is indeed possible for now. But the joint information being leaked here — the winning k-anonymous "renderURL template" and the contextual information including perhaps a first-party identifier from the site where the ad will appear — are joint inputs to the reportWin() function.

That's why I think of the attack you're describing ad being about allowing event-level reporting, and why I'd say the presence or absence of deprecatedReplaceInURN is not really relevant.

(As mentioned above, this API exists because having a manifestation of this leak in rendering, not just reporting, is important for ad tech adoption! Specifically, the current online ads consent framework in the EU is based on the publisher website asking the user for some consent and then passing along a string indicating what the user said to all the parties involved in doing the rendering. This is wildly not k-anonymous, and some new flow will be needed to handle consent checks and privacy at the same time. But that is a problem we are putting off until after the removal of 3rd-party cookies.)

[michaelkleber]

We're adding another way to trigger the deprecatedReplaceInURN functionality, to make it available to more parties.

The deprecatedReplaceInURN JS API was designed for use by the top-level seller (with code on the publisher page). We should make it easier for component sellers to use it also. We can enable that by offering a way to specify replacements directly in the auction config object:

navigator.runAdAuction({ 'deprecatedReplaceInURN': {'${X}': '1'}, ... });

For component auctions, each component seller's config could include these replacements, and they would only be done if the winner of that component auction won the top-level auction. For example, you could replace the renderURL macro ${COMPONENT_SELLER} with the particular component seller who won the auction:

navigator.runAdAuction({ 'componentAuctions': [
    {'seller': 'https://ssp1.com',
     'deprecatedReplaceInURN': {'${COMPONENT_SELLER}': 'ssp1'}, ... },
    {'seller': 'https://ssp2.com',
     'deprecatedReplaceInURN': {'${COMPONENT_SELLER}': 'ssp2'}, ... },
    ... });

We could also let the replacement values be provided via the Promise mechanism, if they might not be known until after runAdAuction time. Please let us know if that change is valuable.

This might help with the problem described in #931 as well. But note that we still don't support replacement inside the body of a creative, only in the render URL.

[ajvelasquezgoog]

Also please be aware that we have set a target release of Chrome M123. Thanks!

[JacobGo]

We could also let the replacement values be provided via the Promise mechanism, if they might not be known until after runAdAuction time. Please let us know if that change is valuable.

Promise-based deprecatedReplaceInURN will be highly valuable for Protected Audience integrations using parallelized auctions where the SSP-specific signals are derived server-side. While hardcoding the values may be an viable option for some sellers, we would like the flexibility of a deferred input, as done previously with several other runAdAuction fields.

[dmdabbs]

For example, you could replace the renderURL macro ${COMPONENT_SELLER} with the particular component seller who won the auction...

Thank you @michaelkleber. Chrome knows the component seller for which it is making substitutions, so consider having it handle this macro (or other info safe for Chrome to substitute as it does for AD_WIDTH, AD_HEIGHT). The adtech community would seek consensus for others.

[michaelkleber]

Hmm — I guess if the ad tech community agrees that there is some specific thing that we should treat as a special macro value, then we could do so. But I feel like people would want ...&seller=gam&... rather than ...&seller=https%3A%2F%2Fwww.doubleclick.net&....

[omriariav]

@michaelkleber - Hi, I have a quick question: Today, the navigator.deprecatedReplaceInURN is now available in the global context. Will it remain like this after the Chrome 123 change?

[michaelkleber]

@omriariav Yes — we're adding a new way to perform macro substitution through the auction config, in addition to the existing JS way.

[thegreatfatzby]

Hey @michaelkleber trying to understand the intended breadth of substitution supported here, as in what parts of the renderURL can be replaced? From what I can see the %%macro%% format is supported for both path and query string param swapping, and the ${macro} format is supported for path, qs, domain, and subdomain. I haven't tried scheme b/c I'm assuming it'll be rejected on non-https, and I don't see much use even if it passed.

In particular I'm curious about the domain/subdomain and path parts. Hypothetically if that is supported it could allow us to make a little protocol, even with an auditable script(s) that support it, to allow for a lot of what we'd need with vast and native rendering, while still allowing both the DSP and SSP to use their own tech, avoiding extra redirects, etc.

[omriariav]

We added feedback to the native proposal. The 'redirect' option uses the deprecatedReplaceInURN. Having the buyer's render URL redirect to the seller's rendering service URL can cause latency and increase the seller's infra costs. I also saw the addition that the deprecatedReplaceInURN can only replace url query parameters. Can we find a way to support domain replacement while maintaining privacy requirements? happy to discuss options. thank you.

[rdgordon-index]

There was a discussion about macros in the recent WICG meeting -- https://github.com/WICG/turtledove/blob/main/meetings/2024-03-20-FLEDGE-call-minutes.md#fabian-h%C3%B6ring-short-term-options--macro-replacement-for-vastnative-workflow

[timphsieh-google]

As of M123, deprecatedReplacedInURN() replaces macro for hostname as long as the macros are lowercase (e.g. ${seller_render_url}).

@michaelkleber, based on the 2024-03-20 WICG meeting, can you confirm that:

• In the short term, ad techs can use deprecatedReplacedInURN() to replace the macro for hostname.
• In the long term, deprecatedReplacedInURN() will be deprecated. However, before deprecatedReplacedInURN() is deprecated, there will be a safer mechanism to replace macro for hostname. Ad tech will have time to migrate to the safer mechanism before deprecatedReplacedInURN() is removed.

[michaelkleber]

Yes!

[timphsieh-google]

@michaelkleber

I saw that the Protected Audience spec for deprecatedRenderURLReplacements seems to have a different spec than your above comment. More specifically, the example in the spec shows a potentially incorrect dictionary?:

'deprecatedRenderURLReplacements':{{'${SELLER}':'exampleSSP'},
                                    {'%%SELLER_ALT%%':'exampleSSP'}},

However, your above comment mentioned that deprecatedRenderURLReplacements should be a dictionary.

Separately, I wasn't able to use deprecatedRenderURLReplacements to replace any macro in M125 when I passed deprecatedRenderURLReplacements to the top level auctionConfig. (I haven't tested component's auctionConfig yet.

[timphsieh-google]

To answer my own question, it looks like the Chrome feature flag --enable-features=FledgeDeprecatedRenderURLReplacements needs to be enabled to get this to work.

In addition, it looks like deprecatedRenderURLReplacements works for a dictionary (similar to deprecatedReplaceInURN()). The spec's example is likely a typo.

[yonf-fel]

Hi,
we advice our partners to use the next renderUrl format - https://${seller_url}/%%seller_param%%pr1=val1...
we found that this format is the best for replacing origin(host) and path(+query params as an option).

can you confirm that the next format will work during support for deprecatedReplacedInURN?

[omriariav]

@timphsieh-google can you please review @yonf-fel last question, please?

[timphsieh-google]

Yes. I also verified that the above suggestion could certainly work. However, I think there could be a sligtly better format:

https://${video_rendering_url}?${video_component_url}https%3A%2F%2Fbuyer.com%2F%3Fvideo_ad%3D123

This way we can use the same style of macros.

@michaelkleber and @JensenPaul, I confirmed that the above would work in M126. Can you help confirm that the above URL format could continue to work?

[fhoering]

The latest proposal seems fine to me. The only downside is that after IG creation Chrome adds an additional / before the question mark.

After calling navigator.joinAdInterestGroup

https://${video_rendering_url}?${video_component_url}https%3A%2F%2Fbuyer.com%2F%3Fvideo_ad%3D123

becomes

https://${video_rendering_url}/?${video_component_url}https%3A%2F%2Fbuyer.com%2F%3Fvideo_ad%3D123
(additional / sign in the middle

@michaelkleber and @JensenPaul
Can you confirm that this will not create issues somewhere in Chrome for caching or page loading ?

[JensenPaul]

@michaelkleber and @JensenPaul, I confirmed that the above would work in M126. Can you help confirm that the above URL format could continue to work?

@timphsieh-google, the macro replacement code hasn't been modified since it landed 9 months ago, and I don't know of any plans to modify it. So long as the URL is valid before and after replacements, I don't see a reason it would not continue to work.

after IG creation Chrome adds an additional / before the question mark.

@fhoering, I believe this is a byproduct of standard URL parsing. When I put new URL("https://a?").toString() in the JavaScript console of all the browsers I tried, I get "https://a/?".

@michaelkleber and @JensenPaul
Can you confirm that this will not create issues somewhere in Chrome for caching or page loading ?

@fhoering, caching and page loading are done with URLs after the macro replacements are completed, so they should be unaffected by the URL's state prior to macro replacement.

[JensenPaul]

A quick update on the status of rolling out of deprecatedRenderURLReplacements: This support is enabled in 50% of Chrome Canary and Dev and Beta traffic, and 1% of Chrome Stable traffic. We hope to very soon have it enabled for 100% of Chrome traffic. Note that we’re not rolling out to labeled Chrome-facilitated testing traffic to avoid disrupting ongoing experiments and testing.

[omriariav]

@JensenPaul Hi, what's the latest on the rollout? I tried searching for the feature status at https://chromestatus.com/features but couldn't find it. Do you have a plan to add this to a stable release?

[omriariav]

@JensenPaul hi, following up here if you can please help me gather more visibility on when it will reach stable?

[dmdabbs]

Hello @JensenPaul. Back in May you wrote:

Note that we’re not rolling out (deprecatedRenderURLReplacements) to labeled Chrome-facilitated testing traffic to avoid disrupting ongoing experiments and testing.

As I understand it, the CMA-aligned "market testing" has completed. Chrome has extended the current set of labels (evaluating possible changes to the labeling configuration), and you "expect ad-techs to continue to use the labels to evaluate and optimize deployments of the Privacy Sandbox Ads APIs." Evaluating features across all users would be helpful.

[dmdabbs]

Hello @JensenPaul. The FledgeDeprecatedRenderURLReplacements cmdline flag no longer works in Canary. I've had this flag set for quite some time.

Noting also that back in May you "hope to very soon have it enabled for 100% of Chrome traffic."

[JensenPaul]

@JensenPaul hi, following up here if you can please help me gather more visibility on when it will reach stable?

This went to 100% of Chrome Stable population May 20th.

Hello @JensenPaul. The FledgeDeprecatedRenderURLReplacements cmdline flag no longer works in Canary. I've had this flag set for quite some time.

The flag was enabled by default in Canary May 17th in this CL.