Extending FLEDGE to support coordinated experiments #266

abrik0131 · 2022-03-11T01:50:34Z

In #191 @JensenPaul suggested that runAdAuction take a 16-bit integer and pass it on the trusted server fetches. Here's a concrete proposal for how this could look.

Example Usage

The browser visits a publisher page.
Ad tag JS receives sellerExperimentGroupId and buyerExperimentGroupId as a part of the contextual call.
Ad tag JS invokes navigator.runAdAuction() and includes sellerExperimentGroupId as part of auction config and buyerExperimentGroupId as part of perBuyerSignals.
The browser includes sellerExperimentGroupId on the seller Trusted Server call and buyerExperimentGroupId on the buyer Trusted Server calls.
Seller trusted servers divert experiments by considering sellerExperimentGroupId and prepare appropriate responses. Buyer servers divert experiments by considering buyerExperimentGroupId and prepare appropriate responses.
buyerExperimentGroupId is available to generateBid() via perBuyerSignals. sellerExperimentGroupId is available to scoreAd() via auction config. Consequently, both generateBid() and scoreAd() will be able to make experiment-specific decisions.
sellerExperimentGroupId is passed to reportResult() via auction config to support experiment-specific logic. buyerExperimentGroupId is passed to reportWin() via perBuyerSignals to support experiment-specific logic. reportResult() may choose to make sellerExperimentGroupId available to reportWin() via sellerSignals.

@JensenPaul

In [WICG#191](WICG#191) @JensenPaul [suggested](WICG#191 (comment)) that runAdAuction take a 16-bit integer and pass it on the trusted server fetches. Here's a concrete proposal for how this could look. ## Example Usage 1. The browser visits a publisher page. 2. Ad tag JS generates `experimentGroupId`. 3. Ad tag JS invokes `navigator.runAdAuction()` and includes `experimentGroupId`. 4. The browser includes `experimentGroupId` on the Trusted Server calls. 5. Servers divert experiments by considering `experimentGroupId`, and prepare appropriate responses. 6. The buyer trusted server may choose to make `experimentGroupId` available to `generateBid()` by including it in its `trustedBiddingSignals` response. `experimentGroupId` is available to `scoreAd()` via auction config. Consequently, both `generateBid()` and `scoreAd()` will be able to make experiment-specific decisions. 7. `experimentGroupId` is passed to `reportResult()` via auction config to support experiment-specific logic in reporting functions. `reportResult()` may choose to make it available to `reportWin()` via `sellerSignals`.

morlovich · 2022-03-14T14:48:17Z

Should the experiment ID go with seller trusted signals as well?

abrik0131 · 2022-03-14T18:02:10Z

Yes, experiment ID (experimentGroupId) should go with seller trusted signals as well. I have added another commit (a94b2b2) for this. Thanks.

morlovich · 2022-03-15T18:34:50Z

@JensenPaul FYI Paul --- would be good if you double-checked this as I am close to wrapping up tests for the implementation

jrmooring · 2022-03-15T20:34:10Z

Could the experimentGroupId be specified per-buyer? This would allow buyer contextual+fledge experiment coordination without a strong reliance on sellers.

The seller could still pass an experiment ID to buyers as part of the contextual flow which buyers could choose to
(1) echo back in an igbid
(2) remap into their own experiment ID space or
(3) ignore, potentially choosing their own experiment ID instead

abrik0131 · 2022-03-25T19:20:06Z

@jrmooring , I've added a new commit to address your comment. Please take a look.

jrmooring · 2022-03-25T22:00:18Z

That looks good to me!

morlovich · 2022-03-28T16:19:02Z

FLEDGE.md

@@ -148,7 +148,10 @@ const myAuctionConfig = {
  'auctionSignals': {...},
  'sellerSignals': {...},
  'sellerTimeout': 100,
-  'perBuyerSignals': {'https://www.example-dsp.com': {...},
+  'sellerExperimentGroupId': 12345,
+  'perBuyerSignals': {'https://www.example-dsp.com': {


So what CL I have pending implements has it as a separate perBuyerExperimentGroupId map, similar to perBuyerTimeouts, and also admitting * for setting the default. What do you think of that interface?

My thought is that if it is a separate field then it's one more signal a seller has to map from the contextual call to runAdAuction(). This may be okay as a one-off, but if you can imagine more buyer-controlled fields being added in the future that the browser needs to understand then it's a change all the sellers need to handle as well. Also need to worry about field name conflicts if they're root level in perBuyerSignals.

Maybe a separate perBuyerConfigurations object that for now just contains the experimentGroupId, but could, in the future, be extended to take any additional buyer-controlled fields that the browser needs to know about?

#276 also runs into this, with perBuyerGroupLimits

On the other hand, with the way partial updating of auction configurations works, there is some advantage of keeping them separate to allow more granular updates.

Is there anything required on my end?

@jeffkaufman could you please expand on "the way partial updating of auction configurations works"?

I like the idea of a perBuyerConfigurations object that contains all buyer-controlled FLEDGE settings like the experimentGroupId (and buyer limits/timeouts, maybe an exploration factor for IG selection, opt-in for bandit algorithm, I'm just making stuff up here).

The thought is that if an SSP does

foreach dsp: perBuyerConfigurations[dsp] = contextualResponse[dsp].buyerConfiguration

then buyers don't have to wait for SSP changes to utilize new buyer-controlled FLEDGE parameters if/when FLEDGE adds them.

could you please expand on "the way partial updating of auction configurations works"?

Sorry, I got confused and thought we were talking about interest groups. Ignore me!

Which means I'm also in favor of a perBuyer section. Might as well merge this with interestGroupBuyers while we're at it.

Note that I've landed https://chromiumdash.appspot.com/commit/9c37c816537202ef21e9447ddfe081093d292c97 which supports this via a separate field, but the intent was very much to incorporate this feedback on better ways of phrasing this in API once a consensus coalesces...

So as implemented, this is:
'perBuyerExperimentGroupIds': {'*': 3498,
'https://example.com': 1203},

As proposed in WICG/turtledove#266 Change-Id: I2c31cbd9fb96ceb2d1f36d42e569ba2a7fe808b0 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3516197 Reviewed-by: Russ Hamilton <behamilton@google.com> Reviewed-by: Daniel Cheng <dcheng@chromium.org> Commit-Queue: Maks Orlovich <morlovich@chromium.org> Cr-Commit-Position: refs/heads/main@{#992957}

morlovich

So this is how what's implemented differs from the current proposal:

morlovich · 2022-07-01T17:02:03Z

FLEDGE.md

@@ -148,7 +148,10 @@ const myAuctionConfig = {
  'auctionSignals': {...},
  'sellerSignals': {...},
  'sellerTimeout': 100,
-  'perBuyerSignals': {'https://www.example-dsp.com': {...},
+  'sellerExperimentGroupId': 12345,
+  'perBuyerSignals': {'https://www.example-dsp.com': {


So as implemented, this is:
'perBuyerExperimentGroupIds': {'*': 3498,
'https://example.com': 1203},

morlovich · 2022-07-01T17:04:03Z

FLEDGE.md

@@ -174,6 +177,8 @@ The returned `auctionResultPromise` object is _opaque_: it is not possible for a

 Optionally, `sellerTimeout` can be specified to restrict the runtime (in milliseconds) of the seller's `scoreAd()` script, and `perBuyerTimeouts` can be specified to restrict the runtime (in milliseconds) of particular buyer's `generateBid()` scripts. If no value is specified for the seller or a particular buyer, a default timeout of 50 ms will be selected. Any timeout higher than 500 ms will be clamped to 500 ms. A key of `'*'` in `perBuyerTimeouts` is used to change the default of unspecified buyers.

+Optionally, `sellerExperimentGroupId` parameter can be specified by the seller to support coordinated experiments with seller trusted server. If specified, this must be an integer between zero and 65535 (16 bits). Optionally,`buyerExperimentGroupId` parameter can be specified by the buyer as part of `perBuyerSignals` to support coordinated experiments with buyer trusted server. If specified, this must also be an integer between zero and 65535 (16 bits).


perBuyerExperimentGroupIds (and not per of perBuyerSignals).
Key of '*' can be used to provide a default for buyers whose origin does not have its own value specified.

morlovich · 2022-07-01T17:05:33Z

FLEDGE.md

@@ -270,11 +275,11 @@ Buyers have three basic jobs in the on-device ad auction:

 Buyers may want to make on-device decisions that take into account real-time data (for example, the remaining budget of an ad campaign).  This need can be met using the interest group's `trustedBiddingSignalsUrl` and `trustedBiddingSignalsKeys` fields.  Once a seller initiates an on-device auction on a publisher page, the browser checks each participating interest group for these fields, and makes an uncredentialed (cookieless) HTTP fetch to a URL of the form:

-    https://www.kv-server.example/getvalues?hostname=publisher.com&keys=key1,key2
+    https://www.kv-server.example/getvalues?hostname=publisher.com&buyerExperimentGroupId=12345&keys=key1,key2


Just experimentGroupId.

morlovich · 2022-07-01T17:05:53Z

FLEDGE.md


-The base URL `https://www.kv-server.example/getvalues` comes from the interest group's `trustedBiddingSignalsUrl`, the hostname of the top-level webpage where the ad will appear `publisher.com` is provided by the browser, and `keys` is a list of `trustedBiddingSignalsKeys` strings, perhaps coalesced (for efficiency) across any number of interest groups that share a `trustedBiddingSignalsUrl`.  The response from the server should be a JSON object whose keys are key1, key2, etc., and whose values will be made available to the buyer's bidding functions (un-coalesced).
+The base URL `https://www.kv-server.example/getvalues` comes from the interest group's `trustedBiddingSignalsUrl`, the hostname of the top-level webpage where the ad will appear `publisher.com` is provided by the browser, `buyerExperimentGroupId` comes from `perBuyerSignals` if provided, and `keys` is a list of `trustedBiddingSignalsKeys` strings, perhaps coalesced (for efficiency) across any number of interest groups that share a `trustedBiddingSignalsUrl`.  The response from the server should be a JSON object whose keys are key1, key2, etc., and whose values will be made available to the buyer's bidding functions (un-coalesced).


experimentGroupId, perBuyerExperimentGroupIds.

morlovich · 2022-07-01T17:06:10Z

FLEDGE.md


-Similarly, sellers may want to fetch information about a specific creative, e.g. the results of some out-of-band ad scanning system.  This works in much the same way, with the base URL coming from the `trustedScoringSignalsUrl` property of the seller's auction configuration object. However, it has two sets of keys: "renderUrls=url1,url2,..." and "adComponentRenderUrls=url1,url2,..." for the main and adComponent renderUrls bids offered in the auction. It is up to the client how and whether to aggregate the fetches with the URLs of multiple bidders.  The response to this request should be in the form:
+Similarly, sellers may want to fetch information about a specific creative, e.g. the results of some out-of-band ad scanning system.  This works in much the same way, with the base URL coming from the `trustedScoringSignalsUrl` property of the seller's auction configuration object. The parameter `sellerExperimentGroupId` comes from the auction configuration if provided. However, the URL has two sets of keys: "renderUrls=url1,url2,..." and "adComponentRenderUrls=url1,url2,..." for the main and adComponent renderUrls bids offered in the auction. It is up to the client how and whether to aggregate the fetches with the URLs of multiple bidders.  The response to this request should be in the form:


Similarly, just experimentGroupId.

abrik0131 · 2022-07-08T00:11:01Z

@morlovich thanks for the comments. I've updated FLEDGE.md. I think now the PR is in sync. with the implementation. Please take a look.

FLEDGE.md

morlovich · 2022-07-08T13:51:22Z

So this looks good to me --- thanks! --- but from talking to Paul there is some LawyerProcedureStuff(tm) that it needs to go through before landing which thankfully is something he can help with.

Co-authored-by: Paul Jensen <JensenPaul@users.noreply.github.com>

abrik0131 mentioned this pull request Mar 11, 2022

Extending FLEDGE to support coordinated experiments #191

Closed

Update FLEDGE.md

a94b2b2

abrik0131 changed the title ~~Update FLEDGE.md~~ Extending FLEDGE to support coordinated experiments Mar 24, 2022

Update FLEDGE.md

91856b1

Update FLEDGE.md

7018693

morlovich reviewed Mar 28, 2022

View reviewed changes

morlovich suggested changes Jul 1, 2022

View reviewed changes

Update FLEDGE.md

4bde316

JensenPaul reviewed Jul 8, 2022

View reviewed changes

FLEDGE.md Outdated Show resolved Hide resolved

Update FLEDGE.md

82de1c0

Co-authored-by: Paul Jensen <JensenPaul@users.noreply.github.com>

jrmooring mentioned this pull request Aug 8, 2022

Priority vector re-prioritization proposal #329

Merged

Merge branch 'main' into patch-1

c3a6617

JensenPaul merged commit c6b0c14 into WICG:main Aug 16, 2022

fhoering mentioned this pull request Nov 16, 2023

Protected Audience AB testing #909

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extending FLEDGE to support coordinated experiments #266

Extending FLEDGE to support coordinated experiments #266

abrik0131 commented Mar 11, 2022 •

edited

Loading

morlovich commented Mar 14, 2022

abrik0131 commented Mar 14, 2022

morlovich commented Mar 15, 2022

jrmooring commented Mar 15, 2022

abrik0131 commented Mar 25, 2022

jrmooring commented Mar 25, 2022

morlovich Mar 28, 2022

jrmooring Mar 28, 2022

jeffkaufman Mar 30, 2022

abrik0131 Apr 12, 2022

jrmooring Apr 15, 2022

jeffkaufman Apr 20, 2022

jeffkaufman Apr 20, 2022

morlovich Apr 21, 2022

morlovich Jul 1, 2022

morlovich left a comment

morlovich Jul 1, 2022

morlovich Jul 1, 2022

morlovich Jul 1, 2022

morlovich Jul 1, 2022

morlovich Jul 1, 2022

abrik0131 commented Jul 8, 2022

morlovich commented Jul 8, 2022

		@@ -174,6 +177,8 @@ The returned `auctionResultPromise` object is _opaque_: it is not possible for a

		Optionally, `sellerTimeout` can be specified to restrict the runtime (in milliseconds) of the seller's `scoreAd()` script, and `perBuyerTimeouts` can be specified to restrict the runtime (in milliseconds) of particular buyer's `generateBid()` scripts. If no value is specified for the seller or a particular buyer, a default timeout of 50 ms will be selected. Any timeout higher than 500 ms will be clamped to 500 ms. A key of `'*'` in `perBuyerTimeouts` is used to change the default of unspecified buyers.

		Optionally, `sellerExperimentGroupId` parameter can be specified by the seller to support coordinated experiments with seller trusted server. If specified, this must be an integer between zero and 65535 (16 bits). Optionally,`buyerExperimentGroupId` parameter can be specified by the buyer as part of `perBuyerSignals` to support coordinated experiments with buyer trusted server. If specified, this must also be an integer between zero and 65535 (16 bits).


		The base URL `https://www.kv-server.example/getvalues` comes from the interest group's `trustedBiddingSignalsUrl`, the hostname of the top-level webpage where the ad will appear `publisher.com` is provided by the browser, and `keys` is a list of `trustedBiddingSignalsKeys` strings, perhaps coalesced (for efficiency) across any number of interest groups that share a `trustedBiddingSignalsUrl`. The response from the server should be a JSON object whose keys are key1, key2, etc., and whose values will be made available to the buyer's bidding functions (un-coalesced).
		The base URL `https://www.kv-server.example/getvalues` comes from the interest group's `trustedBiddingSignalsUrl`, the hostname of the top-level webpage where the ad will appear `publisher.com` is provided by the browser, `buyerExperimentGroupId` comes from `perBuyerSignals` if provided, and `keys` is a list of `trustedBiddingSignalsKeys` strings, perhaps coalesced (for efficiency) across any number of interest groups that share a `trustedBiddingSignalsUrl`. The response from the server should be a JSON object whose keys are key1, key2, etc., and whose values will be made available to the buyer's bidding functions (un-coalesced).


		Similarly, sellers may want to fetch information about a specific creative, e.g. the results of some out-of-band ad scanning system. This works in much the same way, with the base URL coming from the `trustedScoringSignalsUrl` property of the seller's auction configuration object. However, it has two sets of keys: "renderUrls=url1,url2,..." and "adComponentRenderUrls=url1,url2,..." for the main and adComponent renderUrls bids offered in the auction. It is up to the client how and whether to aggregate the fetches with the URLs of multiple bidders. The response to this request should be in the form:
		Similarly, sellers may want to fetch information about a specific creative, e.g. the results of some out-of-band ad scanning system. This works in much the same way, with the base URL coming from the `trustedScoringSignalsUrl` property of the seller's auction configuration object. The parameter `sellerExperimentGroupId` comes from the auction configuration if provided. However, the URL has two sets of keys: "renderUrls=url1,url2,..." and "adComponentRenderUrls=url1,url2,..." for the main and adComponent renderUrls bids offered in the auction. It is up to the client how and whether to aggregate the fetches with the URLs of multiple bidders. The response to this request should be in the form:

Extending FLEDGE to support coordinated experiments #266

Extending FLEDGE to support coordinated experiments #266

Conversation

abrik0131 commented Mar 11, 2022 • edited Loading

Example Usage

morlovich commented Mar 14, 2022

abrik0131 commented Mar 14, 2022

morlovich commented Mar 15, 2022

jrmooring commented Mar 15, 2022

abrik0131 commented Mar 25, 2022

jrmooring commented Mar 25, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

morlovich left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abrik0131 commented Jul 8, 2022

morlovich commented Jul 8, 2022

abrik0131 commented Mar 11, 2022 •

edited

Loading