Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Priority vector re-prioritization proposal #329

Merged
merged 28 commits into from
Nov 11, 2022
Merged
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update FLEDGE.md
MattMenke2 authored Jul 21, 2022

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit d4eeaa8fd8bac4328ea8a223b82eca965cb4af4b
23 changes: 17 additions & 6 deletions FLEDGE.md
Original file line number Diff line number Diff line change
@@ -126,7 +126,7 @@ The browser will remain in an interest group for only a limited amount of time.

The `priority` is used to select which interest groups participate in an auction when the number of interest groups are limited by the `perBuyerGroupLimits` attribute of the auction config. If not specified, a `priority` of `0.0` is assigned. There is no special meaning to these values. These values are only used to select interest groups to participate in an auction such that if there is an interest group participating in the auction with priority `x`, all interest groups with the same owner having a priority `y` where `y > x` should also participate (i.e. `generateBid` will be called). In the case where some but not all interest groups with equal priority can participate in an auction due to `perBuyerGroupLimits`, the participating interest groups will be uniformly randomly chosen from the set of interest groups with that priority.

`priorityVector`, `prioritySignalsOverrides`, and `enableBiddingSignalsPrioritization`, are optional values use to dymnically calculate a priority used in place of `priority`. `priorityVector` and `prioritySignalsOverrides` are mappings of strings to Javscript numbers, while enableBiddingSignalsPrioritization is a bool that defaults to false. See [Prioritizing Interest Groups](#35-prioritizing-interest-groups) for a description of these fields.
`priorityVector`, `prioritySignalsOverrides`, and `enableBiddingSignalsPrioritization`, are optional values use to dymnically calculate a priority used in place of `priority`. `priorityVector` and `prioritySignalsOverrides` are mappings of strings to Javscript numbers, while enableBiddingSignalsPrioritization is a bool that defaults to `false`. See [Prioritizing Interest Groups](#35-prioritizing-interest-groups) for a description of these fields.

The `userBiddingSignals` is for storage of additional metadata that the owner can use during on-device bidding, and the `trustedBiddingSignals` attributes provide another mechanism for making real-time data available for use at bidding time.

@@ -325,23 +325,34 @@ Buyers have three basic jobs in the on-device ad auction:
Buyers may want to make on-device decisions that take into account real-time data (for example, the remaining budget of an ad campaign). This need can be met using the interest group's `trustedBiddingSignalsUrl` and `trustedBiddingSignalsKeys` fields. Once a seller initiates an on-device auction on a publisher page, the browser checks each participating interest group for these fields, and makes an uncredentialed (cookieless) HTTP fetch to a URL of the form:
https://www.kv-server.example/getvalues?hostname=publisher.com&keys=key1,key2
https://www.kv-server.example/getvalues?hostname=publisher.com&keys=key1,key2&interestGroups=name1,name2
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This way of sending data loses the association between keys and Interest Group (IG) names. For example, we might have IG 1 with key 'key1' and IG2 with keys ['key2', 'key3'].

This may become awkward and limiting for usage on the trusted server in the future. For example, it may often be the case that computing the bidding signals corresponding to some key may overlap with computing the perInterestGroupData for the corresponding IG.

Can we have an API that preserves the relationship between keys and IG names? For example, we could require keys and to be in the same order as IG names, with keys from different IGs separated by semicolons instead of commas.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The does potentially make requests bigger, since we need to duplicate keys from multiple IGs. It would also require significant changes to the trusted server code. Are you thinking in terms of things that might be useful down the line, or that would be useful now, with the single perInterestGroupData field?

I'd hope that if something depends on key, it would be something that could be applied to all IGs with the same key, though that may be wishful thinking.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least with our usage, preserving the association means that we could make the request significantly smaller. If we cannot rely on the IG name->key association being available, the IG name potentially needs to be passed twice once as part of the key and once in the interestGroups field. There are ways one could imagine using the API where one would have many duplicate keys across IGs, but I am not sure if anyone uses the API in that way. (As far as I can tell those designs typically involve moving more computation onto the client, with corresponding latency and data integrity concerns.)

Another related issue, supposing we wish to filter out the IG with negative priority with this API, we would also prefer to return empty bidding signal responses for the corresponding key(s). This either involves looking up a key to IG mapping on the server, duplicating data in the key, or doing redundant computations on the trusted server.

Whether or not all of this will be possible likely depends on details of the trusted server model, but I think it is desirable to keep the options open by preserving the information.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least with our usage, preserving the association means that we could make the request significantly smaller. If we cannot rely on the IG name->key association being available, the IG name potentially needs to be passed twice once as part of the key and once in the interestGroups field. There are ways one could imagine using the API where one would have many duplicate keys across IGs, but I am not sure if anyone uses the API in that way. (As far as I can tell those designs typically involve moving more computation onto the client, with corresponding latency and data integrity concerns.)

Another related issue, supposing we wish to filter out the IG with negative priority with this API, we would also prefer to return empty bidding signal responses for the corresponding key(s). This either involves looking up a key to IG mapping on the server, duplicating data in the key, or doing redundant computations on the trusted server.

Whether or not all of this will be possible likely depends on details of the trusted server model, but I think it is desirable to keep the options open by preserving the information.

Seems like for your use case, it would make more sense to make unique keys for each IG, rather than shared keys between IGs. One of the reasons to use keys, instead of just IG name, is so that IGs can share data.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like for your use case, it would make more sense to make unique keys for each IG, rather than shared keys between IGs. One of the reasons to use keys, instead of just IG name, is so that IGs can share data.

Yes, indeed that is how we are using the API. Please see my original concern about losing the association:

This may become awkward and limiting for usage on the trusted server in the future. For example, it may often be the case that computing the bidding signals corresponding to some key may overlap with computing the perInterestGroupData for the corresponding IG.

If we lose the association then this effectively limits the future ability of the trusted bidding server to use the interestGroupNames in computing the key response, or sharing computation between the two different outputs.

The base URL `https://www.kv-server.example/getvalues` comes from the interest group's `trustedBiddingSignalsUrl`, the hostname of the top-level webpage where the ad will appear `publisher.com` is provided by the browser, and `keys` is a list of `trustedBiddingSignalsKeys` strings. The requests may be coalesced (for efficiency) across any number of interest groups that share a `trustedBiddingSignalsUrl` (which means they also share an owner).
The base URL `https://www.kv-server.example/getvalues` comes from the interest group's `trustedBiddingSignalsUrl`, the hostname of the top-level webpage where the ad will appear `publisher.com` is provided by the browser, `keys` is a list of `trustedBiddingSignalsKeys` strings, and interestGroupNames is a list of the names of interest groups data is being fetched for. The requests may be coalesced (for efficiency) across any number of interest groups that share a `trustedBiddingSignalsUrl` (which means they also share an owner).
The response from the server should be a JSON object of the form:
```
{ 'keys': {
'key1': arbitrary_json,
'key2': arbitrary_json,
...}
...},
'perInterestGroupData': {
'name1': {
'priorityVector': {
'signal1': number,
'signal2': number,
...}
},
...
}
}
```
and the server must include the HTTP response header `X-fledge-bidding-signals-format-version: 2`. If the server does not include the header, the response will assumed to be an in older format, where the response is only the contents of the `keys` dictionary.
The value of each key that an interest group has in its `trustedBiddingSignalsKeys` list will be passed to the interest group's generateBid() function as the `trustedBiddingSignals` parameter. Values missing from the JSON object will be set to null. If the JSON download fails, or there are no `trustedBiddingSignalsKeys` or `trustedBiddingSignalsUrl` in the interest group, then the `trustedBiddingSignals` argument to generateBid() will be null.
The value of each key that an interest group has in its `trustedBiddingSignalsKeys` list will be passed from the `keys` dictionary to the interest group's generateBid() function as the `trustedBiddingSignals` parameter. Values missing from the JSON object will be set to null. If the JSON download fails, or there are no `trustedBiddingSignalsKeys` or `trustedBiddingSignalsUrl` in the interest group, then the `trustedBiddingSignals` argument to generateBid() will be null.
The `perInterestGroupData` dictionary contains optional data for interest groups whose names were included in the request URL. The `priorityVector` will be used as the priority for an interest group, if that interest group has `enableBiddingSignalsPrioritization` set to `true` in its definition. See [Prioritizing Interest Groups](#35-prioritizing-interest-groups) for more information.
Similarly, sellers may want to fetch information about a specific creative, e.g. the results of some out-of-band ad scanning system. This works in much the same way, with the base URL coming from the `trustedScoringSignalsUrl` property of the seller's auction configuration object. However, it has two sets of keys: "renderUrls=url1,url2,..." and "adComponentRenderUrls=url1,url2,..." for the main and adComponent renderUrls bids offered in the auction. It is up to the client how and whether to aggregate the fetches with the URLs of multiple bidders. The response to this request should be in the form:
@@ -462,7 +473,7 @@ The browser-generated `prioritySignals` object contains the following values:
If the resulting dot-product is negative, the interest group is immediately removed from an auction (Note that if there's no `priorityVector`, interest groups with negative values currently are not filtered from auctions). After calculting new priorities, as needed, and filtering out interest groups with negative calculated priorities, the `perBuyerGroupLimits` value is applied to all interest groups of a given order, unless the interest group's `enableBiddingSignalsPrioritization` field is present and true.
If `enableBiddingSignalsPrioritization` is true, then rather than applying `perBuyerGroupLimits` immediately after the the calculating the sparse dot product as desribed above (if `priorityVector` is non-null), then instead, group limit enforcement is delayed until after fetching the trusted bidding signals. In this case, if the trusted bidding signals specify a per-interest-group `priorityVector` for an interest group, the dot product of that `priorityVector` is a again taken with a `prioritySignals` vector. The `prioritySignals` vector is the same as in the first calculation, except that it may have a non-zero `browserSignals.firstDotProductPriority` value. If this dot product is negative, the interest group is removed from the auction. If there is no `priorityVector` for an interest group, the priority from earlier in the auction is used instead. Once all priorities have been calculated, then `perBuyerGroupLimits` is applied, and the auction continues as normal.
If `enableBiddingSignalsPrioritization` is `true`, then rather than applying `perBuyerGroupLimits` immediately after the the calculating the sparse dot product as desribed above (if `priorityVector` is non-null), then instead, group limit enforcement is delayed until after fetching the trusted bidding signals. In this case, if the trusted bidding signals specify a per-interest-group `priorityVector` for an interest group, the dot product of that `priorityVector` is a again taken with a `prioritySignals` vector. The `prioritySignals` vector is the same as in the first calculation, except that it may have a non-zero `browserSignals.firstDotProductPriority` value. If this dot product is negative, the interest group is removed from the auction. If there is no `priorityVector` for an interest group, the priority from earlier in the auction is used instead. Once all priorities have been calculated, then `perBuyerGroupLimits` is applied, and the auction continues as normal.
The advantage of having not using `enableBiddingSignalsPrioritization` is that interest groups can be filtered out before bidding signals are fetched. In auctions with large numbers of buyers auctions, some buyers may also be able to filter themselves out entirely, which could significantly improve performance.