Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Adapter: FeedAd #4104

Merged
merged 12 commits into from
Jan 30, 2025
Merged

New Adapter: FeedAd #4104

merged 12 commits into from
Jan 30, 2025

Conversation

andwun
Copy link
Contributor

@andwun andwun commented Dec 13, 2024

This PR adds the FeedAd PBS adapter
Docs PR: prebid/prebid.github.io#5776

Copy link

Code coverage summary

Note:

  • Prebid team doesn't anticipate tests covering code paths that might result in marshal and unmarshal errors
  • Coverage summary encompasses all commits leading up to the latest one, 67f84c8

feedad

Refer here for heat map coverage report

github.com/prebid/prebid-server/v3/adapters/feedad/feedad.go:17:	getHeaders	90.9%
github.com/prebid/prebid-server/v3/adapters/feedad/feedad.go:41:	MakeBids	66.7%
github.com/prebid/prebid-server/v3/adapters/feedad/feedad.go:84:	MakeRequests	80.0%
github.com/prebid/prebid-server/v3/adapters/feedad/feedad.go:100:	Builder		100.0%
total:									(statements)	77.8%

@bsardo bsardo changed the title FeedAd: Initial version of PBS adapter New Adapter: FeedAd Dec 13, 2024
headers.Add("Accept", "application/json")
headers.Add("Content-Type", "application/json;charset=utf-8")
headers.Add("X-FA-PBS-Adapter-Version", feedAdAdapterVersion)
headers.Add("X-Openrtb-Version", "2.5")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you able to support OpenRTB 2.6 instead?

Copy link
Contributor Author

@andwun andwun Dec 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m afraid, we don’t have support for oRTB 2.6, yet, but it’s on our roadmap for next year.
Can we please start with oRTB 2.5 for now?

Copy link

Code coverage summary

Note:

  • Prebid team doesn't anticipate tests covering code paths that might result in marshal and unmarshal errors
  • Coverage summary encompasses all commits leading up to the latest one, 842aee5

feedad

Refer here for heat map coverage report

github.com/prebid/prebid-server/v3/adapters/feedad/feedad.go:17:	getHeaders	90.9%
github.com/prebid/prebid-server/v3/adapters/feedad/feedad.go:41:	MakeBids	66.7%
github.com/prebid/prebid-server/v3/adapters/feedad/feedad.go:84:	MakeRequests	80.0%
github.com/prebid/prebid-server/v3/adapters/feedad/feedad.go:100:	Builder		100.0%
total:									(statements)	77.8%

@andwun andwun requested a review from SyntaxNode December 17, 2024 12:03
@bsardo bsardo added the adapter label Jan 6, 2025
@bsardo bsardo self-assigned this Jan 20, 2025
@bsardo
Copy link
Collaborator

bsardo commented Jan 22, 2025

@Rothalack can you please review?

@bsardo bsardo assigned guscarreon and unassigned guscarreon Jan 24, 2025
@Rothalack
Copy link
Contributor

I've sent a verification email to your maintainer email.

@andwun
Copy link
Contributor Author

andwun commented Jan 27, 2025

Hi @Rothalack , you should have received a response to your confirmation email.

@Rothalack
Copy link
Contributor

I got the response, thank you!

Comment on lines 45 to 55
var invalidParams = []string{
`{}`,
`{"clientToken":"","placementId":"some-placementid"}`,
`{"clientToken":"some-clienttoken","placementId":""}`,
`{"clientToken":"some-clienttoken","placementId":"some-placementid","sdkOptions":"complete-garbage"}`,
`{"clientToken":"some-clienttoken","placementId":"some-placementid","sdkOptions":{"advertising_id":{}}}`,
`{"clientToken":"some-clienttoken","placementId":"some-placementid","sdkOptions":{"advertising_id":{}}}`,
`{"clientToken":"some-clienttoken","placementId":"some-placementid","sdkOptions":{"app_name":{}}}`,
`{"clientToken":"some-clienttoken","placementId":"some-placementid","sdkOptions":{"bundle_id":{}}}`,
`{"clientToken":"some-clienttoken","placementId":"some-placementid","sdkOptions":{"hybrid_platform":{}}}`,
`{"clientToken":"some-clienttoken","placementId":"some-placementid","sdkOptions":{"limit_ad_tracking":{}}}`,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest adding some more invalid param test cases to ensure that your regex for placementId is correct.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines 36 to 43
var validParams = []string{
`{"clientToken":"some-clienttoken","placementId":"some-placementid"}`,
`{"clientToken":"some-clienttoken","placementId":"some-placementid","sdkOptions":{}}`,
`{"clientToken":"some-clienttoken","placementId":"some-placementid","sdkOptions":{"hybrid_platform":"ios"}}`,
`{"clientToken":"some-clienttoken","placementId":"some-placementid","sdkOptions":{"hybrid_platform":"windows"}}`,
`{"clientToken":"some-clienttoken","decoration":"some-decoration","placementId":"some-placementid","sdkOptions":{"advertising_id":"","app_name":"","bundle_id":"","hybrid_app":false,"hybrid_platform":"","limit_ad_tracking":false}}`,
`{"clientToken":"some-clienttoken","decoration":"some-decoration","placementId":"some-placementid","sdkOptions":{"advertising_id":"some-advertisingid","app_name":"some-appname","bundle_id":"some-bundleid","hybrid_app":true,"hybrid_platform":"android","limit_ad_tracking":true}}`,
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest adding some more valid param test cases to ensure that your regex for placementId is correct.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -0,0 +1,17 @@
endpoint: "https://ortb.feedad.com/1/prebid/requests"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified endpoint is reachable:

curl -i --location --request POST https://ortb.feedad.com/1/prebid/requests
HTTP/2 400
cache-control: private, no-cache, no-store
content-type: text/plain; charset=utf-8
x-cloud-trace-context: 1330f4943e1c24fead2cd0c791a59b3d
date: Tue, 28 Jan 2025 19:35:58 GMT
server: Google Frontend
content-length: 15
via: 1.1 google
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000

400 Bad Request

endpointCompression: gzip
maintainer:
email: support@feedad.com
gvlVendorID: 781
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified GVL ID

curl https://vendor-list.consensu.org/v3/vendor-list.json | jq '.vendors."781"'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  658k  100  658k    0     0  3647k      0 --:--:-- --:--:-- --:--:-- 3640k
{
  "id": 781,
  "name": "FeedAd GmbH",
  "purposes": [
    1,
    3,
    4,
    5,
    6,
    8,
    9
  ],
  "legIntPurposes": [
    2,
    7,
    10
  ],
  "flexiblePurposes": [
    2,
    7,
    10
  ],
  "specialPurposes": [
    1,
    2,
    3
  ],
  "features": [
    2,
    3
  ],
  "specialFeatures": [
    1,
    2
  ],
  "overflow": {
    "httpGetLimit": 128
  },
  "cookieMaxAgeSeconds": 2592000,
  "usesCookies": true,
  "cookieRefresh": true,
  "urls": [
    {
      "langId": "en",
      "privacy": "https://feedad.com/privacy/",
      "legIntClaim": "https://feedad.com/privacy/#legal-basis"
    }
  ],
  "usesNonCookieAccess": true,
  "dataRetention": {
    "stdRetention": 30,
    "purposes": {},
    "specialPurposes": {
      "3": 4320
    }
  },
  "dataDeclaration": [
    1,
    2,
    3,
    6,
    7,
    8,
    9,
    10,
    11
  ],
  "deviceStorageDisclosureUrl": "https://api.feedad.com/tcf-device-disclosures.json"
}

Comment on lines +15 to +17
iframe:
url: https://ortb.feedad.com/1/usersyncs/supply?gdpr={{.GDPR}}&gdpr_consent={{.GDPRConsent}}&gpp={{.GPP}}&gpp_sid={{.GPPSID}}&us_privacy={{.USPrivacy}}&redirect={{.RedirectURL}}
userMacro: $UID
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm getting a 404 using this URL generated by my localhost cookie_sync endpoint: https://ortb.feedad.com/1/usersyncs/supply?gdpr=0&gdpr_consent=CQL_D2nQL_D2nPjAAAENCZCAAP_AAH_AAAAAGGwAQGGgYbABAYaAAA.II7Nd_X__bX9n-_7_6ft0eY1f9_r37uQzDhfNs-8F3L_W_LwX32E7NF36tq4KmR4ku1bBIQNtHMnUDUmxaolVrzHsak2cpyNKJ_JkknsZe2dYGF9Pn9lD-YKZ7_5_9_f52T_9_9_-39z3_9f___dv_-__-vjf_599n_v9fV_78_Kf9______-____________8A&gpp=&gpp_sid=&us_privacy=&redirect=http%3A%2F%2Flocalhost%3A8000%2Fsetuid%3Fbidder%3Dfeedad%26gdpr%3D0%26gdpr_consent%3DCQL_D2nQL_D2nPjAAAENCZCAAP_AAH_AAAAAGGwAQGGgYbABAYaAAA.II7Nd_X__bX9n-_7_6ft0eY1f9_r37uQzDhfNs-8F3L_W_LwX32E7NF36tq4KmR4ku1bBIQNtHMnUDUmxaolVrzHsak2cpyNKJ_JkknsZe2dYGF9Pn9lD-YKZ7_5_9_f52T_9_9_-39z3_9f___dv_-__-vjf_599n_v9fV_78_Kf9______-____________8A%26gpp%3D%26gpp_sid%3D%26f%3Db%26uid%3D%24UID

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, actually you should be getting a 400 Bad Request, since we only allow the scheme https in redirect urls.

Quick curl example with example.com:

curl -i --location 'https://ortb.feedad.com/1/usersyncs/supply?gdpr=0&gdpr_consent=CQL_D2nQL_D2nPjAAAENCZCAAP_AAH_AAAAAGGwAQGGgYbABAYaAAA.II7Nd_X__bX9n-_7_6ft0eY1f9_r37uQzDhfNs-8F3L_W_LwX32E7NF36tq4KmR4ku1bBIQNtHMnUDUmxaolVrzHsak2cpyNKJ_JkknsZe2dYGF9Pn9lD-YKZ7_5_9_f52T_9_9_-39z3_9f___dv_-__-vjf_599n_v9fV_78_Kf9______-____________8A&gpp=&gpp_sid=&us_privacy=&redirect=https%3A%2F%2Fexample.com%2F%24UID'
HTTP/2 200 
cache-control: private, no-cache, no-store
content-type: text/html; charset=utf-8
x-cloud-trace-context: f012169dfb1e168d52f044ad1f228369
date: Wed, 29 Jan 2025 11:19:14 GMT
server: Google Frontend
content-length: 150
via: 1.1 google
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000

<!DOCTYPE html>
<html lang="en">

    <iframe src="https://example.com/23f687c1-f025-434f-b26b-b6773d1b3a5f" style="display: none;"></iframe>

</html>

Copy link
Collaborator

@bsardo bsardo Jan 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified cookie sync iframe works by sending a request generated by my localhost cookie sync endpoint with https://localhost:8000/setuid as the redirect param. Received 200 response with iframe html element. Modified the iframe url from https to http and manually triggered that url to verify it hit my localhost setuid endpoint where the cookie was verified to contain the feedad id.

Screenshot 2025-01-29 at 2 32 48 PM Screenshot 2025-01-29 at 2 33 15 PM Screenshot 2025-01-29 at 2 36 51 PM


if request.Device != nil {
if len(request.Device.IPv6) > 0 {
headers.Add("X-Forwarded-For", request.Device.IPv6)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please either modify your banner-app.json or banner-site.json test to have an ipv6 instead of ip or add a supplemental JSON test that has ipv6 set to cover this case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines 42 to 58
if responseData.StatusCode == http.StatusNoContent {
return nil, nil
}

if responseData.StatusCode == http.StatusBadRequest {
err := &errortypes.BadInput{
Message: "Unexpected status code: 400. Bad request from publisher. Run with request.debug = 1 for more info.",
}
return nil, []error{err}
}

if responseData.StatusCode != http.StatusOK {
err := &errortypes.BadServerResponse{
Message: fmt.Sprintf("Unexpected status code: %d. Run with request.debug = 1 for more info.", responseData.StatusCode),
}
return nil, []error{err}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that the prebid docs do not reflect this yet but please use the adapter utility functions adapters/response.go#IsResponseStatusCodeNoContent and adapters/response.go#CheckResponseStatusCodeForErrors to handle invalid status codes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines 42 to 58
if responseData.StatusCode == http.StatusNoContent {
return nil, nil
}

if responseData.StatusCode == http.StatusBadRequest {
err := &errortypes.BadInput{
Message: "Unexpected status code: 400. Bad request from publisher. Run with request.debug = 1 for more info.",
}
return nil, []error{err}
}

if responseData.StatusCode != http.StatusOK {
err := &errortypes.BadServerResponse{
Message: fmt.Sprintf("Unexpected status code: %d. Run with request.debug = 1 for more info.", responseData.StatusCode),
}
return nil, []error{err}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add supplemental JSON tests for status codes 204, 400 and 500.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var response openrtb2.BidResponse
err := jsonutil.Unmarshal(responseData.Body, &response)
if err != nil {
return nil, []error{err}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a supplemental JSON test case where the response is malformed. This can be done by simply setting the mock response body to a string instead of a JSON object.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}

requestData := &adapters.RequestData{
Method: "POST",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: I suggest using http.MethodPost instead of "POST".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link

Code coverage summary

Note:

  • Prebid team doesn't anticipate tests covering code paths that might result in marshal and unmarshal errors
  • Coverage summary encompasses all commits leading up to the latest one, 770ffd8

feedad

Refer here for heat map coverage report

github.com/prebid/prebid-server/v3/adapters/feedad/feedad.go:15:	getHeaders	100.0%
github.com/prebid/prebid-server/v3/adapters/feedad/feedad.go:39:	MakeBids	100.0%
github.com/prebid/prebid-server/v3/adapters/feedad/feedad.go:72:	MakeRequests	80.0%
github.com/prebid/prebid-server/v3/adapters/feedad/feedad.go:88:	Builder		100.0%
total:									(statements)	96.9%

@andwun
Copy link
Contributor Author

andwun commented Jan 29, 2025

Thank you @bsardo for reviewing this PR.
All your feedback should be addressed by the new commits.

Copy link
Collaborator

@bsardo bsardo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

headers.Add("X-Forwarded-For", request.Device.IPv6)
}

if len(request.Device.IP) > 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Under this logic, if a request comes with both root.device.ip and root.device.ipv6, ipv4 would get prioritized over ipv6. Is this convenient? Or should the order of the if statements be reversed to prioritize ipv6 into the X-Forwarded-For header if a request with both values comes in?

   .
   .
  "device": {
    "ua": "test-user-agent",
    "ip": "123.123.123.123",
    "ipv6": "2001:db8:3333:4444:5555:6666:7777:8888",
    "language": "en",
    "dnt": 0
  },
   .
   .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For us, this doesn't really matter.
The only real use case I can think of for looking at the header, would be filtering traffic before it reaches our API.
When processing in the API, we still have both fields ip and ipv6 in the request payload.

Can this really happen that both fields are filled anyway?
Isn't the client IP usually taken from the IP connection the device is connecting to the server?
How can it connect with both IPv4 and IPv6?

Anyway, I don't see a real issue here to be honest.

bidResponse.Bids,
&adapters.TypedBid{
Bid: &seatBid.Bid[i],
BidType: openrtb_ext.BidTypeBanner,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we expect all of the bids to be banners? Is there a possibility now or in the future that we get Video, Native or Audio? If so, should we grab the type from the response instead of hard-coding the banner value?

57     for _, seatBid := range response.SeatBid {
58         for i := range seatBid.Bid {
   +           mediaType, err := getMediaTypeForBid(&seatBid.Bid[i])
   +           if err != nil {
   +               continue
   +           }
59             bidResponse.Bids = append(
60                 bidResponse.Bids,
61                 &adapters.TypedBid{
62                     Bid:     &seatBid.Bid[i],
63 -                   BidType: openrtb_ext.BidTypeBanner,
   +                   mediaType,
64                 },
65             )
66         }
67     }
adapters/feedad/feedad.go

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we only support banners right now.
See feedad.yaml:

capabilities:
  app:
    mediaTypes:
      - banner
  site:
    mediaTypes:
      - banner

@andwun
Copy link
Contributor Author

andwun commented Jan 29, 2025

Thank you @guscarreon for reviewing.
I tried to address your comments inline.
Please take another look and let me know what you think.

Copy link
Contributor

@guscarreon guscarreon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @andwun

@bsardo bsardo merged commit a266557 into prebid:master Jan 30, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants