feat: improve server side cookie handling via more accurate top simulation

[AtofStryker]

• Closes Improve Cookie handling with experimentalSessionAndOrigin enabled #23551, Google Authentication recaptcha'd sometimes after second+ time through test  #23603

User facing changelog

More accurately sends/sets cookies in cy.origin when experimentalSessionAndOrigin is enabled.

Additional details

We have been gradually making improvements to cookie handling in cy.origin via #22320 #22594 #22963 #23438 and #23643

For full details on why this is needed, please see #23551. This also seems to fix some of the inconsistencies in request length we were seeing in browsers, as well as seems to reduce recaptcha's in third party authentication providers almost entirely

This PR implements the ideas in #23551, via

• patching fetch and xmlHttpRequest in the spec bridge and AUT window when cy.origin is active. The patches for fetch and xmlHttpRequest capture the absolute URL of the request being made, and if valid, are sent to the Cypress server over websocket to store the credential information of the request, as well as the resource type. Right now, this sends the absolute url to the websocket, which is hashed and stored in a Map. A map was used on the backend as it is easier to clear the contents out without losing the reference (as opposed to an object literal and iterating through the keys and deleting one by one) when we clear/reset.
  • It is worth noting that I tried adding headers in the patch requests initially, but this would sometimes trigger preflight requests for simple requests, which changes the behavior of the request itself and would likely impact users.
• adds addition X-Cypress-Request Header to the web extension and electron through the webRequest API (similar to X-Cypress-AUT-is-AUT-Frame) to add the resource type is a request. This value is set to 'true' as the extension/electron cannot discern the difference between an xhr vs fetch request. However, CDP is able to discern the difference, and sends X-Cypress-Request with a value of xhr or fetch instead of 'true'. If X-Cypress-Request's value is xhr or fetch instead of 'true', this resource type takes precedence over whatever was sent over the socket as it is more reliable and accurate to the request itself.
• For the Map mentioned in bullet 1, each request is stored in the Map via a key of 'absolute url string md5 hash' to an array of resourceType/credentials matching that url, since there might be many. The logic currently works as a FIFO queue, and pushes items onto the queue from the front end websocket, and are removed from the front of the queue when the request makes it the server. If the request cannot be found in the queue, defaults to the resource type are applied. If no resourceType is known by the request, we assume xhr given the X-Cypress-Request header is present.
  • We also might want to go off of a few other things we may have knowledge of before the request is made, such as hashing headers/body/request method (GET, POST, etc) to make this map more accurate to prevent unwanted collisions. (TODO: add tests for exercising the queue functionality).
  • This could be an issue for something like graphQL that call the same url multiple times and should be tested
• Inside the request/response middleware, we try to determine whether or not top needs to be simulated. This means that if the AUT has navigated away from the origin policy of top, we need to treat the AUT as top and simulate it. This is what happens with cy.origin.
  • If the experimentalSessionAndOrigin flag isn't set, and/or top does NOT need to be simulated, we don't attempt any additional logic on handling cookies as they should be sent/set in the browser normally without any issue.
  • However, if the experimentalSessionAndOrigin flag is set AND top needs to be simulated, we then attempt to simulate how attaching/setting cookies would be having requests/responses respectively if the AUT url were top.
    • For attaching cookies, if a request is an xhr/fetch request, we take into account the withCredentials and credentials options on whether or not to attach cookies. If we should attach cookies, we then determine the sameSiteContext for which cookies to apply. This is a bit confusing as strict, lax, and none cookies are all sent with same-site context assuming the request is configured correctly. To illustrate, I have added the following tables in Xhr and Fetch credentials examples. There are several unit tests available to verify this behavior. If a request is not xhr/fetch, we attach cookies given the site context as there are no additional configuration options needed (which is much simpler 😄 ).
    • For setting cookies in the response, we send the Set-Cookie header regardless of the top simulation/experimental flag. However, if the experimental flag is set and top needs to be simulated, we pass cookies through to our cookie jar and use the same logic as we do for attaching cookies (see bullet point above). The only additional things we need to check for is the SameSite attribute. cross-site requests cannot set Lax Cookies unless it is an AUTFrame request (a top level navigation event in the AUT frame), and cannot set Strict cookies all together. If SameSite=none is used, the cookie must be Secure

Xhr and Fetch credentials examples

Origin URL	Request URL	Origin	Site
http://app.testme.com:3500/	http://app.testme.com:3500/test-request	same-origin	same-origin(same-site)
http://app.testme.com:3500/	http://app.testme.com:3501/test-request	cross-origin	same-site
http://app.testme.com:3500/	http://www.testme.com:3500/test-request	cross-origin	same-site
http://app.testme.com:3500/	http://app.testme2.com:3500/test-request	cross-origin	cross-site

Given the above requests, lets say we have the following cookies set in our browser and top is on origin http://app.testme.com:3500/

Key	Value	Domain	SameSite
foo1	bar1	app.testme.com	Lax
foo2	bar2	.testme.com	Strict
foo3	bar3	www.testme.com	Strict
foo4	bar4	.testme.com	None (secure)
foo5	bar5	.testme2.com	None (secure)

Given the following request configurations, these are the cookies the would be applied

Request URL	ResourceType	Credential Status	Cookies Sent	Can Response set cookies?
http://app.testme.com:3500/test-request	fetch	omit		no
http://app.testme.com:3500/test-request	fetch	same-origin (default)	foo1=bar1;foo2=bar2;foo4=bar4	yes (1st/3rd party)
http://app.testme.com:3500/test-request	fetch	include	foo1=bar1;foo2=bar2;foo4=bar4	yes (1st/3rd party)
http://app.testme.com:3501/test-request	fetch	omit		no
http://app.testme.com:3501/test-request	fetch	same-origin (default)		no
http://app.testme.com:3501/test-request	fetch	include	foo1=bar1;foo2=bar2;foo4=bar4	yes (1st/3rd party)
http://www.testme.com:3500/test-request	fetch	omit		no
http://www.testme.com:3500/test-request	fetch	same-origin (default)		no
http://www.testme.com:3500/test-request	fetch	include	foo2=bar2;foo3=bar3;foo4=bar4	yes (1st/3rd party)
http://www.testme2.com:3500/test-request	fetch	omit		no
http://www.testme2.com:3500/test-request	fetch	same-origin (default)		no
http://www.testme2.com:3500/test-request	fetch	include	foo5=bar5	yes (3rd party only)
http://app.testme.com:3500/test-request	xhr	false (default)	foo1=bar1;foo2=bar2;foo4=bar4	yes (1st/3rd party)
http://app.testme.com:3500/test-request	xhr	true	foo1=bar1;foo2=bar2;foo4=bar4	yes (1st/3rd party)
http://app.testme.com:3501/test-request	xhr	false (default)		no
http://app.testme.com:3501/test-request	xhr	true	foo1=bar1;foo2=bar2;foo4=bar4	yes (1st/3rd party)
http://www.testme.com:3500/test-request	xhr	false (default)		no
http://www.testme.com:3500/test-request	xhr	true	foo2=bar2;foo3=bar3;foo4=bar4	yes (1st/3rd party)
http://www.testme2.com:3500/test-request	xhr	false (default)		no
http://www.testme2.com:3500/test-request	xhr	true	foo5=bar5	yes (3rd party only)

TODOS

• fix the refactor for remoteStates/cors package to omit subdomain where we care about it when it pertains to document.domain to prevent unecessary reloads of the AUT
• add check for AUTFrame request in set cookie in the jar
• add other things to hash off of when sending requests to the server to make requests more specific/resilient against collisions

Steps to test

How has the user experience changed?

PR Tasks

• Have tests been added/updated?
• Has the original issue (or this PR, if no issue exists) been tagged with a release in ZenHub? (user-facing changes only)
• Has a PR for user-facing changes been opened in cypress-documentation?
• Have API changes been updated in the type definitions?

[cypress-bot]

Thanks for taking the time to open a PR!

• Create a Draft Pull Request if your PR is not ready for review. Mark the PR as Ready for Review when you're ready for a Cypress team member to review the PR.
• Become familiar with the Code Review Checklist for guidelines on coding standards and what needs to be done before a PR can be merged.

[AtofStryker]

I need to update this logic to include the AUTFrame on the request to allow Lax cookies, and throw out Strict cookies all together if the request is cross-site (I believe tough-cookie handles this case, but not the first)

[AtofStryker]

going to work on splitting this draft into several smaller PRs linked to each other. I also think it's a good idea to wait on any origin method naming until #23297 lands in develop

[AtofStryker]

should be autParentOrigin here

[AtofStryker]

This PR is ultimately going to be closed for the feature branch PR on branch . I will continue to keep this draft up for a wholistic view of the changes, but will close once the feature branch PR is ready for review.

[AtofStryker]

closing in favor of #23872