Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable to playback track 1 session record for track 2 mgmt sdk test cases #11535

Merged
merged 3 commits into from
Apr 27, 2020

Conversation

erich-wang
Copy link
Member

The update will support to read track 1 session record for track 2 mgmt sdk test cases in playback mode. In record mode, it will save session record in v2 format.

if (isTrack1Record)
{
//there's no domain name for request uri in track 1 record, so add it from reqeust uri
int len = 8; //length of "https://"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we positive that schema is always https?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the len we used as start index for searching "/" to get end index of domain name, so 8 is good here even for http://

{
if (Mode == RecordedTestMode.Playback && IsTrack1SessionRecord())
{
return _session.Names[callerMethodName].Dequeue();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who adds items to _session.Names when test is re-recorded?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for re-record, the record file will be in track 2 format and Names is obsolete.

@@ -84,7 +98,11 @@ public virtual RecordEntry FindMatch(Request request, IList<RecordEntry> entries
score++;
}

//we only check Uri + RequestMethod for track1 record
if (!isTrack1Record)
{
score += CompareHeaderDictionaries(headers, entry.Request.Headers, ExcludeHeaders);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes recordings way less useful, validating neither body nor headers.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This compromise is not ideal, this is best I could figure out, please let me know if any other good suggestion.

@pakrym
Copy link
Contributor

pakrym commented Apr 23, 2020

What's you plan for management plane track 2 recordings? Do you ever plan to re-record tests? what about adding new tests?

@erich-wang
Copy link
Member Author

What's you plan for management plane track 2 recordings? Do you ever plan to re-record tests? what about adding new tests?

All new tests will be recorded in new format. For existing ones, we plan to do it after finishing higher priority things, like release.

@pakrym
Copy link
Contributor

pakrym commented Apr 24, 2020

I really have trouble understanding what we are trying to save by reusing the Track 1 recordings, at the point where running tests Live works it's a matter of running 1 command to update the recordings and get higher quality verification with that.

I'm worried of this code becoming an instant legacy.

int domainEndingIndex = uri.IndexOf('/', len);
if (domainEndingIndex > 0)
{
recordRequestUri = uri.Substring(0, domainEndingIndex) + recordRequestUri;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move this logic into RecordEntry.Deserialize this would allow us to drop the additional RequestUri property.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no domain name for request uri in track 1 record, so I get domain name from real request uri and add domain name to RequestUri of track 1 for comparison, the problem is we don't know real request uri during de-serializing. Please let me know if I have some misunderstanding.

The real request uri: https://xxx.com/subscriptions/xxxx
RequestUri in track 1: /subscriptions/xxxxx
RequestUri in track 2: https://xxx.com/subscriptions/xxxx

@@ -17,6 +17,9 @@ public class RecordEntry

public string RequestUri { get; set; }

//Used only for deserializing track 1 session record files
public string EncodedRequestUri { get; set; }
Copy link
Contributor

@pakrym pakrym Apr 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's have an explicit bool IsTrack1Recording property.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good suggestion.

@erich-wang
Copy link
Member Author

I really have trouble understanding what we are trying to save by reusing the Track 1 recordings, at the point where running tests Live works it's a matter of running 1 command to update the recordings and get higher quality verification with that.

I'm worried of this code becoming an instant legacy.

There're two reasons to reuse track 1 recordings:

  1. Major reason is to use track 1 recordings to verify the correction of test case migrations;
  2. Imaging for compute/network, there're more than 200 test cases and each test cases may take up to 30 minutes for re-recording, we don't have time to record all of them one-time.

@erich-wang erich-wang force-pushed the erich/test-framework-track1 branch from 267172d to 2cf3255 Compare April 26, 2020 02:25
@pakrym pakrym merged commit 9d389d3 into master Apr 27, 2020
@pakrym pakrym deleted the erich/test-framework-track1 branch April 27, 2020 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants