Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore] Schema Processor Revamp [Part 3] - Translation #37403

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

ankitpatel96
Copy link
Contributor

@ankitpatel96 ankitpatel96 commented Jan 22, 2025

Description

This is a slice of changes from #35248
This PR details Translation which defines the complete abstraction of schema translation file. It manages Revisions to move between two versions of a schema

Testing

Unit tests

@github-actions github-actions bot added the processor/schema Schema processor label Jan 22, 2025
@ankitpatel96 ankitpatel96 changed the title split out translation chunk Schema Processor Revamp [Part 1] - Translation Jan 22, 2025
@ankitpatel96 ankitpatel96 changed the title Schema Processor Revamp [Part 1] - Translation [chore] Schema Processor Revamp [Part 3] - Translation Jan 22, 2025
@ankitpatel96 ankitpatel96 marked this pull request as ready for review January 22, 2025 16:19
@ankitpatel96 ankitpatel96 requested a review from a team as a code owner January 22, 2025 16:19
Copy link
Member

@tigrannajaryan tigrannajaryan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM, a few small comments.

Comment on lines 197 to 203
err = rev.all.Rollback(log)
if err != nil {
return err
}
err = rev.logs.Rollback(log)
if err != nil {
return err
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see in ApplyAllResourceChanges the Revert case calls Rollback() funcs in the reverse order compared to the order the Update case calls Apply funcs. I can't tell immediately if it matters or no, but that order reversal intuitively sounds right to me.

Why are we not doing the same here (and elsewhere)? Is it important or no? Please make it consistent (and correct - if order matters). :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does matter! Nice catch, thanks

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ordering still seems to be inconsistent. Here Revert does all, then logs. For spans Revert does spans, then all.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was an accident. I've double checked it and I think I fixed it all.

@ankitpatel96 ankitpatel96 force-pushed the ankit-schema-processor-3-translation branch from bd3c593 to 6a07e81 Compare January 26, 2025 18:48
Comment on lines 197 to 203
err = rev.all.Rollback(log)
if err != nil {
return err
}
err = rev.logs.Rollback(log)
if err != nil {
return err
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ordering still seems to be inconsistent. Here Revert does all, then logs. For spans Revert does spans, then all.

)

//go:embed testdata
var testdataFiles embed.FS
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: do we need to embed the test files or they can be just read as regular files?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not the most experienced go developer - is there any reason this is considered bad practice? I just thought it was convenient.

)
require.NoError(t, err, "Must not error when creating translator")

fixture.ParallelRaceCompute(t, 10, func() error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where in the current implementation do we have concurrency that needs race testing? I am probably missing it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there is any to be honest - a lot of this part's code is the contribution of @MovieStoreGuy so maybe he could speak as to his intention. Within this part, there's no concurrency. In the full processor, a lot of this code can be called under concurrent conditions (processors in general get called concurrently). The Manager piece gets called concurrently and has some logic around that.

@ankitpatel96 ankitpatel96 force-pushed the ankit-schema-processor-3-translation branch from c1bd4df to ee7947e Compare January 27, 2025 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
processor/schema Schema processor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants