Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storeSeq & mlBase : clarity refactoring #2954

Merged
merged 2 commits into from
Dec 28, 2021
Merged

storeSeq & mlBase : clarity refactoring #2954

merged 2 commits into from
Dec 28, 2021

Conversation

Cyan4973
Copy link
Contributor

@Cyan4973 Cyan4973 commented Dec 23, 2021

changed ZSTD_storeSeq() interface to accept matchLength instead of mlBase as parameter.

The new interface feels more logical, and removes the need to do - MINMATCH at every call site.
The new contract is checked with an assert().

Also : changed seqDef.matchLength into seqDef.mlBase
since it more accurately describes what is stored in this field (== matchLength - MINMATCH).

instead of mlBase.

This removes the need to do `- MINMATCH` at every call site.

The new interface contract is checked with an `assert()`.
since this is effectively what is stored in this field (== matchLength - MINMATCH).
This makes it clearer what needs to be done when reading from / writing to this field.
@Cyan4973
Copy link
Contributor Author

Cyan4973 commented Dec 24, 2021

There is a second PR coming, focused on offCode sum-type parameter which combines repCode and offset.

Unfortunately, the sum-type numeric representation is a leaky abstractions, with many subtle consequences throughout the code. Even when keeping ambitions low (i.e. not change behavior, just make the code clearer), the resulting PR is still complex, updating a lot of lines across many files.

That's why this PR, focused on matchLength / mlBase is presented first, separately, as it's relatively easier to review on its own.

note : the lone failing test is just a CI issue, travisci seems to have difficulties with all hw aarch64 jobs lately.

Copy link
Contributor

@felixhandte felixhandte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

U16 litLength;
U16 matchLength;
U16 mlBase; /* mlBase == matchLength - MINMATCH */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would a more natural name be mlCode, by analogy to offCode?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mlCode is already taken,
it relates to the FSE code actually entropy-encoded.
mlBase is the starting value that leads to later transformation mlBase => mlCode + mlBits.

@Cyan4973 Cyan4973 merged commit 0e83d15 into dev Dec 28, 2021
@Cyan4973 Cyan4973 deleted the storeSeq_ml branch January 13, 2023 04:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants