Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ORT/atstccfg Deterministic Config Generation #4557

Merged
merged 5 commits into from
May 26, 2020

Conversation

rob05c
Copy link
Member

@rob05c rob05c commented Mar 27, 2020

This changes ORT config gen to be deterministic. Configs should always be identical, except for the timestamp in the comment header.

This should make future changes much safer and easier to identify, easier to identify bugs, and just easier and safer to diff configs to see changes in general. With deterministic gen, diffing should be as simple as a line diff, e.g. the POSIX diff command.

  • This PR is not related to any other Issue

Includes tests. No docs, no changelog, no documented interface change (we don't document config order).

Which Traffic Control components are affected by this PR?

  • Traffic Ops ORT

What is the best way to verify this PR?

Run ORT and/or it's helper atstccfg multiple times, and verify the output files are identical except for the comment header timestamp.

If this is a bug fix, what versions of Traffic Control are affected?

Not a bug fix.

The following criteria are ALL met by this PR

  • This PR includes tests OR I have explained why tests are unnecessary
  • This PR includes documentation OR I have explained why documentation is unnecessary
  • This PR includes an update to CHANGELOG.md OR such an update is not necessary
  • This PR includes any and all required license headers
  • This PR ensures that database migration sequence is correct OR this PR does not include a database migration
  • This PR DOES NOT FIX A SERIOUS SECURITY VULNERABILITY (see the Apache Software Foundation's security guidelines for details)

Additional Information

@rob05c rob05c added new feature A new feature, capability or behavior Traffic Ops ORT *DEPRECATED* related to the traffic_ops_ort.pl script labels Mar 27, 2020
rob05c added 5 commits March 31, 2020 12:11
Adds a header to the multipart file, with the line comment syntax of
the file (if it has one).

This lets ORT safely strip line comments with times for diffing.
Removes TCCfg from GetAllConfigs, making the function Pure, which
makes it possible/easier to unit test.
@rob05c rob05c force-pushed the ort-deterministic-config-gen branch from b5323b6 to ddd2223 Compare March 31, 2020 18:11
Copy link
Contributor

@alficles alficles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a lot of changes here, but it's really just a lot of the same change. There are some oddities, like the hard-coded boundary, but I don't see a great way around it. If we document the atstccfg interface, this has implications for that, but it's not currently documented, so there's nothing to update.

Looks good to me.


// Note LineCommentLogsDotXML must be a single-line comment!
// But this file doesn't have a single-line format, so we use <!-- for the header and promise it's on a single line
// Note! if this file is ever changed to have multi-line comments, LineCommentLogsDotXML will have to be changed to the empty string.
hdrComment := GenericHeaderComment(profileName, toToolName, toURL)
hdrComment = strings.Replace(hdrComment, `# `, ``, -1)
hdrComment = strings.Replace(hdrComment, "\n", ``, -1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably just replace # with <!-- and \n with -->\n? That would make it at least work in the future if someone returns a multiline comment. This works, though, doesn't change current behaviour, and won't break in the future. This is fine, though.

@dneuman64
Copy link
Contributor

Good enough for Chris, good enough for me 👍

@dneuman64 dneuman64 merged commit d74de45 into apache:master May 26, 2020
@zrhoffman zrhoffman mentioned this pull request Jun 10, 2020
17 tasks
rob05c added a commit to rob05c/trafficcontrol that referenced this pull request Sep 17, 2020
* Make ORT gen deterministic

* Add ORT/atstccfg Line-Comment header

Adds a header to the multipart file, with the line comment syntax of
the file (if it has one).

This lets ORT safely strip line comments with times for diffing.

* Remove ORT/atstccfg unnecessary io param

Removes TCCfg from GetAllConfigs, making the function Pure, which
makes it possible/easier to unit test.

* Change ORT/atstccfg magic string to const

* Add ORT/atstccfg unit test for determinism

(cherry picked from commit d74de45)
rawlinp pushed a commit that referenced this pull request Sep 18, 2020
* Make ORT gen deterministic

* Add ORT/atstccfg Line-Comment header

Adds a header to the multipart file, with the line comment syntax of
the file (if it has one).

This lets ORT safely strip line comments with times for diffing.

* Remove ORT/atstccfg unnecessary io param

Removes TCCfg from GetAllConfigs, making the function Pure, which
makes it possible/easier to unit test.

* Change ORT/atstccfg magic string to const

* Add ORT/atstccfg unit test for determinism

(cherry picked from commit d74de45)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new feature A new feature, capability or behavior Traffic Ops ORT *DEPRECATED* related to the traffic_ops_ort.pl script
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants