Should we standardize on using shared version file for stack docs?

[dedemorton]

During release day, someone always needs to chase down the PRs that contain doc bumps and verify that they've been merged. We could avoid the hassle by using the shared version file for all relevant projects, but this has some problems, too. I've captured my thoughts for discussion. Please add your thoughts! Let's discuss.

Pros:

• One place to update doc paths used across stack docs. Helps release day go more smoothly.
• Doc release manager is more likely to know how the attributes are used and not introduce problems. For example, every few months, someone in dev forgets that the major version should be n.x, not n.0, and the RPM repo downloads break.

Cons:

• Version info lives in a separate repo where contributors can't find it easily unless they understand how our doc build works.
• Most projects still need a separate version file to set versions of other software mentioned in the docs (like go-version, python version, etc). Having a pair of eyes on that file on a regular basis is a good thing.
• Some project, like Beats, use the version info to generate docs, so the version will still need to be bumped in those projects. If the doc bump happens outside of the file, people are more likely to forget to bump the version.
• Making one mistake breaks all projects (not sure if this is a pro or con).
• Each time a new branch is cut, someone will need to update the include to point to the new version file. This is no more work than bumping, but maybe slightly less obvious and intuitive since the shared version files live in a separate repo. The person creating the branch might not know that there are are files named versions67.asciidoc, versions70.asciidoc, and so on. We complicate this further by having a shared attributes file that uses a different versioning strategy (everyone points to attributes.asciidoc unless something changes and they need older attribute settings).

[nik9000]

I'd like to have play with this some. I've had it in the back of my mind for a while but figured I'd wait until after the Asciidoctor migration. I'll think on it some more, and see if I can get it in earlier.

[lcawl]

Having just been affected by this again when the creation of the 7.4 branches here are some thoughts:

Our list of books continues to grow. We have a wiki page that lists where the doc version info is for most of those books, we still inevitably miss some, which can cause broken links and possible delays releasing the new content. To me, this is a reason to try to use a shared file for as many as possible. Otherwise, I think we need to do multiple small updates to the conf.yaml file (i.e. don't block 7.4 book X release because 7.4 book Y attributes aren't up-to-date). Even then, you'd end up with broken links if 7.4 book X links to (non-existent) 7.4 book Y.

[lcawl]

Each time a new branch is cut, someone will need to update the include to point to the new version file.

We have talked in the past about the fact that some of our configuration files (including these shared attribute files) are version-specific. That is to say, if we had these shared files in a repo that had version branches, we would only need one copy of the file (e.g. "attributes.asciidoc" and "versions.asciidoc") and would not need to ensure each other repo is using the appropriately versioned filename.

[bmorelli25]

We have talked in the past about the fact that some of our configuration files (including these shared attribute files) are version-specific. That is to say, if we had these shared files in a repo that had version branches, we would only need one copy of the file (e.g. "attributes.asciidoc" and "versions.asciidoc") and would not need to ensure each other repo is using the appropriately versioned filename.

Is there a downside to this solution? Yes, we miss some of the products that are versioned differently, like cloud, APM Agents, ES Clients, etc., but this simplifies a large portion of the release/branch tasks. That sounds like a big win.

[nik9000]

If it is in a repo that has versioned branches we'd need to include that repo when we build any docs with `--doc`. We *could* do that automatically though. I feel like that "hides" the files more than just adding the version to the name though.

…

On Thu, Aug 29, 2019 at 5:59 PM Brandon Morelli ***@***.***> wrote: We have talked in the past about the fact that some of our configuration files (including these shared attribute files) are version-specific. That is to say, if we had these shared files in a repo that had version branches, we would only need one copy of the file (e.g. "attributes.asciidoc" and "versions.asciidoc") and would not need to ensure each other repo is using the appropriately versioned filename. Is there a downside to this solution? Yes, we miss some of the products that are versioned differently, like cloud, APM Agents, ES Clients, etc., but this simplifies a large portion of the release/branch tasks. That sounds like a big win. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#804?email_source=notifications&email_token=AABUXIXAGYWSRFF7JVQS4SLQHBBDTA5CNFSM4HFCGNTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5P6LQA#issuecomment-526378432>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABUXIX5EIIMCIRNV53R2WTQHBBDTANCNFSM4HFCGNTA> .

[karenzone]

Elastic Common Schema (ECS) is versioned independently of the stack, yet is applicable to several products in the stack. We need the ability to lock ECS versions to stack versions. For example, 7.0 - 7.3 -> ECS 1.0, 7.4+ -> ECS 1.1, etc.

I could easily handle this by setting attributes in Logstash, but that would only perpetuate the problem we're trying to solve.

We currently link to ECS from at least four repos: logstash, beats, kibana, and stack-docs. I expect more.

[nik9000]

We have some ability to `map_branches` right now, but that is more to link from a stack-versioned book to a non-stack-versioned book. I think you could include the attributes file for some particular version of the stack in the ECS docs. That'd make it simple to link outbound to that version. You'd have to pick which version you want to use.

…

On Fri, Aug 30, 2019 at 12:05 PM Karen Metts ***@***.***> wrote: Elastic Common Schema (ECS) is versioned independently of the stack, yet is applicable to several products in the stack. We need the ability to lock ECS versions to stack versions. For example, 7.0 - 7.3 -> ECS 1.0, 7.4+ -> ECS 1.1, etc. I could easily handle this by setting attributes in Logstash, but that would only perpetuate the problem we're trying to solve. We currently link to ECS from at least four repos: logstash, beats, kibana, and stack-docs. I expect more. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#804?email_source=notifications&email_token=AABUXIXZQ6K7NNF23NI2TDTQHFAK3A5CNFSM4HFCGNTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5SCPYY#issuecomment-526657507>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABUXIU6CBQ6CBRCBTOWORTQHFAK3ANCNFSM4HFCGNTA> .

[lcawl]

I think you could include the attributes file for some particular version of the stack in the ECS docs. That'd make it simple to link outbound to that version. You'd have to pick which version you want to use.

From my perspective, the ideal would be to have as many of the shared attributes as possible in a repo that has branches. The branches would align with the stack version branches (e.g. 6.8, 7.4, etc). The attribute files (whether they contain URL paths, release statuses, product versions, feature names, ECS versions, whatever) would therefore not need to have version-specific names. We could reference them once from the appropriate books and not have to worry about updating all the books each time a new branch (and therefore new version file names) came out.

If it's not feasible for the docs repo to have branches like that, I think we should consider some other repo (or create a docs-config kind of repo) that can serve that purpose. If that means that all our individual book builds gain a dependency on that repo, I think it's still worth it. Especially now that we have previews and the majority of our content contributors can see their changes there instead of worrying about doing the builds locally.

[lcawl]

FYI @karenzone and @nik9000 and I met and ended up coming up with another option. Nik implemented #1147 which causes the docs build to determine the appropriate branch and declare a "source_branch" attribute. That attribute value can then be used to find the appropriate shared version attribute file (see #1148). I've tested it in the Stack Overview (see elastic/stack-docs#493) and am pleased with the results. This means we don't have to keep updating the file names -- they're correctly derived.

I am going to implement the same solution in the other books that currently use the shared version attribute files. If this helps anyone else jump on the bandwagon, that's great. If you want more info from me, just let me know.

[bmorelli25]

It turns out that the shared version files introduced in #1147 won't work for APM Server 😢yet. The release target expects a doc branch version in version.asciidoc in order to generate artifacts. The include statement that pulls in the docs version file doesn't work for that purpose. One of the developers said he could point it to the file in the docs repo, but then the build tool wouldn't know how to resolve {apm_server_version} in the APM Server repo.

For now, I've reverted the change to bring tests back to green. More information, along with a proposed solution is in elastic/beats#13765. cc @dedemorton because this will impact you.

[dedemorton]

@bmorelli25 Can I close this in favor of Deb's issue, which was created before this one? #253