UPSTREAM: <carry>: Add OpenShift tooling, images, configs and docs

UPSTREAM: <carry>: Copy hack scripts and tools from openshift/origin

UPSTREAM: <carry>: Fix shellcheck failures for copied openshift-hack bash

UPSTREAM: <carry>: Enable build, test and verify

UPSTREAM: <carry>: Copy README content from origin

UPSTREAM: <carry>: Copy watch-termination command from openshift/origin

UPSTREAM: <carry>: Switch image and rpm build to golang 1.14

UPSTREAM: <carry>: Copy test annotation from origin

UPSTREAM: <carry>: Build openshift-compatible kube e2e binary

UPSTREAM: <carry>: Updating openshift-hack/images/hyperkube/Dockerfile.rhel baseimages to mach ocp-build-data config

UPSTREAM: <carry>: Update test annotation rules

UPSTREAM: <carry>: Enable k8s-e2e-serial

UPSTREAM: <carry>: Update test annotation rules

UPSTREAM: <carry>: Build with golang 1.15

UPSTREAM: <carry>: (squash) Stop installing recent bash and protoc from source

UPSTREAM: <carry>: Add rebase instructions

UPSTREAM: <carry>: (squash) Update README.openshift to reflect transition

UPSTREAM: <carry>: (squash) Stop annotating origin tests with [Suite:openshift]

The detection logic was error-prone (different results based on the
repo existing in GOPATH vs not) and whether a test comes from origin
can be inferred from the absence of the `[Suite:k8s]` tag.

UPSTREAM: <carry>: (squash) Update hyperkube version

UPSTREAM: <carry>: (squash) Update OpenShift docs

UPSTREAM: <carry>: watch-termination: fix deletion race and write non-graceful message also to termination.log

UPSTREAM: <carry>: watch-termination: avoid false positives of NonGracefulTermination events

UPSTREAM: <carry>: (squash) remove servicecatalog e2e that was dropped upstream

UPSTREAM: <carry>: (squash) Retry upstream flakes

UPSTREAM: <carry>: (squash) Add detail to rebase doc

- Add new section 'Maintaining this document'
- Move checklist above the instructions to emphasize their importance
- Add new section 'Reacting to new commits'
- Mention that generated changes in carries should be dropped

.gitignore

@@ -131,3 +131,6 @@ zz_generated_*_test.go
 
 # generated by verify-vendor.sh
 vendordiff.patch
+
+# Ignore openshift source archives produced as part of rpm build
+openshift*.tar.gz

README.openshift.md

@@ -0,0 +1,73 @@
+# OpenShift's fork of k8s.io/kubernetes
+
+This respository contains core Kubernetes components with OpenShift-specific patches.
+
+## Cherry-picking an upstream commit into openshift/kubernetes: Why, how, and when.
+
+`openshift/kubernetes` carries patches on top of each rebase in one of two ways:
+
+1. *periodic rebases* against an upstream Kubernetes tag.  Eventually,
+any code you have in upstream Kubernetes will land in Openshift via
+this mechanism.
+
+2. Cherry-picked patches for important *bug fixes*.  We really try to
+limit feature back-porting entirely. Unless there are exceptional circumstances, your backport should at least be merged in kubernetes master branch. With every carry patch (not included in upstream) you are introducing a maintenance burden for the team managing rebases.
+
+### For Openshift newcomers: Pick my Kubernetes fix into Openshift vs. wait for the next rebase?
+
+Assuming you read the bullets above... If your patch is really far behind, for
+example, if there have been 5 commits modifying the directory you care about,
+cherry picking will be increasingly difficult and you should consider waiting
+for the next rebase, which will likely include the commit you care about or at
+least decrease the amount of cherry picks you need to do to merge.
+
+To really know the answer, you need to know *how many commits behind you are in
+a particular directory*, often.
+
+To do this, just use git log, like so (using pkg/scheduler/ as an example).
+
+```
+MYDIR=pkg/scheduler/algorithm git log --oneline --
+  ${MYDIR} | grep UPSTREAM | cut -d' ' -f 4-10 | head -1
+```
+
+The commit message printed above will tell you:
+
+- what the LAST commit in Kubernetes was (which effected
+"/pkg/scheduler/algorithm")
+- directory, which will give you an intuition about how "hot" the code you are
+cherry picking is.  If it has changed a lot, recently, then that means you
+probably will want to wait for a rebase to land.
+
+### Cherry-picking an upstream change
+
+Since `openshift/kubernetes` closely resembles `k8s.io/kubernetes`,
+cherry-picking largely involves proposing upstream commits in a PR to our
+downstream fork. Other than the usual potential for merge conflicts, the
+commit messages for all commits proposed to `openshift/kubernetes` must
+reflect the following:
+
+- `UPSTREAM: <UPSTREAM PR ID>:` The prefix for upstream commits to ensure
+  correct handling during a future rebase. The person performing the rebase
+  will know to omit a commit with this prefix if the referenced PR is already
+  present in the new base history.
+- `UPSTREAM: <drop>:` The prefix for downstream commits of code that is
+  generated (i.e. via `make update`) or that should not be retained by the
+  next rebase.
+- `UPSTREAM: <carry>:` The prefix for downstream commits that maintain
+  downstream-specific behavior (i.e. to ensure an upstream change is
+  compatible with OpenShift). Commits with this are usually retained across
+  rebases.
+
+## Updating openshift/kubernetes to a new upstream release
+
+Instructions for rebasing `openshift/kubernetes` are maintained in a [separate
+document](REBASE.openshift.md).
+
+## RPM Packaging
+
+A specfile is included in this repo which can be used to produce RPMs
+including the openshift binary. While the specfile will be kept up to
+date with build requirements the version is not updated. Building the
+rpm with the `openshift-hack/build-rpms.sh` helper script will ensure
+that the version is set correctly.

REBASE.openshift.md

@@ -0,0 +1,298 @@
+# Maintaining openshift/kubernetes
+
+OpenShift is based on upstream Kubernetes. With every release of Kubernetes that is
+intended to be shipped as OCP, it is necessary to incorporate the upstream changes
+while ensuring that our downstream customizations are maintained.
+
+## Rebasing for releases < 4.6
+
+The instructions in this document apply to OpenShift releases 4.6 and
+above. For previous releases, please see the [rebase
+enhancement](https://github.com/openshift/enhancements/blob/master/enhancements/rebase.md).
+
+## Maintaining this document
+
+An openshift/kubernetes rebase is a complex process involving many manual and
+potentially error-prone steps. If, while performing a rebase, you find areas where
+the documented procedure is unclear or missing detail, please update this document
+and include the change in the rebase PR. This will ensure that the instructions are
+as comprehensive and accurate as possible for the person performing the next
+rebase.
+
+## Rebase Checklists
+
+The checklists provided below highlight the key responsibilities of
+someone performing an openshift/kubernetes rebase.
+
+In preparation for submitting a PR to the [openshift fork of
+kubernetes](https://github.com/openshift/kubernetes), the following
+should be true:
+
+- [ ] The new rebase branch has been created from the upstream tag
+- [ ] The new rebase branch includes relevant carries from target branch
+- [ ] Dependencies have been updated
+- [ ] Hyperkube dockerfile version has been updated
+- [ ] `make update` has been invoked and the results committed
+- [ ] `make` executes without error
+- [ ] `make verify` executes without error
+- [ ] `make test` executes without error
+- [ ] The upstream tag is pushed to `openshift/kubernetes` to ensure that
+      build artifacts are versioned correctly
+      - Upstream tooling uses the value of the most recent tag (e.g. `v1.20.0`)
+        in the branch history as the version of the binaries it builds.
+      - Pushing the tag is easy as
+```
+git push git@github.com:openshift/kubernetes.git refs/tags/v1.20.0
+```
+
+Details to include in the description of the PR:
+
+- [ ] A link to the rebase spreadsheet for the benefit for reviewers
+
+After the rebase PR has merged to `openshift/kubernetes`, vendor the changes
+into `openshift/origin` to ensure that the openshift-tests binary reflects
+the upstream test changes introduced by the rebase:
+
+- [ ] Find the SHA of the merge commit after your PR lands in `openshift/kubernetes`
+- [ ] Run `hack/update-kube-vendor.sh <o/k SHA>` in a clone of the `origin`
+      repo and commit the results
+- [ ] Run `make update` and commit the results
+- [ ] Submit as a PR to `origin`
+
+As a final step, send an email to the aos-devel mailing list announcing the
+rebase. Make sure to include:
+
+- [ ] The new version of upstream Kubernetes that OpenShift is now based on
+- [ ] Link(s) to upstream changelog(s) detailing what has changed since the last rebase landed
+- [ ] A reminder to component maintainers to bump their dependencies
+- [ ] Relevant details of the challenges involved in landing the rebase that
+      could benefit from a wider audience.
+
+## Getting started
+
+Before incorporating upstream changes you may want to:
+
+- Read this document
+- Get familiar with tig (text-mode interface for git)
+- Find the best tool for resolving merge conflicts
+- Use diff3 conflict resolution strategy
+   (https://blog.nilbus.com/take-the-pain-out-of-git-conflict-resolution-use-diff3/)
+- Teach Git to remember how you’ve resolved a conflict so that the next time it can
+  resolve it automatically (https://git-scm.com/book/en/v2/Git-Tools-Rerere)
+
+## Preparing the local repo clone
+
+Clone from a personal fork of kubernetes via a pushable (ssh) url:
+
+```
+git clone git@github.com:<user id>/kubernetes
+```
+
+Add a remote for upstream and fetch its branches:
+
+```
+git remote add --fetch upstream https://github.com/kubernetes/kubernetes
+```
+
+Add a remote for the openshift fork and fetch its branches:
+
+```
+git remote add --fetch openshift https://github.com/openshift/kubernetes
+```
+
+## Creating a new local branch for the new rebase
+
+- Branch the target `k8s.io/kubernetes` release tag (e.g. `v1.20.0`) to a new
+  local branch
+
+```
+git checkout -b rebase-1.20.0 v1.20.0
+```
+
+- Merge `openshift(master)` branch into the `rebase-1.20.0` branch with merge
+  strategy `ours`. It discards all changes from the other branch (`openshift/master`)
+  and create a merge commit. This leaves the content of your branch unchanged,
+  and when you next merge with the other branch, Git will only consider changes made
+  from this point forward.  (Do not confuse this with `ours` conflict resolution
+  strategy for `recursive` merge strategy, `-X` option.)
+
+```
+git merge -s ours openshift/master
+```
+
+## Creating a spreadsheet of carry commits from the previous release
+
+Given the upstream tag (e.g. `v1.19.2`) of the most recent rebase and the name
+of the branch that is targeted for rebase (e.g. `openshift/master`), generate a tsv file
+containing the set of carry commits that need to be considered for picking:
+
+```
+echo 'Comment Sha\tAction\tClean\tSummary\tCommit link\tPR link' > ~/Documents/v1.19.2.tsv
+```
+```
+git log $( git merge-base openshift/master v1.19.2 )..openshift/master --ancestry-path --reverse --no-merges --pretty='tformat:%x09%h%x09%x09%x09%s%x09https://github.com/openshift/kubernetes/commit/%h?w=1' | grep -E $'\t''UPSTREAM: .*'$'\t' | sed -E 's~UPSTREAM: ([0-9]+)(:.*)~UPSTREAM: \1\2\thttps://github.com/kubernetes/kubernetes/pull/\1~' >> ~/Documents/v1.19.2.tsv
+```
+
+This tsv file can be imported into a google sheets spreadsheet to track the
+progress of picking commits to the new rebase branch. The spreadsheet can also
+be a way of communicating with rebase reviewers. For an example of this
+communication, please see the [the spreadsheet used for the 1.19
+rebase](https://docs.google.com/spreadsheets/d/10KYptJkDB1z8_RYCQVBYDjdTlRfyoXILMa0Fg8tnNlY/edit).
+
+## Picking commits from the previous rebase branch to the new branch
+Go through the spreadsheet and for every commit set one of the appropriate actions:
+ - `p`, to pick the commit
+ - `s`, to squash it (add a comment with the sha of the target)
+ - `d`, to drop the commit (if it is not obvious, comment why)
+
+Set up conditional formatting in the google sheet to color these lines appropriately.
+
+Commits carried on rebase branches have commit messages prefixed as follows:
+
+- `UPSTREAM: <carry>:`
+  - A persistent carry that should probably be picked for the subsequent rebase branch.
+  - In general, these commits are used to modify behavior for consistency or
+    compatibility with openshift.
+- `UPSTREAM: <drop>:`
+  - A carry that should probably not be picked for the subsequent rebase branch.
+  - In general, these commits are used to maintain the codebase in ways that are
+    branch-specific, like the update of generated files or dependencies.
+- `UPSTREAM: 77870:`
+  - The number identifies a PR in upstream kubernetes
+    (i.e. `https://github.com/kubernetes/kubernetes/pull/<pr id>`)
+  - A commit with this message should only be picked into the subsequent rebase branch
+    if the commits of the referenced PR are not included in the upstream branch.
+  - To check if a given commit is included in the upstream branch, open the referenced
+    upstream PR and check any of its commits for the release tag (e.g. `v.1.20.0`)
+    targeted by the new rebase branch. For example:
+    - <img src="openshift-hack/commit-tag.png">
+
+With these guidelines in mind, pick the appropriate commits from the previous rebase
+branch into the new rebase branch. Create a new filter view in the spreadsheet to allow
+you get a view where `Action==p || Action==s` and copy paste the shas to `git cherry-pick`
+command. Use `tr '\n' ' ' <<< "<line_separated_commits>"` to get a space separated list
+from the copy&paste.
+
+Where it makes sense to do so, squash carried changes that are tightly coupled to
+simplify future rebases. If the commit message of a carry does not conform to
+expectations, feel free to revise and note the change in the spreadsheet row for the
+commit.
+
+If you first pick all the pick+squash commits first and push them for review it is easier for you
+and your reviewers to check the code changes and you squash it at the end.
+
+Explicit commit rules:
+- Anything touching `openshift-hack/`, openshift specific READMEs or similar files
+  should be squashed to 1 commit named "UPSTREAM: <carry>: Add OpenShift specific files"
+- Updating generated files coming from kubernetes should be `<drop>` commit
+- Generated changes should never be mixed with non-generated changes. If a carry is
+  ever seen to contain generated changes, those changes should be dropped.
+
+## Update the hyperkube image version to the release tag
+
+The [hyperkube dockerfile](openshift-hack/images/hyperkube/Dockerfile.rhel)
+hard-codes the Kubernetes version in an image label. It's necessary to manually
+set this label to the new release tag. Prefix the commit summary with
+`UPSTREAM: <carry>: (squash)` and squash it before merging the rebase PR.
+
+## Updating dependencies
+
+Once the commits are all picked from the previous rebase branch, and your PR
+is mostly ready, each of the following repositories need to be updated to depend
+on the upstream tag targeted by the rebase:
+
+- https://github.com/openshift/api
+- https://github.com/openshift/apiserver-library-go
+- https://github.com/openshift/client-go
+- https://github.com/openshift/library-go
+
+Often these repositories are updated in parallel by other team members, so make
+sure to ask around before starting the work of bumping their dependencies.
+
+Once the above repos have been updated to depend on the target release,
+it will be necessary to update `go.mod` to point to the appropriate revision
+of these repos by running `hack/pin-dependency.sh` for each of them and then running
+`hack/update-vendor.sh` (as per the [upstream documentation](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/vendor.md#adding-or-updating-a-dependency)).
+
+Make sure to commit the result of a vendoring update with `UPSTREAM: <drop>: bump(*)`.
+If you have already bumped the dependencies to get the repo to compile,
+don't forget to squash the commits before merging the PR.
+
+### Updating dependencies for pending bumps
+
+The upstream `hack/pin-dependency.sh` script only supports setting dependency
+for the original repository. To pin to a fork branch that has not yet been
+merged (i.e. to test a rebase ahead of shared library bumps having merged), the
+following `go mod` invocations are suggested:
+
+```
+go mod edit -replace github.com/openshift/<lib>=github.com/<username>/<lib>@SHA
+go mod tidy && go mod vendor
+```
+
+Alternatively, you can edit `go.mod` file manually with your favourite editor and use search&replace.
+
+## Review test annotation rules
+
+The names of upstream e2e tests are annotated according to the a set of
+[declarative rules](openshift-hack/e2e/annotate/rules.go). These annotations
+are used to group tests into suites and to skip tests that are known not to be
+incompatible with some or all configurations of OpenShift.
+
+When performing a rebase, it is important to review the rules to
+ensure they are still relevant:
+
+- [ ] Ensure that `[Disabled:Alpha]` rules are appropriate for the current kube
+      level. Alpha features that are not enabled by default should be targeted
+      by this annotation to ensure that tests of those features are skipped.
+- [ ] Add new skips (along with a bz to track resolution) where e2e tests fail
+      consistently.
+
+Test failures representing major issues affecting cluster capability will
+generally need to be addressed before merge of the rebase PR, but minor issues
+(e.g. tests that fail to execute correctly but don't appear to reflect a
+regression in behavior) can often be skipped and addressed post-merge.
+
+## Updating generated files
+
+- Update generated files by running `make update`
+  - This step depends on etcd being installed in the path, which can be
+    accomplished by running `hack/install-etcd.sh`.
+  - Alternatively, run it in the same container as CI is using for build_root that already has
+    the etcd at correct version
+```
+podman run -it --rm -v $( pwd ):/go/k8s.io/kubernetes:Z --workdir=/go/k8s.io/kubernetes registry.svc.ci.openshift.org/openshift/release:rhel-8-release-golang-1.15-openshift-4.7 make update OS_RUN_WITHOUT_DOCKER=yes
+```
+- Commit the resulting changes as `UPSTREAM: <drop>: make update`.
+
+## Building and testing
+
+- Build the code with `make`
+- Test the code with `make test`
+  - Where test failures are encountered and can't be trivially resolved, the
+    spreadsheet can be used to track those failures to their resolution. The
+    example spreadsheet should have a sheet that demonstrates this tracking.
+  - Where a test failure proves challenging to fix without specialized knowledge,
+    make sure to coordinate with the team(s) responsible for area(s) of focus
+    exhibiting test failure. If in doubt, ask for help!
+- Verify the code with `make verify`
+
+## Reacting to new commits
+
+Inevitably, a rebase will take long enough that new commits will end up being
+merged to the targeted openshift/kubernetes branch after the rebase is
+underway. The following strategy is suggested to minimize the cost of incorporating
+these new commits:
+
+- rename existing rebase branch (e.g. 1.20.0-beta.2 -> 1.20.0-beta.2-old)
+- create new rebase branch from HEAD of master
+- merge the target upstream tag (e.g. 1.20.0-beta.2) with strategy ours
+- pick all carries from renamed rebase branch (e.g. 1.20.0-beta.2-old)
+- pick new carries from the openshift/kubernetes target branch
+- add details of the new carries to the spreadsheet
+- update generated files
+
+With good tooling, the cost of this procedure should be ~10 minutes at
+most. Re-picking carries should not result in conflicts since the base of the
+rebase branch will be the same as before. The only potential sources of conflict
+will be the newly added commits.

build/run.sh

@@ -25,6 +25,12 @@ set -o pipefail
 KUBE_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
 source "$KUBE_ROOT/build/common.sh"
 
+# Allow running without docker (e.g. in openshift ci)
+if [[ "${OS_RUN_WITHOUT_DOCKER:-}" ]]; then
+  "${@}"
+  exit 0
+fi
+
 KUBE_RUN_COPY_OUTPUT="${KUBE_RUN_COPY_OUTPUT:-y}"
 
 kube::build::verify_prereqs