Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 1.1.1 issue template and input manifests #1275

Merged
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
6987c9a
Add 1.1.1 related templates and manifests
peterzhuamazon Dec 6, 2021
f2da32e
More changes to template and bundle code to make build 1.1.1 opensear…
peterzhuamazon Dec 7, 2021
d3ac3fa
Add more changes
peterzhuamazon Dec 8, 2021
49e49a5
Add arm64 and docker build code
peterzhuamazon Dec 8, 2021
a169e38
Restore Jenkinsfile
peterzhuamazon Dec 8, 2021
7ef3b40
Merge branch 'main' of github.com:peterzhuamazon/opensearch-build int…
peterzhuamazon Dec 8, 2021
d80baf9
Restore Jenkinsfile
peterzhuamazon Dec 8, 2021
450c964
Restore build from source python code
peterzhuamazon Dec 8, 2021
5220b27
Add 1.1.1 related templates and manifests
peterzhuamazon Dec 6, 2021
c48dcbd
More changes to template and bundle code to make build 1.1.1 opensear…
peterzhuamazon Dec 7, 2021
d166b9b
Add more changes
peterzhuamazon Dec 8, 2021
423adfe
Add arm64 and docker build code
peterzhuamazon Dec 8, 2021
438bc16
Restore Jenkinsfile
peterzhuamazon Dec 8, 2021
b09dfab
Restore Jenkinsfile
peterzhuamazon Dec 8, 2021
6ff89d3
Restore build from source python code
peterzhuamazon Dec 8, 2021
9633ec8
Readd functional tests
peterzhuamazon Dec 8, 2021
c467c07
Readd functional tests
peterzhuamazon Dec 8, 2021
8024c90
Switch checks of build from source to gradle related checks
peterzhuamazon Dec 8, 2021
a01c1af
Remove checks for dashboards as it is not there yet
peterzhuamazon Dec 8, 2021
10c4f4d
Fix ci check link is not matching the new setup in s3
peterzhuamazon Dec 8, 2021
81079e1
Merge branch 'main' of github.com:peterzhuamazon/opensearch-build int…
peterzhuamazon Dec 8, 2021
3ec4b08
Add more changes to resolve issues with templates and tests
peterzhuamazon Dec 8, 2021
92b285e
More updates to the templates sent to owners
peterzhuamazon Dec 9, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions .github/meta_issue_template/release-1.1.1-plugin-issues-min.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
This is a component issue for release 1.1.1.
peterzhuamazon marked this conversation as resolved.
Show resolved Hide resolved
Coming from [release issue 1.1.1](https://github.com/opensearch-project/opensearch-build/issues/870), release version 1.1.1. Please follow the following checklist.

<details><summary>How to use this component issue</summary>
<p>

## This Component Issue
peterzhuamazon marked this conversation as resolved.
Show resolved Hide resolved
This component issue captures the state of the OpenSearch release, on component/plugin level, its assignee is responsible for driving the release of the component. Please contact them or @mention them on this issue for help.

## Release Steps
There are several steps to the release process, components that are behind present risk to the release. Component owners resolve tasks on this ticket to communicate with the overall release owner.

Steps have completion dates for coordinating efforts between the components of a release; components can start as soon as they are ready far in advance of a future release.

You can find all the corresponding dates of each step in the release issue above.

</p>
</details>


### You can find all the date in above issue
### (We make changes to both OpenSearch and OpenSearch-Dashboards for 1.1.1)
peterzhuamazon marked this conversation as resolved.
Show resolved Hide resolved

### Preparation

- [ ] Assign this issue to a release owner.
- [ ] All the tasks in this issue have been reviewed by the release owner.
- [ ] Create, update, triage and label all features and issues targeted for this release with v1.1.1.

### CI/CD

- [ ] If including changes in this release, increment plugin version on 1.1 branch to `1.1.1`, otherwise stay on `1.1.0`.
peterzhuamazon marked this conversation as resolved.
Show resolved Hide resolved
peterzhuamazon marked this conversation as resolved.
Show resolved Hide resolved
- [ ] All code changes for 1.1.1 are complete.
- [ ] Ensure working and passing CI.

### Pre-Release

- [ ] Merge any changes to 1.1 you need for the patch
peterzhuamazon marked this conversation as resolved.
Show resolved Hide resolved
- [ ] Complete integration and sanity tests, and update results in the comment, contact corresponding assigner in meta issue above if needed.
peterzhuamazon marked this conversation as resolved.
Show resolved Hide resolved
- [ ] Fix bugs that you find during the integration test, wait for a new build and test again.
peterzhuamazon marked this conversation as resolved.
Show resolved Hide resolved
- [ ] All intermittent test failures have issues filed

### Release

- [ ] Complete [documentation](https://github.com/opensearch-project/documentation-website).
- [ ] Gather, review and publish release notes.
- [ ] Verify all issued labeled for this release are closed or labelled for the next release.

### Post Release

- [ ] Create [a release tag](https://github.com/opensearch-project/.github/blob/main/RELEASING.md#tagging).
- [ ] Suggest improvements to [this template](https://github.com/opensearch-project/opensearch-build/blob/main/.github/ISSUE_TEMPLATE/release_template.md).
- [ ] Conduct a postmortem, and publish its results.
53 changes: 53 additions & 0 deletions .github/meta_issue_template/release-1.1.1-plugin-issues.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
This is a component issue for release 1.1.1.
Coming from [release issue 1.1.1](https://github.com/opensearch-project/opensearch-build/issues/870), release version 1.1.1. Please follow the following checklist.

<details><summary>How to use this component issue</summary>
<p>

## This Component Issue
This component issue captures the state of the OpenSearch release, on component/plugin level, its assignee is responsible for driving the release of the component. Please contact them or @mention them on this issue for help.

## Release Steps
There are several steps to the release process, components that are behind present risk to the release. Component owners resolve tasks on this ticket to communicate with the overall release owner.

Steps have completion dates for coordinating efforts between the components of a release; components can start as soon as they are ready far in advance of a future release.

You can find all the corresponding dates of each step in the release issue above.

</p>
</details>


### You can find all the date in above issue
### (We make changes to both OpenSearch and OpenSearch-Dashboards for 1.1.1)

### Preparation

- [ ] Assign this issue to a release owner.
- [ ] All the tasks in this issue have been reviewed by the release owner.
- [ ] Create, update, triage and label all features and issues targeted for this release with v1.1.1.

### CI/CD

- [ ] If including changes in this release, increment plugin version on 1.1 branch to `1.1.1.0`, otherwise stay on `1.1.0.0`.
- [ ] All code changes for 1.1.1 are complete.
- [ ] Ensure working and passing CI.

### Pre-Release

- [ ] Merge any changes to 1.1 you need for the patch
- [ ] Complete integration and sanity tests, and update results in the comment, contact corresponding assigner in meta issue above if needed.
- [ ] Fix bugs that you find during the integration test, wait for a new build and test again.
- [ ] All intermittent test failures have issues filed

### Release

- [ ] Complete [documentation](https://github.com/opensearch-project/documentation-website).
- [ ] Gather, review and publish release notes.
- [ ] Verify all issued labeled for this release are closed or labelled for the next release.

### Post Release

- [ ] Create [a release tag](https://github.com/opensearch-project/.github/blob/main/RELEASING.md#tagging).
- [ ] Suggest improvements to [this template](https://github.com/opensearch-project/opensearch-build/blob/main/.github/ISSUE_TEMPLATE/release_template.md).
- [ ] Conduct a postmortem, and publish its results.
55 changes: 55 additions & 0 deletions .github/meta_issue_template/release-1.3.0-plugin-issues.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
This is a component issue for release 1.3.0.
peternied marked this conversation as resolved.
Show resolved Hide resolved
Coming from [release issue 1.3.0](https://github.com/opensearch-project/opensearch-build/issues/870), release version 1.3.0. Please follow the following checklist.

<details><summary>How to use this component issue</summary>
<p>

## This Component Issue
This component issue captures the state of the OpenSearch release, on component/plugin level, its assignee is responsible for driving the release of the component. Please contact them or @mention them on this issue for help.

## Release Steps
There are several steps to the release process, components that are behind present risk to the release. Component owners resolve tasks on this ticket to communicate with the overall release owner.

Steps have completion dates for coordinating efforts between the components of a release; components can start as soon as they are ready far in advance of a future release.

You can find all the corresponding dates of each step in the release issue above.

</p>
</details>


### You can find all the date in above issue

### Preparation

- [ ] Assign this issue to a release owner.
- [ ] Finalize scope and feature set and update [the Public Roadmap](https://github.com/orgs/opensearch-project/projects/1).
- [ ] All the tasks in this issue have been reviewed by the release owner.
- [ ] Create, update, triage and label all features and issues targeted for this release with v1.3.0.

### CI/CD

- [ ] If including changes in this release, increment plugin version on 1.3 branch to `1.3.0.0`, otherwise stay on `1.3.0.0`.
- [ ] All code changes for 1.3.0 are complete.
- [ ] Ensure working and passing CI.
- [ ] Re(add) this repo to the (if not exist) [opensearch-dashboards input manifest](https://github.com/opensearch-project/opensearch-build/blob/main/manifests/1.3.0/opensearch-dashboards-1.3.0.yml).

### Pre-Release

- [ ] Update your branch in the [opensearch-dashboards input manifest](https://github.com/opensearch-project/opensearch-build/blob/main/manifests/1.3.0/opensearch-dashboards-1.3.0.yml).
- [ ] Merge any changes to 1.3 you need for the patch
- [ ] Complete integration tests, and update results in the comment, contact corresponding assigner in meta issue above if needed.
- [ ] Fix bugs that you find during the integration test, wait for a new build and test again.
- [ ] All intermittent test failures have issues filed

### Release

- [ ] Complete [documentation](https://github.com/opensearch-project/documentation-website).
- [ ] Gather, review and publish release notes.
- [ ] Verify all issued labeled for this release are closed or labelled for the next release.

### Post Release

- [ ] Create [a release tag](https://github.com/opensearch-project/.github/blob/main/RELEASING.md#tagging).
- [ ] Suggest improvements to [this template](https://github.com/opensearch-project/opensearch-build/blob/main/.github/ISSUE_TEMPLATE/release_template.md).
- [ ] Conduct a postmortem, and publish its results.
51 changes: 26 additions & 25 deletions manifests/1.1.1/opensearch-1.1.1.yml
Original file line number Diff line number Diff line change
@@ -1,73 +1,74 @@
---
schema-version: "1.0"
ci:
image:
name: "opensearchstaging/ci-runner:centos7-x64-arm64-jdkmulti-node10.24.1-cypress6.9.1-20211028"
build:
name: OpenSearch
version: 1.1.1
patches:
- 1.1.0
components:
- name: OpenSearch
repository: https://github.com/opensearch-project/OpenSearch.git
ref: "1.1"
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- gradle:publish
- gradle:properties:version
- manifest:component
- name: common-utils
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: job-scheduler
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: dashboards-reports
repository: https://github.com/opensearch-project/dashboards-reports.git
working_directory: reports-scheduler
ref: "1.1"
platforms:
- linux
checks:
- gradle:publish
- gradle:properties:version
- name: sql
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: alerting
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: security
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: cross-cluster-replication
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
checks:
- manifest:component
- name: performance-analyzer-rca
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: performance-analyzer
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: index-management
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: k-NN
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
platforms:
- darwin
- linux
- name: anomaly-detection
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: asynchronous-search
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
checks:
- manifest:component
- name: dashboards-reports
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: dashboards-notebooks
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
38 changes: 38 additions & 0 deletions manifests/1.1.1/opensearch-dashboards-1.1.1.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
schema-version: "1.0"
ci:
image:
name: "opensearchstaging/ci-runner:centos7-x64-arm64-jdkmulti-node10.24.1-cypress6.9.1-20211028"
build:
name: OpenSearch Dashboards
version: 1.1.1
patches:
- 1.1.0
components:
- name: reportsDashboards
repository: https://github.com/opensearch-project/dashboards-reports
ref: "1.1"
working_directory: "dashboards-reports"
platforms:
- linux
- name: OpenSearch-Dashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: alertingDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: securityDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: indexManagementDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: queryWorkbenchDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: notebooksDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: ganttChartDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: traceAnalyticsDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: anomalyDetectionDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: functionalTestDashboards
repository: https://github.com/opensearch-project/opensearch-dashboards-functional-test.git
ref: "main"
4 changes: 3 additions & 1 deletion src/ci_workflow/ci_check_manifest_component.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@ def __init__(self, component, url):

def check(self):
for architecture in BuildArgs.SUPPORTED_ARCHITECTURES:
url = "/".join([self.component.dist, architecture, "manifest.yml"])
# Since we only have 'linux' builds now we hard code it to 'linux'
# Once we have all platform builds we can adding a second loop for 'BuildArgs.SUPPORTED_PLATFORMS'
Copy link
Member

@dblock dblock Dec 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can it be self.target.platform instead of hard-coded linux, and then the comment is moot?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This target is CiTarget, which does not have the platform var.

url = "/".join([self.component.dist, "linux", architecture, "builds", self.target.name, "manifest.yml"])
self.build_manifest = BuildManifest.from_url(url)
if self.component.name in self.build_manifest.components:
logging.info(f"Found {self.component.name} in {url}.")
Expand Down
2 changes: 1 addition & 1 deletion src/ci_workflow/ci_input_manifest.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ def __init__(self, file, args):

def __check__(self):

target = CiTarget(version=self.manifest.build.version, snapshot=self.args.snapshot)
target = CiTarget(version=self.manifest.build.version, name=self.manifest.build.name.lower().replace(' ', '-'), snapshot=self.args.snapshot)
peterzhuamazon marked this conversation as resolved.
Show resolved Hide resolved

with TemporaryDirectory(keep=self.args.keep, chdir=True) as work_dir:
logging.info(f"Sanity-testing in {work_dir.name}")
Expand Down
4 changes: 3 additions & 1 deletion src/ci_workflow/ci_target.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,12 @@

class CiTarget:
version: str
name: str
snapshot: bool

def __init__(self, version, snapshot=True):
def __init__(self, version, name, snapshot=True):
self.version = version
self.name = name
self.snapshot = snapshot

@property
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ def __mock_dependencies(self, props="", snapshot=False, gradle_project=None):
return TestCiCheckGradleDependencies.DummyDependencies(
component=MagicMock(),
git_repo=git_repo,
target=CiTarget(version="1.1.0", snapshot=snapshot),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=snapshot),
args=gradle_project,
)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ def __mock_check(self, props=None):
return CiCheckGradleDependenciesOpenSearchVersion(
component=MagicMock(),
git_repo=MagicMock(),
target=CiTarget(version="1.1.0", snapshot=True),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=True),
args=None,
)

Expand Down Expand Up @@ -52,7 +52,7 @@ def test_executes_gradle_command(self):
check = CiCheckGradleDependenciesOpenSearchVersion(
component=MagicMock(),
git_repo=MagicMock(),
target=CiTarget(version="1.1.0", snapshot=True),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=True),
args=None,
)
check.git_repo.output.assert_called_once_with(
Expand All @@ -63,7 +63,7 @@ def test_executes_gradle_command_with_arg(self):
check = CiCheckGradleDependenciesOpenSearchVersion(
component=MagicMock(),
git_repo=MagicMock(),
target=CiTarget(version="1.1.0", snapshot=True),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=True),
args="plugin",
)
check.git_repo.output.assert_called_once_with(
Expand Down
4 changes: 2 additions & 2 deletions tests/tests_ci_workflow/test_ci_check_gradle_properties.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ def test_executes_gradle_properties(self):
TestCiCheckGradleProperties.DummyProperties(
component=MagicMock(),
git_repo=git_repo,
target=CiTarget(version="1.1.0", snapshot=False),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=False),
)

git_repo.output.assert_called_once_with("./gradlew properties -Dopensearch.version=1.1.0 -Dbuild.snapshot=false")
Expand All @@ -35,7 +35,7 @@ def test_executes_gradle_properties_snapshot(self):
TestCiCheckGradleProperties.DummyProperties(
component=MagicMock(),
git_repo=git_repo,
target=CiTarget(version="1.1.0", snapshot=True),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=True),
)

git_repo.output.assert_called_once_with("./gradlew properties -Dopensearch.version=1.1.0-SNAPSHOT -Dbuild.snapshot=true")
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ def __mock_check(self, props=None, component=None, snapshot=True):
return CiCheckGradlePropertiesVersion(
component=component or MagicMock(),
git_repo=MagicMock(),
target=CiTarget(version="1.1.0", snapshot=snapshot),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=snapshot),
)

def test_has_version(self):
Expand Down
Loading