Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 1.1.1 issue template and input manifests #1275

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
6987c9a
Add 1.1.1 related templates and manifests
peterzhuamazon Dec 6, 2021
f2da32e
More changes to template and bundle code to make build 1.1.1 opensear…
peterzhuamazon Dec 7, 2021
d3ac3fa
Add more changes
peterzhuamazon Dec 8, 2021
49e49a5
Add arm64 and docker build code
peterzhuamazon Dec 8, 2021
a169e38
Restore Jenkinsfile
peterzhuamazon Dec 8, 2021
7ef3b40
Merge branch 'main' of github.com:peterzhuamazon/opensearch-build int…
peterzhuamazon Dec 8, 2021
d80baf9
Restore Jenkinsfile
peterzhuamazon Dec 8, 2021
450c964
Restore build from source python code
peterzhuamazon Dec 8, 2021
5220b27
Add 1.1.1 related templates and manifests
peterzhuamazon Dec 6, 2021
c48dcbd
More changes to template and bundle code to make build 1.1.1 opensear…
peterzhuamazon Dec 7, 2021
d166b9b
Add more changes
peterzhuamazon Dec 8, 2021
423adfe
Add arm64 and docker build code
peterzhuamazon Dec 8, 2021
438bc16
Restore Jenkinsfile
peterzhuamazon Dec 8, 2021
b09dfab
Restore Jenkinsfile
peterzhuamazon Dec 8, 2021
6ff89d3
Restore build from source python code
peterzhuamazon Dec 8, 2021
9633ec8
Readd functional tests
peterzhuamazon Dec 8, 2021
c467c07
Readd functional tests
peterzhuamazon Dec 8, 2021
8024c90
Switch checks of build from source to gradle related checks
peterzhuamazon Dec 8, 2021
a01c1af
Remove checks for dashboards as it is not there yet
peterzhuamazon Dec 8, 2021
10c4f4d
Fix ci check link is not matching the new setup in s3
peterzhuamazon Dec 8, 2021
81079e1
Merge branch 'main' of github.com:peterzhuamazon/opensearch-build int…
peterzhuamazon Dec 8, 2021
3ec4b08
Add more changes to resolve issues with templates and tests
peterzhuamazon Dec 8, 2021
92b285e
More updates to the templates sent to owners
peterzhuamazon Dec 9, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions .github/meta_issue_template/release-1.1.1-component-issues.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
This is a component issue for release 1.1.1.
Coming from [release issue 1.1.1](https://github.com/opensearch-project/opensearch-build/issues/870), release version 1.1.1. Please follow the following checklist.

<details><summary>How to use this component issue</summary>
<p>

## This Component Issue
This component issue captures the state of the OpenSearch release, on component/plugin level, its assignee is responsible for driving the release of the component. Please contact them or @mention them on this issue for help.

## Release Steps
There are several steps to the release process, components that are behind present risk to the release. Component owners resolve tasks on this ticket to communicate with the overall release owner.

Steps have completion dates for coordinating efforts between the components of a release; components can start as soon as they are ready far in advance of a future release.

You can find all the corresponding dates of each step in the release issue above.

### What should I do if my plugin isn't making any changes?
If including changes in this release, increment the version on 1.1 branch to `1.1.1` for Min/Core, and `1.1.1.0` for components. Otherwise, keep the version number unchanged for both.

</p>
</details>


### You can find all the date in above issue

### Preparation

- [ ] Assign this issue to a release owner.
- [ ] All the tasks in this issue have been reviewed by the release owner.
- [ ] Create, update, triage and label all features and issues targeted for this release with v1.1.1.


### CI/CD

- [ ] All code changes for 1.1.1 are complete.
- [ ] Ensure working and passing CI.

### Pre-Release

- [ ] Confirm that all changes for 1.1.1 have been merged.
- [ ] Complete integration and sanity tests, and update results in the comment, [example](https://github.com/opensearch-project/opensearch-build/issues/1118).
- [ ] Find/fix bugs using latest tarball and docker image provided in meta issue.
- [ ] Completed release candidate testing build #TBD.
- [ ] All intermittent test failures have issues filed.

### Release

- [ ] Complete [documentation](https://github.com/opensearch-project/documentation-website).
- [ ] Gather, review and publish release notes.
- [ ] Verify all issued labeled for this release are closed or labelled for the next release.

### Post Release

- [ ] Create [a release tag](https://github.com/opensearch-project/.github/blob/main/RELEASING.md#tagging).
- [ ] Suggest improvements to [this template](https://github.com/opensearch-project/opensearch-build/blob/main/.github/ISSUE_TEMPLATE/release_template.md).
- [ ] Conduct a retrospect, and publish its results.
59 changes: 59 additions & 0 deletions .github/meta_issue_template/release-1.3.0-component-issues.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
This is a component issue for release 1.3.0.
Coming from [release issue 1.3.0](https://github.com/opensearch-project/opensearch-build/issues/870), release version 1.3.0. Please follow the following checklist.

<details><summary>How to use this component issue</summary>
<p>

## This Component Issue
This component issue captures the state of the OpenSearch release, on component/plugin level, its assignee is responsible for driving the release of the component. Please contact them or @mention them on this issue for help.

## Release Steps
There are several steps to the release process, components that are behind present risk to the release. Component owners resolve tasks on this ticket to communicate with the overall release owner.

Steps have completion dates for coordinating efforts between the components of a release; components can start as soon as they are ready far in advance of a future release.

You can find all the corresponding dates of each step in the release issue above.

### What should I do if my plugin isn't making any changes?
If including changes in this release, increment the version on 1.1 branch to `1.1.1` for Min/Core, and `1.1.1.0` for components. Otherwise, keep the version number unchanged for both.

</p>
</details>


### You can find all the date in above issue

### Preparation

- [ ] Assign this issue to a release owner.
- [ ] Finalize scope and feature set and update [the Public Roadmap](https://github.com/orgs/opensearch-project/projects/1).
- [ ] All the tasks in this issue have been reviewed by the release owner.
- [ ] Create, update, triage and label all features and issues targeted for this release with v1.3.0.


### CI/CD

- [ ] All code changes for 1.3.0 are complete.
- [ ] Ensure working and passing CI.
- [ ] Re(add) this repo to the (if not exist) [opensearch-dashboards input manifest](https://github.com/opensearch-project/opensearch-build/blob/main/manifests/1.3.0/opensearch-dashboards-1.3.0.yml).

### Pre-Release

- [ ] Update your branch in the [opensearch-dashboards input manifest](https://github.com/opensearch-project/opensearch-build/blob/main/manifests/1.3.0/opensearch-dashboards-1.3.0.yml).
- [ ] Confirm that all changes for 1.1.1 have been merged.
- [ ] Complete integration tests, and update results in the comment, [example](https://github.com/opensearch-project/opensearch-build/issues/1118).
- [ ] Find/fix bugs using latest tarball and docker image provided in meta issue.
- [ ] Completed release candidate testing build #TBD.
- [ ] All intermittent test failures have issues filed.

### Release

- [ ] Complete [documentation](https://github.com/opensearch-project/documentation-website).
- [ ] Gather, review and publish release notes.
- [ ] Verify all issued labeled for this release are closed or labelled for the next release.

### Post Release

- [ ] Create [a release tag](https://github.com/opensearch-project/.github/blob/main/RELEASING.md#tagging).
- [ ] Suggest improvements to [this template](https://github.com/opensearch-project/opensearch-build/blob/main/.github/ISSUE_TEMPLATE/release_template.md).
- [ ] Conduct a retrospect, and publish its results.
51 changes: 26 additions & 25 deletions manifests/1.1.1/opensearch-1.1.1.yml
Original file line number Diff line number Diff line change
@@ -1,73 +1,74 @@
---
schema-version: "1.0"
ci:
image:
name: "opensearchstaging/ci-runner:centos7-x64-arm64-jdkmulti-node10.24.1-cypress6.9.1-20211028"
build:
name: OpenSearch
version: 1.1.1
patches:
- 1.1.0
components:
- name: OpenSearch
repository: https://github.com/opensearch-project/OpenSearch.git
ref: "1.1"
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- gradle:publish
- gradle:properties:version
- manifest:component
- name: common-utils
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: job-scheduler
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: dashboards-reports
repository: https://github.com/opensearch-project/dashboards-reports.git
working_directory: reports-scheduler
ref: "1.1"
platforms:
- linux
checks:
- gradle:publish
- gradle:properties:version
- name: sql
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: alerting
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: security
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: cross-cluster-replication
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
checks:
- manifest:component
- name: performance-analyzer-rca
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: performance-analyzer
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: index-management
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: k-NN
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
platforms:
- darwin
- linux
- name: anomaly-detection
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: asynchronous-search
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
checks:
- manifest:component
- name: dashboards-reports
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
- name: dashboards-notebooks
dist: https://ci.opensearch.org/ci/dbc/builds/1.1.0/405
dist: https://ci.opensearch.org/ci/dbc/bundle-build/1.1.0/20210930
checks:
- manifest:component
38 changes: 38 additions & 0 deletions manifests/1.1.1/opensearch-dashboards-1.1.1.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
schema-version: "1.0"
ci:
image:
name: "opensearchstaging/ci-runner:centos7-x64-arm64-jdkmulti-node10.24.1-cypress6.9.1-20211028"
build:
name: OpenSearch Dashboards
version: 1.1.1
patches:
- 1.1.0
components:
- name: reportsDashboards
repository: https://github.com/opensearch-project/dashboards-reports
ref: "1.1"
working_directory: "dashboards-reports"
platforms:
- linux
- name: OpenSearch-Dashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: alertingDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: securityDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: indexManagementDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: queryWorkbenchDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: notebooksDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: ganttChartDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: traceAnalyticsDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: anomalyDetectionDashboards
dist: https://ci.opensearch.org/ci/dbc/bundle-build-dashboards/1.1.0/20210930
- name: functionalTestDashboards
repository: https://github.com/opensearch-project/opensearch-dashboards-functional-test.git
ref: "main"
4 changes: 3 additions & 1 deletion src/ci_workflow/ci_check_manifest_component.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@ def __init__(self, component, url):

def check(self):
for architecture in BuildArgs.SUPPORTED_ARCHITECTURES:
url = "/".join([self.component.dist, architecture, "manifest.yml"])
# Since we only have 'linux' builds now we hard code it to 'linux'
# Once we have all platform builds on S3 we can then add a second loop for 'BuildArgs.SUPPORTED_PLATFORMS'
url = "/".join([self.component.dist, "linux", architecture, "builds", self.target.name, "manifest.yml"])
self.build_manifest = BuildManifest.from_url(url)
if self.component.name in self.build_manifest.components:
logging.info(f"Found {self.component.name} in {url}.")
Expand Down
2 changes: 1 addition & 1 deletion src/ci_workflow/ci_input_manifest.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ def __init__(self, file, args):

def __check__(self):

target = CiTarget(version=self.manifest.build.version, snapshot=self.args.snapshot)
target = CiTarget(version=self.manifest.build.version, name=self.manifest.build.filename, snapshot=self.args.snapshot)

with TemporaryDirectory(keep=self.args.keep, chdir=True) as work_dir:
logging.info(f"Sanity-testing in {work_dir.name}")
Expand Down
4 changes: 3 additions & 1 deletion src/ci_workflow/ci_target.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,12 @@

class CiTarget:
version: str
name: str
snapshot: bool

def __init__(self, version, snapshot=True):
def __init__(self, version, name, snapshot=True):
self.version = version
self.name = name
self.snapshot = snapshot

@property
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ def __mock_dependencies(self, props="", snapshot=False, gradle_project=None):
return TestCiCheckGradleDependencies.DummyDependencies(
component=MagicMock(),
git_repo=git_repo,
target=CiTarget(version="1.1.0", snapshot=snapshot),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=snapshot),
args=gradle_project,
)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ def __mock_check(self, props=None):
return CiCheckGradleDependenciesOpenSearchVersion(
component=MagicMock(),
git_repo=MagicMock(),
target=CiTarget(version="1.1.0", snapshot=True),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=True),
args=None,
)

Expand Down Expand Up @@ -52,7 +52,7 @@ def test_executes_gradle_command(self):
check = CiCheckGradleDependenciesOpenSearchVersion(
component=MagicMock(),
git_repo=MagicMock(),
target=CiTarget(version="1.1.0", snapshot=True),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=True),
args=None,
)
check.git_repo.output.assert_called_once_with(
Expand All @@ -63,7 +63,7 @@ def test_executes_gradle_command_with_arg(self):
check = CiCheckGradleDependenciesOpenSearchVersion(
component=MagicMock(),
git_repo=MagicMock(),
target=CiTarget(version="1.1.0", snapshot=True),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=True),
args="plugin",
)
check.git_repo.output.assert_called_once_with(
Expand Down
4 changes: 2 additions & 2 deletions tests/tests_ci_workflow/test_ci_check_gradle_properties.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ def test_executes_gradle_properties(self):
TestCiCheckGradleProperties.DummyProperties(
component=MagicMock(),
git_repo=git_repo,
target=CiTarget(version="1.1.0", snapshot=False),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=False),
)

git_repo.output.assert_called_once_with("./gradlew properties -Dopensearch.version=1.1.0 -Dbuild.snapshot=false")
Expand All @@ -35,7 +35,7 @@ def test_executes_gradle_properties_snapshot(self):
TestCiCheckGradleProperties.DummyProperties(
component=MagicMock(),
git_repo=git_repo,
target=CiTarget(version="1.1.0", snapshot=True),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=True),
)

git_repo.output.assert_called_once_with("./gradlew properties -Dopensearch.version=1.1.0-SNAPSHOT -Dbuild.snapshot=true")
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ def __mock_check(self, props=None, component=None, snapshot=True):
return CiCheckGradlePropertiesVersion(
component=component or MagicMock(),
git_repo=MagicMock(),
target=CiTarget(version="1.1.0", snapshot=snapshot),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=snapshot),
)

def test_has_version(self):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ def test_executes_gradle_command(self):
check = CiCheckGradlePublishToMavenLocal(
component=MagicMock(),
git_repo=MagicMock(),
target=CiTarget(version="1.1.0", snapshot=False),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=False),
)
check.check()
check.git_repo.execute.assert_called_once_with("./gradlew publishToMavenLocal -Dopensearch.version=1.1.0 -Dbuild.snapshot=false")
Expand All @@ -25,7 +25,7 @@ def test_executes_gradle_command_snapshot(self):
check = CiCheckGradlePublishToMavenLocal(
component=MagicMock(),
git_repo=MagicMock(),
target=CiTarget(version="1.1.0", snapshot=True),
target=CiTarget(version="1.1.0", name="opensearch", snapshot=True),
)
check.check()
check.git_repo.execute.assert_called_once_with("./gradlew publishToMavenLocal -Dopensearch.version=1.1.0-SNAPSHOT -Dbuild.snapshot=true")
Loading