Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

airbyte-ci: Add --yes option #33206

Merged
merged 6 commits into from
Dec 7, 2023
Merged

airbyte-ci: Add --yes option #33206

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
2a4a16a
Add --yes option
bnchrch Dec 6, 2023
e3a4d88
Add change log
bnchrch Dec 6, 2023
52a992f
Add missing file
bnchrch Dec 6, 2023
c63fec7
Automated Commit - Formatting Changes
bnchrch Dec 6, 2023
f1cdc72
Add auto update only agree flag
bnchrch Dec 7, 2023
e61822b
Automated Commit - Formatting Changes
bnchrch Dec 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 14 additions & 12 deletions airbyte-ci/connectors/pipelines/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,17 +124,18 @@ At this point you can run `airbyte-ci` commands.

#### Options

| Option | Default value | Mapped environment variable | Description |
| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------- | ------------------------------------------------------------------------------------------- |
| `--enable-dagger-run/--disable-dagger-run` | `--enable-dagger-run`` | | Disables the Dagger terminal UI. | | | | | | |
| `--is-local/--is-ci` | `--is-local` | | Determines the environment in which the CLI runs: local environment or CI environment. |
| `--git-branch` | The checked out git branch name | `CI_GIT_BRANCH` | The git branch on which the pipelines will run. |
| `--git-revision` | The current branch head | `CI_GIT_REVISION` | The commit hash on which the pipelines will run. |
| `--diffed-branch` | `origin/master` | | Branch to which the git diff will happen to detect new or modified files. |
| `--gha-workflow-run-id` | | | GHA CI only - The run id of the GitHub action workflow |
| `--ci-context` | `manual` | | The current CI context: `manual` for manual run, `pull_request`, `nightly_builds`, `master` |
| `--pipeline-start-timestamp` | Current epoch time | `CI_PIPELINE_START_TIMESTAMP` | Start time of the pipeline as epoch time. Used for pipeline run duration computation. |
| `--show-dagger-logs/--hide-dagger-logs` | `--hide-dagger-logs` | | Flag to show or hide the dagger logs. |
| Option | Default value | Mapped environment variable | Description |
| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `--yes/--y` | False | | Agrees to all prompts. |
| `--enable-dagger-run/--disable-dagger-run` | `--enable-dagger-run` | | Disables the Dagger terminal UI. |
| `--is-local/--is-ci` | `--is-local` | | Determines the environment in which the CLI runs: local environment or CI environment. |
| `--git-branch` | The checked out git branch name | `CI_GIT_BRANCH` | The git branch on which the pipelines will run. |
| `--git-revision` | The current branch head | `CI_GIT_REVISION` | The commit hash on which the pipelines will run. |
| `--diffed-branch` | `origin/master` | | Branch to which the git diff will happen to detect new or modified files. |
| `--gha-workflow-run-id` | | | GHA CI only - The run id of the GitHub action workflow |
| `--ci-context` | `manual` | | The current CI context: `manual` for manual run, `pull_request`, `nightly_builds`, `master` |
| `--pipeline-start-timestamp` | Current epoch time | `CI_PIPELINE_START_TIMESTAMP` | Start time of the pipeline as epoch time. Used for pipeline run duration computation. |
| `--show-dagger-logs/--hide-dagger-logs` | `--hide-dagger-logs` | | Flag to show or hide the dagger logs. |

### <a id="connectors-command-subgroup"></a>`connectors` command subgroup

Expand All @@ -152,7 +153,7 @@ Available commands:
| `--metadata-query` | False | | | Filter connectors by the `data` field in the metadata file using a [simpleeval](https://github.com/danthedeckie/simpleeval) query. e.g. 'data.ab_internal.ql == 200' |
| `--use-local-cdk` | False | False | | Build with the airbyte-cdk from the local repository. " "This is useful for testing changes to the CDK. |
| `--language` | True | | | Select connectors with a specific language: `python`, `low-code`, `java`. Can be used multiple times to select multiple languages. |
| `--modified` | False | False | | Run the pipeline on only the modified connectors on the branch or previous commit (depends on the pipeline implementation). |
| `--modified` | False | False | | Run the pipeline on only the modified connectors on the branch or previous commit (depends on the pipeline implementation). |
| `--concurrency` | False | 5 | | Control the number of connector pipelines that can run in parallel. Useful to speed up pipelines or control their resource usage. |
| `--metadata-change-only/--not-metadata-change-only` | False | `--not-metadata-change-only` | | Only run the pipeline on connectors with changes on their metadata.yaml file. |
| `--enable-dependency-scanning / --disable-dependency-scanning` | False | ` --disable-dependency-scanning` | | When enabled the dependency scanning will be performed to detect the connectors to select according to a dependency change. |
Expand Down Expand Up @@ -451,6 +452,7 @@ This command runs the Python tests for a airbyte-ci poetry package.
## Changelog
| Version | PR | Description |
| ------- | ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
| 2.10.4 | [#33206](https://github.com/airbytehq/airbyte/pull/33206) | Add "-y/--yes" Flag to allow preconfirmation of prompts |
| 2.10.3 | [#33080](https://github.com/airbytehq/airbyte/pull/33080) | Fix update failing due to SSL error on install. |
| 2.10.2 | [#33008](https://github.com/airbytehq/airbyte/pull/33008) | Fix local `connector build`. |
| 2.10.1 | [#32928](https://github.com/airbytehq/airbyte/pull/32928) | Fix BuildConnectorImages constructor. |
Expand Down
3 changes: 2 additions & 1 deletion airbyte-ci/connectors/pipelines/pipelines/airbyte_ci/connectors/publish/commands.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
from pipelines.airbyte_ci.connectors.pipeline import run_connectors_pipelines
from pipelines.airbyte_ci.connectors.publish.context import PublishConnectorContext
from pipelines.airbyte_ci.connectors.publish.pipeline import reorder_contexts, run_connector_publish_pipeline
from pipelines.cli.confirm_prompt import confirm
from pipelines.cli.dagger_pipeline_command import DaggerPipelineCommand
from pipelines.consts import ContextState
from pipelines.helpers.utils import fail_if_missing_docker_hub_creds
Expand Down Expand Up @@ -71,7 +72,7 @@ async def publish(
ctx.obj["metadata_service_bucket_name"] = metadata_service_bucket_name
ctx.obj["metadata_service_gcs_credentials"] = metadata_service_gcs_credentials
if ctx.obj["is_local"]:
click.confirm(
confirm(
"Publishing from a local environment is not recommended and requires to be logged in Airbyte's DockerHub registry, do you want to continue?",
abort=True,
)
Expand Down
2 changes: 2 additions & 0 deletions airbyte-ci/connectors/pipelines/pipelines/cli/airbyte_ci.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
from pipelines import main_logger
from pipelines.cli.auto_update import __installed_version__, check_for_upgrade
from pipelines.cli.click_decorators import click_append_to_context_object, click_ignore_unused_kwargs, click_merge_args_into_context_obj
from pipelines.cli.confirm_prompt import pre_confirm_flag
from pipelines.cli.lazy_group import LazyGroup
from pipelines.cli.telemetry import click_track_command
from pipelines.consts import DAGGER_WRAP_ENV_VAR_NAME, CIContext
Expand Down Expand Up @@ -167,6 +168,7 @@ def is_current_process_wrapped_by_dagger_run() -> bool:
},
)
@click.version_option(__installed_version__)
@pre_confirm_flag
@click.option("--enable-dagger-run/--disable-dagger-run", default=is_dagger_run_enabled_by_default)
@click.option("--enable-update-check/--disable-update-check", default=True)
@click.option("--enable-auto-update/--disable-auto-update", default=True)
Expand Down
4 changes: 2 additions & 2 deletions airbyte-ci/connectors/pipelines/pipelines/cli/auto_update.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@
import os
import sys

import asyncclick as click
import requests
from pipelines import main_logger
from pipelines.cli.confirm_prompt import confirm
from pipelines.consts import LOCAL_PIPELINE_PACKAGE_PATH
from pipelines.external_scripts.airbyte_ci_install import RELEASE_URL, get_airbyte_os_name

Expand Down Expand Up @@ -102,7 +102,7 @@ def check_for_upgrade(
logging.warning(upgrade_error_message)

# Ask the user if they want to upgrade
if enable_auto_update and click.confirm("Do you want to automatically upgrade?", default=True):
if enable_auto_update and confirm("Do you want to automatically upgrade?", default=True):
# if the current command contains `airbyte-ci-dev` is the dev version of the command
logging.info(f"[{'DEV' if is_dev_version else 'BINARY'}] Upgrading pipelines...")

Expand Down
2 changes: 1 addition & 1 deletion airbyte-ci/connectors/pipelines/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "pipelines"
version = "2.10.3"
version = "2.10.4"
description = "Packaged maintained by the connector operations team to perform CI for connectors' pipelines"
authors = ["Airbyte <contact@airbyte.io>"]

Expand Down
Loading