From d1f6dc174172d6a0c920b8df1cba38f833af3941 Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Mon, 22 Feb 2021 16:21:47 -0800 Subject: [PATCH 01/74] Fix push image condition (#767) Signed-off-by: Haytham Abuelfutuh --- .github/workflows/sandbox.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/sandbox.yml b/.github/workflows/sandbox.yml index 43e06851f3..4fb2db51ff 100644 --- a/.github/workflows/sandbox.yml +++ b/.github/workflows/sandbox.yml @@ -25,7 +25,7 @@ jobs: build_extra_args: "--target=default --compress=true" context: ./ dockerfile: docker/sandbox/Dockerfile - push_image_and_stages: ${{ github.event.name == 'release' }} + push_image_and_stages: ${{ github.event_name == 'release' }} push-sandbox-dind-image: name: Push sandbox DinD image to GHCR runs-on: ubuntu-latest @@ -44,4 +44,4 @@ jobs: build_extra_args: "--target=dind --compress=true" context: ./ dockerfile: docker/sandbox/Dockerfile - push_image_and_stages: ${{ github.event.name == 'release' }} + push_image_and_stages: ${{ github.event_name == 'release' }} From cc974e6feed2698cd23217409bd877c4a9449b34 Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Tue, 23 Feb 2021 13:19:46 -0800 Subject: [PATCH 02/74] Add auth config examples Signed-off-by: Haytham Abuelfutuh --- rsts/howto/authentication/index.rst | 89 +++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) diff --git a/rsts/howto/authentication/index.rst b/rsts/howto/authentication/index.rst index eb4fe6d689..af861e082b 100644 --- a/rsts/howto/authentication/index.rst +++ b/rsts/howto/authentication/index.rst @@ -59,6 +59,95 @@ Flyte Admin Configuration ========================= Please refer to the `inline documentation `_ on the ``Config`` object in the ``auth`` package for a discussion on the settings required. +Example Configurations +====================== + +Common Configuration +#################### + +Flyte Admin secures cookies using AES encryption. In order to achieve that, follow the steps below: + +1. Generate secure cookie keys. Run this command to generate new keys: + +.. prompt:: bash + + go test -v github.com/lyft/flyteadmin/pkg/auth -run TestSecureCookieLifecycle + +2. Create two secrets in the ``flyte`` namespace with the generated keys: + +.. prompt:: bash + + kubectl create secret generic flyteadmin-cookie-blockkey -n flyte --from-literal=blockkey= + +.. prompt:: bash + + kubectl create secret generic flyteadmin-cookie-hashkey -n flyte --from-literal=hashkey= + +3. Configure FlyteAdmin deployment to mount them to FlyteAdmin pod. + e.g. Add the following to your existing configuration + + .. code-block:: yaml + + containers: + name: flyteadmin + volumeMounts: + - mountPath: /etc/secrets/oauth + name: oauth + readOnly: true + - mountPath: /etc/secrets/hashkey + name: hashkey + readOnly: true + - mountPath: /etc/secrets/blockkey + name: blockkey + readOnly: true + volumes: + - name: oauth + secret: + defaultMode: 420 + secretName: flyteadmin-oauth-client + - name: hashkey + secret: + defaultMode: 420 + secretName: flyteadmin-cookie-hashkey + - name: blockkey + secret: + defaultMode: 420 + secretName: flyteadmin-cookie-blockkey + +Google IdP +########## + +1. Follow `Google Docs `__ on how to configure the IdP for OpenIDConnect. + +2. Create a secret in the ``flyte`` namespace with the value of the client password. + +.. prompt:: bash + + kubectl create secret generic -n flyte flyteadmin-oauth-client --from-literal=secret= + +3. Configure Flyte Admin with the following configuration: + +.. code-block:: yaml + + server: + security: + useAuth: true + oauth: + scopes: + - profile + - openid + claims: + iss: https://accounts.google.com + aud: + clientId: + clientSecretFile: "/etc/secrets/oauth/secret" + cookieHashKeyFile: "/etc/secrets/hashkey/hashkey" + cookieBlockKeyFile: "/etc/secrets/blockkey/blockkey" + authorizeUrl: "https://accounts.google.com/o/oauth2/v2/auth?access_type=offline" + tokenUrl: "https://oauth2.googleapis.com/token" + callbackUrl: "http://localhost:8088/callback" + redirectUrl: "/api/v1/projects" + ****** CI ****** From 4ca3169e5573e75d5b0fd6bac83c2edf6a9d2622 Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Thu, 25 Feb 2021 14:57:15 -0800 Subject: [PATCH 03/74] update Signed-off-by: Haytham Abuelfutuh --- rsts/howto/authentication/index.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/rsts/howto/authentication/index.rst b/rsts/howto/authentication/index.rst index af861e082b..836096dcf4 100644 --- a/rsts/howto/authentication/index.rst +++ b/rsts/howto/authentication/index.rst @@ -82,6 +82,7 @@ Flyte Admin secures cookies using AES encryption. In order to achieve that, foll .. prompt:: bash kubectl create secret generic flyteadmin-cookie-hashkey -n flyte --from-literal=hashkey= + kubectl create secret generic flytepropeller-oauth -n flyte --from-literal=secret=5aATwGcDZmFd3n0mLDbuR1uA 3. Configure FlyteAdmin deployment to mount them to FlyteAdmin pod. e.g. Add the following to your existing configuration From 71d177dd5fb445e07e5a79b2d5115fdca68d055a Mon Sep 17 00:00:00 2001 From: Katrina Rogan Date: Tue, 23 Feb 2021 13:31:25 -0800 Subject: [PATCH 04/74] Update docs on configuring flyteadmin matchable attributes (#772) Signed-off-by: Haytham Abuelfutuh --- .../howto/managing_customizable_resources.rst | 49 +++++++++++++++---- 1 file changed, 39 insertions(+), 10 deletions(-) diff --git a/rsts/howto/managing_customizable_resources.rst b/rsts/howto/managing_customizable_resources.rst index 4bd0f28ee7..671c14b471 100644 --- a/rsts/howto/managing_customizable_resources.rst +++ b/rsts/howto/managing_customizable_resources.rst @@ -20,7 +20,7 @@ Flyte allows these custom settings along the following combination of dimensions Please see the :ref:`divedeep-projects` document for more information on projects and domains. Along these dimensions, the following settings are configurable. Note that not all three of the combinations above are valid for each of these settings. - Defaults for task resource requests and limits (when not specified by the author of the task). -- Settings for the cluster resource configuration that feeds into Admin's cluster resource manager. +- Settings for project-namespaced cluster resource configuration that feeds into Admin's cluster resource manager. - Execution queues that are used for Dynamic Tasks. Read more about execution queues here, but effectively they're meant to be used with constructs like AWS Batch. - Determining how workflow executions get assigned to clusters in a multi-cluster Flyte deployment. @@ -28,7 +28,9 @@ The proto definition is the definitive source of which `matchable attributes `_ can be customized. -Each of the four above settings are discussed below. Also, since the flyte-cli tool does not yet hit these endpoints, we are including some sample ``curl`` commands for administrators to reference. +Each of the four above settings are discussed below. Eventually all of these customizations will be overridable using +:std:ref:`flytectl`. Until then, flyte-cli command line options can be used to modify frequent use-cases, and barring +that we show examples using curl. Task Resources @@ -47,6 +49,15 @@ in the flyteadmin config are used. The override values from the database are assigned at execution time. +To update individual project-domain attributes, use the following as an example: + +.. prompt:: bash + + curl --request PUT 'https://flyte.company.net/api/v1/project_domain_attributes/projectname/staging' \ + --header 'Content-Type: application/json' --data-raw \ + '{"attributes":{"matchingAttributes":{"taskResourceAttributes":{"defaults":{"cpu": "1000", "memory": "5000Gi"}, "limits": {"cpu": "4000"}}}}' + + Cluster Resources ================= @@ -57,22 +68,32 @@ In the absence of custom override values, templateData from the `flyteadmin conf Note that these settings can only take on domain, or a project and domain specificity. Since Flyte has not tied in the notion of a workflow or a launch plan to any Kubernetes constructs, specifying a workflow or launch plan name doesn't make any sense. +Running the following, will make it so that when Admin fills in cluster resource templates, the K8s namespace ``flyteexamples-development`` will have a resource quota of 1000 CPU cores and 5TB of memory. + +.. prompt:: bash + + flyte-cli -h localhost:30081 -p flyteexamples -d development update-cluster-resource-attributes \ + --attributes projectQuotaCpu 1000 --attributes projectQuotaMemory 5000Gi + -Command -------- -Running the following, will make it so that when Admin fills in cluster resource templates, the K8s namespace ``projectname-staging`` will have a resource quota of 1000 CPU cores and 5TB of memory. +These values will in turn be used to fill in the template fields, for example: -.. code-block:: console +.. rli:: https://raw.githubusercontent.com/flyteorg/flyte/master/kustomize/base/single_cluster/headless/config/clusterresource-templates/ab_project-resource-quota.yaml - curl --request PUT 'https://flyte.company.net/api/v1/project_domain_attributes/projectname/staging' --header 'Content-Type: application/json' --data-raw '{"attributes":{"matchingAttributes":{"clusterResourceAttributes":{"attributes":{"projectQuotaCpu": "1000", "projectQuotaMemory": "5000Gi"}}}}}' +from the base of this repository for the ``flyteexamples-development`` namespace and that namespace only. +For other namespaces, the `platform defaults `__ would still be applied. -These values will in turn be used to fill in the template fields in the file at ``kustomize/overlays/sandbox/admindeployment/clusterresource-templates/ab_project-resource-quota.yaml`` from the base of this repository for the ``projectname-staging`` namespace and that namespace only. For other namespaces, the defaults at the bottom of `this file `__ would still be applied. +.. note:: + + The template values, e.g. ``projectQuotaCpu`` or ``projectQuotaMemory`` are freeform strings. You must ensure that + they match the template placeholders in your `template file `__ + for your changes to take effect. Execution Queues ================ -Execution queues are use to determine where dynamic tasks run. +Execution queues are use to determine where tasks yielded by a dynamic :py:func:`flytekit:flytekit.maptask` run. Execution queues themselves are currently defined in the `flyteadmin config `__. @@ -82,14 +103,22 @@ stored in the Admin database. .. prompt:: bash - curl --request PUT 'https://flyte.company.net/api/v1/workflow_attributes/project/domain/YourWorkflowName' --header 'Content-Type: application/json' --data-raw '{"attributes":{"matchingAttributes":{"executionQueueAttributes":{"tags":["my_queue"]}}}}' + flyte-cli -h localhost:30081 -p flyteexamples -d development update-execution-queue-attributes \ + --tags critical --tags gpu_intensive +You can view existing attributes for which tags can be assigned by visting `http://localhost:30081/api/v1/matchable_attributes?resource_type=3 `__. Execution Cluster Label ======================= This allows forcing a matching execution to always execute on a specific kubernetes cluster. +You can set this using flyte-cli: + +.. prompt:: bash + + flyte-cli -h localhost:30081 -p flyteexamples -d development update-execution-cluster-label --value mycluster + ********* Hierarchy From a89f8cfb77b47f75da6d61198d2ff5acd236b031 Mon Sep 17 00:00:00 2001 From: Ketan Umare <16888709+kumare3@users.noreply.github.com> Date: Tue, 23 Feb 2021 16:37:55 -0800 Subject: [PATCH 05/74] Fix docs: Broken links etc (#771) * fix broken links * Replace references * Fix more issues * Update rsts/faq/gcp.rst Co-authored-by: Katrina Rogan * Update rsts/howto/flytecli.rst Co-authored-by: Yee Hing Tong * Update docs on configuring flyteadmin matchable attributes (#772) Co-authored-by: Katrina Rogan Co-authored-by: Yee Hing Tong Signed-off-by: Haytham Abuelfutuh --- rsts/dive_deep/admin.rst | 20 +++++------ rsts/dive_deep/admin_service.rst | 34 +++++++++---------- rsts/dive_deep/architecture.rst | 2 +- rsts/dive_deep/customizable_resources.rst | 10 +++--- rsts/faq/gcp.rst | 12 ++++--- rsts/howto/authentication/index.rst | 4 +-- rsts/howto/enable_backend_plugin.rst | 4 +-- rsts/howto/execute_single_task.rst | 2 +- rsts/howto/flytecli.rst | 2 +- .../howto/managing_customizable_resources.rst | 26 ++++++++++---- rsts/howto/multi_cluster/index.rst | 2 +- rsts/howto/notifications.rst | 2 +- rsts/howto/productionize/production.rst | 30 ++++++++-------- rsts/howto/sandbox.rst | 4 +-- rsts/howto/serviceaccount.rst | 2 +- rsts/plugins/spark_k8s.rst | 4 +-- rsts/tutorials/first_example.rst | 2 +- 17 files changed, 89 insertions(+), 73 deletions(-) diff --git a/rsts/dive_deep/admin.rst b/rsts/dive_deep/admin.rst index a6dcd7d879..a25f9e56a2 100644 --- a/rsts/dive_deep/admin.rst +++ b/rsts/dive_deep/admin.rst @@ -18,7 +18,7 @@ RPC FlyteAdmin uses the `grpc-gateway `__ library to serve incoming gRPC and HTTP requests with identical handlers. For a more detailed overview of the API, including request and response entities, see the admin -service `definition `__. The +service `definition `__. The RPC handlers are a thin shim that enforce some request structure validation and call out to appropriate :ref:`manager `. methods to process requests. @@ -59,14 +59,14 @@ The managers utilize additional components to process requests. These additional Repository ---------- Serialized entities (tasks, workflows, launch plans) and executions (workflow-, node- and task-) are stored as protos defined -`here `__. -We use the excellent `gorm `__ library to interface with our database, which currently supports a postgres +`here `__. +We use the excellent `gorm `__ library to interface with our database, which currently supports a postgres implementation. The actual code for issuing queries with gorm can be found in the -`gormimpl `__ directory. +`gormimpl `__ directory. Models ++++++ -Database models are defined in the `models `__ directory and correspond 1:1 with database tables [0]_. +Database models are defined in the `models `__ directory and correspond 1:1 with database tables [0]_. The full set of database tables includes: @@ -79,9 +79,9 @@ The full set of database tables includes: - task_executions - workflows -These database models inherit primary keys and indexes as defined in the corresponding `models `__ file. +These database models inherit primary keys and indexes as defined in the corresponding `models `__ file. -The repositories code also includes `transformers `__. +The repositories code also includes `transformers `__. These convert entities from the database format to a response format for the external API. If you change either of these structures, you will find you must change the corresponding transformers. @@ -114,7 +114,7 @@ As the name implies, ``common`` houses shared components used across different f Data ---- -Data interfaces are primarily handled by the `storage `__ library implemented in flytestdlib. However, neither this nor the underlying `stow `__ library expose `HEAD `__ support so the data package in admin exists as the layer responsible for additional, remote data operations. +Data interfaces are primarily handled by the `storage `__ library implemented in flytestdlib. However, neither this nor the underlying `stow `__ library expose `HEAD `__ support so the data package in admin exists as the layer responsible for additional, remote data operations. Errors ------ @@ -125,7 +125,7 @@ The errors directory contains centrally defined errors that are designed for com Runtime ------- -Values specific to the flyteadmin application as well as task and workflow registration and execution are configured in the `runtime `__ directory. These interfaces expose values configured in the ``flyteadmin`` top-level key in the application config. +Values specific to the flyteadmin application as well as task and workflow registration and execution are configured in the `runtime `__ directory. These interfaces expose values configured in the ``flyteadmin`` top-level key in the application config. .. _divedeep-admin-workflowengine: @@ -134,4 +134,4 @@ Workflowengine This directory contains interfaces to build and execute workflows leveraging flytepropeller compiler and client components. -.. [0] Unfortunately, given unique naming constraints, some models are redefined in `migration_models `__ to guarantee unique index values. +.. [0] Unfortunately, given unique naming constraints, some models are redefined in `migration_models `__ to guarantee unique index values. diff --git a/rsts/dive_deep/admin_service.rst b/rsts/dive_deep/admin_service.rst index 1217bda081..005926e37e 100644 --- a/rsts/dive_deep/admin_service.rst +++ b/rsts/dive_deep/admin_service.rst @@ -6,7 +6,7 @@ FlyteAdmin Service Background Entities ======== -The `admin service definition `__ defines REST operations for the entities +The `admin service definition `__ defines REST operations for the entities flyteadmin administers. As a refresher, the primary :ref:`entities ` across Flyte map similarly to FlyteAdmin entities. @@ -26,20 +26,20 @@ Permitted operations: - Get - List -The above are designated by an `identifier `_ +The above are designated by an `identifier `_ which consists of a project, domain, name and version specification. These entities are for the most part immutable. To update one of these specific entities, the updated version must be reregistered with a unique and new version identifier attribute. -One caveat is that launch plan state can toggle between `ACTIVE or INACTIVE `_. +One caveat is that launch plan state can toggle between `ACTIVE or INACTIVE `_. At most one launch plan version across a shared project, domain and name specification can be active at a time. The state affects scheduled launch plans only. An inactive launch plan can still be used to launch individual executions. However, only an active launch plan runs on a schedule (if it has a schedule defined). Static entities metadata (Named Entities) +++++++++++++++++++++++++++++++++++++++++ -A `named entity `__ includes metadata for one of the above entities +A `named entity `__ includes metadata for one of the above entities (workflow, task or launch plan) across versions. A named entity includes a resource type (workflow, task or launch plan) and an -`id `__ which is composed of project, domain and name. +`id `__ which is composed of project, domain and name. A named entity also includes metadata, which are mutable attributes about the referenced entity. This metadata includes: @@ -72,7 +72,7 @@ Permitted operations: After an execution begins, flyte propeller monitors the execution and sends events which admin uses to update the above executions. -These `events `_ include +These `events `_ include - WorkflowExecutionEvent - NodeExecutionEvent @@ -174,11 +174,11 @@ Filterable fields vary based on entity types: - updated_at - workflows.{any workflow field above} (for example: workflow.domain) - state (you must use the integer enum e.g. 1) - - States are defined in `launch_plan.proto `_. + - States are defined in `launch_plan.proto `_. - Named Entity Metadata - state (you must use the integer enum e.g. 1) - - States are defined in `common.proto `_. + - States are defined in `common.proto `_. - Executions (Workflow executions) - project @@ -187,12 +187,12 @@ Filterable fields vary based on entity types: - workflow.{any workflow field above} (for example: workflow.domain) - launch_plan.{any launch plan field above} (for example: launch_plan.name) - phase (you must use the upper-cased string name e.g. RUNNING) - - Phases are defined in `execution.proto `__. + - Phases are defined in `execution.proto `__. - execution_created_at - execution_updated_at - duration (in seconds) - mode (you must use the integer enum e.g. 1) - - Modes are defined in `execution.proto `__. + - Modes are defined in `execution.proto `__. - user (authenticated user or role from flytekit config) - Node Executions @@ -200,7 +200,7 @@ Filterable fields vary based on entity types: - node_id - execution.{any execution field above} (for example: execution.domain) - phase (you must use the upper-cased string name e.g. QUEUED) - - Phases are defined in `execution.proto `__. + - Phases are defined in `execution.proto `__. - started_at - node_execution_created_at - node_execution_updated_at @@ -212,7 +212,7 @@ Filterable fields vary based on entity types: - execution.{any execution field above} (for example: execution.domain) - node_execution.{any node execution field above} (for example: node_execution.phase) - phase (you must use the upper-cased string name e.g. SUCCEEDED) - - Phases are defined in `execution.proto `__. + - Phases are defined in `execution.proto `__. - started_at - task_execution_created_at - task_execution_updated_at @@ -268,7 +268,7 @@ Only a subset of fields are supported for sorting list queries. The explicit lis - created_at - updated_at - state (you must use the integer enum e.g. 1) - - States are defined in `launch_plan.proto `_. + - States are defined in `launch_plan.proto `_. - ListWorkflowIds - project @@ -279,18 +279,18 @@ Only a subset of fields are supported for sorting list queries. The explicit lis - domain - name - phase (you must use the upper-cased string name e.g. RUNNING) - - Phases are defined in `execution.proto `__. + - Phases are defined in `execution.proto `__. - execution_created_at - execution_updated_at - duration (in seconds) - mode (you must use the integer enum e.g. 1) - - Modes are defined `execution.proto `__. + - Modes are defined `execution.proto `__. - ListNodeExecutions - node_id - retry_attempt - phase (you must use the upper-cased string name e.g. QUEUED) - - Phases are defined in `execution.proto `__. + - Phases are defined in `execution.proto `__. - started_at - node_execution_created_at - node_execution_updated_at @@ -299,7 +299,7 @@ Only a subset of fields are supported for sorting list queries. The explicit lis - retry_attempt - phase (you must use the upper-cased string name e.g. SUCCEEDED) - - Phases are defined in `execution.proto `__. + - Phases are defined in `execution.proto `__. - started_at - task_execution_created_at - task_execution_updated_at diff --git a/rsts/dive_deep/architecture.rst b/rsts/dive_deep/architecture.rst index eeb8f8d64d..b8591e2671 100644 --- a/rsts/dive_deep/architecture.rst +++ b/rsts/dive_deep/architecture.rst @@ -11,7 +11,7 @@ FlyteIDL In Flyte, entities like "Workflows", "Tasks", "Launch Plans", and "Schedules" are recognized by multiple system components. In order for components to communicate effectively, they need a shared understanding about the structure of these entities. -The Flyte IDL (Interface Definition Language) is where shared Flyte entities are defined. This IDL also defines the RPC service definition for the `core Flyte API `_. +The Flyte IDL (Interface Definition Language) is where shared Flyte entities are defined. This IDL also defines the RPC service definition for the `core Flyte API `_. FlyteIDL uses the `protobuf `_ schema to describe entities. Clients are generated for Python, Golang, and JavaScript and imported by Flyte components. diff --git a/rsts/dive_deep/customizable_resources.rst b/rsts/dive_deep/customizable_resources.rst index aa74435173..f3e1975edb 100644 --- a/rsts/dive_deep/customizable_resources.rst +++ b/rsts/dive_deep/customizable_resources.rst @@ -4,7 +4,7 @@ Adding customizable resources ############################# -For background on customizable resources, see :ref:`howto-managing-customizable-resources`. As a quick refresher, custom resources allow you to manage configurations for specific combinations of user projects, domains and workflows that override default values. Examples of such resources include execution clusters, task resource defaults, and `more `__. +For background on customizable resources, see :ref:`howto-managing-customizable-resources`. As a quick refresher, custom resources allow you to manage configurations for specific combinations of user projects, domains and workflows that override default values. Examples of such resources include execution clusters, task resource defaults, and `more `__. Example @@ -16,7 +16,7 @@ Now one of your user teams comes to you and wants to ensure that their critical Flyte IDL ^^^^^^^^^ -You'll want to introduce a new `matchable attribute `__, including a unique enum value and proto message definition. For example +You'll want to introduce a new `matchable attribute `__, including a unique enum value and proto message definition. For example :: @@ -37,13 +37,13 @@ You'll want to introduce a new `matchable attribute `__ for an example of what is required. +See the changes in this `file `__ for an example of what is required. Flyte admin ^^^^^^^^^^^ -Once your idl changes are released, update the logic of flyteadmin to `fetch `__ your new matchable priority resource and use it when creating executions or wherever makes sense for your use case. +Once your idl changes are released, update the logic of flyteadmin to `fetch `__ your new matchable priority resource and use it when creating executions or wherever makes sense for your use case. For example: @@ -70,6 +70,6 @@ For example: Flytekit ^^^^^^^^ -For convenience, add a flyte-cli wrapper for updating your new attribute. See `this PR `__ for the full required set of changes. +For convenience, add a flyte-cli wrapper for updating your new attribute. See `this PR `__ for the full required set of changes. That's it! You now have a new matchable attribute you can configure as the needs of your users evolve. diff --git a/rsts/faq/gcp.rst b/rsts/faq/gcp.rst index 88202c1c59..f1d627c6d6 100644 --- a/rsts/faq/gcp.rst +++ b/rsts/faq/gcp.rst @@ -7,10 +7,15 @@ Google Cloud Platform I tried to run examples, but task fails with 401 error? ------------------------------------------------------- Steps: - - Are you using Workload Identity, then you have to pass in the ServiceAccount when you create the launchplan. Docs here https://lyft.github.io/flyte/user/features/roles.html?highlight=serviceaccount#kubernetes-serviceaccount-examples More information about WorkloadIdentity at https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity - - If you are just using a simple Nodepool wide permissions then check the cluster's ServiceACcount for Storage permissions. Do they look fine? - - If not, then start a dummy pod in the intended namespace and check for :: + #. Are you using Workload Identity, then you have to pass in the ServiceAccount when you create the launchplan. + - Refer to docs :ref:`howto-serviceaccounts` + - More information about WorkloadIdentity at https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity + #. If you are just using a simple Nodepool wide permissions then check the cluster's ServiceAccount for Storage permissions. Do they look fine? + + #. If not, then start a dummy pod in the intended namespace and check for + +:: gcloud auth list @@ -19,4 +24,3 @@ I tried to run examples, but task fails with 401 error? FlytePropeller uses Google Application credentials, but gsutil does not use these credentials - diff --git a/rsts/howto/authentication/index.rst b/rsts/howto/authentication/index.rst index 836096dcf4..9ff67eee9e 100644 --- a/rsts/howto/authentication/index.rst +++ b/rsts/howto/authentication/index.rst @@ -57,7 +57,7 @@ Flyte Admin will require that the application in your identity provider be confi Flyte Admin Configuration ========================= -Please refer to the `inline documentation `_ on the ``Config`` object in the ``auth`` package for a discussion on the settings required. +Please refer to the `inline documentation `_ on the ``Config`` object in the ``auth`` package for a discussion on the settings required. Example Configurations ====================== @@ -164,7 +164,7 @@ The following is a listing of the Flytekit configuration values we set in CI, al * ``export FLYTE_CREDENTIALS_AUTH_MODE=basic`` This tells the SDK to use basic authentication. If not set, Flytekit will assume you want to use the standard OAuth based three-legged flow. * ``export FLYTE_CREDENTIALS_AUTHORIZATION_METADATA_KEY=text`` - At Lyft, we set this to conform to this `header config `_ on the Admin side. + At Lyft, we set this to conform to this `header config `_ on the Admin side. * ``export FLYTE_CREDENTIALS_SCOPE=text`` When using basic authentication, you'll need to specify a scope to the IDP (instead of ``openid``, as that's only for OAuth). Set that here. * ``export FLYTE_PLATFORM_AUTH=True`` diff --git a/rsts/howto/enable_backend_plugin.rst b/rsts/howto/enable_backend_plugin.rst index 00745e7a06..c3b57c59e9 100644 --- a/rsts/howto/enable_backend_plugin.rst +++ b/rsts/howto/enable_backend_plugin.rst @@ -4,7 +4,7 @@ How do I enable backend plugins? ################################# -.. tip:: Flyte Backend plugins are awesome, but are not required to extend Flyte! You can always write a `flytekit-only plugin`. Refer to :std:ref:`sphx_glr_auto_core_advanced_custom_task_plugin.py`. Also refer to :ref:`howto-create-plugins`. +.. tip:: Flyte Backend plugins are awesome, but are not required to extend Flyte! You can always write a flytekit-only plugins. Refer to :ref:`plugins_extend_intro`. Flyte has a unique capability of adding backend plugins. Backend plugins enable Flyte platform to add new capabilities. This has several advantages, @@ -18,7 +18,7 @@ Ok, How do I enable the backend plugins? ========================================= To enable a backend plugin you have to add the ``ID`` of the plugin to the enabled plugins list. The ``enabled-plugins`` is available under the ``tasks > task-plugins`` section of FlytePropeller's configuration. -The `plugin configuration structure is defined here `_. An example of the config follows, +The `plugin configuration structure is defined here `_. An example of the config follows, .. rli:: https://raw.githubusercontent.com/flyteorg/flyte/master/kustomize/overlays/sandbox/config/propeller/enabled_plugins.yaml :language: yaml diff --git a/rsts/howto/execute_single_task.rst b/rsts/howto/execute_single_task.rst index ae7d788ec0..ffbabf05c6 100644 --- a/rsts/howto/execute_single_task.rst +++ b/rsts/howto/execute_single_task.rst @@ -32,7 +32,7 @@ After you've built an image with your updated task code, create an execution usi Just like workflow executions, you can optionally pass a user-defined name, labels, annotations, and/or notifications when launching a single task. -The type of ``my_single_task_execution`` is `SdkWorkflowExecution `_ +The type of ``my_single_task_execution`` is `SdkWorkflowExecution `_ and has the full set of methods and functionality available for conventional WorkflowExecutions. diff --git a/rsts/howto/flytecli.rst b/rsts/howto/flytecli.rst index afc72779c6..043fc13f5e 100644 --- a/rsts/howto/flytecli.rst +++ b/rsts/howto/flytecli.rst @@ -38,7 +38,7 @@ the root directory of the flytecli package, and install the dependencies with ``pip install -e .``. -.. _doc: https://virtualenv.pypa.io/en/latest/installation/ +.. _doc: https://virtualenv.pypa.io/en/latest/installation.html Install from pypi [recommended] ------------------------------- diff --git a/rsts/howto/managing_customizable_resources.rst b/rsts/howto/managing_customizable_resources.rst index 671c14b471..eb9c77828d 100644 --- a/rsts/howto/managing_customizable_resources.rst +++ b/rsts/howto/managing_customizable_resources.rst @@ -25,7 +25,7 @@ Please see the :ref:`divedeep-projects` document for more information on project - Determining how workflow executions get assigned to clusters in a multi-cluster Flyte deployment. The proto definition is the definitive source of which -`matchable attributes `_ +`matchable attributes `_ can be customized. Each of the four above settings are discussed below. Eventually all of these customizations will be overridable using @@ -44,7 +44,7 @@ This includes setting default value for task resource requests and limits for th - storage In the absence of an override the global -`default values `__ +`default values `__ in the flyteadmin config are used. The override values from the database are assigned at execution time. @@ -62,9 +62,9 @@ To update individual project-domain attributes, use the following as an example: Cluster Resources ================= -These are free-form key-value pairs which are used when filling in the templates that Admin feeds into its cluster manager. The keys represent templatized variables in `clusterresource template yaml `__ and the values are what you want to see filled in. +These are free-form key-value pairs which are used when filling in the templates that Admin feeds into its cluster manager. The keys represent templatized variables in `clusterresource template yaml `__ and the values are what you want to see filled in. -In the absence of custom override values, templateData from the `flyteadmin config `__ is used as a default. +In the absence of custom override values, templateData from the `flyteadmin config `__ is used as a default. Note that these settings can only take on domain, or a project and domain specificity. Since Flyte has not tied in the notion of a workflow or a launch plan to any Kubernetes constructs, specifying a workflow or launch plan name doesn't make any sense. @@ -83,6 +83,18 @@ These values will in turn be used to fill in the template fields, for example: from the base of this repository for the ``flyteexamples-development`` namespace and that namespace only. For other namespaces, the `platform defaults `__ would still be applied. +======= + + flyte-cli -h localhost:30081 -p flyteexamples -d development update-cluster-resource-attributes \ + --attributes projectQuotaCpu 1000 --attributes projectQuotaMemory 5000Gi + + +These values will in turn be used to fill in the template fields, for example: + +.. rli:: https://raw.githubusercontent.com/flyteorg/flyte/master/kustomize/base/single_cluster/headless/config/clusterresource-templates/ab_project-resource-quota.yaml + +from the base of this repository for the ``flyteexamples-development`` namespace and that namespace only. +For other namespaces, the `platform defaults `__ would still be applied. .. note:: @@ -96,7 +108,7 @@ Execution Queues Execution queues are use to determine where tasks yielded by a dynamic :py:func:`flytekit:flytekit.maptask` run. Execution queues themselves are currently defined in the -`flyteadmin config `__. +`flyteadmin config `__. The **attributes** associated with an execution queue must match the **tags** for workflow executions. The tags are associated with configurable resources stored in the Admin database. @@ -160,11 +172,11 @@ All other inbound CreateExecution requests would use the default values specifie Debugging ********* -Use the `get `__ endpoint +Use the `get `__ endpoint to see if overrides exist for a specific resource. E.g. `https://example.com/api/v1/project_domain_attributes/widgetmodels/production?resource_type=2 `__ To get the global state of the world, use the list all endpoint, e.g. `https://example.com/api/v1/matchable_attributes?resource_type=2 `__. -The resource type enum (int) is defined in the `proto `__. +The resource type enum (int) is defined in the `proto `__. diff --git a/rsts/howto/multi_cluster/index.rst b/rsts/howto/multi_cluster/index.rst index 8df12ddf64..77cb9cc275 100644 --- a/rsts/howto/multi_cluster/index.rst +++ b/rsts/howto/multi_cluster/index.rst @@ -132,7 +132,7 @@ Deploy the user/control plane to one cluster (you could use one of 3 existing cl FlyteAdmin Remote Cluster Access ********************************* -Some deployments of Flyte may choose to run the control plane separate from the data plane. Flyte Admin is designed to create kubernetes resources in one or more Flyte data plane clusters. For Admin to access remote clusters, it needs credentials to each cluster. In kubernetes, scoped service credentials are created by configuring a “Role” resource in a Kubernetes cluster. When you attach that role to a “ServiceAccount”, Kubernetes generates a bearer token that permits access. We create a flyteadmin `ServiceAccount `_ in each data plane cluster to generate these tokens. +Some deployments of Flyte may choose to run the control plane separate from the data plane. Flyte Admin is designed to create kubernetes resources in one or more Flyte data plane clusters. For Admin to access remote clusters, it needs credentials to each cluster. In kubernetes, scoped service credentials are created by configuring a “Role” resource in a Kubernetes cluster. When you attach that role to a “ServiceAccount”, Kubernetes generates a bearer token that permits access. We create a flyteadmin `ServiceAccount `_ in each data plane cluster to generate these tokens. When you first create the Flyte Admin ServiceAccount in a new cluster, a bearer token is generated, and will continue to allow access unless the ServiceAccount is deleted. Once we create the Flyte Admin ServiceAccount on a cluster, we should never delete it. In order to feed the credentials to Flyte Admin, you must retrieve them from your new data plane cluster, and upload them to Admin somehow (within Lyft, we use Confidant for example). diff --git a/rsts/howto/notifications.rst b/rsts/howto/notifications.rst index dc9b8aa058..d38d1702b9 100644 --- a/rsts/howto/notifications.rst +++ b/rsts/howto/notifications.rst @@ -107,7 +107,7 @@ Let's look at the following config section and go into what each value represent * **sender**: Your verified SES email sender * **body**: Configurable email body used in notifications -The full set of parameters which can be used for email templating are checked into `code `_. +The full set of parameters which can be used for email templating are checked into `code `_. .. _admin-config-example: diff --git a/rsts/howto/productionize/production.rst b/rsts/howto/productionize/production.rst index 2627e51c33..e9c0d9ed98 100644 --- a/rsts/howto/productionize/production.rst +++ b/rsts/howto/productionize/production.rst @@ -12,14 +12,14 @@ A Flyte deployment contains around 50 kubernetes resources. The Flyte team has chosen to use the "kustomize" tool to manage these configs. Take a moment to read the `kustomize docs `__. Understanding kustomize will be important to modifying Flyte configurations. -The ``/kustomize`` directory in the `flyte repository `__ is designed for use with ``kustomize`` to tailor Flyte deployments to your needs. +The ``/kustomize`` directory in the `flyte repository `__ is designed for use with ``kustomize`` to tailor Flyte deployments to your needs. Important subdirectories are described below. base - The `base directory `__ contains minimal configurations for each Flyte component. + The `base directory `__ contains minimal configurations for each Flyte component. dependencies - The `dependencies directory `__ contains deploy configurations for components like ``PostgreSQL`` that Flyte depends on. + The `dependencies directory `__ contains deploy configurations for components like ``PostgreSQL`` that Flyte depends on. These directories were designed so that you can modify them using ``kustomize`` to generate a custom Flyte deployment. In fact, this is how we create the ``sandbox`` deployment. @@ -31,21 +31,21 @@ Understanding the Sandbox The sandbox deployment is managed by a set of kustomize `overlays `__ that alter the ``base`` configurations to compose the sandbox deployment. -The sandbox overlays live in the `kustomize/overlays/sandbox `__ directory. There are overlays for each component, and a "flyte" overlay that aggregates the components into a single deploy file. +The sandbox overlays live in the `kustomize/overlays/sandbox `__ directory. There are overlays for each component, and a "flyte" overlay that aggregates the components into a single deploy file. **Component Overlays** For each modified component, there is an kustomize overlay at ``kustomize/overlays/sandbox/{{ component }}``. The overlay will typically reference the ``base`` for that component, and modify it to the needs of the sandbox. - Using kustomize "patches", we add or override specific configs from the ``base`` resources. For example, in the "console" overlay, we specify a patch in the `kustomization.yaml `__. This patch adds memory and cpu limits to the console deployment config. + Using kustomize "patches", we add or override specific configs from the ``base`` resources. For example, in the "console" overlay, we specify a patch in the `kustomization.yaml `__. This patch adds memory and cpu limits to the console deployment config. - Each Flyte component requires at least one configuration file. The configuration files for each component live in the component overlay. For example, the FlyteAdmin config lives at `kustomize/overlays/sandbox/admindeployment/flyteadmin_config.yaml `__. These files get included as Kubernetes configmaps and mounted into pods. + Each Flyte component requires at least one configuration file. The configuration files for each component live in the component overlay. For example, the FlyteAdmin config lives at `kustomize/overlays/sandbox/admindeployment/flyteadmin_config.yaml `__. These files get included as Kubernetes configmaps and mounted into pods. **Flyte Overlay** The ``flyte`` overlay is meant to aggregate the components into a single deployment file. - The `kustomization.yaml overlay `__ in that directory lists the components to be included in the deploy. + The `kustomization.yaml overlay `__ in that directory lists the components to be included in the deploy. - We run ``kustomize build`` against the ``flyte`` directory to generate the complete `sandbox deployment yaml `__ we used earlier to deploy Flyte sandbox. + We run ``kustomize build`` against the ``flyte`` directory to generate the complete `sandbox deployment yaml `__ we used earlier to deploy Flyte sandbox. Creating Your Own Deployment **************************** @@ -59,7 +59,7 @@ NOTE: To do this, check out the ``flyte`` repo :: - git clone https://github.com/lyft/flyte.git + git clone https://github.com/flyteorg/flyte.git Copy the sandbox configuration to a new directory on your machine, and enter the new directory :: @@ -68,8 +68,8 @@ Copy the sandbox configuration to a new directory on your machine, and enter the Since the ``base`` files are not in your local copy, you'll need to make some slight modifications to reference the ``base`` files from our GitHub repository. :: - find . -name kustomization.yaml -print0 | xargs -0 sed -i.bak 's~../../../base~github.com/lyft/flyte/kustomize/base~' - find . -name kustomization.yaml -print0 | xargs -0 sed -i.bak 's~../../../dependencies~github.com/lyft/flyte/kustomize/dependencies~' + find . -name kustomization.yaml -print0 | xargs -0 sed -i.bak 's~../../../base~github.com/flyteorg/flyte/kustomize/base~' + find . -name kustomization.yaml -print0 | xargs -0 sed -i.bak 's~../../../dependencies~github.com/flyteorg/flyte/kustomize/dependencies~' find . -name '*.bak' | xargs rm You should now be able to run kustomize against the ``flyte`` directory :: @@ -94,7 +94,7 @@ First, you'll need to set up a reliable PostgreSQL database. The easiest way ach Next, remove old sandbox database by opening up the ``flyte/kustomization.yaml`` file and deleting database component. :: - - github.com/lyft/flyte/kustomize/dependencies/database + - github.com/flyteorg/flyte/kustomize/dependencies/database With this line removed, you can re-run ``kustomize build flyte > flyte_generated.yaml`` and see that the the postgres deployment has been removed from the ``flyte_generated.yaml`` file. @@ -119,7 +119,7 @@ Do the same thing in ``datacatalog/datacatalog_config.yaml``, but use the dbname Note: *You can mount the database password into the pod and use the "passwordPath" config to point to a file on disk instead of specifying the password here* -Next, remove the "check-db-ready" init container from `admindeployment/admindeployment.yaml `__. This check is no longer needed. +Next, remove the "check-db-ready" init container from `admindeployment/admindeployment.yaml `__. This check is no longer needed. Production Grade Object Store ***************************** @@ -133,7 +133,7 @@ To start, `create an s3 bucket flyte_generated.yaml`` and see that the sandbox object store has been removed from the ``flyte_generated.yaml`` file. @@ -152,7 +152,7 @@ Change the ``storage`` configuration in each of these configs to use your new s3 Note: *To use IAM roles for authentication, switch to the "iam" auth-type.* -Next, open ``propeller/plugins/config.yaml`` and remove the `default-env-vars `__ (no need to replace them, the default behavior is sufficient). +Next, open ``propeller/plugins/config.yaml`` and remove the `default-env-vars `__ (no need to replace them, the default behavior is sufficient). Now if you re-run ``kustomize build flyte > flyte_generated.yaml``, you should see that the configmaps have been updated. diff --git a/rsts/howto/sandbox.rst b/rsts/howto/sandbox.rst index dab5eedb8a..39f5fe671b 100644 --- a/rsts/howto/sandbox.rst +++ b/rsts/howto/sandbox.rst @@ -54,7 +54,7 @@ Flyte configuration on your remote cluster. #. The Flyte sandbox can be deployed with a single command :: - kubectl create -f https://raw.githubusercontent.com/lyft/flyte/master/deployment/sandbox/flyte_generated.yaml + kubectl create -f https://raw.githubusercontent.com/flyteorg/flyte/master/deployment/sandbox/flyte_generated.yaml #. You can now port-forward (or if you have load-balancer enabled then get an LB) to connect to remote FlyteConsole, as follows:: @@ -70,4 +70,4 @@ Deploy Flyte Sandbox environment to a shared kubernetes cluster The goal here is to deploy to an existing Kubernetes cluster - within one namespace only. This would allow multiple Flyte clusters to run within one K8s cluster. -.. caution:: coming soon! \ No newline at end of file +.. caution:: coming soon! diff --git a/rsts/howto/serviceaccount.rst b/rsts/howto/serviceaccount.rst index 0c7ebad4de..bf720e9f0c 100644 --- a/rsts/howto/serviceaccount.rst +++ b/rsts/howto/serviceaccount.rst @@ -1,4 +1,4 @@ -. _howto_serviceaccounts: +.. _howto-serviceaccounts: ###################################################################### How do I use Kubernetes ServiceAccounts to access my cloud resources? diff --git a/rsts/plugins/spark_k8s.rst b/rsts/plugins/spark_k8s.rst index 484650cbbe..0b4aa016fc 100644 --- a/rsts/plugins/spark_k8s.rst +++ b/rsts/plugins/spark_k8s.rst @@ -36,10 +36,10 @@ Cons: How to enable Spark in flyte backend? ====================================== -Flyte Spark uses the `Spark On K8s Operator `_ and a custom built `Flyte Spark Plugin `_. +Flyte Spark uses the `Spark On K8s Operator `_ and a custom built `Flyte Spark Plugin `_. The plugin is a backend plugin and you have to enable it in your deployment. To enable a plugin follow the steps in :ref:`howto-enable-backend-plugins`. -You can optionally configure the Plugin as per the - `backend Config Structure `_ and an example Config is defined +You can optionally configure the Plugin as per the - `backend Config Structure `_ and an example Config is defined `here `_, which looks like, .. rli:: https://raw.githubusercontent.com/flyteorg/flyte/master/kustomize/overlays/sandbox/config/propeller/plugins/spark.yaml diff --git a/rsts/tutorials/first_example.rst b/rsts/tutorials/first_example.rst index aa94e15713..7511fcded5 100644 --- a/rsts/tutorials/first_example.rst +++ b/rsts/tutorials/first_example.rst @@ -50,4 +50,4 @@ Similarly, you can call this workflow and iterate locally before moving on to register it with Flyte. -.. tip:: Every invocation of a Flyte workflow requires specifying keyword args. +.. tip:: Every invocation of a Flyte workflow requires specifying keyword arguments as in the example - ``hello_world(name="name")``. Calling the workflow without the keyword ``name`` will raise an ``AssertionError``. From ba7b78f9e666f42c7716371df93aaa241b65f492 Mon Sep 17 00:00:00 2001 From: Ketan Umare <16888709+kumare3@users.noreply.github.com> Date: Wed, 24 Feb 2021 13:59:29 -0800 Subject: [PATCH 06/74] Updating Getting started to use the single Image (#775) * Updating Getting started to use the single Image * Updated Signed-off-by: Haytham Abuelfutuh --- rsts/community/docs.rst | 8 +-- rsts/dive_deep/tasks.rst | 2 +- rsts/howto/sandbox.rst | 88 ++++++++++++++++++++++++++++++-- rsts/tutorials/first_example.rst | 24 +++++---- rsts/tutorials/first_run.rst | 7 +-- rsts/tutorials/run_on_flyte.rst | 79 ++++------------------------ 6 files changed, 117 insertions(+), 91 deletions(-) diff --git a/rsts/community/docs.rst b/rsts/community/docs.rst index cef590e973..2e33df8440 100644 --- a/rsts/community/docs.rst +++ b/rsts/community/docs.rst @@ -102,17 +102,17 @@ Here are some examples, first the code and then the link .. code-block:: text - Task: :std:doc:`flytekit:tasks` + Task: :std:doc:`generated/flytekit.task` -Task: :std:doc:`flytekit:tasks` +Task: :std:doc:`generated/flytekit.task` ----- .. code-block:: text - :std:doc:`Using custom words` + :std:doc:`Using custom words ` -:std:doc:`Using custom words` +:std:doc:`Using custom words ` Python ^^^^^^^ diff --git a/rsts/dive_deep/tasks.rst b/rsts/dive_deep/tasks.rst index 052ec5d380..5317b78003 100644 --- a/rsts/dive_deep/tasks.rst +++ b/rsts/dive_deep/tasks.rst @@ -59,7 +59,7 @@ Types ----- Since it's impossible to define the unit of execution of a task the same way for all kinds of tasks, Flyte allows different task types in the system. Flyte comes with a set of defined, battle tested task types but also allows for a very flexible model to -introducing new :ref:`howto-create-plugins`. +introducing new :ref:`plugins_extend_intro`. Fault tolerance --------------- diff --git a/rsts/howto/sandbox.rst b/rsts/howto/sandbox.rst index 39f5fe671b..cd086a8605 100644 --- a/rsts/howto/sandbox.rst +++ b/rsts/howto/sandbox.rst @@ -1,4 +1,4 @@ -.. _howto_sandbox: +.. _howto-sandbox: ################################ How do I try out/install Flyte? @@ -21,14 +21,94 @@ Flyte can be run using a Kubernetes cluster only. This installs all the dependen Deploy Flyte Sandbox environment locally - on your laptop ********************************************************* -Flyte Sandbox deployment can be run locally on your laptop. This will allow only some of the functionality, but is great to get started with. -Refer to :ref:`tutorials-getting-started-run-on-flyte`. +Ensure ``kubectl`` is installed. Follow `kubectl installation docs `_. On Mac:: + + brew install kubectl + + + +.. tabs:: + + .. tab:: Docker Image + + Refer to :ref:`flyte-tutorials-firstrun` + + .. tab:: k3d + + #. Install k3d Using ``curl``:: + + curl -s https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash + + Or Using ``wget`` :: + + wget -q -O - https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash + + #. Start a new K3s cluster called flyte:: + + k3d cluster create -p "30081:30081" --no-lb --k3s-server-arg '--no-deploy=traefik' --k3s-server-arg '--no-deploy=servicelb' flyte + + #. Ensure the context is set to the new cluster:: + + kubectl config set-context flyte + + #. Install Flyte:: + + kubectl create -f https://raw.githubusercontent.com/flyteorg/flyte/master/deployment/sandbox/flyte_generated.yaml + + + #. Connect to `FlyteConsole `__ + #. [Optional] You can delete the cluster once you are done with the tutorial using - :: + + k3d cluster delete flyte + + + .. note:: + + #. Sometimes Flyteconsole will not open up. This is probably because your docker networking is impacted. One solution is to restart docker and re-do the previous steps. + #. To debug you can try a simple excercise - run nginx as follows:: + + docker run -it --rm -p 8083:80 nginx + + Now connect to `locahost:8083 `__. If this does not work, then for sure the networking is impacted, please restart docker daemon. + + .. tab:: Docker-Mac + K8s + + #. `Install Docker for mac with Kubernetes as explained here `_ + #. Make sure Kubernetes is started and once started make sure your kubectx is set to the `docker-desktop` cluster, typically :: + + kubectl config set-context docker-desktop + + #. Install Flyte:: + + kubectl create -f https://raw.githubusercontent.com/flyteorg/flyte/master/deployment/sandbox/flyte_generated.yaml + + + #. Connect to `FlyteConsole `__ + + .. tab:: Using Minikube (Not recommended) + + #. Install `Minikube `_ + + #. Install Flyte:: + + kubectl create -f https://raw.githubusercontent.com/flyteorg/flyte/master/deployment/sandbox/flyte_generated.yaml + + + .. note:: + + - Minikube runs in a Virtual Machine on your host + - So if you try to access the flyte console on localhost, that will not work, because the Virtual Machine has a different IP address. + - Flyte runs within Kubernetes (minikube), thus to access FlyteConsole, you cannot just use https://localhost:30081/console, you need to use the IP address of the minikube VM instead of localhost + - Refer to https://kubernetes.io/docs/tutorials/hello-minikube/ to understand how to access a + also to register workflows, tasks etc or use the CLI to query Flyte service, you have to use the IP address. + - If you are building an image locally and want to execute on Minikube hosted Flyte environment, please push the image to docker registry running on the Minikube VM. + - Another alternative is to change the docker host, to build the docker image on the Minikube hosted docker daemon. https://minikube.sigs.k8s.io/docs/handbook/pushing/ provides more detailed information about this process. As a TL;DR, Flyte can only run images that are accessible to Kubernetes. To make an image accessible, you could either push it to a remote registry or to a regisry that is available to Kuberentes. In case on minikube this registry is the one thats running on the VM. .. _howto-sandbox-dedicated-k8s-cluster: ****************************************************************** -Deploy Flyte Sandbox environment to a dedicated kubernetes cluster +Deploy Flyte Sandbox environment to a Cloud Kubernetes cluster ****************************************************************** Cluster Requirements diff --git a/rsts/tutorials/first_example.rst b/rsts/tutorials/first_example.rst index 7511fcded5..2bcc5c5c09 100644 --- a/rsts/tutorials/first_example.rst +++ b/rsts/tutorials/first_example.rst @@ -12,15 +12,21 @@ By the end of this getting started guide you'll be familiar with how easy it is Prerequisites ************* -Ensure that you have `git `__ installed. +#. Ensure that you have `git `__ installed. -First install the python Flytekit SDK and clone the ``flytekit-python-template`` repo :: +#. Let us make a virutal environment **(recommended)** - and then install flytekit using + ``--pre`` is used because we are currently using the beta version of flytekit 0.16.0, this introduces a completely new SDK for authoring workflows + :: - pip install flytekit==0.16.0b6 - git clone git@github.com:flyteorg/flytekit-python-template.git flyteexamples - cd flyteexamples - rm -rf .git - git init + pip install --pre flytekit + + +#. Now we will use the ``flytekit-python-template`` repo to create our own git repository called ``flyteexamples`` :: + + git clone git@github.com:flyteorg/flytekit-python-template.git flyteexamples + cd flyteexamples + rm -rf .git + git init Flyte Tasks and Workflows @@ -31,8 +37,8 @@ Let's take a look at the example workflow found in `myapp/workflows/example.py < .. rli:: https://raw.githubusercontent.com/flyteorg/flytekit-python-template/main/myapp/workflows/example.py :language: python -The most basic Flyte primitive is a :std:doc:`task `. -Flyte tasks are units of work that can be composed in a :std:doc:`workflow `. +The most basic Flyte primitive is a :std:doc:`task ` +Flyte tasks are units of work that can be composed in a :std:doc:`workflow ` You can call this task diff --git a/rsts/tutorials/first_run.rst b/rsts/tutorials/first_run.rst index 3329311138..d403e5d0a9 100644 --- a/rsts/tutorials/first_run.rst +++ b/rsts/tutorials/first_run.rst @@ -1,6 +1,7 @@ -.. _flyte-tutorials-firstrun: .. currentmodule:: firstrun +.. _flyte-tutorials-firstrun: + ############################################ Getting Started with Flyte ############################################ @@ -11,9 +12,9 @@ Flyte enables scalable, reproducable and reliable orchestration of massively lar With `docker installed `__, run this command: :: - docker run --rm --privileged -p 30081:30081 ghcr.io/flyteorg/flyte-sandbox + docker run --rm --privileged -p 30081:30081 -p 30084:30084 ghcr.io/flyteorg/flyte-sandbox -This creates a local Flyte sandbox. Once the sandbox is ready, you should see the following message: `Flyte is ready! Flyte UI is available at http://localhost:30081/console.`. Go ahead and visit that to check out the Flyte UI. +This creates a local Flyte sandbox. Once the sandbox is ready, you should see the following message: ``Flyte is ready! Flyte UI is available at http://localhost:30081/console``. Go ahead and visit http://localhost:30081/console. A quick visual tour for launching your first Workflow: diff --git a/rsts/tutorials/run_on_flyte.rst b/rsts/tutorials/run_on_flyte.rst index 075a2d1e86..215c0654b7 100644 --- a/rsts/tutorials/run_on_flyte.rst +++ b/rsts/tutorials/run_on_flyte.rst @@ -11,71 +11,16 @@ Installing Flyte Locally This guide will walk you through a quick installation of Flyte on your laptop and then how to register and execute your workflows against this deployment. (The tabs below have an option to install Flyte on a cloud provider as well) -.. rubric:: Estimated time to complete: 5 minutes. +.. rubric:: Estimated time to complete: 1 minutes. Prerequisites ============= -#. Ensure ``kubectl`` is installed. Follow `kubectl installation docs `_. On Mac:: +Make Sure you have followed :ref:`flyte-tutorials-firstrun` and have a running docker container and can access FlyteConsole on http://localhost:30081/console. - brew install kubectl - -#. If running locally ensure you have docker installed - as explained `here `_ -#. If you prefer to run the Flyte test cluster on a cloud environment like `AWS EKS `_, `Google GKE `__, then follow the instructions in a tab below. The rest of it is mostly the same. - -Steps -====== - -.. tabs:: - - .. tab:: Using k3d - - #. Install k3d Using ``curl``:: - - curl -s https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash - - Or Using ``wget`` :: - - wget -q -O - https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash - - #. Start a new K3s cluster called flyte:: - - k3d cluster create -p "30081:30081" --no-lb --k3s-server-arg '--no-deploy=traefik' --k3s-server-arg '--no-deploy=servicelb' flyte - - #. Ensure the context is set to the new cluster:: - - kubectl config set-context flyte - - #. Install Flyte:: - - kubectl create -f https://raw.githubusercontent.com/flyteorg/flyte/master/deployment/sandbox/flyte_generated.yaml - - - #. Connect to `FlyteConsole `__ - #. [Optional] You can delete the cluster once you are done with the tutorial using - :: - - k3d cluster delete flyte - - - .. note:: - - #. Sometimes Flyteconsole will not open up. This is probably because your docker networking is impacted. One solution is to restart docker and re-do the previous steps. - #. To debug you can try a simple excercise - run nginx as follows:: - - docker run -it --rm -p 8083:80 nginx - - Now connect to `locahost:8083 `__. If this does not work, then for sure the networking is impacted, please restart docker daemon. - - .. tab:: Using Docker for Mac - - Coming soon - - .. tab:: Install Flyte in EKS/GKE etc - - Refer to the :ref:`howto-sandbox-dedicated-k8s-cluster` guide. - - Once you've deployed flyte sandbox to a cloud provider, read on below to run your first workflow remotely. +.. tip:: To check if your flyte-sandbox container is running you can run ``docker ps`` and it should show image ``ghcr.io/flyteorg/flyte-sandbox `` running +.. tip:: If you prefer using k3d, Minikube, docker for mac, or a hosted Kubernetes cluster like AWS-EKS, GCP-GKE, Azure Kubernetes refer to :ref:`howto-sandbox`. It is recommended that you use a simple Docker based approach when you are first getting started with Flyte. .. _tutorials-run-flyte-laptop: @@ -85,14 +30,7 @@ Running your Flyte Workflows Registration ============ - -Set-up ------- - -In a separate process, port-forward the flyte minio service. If you're using a locally deployed Flyte sandbox, simply: :: - - kubectl -n flyte port-forward service/minio 9000 - +Registration is the process of shipping your code to Flyte backend. This creates an immutable, versioned record of your code with FlyteAdmin service. Register your workflows ----------------------- @@ -100,10 +38,11 @@ Register your workflows From within root directory of ``flyteexamples`` you created :ref:`previously ` feel free to make any changes and then register :: - FLYTE_AWS_ENDPOINT=http://localhost:9000 FLYTE_AWS_ACCESS_KEY_ID=minio \ - FLYTE_AWS_SECRET_ACCESS_KEY=miniostorage make fast_register + FLYTE_AWS_ENDPOINT=http://localhost:30084/ FLYTE_AWS_ACCESS_KEY_ID=minio \ + FLYTE_AWS_SECRET_ACCESS_KEY=miniostorage make fast_register + -If you're port-forwarding minio somewhere else, substitute the ``FLYTE_AWS_ENDPOINT`` accordingly. +.. tip:: Flyte sandbox uses minio as a substitue for S3/GCS etc, in the first command we port-forwarded it to 30084. If you use s3/gcs or a different port-forward you can drop or change ``FLYTE_AWS_ENDPOINT`` accordingly. .. rubric:: Boom! It's that simple. From 32a594b6abd8931284a8ba92ae4265ef1f7ac23e Mon Sep 17 00:00:00 2001 From: Ketan Umare <16888709+kumare3@users.noreply.github.com> Date: Thu, 25 Feb 2021 08:37:07 -0800 Subject: [PATCH 07/74] Update readme to have a getting started section (#777) * Readme updated * Readme updated * Updated * changelog * Meeting link updated * updated * Update README.md Co-authored-by: Haytham Abuelfutuh * Update rsts/community/index.rst Co-authored-by: Haytham Abuelfutuh Co-authored-by: Haytham Abuelfutuh Signed-off-by: Haytham Abuelfutuh --- README.md | 63 ++++++++++++++++++++++++++-------------- rsts/community/index.rst | 2 +- 2 files changed, 43 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index a8d6171797..0ef11abede 100644 --- a/README.md +++ b/README.md @@ -11,11 +11,19 @@ ![Twitter Follow](https://img.shields.io/twitter/follow/flyteorg?label=Follow&style=social) [![Slack Status](https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&style=social)](https://forms.gle/UVuek9WfBoweiqcJA) -Flyte is a container-native, type-safe workflow and pipelines platform optimized for large scale processing and machine learning written in Golang. Workflows can be written in any language, with out of the box support for Python, Java and Scala. +Flyte is a production-grade, container-native, type-safe workflow and pipelines platform optimized for large scale processing and machine learning written in Golang. Workflows can be written in any language, with out of the box support for [Python](https://github.com/flyteorg/flytekit), [Java and Scala](https://github.com/spotify/flytekit-java). -# Homepage -- [flyte.org](https://flyte.org) -- [Docs](https://flyte.readthedocs.io/en/latest/) +--- + +[HomePage](https://flyte.org) | +[Quickstart](#quickstart) | +[Documentation](https://docs.flyte.org/) | +[Features](#features) | +[Community & Resources](#community--resources) | +[Changelogs](CHANGELOG/) | +[Components](#component-repos) + +--- # Introduction Flyte is a fabric that connects disparate computation backends using a type safe data dependency graph. It records all changes to a pipeline, making it possible to rewind time. It also stores @@ -25,15 +33,30 @@ Flyte is more than a workflow engine, it provides workflows as a core concepts, producer-consumer order creates a workflow. Flyte workflows are pure specification and can be created using any language. Every task can also by any language. We do provide first class support for python, making it perfect for modern Machine Learning and Data processing pipelines. -# Resources +# QuickStart +With [docker installed](https://docs.docker.com/get-docker/), run this command: + +```bash + docker run --rm --privileged -p 30081:30081 -p 30084:30084 ghcr.io/flyteorg/flyte-sandbox +``` + +This creates a local Flyte sandbox. Once the sandbox is ready, you should see the following message: ``Flyte is ready! Flyte UI is available at http://localhost:30081/console``. Go ahead and visit http://localhost:30081/console. +A quick visual tour of the console + +![Flyte console Example](https://github.com/flyteorg/flyte/raw/static-resources/img/first-run-console-2.gif) + +Refer to [Docs - Getting Started](https://docs.flyte.org/en/latest/index.html) for complete end to end example. + +# Community & Resources Resources that would help you get a better understanding of Flyte. -# Communication channels +## Communication channels - [Slack Org](https://forms.gle/UVuek9WfBoweiqcJA) - [Email list](https://groups.google.com/a/flyte.org/g/users) -# Biweekly Community Sync -- Starting April 21 2020, the Flyte community meets every other Tuesday at 9:00 AM PST (US West coast time). +## Biweekly Community Sync +- 📣  Flyte OSS Community Sync Every alternate Tuesday, 9am-10am PDT ([Checkout the events calendar & subscribe](https://calendar.google.com/calendar/embed?src=admin%40flyte.org&ctz=America%2FLos_Angeles) + - You can join the [zoom link]( https://us04web.zoom.us/j/71298741279?pwd=TDR1RUppQmxGaDRFdzBOa2lHN1dsZz09). - Meeting notes are captured in [Doc](https://docs.google.com/document/d/1Jb6eOPOzvTaHjtPEVy7OR2O5qK1MhEs3vv56DX2dacM/edit#heading=h.c5ha25xc546e) - Demo Signup [Sheet](https://docs.google.com/spreadsheets/d/1Bz-s3SpMm1hGl6Z5y3HcxrNBcg9ZhSgqVgmCxK_eEhs/edit#gid=0) @@ -93,30 +116,33 @@ Resources that would help you get a better understanding of Flyte. - Snappy Console - Python CLI - Written in Golang and optimized for performance of large running jobs - -## In Progress - Golang CLI - flytectl -## Coming Soon - - Reactive pipelines +## Inprogress - Grafana templates (user/system observability) - - More integrations - + - helm chart for Flyte + - Performance optimization + - Flink-K8s + # Available Plugins - Containers - K8s Pods - AWS Batch Arrays - K8s Pod arrays - K8s Spark (native pyspark and java/scala) + - AWS Athena - Qubole Hive - Presto Queries - Distributed Pytorch (K8s Native) - Pytorch Operator - Sagemaker (builtin algorithms & custom models) - Distributed Tensorflow (K8s Native) - TFOperator - Papermill Notebook execution (python and spark) + - Type safe and data checking for Pandas dataframe using Pandera + +## Coming Soon + - Reactive pipelines + - More integrations -## Coming soon - - Flink-K8s # Current Usage - [Freenome](https://www.freenome.com/) @@ -125,10 +151,6 @@ Resources that would help you get a better understanding of Flyte. - [Spotify](https://www.spotify.com/) - [USU Group](https://www.usu.com/) -# Changelogs -[Changelogs](CHANGELOG/README.md) - - # Component Repos Repo | Language | Purpose | Status --- | --- | --- | --- @@ -185,4 +207,3 @@ Thank you to the community for making Flyte possible. - [@catalinii](https://github.com/catalinii) - [@kumare3](https://github.com/kumare3) - diff --git a/rsts/community/index.rst b/rsts/community/index.rst index 6fbb8e96b5..65374e4d7d 100644 --- a/rsts/community/index.rst +++ b/rsts/community/index.rst @@ -12,10 +12,10 @@ so please join us on the - `Slack `_ - `Email `_ - `Twitter `_ +- Community Sync every other Tuesday, 9:00 AM PDT/PST please check out the `calendar `_ We love contributions, so please contribute to - docs - examples - new plugins or plugin ideas - just feedback in general - From 20e9e461362b2644afe5bf92144be1c2ff6372e4 Mon Sep 17 00:00:00 2001 From: Ketan Umare <16888709+kumare3@users.noreply.github.com> Date: Thu, 25 Feb 2021 15:22:19 -0800 Subject: [PATCH 08/74] Update readme badges Signed-off-by: Haytham Abuelfutuh --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 0ef11abede..909152c6ff 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,8 @@ ![Flyte Logo](rsts/images/flyte_lockup_gradient_on_light.png "Flyte Logo") [![Current Release](https://img.shields.io/github/release/lyft/flyte.svg)](https://github.com/lyft/flyte/releases/latest) -[![Build Status](https://travis-ci.org/lyft/flyte.svg?branch=master)](https://travis-ci.org/lyft/flyte) +[![Sandbox Build](https://github.com/flyteorg/flyte/actions/workflows/sandbox.yml/badge.svg)](https://github.com/flyteorg/flyte/actions/workflows/sandbox.yml) +[![End to End tests](https://github.com/flyteorg/flyte/actions/workflows/tests.yml/badge.svg)](https://github.com/flyteorg/flyte/actions/workflows/tests.yml) [![License](https://img.shields.io/badge/LICENSE-Apache2.0-ff69b4.svg)](http://www.apache.org/licenses/LICENSE-2.0.html) ![Commit activity](https://img.shields.io/github/commit-activity/w/lyft/flyte.svg?style=plastic) ![Commit since last release](https://img.shields.io/github/commits-since/lyft/flyte/latest.svg?style=plastic) From 4260e6e0ecfdf8c2f3b667faaa15fdc51f79dbfb Mon Sep 17 00:00:00 2001 From: Ketan Umare <16888709+kumare3@users.noreply.github.com> Date: Fri, 26 Feb 2021 13:06:57 -0800 Subject: [PATCH 09/74] Default Grafana templates for various Flyte components using Prometheus (#779) - These dashboards as also published to grafana marketplace Signed-off-by: Ketan Umare Signed-off-by: Haytham Abuelfutuh --- .gitignore | 1 + Makefile | 14 +- .../prometheus/flyteadmin-dashboard.json | 7749 +++++++++++++++++ .../prometheus/flytepropeller-dashboard.json | 2115 +++++ .../stats/prometheus/flyteuser-dashboard.json | 1691 ++++ requirements.in | 1 + requirements.txt | 10 + stats/README.md | 19 + stats/flyteadmin_dashboard.py | 149 + stats/flytepropeller_dashboard.py | 417 + stats/flyteuser_dashboard.py | 287 + stats/prom.yaml | 20 + 12 files changed, 12468 insertions(+), 5 deletions(-) create mode 100644 deployment/stats/prometheus/flyteadmin-dashboard.json create mode 100644 deployment/stats/prometheus/flytepropeller-dashboard.json create mode 100644 deployment/stats/prometheus/flyteuser-dashboard.json create mode 100644 requirements.in create mode 100644 requirements.txt create mode 100644 stats/README.md create mode 100644 stats/flyteadmin_dashboard.py create mode 100644 stats/flytepropeller_dashboard.py create mode 100644 stats/flyteuser_dashboard.py create mode 100644 stats/prom.yaml diff --git a/.gitignore b/.gitignore index 25dff9d99e..bdde967a1d 100644 --- a/.gitignore +++ b/.gitignore @@ -14,3 +14,4 @@ docs/_sources/ docs/flytekit/flytekit.interfaces.html docs/searchindex.js docs/ +__pycache__/ diff --git a/Makefile b/Makefile index 9081cf87d3..3474016751 100644 --- a/Makefile +++ b/Makefile @@ -20,11 +20,6 @@ end2end: end2end_execute: @end2end/execute.sh -# updates referenced docs from other repositories (e.g. flyteidl, flytekit) -.PHONY: generate-dependent-repo-docs -generate-dependent-repo-docs: - @FLYTEKIT_VERSION=0.15.4 FLYTEIDL_VERSION=0.18.11 ./script/update_ref_docs.sh - .PHONY: install-piptools install-piptools: pip install -U pip-tools @@ -33,3 +28,12 @@ install-piptools: doc-requirements.txt: doc-requirements.in install-piptools $(call PIP_COMPILE,doc-requirements.in) +.PHONY: requirements.txt +requirements.txt: requirements.in install-piptools + $(call PIP_COMPILE,requirements.in) + +.PHONY: stats +stats: + @generate-dashboard -o deployment/stats/prometheus/flytepropeller-dashboard.json stats/flytepropeller_dashboard.py + @generate-dashboard -o deployment/stats/prometheus/flyteadmin-dashboard.json stats/flyteadmin_dashboard.py + @generate-dashboard -o deployment/stats/prometheus/flyteuser-dashboard.json stats/flyteuser_dashboard.py diff --git a/deployment/stats/prometheus/flyteadmin-dashboard.json b/deployment/stats/prometheus/flyteadmin-dashboard.json new file mode 100644 index 0000000000..1742e9b2bd --- /dev/null +++ b/deployment/stats/prometheus/flyteadmin-dashboard.json @@ -0,0 +1,7749 @@ +{ + "__inputs": [ + { + "description": "Prometheus server that connects to Flyte", + "label": "Prometheus", + "name": "DS_PROM", + "pluginId": "prometheus", + "pluginName": "Prometheus", + "type": "datasource" + } + ], + "annotations": { + "list": [] + }, + "description": "Flyte Admin/Control Plane Dashboard. This is great for monitoring FlyteAdmin and the Service API.", + "editable": false, + "gnetId": null, + "hideControls": false, + "id": null, + "links": [], + "refresh": "10s", + "rows": [ + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 1, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_execution:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_execution:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_execution:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_execution:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_execution return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 2, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_execution:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_execution:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_execution success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 3, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:create_execution:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_execution Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "create_execution stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 4, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_launch_plan:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_launch_plan:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_launch_plan:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_launch_plan:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_launch_plan return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 5, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_launch_plan:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_launch_plan:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_launch_plan success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 6, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:create_launch_plan:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_launch_plan Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "create_launch_plan stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 7, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_task:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_task:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_task:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_task:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_task return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 8, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_task:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_task:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_task success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 9, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:create_task:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_task Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "create_task stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 10, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_workflow:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_workflow:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_workflow:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_workflow:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_workflow return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 11, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_workflow:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_workflow:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_workflow success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 12, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:create_workflow:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_workflow Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "create_workflow stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 13, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_node_execution:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_node_execution:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_node_execution:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_node_execution:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_node_execution return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 14, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_node_execution:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_node_execution:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_node_execution success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 15, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:create_node_execution:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_node_execution Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "create_node_execution stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 16, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_task_execution:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_task_execution:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_task_execution:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_task_execution:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_task_execution return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 17, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_task_execution:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:create_task_execution:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_task_execution success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 18, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:create_task_execution:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create_task_execution Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "create_task_execution stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 19, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_execution:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_execution:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_execution:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_execution:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_execution return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 20, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_execution:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_execution:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_execution success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 21, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:get_execution:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_execution Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "get_execution stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 22, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_launch_plan:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_launch_plan:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_launch_plan:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_launch_plan:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_launch_plan return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 23, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_launch_plan:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_launch_plan:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_launch_plan success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 24, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:get_launch_plan:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_launch_plan Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "get_launch_plan stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 25, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_task:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_task:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_task:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_task:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_task return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 26, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_task:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_task:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_task success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 27, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:get_task:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_task Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "get_task stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 28, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_workflow:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_workflow:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_workflow:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_workflow:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_workflow return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 29, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_workflow:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_workflow:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_workflow success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 30, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:get_workflow:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_workflow Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "get_workflow stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 31, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_node_execution:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_node_execution:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_node_execution:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_node_execution:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_node_execution return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 32, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_node_execution:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_node_execution:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_node_execution success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 33, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:get_node_execution:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_node_execution Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "get_node_execution stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 34, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_task_execution:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_task_execution:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_task_execution:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_task_execution:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_task_execution return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 35, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_task_execution:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_task_execution:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_task_execution success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 36, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:get_task_execution:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_task_execution Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "get_task_execution stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 37, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_active_launch_plan:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_active_launch_plan:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_active_launch_plan:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_active_launch_plan:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_active_launch_plan return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 38, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_active_launch_plan:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:get_active_launch_plan:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_active_launch_plan success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 39, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:get_active_launch_plan:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get_active_launch_plan Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "get_active_launch_plan stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 40, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_execution:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_execution:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_execution:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_execution:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_execution return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 41, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_execution:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_execution:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_execution success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 42, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:list_execution:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_execution Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "list_execution stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 43, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_launch_plan:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_launch_plan:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_launch_plan:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_launch_plan:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_launch_plan return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 44, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_launch_plan:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_launch_plan:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_launch_plan success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 45, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:list_launch_plan:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_launch_plan Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "list_launch_plan stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 46, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_task:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_task:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_task:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_task:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_task return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 47, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_task:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_task:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_task success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 48, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:list_task:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_task Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "list_task stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 49, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_workflow:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_workflow:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_workflow:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_workflow:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_workflow return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 50, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_workflow:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_workflow:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_workflow success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 51, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:list_workflow:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_workflow Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "list_workflow stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 52, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_node_execution:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_node_execution:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_node_execution:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_node_execution:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_node_execution return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 53, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_node_execution:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_node_execution:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_node_execution success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 54, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:list_node_execution:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_node_execution Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "list_node_execution stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 55, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_task_execution:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_task_execution:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_task_execution:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_task_execution:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_task_execution return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 56, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_task_execution:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_task_execution:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_task_execution success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 57, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:list_task_execution:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_task_execution Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "list_task_execution stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 58, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_active_launch_plan:codes:OK[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "ok", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_active_launch_plan:codes:InvalidArgument[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "invalid-args", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_active_launch_plan:codes:AlreadyExists[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "already-exists", + "metric": "", + "refId": "C", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_active_launch_plan:codes:FailedPrecondition[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failed-precondition", + "metric": "", + "refId": "D", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_active_launch_plan return codes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 59, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_active_launch_plan:errors[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "errors", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(irate(flyte:admin:list_active_launch_plan:success[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_active_launch_plan success vs errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 60, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:list_active_launch_plan:duration_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_active_launch_plan Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "list_active_launch_plan stats" + } + ], + "schemaVersion": 12, + "sharedCrosshair": false, + "style": "dark", + "tags": [ + "flyte", + "prometheus", + "flyteadmin", + "flyte-controlplane" + ], + "templating": { + "list": [] + }, + "time": { + "from": "now-1h", + "to": "now" + }, + "timepicker": { + "refresh_intervals": [ + "5s", + "10s", + "30s", + "1m", + "5m", + "15m", + "30m", + "1h", + "2h", + "1d" + ], + "time_options": [ + "5m", + "15m", + "1h", + "6h", + "12h", + "24h", + "2d", + "7d", + "30d" + ] + }, + "timezone": "utc", + "title": "Flyte Admin Dashboard (via Prometheus)", + "uid": null, + "version": 0 +} diff --git a/deployment/stats/prometheus/flytepropeller-dashboard.json b/deployment/stats/prometheus/flytepropeller-dashboard.json new file mode 100644 index 0000000000..3dbd64bc4d --- /dev/null +++ b/deployment/stats/prometheus/flytepropeller-dashboard.json @@ -0,0 +1,2115 @@ +{ + "__inputs": [ + { + "description": "Prometheus server that connects to Flyte", + "label": "Prometheus", + "name": "DS_PROM", + "pluginId": "prometheus", + "pluginName": "Prometheus", + "type": "datasource" + } + ], + "annotations": { + "list": [] + }, + "description": "Flyte Propeller Dashboard. This is great for monitoring FlytePropeller / Flyte data plane deployments. This is mostly useful for the Flyte deployment maintainer", + "editable": false, + "gnetId": null, + "hideControls": false, + "id": null, + "links": [], + "refresh": "10s", + "rows": [ + { + "collapse": false, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 1, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:free_workers_count) by (kubernetes_pod_name)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Free workers count", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 2, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:round:abort_error[5m]))*300", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "System errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 3, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(deriv(flyte:propeller:all:round:system_error_unlabeled[5m]))*300", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "System errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 4, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "{__name__=~\"flyte:propeller:all:node:plugin:.*_failure_unlabeled\"}", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "{__name__=~\"flyte:propeller:all:node:plugin:.*_success_unlabeled\"}", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Plugin Failures", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 5, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:round:raw_unlabeled_ms[5m])) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "round Latency by quantile", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 6, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:round:raw_ms[5m])) by (wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "round Latency per workflow", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 7, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:round:panic_unlabeled[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Round panic", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 8, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:collector:flyteworkflow) by (project)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Running Workflows per project", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "Core metrics" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 9, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 6, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "(sum(rate(flyte:propeller:all:metastore:cache_hit[5m])) * 100) / (sum(rate(flyte:propeller:all:metastore:cache_miss[5m])) + sum(rate(flyte:propeller:all:metastore:cache_hit[5m])))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "cache hit percentage", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 10, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 6, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:metastore:head_failure_unlabeled[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "head-failure", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:metastore:bad_container_unlabeled[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "bad-container", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:metastore:bad_key_unlabeled[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "bad-key", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:metastore:read_failure_unlabeled[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "read-failure", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:metastore:write_failure_unlabeled[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "write-failure", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Failures from metastore", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "Metastore failures and cache" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 11, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 3, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:metastore:copy:overall_unlabeled_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Metastore copy latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 12, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 3, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:metastore:write_ms) by (quantile, wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Metastore write latency by workflow", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 13, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 3, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:metastore:read_open_ms) by (quantile, wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Metastore read open latency by workflow", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 14, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 3, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:metastore:head_ms) by (quantile, wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Metastore head latency by workflow", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 15, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 3, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:metastore:proto_fetch_ms) by (quantile, wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "proto-fetch", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(flyte:propeller:all:metastore:remote_fetch_ms) by (quantile, wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "remote-fetch", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Metastore fetch latency by workflow", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "Metastore latencies" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 16, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:node:node_exec_latency_us) by (quantile, wf) / 1000", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Node Exec latency quantile and workflow", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 17, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:node:node_input_latency_ms) by (quantile, wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Node Input latency quantile and workflow", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 18, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:node:event_recording:success_duration_ms) by (quantile, wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(flyte:propeller:all:node:event_recording:failure_duration_ms) by (quantile, wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Node Event event recording latency quantile and workflow", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "Node Metrics" + } + ], + "schemaVersion": 12, + "sharedCrosshair": false, + "style": "dark", + "tags": [ + "flyte", + "prometheus", + "flytepropeller", + "flyte-dataplane" + ], + "templating": { + "list": [] + }, + "time": { + "from": "now-1h", + "to": "now" + }, + "timepicker": { + "refresh_intervals": [ + "5s", + "10s", + "30s", + "1m", + "5m", + "15m", + "30m", + "1h", + "2h", + "1d" + ], + "time_options": [ + "5m", + "15m", + "1h", + "6h", + "12h", + "24h", + "2d", + "7d", + "30d" + ] + }, + "timezone": "utc", + "title": "Flyte Propeller Dashboard (via Prometheus)", + "uid": null, + "version": 0 +} diff --git a/deployment/stats/prometheus/flyteuser-dashboard.json b/deployment/stats/prometheus/flyteuser-dashboard.json new file mode 100644 index 0000000000..3ec252f2e4 --- /dev/null +++ b/deployment/stats/prometheus/flyteuser-dashboard.json @@ -0,0 +1,1691 @@ +{ + "__inputs": [ + { + "description": "Prometheus server that connects to Flyte", + "label": "Prometheus", + "name": "DS_PROM", + "pluginId": "prometheus", + "pluginName": "Prometheus", + "type": "datasource" + } + ], + "annotations": { + "list": [] + }, + "description": "Flyte User Dashboard. This is great to get a birds-eye and drill down view of executions in your Flyte cluster. Useful for the user.", + "editable": false, + "gnetId": null, + "hideControls": false, + "id": null, + "links": [], + "refresh": "10s", + "rows": [ + { + "collapse": false, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 1, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:workflow:accepted{project=~\"$project\", domain=~\"$domain\", wf=~\"$workflow\"}[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Accepted Workflow", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 2, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:workflow:success_duration_ms_count{project=~\"$project\", domain=~\"$domain\", wf=~\"$workflow\"}[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Successful Workflow", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 3, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:workflow:failure_duration_ms_count{project=~\"$project\", domain=~\"$domain\", wf=~\"$workflow\"}[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Failed Workflow", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 4, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:workflow:workflow_aborted{project=~\"$project\", domain=~\"$domain\", wf=~\"$workflow\"}[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Aborted Workflow", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 5, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:workflow:success_duration_ms{project=~\"$project\", domain=~\"$domain\", wf=~\"$workflow\"}) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Successful workflow execution time by Quantile", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 6, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:workflow:failed_duration_ms{project=~\"$project\", domain=~\"$domain\", wf=~\"$workflow\"}) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Failed workflow execution time by Quantile", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 7, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:node:queueing_latency_ms{project=~\"$project\", domain=~\"$domain\", wf=~\"$workflow\"}) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Node queuing latency by Quantile", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "Workflow Stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 8, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 6, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "kube_resourcequota{resource=\"limits.cpu\", namespace=\"$project-$domain\", type=\"hard\"}", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "max cpu", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "kube_resourcequota{resource=\"limits.cpu\", namespace=\"$project-$domain\", type=\"used\"}", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "used cpu", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "CPU Limits vs usage", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 9, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 6, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "kube_resourcequota{resource=\"limits.memory\", namespace=\"$project-$domain\", type=\"hard\"}", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "max mem", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "kube_resourcequota{resource=\"limits.memory\", namespace=\"$project-$domain\", type=\"used\"}", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "used mem", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Mem Limits vs usage", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "Kubernetes Quota Usage stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 10, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(kube_pod_container_status_waiting * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=\"\",namespace=~\"$project-$domain\",label_workflow_name=~\"$workflow\"}) by (namespace, label_execution_id, label_task_name, label_node_id, label_workflow_name) > 0", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Pending tasks", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 11, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "(max(container_memory_rss{image!=\"\"} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=\"\",namespace=~\"$project-$domain\",label_workflow_name=~\"$workflow\"} * on(pod) group_left(phase) kube_pod_status_phase{phase=\"Running\"}) by (namespace, pod, label_execution_id, label_task_name, label_node_id, label_workflow_name) / max(kube_pod_container_resource_limits_memory_bytes{container!=\"\"} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=\"\"} * on(pod) group_left(phase) kube_pod_status_phase{phase=\"Running\"}) by (namespace, pod, label_execution_id, label_task_name, label_node_id, label_workflow_name)) > 0", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Memory Usage Percentage", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 12, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 4, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "(sum(rate(container_cpu_usage_seconds_total{image!=\"\"}[2m]) * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=\"\",namespace=~\"$project-$domain\",label_workflow_name=~\"$workflow\"} * on(pod) group_left(phase) kube_pod_status_phase{phase=\"Running\"}) by (namespace, pod, label_execution_id, label_task_name, label_node_id, label_workflow_name) / sum(kube_pod_container_resource_limits_cpu_cores{container!=\"\"} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=\"\"} * on(pod) group_left(phase) kube_pod_status_phase{phase=\"Running\"}) by (namespace, pod, label_execution_id, label_task_name, label_node_id, label_workflow_name)) > 0", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "CPU Usage Percentage", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "Task stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 13, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 6, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:node:user_error_duration_ms_count{project=~\"$project\",domain=~\"$domain\",wf=~\"$project:$domain:$workflow\"}[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "User errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 14, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 6, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:node:system_error_duration_ms_count{project=~\"$project\",domain=~\"$domain\",wf=~\"$project:$domain:$workflow\"}[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "System errors", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "Error (System vs user)" + } + ], + "schemaVersion": 12, + "sharedCrosshair": false, + "style": "dark", + "tags": [ + "flyte", + "prometheus", + "flyteuser", + "flyte-user" + ], + "templating": { + "list": [ + { + "allValue": null, + "current": { + "tags": [], + "text": null, + "value": null + }, + "datasource": "${DS_PROM}", + "hide": 0, + "includeAll": false, + "label": null, + "multi": false, + "name": "project", + "options": [], + "query": "label_values(flyte:propeller:all:collector:flyteworkflow, project)", + "refresh": 1, + "regex": null, + "sort": true, + "tagValuesQuery": null, + "tagsQuery": null, + "type": "query", + "useTags": false + }, + { + "allValue": null, + "current": { + "tags": [], + "text": null, + "value": null + }, + "datasource": "${DS_PROM}", + "hide": 0, + "includeAll": false, + "label": null, + "multi": false, + "name": "domain", + "options": [], + "query": "label_values(flyte:propeller:all:collector:flyteworkflow, domain)", + "refresh": 1, + "regex": null, + "sort": true, + "tagValuesQuery": null, + "tagsQuery": null, + "type": "query", + "useTags": false + }, + { + "allValue": ".*", + "current": { + "tags": [], + "text": null, + "value": null + }, + "datasource": "${DS_PROM}", + "hide": 0, + "includeAll": true, + "label": null, + "multi": false, + "name": "workflow", + "options": [], + "query": "label_values(flyte:propeller:all:collector:flyteworkflow, wf)", + "refresh": 1, + "regex": null, + "sort": true, + "tagValuesQuery": null, + "tagsQuery": null, + "type": "query", + "useTags": false + } + ] + }, + "time": { + "from": "now-1h", + "to": "now" + }, + "timepicker": { + "refresh_intervals": [ + "5s", + "10s", + "30s", + "1m", + "5m", + "15m", + "30m", + "1h", + "2h", + "1d" + ], + "time_options": [ + "5m", + "15m", + "1h", + "6h", + "12h", + "24h", + "2d", + "7d", + "30d" + ] + }, + "timezone": "utc", + "title": "Flyte User Dashboard (via Prometheus)", + "uid": null, + "version": 0 +} diff --git a/requirements.in b/requirements.in new file mode 100644 index 0000000000..7c4e34693c --- /dev/null +++ b/requirements.in @@ -0,0 +1 @@ +grafanalib diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000000..800331e6e7 --- /dev/null +++ b/requirements.txt @@ -0,0 +1,10 @@ +# +# This file is autogenerated by pip-compile +# To update, run: +# +# pip-compile requirements.in +# +attrs==20.3.0 + # via grafanalib +grafanalib==0.5.9 + # via -r requirements.in diff --git a/stats/README.md b/stats/README.md new file mode 100644 index 0000000000..2ecea73bd5 --- /dev/null +++ b/stats/README.md @@ -0,0 +1,19 @@ +## Developing stats + +These scripts generate dashboards by running the following make target in the +flyteorg/flyte root folder. + +```bash +make stats +``` + +Ensure that you have installed the requirements.txt + +```bash +pip install -r requirements.txt +``` + +Refer to [Grafanalib](https://github.com/weaveworks/grafanalib) to understand +how to write the dashboards. + +Currently the dashboards are manually uploaded to [Grafana marketplace](https://grafana.com/grafana/dashboards?search=flyte) diff --git a/stats/flyteadmin_dashboard.py b/stats/flyteadmin_dashboard.py new file mode 100644 index 0000000000..c7c660e74b --- /dev/null +++ b/stats/flyteadmin_dashboard.py @@ -0,0 +1,149 @@ +import typing +from grafanalib.core import ( + Alert, AlertCondition, Dashboard, Graph, + GreaterThan, OP_AND, OPS_FORMAT, Row, RTYPE_SUM, SECONDS_FORMAT, + SHORT_FORMAT, single_y_axis, Target, TimeRange, YAxes, YAxis, DataSourceInput +) + +# ------------------------------ +# For Gostats we recommend using +# Grafana dashboard ID 10826 - https://grafana.com/grafana/dashboards/10826 +# + +DATASOURCE_NAME = "DS_PROM" +DATASOURCE = "${%s}" % DATASOURCE_NAME + + +class FlyteAdmin(object): + APIS = [ + "create_execution", + "create_launch_plan", + "create_task", + "create_workflow", + "create_node_execution", + "create_task_execution", + "get_execution", + "get_launch_plan", + "get_task", + "get_workflow", + "get_node_execution", + "get_task_execution", + "get_active_launch_plan", + "list_execution", + "list_launch_plan", + "list_task", + "list_workflow", + "list_node_execution", + "list_task_execution", + "list_active_launch_plan", + ] + + @staticmethod + def error_codes(api: str, interval: int = 1) -> Graph: + return Graph( + title=f"{api} return codes", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(irate(flyte:admin:{api}:codes:OK[{interval}m]))', + legendFormat="ok", + refId='A', + ), + Target( + expr=f'sum(irate(flyte:admin:{api}:codes:InvalidArgument[{interval}m]))', + legendFormat="invalid-args", + refId='B', + ), + Target( + expr=f'sum(irate(flyte:admin:{api}:codes:AlreadyExists[{interval}m]))', + legendFormat="already-exists", + refId='C', + ), + Target( + expr=f'sum(irate(flyte:admin:{api}:codes:FailedPrecondition[{interval}m]))', + legendFormat="failed-precondition", + refId='D', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ) + + @staticmethod + def error_vs_success(api: str, interval: int = 1) -> Graph: + return Graph( + title=f"{api} success vs errors", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(irate(flyte:admin:{api}:errors[{interval}m]))', + legendFormat="errors", + refId='A', + ), + Target( + expr=f'sum(irate(flyte:admin:{api}:success[{interval}m]))', + legendFormat="success", + refId='B', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ) + + @staticmethod + def api_latency(api: str, interval: int = 1) -> Graph: + return Graph( + title=f"{api} Latency", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:admin:{api}:duration_ms[{interval}m])) by (quantile)', + refId='A', + ), + ], + yAxes=single_y_axis(format=SECONDS_FORMAT), + ) + + @staticmethod + def create_api_row(api: str, collapse: bool, interval: int = 1) -> Row: + return Row( + title=f"{api} stats", + collapse=collapse, + panels=[ + FlyteAdmin.error_codes(api, interval), + FlyteAdmin.error_vs_success(api, interval), + FlyteAdmin.api_latency(api, interval), + ] + ) + + @staticmethod + def create_all_apis(interval: int = 5) -> typing.List[Row]: + rows = [] + for api in FlyteAdmin.APIS: + rows.append(FlyteAdmin.create_api_row(api, collapse=True, interval=interval)) + return rows + + +dashboard = Dashboard( + tags=["flyte", "prometheus", "flyteadmin", "flyte-controlplane"], + inputs=[ + DataSourceInput( + name=DATASOURCE_NAME, + label="Prometheus", + description="Prometheus server that connects to Flyte", + pluginId="prometheus", + pluginName="Prometheus", + ), + ], + editable=False, + title="Flyte Admin Dashboard (via Prometheus)", + rows=FlyteAdmin.create_all_apis(interval=5), + description="Flyte Admin/Control Plane Dashboard. This is great for monitoring FlyteAdmin and the Service API.", +).auto_panel_ids() + +if __name__ == "__main__": + print(dashboard.to_json_data()) diff --git a/stats/flytepropeller_dashboard.py b/stats/flytepropeller_dashboard.py new file mode 100644 index 0000000000..f3cd01618d --- /dev/null +++ b/stats/flytepropeller_dashboard.py @@ -0,0 +1,417 @@ +import typing +from grafanalib.core import ( + Alert, AlertCondition, Dashboard, Graph, + GreaterThan, OP_AND, OPS_FORMAT, Row, RTYPE_SUM, SECONDS_FORMAT, + SHORT_FORMAT, single_y_axis, Target, TimeRange, YAxes, YAxis, MILLISECONDS_FORMAT, DataSourceInput +) + +# ------------------------------ +# For Gostats we recommend using +# Grafana dashboard ID 10826 - https://grafana.com/grafana/dashboards/10826 +# + +DATASOURCE_NAME = "DS_PROM" +DATASOURCE = "${%s}" % DATASOURCE_NAME + + +class FlytePropeller(object): + + @staticmethod + def create_free_workers() -> Graph: + return Graph( + title="Free workers count", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(flyte:propeller:all:free_workers_count) by (kubernetes_pod_name)', + refId='A', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ) + + @staticmethod + def round_latency_per_wf(interval: int = 1) -> Graph: + return Graph( + title=f"round Latency per workflow", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:round:raw_ms[{interval}m])) by (wf)', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ) + + @staticmethod + def round_latency(interval: int = 1) -> Graph: + return Graph( + title=f"round Latency by quantile", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:round:raw_unlabeled_ms[{interval}m])) by (quantile)', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ) + + @staticmethod + def round_panic() -> Graph: + return Graph( + title="Round panic", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(rate(flyte:propeller:all:round:panic_unlabeled[5m]))', + refId='A', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ) + + @staticmethod + def streak_length() -> Graph: + return Graph( + title="Avg streak length", + dataSource=DATASOURCE, + targets=[ + Target( + expr='avg(flyte:propeller:all:round:streak_length_unlabeled)', + refId='A', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ) + + @staticmethod + def system_errors() -> Graph: + return Graph( + title="System errors", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(deriv(flyte:propeller:all:round:system_error_unlabeled[5m]))*300', + refId='A', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ) + + @staticmethod + def abort_errors() -> Graph: + return Graph( + title="System errors", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(rate(flyte:propeller:all:round:abort_error[5m]))*300', + refId='A', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ) + + @staticmethod + def workflows_per_project() -> Graph: + return Graph( + title=f"Running Workflows per project", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(flyte:propeller:all:collector:flyteworkflow) by (project)', + refId='A', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ) + + @staticmethod + def plugin_success_vs_failures() -> Graph: + """ + TODO We need to convert the plugin names to be labels, so that prometheus can perform queries correctly + """ + return Graph( + title=f"Plugin Failures", + dataSource=DATASOURCE, + targets=[ + Target( + expr='{__name__=~"flyte:propeller:all:node:plugin:.*_failure_unlabeled"}', + refId='A', + ), + Target( + expr='{__name__=~"flyte:propeller:all:node:plugin:.*_success_unlabeled"}', + refId='A', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ) + + @staticmethod + def node_exec_latency() -> Graph: + return Graph( + title=f"Node Exec latency quantile and workflow", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(flyte:propeller:all:node:node_exec_latency_us) by (quantile, wf) / 1000', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ) + + @staticmethod + def node_event_recording_latency() -> Graph: + return Graph( + title=f"Node Event event recording latency quantile and workflow", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(flyte:propeller:all:node:event_recording:success_duration_ms) by (quantile, wf)', + refId='A', + ), + Target( + expr=f'sum(flyte:propeller:all:node:event_recording:failure_duration_ms) by (quantile, wf)', + refId='B', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ) + + @staticmethod + def node_input_latency() -> Graph: + return Graph( + title=f"Node Input latency quantile and workflow", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(flyte:propeller:all:node:node_input_latency_ms) by (quantile, wf)', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ) + + @staticmethod + def metastore_failures(): + # Copy counts sum(rate(flyte:propeller:all:metastore:copy:overall_unlabeled_ms_count[5m])) + return Graph( + title=f"Failures from metastore", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:metastore:head_failure_unlabeled[5m]))', + legendFormat="head-failure", + refId='A', + ), + Target( + expr=f'sum(rate(flyte:propeller:all:metastore:bad_container_unlabeled[5m]))', + legendFormat="bad-container", + refId='A', + ), + Target( + expr=f'sum(rate(flyte:propeller:all:metastore:bad_key_unlabeled[5m]))', + legendFormat="bad-key", + refId='A', + ), + Target( + expr=f'sum(rate(flyte:propeller:all:metastore:read_failure_unlabeled[5m]))', + legendFormat="read-failure", + refId='A', + ), + Target( + expr=f'sum(rate(flyte:propeller:all:metastore:write_failure_unlabeled[5m]))', + legendFormat="write-failure", + refId='A', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ) + + @staticmethod + def metastore_cache_hit_percentage(interval: int) -> Graph: + """ + TODO replace with metric math maybe? + """ + return Graph( + title="cache hit percentage", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'(sum(rate(flyte:propeller:all:metastore:cache_hit[{interval}m])) * 100) / (sum(rate(flyte:propeller:all:metastore:cache_miss[{interval}m])) + sum(rate(flyte:propeller:all:metastore:cache_hit[{interval}m])))', + refId='A', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ) + + @staticmethod + def metastore_latencies(collapse: bool) -> Row: + return Row( + title=f"Metastore latencies", + collapse=collapse, + panels=[ + Graph( + title=f"Metastore copy latency", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(flyte:propeller:all:metastore:copy:overall_unlabeled_ms) by (quantile)', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ), + Graph( + title=f"Metastore write latency by workflow", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(flyte:propeller:all:metastore:write_ms) by (quantile, wf)', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ), + Graph( + title=f"Metastore read open latency by workflow", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(flyte:propeller:all:metastore:read_open_ms) by (quantile, wf)', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ), + Graph( + title=f"Metastore head latency by workflow", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(flyte:propeller:all:metastore:head_ms) by (quantile, wf)', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ), + Graph( + title=f"Metastore fetch latency by workflow", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(flyte:propeller:all:metastore:proto_fetch_ms) by (quantile, wf)', + legendFormat="proto-fetch", + refId='A', + ), + + Target( + expr='sum(flyte:propeller:all:metastore:remote_fetch_ms) by (quantile, wf)', + legendFormat="remote-fetch", + refId='B', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ), + ] + ) + + @staticmethod + def metastore_metrics(interval: int, collapse: bool) -> Row: + return Row( + title="Metastore failures and cache", + collapse=collapse, + panels=[ + FlytePropeller.metastore_cache_hit_percentage(interval), + FlytePropeller.metastore_failures(), + ], + ) + + @staticmethod + def node_metrics(collapse: bool) -> Row: + return Row( + title="Node Metrics", + collapse=collapse, + panels=[ + FlytePropeller.node_exec_latency(), + FlytePropeller.node_input_latency(), + FlytePropeller.node_event_recording_latency(), + ], + ) + + @staticmethod + def core_metrics(interval: int, collapse: bool) -> Row: + return Row( + title="Core metrics", + collapse=collapse, + panels=[ + FlytePropeller.create_free_workers(), + FlytePropeller.abort_errors(), + FlytePropeller.system_errors(), + FlytePropeller.plugin_success_vs_failures(), + FlytePropeller.round_latency(interval), + FlytePropeller.round_latency_per_wf(interval), + FlytePropeller.round_panic(), + FlytePropeller.workflows_per_project(), + ], + ) + + @staticmethod + def create_all_rows(interval: int = 5) -> typing.List[Row]: + return [ + FlytePropeller.core_metrics(interval, False), + FlytePropeller.metastore_metrics(interval, True), + FlytePropeller.metastore_latencies(True), + FlytePropeller.node_metrics(True), + ] + + +dashboard = Dashboard( + tags=["flyte", "prometheus", "flytepropeller", "flyte-dataplane"], + inputs=[ + DataSourceInput( + name=DATASOURCE_NAME, + label="Prometheus", + description="Prometheus server that connects to Flyte", + pluginId="prometheus", + pluginName="Prometheus", + ), + ], + editable=False, + title="Flyte Propeller Dashboard (via Prometheus)", + rows=FlytePropeller.create_all_rows(interval=5), + description="Flyte Propeller Dashboard. This is great for monitoring FlytePropeller / Flyte data plane deployments. This is mostly useful for the Flyte deployment maintainer", +).auto_panel_ids() + +if __name__ == "__main__": + print(dashboard.to_json_data()) diff --git a/stats/flyteuser_dashboard.py b/stats/flyteuser_dashboard.py new file mode 100644 index 0000000000..a094108955 --- /dev/null +++ b/stats/flyteuser_dashboard.py @@ -0,0 +1,287 @@ +import typing +from grafanalib.core import ( + Alert, AlertCondition, Dashboard, Graph, + GreaterThan, OP_AND, OPS_FORMAT, Row, RTYPE_SUM, SECONDS_FORMAT, + SHORT_FORMAT, single_y_axis, Target, TimeRange, YAxes, YAxis, MILLISECONDS_FORMAT, Templating, Template, + DataSourceInput +) + +DATASOURCE_NAME = "DS_PROM" +DATASOURCE = "${%s}" % DATASOURCE_NAME + + +class FlyteUserDashboard(object): + + @staticmethod + def workflow_stats(collapse: bool) -> Row: + return Row( + title="Workflow Stats", + collapse=collapse, + panels=[ + Graph( + title="Accepted Workflow", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(rate(flyte:propeller:all:workflow:accepted{project=~"$project", domain=~"$domain", wf=~"$workflow"}[5m]))', + refId='A', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ), + Graph( + title="Successful Workflow", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(rate(flyte:propeller:all:workflow:success_duration_ms_count{project=~"$project", domain=~"$domain", wf=~"$workflow"}[5m]))', + refId='A', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ), + Graph( + title="Failed Workflow", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(rate(flyte:propeller:all:workflow:failure_duration_ms_count{project=~"$project", domain=~"$domain", wf=~"$workflow"}[5m]))', + refId='A', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ), + Graph( + title="Aborted Workflow", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(rate(flyte:propeller:all:workflow:workflow_aborted{project=~"$project", domain=~"$domain", wf=~"$workflow"}[5m]))', + refId='A', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ), + Graph( + title="Successful workflow execution time by Quantile", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(flyte:propeller:all:workflow:success_duration_ms{project=~"$project", domain=~"$domain", wf=~"$workflow"}) by (quantile)', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ), + Graph( + title="Failed workflow execution time by Quantile", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(flyte:propeller:all:workflow:failed_duration_ms{project=~"$project", domain=~"$domain", wf=~"$workflow"}) by (quantile)', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ), + Graph( + title="Node queuing latency by Quantile", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(flyte:propeller:all:node:queueing_latency_ms{project=~"$project", domain=~"$domain", wf=~"$workflow"}) by (quantile)', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ), + ]) + + @staticmethod + def quota_stats(collapse: bool) -> Row: + return Row( + title="Kubernetes Quota Usage stats", + collapse=collapse, + panels=[ + Graph( + title="CPU Limits vs usage", + dataSource=DATASOURCE, + targets=[ + Target( + expr='kube_resourcequota{resource="limits.cpu", namespace="$project-$domain", type="hard"}', + refId='A', + legendFormat="max cpu", + ), + Target( + expr='kube_resourcequota{resource="limits.cpu", namespace="$project-$domain", type="used"}', + refId='B', + legendFormat="used cpu", + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ), + Graph( + title="Mem Limits vs usage", + dataSource=DATASOURCE, + targets=[ + Target( + expr='kube_resourcequota{resource="limits.memory", namespace="$project-$domain", type="hard"}', + refId='A', + legendFormat="max mem", + ), + Target( + expr='kube_resourcequota{resource="limits.memory", namespace="$project-$domain", type="used"}', + refId='B', + legendFormat="used mem", + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ), + ]) + + @staticmethod + def resource_stats(collapse: bool) -> Row: + return Row( + title="Task stats", + collapse=collapse, + panels=[ + Graph( + title="Pending tasks", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(kube_pod_container_status_waiting * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !="",namespace=~"$project-$domain",label_workflow_name=~"$workflow"}) by (namespace, label_execution_id, label_task_name, label_node_id, label_workflow_name) > 0', + refId='A', + ), + ], + yAxes=single_y_axis(format=SHORT_FORMAT), + ), + Graph( + title="Memory Usage Percentage", + dataSource=DATASOURCE, + targets=[ + Target( + expr='(max(container_memory_rss{image!=""} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !="",namespace=~"$project-$domain",label_workflow_name=~"$workflow"} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, pod, label_execution_id, label_task_name, label_node_id, label_workflow_name) / max(kube_pod_container_resource_limits_memory_bytes{container!=""} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=""} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, pod, label_execution_id, label_task_name, label_node_id, label_workflow_name)) > 0', + refId='A', + ), + ], + yAxes=single_y_axis(format=SHORT_FORMAT), + ), + Graph( + title="CPU Usage Percentage", + dataSource=DATASOURCE, + targets=[ + Target( + expr='(sum(rate(container_cpu_usage_seconds_total{image!=""}[2m]) * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !="",namespace=~"$project-$domain",label_workflow_name=~"$workflow"} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, pod, label_execution_id, label_task_name, label_node_id, label_workflow_name) / sum(kube_pod_container_resource_limits_cpu_cores{container!=""} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=""} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, pod, label_execution_id, label_task_name, label_node_id, label_workflow_name)) > 0', + refId='A', + ), + ], + yAxes=single_y_axis(format=SHORT_FORMAT), + ), + ]) + + @staticmethod + def errors(collapse: bool) -> Row: + return Row( + title="Error (System vs user)", + collapse=collapse, + panels=[ + Graph( + title="User errors", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(rate(flyte:propeller:all:node:user_error_duration_ms_count{project=~"$project",domain=~"$domain",wf=~"$project:$domain:$workflow"}[5m]))', + refId='A', + ), + ], + yAxes=single_y_axis(format=SHORT_FORMAT), + ), + Graph( + title="System errors", + dataSource=DATASOURCE, + targets=[ + Target( + expr='sum(rate(flyte:propeller:all:node:system_error_duration_ms_count{project=~"$project",domain=~"$domain",wf=~"$project:$domain:$workflow"}[5m]))', + refId='A', + ), + ], + yAxes=single_y_axis(format=SHORT_FORMAT), + ), + ]) + + @staticmethod + def create_all_rows(interval: int) -> typing.List[Row]: + return [ + FlyteUserDashboard.workflow_stats(False), + FlyteUserDashboard.quota_stats(True), + FlyteUserDashboard.resource_stats(True), + FlyteUserDashboard.errors(True), + ] + + +project_template = Template( + name="project", + dataSource=DATASOURCE, + query="label_values(flyte:propeller:all:collector:flyteworkflow, project)", + sort=True, +) + +domain_template = Template( + name="domain", + dataSource=DATASOURCE, + query="label_values(flyte:propeller:all:collector:flyteworkflow, domain)", + sort=True, +) + +wf_template = Template( + name="workflow", + dataSource=DATASOURCE, + query="label_values(flyte:propeller:all:collector:flyteworkflow, wf)", + sort=True, + allValue=".*", + includeAll=True, +) + +dashboard = Dashboard( + tags=["flyte", "prometheus", "flyteuser", "flyte-user"], + inputs=[ + DataSourceInput( + name=DATASOURCE_NAME, + label="Prometheus", + description="Prometheus server that connects to Flyte", + pluginId="prometheus", + pluginName="Prometheus", + ), + ], + editable=False, + title="Flyte User Dashboard (via Prometheus)", + rows=FlyteUserDashboard.create_all_rows(interval=5), + templating=Templating(list=[ + project_template, + domain_template, + wf_template, + ]), + description="Flyte User Dashboard. This is great to get a birds-eye and drill down view of executions in your Flyte cluster. Useful for the user.", +).auto_panel_ids() + +if __name__ == "__main__": + print(dashboard.to_json_data()) diff --git a/stats/prom.yaml b/stats/prom.yaml new file mode 100644 index 0000000000..6268f856a3 --- /dev/null +++ b/stats/prom.yaml @@ -0,0 +1,20 @@ +- name: flyterunningtask_metrics + rules: + - record: flyterunningtask:pending + expr: sum(kube_pod_container_status_waiting * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=""}) by (namespace, label_execution_id, label_task_name, label_node_id, label_workflow_name) > 0 + - record: flyterunningtask:cpu:limit + expr: sum(kube_pod_container_resource_limits_cpu_cores{container!=""} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=""} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, label_execution_id, label_task_name, label_node_id, label_workflow_name) > 0 + - record: flyterunningtask:cpu:request + expr: sum(kube_pod_container_resource_requests_cpu_cores{container!=""} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=""} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, label_execution_id, label_task_name, label_node_id, label_workflow_name) > 0 + - record: flyterunningtask:cpu:usage:percentage + expr: (sum(rate(container_cpu_usage_seconds_total{image!=""}[2m]) * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=""} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, label_execution_id, label_task_name, label_node_id, label_workflow_name) / sum(kube_pod_container_resource_limits_cpu_cores{container!=""} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=""} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, label_execution_id, label_task_name, label_node_id, label_workflow_name)) > 0 + - record: flyterunningtask:pods:cpu:usage:percentage + expr: (sum(rate(container_cpu_usage_seconds_total{image!=""}[2m]) * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=""} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, pod, label_execution_id, label_task_name, label_node_id, label_workflow_name) / sum(kube_pod_container_resource_limits_cpu_cores{container!=""} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=""} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, pod, label_execution_id, label_task_name, label_node_id, label_workflow_name)) > 0 + - record: flyterunningtask:memory:limit + expr: sum(kube_pod_container_resource_limits_memory_bytes{container!=""} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=""} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, label_execution_id, label_task_name, label_node_id, label_workflow_name) > 0 + - record: flyterunningtask:memory:request + expr: sum(kube_pod_container_resource_requests_memory_bytes{container!=""} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=""} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, label_execution_id, label_task_name, label_node_id, label_workflow_name) > 0 + - record: flyterunningtask:memory:usage:percentage + expr: (max(container_memory_rss{image!=""} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=""} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, label_execution_id, label_task_name, label_node_id, label_workflow_name) / max(kube_pod_container_resource_limits_memory_bytes{container!=""} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=""} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, label_execution_id, label_task_name, label_node_id, label_workflow_name)) > 0 + - record: flyterunningtask:pods:memory:usage:percentage + expr: (max(container_memory_rss{image!=""} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=""} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, pod, label_execution_id, label_task_name, label_node_id, label_workflow_name) / max(kube_pod_container_resource_limits_memory_bytes{container!=""} * on(pod) group_left(label_execution_id, label_task_name, label_node_id, label_workflow_name) kube_pod_labels{label_execution_id !=""} * on(pod) group_left(phase) kube_pod_status_phase{phase="Running"}) by (namespace, pod, label_execution_id, label_task_name, label_node_id, label_workflow_name)) > 0 From cfe7784df89013056a9b26e9f4147887a27e8608 Mon Sep 17 00:00:00 2001 From: Ketan Umare <16888709+kumare3@users.noreply.github.com> Date: Sat, 27 Feb 2021 13:50:02 -0800 Subject: [PATCH 10/74] Docs Update: Extending flyte & deploying observability (#780) Signed-off-by: Ketan Umare Signed-off-by: Haytham Abuelfutuh --- rsts/dive_deep/observability.rst | 10 +---- rsts/howto/monitoring/index.rst | 17 ++++++- rsts/plugins/extend/flyte_backend.rst | 60 ++++++++++++++++++++++++- rsts/plugins/extend/flytekit_python.rst | 7 ++- rsts/plugins/extend/intro.rst | 2 + 5 files changed, 85 insertions(+), 11 deletions(-) diff --git a/rsts/dive_deep/observability.rst b/rsts/dive_deep/observability.rst index 4b7354095c..c9b4d69b0b 100644 --- a/rsts/dive_deep/observability.rst +++ b/rsts/dive_deep/observability.rst @@ -3,6 +3,8 @@ Metrics for your executions =========================== +.. tip:: Refer to :ref:`howto-monitoring` to see/use prebuilt dashboards published to Grafana Marketplace. The following section explains some other metrics that are very important. + Flyte-Provided Metrics ~~~~~~~~~~~~~~~~~~~~~~~ Whenever you run a workflow, Flyte Platform automatically emits high-level metrics. These metrics follow a consistent schema and @@ -57,14 +59,6 @@ wf This is the name of the workflow that was executing when this m ``{{project}}:{{domain}}:{{workflow_name}}`` =============== ================================================================================= -Flyte Task Container Metrics -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -To be built. - -Default Project Dashboard -^^^^^^^^^^^^^^^^^^^^^^^^^^^ -To be built. - User Stats With Flyte ~~~~~~~~~~~~~~~~~~~~~~ The workflow parameters object that the SDK injects into the various tasks has a statsd handle that users should call diff --git a/rsts/howto/monitoring/index.rst b/rsts/howto/monitoring/index.rst index 3e6bda1f15..044864cbc8 100644 --- a/rsts/howto/monitoring/index.rst +++ b/rsts/howto/monitoring/index.rst @@ -1,5 +1,20 @@ -.. _howto_monitoring: +.. _howto-monitoring: ###################################### How do I monitor my Flyte deployment? ###################################### + +.. tip:: The flyte core team publishes a maintains Grafana dashboards built using prometheus data source and can be found `here `_. + +Flyte Backend is written in Golang and exposes stats using Prometheus. The Stats themselves are labeled with the Workflow, Task, Project & Domain whereever appropriate. + +The dashboards are divided into primarily 2 types + +- User facing dashboards. These are dashboards that a user can use to triage/investigate/observe performance and characterisitics for their Workflows and tasks. + The User facing dashboard is published under Grafana marketplace ID `13980 `_ + +- System Dashboards. These dashboards are useful for the system maintainer to maintain their Flyte deployments. These are further divided into + - DataPlane/FlytePropeller dashboards published @ `13979 `_ + - ControlPlane/Flyteadmin dashboards published @ `13981 `_ + +These are basic dashboards and do no include all the metrics that are exposed by Flyte. You can contribute to the dashboards and help us improve them - by referring to the build scripts `here `_. diff --git a/rsts/plugins/extend/flyte_backend.rst b/rsts/plugins/extend/flyte_backend.rst index 680d54698c..a1524db9cf 100644 --- a/rsts/plugins/extend/flyte_backend.rst +++ b/rsts/plugins/extend/flyte_backend.rst @@ -2,4 +2,62 @@ ######################################## Implement Backend Extensions (advanced) -######################################## \ No newline at end of file +######################################## + +Now that you have landed here, we can assume that you have exhausted your options of extending and want to extend Flyte in a way that adds new capabilities to the platform. + +Let us try to recap, why we should implement a backend plugin, + +#. We want to add a new capability to the Flyte Platform, for e.g. + * ability to talk to a new service like AWS Sagemaker, Snowflake, Redshift, Athena, BigQuery etc + * ability to orchestrate a set of containers in a new way like Spark, Flink, Distributed training on Kubernetes (usually using a Kubernetes operator) + * use a new container orchestration engine like AWS Batch/ECS, Hashicorp' Nomad + * use a completely new runtime like AWS Lambda, KNative etc +#. For the case of talking to a new service like in 1.a - this can be done using flytekit extensions and usually is the better way to get started. But, once matured most of these extensions are better to be migrated to the backend. For the rest of the cases, it is possible to extend flytekit to achieve these scenarios, but this is less desirable, because of the associated overhead of first launching a container that launches these jobs downstream. +#. You want to retain the capability to updating the plugin implementation and roll out new changes, fixes without affecting the users code, or requiring them to update versions of their plugins etc +#. You want the same plugin to be accessible across multiple language SDK's - build a universal plugin + +Basics +======= +We will try to understand the components of a backend plugin using an example plugin - :ref:`plugins-spark-k8s` A Flyte backend extension consists of 3 parts + +Interface specification +------------------------ +Usually Flyte extensions need information that is not covered by a :std:ref:`Flyte TaskTemplate `. The TaskTemplate consists of a +the interface, task_type identifier, some metadata and other fields. An important field to note here is - :std:ref:`api_field_flyteidl.core.tasktemplate.custom`. The custom field is essentially an unstructured JSON. +This makes it possible to extend a task-template beyond the default supported targets -- :std:ref:`api_field_flyteidl.core.tasktemplate.container` (WIP, sql etc). + +The motivation of the Custom field, is to marshal a JSON structure that specifies information beyond what a regular TaskTemplate can capture. The actual structure of the JSON is known only to the implemented backend-plugin and the SDK components. The core Flyte platform, does not understand of look into the specifics of this structure. +It is highly recommended to use an interface definition lanugage like Protobuf, OpenAPISpec etc to declare specify the structure of the JSON. From here, on we refer to this as the ``Plugin Specification``. + +For Spark we decided to use Protobuf to specify the plugin as can be seen `here `_. Note it is not necessary to have the Plugin structure specified in FlyteIDL, we do it for simplicity, ease of maintenance alongwith the core platform and because of existing tooling to generate code for protobuf. + +Flytekit Plugin implementation +-------------------------------- +Now that you have a specification, we have to implement a method to generate this new TaskTemplate, with the special custom field. Also, this is where the UX design comes into play. You want to write the best possible interface in the SDK that users are delighted to use. The end goal is to create the TaskTemplate with the Custom field populated with the actual JSON structure. +We will currently refer to Flytekit - python as an example for extending and implementing the SDK. (For java refer to other docs). +The SDK task should be implemented as an extension of :py:class:`flytekit.extend.PythonTask`, or more commonly :py:class:`flytekit.PythonFunctionTask`. +In the case of Spark, we extend the :py:class:`flytekit.PythonFunctionTask`, as shown `here `_. + +The SparkTask is implemented as a regular flytekit plugin, with one exception, the ``Custom`` field is now actually the ``SparkJob`` protocol buffer. Flytekit base classes when serializing a task, will automatically invoke the `get_custom method `_. + + +FlytePropeller backend Plugin +------------------------------ +The backend plugin is where the actual logic of the execution is implemented. The backend plugin uses Flyte - PluginMachinery inteface to implement a plugin which can be one of the following supported types + +#. A `Kubernetes operator Plugin `_ +#. A Web API plugin - `Async `_ or `Sync `_. +#. Or if none of the above fits then - a `Core Plugin `_ + +Kubernetes operator Plugin +^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. todo: fill in + +Web API plugin +^^^^^^^^^^^^^^^ +.. todo: fill in + +Catch-all - Core Plugin +^^^^^^^^^^^^^^^^^^^^^^^^ +.. todo: fill in \ No newline at end of file diff --git a/rsts/plugins/extend/flytekit_python.rst b/rsts/plugins/extend/flytekit_python.rst index e061d4ccb1..2bd1f1de4f 100644 --- a/rsts/plugins/extend/flytekit_python.rst +++ b/rsts/plugins/extend/flytekit_python.rst @@ -2,4 +2,9 @@ ################################## Extend flytekit (python) -################################## \ No newline at end of file +################################## + +Extending Flytekit is desirable whether you are writing a backend plugins or a Flytekit only plugin. In this section we will cover the basics of how to extend Flytekit to add new plugins - task-types. +In the next section of :ref:`extend-plugin-flyte-backend`, we will talk about backend plugins, which includes extending flytekit - but specifically for the backend plugin. + +.. caution:: Work in progress. For a simple example of how to write a flytekit plugin refer to :std:ref:`advanced_custom_task_plugin`. But, remember flytekit can be extended beyond this simple method - so feel free to ask us a question in the slack channel. \ No newline at end of file diff --git a/rsts/plugins/extend/intro.rst b/rsts/plugins/extend/intro.rst index 7651676b80..18b7896393 100644 --- a/rsts/plugins/extend/intro.rst +++ b/rsts/plugins/extend/intro.rst @@ -4,6 +4,8 @@ When & How to Extend Flyte ########################### +.. caution:: These docs are still work in progress. Please read through and if you have any questions don't shy away from either filing a github issue or ping us in the Slack channel. The community loves plugins and would love to help you in any way. + The Core of Flyte is a container execution engine, where you can write one or more tasks and string them together to form a data dependency DAG - called a ``workflow``. If your work involves writing simple python or java tasks that can either perform operations on their own or can call out to external services - then there is **NO NEED to extend FLYTE**. From 2e28f1a7398f8e5aec96b7027a959d3e6b88318b Mon Sep 17 00:00:00 2001 From: Jeev B Date: Mon, 1 Mar 2021 17:27:55 -0800 Subject: [PATCH 11/74] Fix issues with kustomize templates (#783) * Fix issues with kustomize templates Signed-off-by: Jeev B * Use remote URL for contour-ingress-controller Signed-off-by: Jeev B * fix Signed-off-by: Jeev B * fix Signed-off-by: Jeev B * Remove contour from test overlay Signed-off-by: Jeev B Signed-off-by: Haytham Abuelfutuh --- deployment/eks/flyte_generated.yaml | 86 +- deployment/gcp/flyte_generated.yaml | 64 +- deployment/sandbox/flyte_generated.yaml | 890 +++++--- deployment/test/flyte_generated.yaml | 302 --- docker/sandbox/wait-for-flyte.sh | 2 +- end2end/execute.sh | 2 - .../addons}/alb_ingress/README.md | 0 .../addons}/alb_ingress/ingress.yaml | 0 .../addons}/cloudsqlproxy/deployment.yaml | 0 .../addons}/cloudsqlproxy/kustomization.yaml | 0 .../addons}/cloudsqlproxy/service.yaml | 0 .../kustomization.yaml | 4 + .../contour_ingress_controller/service.yaml | 12 +- .../addons}/database/database.yaml | 0 .../addons}/database/kustomization.yaml | 0 .../addons}/redis/deployment.yaml | 0 .../addons}/redis/kustomization.yaml | 0 .../addons}/redis/service.yaml | 0 .../addons}/storage/kustomization.yaml | 0 .../addons}/storage/storage.yaml | 0 .../kfoperators/namespace/kustomization.yaml | 2 + .../kfoperators/namespace/namespace.yaml | 4 + .../kfoperators/pytorch/kustomization.yaml | 3 +- .../kfoperators/tensorflow/kustomization.yaml | 1 + .../headless/kustomization.yaml | 1 + .../base/single_cluster/headless/pass.txt | 1 - .../eks/datacatalog/kustomization.yaml | 5 - .../eks/{ => flyte}/admin/deployment.yaml | 0 .../eks/{ => flyte}/admin/service.yaml | 0 .../eks/{ => flyte}/admin/serviceaccount.yaml | 0 .../eks/{ => flyte}/config/admin/db.yaml | 0 .../{ => flyte}/config/admin/remote_data.yaml | 0 .../config/admin/task_resource_defaults.yaml | 0 .../ac_project-copilot-dataconfig.yaml | 0 .../ad_spark-role.yaml | 0 .../ae_spark-service-account.yaml | 0 .../af_spark-role-binding.yaml | 0 .../{ => flyte}/config/common/storage.yaml | 0 .../{ => flyte}/config/datacatalog/db.yaml | 0 .../{ => flyte}/config/propeller/core.yaml | 0 .../config/propeller/enabled_plugins.yaml | 0 .../propeller/plugins/catalog_cache.yaml | 0 .../config/propeller/plugins/k8s.yaml | 0 .../config/propeller/plugins/qubole.yaml | 0 .../config/propeller/plugins/spark.yaml | 0 .../config/propeller/plugins/task_logs.yaml | 0 .../eks/{ => flyte}/console/service.yaml | 0 .../{ => flyte}/datacatalog/deployment.yaml | 0 .../eks/{ => flyte}/datacatalog/service.yaml | 0 .../eks/{ => flyte}/ingress/ingress.yaml | 0 .../overlays/eks/flyte/kustomization.yaml | 64 + .../eks/{ => flyte}/propeller/deployment.yaml | 0 .../eks/{ => flyte}/propeller/service.yaml | 0 .../{ => flyte}/propeller/serviceaccount.yaml | 0 kustomize/overlays/eks/kustomization.yaml | 115 +- .../gcp/datacatalog/kustomization.yaml | 5 - .../gcp/dependencies/redis/deployment.yaml | 59 - .../gcp/dependencies/redis/kustomization.yaml | 3 - .../gcp/dependencies/redis/service.yaml | 16 - .../gcp/{ => flyte}/admin/deployment.yaml | 0 .../gcp/{ => flyte}/admin/service.yaml | 0 .../gcp/{ => flyte}/config/admin/db.yaml | 0 .../{ => flyte}/config/admin/remote_data.yaml | 0 .../config/admin/task_resource_defaults.yaml | 0 .../ac_project-copilot-dataconfig.yaml | 0 .../ad_spark-role.yaml | 0 .../ae_spark-service-account.yaml | 0 .../af_spark-role-binding.yaml | 0 .../{ => flyte}/config/common/storage.yaml | 0 .../{ => flyte}/config/datacatalog/db.yaml | 0 .../{ => flyte}/config/propeller/core.yaml | 0 .../config/propeller/enabled_plugins.yaml | 0 .../propeller/plugins/catalog_cache.yaml | 0 .../config/propeller/plugins/k8s.yaml | 0 .../config/propeller/plugins/qubole.yaml | 0 .../config/propeller/plugins/spark.yaml | 0 .../config/propeller/plugins/task_logs.yaml | 0 .../gcp/{ => flyte}/console/service.yaml | 0 .../{ => flyte}/datacatalog/deployment.yaml | 0 .../gcp/{ => flyte}/datacatalog/service.yaml | 0 .../overlays/gcp/flyte/kustomization.yaml | 61 + .../gcp/{ => flyte}/propeller/deployment.yaml | 0 .../gcp/{ => flyte}/propeller/service.yaml | 0 kustomize/overlays/gcp/kustomization.yaml | 112 +- .../contour_ingress_controller/contour.yaml | 1880 ----------------- .../kustomization.yaml | 5 - .../dependencies/redis/deployment.yaml | 59 - .../dependencies/redis/kustomization.yaml | 3 - .../sandbox/dependencies/redis/service.yaml | 16 - .../sandbox/{ => flyte}/admin/deployment.yaml | 0 .../sandbox/{ => flyte}/config/admin/db.yaml | 0 .../config/admin/task_resource_defaults.yaml | 0 .../ac_project-copilot-dataconfig.yaml | 0 .../ad_spark-role.yaml | 0 .../ae_spark-service-account.yaml | 0 .../af_spark-role-binding.yaml | 0 .../{ => flyte}/config/common/logger.yaml | 0 .../{ => flyte}/config/common/storage.yaml | 0 .../{ => flyte}/config/datacatalog/db.yaml | 0 .../config/propeller/enabled_plugins.yaml | 0 .../config/propeller/plugins/k8s.yaml | 0 .../config/propeller/plugins/task_logs.yaml | 0 .../config/propeller/resource_manager.yaml | 0 .../overlays/sandbox/flyte/kustomization.yaml | 42 + kustomize/overlays/sandbox/kustomization.yaml | 85 +- .../{dependencies => }/nodeport-services.yaml | 0 .../custom-config.yaml | 134 -- .../deployment.yaml | 66 - .../kustomization.yaml | 5 - .../contour_ingress_controller/rbac.yaml | 66 - .../contour_ingress_controller/service.yaml | 17 - .../test/dependencies/database/database.yaml | 41 - .../dependencies/database/kustomization.yaml | 2 - .../dependencies/storage/kustomization.yaml | 2 - .../test/dependencies/storage/storage.yaml | 63 - .../test/{ => flyte}/admin/deployment.yaml | 0 .../test/{ => flyte}/config/admin/db.yaml | 0 .../ac_project-copilot-dataconfig.yaml | 0 .../{ => flyte}/config/common/logger.yaml | 0 .../{ => flyte}/config/common/storage.yaml | 0 .../{ => flyte}/config/datacatalog/db.yaml | 0 .../config/propeller/enabled_plugins.yaml | 0 .../config/propeller/plugins/k8s.yaml | 0 .../config/propeller/resource_manager.yaml | 0 .../overlays/test/flyte/kustomization.yaml | 49 + kustomize/overlays/test/kustomization.yaml | 103 +- 126 files changed, 1055 insertions(+), 3397 deletions(-) rename kustomize/{overlays/eks/dependencies => base/addons}/alb_ingress/README.md (100%) rename kustomize/{overlays/eks/dependencies => base/addons}/alb_ingress/ingress.yaml (100%) rename kustomize/{overlays/gcp/dependencies => base/addons}/cloudsqlproxy/deployment.yaml (100%) rename kustomize/{overlays/gcp/dependencies => base/addons}/cloudsqlproxy/kustomization.yaml (100%) rename kustomize/{overlays/gcp/dependencies => base/addons}/cloudsqlproxy/service.yaml (100%) create mode 100644 kustomize/base/addons/contour_ingress_controller/kustomization.yaml rename kustomize/{overlays/sandbox/dependencies => base/addons}/contour_ingress_controller/service.yaml (60%) rename kustomize/{overlays/sandbox/dependencies => base/addons}/database/database.yaml (100%) rename kustomize/{overlays/sandbox/dependencies => base/addons}/database/kustomization.yaml (100%) rename kustomize/{overlays/eks/dependencies => base/addons}/redis/deployment.yaml (100%) rename kustomize/{overlays/eks/dependencies => base/addons}/redis/kustomization.yaml (100%) rename kustomize/{overlays/eks/dependencies => base/addons}/redis/service.yaml (100%) rename kustomize/{overlays/sandbox/dependencies => base/addons}/storage/kustomization.yaml (100%) rename kustomize/{overlays/sandbox/dependencies => base/addons}/storage/storage.yaml (100%) create mode 100644 kustomize/base/operators/kfoperators/namespace/kustomization.yaml create mode 100644 kustomize/base/operators/kfoperators/namespace/namespace.yaml delete mode 100644 kustomize/base/single_cluster/headless/pass.txt delete mode 100644 kustomize/overlays/eks/datacatalog/kustomization.yaml rename kustomize/overlays/eks/{ => flyte}/admin/deployment.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/admin/service.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/admin/serviceaccount.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/admin/db.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/admin/remote_data.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/admin/task_resource_defaults.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/clusterresource-templates/ad_spark-role.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/clusterresource-templates/ae_spark-service-account.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/clusterresource-templates/af_spark-role-binding.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/common/storage.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/datacatalog/db.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/propeller/core.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/propeller/enabled_plugins.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/propeller/plugins/catalog_cache.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/propeller/plugins/k8s.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/propeller/plugins/qubole.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/propeller/plugins/spark.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/config/propeller/plugins/task_logs.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/console/service.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/datacatalog/deployment.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/datacatalog/service.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/ingress/ingress.yaml (100%) create mode 100644 kustomize/overlays/eks/flyte/kustomization.yaml rename kustomize/overlays/eks/{ => flyte}/propeller/deployment.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/propeller/service.yaml (100%) rename kustomize/overlays/eks/{ => flyte}/propeller/serviceaccount.yaml (100%) delete mode 100644 kustomize/overlays/gcp/datacatalog/kustomization.yaml delete mode 100644 kustomize/overlays/gcp/dependencies/redis/deployment.yaml delete mode 100644 kustomize/overlays/gcp/dependencies/redis/kustomization.yaml delete mode 100644 kustomize/overlays/gcp/dependencies/redis/service.yaml rename kustomize/overlays/gcp/{ => flyte}/admin/deployment.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/admin/service.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/admin/db.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/admin/remote_data.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/admin/task_resource_defaults.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/clusterresource-templates/ad_spark-role.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/clusterresource-templates/ae_spark-service-account.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/clusterresource-templates/af_spark-role-binding.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/common/storage.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/datacatalog/db.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/propeller/core.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/propeller/enabled_plugins.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/propeller/plugins/catalog_cache.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/propeller/plugins/k8s.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/propeller/plugins/qubole.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/propeller/plugins/spark.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/config/propeller/plugins/task_logs.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/console/service.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/datacatalog/deployment.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/datacatalog/service.yaml (100%) create mode 100644 kustomize/overlays/gcp/flyte/kustomization.yaml rename kustomize/overlays/gcp/{ => flyte}/propeller/deployment.yaml (100%) rename kustomize/overlays/gcp/{ => flyte}/propeller/service.yaml (100%) delete mode 100644 kustomize/overlays/sandbox/dependencies/contour_ingress_controller/contour.yaml delete mode 100644 kustomize/overlays/sandbox/dependencies/contour_ingress_controller/kustomization.yaml delete mode 100644 kustomize/overlays/sandbox/dependencies/redis/deployment.yaml delete mode 100644 kustomize/overlays/sandbox/dependencies/redis/kustomization.yaml delete mode 100644 kustomize/overlays/sandbox/dependencies/redis/service.yaml rename kustomize/overlays/sandbox/{ => flyte}/admin/deployment.yaml (100%) rename kustomize/overlays/sandbox/{ => flyte}/config/admin/db.yaml (100%) rename kustomize/overlays/sandbox/{ => flyte}/config/admin/task_resource_defaults.yaml (100%) rename kustomize/overlays/sandbox/{ => flyte}/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml (100%) rename kustomize/overlays/sandbox/{ => flyte}/config/clusterresource-templates/ad_spark-role.yaml (100%) rename kustomize/overlays/sandbox/{ => flyte}/config/clusterresource-templates/ae_spark-service-account.yaml (100%) rename kustomize/overlays/sandbox/{ => flyte}/config/clusterresource-templates/af_spark-role-binding.yaml (100%) rename kustomize/overlays/sandbox/{ => flyte}/config/common/logger.yaml (100%) rename kustomize/overlays/sandbox/{ => flyte}/config/common/storage.yaml (100%) rename kustomize/overlays/sandbox/{ => flyte}/config/datacatalog/db.yaml (100%) rename kustomize/overlays/sandbox/{ => flyte}/config/propeller/enabled_plugins.yaml (100%) rename kustomize/overlays/sandbox/{ => flyte}/config/propeller/plugins/k8s.yaml (100%) rename kustomize/overlays/sandbox/{ => flyte}/config/propeller/plugins/task_logs.yaml (100%) rename kustomize/overlays/sandbox/{ => flyte}/config/propeller/resource_manager.yaml (100%) create mode 100644 kustomize/overlays/sandbox/flyte/kustomization.yaml rename kustomize/overlays/sandbox/{dependencies => }/nodeport-services.yaml (100%) delete mode 100644 kustomize/overlays/test/dependencies/contour_ingress_controller/custom-config.yaml delete mode 100644 kustomize/overlays/test/dependencies/contour_ingress_controller/deployment.yaml delete mode 100644 kustomize/overlays/test/dependencies/contour_ingress_controller/kustomization.yaml delete mode 100644 kustomize/overlays/test/dependencies/contour_ingress_controller/rbac.yaml delete mode 100644 kustomize/overlays/test/dependencies/contour_ingress_controller/service.yaml delete mode 100644 kustomize/overlays/test/dependencies/database/database.yaml delete mode 100644 kustomize/overlays/test/dependencies/database/kustomization.yaml delete mode 100644 kustomize/overlays/test/dependencies/storage/kustomization.yaml delete mode 100644 kustomize/overlays/test/dependencies/storage/storage.yaml rename kustomize/overlays/test/{ => flyte}/admin/deployment.yaml (100%) rename kustomize/overlays/test/{ => flyte}/config/admin/db.yaml (100%) rename kustomize/overlays/test/{ => flyte}/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml (100%) rename kustomize/overlays/test/{ => flyte}/config/common/logger.yaml (100%) rename kustomize/overlays/test/{ => flyte}/config/common/storage.yaml (100%) rename kustomize/overlays/test/{ => flyte}/config/datacatalog/db.yaml (100%) rename kustomize/overlays/test/{ => flyte}/config/propeller/enabled_plugins.yaml (100%) rename kustomize/overlays/test/{ => flyte}/config/propeller/plugins/k8s.yaml (100%) rename kustomize/overlays/test/{ => flyte}/config/propeller/resource_manager.yaml (100%) create mode 100644 kustomize/overlays/test/flyte/kustomization.yaml diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index f0a55da47b..e47d98fb0e 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -3,6 +3,16 @@ kind: Namespace metadata: name: flyte --- +apiVersion: v1 +kind: Namespace +metadata: + name: kubeflow +--- +apiVersion: v1 +kind: Namespace +metadata: + name: sparkoperator +--- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: @@ -7621,13 +7631,13 @@ metadata: app: pytorch-operator kustomize.component: pytorch-operator name: pytorch-operator - namespace: flyte + namespace: kubeflow --- apiVersion: v1 kind: ServiceAccount metadata: name: sparkoperator - namespace: flyte + namespace: sparkoperator --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole @@ -7924,7 +7934,7 @@ roleRef: subjects: - kind: ServiceAccount name: pytorch-operator - namespace: flyte + namespace: kubeflow --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding @@ -7937,7 +7947,7 @@ roleRef: subjects: - kind: ServiceAccount name: sparkoperator - namespace: flyte + namespace: sparkoperator --- apiVersion: v1 data: @@ -8365,6 +8375,22 @@ spec: --- apiVersion: v1 kind: Service +metadata: + annotations: + cloud.google.com/load-balancer-type: Internal + name: datacatalog-metrics + namespace: flyte +spec: + ports: + - name: http-metrics + port: 10254 + protocol: TCP + selector: + app: datacatalog + type: LoadBalancer +--- +apiVersion: v1 +kind: Service metadata: annotations: external-dns.alpha.kubernetes.io/hostname: flyteadmin.subdomain.mydomain.com @@ -8414,23 +8440,17 @@ apiVersion: v1 kind: Service metadata: annotations: - prometheus.io/path: /metrics - prometheus.io/port: "8443" - prometheus.io/scrape: "true" - labels: - app: pytorch-operator - kustomize.component: pytorch-operator - name: pytorch-operator + cloud.google.com/load-balancer-type: Internal + name: flytepropeller namespace: flyte spec: ports: - - name: monitoring-port - port: 8443 - targetPort: 8443 + - name: http-metrics + port: 10254 + protocol: TCP selector: - kustomize.component: pytorch-operator - name: pytorch-operator - type: ClusterIP + app: flytepropeller + type: LoadBalancer --- apiVersion: v1 kind: Service @@ -8451,9 +8471,31 @@ spec: --- apiVersion: v1 kind: Service +metadata: + annotations: + prometheus.io/path: /metrics + prometheus.io/port: "8443" + prometheus.io/scrape: "true" + labels: + app: pytorch-operator + kustomize.component: pytorch-operator + name: pytorch-operator + namespace: kubeflow +spec: + ports: + - name: monitoring-port + port: 8443 + targetPort: 8443 + selector: + kustomize.component: pytorch-operator + name: pytorch-operator + type: ClusterIP +--- +apiVersion: v1 +kind: Service metadata: name: spark-webhook - namespace: flyte + namespace: sparkoperator spec: ports: - name: webhook @@ -8760,7 +8802,7 @@ metadata: labels: kustomize.component: pytorch-operator name: pytorch-operator - namespace: flyte + namespace: kubeflow spec: replicas: 1 selector: @@ -8799,7 +8841,7 @@ metadata: app.kubernetes.io/name: sparkoperator app.kubernetes.io/version: v2.4.5-v1beta2 name: sparkoperator - namespace: flyte + namespace: sparkoperator spec: replicas: 1 selector: @@ -8954,7 +8996,7 @@ metadata: app.kubernetes.io/name: sparkoperator app.kubernetes.io/version: v2.4.5-v1beta2 name: sparkoperator-init - namespace: flyte + namespace: sparkoperator spec: backoffLimit: 3 template: @@ -8967,7 +9009,7 @@ spec: - command: - /usr/bin/gencerts.sh - --namespace - - flyte + - sparkoperator - -p image: gcr.io/spark-operator/spark-operator:v1beta2-1.1.2-2.4.5 imagePullPolicy: IfNotPresent diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index 989d75e842..0d9b307eff 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -3,6 +3,16 @@ kind: Namespace metadata: name: flyte --- +apiVersion: v1 +kind: Namespace +metadata: + name: kubeflow +--- +apiVersion: v1 +kind: Namespace +metadata: + name: sparkoperator +--- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: @@ -7617,13 +7627,13 @@ metadata: app: pytorch-operator kustomize.component: pytorch-operator name: pytorch-operator - namespace: flyte + namespace: kubeflow --- apiVersion: v1 kind: ServiceAccount metadata: name: sparkoperator - namespace: flyte + namespace: sparkoperator --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole @@ -7920,7 +7930,7 @@ roleRef: subjects: - kind: ServiceAccount name: pytorch-operator - namespace: flyte + namespace: kubeflow --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding @@ -7933,7 +7943,7 @@ roleRef: subjects: - kind: ServiceAccount name: sparkoperator - namespace: flyte + namespace: sparkoperator --- apiVersion: v1 data: @@ -8449,6 +8459,23 @@ spec: --- apiVersion: v1 kind: Service +metadata: + labels: + app: redis-resource-manager + name: redis-resource-manager + namespace: flyte +spec: + ports: + - name: redis + port: 6379 + protocol: TCP + targetPort: redis + selector: + app: redis-resource-manager + type: ClusterIP +--- +apiVersion: v1 +kind: Service metadata: annotations: prometheus.io/path: /metrics @@ -8458,7 +8485,7 @@ metadata: app: pytorch-operator kustomize.component: pytorch-operator name: pytorch-operator - namespace: flyte + namespace: kubeflow spec: ports: - name: monitoring-port @@ -8471,26 +8498,9 @@ spec: --- apiVersion: v1 kind: Service -metadata: - labels: - app: redis-resource-manager - name: redis-resource-manager - namespace: flyte -spec: - ports: - - name: redis - port: 6379 - protocol: TCP - targetPort: redis - selector: - app: redis-resource-manager - type: ClusterIP ---- -apiVersion: v1 -kind: Service metadata: name: spark-webhook - namespace: flyte + namespace: sparkoperator spec: ports: - name: webhook @@ -8824,7 +8834,7 @@ metadata: labels: kustomize.component: pytorch-operator name: pytorch-operator - namespace: flyte + namespace: kubeflow spec: replicas: 1 selector: @@ -8863,7 +8873,7 @@ metadata: app.kubernetes.io/name: sparkoperator app.kubernetes.io/version: v2.4.5-v1beta2 name: sparkoperator - namespace: flyte + namespace: sparkoperator spec: replicas: 1 selector: @@ -9018,7 +9028,7 @@ metadata: app.kubernetes.io/name: sparkoperator app.kubernetes.io/version: v2.4.5-v1beta2 name: sparkoperator-init - namespace: flyte + namespace: sparkoperator spec: backoffLimit: 3 template: @@ -9031,7 +9041,7 @@ spec: - command: - /usr/bin/gencerts.sh - --namespace - - flyte + - sparkoperator - -p image: gcr.io/spark-operator/spark-operator:v1beta2-1.1.2-2.4.5 imagePullPolicy: IfNotPresent diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index d119bfd043..3f5300cebd 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -3,11 +3,16 @@ kind: Namespace metadata: name: flyte --- +apiVersion: v1 +kind: Namespace +metadata: + name: projectcontour +--- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.2.9 + controller-gen.kubebuilder.io/version: v0.4.1 creationTimestamp: null name: extensionservices.projectcontour.io spec: @@ -40,10 +45,28 @@ spec: description: ExtensionServiceSpec defines the desired state of an ExtensionService resource. properties: loadBalancerPolicy: - description: The policy for load balancing GRPC service requests. Note that the `Cookie` load balancing strategy cannot be used here. + description: The policy for load balancing GRPC service requests. Note that the `Cookie` and `RequestHash` load balancing strategies cannot be used here. properties: + requestHashPolicies: + description: RequestHashPolicies contains a list of hash policies to apply when the `RequestHash` load balancing strategy is chosen. If an element of the supplied list of hash policies is invalid, it will be ignored. If the list of hash policies is empty after validation, the load balancing strategy will fall back the the default `RoundRobin`. + items: + description: RequestHashPolicy contains configuration for an individual hash policy on a request attribute. + properties: + headerHashOptions: + description: HeaderHashOptions should be set when request header hash based load balancing is desired. It must be the only hash option field set, otherwise this request hash policy object will be ignored. + properties: + headerName: + description: HeaderName is the name of the HTTP request header that will be used to calculate the hash key. If the header specified is not present on a request, no hash will be produced. + minLength: 1 + type: string + type: object + terminal: + description: Terminal is a flag that allows for short-circuiting computing of a hash for a given request. If set to true, and the request attribute specified in the attribute hash options is present, no further hash policies will be used to calculate a hash for the request. + type: boolean + type: object + type: array strategy: - description: Strategy specifies the policy used to balance requests across the pool of backend pods. Valid policy names are `Random`, `RoundRobin`, `WeightedLeastRequest`, `Random` and `Cookie`. If an unknown strategy name is specified or no policy is supplied, the default `RoundRobin` policy is used. + description: Strategy specifies the policy used to balance requests across the pool of backend pods. Valid policy names are `Random`, `RoundRobin`, `WeightedLeastRequest`, `Cookie`, and `RequestHash`. If an unknown strategy name is specified or no policy is supplied, the default `RoundRobin` policy is used. type: string type: object protocol: @@ -244,7 +267,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.2.9 + controller-gen.kubebuilder.io/version: v0.4.1 creationTimestamp: null name: httpproxies.projectcontour.io spec: @@ -429,8 +452,26 @@ spec: loadBalancerPolicy: description: The load balancing policy for this route. properties: + requestHashPolicies: + description: RequestHashPolicies contains a list of hash policies to apply when the `RequestHash` load balancing strategy is chosen. If an element of the supplied list of hash policies is invalid, it will be ignored. If the list of hash policies is empty after validation, the load balancing strategy will fall back the the default `RoundRobin`. + items: + description: RequestHashPolicy contains configuration for an individual hash policy on a request attribute. + properties: + headerHashOptions: + description: HeaderHashOptions should be set when request header hash based load balancing is desired. It must be the only hash option field set, otherwise this request hash policy object will be ignored. + properties: + headerName: + description: HeaderName is the name of the HTTP request header that will be used to calculate the hash key. If the header specified is not present on a request, no hash will be produced. + minLength: 1 + type: string + type: object + terminal: + description: Terminal is a flag that allows for short-circuiting computing of a hash for a given request. If set to true, and the request attribute specified in the attribute hash options is present, no further hash policies will be used to calculate a hash for the request. + type: boolean + type: object + type: array strategy: - description: Strategy specifies the policy used to balance requests across the pool of backend pods. Valid policy names are `Random`, `RoundRobin`, `WeightedLeastRequest`, `Random` and `Cookie`. If an unknown strategy name is specified or no policy is supplied, the default `RoundRobin` policy is used. + description: Strategy specifies the policy used to balance requests across the pool of backend pods. Valid policy names are `Random`, `RoundRobin`, `WeightedLeastRequest`, `Cookie`, and `RequestHash`. If an unknown strategy name is specified or no policy is supplied, the default `RoundRobin` policy is used. type: string type: object pathRewritePolicy: @@ -457,6 +498,103 @@ spec: permitInsecure: description: Allow this path to respond to insecure requests over HTTP which are normally not permitted when a `virtualhost.tls` block is present. type: boolean + rateLimitPolicy: + description: The policy for rate limiting on the route. + properties: + global: + description: Global defines global rate limiting parameters, i.e. parameters defining descriptors that are sent to an external rate limit service (RLS) for a rate limit decision on each request. + properties: + descriptors: + description: Descriptors defines the list of descriptors that will be generated and sent to the rate limit service. Each descriptor contains 1+ key-value pair entries. + items: + description: RateLimitDescriptor defines a list of key-value pair generators. + properties: + entries: + description: Entries is the list of key-value pair generators. + items: + description: RateLimitDescriptorEntry is a key-value pair generator. Exactly one field on this struct must be non-nil. + properties: + genericKey: + description: GenericKey defines a descriptor entry with a static key and value. + properties: + key: + description: Key defines the key of the descriptor entry. If not set, the key is set to "generic_key". + type: string + value: + description: Value defines the value of the descriptor entry. + minLength: 1 + type: string + type: object + remoteAddress: + description: RemoteAddress defines a descriptor entry with a key of "remote_address" and a value equal to the client's IP address (from x-forwarded-for). + type: object + requestHeader: + description: RequestHeader defines a descriptor entry that's populated only if a given header is present on the request. The descriptor key is static, and the descriptor value is equal to the value of the header. + properties: + descriptorKey: + description: DescriptorKey defines the key to use on the descriptor entry. + minLength: 1 + type: string + headerName: + description: HeaderName defines the name of the header to look for on the request. + minLength: 1 + type: string + type: object + type: object + minItems: 1 + type: array + type: object + minItems: 1 + type: array + type: object + local: + description: Local defines local rate limiting parameters, i.e. parameters for rate limiting that occurs within each Envoy pod as requests are handled. + properties: + burst: + description: Burst defines the number of requests above the requests per unit that should be allowed within a short period of time. + format: int32 + type: integer + requests: + description: Requests defines how many requests per unit of time should be allowed before rate limiting occurs. + format: int32 + minimum: 1 + type: integer + responseHeadersToAdd: + description: ResponseHeadersToAdd is an optional list of response headers to set when a request is rate-limited. + items: + description: HeaderValue represents a header name/value pair + properties: + name: + description: Name represents a key of a header + minLength: 1 + type: string + value: + description: Value represents the value of a header specified by a key + minLength: 1 + type: string + required: + - name + - value + type: object + type: array + responseStatusCode: + description: ResponseStatusCode is the HTTP status code to use for responses to rate-limited requests. Codes must be in the 400-599 range (inclusive). If not specified, the Envoy default of 429 (Too Many Requests) is used. + format: int32 + maximum: 599 + minimum: 400 + type: integer + unit: + description: Unit defines the period of time within which requests over the limit will be rate limited. Valid values are "second", "minute" and "hour". + enum: + - second + - minute + - hour + type: string + required: + - requests + - unit + type: object + type: object requestHeadersPolicy: description: The policy for managing request headers during proxying. properties: @@ -716,10 +854,28 @@ spec: - name type: object loadBalancerPolicy: - description: The load balancing policy for the backend services. + description: The load balancing policy for the backend services. Note that the `Cookie` and `RequestHash` load balancing strategies cannot be used here. properties: + requestHashPolicies: + description: RequestHashPolicies contains a list of hash policies to apply when the `RequestHash` load balancing strategy is chosen. If an element of the supplied list of hash policies is invalid, it will be ignored. If the list of hash policies is empty after validation, the load balancing strategy will fall back the the default `RoundRobin`. + items: + description: RequestHashPolicy contains configuration for an individual hash policy on a request attribute. + properties: + headerHashOptions: + description: HeaderHashOptions should be set when request header hash based load balancing is desired. It must be the only hash option field set, otherwise this request hash policy object will be ignored. + properties: + headerName: + description: HeaderName is the name of the HTTP request header that will be used to calculate the hash key. If the header specified is not present on a request, no hash will be produced. + minLength: 1 + type: string + type: object + terminal: + description: Terminal is a flag that allows for short-circuiting computing of a hash for a given request. If set to true, and the request attribute specified in the attribute hash options is present, no further hash policies will be used to calculate a hash for the request. + type: boolean + type: object + type: array strategy: - description: Strategy specifies the policy used to balance requests across the pool of backend pods. Valid policy names are `Random`, `RoundRobin`, `WeightedLeastRequest`, `Random` and `Cookie`. If an unknown strategy name is specified or no policy is supplied, the default `RoundRobin` policy is used. + description: Strategy specifies the policy used to balance requests across the pool of backend pods. Valid policy names are `Random`, `RoundRobin`, `WeightedLeastRequest`, `Cookie`, and `RequestHash`. If an unknown strategy name is specified or no policy is supplied, the default `RoundRobin` policy is used. type: string type: object services: @@ -910,6 +1066,103 @@ spec: fqdn: description: The fully qualified domain name of the root of the ingress tree all leaves of the DAG rooted at this object relate to the fqdn. type: string + rateLimitPolicy: + description: The policy for rate limiting on the virtual host. + properties: + global: + description: Global defines global rate limiting parameters, i.e. parameters defining descriptors that are sent to an external rate limit service (RLS) for a rate limit decision on each request. + properties: + descriptors: + description: Descriptors defines the list of descriptors that will be generated and sent to the rate limit service. Each descriptor contains 1+ key-value pair entries. + items: + description: RateLimitDescriptor defines a list of key-value pair generators. + properties: + entries: + description: Entries is the list of key-value pair generators. + items: + description: RateLimitDescriptorEntry is a key-value pair generator. Exactly one field on this struct must be non-nil. + properties: + genericKey: + description: GenericKey defines a descriptor entry with a static key and value. + properties: + key: + description: Key defines the key of the descriptor entry. If not set, the key is set to "generic_key". + type: string + value: + description: Value defines the value of the descriptor entry. + minLength: 1 + type: string + type: object + remoteAddress: + description: RemoteAddress defines a descriptor entry with a key of "remote_address" and a value equal to the client's IP address (from x-forwarded-for). + type: object + requestHeader: + description: RequestHeader defines a descriptor entry that's populated only if a given header is present on the request. The descriptor key is static, and the descriptor value is equal to the value of the header. + properties: + descriptorKey: + description: DescriptorKey defines the key to use on the descriptor entry. + minLength: 1 + type: string + headerName: + description: HeaderName defines the name of the header to look for on the request. + minLength: 1 + type: string + type: object + type: object + minItems: 1 + type: array + type: object + minItems: 1 + type: array + type: object + local: + description: Local defines local rate limiting parameters, i.e. parameters for rate limiting that occurs within each Envoy pod as requests are handled. + properties: + burst: + description: Burst defines the number of requests above the requests per unit that should be allowed within a short period of time. + format: int32 + type: integer + requests: + description: Requests defines how many requests per unit of time should be allowed before rate limiting occurs. + format: int32 + minimum: 1 + type: integer + responseHeadersToAdd: + description: ResponseHeadersToAdd is an optional list of response headers to set when a request is rate-limited. + items: + description: HeaderValue represents a header name/value pair + properties: + name: + description: Name represents a key of a header + minLength: 1 + type: string + value: + description: Value represents the value of a header specified by a key + minLength: 1 + type: string + required: + - name + - value + type: object + type: array + responseStatusCode: + description: ResponseStatusCode is the HTTP status code to use for responses to rate-limited requests. Codes must be in the 400-599 range (inclusive). If not specified, the Envoy default of 429 (Too Many Requests) is used. + format: int32 + maximum: 599 + minimum: 400 + type: integer + unit: + description: Unit defines the period of time within which requests over the limit will be rate limited. Valid values are "second", "minute" and "hour". + enum: + - second + - minute + - hour + type: string + required: + - requests + - unit + type: object + type: object tls: description: If present the fields describes TLS properties of the virtual host. The SNI names that will be matched on are described in fqdn, the tls.secretName secret must contain a certificate that itself contains a name that matches the FQDN. properties: @@ -927,7 +1180,7 @@ spec: description: EnableFallbackCertificate defines if the vhost should allow a default certificate to be applied which handles all requests which don't match the SNI defined in this vhost. type: boolean minimumProtocolVersion: - description: Minimum TLS version this vhost should negotiate + description: MinimumProtocolVersion is the minimum TLS version this vhost should negotiate. Valid options are `1.2` (default) and `1.3`. Any other value defaults to TLS 1.2. type: string passthrough: description: Passthrough defines whether the encrypted TLS handshake will be passed through to the backing cluster. Either Passthrough or SecretName must be specified, but not both. @@ -1076,6 +1329,29 @@ spec: ip: description: IP is set for load-balancer ingress points that are IP based (typically GCE or OpenStack load-balancers) type: string + ports: + description: Ports is a list of records of service ports If used, every port defined in the service should have an entry in it + items: + properties: + error: + description: 'Error is to record the problem with the service port The format of the error shall comply with the following rules: - built-in error values shall be specified in this file and those shall use CamelCase names - cloud provider specific error values must have names that comply with the format foo.example.com/CamelCase. --- The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)' + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + port: + description: Port is the port number of the service port of which status is recorded here + format: int32 + type: integer + protocol: + default: TCP + description: 'Protocol is the protocol of the service port of which status is recorded here The supported values are: "TCP", "UDP", "SCTP"' + type: string + required: + - port + - protocol + type: object + type: array + x-kubernetes-list-type: atomic type: object type: array type: object @@ -1099,7 +1375,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.2.9 + controller-gen.kubebuilder.io/version: v0.4.1 creationTimestamp: null name: tlscertificatedelegations.projectcontour.io spec: @@ -1302,44 +1578,44 @@ spec: apiVersion: v1 kind: ServiceAccount metadata: - name: contour + name: datacatalog namespace: flyte --- apiVersion: v1 kind: ServiceAccount metadata: - name: contour-certgen + name: flyteadmin namespace: flyte --- apiVersion: v1 kind: ServiceAccount metadata: - name: datacatalog + name: flytepropeller namespace: flyte --- apiVersion: v1 kind: ServiceAccount metadata: - name: envoy - namespace: flyte + name: contour + namespace: projectcontour --- apiVersion: v1 kind: ServiceAccount metadata: - name: flyteadmin - namespace: flyte + name: contour-certgen + namespace: projectcontour --- apiVersion: v1 kind: ServiceAccount metadata: - name: flytepropeller - namespace: flyte + name: envoy + namespace: projectcontour --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: contour-certgen - namespace: flyte + namespace: projectcontour rules: - apiGroups: - "" @@ -1421,10 +1697,7 @@ rules: - apiGroups: - networking.k8s.io resources: - - gatewayclasses - - gateways - - httproutes - - tcproutes + - ingressclasses verbs: - get - list @@ -1445,6 +1718,17 @@ rules: - create - get - update +- apiGroups: + - networking.x-k8s.io + resources: + - backendpolicies + - gateways + - httproutes + - tlsroutes + verbs: + - get + - list + - watch - apiGroups: - projectcontour.io resources: @@ -1543,7 +1827,7 @@ apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: contour - namespace: flyte + namespace: projectcontour roleRef: apiGroup: rbac.authorization.k8s.io kind: Role @@ -1551,7 +1835,7 @@ roleRef: subjects: - kind: ServiceAccount name: contour-certgen - namespace: flyte + namespace: projectcontour --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding @@ -1564,7 +1848,7 @@ roleRef: subjects: - kind: ServiceAccount name: contour - namespace: flyte + namespace: projectcontour --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding @@ -1630,92 +1914,6 @@ metadata: namespace: flyte --- apiVersion: v1 -data: - contour.yaml: | - # - # server: - # determine which XDS Server implementation to utilize in Contour. - # xds-server-type: contour - # - # should contour expect to be running inside a k8s cluster - # incluster: true - # - # path to kubeconfig (if not running inside a k8s cluster) - # kubeconfig: /path/to/.kube/config - # - # disable HTTPProxy permitInsecure field - disablePermitInsecure: false - tls: - # minimum TLS version that Contour will negotiate - # minimum-protocol-version: "1.1" - # Defines the Kubernetes name/namespace matching a secret to use - # as the fallback certificate when requests which don't match the - # SNI defined for a vhost. - fallback-certificate: - # name: fallback-secret-name - # namespace: projectcontour - envoy-client-certificate: - # name: envoy-client-cert-secret-name - # namespace: projectcontour - # The following config shows the defaults for the leader election. - # leaderelection: - # configmap-name: leader-elect - # configmap-namespace: projectcontour - ### Logging options - # Default setting - accesslog-format: envoy - # To enable JSON logging in Envoy - # accesslog-format: json - # The default fields that will be logged are specified below. - # To customise this list, just add or remove entries. - # The canonical list is available at - # https://godoc.org/github.com/projectcontour/contour/internal/envoy#JSONFields - # json-fields: - # - "@timestamp" - # - "authority" - # - "bytes_received" - # - "bytes_sent" - # - "downstream_local_address" - # - "downstream_remote_address" - # - "duration" - # - "method" - # - "path" - # - "protocol" - # - "request_id" - # - "requested_server_name" - # - "response_code" - # - "response_flags" - # - "uber_trace_id" - # - "upstream_cluster" - # - "upstream_host" - # - "upstream_local_address" - # - "upstream_service_time" - # - "user_agent" - # - "x_forwarded_for" - # - # default-http-versions: - # - "HTTP/2" - # - "HTTP/1.1" - # - # The following shows the default proxy timeout settings. - # timeouts: - # request-timeout: infinity - # connection-idle-timeout: 60s - # stream-idle-timeout: 5m - # max-connection-duration: infinity - # connection-shutdown-grace-period: 5s - # - # Envoy cluster settings. - # cluster: - # configure the cluster dns lookup family - # valid options are: auto (default), v4, v6 - # dns-lookup-family: auto -kind: ConfigMap -metadata: - name: contour - namespace: flyte ---- -apiVersion: v1 data: db.yaml: | database: @@ -1967,71 +2165,157 @@ metadata: --- apiVersion: v1 data: - pass.txt: YXdlc29tZXNhdWNl -kind: Secret -metadata: - name: db-pass-9dgchhk2bm - namespace: flyte -type: Opaque ---- -apiVersion: v1 -kind: Service -metadata: - name: contour - namespace: flyte -spec: - ports: - - name: xds - port: 8001 - protocol: TCP - targetPort: 8001 - selector: - app: contour - type: ClusterIP ---- -apiVersion: v1 -kind: Service -metadata: - annotations: - contour.heptio.com/upstream-protocol.h2c: grpc - name: datacatalog - namespace: flyte -spec: - ports: - - name: http - port: 88 - protocol: TCP - targetPort: 8088 - - name: grpc - port: 89 - protocol: TCP - targetPort: 8089 - selector: - app: datacatalog ---- -apiVersion: v1 -kind: Service -metadata: - annotations: - service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp - name: envoy - namespace: flyte -spec: - externalTrafficPolicy: Local - ports: - - name: http - nodePort: 30081 - port: 80 - protocol: TCP - - name: https - port: 443 - protocol: TCP - selector: - app: envoy - type: LoadBalancer ---- -apiVersion: v1 -kind: Service + contour.yaml: | + # + # server: + # determine which XDS Server implementation to utilize in Contour. + # xds-server-type: contour + # + # Specify the gateway-api Gateway Contour should watch. + # gateway: + # name: contour + # namespace: projectcontour + # + # should contour expect to be running inside a k8s cluster + # incluster: true + # + # path to kubeconfig (if not running inside a k8s cluster) + # kubeconfig: /path/to/.kube/config + # + # Disable RFC-compliant behavior to strip "Content-Length" header if + # "Tranfer-Encoding: chunked" is also set. + # disableAllowChunkedLength: false + # Disable HTTPProxy permitInsecure field + disablePermitInsecure: false + tls: + # minimum TLS version that Contour will negotiate + # minimum-protocol-version: "1.2" + # TLS ciphers to be supported by Envoy TLS listeners when negotiating + # TLS 1.2. + # cipher-suites: + # - '[ECDHE-ECDSA-AES128-GCM-SHA256|ECDHE-ECDSA-CHACHA20-POLY1305]' + # - '[ECDHE-RSA-AES128-GCM-SHA256|ECDHE-RSA-CHACHA20-POLY1305]' + # - 'ECDHE-ECDSA-AES256-GCM-SHA384' + # - 'ECDHE-RSA-AES256-GCM-SHA384' + # Defines the Kubernetes name/namespace matching a secret to use + # as the fallback certificate when requests which don't match the + # SNI defined for a vhost. + fallback-certificate: + # name: fallback-secret-name + # namespace: projectcontour + envoy-client-certificate: + # name: envoy-client-cert-secret-name + # namespace: projectcontour + # The following config shows the defaults for the leader election. + # leaderelection: + # configmap-name: leader-elect + # configmap-namespace: projectcontour + ### Logging options + # Default setting + accesslog-format: envoy + # To enable JSON logging in Envoy + # accesslog-format: json + # The default fields that will be logged are specified below. + # To customise this list, just add or remove entries. + # The canonical list is available at + # https://godoc.org/github.com/projectcontour/contour/internal/envoy#JSONFields + # json-fields: + # - "@timestamp" + # - "authority" + # - "bytes_received" + # - "bytes_sent" + # - "downstream_local_address" + # - "downstream_remote_address" + # - "duration" + # - "method" + # - "path" + # - "protocol" + # - "request_id" + # - "requested_server_name" + # - "response_code" + # - "response_flags" + # - "uber_trace_id" + # - "upstream_cluster" + # - "upstream_host" + # - "upstream_local_address" + # - "upstream_service_time" + # - "user_agent" + # - "x_forwarded_for" + # + # default-http-versions: + # - "HTTP/2" + # - "HTTP/1.1" + # + # The following shows the default proxy timeout settings. + # timeouts: + # request-timeout: infinity + # connection-idle-timeout: 60s + # stream-idle-timeout: 5m + # max-connection-duration: infinity + # delayed-close-timeout: 1s + # connection-shutdown-grace-period: 5s + # + # Envoy cluster settings. + # cluster: + # configure the cluster dns lookup family + # valid options are: auto (default), v4, v6 + # dns-lookup-family: auto + # + # Envoy network settings. + # network: + # Configure the number of additional ingress proxy hops from the + # right side of the x-forwarded-for HTTP header to trust. + # num-trusted-hops: 0 + # + # Configure an optional global rate limit service. + # rateLimitService: + # Identifies the extension service defining the rate limit service, + # formatted as /. + # extensionService: projectcontour/ratelimit + # Defines the rate limit domain to pass to the rate limit service. + # Acts as a container for a set of rate limit definitions within + # the RLS. + # domain: contour + # Defines whether to allow requests to proceed when the rate limit + # service fails to respond with a valid rate limit decision within + # the timeout defined on the extension service. + # failOpen: false +kind: ConfigMap +metadata: + name: contour + namespace: projectcontour +--- +apiVersion: v1 +data: + pass.txt: YXdlc29tZXNhdWNl +kind: Secret +metadata: + name: db-pass-9dgchhk2bm + namespace: flyte +type: Opaque +--- +apiVersion: v1 +kind: Service +metadata: + annotations: + contour.heptio.com/upstream-protocol.h2c: grpc + name: datacatalog + namespace: flyte +spec: + ports: + - name: http + port: 88 + protocol: TCP + targetPort: 8088 + - name: grpc + port: 89 + protocol: TCP + targetPort: 8089 + selector: + app: datacatalog +--- +apiVersion: v1 +kind: Service metadata: annotations: projectcontour.io/upstream-protocol.h2c: grpc @@ -2122,110 +2406,41 @@ spec: app: postgres type: NodePort --- -apiVersion: apps/v1 -kind: Deployment +apiVersion: v1 +kind: Service metadata: - labels: - app: contour name: contour - namespace: flyte + namespace: projectcontour spec: - replicas: 2 + ports: + - name: xds + port: 8001 + protocol: TCP + targetPort: 8001 selector: - matchLabels: - app: contour - strategy: - rollingUpdate: - maxSurge: 50% - type: RollingUpdate - template: - metadata: - annotations: - prometheus.io/port: "8000" - prometheus.io/scrape: "true" - labels: - app: contour - spec: - affinity: - podAntiAffinity: - preferredDuringSchedulingIgnoredDuringExecution: - - podAffinityTerm: - labelSelector: - matchLabels: - app: contour - topologyKey: kubernetes.io/hostname - weight: 100 - containers: - - args: - - serve - - --incluster - - --xds-address=0.0.0.0 - - --xds-port=8001 - - --envoy-service-http-port=80 - - --envoy-service-https-port=443 - - --contour-cafile=/certs/ca.crt - - --contour-cert-file=/certs/tls.crt - - --contour-key-file=/certs/tls.key - - --config-path=/config/contour.yaml - command: - - contour - env: - - name: CONTOUR_NAMESPACE - valueFrom: - fieldRef: - apiVersion: v1 - fieldPath: metadata.namespace - - name: POD_NAME - valueFrom: - fieldRef: - apiVersion: v1 - fieldPath: metadata.name - image: docker.io/projectcontour/contour:v1.11.0 - imagePullPolicy: IfNotPresent - livenessProbe: - httpGet: - path: /healthz - port: 8000 - name: contour - ports: - - containerPort: 8001 - name: xds - protocol: TCP - - containerPort: 8000 - name: metrics - protocol: TCP - - containerPort: 6060 - name: debug - protocol: TCP - readinessProbe: - initialDelaySeconds: 15 - periodSeconds: 10 - tcpSocket: - port: 8001 - volumeMounts: - - mountPath: /certs - name: contourcert - readOnly: true - - mountPath: /config - name: contour-config - readOnly: true - dnsPolicy: ClusterFirst - securityContext: - runAsGroup: 65534 - runAsNonRoot: true - runAsUser: 65534 - serviceAccountName: contour - volumes: - - name: contourcert - secret: - secretName: contourcert - - configMap: - defaultMode: 420 - items: - - key: contour.yaml - path: contour.yaml - name: contour - name: contour-config + app: contour + type: ClusterIP +--- +apiVersion: v1 +kind: Service +metadata: + annotations: + service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp + name: envoy + namespace: projectcontour +spec: + externalTrafficPolicy: Local + ports: + - name: http + nodePort: 30081 + port: 80 + protocol: TCP + - name: https + port: 443 + protocol: TCP + selector: + app: envoy + type: NodePort --- apiVersion: apps/v1 kind: Deployment @@ -2579,6 +2794,111 @@ spec: - emptyDir: {} name: postgres-storage --- +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + app: contour + name: contour + namespace: projectcontour +spec: + replicas: 2 + selector: + matchLabels: + app: contour + strategy: + rollingUpdate: + maxSurge: 50% + type: RollingUpdate + template: + metadata: + annotations: + prometheus.io/port: "8000" + prometheus.io/scrape: "true" + labels: + app: contour + spec: + affinity: + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - podAffinityTerm: + labelSelector: + matchLabels: + app: contour + topologyKey: kubernetes.io/hostname + weight: 100 + containers: + - args: + - serve + - --incluster + - --xds-address=0.0.0.0 + - --xds-port=8001 + - --envoy-service-http-port=80 + - --envoy-service-https-port=443 + - --contour-cafile=/certs/ca.crt + - --contour-cert-file=/certs/tls.crt + - --contour-key-file=/certs/tls.key + - --config-path=/config/contour.yaml + command: + - contour + env: + - name: CONTOUR_NAMESPACE + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.namespace + - name: POD_NAME + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.name + image: docker.io/projectcontour/contour:v1.13.1 + imagePullPolicy: IfNotPresent + livenessProbe: + httpGet: + path: /healthz + port: 8000 + name: contour + ports: + - containerPort: 8001 + name: xds + protocol: TCP + - containerPort: 8000 + name: metrics + protocol: TCP + - containerPort: 6060 + name: debug + protocol: TCP + readinessProbe: + initialDelaySeconds: 15 + periodSeconds: 10 + tcpSocket: + port: 8001 + volumeMounts: + - mountPath: /certs + name: contourcert + readOnly: true + - mountPath: /config + name: contour-config + readOnly: true + dnsPolicy: ClusterFirst + securityContext: + runAsGroup: 65534 + runAsNonRoot: true + runAsUser: 65534 + serviceAccountName: contour + volumes: + - name: contourcert + secret: + secretName: contourcert + - configMap: + defaultMode: 420 + items: + - key: contour.yaml + path: contour.yaml + name: contour + name: contour-config +--- apiVersion: batch/v1beta1 kind: CronJob metadata: @@ -2627,7 +2947,7 @@ metadata: labels: app: envoy name: envoy - namespace: flyte + namespace: projectcontour spec: selector: matchLabels: @@ -2648,7 +2968,7 @@ spec: - shutdown-manager command: - /bin/contour - image: docker.io/projectcontour/contour:v1.11.0 + image: docker.io/projectcontour/contour:v1.13.1 imagePullPolicy: IfNotPresent lifecycle: preStop: @@ -2683,7 +3003,7 @@ spec: fieldRef: apiVersion: v1 fieldPath: metadata.name - image: docker.io/envoyproxy/envoy:v1.16.2 + image: docker.io/envoyproxy/envoy:v1.17.1 imagePullPolicy: IfNotPresent lifecycle: preStop: @@ -2710,8 +3030,10 @@ spec: volumeMounts: - mountPath: /config name: envoy-config + readOnly: true - mountPath: /certs name: envoycert + readOnly: true initContainers: - args: - bootstrap @@ -2730,7 +3052,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: docker.io/projectcontour/contour:v1.11.0 + image: docker.io/projectcontour/contour:v1.13.1 imagePullPolicy: IfNotPresent name: envoy-initconfig volumeMounts: @@ -2756,8 +3078,8 @@ spec: apiVersion: batch/v1 kind: Job metadata: - name: contour-certgen-v1.11.0 - namespace: flyte + name: contour-certgen-v1.13.1 + namespace: projectcontour spec: backoffLimit: 1 completions: 1 @@ -2781,7 +3103,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: docker.io/projectcontour/contour:v1.11.0 + image: docker.io/projectcontour/contour:v1.13.1 imagePullPolicy: Always name: contour restartPolicy: Never diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index 60c64b5def..794dde417f 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -18,147 +18,6 @@ spec: scope: Namespaced version: v1alpha1 --- -apiVersion: apiextensions.k8s.io/v1beta1 -kind: CustomResourceDefinition -metadata: - labels: - component: ingressroute - name: ingressroutes.contour.heptio.com -spec: - additionalPrinterColumns: - - JSONPath: .spec.virtualhost.fqdn - description: Fully qualified domain name - name: FQDN - type: string - - JSONPath: .spec.virtualhost.tls.secretName - description: Secret with TLS credentials - name: TLS Secret - type: string - - JSONPath: .spec.routes[0].match - description: First routes defined - name: First route - type: string - - JSONPath: .status.currentStatus - description: The current status of the IngressRoute - name: Status - type: string - - JSONPath: .status.description - description: Description of the current status - name: Status Description - type: string - group: contour.heptio.com - names: - kind: IngressRoute - plural: ingressroutes - scope: Namespaced - validation: - openAPIV3Schema: - properties: - spec: - properties: - healthCheck: - properties: - healthyThresholdCount: - type: integer - intervalSeconds: - type: integer - path: - pattern: ^\/.*$ - type: string - timeoutSeconds: - type: integer - unhealthyThresholdCount: - type: integer - required: - - path - type: object - routes: - items: - properties: - delegate: - properties: - name: - pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ - type: string - namespace: - pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ - type: string - required: - - name - type: object - match: - pattern: ^\/.*$ - type: string - services: - items: - properties: - healthCheck: - properties: - healthyThresholdCount: - type: integer - intervalSeconds: - type: integer - path: - pattern: ^\/.*$ - type: string - timeoutSeconds: - type: integer - unhealthyThresholdCount: - type: integer - required: - - path - type: object - name: - pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ - type: string - port: - type: integer - strategy: - enum: - - RoundRobin - - WeightedLeastRequest - - Random - - RingHash - - Maglev - type: string - weight: - type: integer - required: - - name - - port - type: object - type: array - required: - - match - type: array - strategy: - enum: - - RoundRobin - - LeastRequest - - Random - - RingHash - - Maglev - type: string - virtualhost: - properties: - aliases: - items: - pattern: ^([a-z0-9]+(-[a-z0-9]+)*\.)+[a-z]{2,}$ - type: string - type: array - fqdn: - pattern: ^([a-zA-Z0-9]+(-[a-zA-Z0-9]+)*\.)+[a-z]{2,}$ - type: string - required: - - routes - version: v1beta1 ---- -apiVersion: v1 -kind: ServiceAccount -metadata: - name: contour - namespace: flyte ---- apiVersion: v1 kind: ServiceAccount metadata: @@ -263,56 +122,6 @@ rules: - deletecollection --- apiVersion: rbac.authorization.k8s.io/v1beta1 -kind: ClusterRole -metadata: - name: contour -rules: -- apiGroups: - - "" - resources: - - configmaps - - endpoints - - nodes - - pods - - secrets - verbs: - - list - - watch -- apiGroups: - - "" - resources: - - nodes - verbs: - - get -- apiGroups: - - "" - resources: - - services - verbs: - - get - - list - - watch -- apiGroups: - - extensions - resources: - - ingresses - verbs: - - get - - list - - watch -- apiGroups: - - contour.heptio.com - resources: - - ingressroutes - verbs: - - get - - list - - watch - - put - - post - - patch ---- -apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: flyteadmin-binding @@ -340,19 +149,6 @@ subjects: name: flytepropeller namespace: flyte --- -apiVersion: rbac.authorization.k8s.io/v1beta1 -kind: ClusterRoleBinding -metadata: - name: contour -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: ClusterRole - name: contour -subjects: -- kind: ServiceAccount - name: contour - namespace: flyte ---- apiVersion: v1 data: aa_namespace.yaml: | @@ -641,22 +437,6 @@ type: Opaque --- apiVersion: v1 kind: Service -metadata: - labels: - app: contour - name: contour - namespace: flyte -spec: - ports: - - nodePort: 30081 - port: 80 - protocol: TCP - selector: - app: contour - type: NodePort ---- -apiVersion: v1 -kind: Service metadata: annotations: contour.heptio.com/upstream-protocol.h2c: grpc @@ -724,88 +504,6 @@ spec: --- apiVersion: apps/v1 kind: Deployment -metadata: - labels: - app: contour - name: contour - namespace: flyte -spec: - replicas: 1 - selector: - matchLabels: - app: contour - template: - metadata: - annotations: - prometheus.io/format: prometheus - prometheus.io/path: /stats - prometheus.io/port: "8002" - prometheus.io/scrape: "true" - labels: - app: contour - spec: - containers: - - args: - - -c - - /config/contour.yaml - - --service-cluster - - cluster0 - - --service-node - - node0 - command: - - envoy - image: docker.io/envoyproxy/envoy-alpine:v1.6.0 - name: envoy-envoyingressv1 - ports: - - containerPort: 80 - name: http - - containerPort: 8002 - name: statsd - resources: - limits: - cpu: 100m - memory: 100Mi - requests: - cpu: 100m - memory: 100Mi - volumeMounts: - - mountPath: /config - name: contour-config - - args: - - serve - - --incluster - - --envoy-http-port=80 - - --debug-http-port=6069 - command: - - contour - image: gcr.io/heptio-images/contour:v0.6.1 - imagePullPolicy: Always - name: contour-unknown - ports: - - containerPort: 8000 - name: contour - dnsPolicy: ClusterFirst - initContainers: - - args: - - bootstrap - - /config/contour.yaml - - --statsd-enabled - command: - - contour - image: gcr.io/heptio-images/contour:v0.6.1 - imagePullPolicy: Always - name: envoy-initconfig - volumeMounts: - - mountPath: /config - name: contour-config - serviceAccountName: contour - terminationGracePeriodSeconds: 30 - volumes: - - emptyDir: {} - name: contour-config ---- -apiVersion: apps/v1 -kind: Deployment metadata: labels: app: datacatalog diff --git a/docker/sandbox/wait-for-flyte.sh b/docker/sandbox/wait-for-flyte.sh index a5266e090f..c99e722944 100755 --- a/docker/sandbox/wait-for-flyte.sh +++ b/docker/sandbox/wait-for-flyte.sh @@ -16,6 +16,6 @@ timeout 600 sh -c "until k3s kubectl get namespace flyte &> /dev/null; do sleep k3s kubectl wait --for=condition=available deployment/datacatalog deployment/flyteadmin deployment/flyteconsole deployment/flytepropeller -n flyte --timeout=10m || ( echo >&2 "Timed out while waiting for the Flyte deployment to start"; exit 1 ) # Wait for envoy proxy to become ready -timeout 600 sh -c 'until [[ $(k3s kubectl get daemonset envoy -n flyte -o jsonpath="{.status.numberReady}") -eq 1 ]]; do sleep 1; done' || ( echo >&2 "Timed out while waiting for the Flyte envoy proxy to start"; exit 1 ) +timeout 600 sh -c 'until [[ $(k3s kubectl get daemonset envoy -n projectcontour -o jsonpath="{.status.numberReady}") -eq 1 ]]; do sleep 1; done' || ( echo >&2 "Timed out while waiting for the Flyte envoy proxy to start"; exit 1 ) echo "Flyte is ready! Flyte UI is available at http://localhost:30081/console." diff --git a/end2end/execute.sh b/end2end/execute.sh index e03a3e25de..a59c312390 100755 --- a/end2end/execute.sh +++ b/end2end/execute.sh @@ -35,8 +35,6 @@ function wait_for_flyte_deploys() { echo "" kubectl -n flyte rollout status deployment minio echo "" - kubectl -n flyte rollout status deployment contour - echo "" echo "Flyte deployed in $SECONDS seconds." } diff --git a/kustomize/overlays/eks/dependencies/alb_ingress/README.md b/kustomize/base/addons/alb_ingress/README.md similarity index 100% rename from kustomize/overlays/eks/dependencies/alb_ingress/README.md rename to kustomize/base/addons/alb_ingress/README.md diff --git a/kustomize/overlays/eks/dependencies/alb_ingress/ingress.yaml b/kustomize/base/addons/alb_ingress/ingress.yaml similarity index 100% rename from kustomize/overlays/eks/dependencies/alb_ingress/ingress.yaml rename to kustomize/base/addons/alb_ingress/ingress.yaml diff --git a/kustomize/overlays/gcp/dependencies/cloudsqlproxy/deployment.yaml b/kustomize/base/addons/cloudsqlproxy/deployment.yaml similarity index 100% rename from kustomize/overlays/gcp/dependencies/cloudsqlproxy/deployment.yaml rename to kustomize/base/addons/cloudsqlproxy/deployment.yaml diff --git a/kustomize/overlays/gcp/dependencies/cloudsqlproxy/kustomization.yaml b/kustomize/base/addons/cloudsqlproxy/kustomization.yaml similarity index 100% rename from kustomize/overlays/gcp/dependencies/cloudsqlproxy/kustomization.yaml rename to kustomize/base/addons/cloudsqlproxy/kustomization.yaml diff --git a/kustomize/overlays/gcp/dependencies/cloudsqlproxy/service.yaml b/kustomize/base/addons/cloudsqlproxy/service.yaml similarity index 100% rename from kustomize/overlays/gcp/dependencies/cloudsqlproxy/service.yaml rename to kustomize/base/addons/cloudsqlproxy/service.yaml diff --git a/kustomize/base/addons/contour_ingress_controller/kustomization.yaml b/kustomize/base/addons/contour_ingress_controller/kustomization.yaml new file mode 100644 index 0000000000..e75e137804 --- /dev/null +++ b/kustomize/base/addons/contour_ingress_controller/kustomization.yaml @@ -0,0 +1,4 @@ +resources: +- https://raw.githubusercontent.com/projectcontour/contour/release-1.13/examples/render/contour.yaml +patchesStrategicMerge: +- service.yaml diff --git a/kustomize/overlays/sandbox/dependencies/contour_ingress_controller/service.yaml b/kustomize/base/addons/contour_ingress_controller/service.yaml similarity index 60% rename from kustomize/overlays/sandbox/dependencies/contour_ingress_controller/service.yaml rename to kustomize/base/addons/contour_ingress_controller/service.yaml index 79e68127b1..a848798e11 100644 --- a/kustomize/overlays/sandbox/dependencies/contour_ingress_controller/service.yaml +++ b/kustomize/base/addons/contour_ingress_controller/service.yaml @@ -2,15 +2,13 @@ apiVersion: v1 kind: Service metadata: - name: contour - labels: - app: contour + name: envoy + namespace: projectcontour spec: # use NodePort to make sure the service is accessible type: NodePort - selector: - app: contour ports: - - protocol: TCP - port: 80 + - port: 80 + name: http + protocol: TCP nodePort: 30081 diff --git a/kustomize/overlays/sandbox/dependencies/database/database.yaml b/kustomize/base/addons/database/database.yaml similarity index 100% rename from kustomize/overlays/sandbox/dependencies/database/database.yaml rename to kustomize/base/addons/database/database.yaml diff --git a/kustomize/overlays/sandbox/dependencies/database/kustomization.yaml b/kustomize/base/addons/database/kustomization.yaml similarity index 100% rename from kustomize/overlays/sandbox/dependencies/database/kustomization.yaml rename to kustomize/base/addons/database/kustomization.yaml diff --git a/kustomize/overlays/eks/dependencies/redis/deployment.yaml b/kustomize/base/addons/redis/deployment.yaml similarity index 100% rename from kustomize/overlays/eks/dependencies/redis/deployment.yaml rename to kustomize/base/addons/redis/deployment.yaml diff --git a/kustomize/overlays/eks/dependencies/redis/kustomization.yaml b/kustomize/base/addons/redis/kustomization.yaml similarity index 100% rename from kustomize/overlays/eks/dependencies/redis/kustomization.yaml rename to kustomize/base/addons/redis/kustomization.yaml diff --git a/kustomize/overlays/eks/dependencies/redis/service.yaml b/kustomize/base/addons/redis/service.yaml similarity index 100% rename from kustomize/overlays/eks/dependencies/redis/service.yaml rename to kustomize/base/addons/redis/service.yaml diff --git a/kustomize/overlays/sandbox/dependencies/storage/kustomization.yaml b/kustomize/base/addons/storage/kustomization.yaml similarity index 100% rename from kustomize/overlays/sandbox/dependencies/storage/kustomization.yaml rename to kustomize/base/addons/storage/kustomization.yaml diff --git a/kustomize/overlays/sandbox/dependencies/storage/storage.yaml b/kustomize/base/addons/storage/storage.yaml similarity index 100% rename from kustomize/overlays/sandbox/dependencies/storage/storage.yaml rename to kustomize/base/addons/storage/storage.yaml diff --git a/kustomize/base/operators/kfoperators/namespace/kustomization.yaml b/kustomize/base/operators/kfoperators/namespace/kustomization.yaml new file mode 100644 index 0000000000..bf20f4df68 --- /dev/null +++ b/kustomize/base/operators/kfoperators/namespace/kustomization.yaml @@ -0,0 +1,2 @@ +resources: +- namespace.yaml diff --git a/kustomize/base/operators/kfoperators/namespace/namespace.yaml b/kustomize/base/operators/kfoperators/namespace/namespace.yaml new file mode 100644 index 0000000000..7a940e4673 --- /dev/null +++ b/kustomize/base/operators/kfoperators/namespace/namespace.yaml @@ -0,0 +1,4 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: kubeflow diff --git a/kustomize/base/operators/kfoperators/pytorch/kustomization.yaml b/kustomize/base/operators/kfoperators/pytorch/kustomization.yaml index 2cab6d5984..a75b8ad07c 100644 --- a/kustomize/base/operators/kfoperators/pytorch/kustomization.yaml +++ b/kustomize/base/operators/kfoperators/pytorch/kustomization.yaml @@ -1,3 +1,4 @@ bases: + - ../namespace - github.com/kubeflow/manifests/pytorch-job/pytorch-job-crds/base?ref=v1.0.0 - - github.com/kubeflow/manifests/pytorch-job/pytorch-operator/base?ref=v1.0.0 \ No newline at end of file + - github.com/kubeflow/manifests/pytorch-job/pytorch-operator/base?ref=v1.0.0 diff --git a/kustomize/base/operators/kfoperators/tensorflow/kustomization.yaml b/kustomize/base/operators/kfoperators/tensorflow/kustomization.yaml index 33b2883011..9d7d7b7def 100644 --- a/kustomize/base/operators/kfoperators/tensorflow/kustomization.yaml +++ b/kustomize/base/operators/kfoperators/tensorflow/kustomization.yaml @@ -1,3 +1,4 @@ bases: + - ../namespace - github.com/kubeflow/manifests/tf-training/tf-job-crds/base?ref=v1.0.0 - github.com/kubeflow/manifests/tf-training/tf-job-operator/base?ref=v1.0.0 diff --git a/kustomize/base/single_cluster/headless/kustomization.yaml b/kustomize/base/single_cluster/headless/kustomization.yaml index c9d371de72..0a863fab06 100644 --- a/kustomize/base/single_cluster/headless/kustomization.yaml +++ b/kustomize/base/single_cluster/headless/kustomization.yaml @@ -4,6 +4,7 @@ kind: Kustomization # All the resources that make up the deployment resources: # global resources +- ../../namespace - ../../ingress - ../../admindeployment - ../../datacatalog diff --git a/kustomize/base/single_cluster/headless/pass.txt b/kustomize/base/single_cluster/headless/pass.txt deleted file mode 100644 index f6b1b7073b..0000000000 --- a/kustomize/base/single_cluster/headless/pass.txt +++ /dev/null @@ -1 +0,0 @@ -awesomesauce diff --git a/kustomize/overlays/eks/datacatalog/kustomization.yaml b/kustomize/overlays/eks/datacatalog/kustomization.yaml deleted file mode 100644 index 96588e8106..0000000000 --- a/kustomize/overlays/eks/datacatalog/kustomization.yaml +++ /dev/null @@ -1,5 +0,0 @@ -resources: - - service.yaml - -patches: - - deployment.yaml diff --git a/kustomize/overlays/eks/admin/deployment.yaml b/kustomize/overlays/eks/flyte/admin/deployment.yaml similarity index 100% rename from kustomize/overlays/eks/admin/deployment.yaml rename to kustomize/overlays/eks/flyte/admin/deployment.yaml diff --git a/kustomize/overlays/eks/admin/service.yaml b/kustomize/overlays/eks/flyte/admin/service.yaml similarity index 100% rename from kustomize/overlays/eks/admin/service.yaml rename to kustomize/overlays/eks/flyte/admin/service.yaml diff --git a/kustomize/overlays/eks/admin/serviceaccount.yaml b/kustomize/overlays/eks/flyte/admin/serviceaccount.yaml similarity index 100% rename from kustomize/overlays/eks/admin/serviceaccount.yaml rename to kustomize/overlays/eks/flyte/admin/serviceaccount.yaml diff --git a/kustomize/overlays/eks/config/admin/db.yaml b/kustomize/overlays/eks/flyte/config/admin/db.yaml similarity index 100% rename from kustomize/overlays/eks/config/admin/db.yaml rename to kustomize/overlays/eks/flyte/config/admin/db.yaml diff --git a/kustomize/overlays/eks/config/admin/remote_data.yaml b/kustomize/overlays/eks/flyte/config/admin/remote_data.yaml similarity index 100% rename from kustomize/overlays/eks/config/admin/remote_data.yaml rename to kustomize/overlays/eks/flyte/config/admin/remote_data.yaml diff --git a/kustomize/overlays/eks/config/admin/task_resource_defaults.yaml b/kustomize/overlays/eks/flyte/config/admin/task_resource_defaults.yaml similarity index 100% rename from kustomize/overlays/eks/config/admin/task_resource_defaults.yaml rename to kustomize/overlays/eks/flyte/config/admin/task_resource_defaults.yaml diff --git a/kustomize/overlays/eks/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml b/kustomize/overlays/eks/flyte/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml similarity index 100% rename from kustomize/overlays/eks/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml rename to kustomize/overlays/eks/flyte/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml diff --git a/kustomize/overlays/eks/config/clusterresource-templates/ad_spark-role.yaml b/kustomize/overlays/eks/flyte/config/clusterresource-templates/ad_spark-role.yaml similarity index 100% rename from kustomize/overlays/eks/config/clusterresource-templates/ad_spark-role.yaml rename to kustomize/overlays/eks/flyte/config/clusterresource-templates/ad_spark-role.yaml diff --git a/kustomize/overlays/eks/config/clusterresource-templates/ae_spark-service-account.yaml b/kustomize/overlays/eks/flyte/config/clusterresource-templates/ae_spark-service-account.yaml similarity index 100% rename from kustomize/overlays/eks/config/clusterresource-templates/ae_spark-service-account.yaml rename to kustomize/overlays/eks/flyte/config/clusterresource-templates/ae_spark-service-account.yaml diff --git a/kustomize/overlays/eks/config/clusterresource-templates/af_spark-role-binding.yaml b/kustomize/overlays/eks/flyte/config/clusterresource-templates/af_spark-role-binding.yaml similarity index 100% rename from kustomize/overlays/eks/config/clusterresource-templates/af_spark-role-binding.yaml rename to kustomize/overlays/eks/flyte/config/clusterresource-templates/af_spark-role-binding.yaml diff --git a/kustomize/overlays/eks/config/common/storage.yaml b/kustomize/overlays/eks/flyte/config/common/storage.yaml similarity index 100% rename from kustomize/overlays/eks/config/common/storage.yaml rename to kustomize/overlays/eks/flyte/config/common/storage.yaml diff --git a/kustomize/overlays/eks/config/datacatalog/db.yaml b/kustomize/overlays/eks/flyte/config/datacatalog/db.yaml similarity index 100% rename from kustomize/overlays/eks/config/datacatalog/db.yaml rename to kustomize/overlays/eks/flyte/config/datacatalog/db.yaml diff --git a/kustomize/overlays/eks/config/propeller/core.yaml b/kustomize/overlays/eks/flyte/config/propeller/core.yaml similarity index 100% rename from kustomize/overlays/eks/config/propeller/core.yaml rename to kustomize/overlays/eks/flyte/config/propeller/core.yaml diff --git a/kustomize/overlays/eks/config/propeller/enabled_plugins.yaml b/kustomize/overlays/eks/flyte/config/propeller/enabled_plugins.yaml similarity index 100% rename from kustomize/overlays/eks/config/propeller/enabled_plugins.yaml rename to kustomize/overlays/eks/flyte/config/propeller/enabled_plugins.yaml diff --git a/kustomize/overlays/eks/config/propeller/plugins/catalog_cache.yaml b/kustomize/overlays/eks/flyte/config/propeller/plugins/catalog_cache.yaml similarity index 100% rename from kustomize/overlays/eks/config/propeller/plugins/catalog_cache.yaml rename to kustomize/overlays/eks/flyte/config/propeller/plugins/catalog_cache.yaml diff --git a/kustomize/overlays/eks/config/propeller/plugins/k8s.yaml b/kustomize/overlays/eks/flyte/config/propeller/plugins/k8s.yaml similarity index 100% rename from kustomize/overlays/eks/config/propeller/plugins/k8s.yaml rename to kustomize/overlays/eks/flyte/config/propeller/plugins/k8s.yaml diff --git a/kustomize/overlays/eks/config/propeller/plugins/qubole.yaml b/kustomize/overlays/eks/flyte/config/propeller/plugins/qubole.yaml similarity index 100% rename from kustomize/overlays/eks/config/propeller/plugins/qubole.yaml rename to kustomize/overlays/eks/flyte/config/propeller/plugins/qubole.yaml diff --git a/kustomize/overlays/eks/config/propeller/plugins/spark.yaml b/kustomize/overlays/eks/flyte/config/propeller/plugins/spark.yaml similarity index 100% rename from kustomize/overlays/eks/config/propeller/plugins/spark.yaml rename to kustomize/overlays/eks/flyte/config/propeller/plugins/spark.yaml diff --git a/kustomize/overlays/eks/config/propeller/plugins/task_logs.yaml b/kustomize/overlays/eks/flyte/config/propeller/plugins/task_logs.yaml similarity index 100% rename from kustomize/overlays/eks/config/propeller/plugins/task_logs.yaml rename to kustomize/overlays/eks/flyte/config/propeller/plugins/task_logs.yaml diff --git a/kustomize/overlays/eks/console/service.yaml b/kustomize/overlays/eks/flyte/console/service.yaml similarity index 100% rename from kustomize/overlays/eks/console/service.yaml rename to kustomize/overlays/eks/flyte/console/service.yaml diff --git a/kustomize/overlays/eks/datacatalog/deployment.yaml b/kustomize/overlays/eks/flyte/datacatalog/deployment.yaml similarity index 100% rename from kustomize/overlays/eks/datacatalog/deployment.yaml rename to kustomize/overlays/eks/flyte/datacatalog/deployment.yaml diff --git a/kustomize/overlays/eks/datacatalog/service.yaml b/kustomize/overlays/eks/flyte/datacatalog/service.yaml similarity index 100% rename from kustomize/overlays/eks/datacatalog/service.yaml rename to kustomize/overlays/eks/flyte/datacatalog/service.yaml diff --git a/kustomize/overlays/eks/ingress/ingress.yaml b/kustomize/overlays/eks/flyte/ingress/ingress.yaml similarity index 100% rename from kustomize/overlays/eks/ingress/ingress.yaml rename to kustomize/overlays/eks/flyte/ingress/ingress.yaml diff --git a/kustomize/overlays/eks/flyte/kustomization.yaml b/kustomize/overlays/eks/flyte/kustomization.yaml new file mode 100644 index 0000000000..b5e9e0173f --- /dev/null +++ b/kustomize/overlays/eks/flyte/kustomization.yaml @@ -0,0 +1,64 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +# Override the namespace +namespace: flyte + +bases: +- ../../../base/single_cluster/complete + +resources: +- datacatalog/service.yaml +- propeller/service.yaml + +patchesStrategicMerge: +- admin/deployment.yaml +- admin/service.yaml +- admin/serviceaccount.yaml +- datacatalog/deployment.yaml +- propeller/deployment.yaml +- propeller/serviceaccount.yaml +- ingress/ingress.yaml +- console/service.yaml + +# Files are read in alphabetical order. To ensure that we create the namespace first, prefix the file name with "aa". + +configMapGenerator: +- behavior: merge + files: + - ./config/admin/db.yaml + - ./config/admin/remote_data.yaml + - ./config/admin/task_resource_defaults.yaml + - ./config/common/storage.yaml + name: flyte-admin-config +- behavior: merge + files: + - ./config/clusterresource-templates/ac_project-copilot-dataconfig.yaml + - ./config/clusterresource-templates/ad_spark-role.yaml + - ./config/clusterresource-templates/ae_spark-service-account.yaml + - ./config/clusterresource-templates/af_spark-role-binding.yaml + name: clusterresource-template +- behavior: merge + files: + - ./config/propeller/core.yaml + - ./config/propeller/enabled_plugins.yaml + - ./config/propeller/plugins/catalog_cache.yaml + - ./config/propeller/plugins/k8s.yaml + - ./config/propeller/plugins/qubole.yaml + - ./config/propeller/plugins/spark.yaml + - ./config/propeller/plugins/task_logs.yaml + - ./config/common/storage.yaml + name: flyte-propeller-config +- behavior: merge + files: + - ./config/common/storage.yaml + - ./config/datacatalog/db.yaml + name: datacatalog-config + +########### +# Use this to record the password for the Google CloudSQL Database +secretGenerator: +- name: db-pass + behavior: merge + literals: + - pass.txt="yourpassword" diff --git a/kustomize/overlays/eks/propeller/deployment.yaml b/kustomize/overlays/eks/flyte/propeller/deployment.yaml similarity index 100% rename from kustomize/overlays/eks/propeller/deployment.yaml rename to kustomize/overlays/eks/flyte/propeller/deployment.yaml diff --git a/kustomize/overlays/eks/propeller/service.yaml b/kustomize/overlays/eks/flyte/propeller/service.yaml similarity index 100% rename from kustomize/overlays/eks/propeller/service.yaml rename to kustomize/overlays/eks/flyte/propeller/service.yaml diff --git a/kustomize/overlays/eks/propeller/serviceaccount.yaml b/kustomize/overlays/eks/flyte/propeller/serviceaccount.yaml similarity index 100% rename from kustomize/overlays/eks/propeller/serviceaccount.yaml rename to kustomize/overlays/eks/flyte/propeller/serviceaccount.yaml diff --git a/kustomize/overlays/eks/kustomization.yaml b/kustomize/overlays/eks/kustomization.yaml index 1d3a63bb9c..fa35ea88e7 100644 --- a/kustomize/overlays/eks/kustomization.yaml +++ b/kustomize/overlays/eks/kustomization.yaml @@ -1,96 +1,39 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization -# Override the namespace -namespace: flyte - bases: -- ../../base/single_cluster/complete - # All the resources that make up the deployment - ######## - # Operators to be enabled - ############# - # Dependencies to be configured - # In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc - # This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) - # Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's - # Add node ports for ease of use locally -resources: +- flyte +######## +# Operators to be enabled +############# +# Dependencies to be configured +# In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc +# This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) +# Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's +# Add node ports for ease of use locally - ../../base/operators/spark - ../../base/operators/kfoperators/pytorch - # TODO Fix deployment of alb_ingress controller - #- ./dependencies/alb_ingress - # Optional dependency -- ./dependencies/redis - -patchesStrategicMerge: -- admin/deployment.yaml -- admin/service.yaml -- admin/serviceaccount.yaml -- datacatalog/deployment.yaml -- propeller/deployment.yaml -- propeller/serviceaccount.yaml -- ingress/ingress.yaml -- console/service.yaml - -# Files are read in alphabetical order. To ensure that we create the namespace first, prefix the file name with "aa". - -configMapGenerator: -- behavior: merge - files: - - ./config/admin/db.yaml - - ./config/admin/remote_data.yaml - - ./config/admin/task_resource_defaults.yaml - - ./config/common/storage.yaml - name: flyte-admin-config -- behavior: merge - files: - - ./config/clusterresource-templates/ac_project-copilot-dataconfig.yaml - - ./config/clusterresource-templates/ad_spark-role.yaml - - ./config/clusterresource-templates/ae_spark-service-account.yaml - - ./config/clusterresource-templates/af_spark-role-binding.yaml - name: clusterresource-template -- behavior: merge - files: - - ./config/propeller/core.yaml - - ./config/propeller/enabled_plugins.yaml - - ./config/propeller/plugins/catalog_cache.yaml - - ./config/propeller/plugins/k8s.yaml - - ./config/propeller/plugins/qubole.yaml - - ./config/propeller/plugins/spark.yaml - - ./config/propeller/plugins/task_logs.yaml - - ./config/common/storage.yaml - name: flyte-propeller-config -- behavior: merge - files: - - ./config/common/storage.yaml - - ./config/datacatalog/db.yaml - name: datacatalog-config - -########### -# Use this to record the password for the Google CloudSQL Database -secretGenerator: -- name: db-pass - behavior: merge - literals: - - pass.txt="yourpassword" +# TODO Fix deployment of alb_ingress controller +#- ../../base/addons/alb_ingress +# Optional dependency +- ../../base/addons/redis # Images that should be used images: - # FlyteAdmin - - name: flyteadmin # match images with this name - newTag: v0.3.29 # override the tag - newName: ghcr.io/lyft/flyteadmin # override the name - # FlyteConsole - - name: flyteconsole # match images with this name - newTag: v0.19.1 # override the tag - newName: ghcr.io/lyft/flyteconsole # override the namep - # Flyte DataCatalog - - name: datacatalog # match images with this name - newTag: v0.2.130 # override the tag - newName: ghcr.io/lyft/datacatalog # override the name - # FlytePropeller - - name: flytepropeller # match images with this name - newTag: v0.5.13 # override the tag - newName: ghcr.io/flyteorg/flytepropeller # override the name +# FlyteAdmin +- name: flyteadmin # match images with this name + newTag: v0.3.29 # override the tag + newName: ghcr.io/lyft/flyteadmin # override the name +# FlyteConsole +- name: flyteconsole # match images with this name + newTag: v0.19.1 # override the tag + newName: ghcr.io/lyft/flyteconsole # override the namep +# Flyte DataCatalog +- name: datacatalog # match images with this name + newTag: v0.2.130 # override the tag + newName: ghcr.io/lyft/datacatalog # override the name +# FlytePropeller +- name: flytepropeller # match images with this name + newTag: v0.5.13 # override the tag + newName: ghcr.io/flyteorg/flytepropeller # override the name diff --git a/kustomize/overlays/gcp/datacatalog/kustomization.yaml b/kustomize/overlays/gcp/datacatalog/kustomization.yaml deleted file mode 100644 index 96588e8106..0000000000 --- a/kustomize/overlays/gcp/datacatalog/kustomization.yaml +++ /dev/null @@ -1,5 +0,0 @@ -resources: - - service.yaml - -patches: - - deployment.yaml diff --git a/kustomize/overlays/gcp/dependencies/redis/deployment.yaml b/kustomize/overlays/gcp/dependencies/redis/deployment.yaml deleted file mode 100644 index 82067d702b..0000000000 --- a/kustomize/overlays/gcp/dependencies/redis/deployment.yaml +++ /dev/null @@ -1,59 +0,0 @@ -apiVersion: apps/v1 -kind: StatefulSet -metadata: - name: redis - namespace: flyte -spec: - replicas: 1 - selector: - matchLabels: - app: redis-resource-manager - serviceName: redis-resource-manager - template: - metadata: - labels: - app: redis-resource-manager - spec: - containers: - - env: - - name: REDIS_PASSWORD - value: mypassword - image: docker.io/bitnami/redis:4.0.2-r1 - imagePullPolicy: IfNotPresent - livenessProbe: - exec: - command: - - redis-cli - - ping - failureThreshold: 3 - initialDelaySeconds: 30 - periodSeconds: 10 - successThreshold: 1 - timeoutSeconds: 5 - name: redis-resource-manager - ports: - - containerPort: 6379 - name: redis - protocol: TCP - readinessProbe: - exec: - command: - - redis-cli - - ping - failureThreshold: 3 - initialDelaySeconds: 5 - periodSeconds: 10 - successThreshold: 1 - timeoutSeconds: 1 - resources: - requests: - cpu: 200m - memory: 128Mi - volumeMounts: - - mountPath: /bitnami - name: redis-data - dnsPolicy: ClusterFirst - restartPolicy: Always - volumes: - - name: redis-data - emptyDir: {} diff --git a/kustomize/overlays/gcp/dependencies/redis/kustomization.yaml b/kustomize/overlays/gcp/dependencies/redis/kustomization.yaml deleted file mode 100644 index a944d005ca..0000000000 --- a/kustomize/overlays/gcp/dependencies/redis/kustomization.yaml +++ /dev/null @@ -1,3 +0,0 @@ -resources: -- deployment.yaml -- service.yaml diff --git a/kustomize/overlays/gcp/dependencies/redis/service.yaml b/kustomize/overlays/gcp/dependencies/redis/service.yaml deleted file mode 100644 index 8c86264ae3..0000000000 --- a/kustomize/overlays/gcp/dependencies/redis/service.yaml +++ /dev/null @@ -1,16 +0,0 @@ -apiVersion: v1 -kind: Service -metadata: - labels: - app: redis-resource-manager - name: redis-resource-manager - namespace: flyte -spec: - ports: - - name: redis - port: 6379 - protocol: TCP - targetPort: redis - selector: - app: redis-resource-manager - type: ClusterIP diff --git a/kustomize/overlays/gcp/admin/deployment.yaml b/kustomize/overlays/gcp/flyte/admin/deployment.yaml similarity index 100% rename from kustomize/overlays/gcp/admin/deployment.yaml rename to kustomize/overlays/gcp/flyte/admin/deployment.yaml diff --git a/kustomize/overlays/gcp/admin/service.yaml b/kustomize/overlays/gcp/flyte/admin/service.yaml similarity index 100% rename from kustomize/overlays/gcp/admin/service.yaml rename to kustomize/overlays/gcp/flyte/admin/service.yaml diff --git a/kustomize/overlays/gcp/config/admin/db.yaml b/kustomize/overlays/gcp/flyte/config/admin/db.yaml similarity index 100% rename from kustomize/overlays/gcp/config/admin/db.yaml rename to kustomize/overlays/gcp/flyte/config/admin/db.yaml diff --git a/kustomize/overlays/gcp/config/admin/remote_data.yaml b/kustomize/overlays/gcp/flyte/config/admin/remote_data.yaml similarity index 100% rename from kustomize/overlays/gcp/config/admin/remote_data.yaml rename to kustomize/overlays/gcp/flyte/config/admin/remote_data.yaml diff --git a/kustomize/overlays/gcp/config/admin/task_resource_defaults.yaml b/kustomize/overlays/gcp/flyte/config/admin/task_resource_defaults.yaml similarity index 100% rename from kustomize/overlays/gcp/config/admin/task_resource_defaults.yaml rename to kustomize/overlays/gcp/flyte/config/admin/task_resource_defaults.yaml diff --git a/kustomize/overlays/gcp/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml b/kustomize/overlays/gcp/flyte/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml similarity index 100% rename from kustomize/overlays/gcp/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml rename to kustomize/overlays/gcp/flyte/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml diff --git a/kustomize/overlays/gcp/config/clusterresource-templates/ad_spark-role.yaml b/kustomize/overlays/gcp/flyte/config/clusterresource-templates/ad_spark-role.yaml similarity index 100% rename from kustomize/overlays/gcp/config/clusterresource-templates/ad_spark-role.yaml rename to kustomize/overlays/gcp/flyte/config/clusterresource-templates/ad_spark-role.yaml diff --git a/kustomize/overlays/gcp/config/clusterresource-templates/ae_spark-service-account.yaml b/kustomize/overlays/gcp/flyte/config/clusterresource-templates/ae_spark-service-account.yaml similarity index 100% rename from kustomize/overlays/gcp/config/clusterresource-templates/ae_spark-service-account.yaml rename to kustomize/overlays/gcp/flyte/config/clusterresource-templates/ae_spark-service-account.yaml diff --git a/kustomize/overlays/gcp/config/clusterresource-templates/af_spark-role-binding.yaml b/kustomize/overlays/gcp/flyte/config/clusterresource-templates/af_spark-role-binding.yaml similarity index 100% rename from kustomize/overlays/gcp/config/clusterresource-templates/af_spark-role-binding.yaml rename to kustomize/overlays/gcp/flyte/config/clusterresource-templates/af_spark-role-binding.yaml diff --git a/kustomize/overlays/gcp/config/common/storage.yaml b/kustomize/overlays/gcp/flyte/config/common/storage.yaml similarity index 100% rename from kustomize/overlays/gcp/config/common/storage.yaml rename to kustomize/overlays/gcp/flyte/config/common/storage.yaml diff --git a/kustomize/overlays/gcp/config/datacatalog/db.yaml b/kustomize/overlays/gcp/flyte/config/datacatalog/db.yaml similarity index 100% rename from kustomize/overlays/gcp/config/datacatalog/db.yaml rename to kustomize/overlays/gcp/flyte/config/datacatalog/db.yaml diff --git a/kustomize/overlays/gcp/config/propeller/core.yaml b/kustomize/overlays/gcp/flyte/config/propeller/core.yaml similarity index 100% rename from kustomize/overlays/gcp/config/propeller/core.yaml rename to kustomize/overlays/gcp/flyte/config/propeller/core.yaml diff --git a/kustomize/overlays/gcp/config/propeller/enabled_plugins.yaml b/kustomize/overlays/gcp/flyte/config/propeller/enabled_plugins.yaml similarity index 100% rename from kustomize/overlays/gcp/config/propeller/enabled_plugins.yaml rename to kustomize/overlays/gcp/flyte/config/propeller/enabled_plugins.yaml diff --git a/kustomize/overlays/gcp/config/propeller/plugins/catalog_cache.yaml b/kustomize/overlays/gcp/flyte/config/propeller/plugins/catalog_cache.yaml similarity index 100% rename from kustomize/overlays/gcp/config/propeller/plugins/catalog_cache.yaml rename to kustomize/overlays/gcp/flyte/config/propeller/plugins/catalog_cache.yaml diff --git a/kustomize/overlays/gcp/config/propeller/plugins/k8s.yaml b/kustomize/overlays/gcp/flyte/config/propeller/plugins/k8s.yaml similarity index 100% rename from kustomize/overlays/gcp/config/propeller/plugins/k8s.yaml rename to kustomize/overlays/gcp/flyte/config/propeller/plugins/k8s.yaml diff --git a/kustomize/overlays/gcp/config/propeller/plugins/qubole.yaml b/kustomize/overlays/gcp/flyte/config/propeller/plugins/qubole.yaml similarity index 100% rename from kustomize/overlays/gcp/config/propeller/plugins/qubole.yaml rename to kustomize/overlays/gcp/flyte/config/propeller/plugins/qubole.yaml diff --git a/kustomize/overlays/gcp/config/propeller/plugins/spark.yaml b/kustomize/overlays/gcp/flyte/config/propeller/plugins/spark.yaml similarity index 100% rename from kustomize/overlays/gcp/config/propeller/plugins/spark.yaml rename to kustomize/overlays/gcp/flyte/config/propeller/plugins/spark.yaml diff --git a/kustomize/overlays/gcp/config/propeller/plugins/task_logs.yaml b/kustomize/overlays/gcp/flyte/config/propeller/plugins/task_logs.yaml similarity index 100% rename from kustomize/overlays/gcp/config/propeller/plugins/task_logs.yaml rename to kustomize/overlays/gcp/flyte/config/propeller/plugins/task_logs.yaml diff --git a/kustomize/overlays/gcp/console/service.yaml b/kustomize/overlays/gcp/flyte/console/service.yaml similarity index 100% rename from kustomize/overlays/gcp/console/service.yaml rename to kustomize/overlays/gcp/flyte/console/service.yaml diff --git a/kustomize/overlays/gcp/datacatalog/deployment.yaml b/kustomize/overlays/gcp/flyte/datacatalog/deployment.yaml similarity index 100% rename from kustomize/overlays/gcp/datacatalog/deployment.yaml rename to kustomize/overlays/gcp/flyte/datacatalog/deployment.yaml diff --git a/kustomize/overlays/gcp/datacatalog/service.yaml b/kustomize/overlays/gcp/flyte/datacatalog/service.yaml similarity index 100% rename from kustomize/overlays/gcp/datacatalog/service.yaml rename to kustomize/overlays/gcp/flyte/datacatalog/service.yaml diff --git a/kustomize/overlays/gcp/flyte/kustomization.yaml b/kustomize/overlays/gcp/flyte/kustomization.yaml new file mode 100644 index 0000000000..b369942368 --- /dev/null +++ b/kustomize/overlays/gcp/flyte/kustomization.yaml @@ -0,0 +1,61 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +# Override the namespace +namespace: flyte + +bases: +- ../../../base/single_cluster/complete + +resources: +- datacatalog/service.yaml +- propeller/service.yaml + +patchesStrategicMerge: +- admin/deployment.yaml +- admin/service.yaml +- datacatalog/deployment.yaml +- propeller/deployment.yaml +- console/service.yaml + +# Files are read in alphabetical order. To ensure that we create the namespace first, prefix the file name with "aa". + +configMapGenerator: +- behavior: merge + files: + - ./config/admin/db.yaml + - ./config/admin/remote_data.yaml + - ./config/admin/task_resource_defaults.yaml + - ./config/common/storage.yaml + name: flyte-admin-config +- behavior: merge + files: + - ./config/clusterresource-templates/ac_project-copilot-dataconfig.yaml + - ./config/clusterresource-templates/ad_spark-role.yaml + - ./config/clusterresource-templates/ae_spark-service-account.yaml + - ./config/clusterresource-templates/af_spark-role-binding.yaml + name: clusterresource-template +- behavior: merge + files: + - ./config/propeller/core.yaml + - ./config/propeller/enabled_plugins.yaml + - ./config/propeller/plugins/catalog_cache.yaml + - ./config/propeller/plugins/k8s.yaml + - ./config/propeller/plugins/qubole.yaml + - ./config/propeller/plugins/spark.yaml + - ./config/propeller/plugins/task_logs.yaml + - ./config/common/storage.yaml + name: flyte-propeller-config +- behavior: merge + files: + - ./config/common/storage.yaml + - ./config/datacatalog/db.yaml + name: datacatalog-config + +########### +# Use this to record the password for the Google CloudSQL Database +secretGenerator: +- name: db-pass + behavior: merge + literals: + - pass.txt="yourpassword" diff --git a/kustomize/overlays/gcp/propeller/deployment.yaml b/kustomize/overlays/gcp/flyte/propeller/deployment.yaml similarity index 100% rename from kustomize/overlays/gcp/propeller/deployment.yaml rename to kustomize/overlays/gcp/flyte/propeller/deployment.yaml diff --git a/kustomize/overlays/gcp/propeller/service.yaml b/kustomize/overlays/gcp/flyte/propeller/service.yaml similarity index 100% rename from kustomize/overlays/gcp/propeller/service.yaml rename to kustomize/overlays/gcp/flyte/propeller/service.yaml diff --git a/kustomize/overlays/gcp/kustomization.yaml b/kustomize/overlays/gcp/kustomization.yaml index 86c77fe034..5e4de67b08 100644 --- a/kustomize/overlays/gcp/kustomization.yaml +++ b/kustomize/overlays/gcp/kustomization.yaml @@ -1,94 +1,38 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization -# Override the namespace -namespace: flyte - bases: -- ../../base/single_cluster/complete - # All the resources that make up the deployment -- datacatalog/service.yaml -- propeller/service.yaml - ######## - # Operators to be enabled - ############# - # Dependencies to be configured - # In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc - # This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) - # Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's - # Add node ports for ease of use locally -resources: +- flyte +######## +# Operators to be enabled +############# +# Dependencies to be configured +# In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc +# This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) +# Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's +# Add node ports for ease of use locally - ../../base/operators/spark - ../../base/operators/kfoperators/pytorch -- ./dependencies/cloudsqlproxy - # Optional dependency -- ./dependencies/redis - -patchesStrategicMerge: -- admin/deployment.yaml -- admin/service.yaml -- datacatalog/deployment.yaml -- propeller/deployment.yaml -- console/service.yaml - -# Files are read in alphabetical order. To ensure that we create the namespace first, prefix the file name with "aa". - -configMapGenerator: -- behavior: merge - files: - - ./config/admin/db.yaml - - ./config/admin/remote_data.yaml - - ./config/admin/task_resource_defaults.yaml - - ./config/common/storage.yaml - name: flyte-admin-config -- behavior: merge - files: - - ./config/clusterresource-templates/ac_project-copilot-dataconfig.yaml - - ./config/clusterresource-templates/ad_spark-role.yaml - - ./config/clusterresource-templates/ae_spark-service-account.yaml - - ./config/clusterresource-templates/af_spark-role-binding.yaml - name: clusterresource-template -- behavior: merge - files: - - ./config/propeller/core.yaml - - ./config/propeller/enabled_plugins.yaml - - ./config/propeller/plugins/catalog_cache.yaml - - ./config/propeller/plugins/k8s.yaml - - ./config/propeller/plugins/qubole.yaml - - ./config/propeller/plugins/spark.yaml - - ./config/propeller/plugins/task_logs.yaml - - ./config/common/storage.yaml - name: flyte-propeller-config -- behavior: merge - files: - - ./config/common/storage.yaml - - ./config/datacatalog/db.yaml - name: datacatalog-config - -########### -# Use this to record the password for the Google CloudSQL Database -secretGenerator: -- name: db-pass - behavior: merge - literals: - - pass.txt="yourpassword" +- ../../base/addons/cloudsqlproxy +# Optional dependency +- ../../base/addons/redis # Images that should be used images: - # FlyteAdmin - - name: flyteadmin # match images with this name - newTag: v0.3.29 # override the tag - newName: ghcr.io/lyft/flyteadmin # override the name - # FlyteConsole - - name: flyteconsole # match images with this name - newTag: v0.19.1 # override the tag - newName: ghcr.io/lyft/flyteconsole # override the namep - # Flyte DataCatalog - - name: datacatalog # match images with this name - newTag: v0.2.130 # override the tag - newName: ghcr.io/lyft/datacatalog # override the name - # FlytePropeller - - name: flytepropeller # match images with this name - newTag: v0.5.13 # override the tag - newName: ghcr.io/flyteorg/flytepropeller # override the name +# FlyteAdmin +- name: flyteadmin # match images with this name + newTag: v0.3.29 # override the tag + newName: ghcr.io/lyft/flyteadmin # override the name +# FlyteConsole +- name: flyteconsole # match images with this name + newTag: v0.19.1 # override the tag + newName: ghcr.io/lyft/flyteconsole # override the namep +# Flyte DataCatalog +- name: datacatalog # match images with this name + newTag: v0.2.130 # override the tag + newName: ghcr.io/lyft/datacatalog # override the name +# FlytePropeller +- name: flytepropeller # match images with this name + newTag: v0.5.13 # override the tag + newName: ghcr.io/flyteorg/flytepropeller # override the name diff --git a/kustomize/overlays/sandbox/dependencies/contour_ingress_controller/contour.yaml b/kustomize/overlays/sandbox/dependencies/contour_ingress_controller/contour.yaml deleted file mode 100644 index 499f77f0b0..0000000000 --- a/kustomize/overlays/sandbox/dependencies/contour_ingress_controller/contour.yaml +++ /dev/null @@ -1,1880 +0,0 @@ -# This file is generated from the individual YAML files by generate-deployment.sh. Do not -# edit this file directly but instead edit the source files and re-render. -# -# Generated from: -# examples/contour/00-common.yaml -# examples/contour/01-contour-config.yaml -# examples/contour/01-crds.yaml -# examples/contour/02-job-certgen.yaml -# examples/contour/02-rbac.yaml -# examples/contour/02-role-contour.yaml -# examples/contour/02-service-contour.yaml -# examples/contour/02-service-envoy.yaml -# examples/contour/03-contour.yaml -# examples/contour/03-envoy.yaml -# - ---- -apiVersion: v1 -kind: ServiceAccount -metadata: - name: contour - namespace: projectcontour ---- -apiVersion: v1 -kind: ServiceAccount -metadata: - name: envoy - namespace: projectcontour - ---- -apiVersion: v1 -kind: ConfigMap -metadata: - name: contour - namespace: projectcontour -data: - contour.yaml: | - # - # server: - # determine which XDS Server implementation to utilize in Contour. - # xds-server-type: contour - # - # should contour expect to be running inside a k8s cluster - # incluster: true - # - # path to kubeconfig (if not running inside a k8s cluster) - # kubeconfig: /path/to/.kube/config - # - # disable HTTPProxy permitInsecure field - disablePermitInsecure: false - tls: - # minimum TLS version that Contour will negotiate - # minimum-protocol-version: "1.1" - # Defines the Kubernetes name/namespace matching a secret to use - # as the fallback certificate when requests which don't match the - # SNI defined for a vhost. - fallback-certificate: - # name: fallback-secret-name - # namespace: projectcontour - envoy-client-certificate: - # name: envoy-client-cert-secret-name - # namespace: projectcontour - # The following config shows the defaults for the leader election. - # leaderelection: - # configmap-name: leader-elect - # configmap-namespace: projectcontour - ### Logging options - # Default setting - accesslog-format: envoy - # To enable JSON logging in Envoy - # accesslog-format: json - # The default fields that will be logged are specified below. - # To customise this list, just add or remove entries. - # The canonical list is available at - # https://godoc.org/github.com/projectcontour/contour/internal/envoy#JSONFields - # json-fields: - # - "@timestamp" - # - "authority" - # - "bytes_received" - # - "bytes_sent" - # - "downstream_local_address" - # - "downstream_remote_address" - # - "duration" - # - "method" - # - "path" - # - "protocol" - # - "request_id" - # - "requested_server_name" - # - "response_code" - # - "response_flags" - # - "uber_trace_id" - # - "upstream_cluster" - # - "upstream_host" - # - "upstream_local_address" - # - "upstream_service_time" - # - "user_agent" - # - "x_forwarded_for" - # - # default-http-versions: - # - "HTTP/2" - # - "HTTP/1.1" - # - # The following shows the default proxy timeout settings. - # timeouts: - # request-timeout: infinity - # connection-idle-timeout: 60s - # stream-idle-timeout: 5m - # max-connection-duration: infinity - # connection-shutdown-grace-period: 5s - # - # Envoy cluster settings. - # cluster: - # configure the cluster dns lookup family - # valid options are: auto (default), v4, v6 - # dns-lookup-family: auto - ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - controller-gen.kubebuilder.io/version: v0.2.9 - creationTimestamp: null - name: extensionservices.projectcontour.io -spec: - preserveUnknownFields: false - group: projectcontour.io - names: - kind: ExtensionService - listKind: ExtensionServiceList - plural: extensionservices - shortNames: - - extensionservice - - extensionservices - singular: extensionservice - scope: Namespaced - versions: - - name: v1alpha1 - schema: - openAPIV3Schema: - description: ExtensionService is the schema for the Contour extension services API. An ExtensionService resource binds a network service to the Contour API so that Contour API features can be implemented by collaborating components. - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: ExtensionServiceSpec defines the desired state of an ExtensionService resource. - properties: - loadBalancerPolicy: - description: The policy for load balancing GRPC service requests. Note that the `Cookie` load balancing strategy cannot be used here. - properties: - strategy: - description: Strategy specifies the policy used to balance requests across the pool of backend pods. Valid policy names are `Random`, `RoundRobin`, `WeightedLeastRequest`, `Random` and `Cookie`. If an unknown strategy name is specified or no policy is supplied, the default `RoundRobin` policy is used. - type: string - type: object - protocol: - description: Protocol may be used to specify (or override) the protocol used to reach this Service. Values may be tls, h2, h2c. If omitted, protocol-selection falls back on Service annotations. - enum: - - h2 - - h2c - type: string - protocolVersion: - description: This field sets the version of the GRPC protocol that Envoy uses to send requests to the extension service. Since Contour always uses the v3 Envoy API, this is currently fixed at "v3". However, other protocol options will be available in future. - enum: - - v3 - type: string - services: - description: Services specifies the set of Kubernetes Service resources that receive GRPC extension API requests. If no weights are specified for any of the entries in this array, traffic will be spread evenly across all the services. Otherwise, traffic is balanced proportionally to the Weight field in each entry. - items: - description: ExtensionServiceTarget defines an Kubernetes Service to target with extension service traffic. - properties: - name: - description: Name is the name of Kubernetes service that will accept service traffic. - type: string - port: - description: Port (defined as Integer) to proxy traffic to since a service can have multiple defined. - exclusiveMaximum: true - maximum: 65536 - minimum: 1 - type: integer - weight: - description: Weight defines proportion of traffic to balance to the Kubernetes Service. - format: int32 - type: integer - required: - - name - - port - type: object - minItems: 1 - type: array - timeoutPolicy: - description: The timeout policy for requests to the services. - properties: - idle: - description: Timeout after which, if there are no active requests for this route, the connection between Envoy and the backend or Envoy and the external client will be closed. If not specified, there is no per-route idle timeout, though a connection manager-wide stream_idle_timeout default of 5m still applies. - pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ - type: string - response: - description: Timeout for receiving a response from the server after processing a request from client. If not supplied, Envoy's default value of 15s applies. - pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ - type: string - type: object - validation: - description: UpstreamValidation defines how to verify the backend service's certificate - properties: - caSecret: - description: Name of the Kubernetes secret be used to validate the certificate presented by the backend - type: string - subjectName: - description: Key which is expected to be present in the 'subjectAltName' of the presented certificate - type: string - required: - - caSecret - - subjectName - type: object - required: - - services - type: object - status: - description: ExtensionServiceStatus defines the observed state of an ExtensionService resource. - properties: - conditions: - description: "Conditions contains the current status of the ExtensionService resource. \n Contour will update a single condition, `Valid`, that is in normal-true polarity. \n Contour will not modify any other Conditions set in this block, in case some other controller wants to add a Condition." - items: - description: "DetailedCondition is an extension of the normal Kubernetes conditions, with two extra fields to hold sub-conditions, which provide more detailed reasons for the state (True or False) of the condition. \n `errors` holds information about sub-conditions which are fatal to that condition and render its state False. \n `warnings` holds information about sub-conditions which are not fatal to that condition and do not force the state to be False. \n Remember that Conditions have a type, a status, and a reason. \n The type is the type of the condition, the most important one in this CRD set is `Valid`. `Valid` is a positive-polarity condition: when it is `status: true` there are no problems. \n In more detail, `status: true` means that the object is has been ingested into Contour with no errors. `warnings` may still be present, and will be indicated in the Reason field. There must be zero entries in the `errors` slice in this case. \n `Valid`, `status: false` means that the object has had one or more fatal errors during processing into Contour. The details of the errors will be present under the `errors` field. There must be at least one error in the `errors` slice if `status` is `false`. \n For DetailedConditions of types other than `Valid`, the Condition must be in the negative polarity. When they have `status` `true`, there is an error. There must be at least one entry in the `errors` Subcondition slice. When they have `status` `false`, there are no serious errors, and there must be zero entries in the `errors` slice. In either case, there may be entries in the `warnings` slice. \n Regardless of the polarity, the `reason` and `message` fields must be updated with either the detail of the reason (if there is one and only one entry in total across both the `errors` and `warnings` slices), or `MultipleReasons` if there is more than one entry." - properties: - errors: - description: "Errors contains a slice of relevant error subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a error), and disappear when not relevant. An empty slice here indicates no errors." - items: - description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." - properties: - message: - description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." - maxLength: 32768 - type: string - reason: - description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." - maxLength: 1024 - minLength: 1 - pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ - type: string - status: - description: Status of the condition, one of True, False, Unknown. - enum: - - "True" - - "False" - - Unknown - type: string - type: - description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" - maxLength: 316 - pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ - type: string - required: - - message - - reason - - status - - type - type: object - type: array - lastTransitionTime: - description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. - format: date-time - type: string - message: - description: message is a human readable message indicating details about the transition. This may be an empty string. - maxLength: 32768 - type: string - observedGeneration: - description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. - format: int64 - minimum: 0 - type: integer - reason: - description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. - maxLength: 1024 - minLength: 1 - pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ - type: string - status: - description: status of the condition, one of True, False, Unknown. - enum: - - "True" - - "False" - - Unknown - type: string - type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) - maxLength: 316 - pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ - type: string - warnings: - description: "Warnings contains a slice of relevant warning subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a warning), and disappear when not relevant. An empty slice here indicates no warnings." - items: - description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." - properties: - message: - description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." - maxLength: 32768 - type: string - reason: - description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." - maxLength: 1024 - minLength: 1 - pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ - type: string - status: - description: Status of the condition, one of True, False, Unknown. - enum: - - "True" - - "False" - - Unknown - type: string - type: - description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" - maxLength: 316 - pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ - type: string - required: - - message - - reason - - status - - type - type: object - type: array - required: - - lastTransitionTime - - message - - reason - - status - - type - type: object - type: array - x-kubernetes-list-map-keys: - - type - x-kubernetes-list-type: map - type: object - type: object - served: true - storage: true - subresources: - status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - controller-gen.kubebuilder.io/version: v0.2.9 - creationTimestamp: null - name: httpproxies.projectcontour.io -spec: - preserveUnknownFields: false - group: projectcontour.io - names: - kind: HTTPProxy - listKind: HTTPProxyList - plural: httpproxies - shortNames: - - proxy - - proxies - singular: httpproxy - scope: Namespaced - versions: - - additionalPrinterColumns: - - description: Fully qualified domain name - jsonPath: .spec.virtualhost.fqdn - name: FQDN - type: string - - description: Secret with TLS credentials - jsonPath: .spec.virtualhost.tls.secretName - name: TLS Secret - type: string - - description: The current status of the HTTPProxy - jsonPath: .status.currentStatus - name: Status - type: string - - description: Description of the current status - jsonPath: .status.description - name: Status Description - type: string - name: v1 - schema: - openAPIV3Schema: - description: HTTPProxy is an Ingress CRD specification. - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: HTTPProxySpec defines the spec of the CRD. - properties: - includes: - description: Includes allow for specific routing configuration to be included from another HTTPProxy, possibly in another namespace. - items: - description: Include describes a set of policies that can be applied to an HTTPProxy in a namespace. - properties: - conditions: - description: 'Conditions are a set of rules that are applied to included HTTPProxies. In effect, they are added onto the Conditions of included HTTPProxy Route structs. When applied, they are merged using AND, with one exception: There can be only one Prefix MatchCondition per Conditions slice. More than one Prefix, or contradictory Conditions, will make the include invalid.' - items: - description: MatchCondition are a general holder for matching rules for HTTPProxies. One of Prefix or Header must be provided. - properties: - header: - description: Header specifies the header condition to match. - properties: - contains: - description: Contains specifies a substring that must be present in the header value. - type: string - exact: - description: Exact specifies a string that the header value must be equal to. - type: string - name: - description: Name is the name of the header to match against. Name is required. Header names are case insensitive. - type: string - notcontains: - description: NotContains specifies a substring that must not be present in the header value. - type: string - notexact: - description: NoExact specifies a string that the header value must not be equal to. The condition is true if the header has any other value. - type: string - present: - description: Present specifies that condition is true when the named header is present, regardless of its value. Note that setting Present to false does not make the condition true if the named header is absent. - type: boolean - required: - - name - type: object - prefix: - description: Prefix defines a prefix match for a request. - type: string - type: object - type: array - name: - description: Name of the HTTPProxy - type: string - namespace: - description: Namespace of the HTTPProxy to include. Defaults to the current namespace if not supplied. - type: string - required: - - name - type: object - type: array - routes: - description: Routes are the ingress routes. If TCPProxy is present, Routes is ignored. - items: - description: Route contains the set of routes for a virtual host. - properties: - authPolicy: - description: AuthPolicy updates the authorization policy that was set on the root HTTPProxy object for client requests that match this route. - properties: - context: - additionalProperties: - type: string - description: Context is a set of key/value pairs that are sent to the authentication server in the check request. If a context is provided at an enclosing scope, the entries are merged such that the inner scope overrides matching keys from the outer scope. - type: object - disabled: - description: When true, this field disables client request authentication for the scope of the policy. - type: boolean - type: object - conditions: - description: 'Conditions are a set of rules that are applied to a Route. When applied, they are merged using AND, with one exception: There can be only one Prefix MatchCondition per Conditions slice. More than one Prefix, or contradictory Conditions, will make the route invalid.' - items: - description: MatchCondition are a general holder for matching rules for HTTPProxies. One of Prefix or Header must be provided. - properties: - header: - description: Header specifies the header condition to match. - properties: - contains: - description: Contains specifies a substring that must be present in the header value. - type: string - exact: - description: Exact specifies a string that the header value must be equal to. - type: string - name: - description: Name is the name of the header to match against. Name is required. Header names are case insensitive. - type: string - notcontains: - description: NotContains specifies a substring that must not be present in the header value. - type: string - notexact: - description: NoExact specifies a string that the header value must not be equal to. The condition is true if the header has any other value. - type: string - present: - description: Present specifies that condition is true when the named header is present, regardless of its value. Note that setting Present to false does not make the condition true if the named header is absent. - type: boolean - required: - - name - type: object - prefix: - description: Prefix defines a prefix match for a request. - type: string - type: object - type: array - enableWebsockets: - description: Enables websocket support for the route. - type: boolean - healthCheckPolicy: - description: The health check policy for this route. - properties: - healthyThresholdCount: - description: The number of healthy health checks required before a host is marked healthy - format: int64 - minimum: 0 - type: integer - host: - description: The value of the host header in the HTTP health check request. If left empty (default value), the name "contour-envoy-healthcheck" will be used. - type: string - intervalSeconds: - description: The interval (seconds) between health checks - format: int64 - type: integer - path: - description: HTTP endpoint used to perform health checks on upstream service - type: string - timeoutSeconds: - description: The time to wait (seconds) for a health check response - format: int64 - type: integer - unhealthyThresholdCount: - description: The number of unhealthy health checks required before a host is marked unhealthy - format: int64 - minimum: 0 - type: integer - required: - - path - type: object - loadBalancerPolicy: - description: The load balancing policy for this route. - properties: - strategy: - description: Strategy specifies the policy used to balance requests across the pool of backend pods. Valid policy names are `Random`, `RoundRobin`, `WeightedLeastRequest`, `Random` and `Cookie`. If an unknown strategy name is specified or no policy is supplied, the default `RoundRobin` policy is used. - type: string - type: object - pathRewritePolicy: - description: The policy for rewriting the path of the request URL after the request has been routed to a Service. - properties: - replacePrefix: - description: ReplacePrefix describes how the path prefix should be replaced. - items: - description: ReplacePrefix describes a path prefix replacement. - properties: - prefix: - description: "Prefix specifies the URL path prefix to be replaced. \n If Prefix is specified, it must exactly match the MatchCondition prefix that is rendered by the chain of including HTTPProxies and only that path prefix will be replaced by Replacement. This allows HTTPProxies that are included through multiple roots to only replace specific path prefixes, leaving others unmodified. \n If Prefix is not specified, all routing prefixes rendered by the include chain will be replaced." - minLength: 1 - type: string - replacement: - description: Replacement is the string that the routing path prefix will be replaced with. This must not be empty. - minLength: 1 - type: string - required: - - replacement - type: object - type: array - type: object - permitInsecure: - description: Allow this path to respond to insecure requests over HTTP which are normally not permitted when a `virtualhost.tls` block is present. - type: boolean - requestHeadersPolicy: - description: The policy for managing request headers during proxying. - properties: - remove: - description: Remove specifies a list of HTTP header names to remove. - items: - type: string - type: array - set: - description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. - items: - description: HeaderValue represents a header name/value pair - properties: - name: - description: Name represents a key of a header - minLength: 1 - type: string - value: - description: Value represents the value of a header specified by a key - minLength: 1 - type: string - required: - - name - - value - type: object - type: array - type: object - responseHeadersPolicy: - description: The policy for managing response headers during proxying. Rewriting the 'Host' header is not supported. - properties: - remove: - description: Remove specifies a list of HTTP header names to remove. - items: - type: string - type: array - set: - description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. - items: - description: HeaderValue represents a header name/value pair - properties: - name: - description: Name represents a key of a header - minLength: 1 - type: string - value: - description: Value represents the value of a header specified by a key - minLength: 1 - type: string - required: - - name - - value - type: object - type: array - type: object - retryPolicy: - description: The retry policy for this route. - properties: - count: - description: NumRetries is maximum allowed number of retries. If not supplied, the number of retries is one. - format: int64 - minimum: 0 - type: integer - perTryTimeout: - description: PerTryTimeout specifies the timeout per retry attempt. Ignored if NumRetries is not supplied. - pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ - type: string - retriableStatusCodes: - description: "RetriableStatusCodes specifies the HTTP status codes that should be retried. \n This field is only respected when you include `retriable-status-codes` in the `RetryOn` field." - items: - format: int32 - type: integer - type: array - retryOn: - description: "RetryOn specifies the conditions on which to retry a request. \n Supported [HTTP conditions](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/router_filter#x-envoy-retry-on): \n - `5xx` - `gateway-error` - `reset` - `connect-failure` - `retriable-4xx` - `refused-stream` - `retriable-status-codes` - `retriable-headers` \n Supported [gRPC conditions](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/router_filter#x-envoy-retry-grpc-on): \n - `cancelled` - `deadline-exceeded` - `internal` - `resource-exhausted` - `unavailable`" - items: - description: RetryOn is a string type alias with validation to ensure that the value is valid. - enum: - - 5xx - - gateway-error - - reset - - connect-failure - - retriable-4xx - - refused-stream - - retriable-status-codes - - retriable-headers - - cancelled - - deadline-exceeded - - internal - - resource-exhausted - - unavailable - type: string - type: array - type: object - services: - description: Services are the services to proxy traffic. - items: - description: Service defines an Kubernetes Service to proxy traffic. - properties: - mirror: - description: If Mirror is true the Service will receive a read only mirror of the traffic for this route. - type: boolean - name: - description: Name is the name of Kubernetes service to proxy traffic. Names defined here will be used to look up corresponding endpoints which contain the ips to route. - type: string - port: - description: Port (defined as Integer) to proxy traffic to since a service can have multiple defined. - exclusiveMaximum: true - maximum: 65536 - minimum: 1 - type: integer - protocol: - description: Protocol may be used to specify (or override) the protocol used to reach this Service. Values may be tls, h2, h2c. If omitted, protocol-selection falls back on Service annotations. - enum: - - h2 - - h2c - - tls - type: string - requestHeadersPolicy: - description: The policy for managing request headers during proxying. Rewriting the 'Host' header is not supported. - properties: - remove: - description: Remove specifies a list of HTTP header names to remove. - items: - type: string - type: array - set: - description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. - items: - description: HeaderValue represents a header name/value pair - properties: - name: - description: Name represents a key of a header - minLength: 1 - type: string - value: - description: Value represents the value of a header specified by a key - minLength: 1 - type: string - required: - - name - - value - type: object - type: array - type: object - responseHeadersPolicy: - description: The policy for managing response headers during proxying. Rewriting the 'Host' header is not supported. - properties: - remove: - description: Remove specifies a list of HTTP header names to remove. - items: - type: string - type: array - set: - description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. - items: - description: HeaderValue represents a header name/value pair - properties: - name: - description: Name represents a key of a header - minLength: 1 - type: string - value: - description: Value represents the value of a header specified by a key - minLength: 1 - type: string - required: - - name - - value - type: object - type: array - type: object - validation: - description: UpstreamValidation defines how to verify the backend service's certificate - properties: - caSecret: - description: Name of the Kubernetes secret be used to validate the certificate presented by the backend - type: string - subjectName: - description: Key which is expected to be present in the 'subjectAltName' of the presented certificate - type: string - required: - - caSecret - - subjectName - type: object - weight: - description: Weight defines percentage of traffic to balance traffic - format: int64 - minimum: 0 - type: integer - required: - - name - - port - type: object - minItems: 1 - type: array - timeoutPolicy: - description: The timeout policy for this route. - properties: - idle: - description: Timeout after which, if there are no active requests for this route, the connection between Envoy and the backend or Envoy and the external client will be closed. If not specified, there is no per-route idle timeout, though a connection manager-wide stream_idle_timeout default of 5m still applies. - pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ - type: string - response: - description: Timeout for receiving a response from the server after processing a request from client. If not supplied, Envoy's default value of 15s applies. - pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ - type: string - type: object - required: - - services - type: object - type: array - tcpproxy: - description: TCPProxy holds TCP proxy information. - properties: - healthCheckPolicy: - description: The health check policy for this tcp proxy - properties: - healthyThresholdCount: - description: The number of healthy health checks required before a host is marked healthy - format: int32 - type: integer - intervalSeconds: - description: The interval (seconds) between health checks - format: int64 - type: integer - timeoutSeconds: - description: The time to wait (seconds) for a health check response - format: int64 - type: integer - unhealthyThresholdCount: - description: The number of unhealthy health checks required before a host is marked unhealthy - format: int32 - type: integer - type: object - include: - description: Include specifies that this tcpproxy should be delegated to another HTTPProxy. - properties: - name: - description: Name of the child HTTPProxy - type: string - namespace: - description: Namespace of the HTTPProxy to include. Defaults to the current namespace if not supplied. - type: string - required: - - name - type: object - includes: - description: "IncludesDeprecated allow for specific routing configuration to be appended to another HTTPProxy in another namespace. \n Exists due to a mistake when developing HTTPProxy and the field was marked plural when it should have been singular. This field should stay to not break backwards compatibility to v1 users." - properties: - name: - description: Name of the child HTTPProxy - type: string - namespace: - description: Namespace of the HTTPProxy to include. Defaults to the current namespace if not supplied. - type: string - required: - - name - type: object - loadBalancerPolicy: - description: The load balancing policy for the backend services. - properties: - strategy: - description: Strategy specifies the policy used to balance requests across the pool of backend pods. Valid policy names are `Random`, `RoundRobin`, `WeightedLeastRequest`, `Random` and `Cookie`. If an unknown strategy name is specified or no policy is supplied, the default `RoundRobin` policy is used. - type: string - type: object - services: - description: Services are the services to proxy traffic - items: - description: Service defines an Kubernetes Service to proxy traffic. - properties: - mirror: - description: If Mirror is true the Service will receive a read only mirror of the traffic for this route. - type: boolean - name: - description: Name is the name of Kubernetes service to proxy traffic. Names defined here will be used to look up corresponding endpoints which contain the ips to route. - type: string - port: - description: Port (defined as Integer) to proxy traffic to since a service can have multiple defined. - exclusiveMaximum: true - maximum: 65536 - minimum: 1 - type: integer - protocol: - description: Protocol may be used to specify (or override) the protocol used to reach this Service. Values may be tls, h2, h2c. If omitted, protocol-selection falls back on Service annotations. - enum: - - h2 - - h2c - - tls - type: string - requestHeadersPolicy: - description: The policy for managing request headers during proxying. Rewriting the 'Host' header is not supported. - properties: - remove: - description: Remove specifies a list of HTTP header names to remove. - items: - type: string - type: array - set: - description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. - items: - description: HeaderValue represents a header name/value pair - properties: - name: - description: Name represents a key of a header - minLength: 1 - type: string - value: - description: Value represents the value of a header specified by a key - minLength: 1 - type: string - required: - - name - - value - type: object - type: array - type: object - responseHeadersPolicy: - description: The policy for managing response headers during proxying. Rewriting the 'Host' header is not supported. - properties: - remove: - description: Remove specifies a list of HTTP header names to remove. - items: - type: string - type: array - set: - description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. - items: - description: HeaderValue represents a header name/value pair - properties: - name: - description: Name represents a key of a header - minLength: 1 - type: string - value: - description: Value represents the value of a header specified by a key - minLength: 1 - type: string - required: - - name - - value - type: object - type: array - type: object - validation: - description: UpstreamValidation defines how to verify the backend service's certificate - properties: - caSecret: - description: Name of the Kubernetes secret be used to validate the certificate presented by the backend - type: string - subjectName: - description: Key which is expected to be present in the 'subjectAltName' of the presented certificate - type: string - required: - - caSecret - - subjectName - type: object - weight: - description: Weight defines percentage of traffic to balance traffic - format: int64 - minimum: 0 - type: integer - required: - - name - - port - type: object - type: array - type: object - virtualhost: - description: Virtualhost appears at most once. If it is present, the object is considered to be a "root" HTTPProxy. - properties: - authorization: - description: This field configures an extension service to perform authorization for this virtual host. Authorization can only be configured on virtual hosts that have TLS enabled. If the TLS configuration requires client certificate /validation, the client certificate is always included in the authentication check request. - properties: - authPolicy: - description: AuthPolicy sets a default authorization policy for client requests. This policy will be used unless overridden by individual routes. - properties: - context: - additionalProperties: - type: string - description: Context is a set of key/value pairs that are sent to the authentication server in the check request. If a context is provided at an enclosing scope, the entries are merged such that the inner scope overrides matching keys from the outer scope. - type: object - disabled: - description: When true, this field disables client request authentication for the scope of the policy. - type: boolean - type: object - extensionRef: - description: ExtensionServiceRef specifies the extension resource that will authorize client requests. - properties: - apiVersion: - description: API version of the referent. If this field is not specified, the default "projectcontour.io/v1alpha1" will be used - minLength: 1 - type: string - name: - description: "Name of the referent. \n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names" - minLength: 1 - type: string - namespace: - description: "Namespace of the referent. If this field is not specifies, the namespace of the resource that targets the referent will be used. \n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/" - minLength: 1 - type: string - type: object - failOpen: - description: If FailOpen is true, the client request is forwarded to the upstream service even if the authorization server fails to respond. This field should not be set in most cases. It is intended for use only while migrating applications from internal authorization to Contour external authorization. - type: boolean - responseTimeout: - description: ResponseTimeout configures maximum time to wait for a check response from the authorization server. Timeout durations are expressed in the Go [Duration format](https://godoc.org/time#ParseDuration). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". The string "infinity" is also a valid input and specifies no timeout. - pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ - type: string - required: - - extensionRef - type: object - corsPolicy: - description: Specifies the cross-origin policy to apply to the VirtualHost. - properties: - allowCredentials: - description: Specifies whether the resource allows credentials. - type: boolean - allowHeaders: - description: AllowHeaders specifies the content for the *access-control-allow-headers* header. - items: - description: CORSHeaderValue specifies the value of the string headers returned by a cross-domain request. - pattern: ^[a-zA-Z0-9!#$%&'*+.^_`|~-]+$ - type: string - type: array - allowMethods: - description: AllowMethods specifies the content for the *access-control-allow-methods* header. - items: - description: CORSHeaderValue specifies the value of the string headers returned by a cross-domain request. - pattern: ^[a-zA-Z0-9!#$%&'*+.^_`|~-]+$ - type: string - type: array - allowOrigin: - description: AllowOrigin specifies the origins that will be allowed to do CORS requests. "*" means allow any origin. - items: - type: string - type: array - exposeHeaders: - description: ExposeHeaders Specifies the content for the *access-control-expose-headers* header. - items: - description: CORSHeaderValue specifies the value of the string headers returned by a cross-domain request. - pattern: ^[a-zA-Z0-9!#$%&'*+.^_`|~-]+$ - type: string - type: array - maxAge: - description: MaxAge indicates for how long the results of a preflight request can be cached. MaxAge durations are expressed in the Go [Duration format](https://godoc.org/time#ParseDuration). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". Only positive values are allowed while 0 disables the cache requiring a preflight OPTIONS check for all cross-origin requests. - type: string - required: - - allowMethods - - allowOrigin - type: object - fqdn: - description: The fully qualified domain name of the root of the ingress tree all leaves of the DAG rooted at this object relate to the fqdn. - type: string - tls: - description: If present the fields describes TLS properties of the virtual host. The SNI names that will be matched on are described in fqdn, the tls.secretName secret must contain a certificate that itself contains a name that matches the FQDN. - properties: - clientValidation: - description: "ClientValidation defines how to verify the client certificate when an external client establishes a TLS connection to Envoy. \n This setting: \n 1. Enables TLS client certificate validation. 2. Requires clients to present a TLS certificate (i.e. not optional validation). 3. Specifies how the client certificate will be validated." - properties: - caSecret: - description: Name of a Kubernetes secret that contains a CA certificate bundle. The client certificate must validate against the certificates in the bundle. - minLength: 1 - type: string - required: - - caSecret - type: object - enableFallbackCertificate: - description: EnableFallbackCertificate defines if the vhost should allow a default certificate to be applied which handles all requests which don't match the SNI defined in this vhost. - type: boolean - minimumProtocolVersion: - description: Minimum TLS version this vhost should negotiate - type: string - passthrough: - description: Passthrough defines whether the encrypted TLS handshake will be passed through to the backing cluster. Either Passthrough or SecretName must be specified, but not both. - type: boolean - secretName: - description: SecretName is the name of a TLS secret in the current namespace. Either SecretName or Passthrough must be specified, but not both. If specified, the named secret must contain a matching certificate for the virtual host's FQDN. - type: string - type: object - required: - - fqdn - type: object - type: object - status: - description: Status is a container for computed information about the HTTPProxy. - properties: - conditions: - description: "Conditions contains information about the current status of the HTTPProxy, in an upstream-friendly container. \n Contour will update a single condition, `Valid`, that is in normal-true polarity. That is, when `currentStatus` is `valid`, the `Valid` condition will be `status: true`, and vice versa. \n Contour will leave untouched any other Conditions set in this block, in case some other controller wants to add a Condition. \n If you are another controller owner and wish to add a condition, you *should* namespace your condition with a label, like `controller.domain.com/ConditionName`." - items: - description: "DetailedCondition is an extension of the normal Kubernetes conditions, with two extra fields to hold sub-conditions, which provide more detailed reasons for the state (True or False) of the condition. \n `errors` holds information about sub-conditions which are fatal to that condition and render its state False. \n `warnings` holds information about sub-conditions which are not fatal to that condition and do not force the state to be False. \n Remember that Conditions have a type, a status, and a reason. \n The type is the type of the condition, the most important one in this CRD set is `Valid`. `Valid` is a positive-polarity condition: when it is `status: true` there are no problems. \n In more detail, `status: true` means that the object is has been ingested into Contour with no errors. `warnings` may still be present, and will be indicated in the Reason field. There must be zero entries in the `errors` slice in this case. \n `Valid`, `status: false` means that the object has had one or more fatal errors during processing into Contour. The details of the errors will be present under the `errors` field. There must be at least one error in the `errors` slice if `status` is `false`. \n For DetailedConditions of types other than `Valid`, the Condition must be in the negative polarity. When they have `status` `true`, there is an error. There must be at least one entry in the `errors` Subcondition slice. When they have `status` `false`, there are no serious errors, and there must be zero entries in the `errors` slice. In either case, there may be entries in the `warnings` slice. \n Regardless of the polarity, the `reason` and `message` fields must be updated with either the detail of the reason (if there is one and only one entry in total across both the `errors` and `warnings` slices), or `MultipleReasons` if there is more than one entry." - properties: - errors: - description: "Errors contains a slice of relevant error subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a error), and disappear when not relevant. An empty slice here indicates no errors." - items: - description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." - properties: - message: - description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." - maxLength: 32768 - type: string - reason: - description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." - maxLength: 1024 - minLength: 1 - pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ - type: string - status: - description: Status of the condition, one of True, False, Unknown. - enum: - - "True" - - "False" - - Unknown - type: string - type: - description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" - maxLength: 316 - pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ - type: string - required: - - message - - reason - - status - - type - type: object - type: array - lastTransitionTime: - description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. - format: date-time - type: string - message: - description: message is a human readable message indicating details about the transition. This may be an empty string. - maxLength: 32768 - type: string - observedGeneration: - description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. - format: int64 - minimum: 0 - type: integer - reason: - description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. - maxLength: 1024 - minLength: 1 - pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ - type: string - status: - description: status of the condition, one of True, False, Unknown. - enum: - - "True" - - "False" - - Unknown - type: string - type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) - maxLength: 316 - pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ - type: string - warnings: - description: "Warnings contains a slice of relevant warning subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a warning), and disappear when not relevant. An empty slice here indicates no warnings." - items: - description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." - properties: - message: - description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." - maxLength: 32768 - type: string - reason: - description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." - maxLength: 1024 - minLength: 1 - pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ - type: string - status: - description: Status of the condition, one of True, False, Unknown. - enum: - - "True" - - "False" - - Unknown - type: string - type: - description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" - maxLength: 316 - pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ - type: string - required: - - message - - reason - - status - - type - type: object - type: array - required: - - lastTransitionTime - - message - - reason - - status - - type - type: object - type: array - x-kubernetes-list-map-keys: - - type - x-kubernetes-list-type: map - currentStatus: - type: string - description: - type: string - loadBalancer: - description: LoadBalancer contains the current status of the load balancer. - properties: - ingress: - description: Ingress is a list containing ingress points for the load-balancer. Traffic intended for the service should be sent to these ingress points. - items: - description: 'LoadBalancerIngress represents the status of a load-balancer ingress point: traffic intended for the service should be sent to an ingress point.' - properties: - hostname: - description: Hostname is set for load-balancer ingress points that are DNS based (typically AWS load-balancers) - type: string - ip: - description: IP is set for load-balancer ingress points that are IP based (typically GCE or OpenStack load-balancers) - type: string - type: object - type: array - type: object - type: object - required: - - metadata - - spec - type: object - served: true - storage: true - subresources: - status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - controller-gen.kubebuilder.io/version: v0.2.9 - creationTimestamp: null - name: tlscertificatedelegations.projectcontour.io -spec: - preserveUnknownFields: false - group: projectcontour.io - names: - kind: TLSCertificateDelegation - listKind: TLSCertificateDelegationList - plural: tlscertificatedelegations - shortNames: - - tlscerts - singular: tlscertificatedelegation - scope: Namespaced - versions: - - name: v1 - schema: - openAPIV3Schema: - description: TLSCertificateDelegation is an TLS Certificate Delegation CRD specification. See design/tls-certificate-delegation.md for details. - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: TLSCertificateDelegationSpec defines the spec of the CRD - properties: - delegations: - items: - description: CertificateDelegation maps the authority to reference a secret in the current namespace to a set of namespaces. - properties: - secretName: - description: required, the name of a secret in the current namespace. - type: string - targetNamespaces: - description: required, the namespaces the authority to reference the the secret will be delegated to. If TargetNamespaces is nil or empty, the CertificateDelegation' is ignored. If the TargetNamespace list contains the character, "*" the secret will be delegated to all namespaces. - items: - type: string - type: array - required: - - secretName - - targetNamespaces - type: object - type: array - required: - - delegations - type: object - status: - description: TLSCertificateDelegationStatus allows for the status of the delegation to be presented to the user. - properties: - conditions: - description: "Conditions contains information about the current status of the HTTPProxy, in an upstream-friendly container. \n Contour will update a single condition, `Valid`, that is in normal-true polarity. That is, when `currentStatus` is `valid`, the `Valid` condition will be `status: true`, and vice versa. \n Contour will leave untouched any other Conditions set in this block, in case some other controller wants to add a Condition. \n If you are another controller owner and wish to add a condition, you *should* namespace your condition with a label, like `controller.domain.com\\ConditionName`." - items: - description: "DetailedCondition is an extension of the normal Kubernetes conditions, with two extra fields to hold sub-conditions, which provide more detailed reasons for the state (True or False) of the condition. \n `errors` holds information about sub-conditions which are fatal to that condition and render its state False. \n `warnings` holds information about sub-conditions which are not fatal to that condition and do not force the state to be False. \n Remember that Conditions have a type, a status, and a reason. \n The type is the type of the condition, the most important one in this CRD set is `Valid`. `Valid` is a positive-polarity condition: when it is `status: true` there are no problems. \n In more detail, `status: true` means that the object is has been ingested into Contour with no errors. `warnings` may still be present, and will be indicated in the Reason field. There must be zero entries in the `errors` slice in this case. \n `Valid`, `status: false` means that the object has had one or more fatal errors during processing into Contour. The details of the errors will be present under the `errors` field. There must be at least one error in the `errors` slice if `status` is `false`. \n For DetailedConditions of types other than `Valid`, the Condition must be in the negative polarity. When they have `status` `true`, there is an error. There must be at least one entry in the `errors` Subcondition slice. When they have `status` `false`, there are no serious errors, and there must be zero entries in the `errors` slice. In either case, there may be entries in the `warnings` slice. \n Regardless of the polarity, the `reason` and `message` fields must be updated with either the detail of the reason (if there is one and only one entry in total across both the `errors` and `warnings` slices), or `MultipleReasons` if there is more than one entry." - properties: - errors: - description: "Errors contains a slice of relevant error subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a error), and disappear when not relevant. An empty slice here indicates no errors." - items: - description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." - properties: - message: - description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." - maxLength: 32768 - type: string - reason: - description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." - maxLength: 1024 - minLength: 1 - pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ - type: string - status: - description: Status of the condition, one of True, False, Unknown. - enum: - - "True" - - "False" - - Unknown - type: string - type: - description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" - maxLength: 316 - pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ - type: string - required: - - message - - reason - - status - - type - type: object - type: array - lastTransitionTime: - description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. - format: date-time - type: string - message: - description: message is a human readable message indicating details about the transition. This may be an empty string. - maxLength: 32768 - type: string - observedGeneration: - description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. - format: int64 - minimum: 0 - type: integer - reason: - description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. - maxLength: 1024 - minLength: 1 - pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ - type: string - status: - description: status of the condition, one of True, False, Unknown. - enum: - - "True" - - "False" - - Unknown - type: string - type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) - maxLength: 316 - pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ - type: string - warnings: - description: "Warnings contains a slice of relevant warning subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a warning), and disappear when not relevant. An empty slice here indicates no warnings." - items: - description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." - properties: - message: - description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." - maxLength: 32768 - type: string - reason: - description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." - maxLength: 1024 - minLength: 1 - pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ - type: string - status: - description: Status of the condition, one of True, False, Unknown. - enum: - - "True" - - "False" - - Unknown - type: string - type: - description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" - maxLength: 316 - pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ - type: string - required: - - message - - reason - - status - - type - type: object - type: array - required: - - lastTransitionTime - - message - - reason - - status - - type - type: object - type: array - x-kubernetes-list-map-keys: - - type - x-kubernetes-list-type: map - type: object - required: - - metadata - - spec - type: object - served: true - storage: true - subresources: - status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] - ---- -apiVersion: v1 -kind: ServiceAccount -metadata: - name: contour-certgen - namespace: projectcontour ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: RoleBinding -metadata: - name: contour - namespace: projectcontour -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: Role - name: contour-certgen -subjects: -- kind: ServiceAccount - name: contour-certgen - namespace: projectcontour ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: Role -metadata: - name: contour-certgen - namespace: projectcontour -rules: -- apiGroups: - - "" - resources: - - secrets - verbs: - - create - - update ---- -apiVersion: batch/v1 -kind: Job -metadata: - name: contour-certgen-v1.11.0 - namespace: projectcontour -spec: - ttlSecondsAfterFinished: 0 - template: - metadata: - labels: - app: "contour-certgen" - spec: - containers: - - name: contour - image: docker.io/projectcontour/contour:v1.11.0 - imagePullPolicy: Always - command: - - contour - - certgen - - --kube - - --incluster - - --overwrite - - --secrets-format=compact - - --namespace=$(CONTOUR_NAMESPACE) - env: - - name: CONTOUR_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - restartPolicy: Never - serviceAccountName: contour-certgen - securityContext: - runAsNonRoot: true - runAsUser: 65534 - runAsGroup: 65534 - parallelism: 1 - completions: 1 - backoffLimit: 1 - ---- - -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRoleBinding -metadata: - name: contour -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: ClusterRole - name: contour -subjects: -- kind: ServiceAccount - name: contour - namespace: projectcontour - -# The following ClusterRole is generated from kubebuilder RBAC tags by -# generate-rbac.sh. Do not edit this file directly but instead edit the source -# files and re-render. - ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - creationTimestamp: null - name: contour -rules: -- apiGroups: - - "" - resources: - - configmaps - verbs: - - create - - get - - update -- apiGroups: - - "" - resources: - - endpoints - verbs: - - get - - list - - watch -- apiGroups: - - "" - resources: - - secrets - verbs: - - get - - list - - watch -- apiGroups: - - "" - resources: - - services - verbs: - - get - - list - - watch -- apiGroups: - - apiextensions.k8s.io - resources: - - customresourcedefinitions - verbs: - - list -- apiGroups: - - networking.k8s.io - resources: - - gatewayclasses - - gateways - - httproutes - - tcproutes - verbs: - - get - - list - - watch -- apiGroups: - - networking.k8s.io - resources: - - ingresses - verbs: - - get - - list - - watch -- apiGroups: - - networking.k8s.io - resources: - - ingresses/status - verbs: - - create - - get - - update -- apiGroups: - - projectcontour.io - resources: - - extensionservices - verbs: - - get - - list - - watch -- apiGroups: - - projectcontour.io - resources: - - extensionservices/status - verbs: - - create - - get - - update -- apiGroups: - - projectcontour.io - resources: - - httpproxies - - tlscertificatedelegations - verbs: - - get - - list - - watch -- apiGroups: - - projectcontour.io - resources: - - httpproxies/status - verbs: - - create - - get - - update - ---- -apiVersion: v1 -kind: Service -metadata: - name: contour - namespace: projectcontour -spec: - ports: - - port: 8001 - name: xds - protocol: TCP - targetPort: 8001 - selector: - app: contour - type: ClusterIP - ---- -apiVersion: v1 -kind: Service -metadata: - name: envoy - namespace: projectcontour - annotations: - # This annotation puts the AWS ELB into "TCP" mode so that it does not - # do HTTP negotiation for HTTPS connections at the ELB edge. - # The downside of this is the remote IP address of all connections will - # appear to be the internal address of the ELB. See docs/proxy-proto.md - # for information about enabling the PROXY protocol on the ELB to recover - # the original remote IP address. - service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp -spec: - externalTrafficPolicy: Local - ports: - - port: 80 - name: http - protocol: TCP - nodePort: 30081 - - port: 443 - name: https - protocol: TCP - selector: - app: envoy - type: LoadBalancer - ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - labels: - app: contour - name: contour - namespace: projectcontour -spec: - replicas: 2 - strategy: - type: RollingUpdate - rollingUpdate: - # This value of maxSurge means that during a rolling update - # the new ReplicaSet will be created first. - maxSurge: 50% - selector: - matchLabels: - app: contour - template: - metadata: - annotations: - prometheus.io/scrape: "true" - prometheus.io/port: "8000" - labels: - app: contour - spec: - affinity: - podAntiAffinity: - preferredDuringSchedulingIgnoredDuringExecution: - - podAffinityTerm: - labelSelector: - matchLabels: - app: contour - topologyKey: kubernetes.io/hostname - weight: 100 - containers: - - args: - - serve - - --incluster - - --xds-address=0.0.0.0 - - --xds-port=8001 - - --envoy-service-http-port=80 - - --envoy-service-https-port=443 - - --contour-cafile=/certs/ca.crt - - --contour-cert-file=/certs/tls.crt - - --contour-key-file=/certs/tls.key - - --config-path=/config/contour.yaml - command: ["contour"] - image: docker.io/projectcontour/contour:v1.11.0 - imagePullPolicy: IfNotPresent - name: contour - ports: - - containerPort: 8001 - name: xds - protocol: TCP - - containerPort: 8000 - name: metrics - protocol: TCP - - containerPort: 6060 - name: debug - protocol: TCP - livenessProbe: - httpGet: - path: /healthz - port: 8000 - readinessProbe: - tcpSocket: - port: 8001 - initialDelaySeconds: 15 - periodSeconds: 10 - volumeMounts: - - name: contourcert - mountPath: /certs - readOnly: true - - name: contour-config - mountPath: /config - readOnly: true - env: - - name: CONTOUR_NAMESPACE - valueFrom: - fieldRef: - apiVersion: v1 - fieldPath: metadata.namespace - - name: POD_NAME - valueFrom: - fieldRef: - apiVersion: v1 - fieldPath: metadata.name - dnsPolicy: ClusterFirst - serviceAccountName: contour - securityContext: - runAsNonRoot: true - runAsUser: 65534 - runAsGroup: 65534 - volumes: - - name: contourcert - secret: - secretName: contourcert - - name: contour-config - configMap: - name: contour - defaultMode: 0644 - items: - - key: contour.yaml - path: contour.yaml - ---- -apiVersion: apps/v1 -kind: DaemonSet -metadata: - labels: - app: envoy - name: envoy - namespace: projectcontour -spec: - updateStrategy: - type: RollingUpdate - rollingUpdate: - maxUnavailable: 10% - selector: - matchLabels: - app: envoy - template: - metadata: - annotations: - prometheus.io/scrape: "true" - prometheus.io/port: "8002" - prometheus.io/path: "/stats/prometheus" - labels: - app: envoy - spec: - containers: - - command: - - /bin/contour - args: - - envoy - - shutdown-manager - image: docker.io/projectcontour/contour:v1.11.0 - imagePullPolicy: IfNotPresent - lifecycle: - preStop: - exec: - command: - - /bin/contour - - envoy - - shutdown - livenessProbe: - httpGet: - path: /healthz - port: 8090 - initialDelaySeconds: 3 - periodSeconds: 10 - name: shutdown-manager - - args: - - -c - - /config/envoy.json - - --service-cluster $(CONTOUR_NAMESPACE) - - --service-node $(ENVOY_POD_NAME) - - --log-level info - command: - - envoy - image: docker.io/envoyproxy/envoy:v1.16.2 - imagePullPolicy: IfNotPresent - name: envoy - env: - - name: CONTOUR_NAMESPACE - valueFrom: - fieldRef: - apiVersion: v1 - fieldPath: metadata.namespace - - name: ENVOY_POD_NAME - valueFrom: - fieldRef: - apiVersion: v1 - fieldPath: metadata.name - ports: - - containerPort: 80 - hostPort: 80 - name: http - protocol: TCP - - containerPort: 443 - hostPort: 443 - name: https - protocol: TCP - readinessProbe: - httpGet: - path: /ready - port: 8002 - initialDelaySeconds: 3 - periodSeconds: 4 - volumeMounts: - - name: envoy-config - mountPath: /config - - name: envoycert - mountPath: /certs - lifecycle: - preStop: - httpGet: - path: /shutdown - port: 8090 - scheme: HTTP - initContainers: - - args: - - bootstrap - - /config/envoy.json - - --xds-address=contour - - --xds-port=8001 - - --xds-resource-version=v3 - - --resources-dir=/config/resources - - --envoy-cafile=/certs/ca.crt - - --envoy-cert-file=/certs/tls.crt - - --envoy-key-file=/certs/tls.key - command: - - contour - image: docker.io/projectcontour/contour:v1.11.0 - imagePullPolicy: IfNotPresent - name: envoy-initconfig - volumeMounts: - - name: envoy-config - mountPath: /config - - name: envoycert - mountPath: /certs - readOnly: true - env: - - name: CONTOUR_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - automountServiceAccountToken: false - serviceAccountName: envoy - terminationGracePeriodSeconds: 300 - volumes: - - name: envoy-config - emptyDir: {} - - name: envoycert - secret: - secretName: envoycert - restartPolicy: Always diff --git a/kustomize/overlays/sandbox/dependencies/contour_ingress_controller/kustomization.yaml b/kustomize/overlays/sandbox/dependencies/contour_ingress_controller/kustomization.yaml deleted file mode 100644 index 5e16119d07..0000000000 --- a/kustomize/overlays/sandbox/dependencies/contour_ingress_controller/kustomization.yaml +++ /dev/null @@ -1,5 +0,0 @@ -resources: - # We cannot just import from remote as namespace transformer causes conflicts - #- https://projectcontour.io/quickstart/v1.11.0/contour.yaml -- contour.yaml - diff --git a/kustomize/overlays/sandbox/dependencies/redis/deployment.yaml b/kustomize/overlays/sandbox/dependencies/redis/deployment.yaml deleted file mode 100644 index 82067d702b..0000000000 --- a/kustomize/overlays/sandbox/dependencies/redis/deployment.yaml +++ /dev/null @@ -1,59 +0,0 @@ -apiVersion: apps/v1 -kind: StatefulSet -metadata: - name: redis - namespace: flyte -spec: - replicas: 1 - selector: - matchLabels: - app: redis-resource-manager - serviceName: redis-resource-manager - template: - metadata: - labels: - app: redis-resource-manager - spec: - containers: - - env: - - name: REDIS_PASSWORD - value: mypassword - image: docker.io/bitnami/redis:4.0.2-r1 - imagePullPolicy: IfNotPresent - livenessProbe: - exec: - command: - - redis-cli - - ping - failureThreshold: 3 - initialDelaySeconds: 30 - periodSeconds: 10 - successThreshold: 1 - timeoutSeconds: 5 - name: redis-resource-manager - ports: - - containerPort: 6379 - name: redis - protocol: TCP - readinessProbe: - exec: - command: - - redis-cli - - ping - failureThreshold: 3 - initialDelaySeconds: 5 - periodSeconds: 10 - successThreshold: 1 - timeoutSeconds: 1 - resources: - requests: - cpu: 200m - memory: 128Mi - volumeMounts: - - mountPath: /bitnami - name: redis-data - dnsPolicy: ClusterFirst - restartPolicy: Always - volumes: - - name: redis-data - emptyDir: {} diff --git a/kustomize/overlays/sandbox/dependencies/redis/kustomization.yaml b/kustomize/overlays/sandbox/dependencies/redis/kustomization.yaml deleted file mode 100644 index a944d005ca..0000000000 --- a/kustomize/overlays/sandbox/dependencies/redis/kustomization.yaml +++ /dev/null @@ -1,3 +0,0 @@ -resources: -- deployment.yaml -- service.yaml diff --git a/kustomize/overlays/sandbox/dependencies/redis/service.yaml b/kustomize/overlays/sandbox/dependencies/redis/service.yaml deleted file mode 100644 index 8c86264ae3..0000000000 --- a/kustomize/overlays/sandbox/dependencies/redis/service.yaml +++ /dev/null @@ -1,16 +0,0 @@ -apiVersion: v1 -kind: Service -metadata: - labels: - app: redis-resource-manager - name: redis-resource-manager - namespace: flyte -spec: - ports: - - name: redis - port: 6379 - protocol: TCP - targetPort: redis - selector: - app: redis-resource-manager - type: ClusterIP diff --git a/kustomize/overlays/sandbox/admin/deployment.yaml b/kustomize/overlays/sandbox/flyte/admin/deployment.yaml similarity index 100% rename from kustomize/overlays/sandbox/admin/deployment.yaml rename to kustomize/overlays/sandbox/flyte/admin/deployment.yaml diff --git a/kustomize/overlays/sandbox/config/admin/db.yaml b/kustomize/overlays/sandbox/flyte/config/admin/db.yaml similarity index 100% rename from kustomize/overlays/sandbox/config/admin/db.yaml rename to kustomize/overlays/sandbox/flyte/config/admin/db.yaml diff --git a/kustomize/overlays/sandbox/config/admin/task_resource_defaults.yaml b/kustomize/overlays/sandbox/flyte/config/admin/task_resource_defaults.yaml similarity index 100% rename from kustomize/overlays/sandbox/config/admin/task_resource_defaults.yaml rename to kustomize/overlays/sandbox/flyte/config/admin/task_resource_defaults.yaml diff --git a/kustomize/overlays/sandbox/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml b/kustomize/overlays/sandbox/flyte/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml similarity index 100% rename from kustomize/overlays/sandbox/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml rename to kustomize/overlays/sandbox/flyte/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml diff --git a/kustomize/overlays/sandbox/config/clusterresource-templates/ad_spark-role.yaml b/kustomize/overlays/sandbox/flyte/config/clusterresource-templates/ad_spark-role.yaml similarity index 100% rename from kustomize/overlays/sandbox/config/clusterresource-templates/ad_spark-role.yaml rename to kustomize/overlays/sandbox/flyte/config/clusterresource-templates/ad_spark-role.yaml diff --git a/kustomize/overlays/sandbox/config/clusterresource-templates/ae_spark-service-account.yaml b/kustomize/overlays/sandbox/flyte/config/clusterresource-templates/ae_spark-service-account.yaml similarity index 100% rename from kustomize/overlays/sandbox/config/clusterresource-templates/ae_spark-service-account.yaml rename to kustomize/overlays/sandbox/flyte/config/clusterresource-templates/ae_spark-service-account.yaml diff --git a/kustomize/overlays/sandbox/config/clusterresource-templates/af_spark-role-binding.yaml b/kustomize/overlays/sandbox/flyte/config/clusterresource-templates/af_spark-role-binding.yaml similarity index 100% rename from kustomize/overlays/sandbox/config/clusterresource-templates/af_spark-role-binding.yaml rename to kustomize/overlays/sandbox/flyte/config/clusterresource-templates/af_spark-role-binding.yaml diff --git a/kustomize/overlays/sandbox/config/common/logger.yaml b/kustomize/overlays/sandbox/flyte/config/common/logger.yaml similarity index 100% rename from kustomize/overlays/sandbox/config/common/logger.yaml rename to kustomize/overlays/sandbox/flyte/config/common/logger.yaml diff --git a/kustomize/overlays/sandbox/config/common/storage.yaml b/kustomize/overlays/sandbox/flyte/config/common/storage.yaml similarity index 100% rename from kustomize/overlays/sandbox/config/common/storage.yaml rename to kustomize/overlays/sandbox/flyte/config/common/storage.yaml diff --git a/kustomize/overlays/sandbox/config/datacatalog/db.yaml b/kustomize/overlays/sandbox/flyte/config/datacatalog/db.yaml similarity index 100% rename from kustomize/overlays/sandbox/config/datacatalog/db.yaml rename to kustomize/overlays/sandbox/flyte/config/datacatalog/db.yaml diff --git a/kustomize/overlays/sandbox/config/propeller/enabled_plugins.yaml b/kustomize/overlays/sandbox/flyte/config/propeller/enabled_plugins.yaml similarity index 100% rename from kustomize/overlays/sandbox/config/propeller/enabled_plugins.yaml rename to kustomize/overlays/sandbox/flyte/config/propeller/enabled_plugins.yaml diff --git a/kustomize/overlays/sandbox/config/propeller/plugins/k8s.yaml b/kustomize/overlays/sandbox/flyte/config/propeller/plugins/k8s.yaml similarity index 100% rename from kustomize/overlays/sandbox/config/propeller/plugins/k8s.yaml rename to kustomize/overlays/sandbox/flyte/config/propeller/plugins/k8s.yaml diff --git a/kustomize/overlays/sandbox/config/propeller/plugins/task_logs.yaml b/kustomize/overlays/sandbox/flyte/config/propeller/plugins/task_logs.yaml similarity index 100% rename from kustomize/overlays/sandbox/config/propeller/plugins/task_logs.yaml rename to kustomize/overlays/sandbox/flyte/config/propeller/plugins/task_logs.yaml diff --git a/kustomize/overlays/sandbox/config/propeller/resource_manager.yaml b/kustomize/overlays/sandbox/flyte/config/propeller/resource_manager.yaml similarity index 100% rename from kustomize/overlays/sandbox/config/propeller/resource_manager.yaml rename to kustomize/overlays/sandbox/flyte/config/propeller/resource_manager.yaml diff --git a/kustomize/overlays/sandbox/flyte/kustomization.yaml b/kustomize/overlays/sandbox/flyte/kustomization.yaml new file mode 100644 index 0000000000..d3485ba7b7 --- /dev/null +++ b/kustomize/overlays/sandbox/flyte/kustomization.yaml @@ -0,0 +1,42 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +# Override the namespace +namespace: flyte + +bases: +- ../../../base/single_cluster/complete + +patchesStrategicMerge: +- admin/deployment.yaml + +configMapGenerator: +- behavior: merge + files: + - ./config/admin/db.yaml + - ./config/admin/task_resource_defaults.yaml + - ./config/common/storage.yaml + - ./config/common/logger.yaml + name: flyte-admin-config + +- behavior: merge + files: + - ./config/clusterresource-templates/ac_project-copilot-dataconfig.yaml + name: clusterresource-template + +- behavior: merge + files: + - ./config/propeller/enabled_plugins.yaml + - ./config/propeller/resource_manager.yaml + - ./config/propeller/plugins/k8s.yaml + - ./config/propeller/plugins/task_logs.yaml + - ./config/common/storage.yaml + - ./config/common/logger.yaml + name: flyte-propeller-config + +- behavior: merge + files: + - ./config/common/storage.yaml + - ./config/common/logger.yaml + - ./config/datacatalog/db.yaml + name: datacatalog-config diff --git a/kustomize/overlays/sandbox/kustomization.yaml b/kustomize/overlays/sandbox/kustomization.yaml index e60cd33045..b3f1f9a6d7 100644 --- a/kustomize/overlays/sandbox/kustomization.yaml +++ b/kustomize/overlays/sandbox/kustomization.yaml @@ -1,13 +1,9 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization -# Override the namespace -namespace: flyte - bases: - - ../../base/single_cluster/complete - # All the resources that make up the deployment +- flyte ######## # Operators to be enabled ############# @@ -16,62 +12,31 @@ bases: # This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) # Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's # Add node ports for ease of use locally -resources: - - ./dependencies/database - - ./dependencies/storage - - ./dependencies/contour_ingress_controller - - ./dependencies/nodeport-services.yaml - - ../../base/namespace +- ../../base/addons/database +- ../../base/addons/storage +- ../../base/addons/contour_ingress_controller -patchesStrategicMerge: - - admin/deployment.yaml - -configMapGenerator: - - behavior: merge - files: - - ./config/admin/db.yaml - - ./config/admin/task_resource_defaults.yaml - - ./config/common/storage.yaml - - ./config/common/logger.yaml - name: flyte-admin-config - - behavior: merge - files: - - ./config/clusterresource-templates/ac_project-copilot-dataconfig.yaml - name: clusterresource-template - - behavior: merge - files: - - ./config/propeller/enabled_plugins.yaml - - ./config/propeller/resource_manager.yaml - - ./config/propeller/plugins/k8s.yaml - - ./config/propeller/plugins/task_logs.yaml - - ./config/common/storage.yaml - - ./config/common/logger.yaml - name: flyte-propeller-config - - behavior: merge - files: - - ./config/common/storage.yaml - - ./config/common/logger.yaml - - ./config/datacatalog/db.yaml - name: datacatalog-config +resources: +- nodeport-services.yaml # Images that should be used images: - # FlyteAdmin - - name: flyteadmin # match images with this name - newTag: v0.3.29 # override the tag - newName: ghcr.io/lyft/flyteadmin # override the name - # FlyteConsole - - name: flyteconsole # match images with this name - newTag: v0.19.1 # override the tag - newName: ghcr.io/lyft/flyteconsole # override the namep - # Flyte DataCatalog - - name: datacatalog # match images with this name - newTag: v0.2.130 # override the tag - newName: ghcr.io/lyft/datacatalog # override the name - # FlytePropeller - - name: flytepropeller # match images with this name - newTag: v0.5.13 # override the tag - newName: ghcr.io/flyteorg/flytepropeller # override the name - # Override postgres image to use alpine based (rather smaller) docker image - - name: postgres - newTag: 10.16-alpine +# FlyteAdmin +- name: flyteadmin # match images with this name + newTag: v0.3.29 # override the tag + newName: ghcr.io/lyft/flyteadmin # override the name +# FlyteConsole +- name: flyteconsole # match images with this name + newTag: v0.19.1 # override the tag + newName: ghcr.io/lyft/flyteconsole # override the namep +# Flyte DataCatalog +- name: datacatalog # match images with this name + newTag: v0.2.130 # override the tag + newName: ghcr.io/lyft/datacatalog # override the name +# FlytePropeller +- name: flytepropeller # match images with this name + newTag: v0.5.13 # override the tag + newName: ghcr.io/flyteorg/flytepropeller # override the name +# Override postgres image to use alpine based (rather smaller) docker image +- name: postgres + newTag: 10.16-alpine diff --git a/kustomize/overlays/sandbox/dependencies/nodeport-services.yaml b/kustomize/overlays/sandbox/nodeport-services.yaml similarity index 100% rename from kustomize/overlays/sandbox/dependencies/nodeport-services.yaml rename to kustomize/overlays/sandbox/nodeport-services.yaml diff --git a/kustomize/overlays/test/dependencies/contour_ingress_controller/custom-config.yaml b/kustomize/overlays/test/dependencies/contour_ingress_controller/custom-config.yaml deleted file mode 100644 index c0fcfee56a..0000000000 --- a/kustomize/overlays/test/dependencies/contour_ingress_controller/custom-config.yaml +++ /dev/null @@ -1,134 +0,0 @@ -apiVersion: apiextensions.k8s.io/v1beta1 -kind: CustomResourceDefinition -metadata: - name: ingressroutes.contour.heptio.com - labels: - component: ingressroute -spec: - group: contour.heptio.com - version: v1beta1 - scope: Namespaced - names: - plural: ingressroutes - kind: IngressRoute - additionalPrinterColumns: - - name: FQDN - type: string - description: Fully qualified domain name - JSONPath: .spec.virtualhost.fqdn - - name: TLS Secret - type: string - description: Secret with TLS credentials - JSONPath: .spec.virtualhost.tls.secretName - - name: First route - type: string - description: First routes defined - JSONPath: .spec.routes[0].match - - name: Status - type: string - description: The current status of the IngressRoute - JSONPath: .status.currentStatus - - name: Status Description - type: string - description: Description of the current status - JSONPath: .status.description - validation: - openAPIV3Schema: - properties: - spec: - required: - - routes - properties: - virtualhost: - properties: - fqdn: - type: string - pattern: ^([a-zA-Z0-9]+(-[a-zA-Z0-9]+)*\.)+[a-z]{2,}$ - aliases: - type: array - items: - type: string - pattern: ^([a-z0-9]+(-[a-z0-9]+)*\.)+[a-z]{2,}$ - strategy: - type: string - enum: - - RoundRobin - - LeastRequest - - Random - - RingHash - - Maglev - healthCheck: - type: object - required: - - path - properties: - path: - type: string - pattern: ^\/.*$ - intervalSeconds: - type: integer - timeoutSeconds: - type: integer - unhealthyThresholdCount: - type: integer - healthyThresholdCount: - type: integer - routes: - type: array - items: - required: - - match - properties: - match: - type: string - pattern: ^\/.*$ - delegate: - type: object - required: - - name - properties: - name: - type: string - pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ # DNS-1123 - namespace: - type: string - pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ # DNS-1123 - services: - type: array - items: - type: object - required: - - name - - port - properties: - name: - type: string - pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ # DNS-1123 - port: - type: integer - weight: - type: integer - strategy: - type: string - enum: - - RoundRobin - - WeightedLeastRequest - - Random - - RingHash - - Maglev - healthCheck: - type: object - required: - - path - properties: - path: - type: string - pattern: ^\/.*$ - intervalSeconds: - type: integer - timeoutSeconds: - type: integer - unhealthyThresholdCount: - type: integer - healthyThresholdCount: - type: integer diff --git a/kustomize/overlays/test/dependencies/contour_ingress_controller/deployment.yaml b/kustomize/overlays/test/dependencies/contour_ingress_controller/deployment.yaml deleted file mode 100644 index c24f2b504c..0000000000 --- a/kustomize/overlays/test/dependencies/contour_ingress_controller/deployment.yaml +++ /dev/null @@ -1,66 +0,0 @@ -# Create the actual deployment -apiVersion: apps/v1 -kind: Deployment -metadata: - name: contour - namespace: heptio-contour - labels: - app: contour -spec: - replicas: 1 - selector: - matchLabels: - app: contour - template: - metadata: - labels: - app: contour - annotations: - prometheus.io/scrape: "true" - prometheus.io/port: "8002" - prometheus.io/path: "/stats" - prometheus.io/format: "prometheus" - spec: - containers: - - image: docker.io/envoyproxy/envoy-alpine:v1.6.0 - name: envoy-envoyingressv1 - ports: - - containerPort: 80 - name: http - - containerPort: 8002 - name: statsd - command: ["envoy"] - args: ["-c", "/config/contour.yaml", "--service-cluster", "cluster0", "--service-node", "node0"] - volumeMounts: - - name: contour-config - mountPath: /config - resources: - requests: - cpu: 100m - memory: 100Mi - limits: - cpu: 100m - memory: 100Mi - - image: gcr.io/heptio-images/contour:v0.6.1 - imagePullPolicy: Always - ports: - - containerPort: 8000 - name: contour - name: contour-unknown - command: ["contour"] - args: ["serve", "--incluster", "--envoy-http-port=80", "--debug-http-port=6069"] - initContainers: - - image: gcr.io/heptio-images/contour:v0.6.1 - imagePullPolicy: Always - name: envoy-initconfig - command: ["contour"] - args: ["bootstrap", "/config/contour.yaml", "--statsd-enabled"] - volumeMounts: - - name: contour-config - mountPath: /config - volumes: - - name: contour-config - emptyDir: {} - dnsPolicy: ClusterFirst - serviceAccountName: contour - terminationGracePeriodSeconds: 30 diff --git a/kustomize/overlays/test/dependencies/contour_ingress_controller/kustomization.yaml b/kustomize/overlays/test/dependencies/contour_ingress_controller/kustomization.yaml deleted file mode 100644 index 83bbe44daf..0000000000 --- a/kustomize/overlays/test/dependencies/contour_ingress_controller/kustomization.yaml +++ /dev/null @@ -1,5 +0,0 @@ -resources: -- rbac.yaml -- custom-config.yaml -- service.yaml -- deployment.yaml diff --git a/kustomize/overlays/test/dependencies/contour_ingress_controller/rbac.yaml b/kustomize/overlays/test/dependencies/contour_ingress_controller/rbac.yaml deleted file mode 100644 index 4492a6f80f..0000000000 --- a/kustomize/overlays/test/dependencies/contour_ingress_controller/rbac.yaml +++ /dev/null @@ -1,66 +0,0 @@ -apiVersion: v1 -kind: ServiceAccount -metadata: - name: contour - namespace: heptio-contour ---- -apiVersion: rbac.authorization.k8s.io/v1beta1 -kind: ClusterRole -metadata: - name: contour -rules: -- apiGroups: - - "" - resources: - - configmaps - - endpoints - - nodes - - pods - - secrets - verbs: - - list - - watch -- apiGroups: - - "" - resources: - - nodes - verbs: - - get -- apiGroups: - - "" - resources: - - services - verbs: - - get - - list - - watch -- apiGroups: - - extensions - resources: - - ingresses - verbs: - - get - - list - - watch -- apiGroups: ["contour.heptio.com"] - resources: ["ingressroutes"] - verbs: - - get - - list - - watch - - put - - post - - patch ---- -apiVersion: rbac.authorization.k8s.io/v1beta1 -kind: ClusterRoleBinding -metadata: - name: contour -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: ClusterRole - name: contour -subjects: -- kind: ServiceAccount - name: contour - namespace: heptio-contour diff --git a/kustomize/overlays/test/dependencies/contour_ingress_controller/service.yaml b/kustomize/overlays/test/dependencies/contour_ingress_controller/service.yaml deleted file mode 100644 index b7e10ab4af..0000000000 --- a/kustomize/overlays/test/dependencies/contour_ingress_controller/service.yaml +++ /dev/null @@ -1,17 +0,0 @@ -# Service -apiVersion: v1 -kind: Service -metadata: - name: contour - namespace: heptio-contour - labels: - app: contour -spec: - # use NodePort to make sure the service is accessible - type: NodePort - selector: - app: contour - ports: - - protocol: TCP - port: 80 - nodePort: 30081 diff --git a/kustomize/overlays/test/dependencies/database/database.yaml b/kustomize/overlays/test/dependencies/database/database.yaml deleted file mode 100644 index de17063c4c..0000000000 --- a/kustomize/overlays/test/dependencies/database/database.yaml +++ /dev/null @@ -1,41 +0,0 @@ ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: postgres - namespace: flyte -spec: - selector: - matchLabels: - app: postgres - template: - metadata: - labels: - app: postgres - spec: - volumes: - - name: postgres-storage - emptyDir: {} - containers: - - image: postgres - name: postgres - env: - - name: POSTGRES_HOST_AUTH_METHOD - value: trust - ports: - - containerPort: 5432 - name: postgres - volumeMounts: - - name: postgres-storage - mountPath: /var/lib/postgresql/data ---- -apiVersion: v1 -kind: Service -metadata: - name: postgres - namespace: flyte -spec: - ports: - - port: 5432 - selector: - app: postgres diff --git a/kustomize/overlays/test/dependencies/database/kustomization.yaml b/kustomize/overlays/test/dependencies/database/kustomization.yaml deleted file mode 100644 index ca2ede8d46..0000000000 --- a/kustomize/overlays/test/dependencies/database/kustomization.yaml +++ /dev/null @@ -1,2 +0,0 @@ -resources: -- database.yaml diff --git a/kustomize/overlays/test/dependencies/storage/kustomization.yaml b/kustomize/overlays/test/dependencies/storage/kustomization.yaml deleted file mode 100644 index 9a80610c4e..0000000000 --- a/kustomize/overlays/test/dependencies/storage/kustomization.yaml +++ /dev/null @@ -1,2 +0,0 @@ -resources: -- storage.yaml diff --git a/kustomize/overlays/test/dependencies/storage/storage.yaml b/kustomize/overlays/test/dependencies/storage/storage.yaml deleted file mode 100644 index 2f72d3c9a6..0000000000 --- a/kustomize/overlays/test/dependencies/storage/storage.yaml +++ /dev/null @@ -1,63 +0,0 @@ ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: minio - namespace: flyte -spec: - selector: - matchLabels: - app: minio - template: - metadata: - labels: - app: minio - spec: - volumes: - - name: minio-storage - emptyDir: {} - containers: - - image: "minio/minio:RELEASE.2020-12-16T05-05-17Z" - name: minio - env: - - name: MINIO_ACCESS_KEY - value: minio - - name: MINIO_SECRET_KEY - value: miniostorage - args: - - server - - /data - ports: - - containerPort: 9000 - name: minio - volumeMounts: - - name: minio-storage - mountPath: /data ---- -apiVersion: v1 -kind: Service -metadata: - name: minio - namespace: flyte -spec: - externalName: minio - ports: - - port: 9000 - selector: - app: minio ---- -apiVersion: extensions/v1beta1 -kind: Ingress -metadata: - name: minio - namespace: flyte - annotations: - nginx.ingress.kubernetes.io/ssl-redirect: "false" -spec: - rules: - - http: - paths: - - path: /minio - backend: - serviceName: minio - servicePort: 9000 diff --git a/kustomize/overlays/test/admin/deployment.yaml b/kustomize/overlays/test/flyte/admin/deployment.yaml similarity index 100% rename from kustomize/overlays/test/admin/deployment.yaml rename to kustomize/overlays/test/flyte/admin/deployment.yaml diff --git a/kustomize/overlays/test/config/admin/db.yaml b/kustomize/overlays/test/flyte/config/admin/db.yaml similarity index 100% rename from kustomize/overlays/test/config/admin/db.yaml rename to kustomize/overlays/test/flyte/config/admin/db.yaml diff --git a/kustomize/overlays/test/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml b/kustomize/overlays/test/flyte/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml similarity index 100% rename from kustomize/overlays/test/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml rename to kustomize/overlays/test/flyte/config/clusterresource-templates/ac_project-copilot-dataconfig.yaml diff --git a/kustomize/overlays/test/config/common/logger.yaml b/kustomize/overlays/test/flyte/config/common/logger.yaml similarity index 100% rename from kustomize/overlays/test/config/common/logger.yaml rename to kustomize/overlays/test/flyte/config/common/logger.yaml diff --git a/kustomize/overlays/test/config/common/storage.yaml b/kustomize/overlays/test/flyte/config/common/storage.yaml similarity index 100% rename from kustomize/overlays/test/config/common/storage.yaml rename to kustomize/overlays/test/flyte/config/common/storage.yaml diff --git a/kustomize/overlays/test/config/datacatalog/db.yaml b/kustomize/overlays/test/flyte/config/datacatalog/db.yaml similarity index 100% rename from kustomize/overlays/test/config/datacatalog/db.yaml rename to kustomize/overlays/test/flyte/config/datacatalog/db.yaml diff --git a/kustomize/overlays/test/config/propeller/enabled_plugins.yaml b/kustomize/overlays/test/flyte/config/propeller/enabled_plugins.yaml similarity index 100% rename from kustomize/overlays/test/config/propeller/enabled_plugins.yaml rename to kustomize/overlays/test/flyte/config/propeller/enabled_plugins.yaml diff --git a/kustomize/overlays/test/config/propeller/plugins/k8s.yaml b/kustomize/overlays/test/flyte/config/propeller/plugins/k8s.yaml similarity index 100% rename from kustomize/overlays/test/config/propeller/plugins/k8s.yaml rename to kustomize/overlays/test/flyte/config/propeller/plugins/k8s.yaml diff --git a/kustomize/overlays/test/config/propeller/resource_manager.yaml b/kustomize/overlays/test/flyte/config/propeller/resource_manager.yaml similarity index 100% rename from kustomize/overlays/test/config/propeller/resource_manager.yaml rename to kustomize/overlays/test/flyte/config/propeller/resource_manager.yaml diff --git a/kustomize/overlays/test/flyte/kustomization.yaml b/kustomize/overlays/test/flyte/kustomization.yaml new file mode 100644 index 0000000000..60b964ed8c --- /dev/null +++ b/kustomize/overlays/test/flyte/kustomization.yaml @@ -0,0 +1,49 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +# Override the namespace +namespace: flyte + +bases: +- ../../../base/single_cluster/headless + +patchesStrategicMerge: +- admin/deployment.yaml + +configMapGenerator: +# the main admin configmap +- name: flyte-admin-config + behavior: merge + files: + - ./config/admin/db.yaml + - ./config/common/storage.yaml + - ./config/common/logger.yaml + +# cluster resource templates +- name: clusterresource-template + behavior: merge + files: + # Files are read in alphabetical order. To ensure that we create the namespace first, prefix the file name with "aa". + - ./config/clusterresource-templates/ac_project-copilot-dataconfig.yaml + +# Flyte Propeller Configuration +- name: flyte-propeller-config + behavior: merge + files: + - ./config/propeller/enabled_plugins.yaml + - ./config/propeller/resource_manager.yaml + - ./config/propeller/plugins/k8s.yaml + - ./config/common/storage.yaml + - ./config/common/logger.yaml + +# TODO Flyte Console Configuration +#- name: flyte-console-config +# files: +# - ./config/console.yaml + +- name: datacatalog-config + behavior: merge + files: + - ./config/datacatalog/db.yaml + - ./config/common/storage.yaml + - ./config/common/logger.yaml diff --git a/kustomize/overlays/test/kustomization.yaml b/kustomize/overlays/test/kustomization.yaml index 1c1832179b..2a2b6b805b 100644 --- a/kustomize/overlays/test/kustomization.yaml +++ b/kustomize/overlays/test/kustomization.yaml @@ -1,81 +1,38 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization -# Override the namespace -namespace: flyte - bases: - - ../../base/single_cluster/headless - # All the resources that make up the deployment -resources: - ############# - # Dependencies to be configured - - ./dependencies/database - # In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc - - ./dependencies/storage - # Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's - - ./dependencies/contour_ingress_controller - - ../../base/namespace - -patches: - - ./admin/deployment.yaml - -configMapGenerator: - # the main admin configmap - - name: flyte-admin-config - behavior: merge - files: - - ./config/admin/db.yaml - - ./config/common/storage.yaml - - ./config/common/logger.yaml - # cluster resource templates - - name: clusterresource-template - behavior: merge - files: - # Files are read in alphabetical order. To ensure that we create the namespace first, prefix the file name with "aa". - - ./config/clusterresource-templates/ac_project-copilot-dataconfig.yaml - - # Flyte Propeller Configuration - - name: flyte-propeller-config - behavior: merge - files: - - ./config/propeller/enabled_plugins.yaml - - ./config/propeller/resource_manager.yaml - - ./config/propeller/plugins/k8s.yaml - - ./config/common/storage.yaml - - ./config/common/logger.yaml - - # TODO Flyte Console Configuration - #- name: flyte-console-config - # files: - # - ./config/console.yaml - - - name: datacatalog-config - behavior: merge - files: - - ./config/datacatalog/db.yaml - - ./config/common/storage.yaml - - ./config/common/logger.yaml +- flyte +######## +# Operators to be enabled +############# +# Dependencies to be configured +# In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc +# This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) +# Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's +# Add node ports for ease of use locally +- ../../base/addons/database +- ../../base/addons/storage # Images that should be used images: - # FlyteAdmin - - name: flyteadmin # match images with this name - newTag: v0.3.29 # override the tag - newName: ghcr.io/lyft/flyteadmin # override the name - # FlyteConsole - - name: flyteconsole # match images with this name - newTag: v0.19.1 # override the tag - newName: ghcr.io/lyft/flyteconsole # override the namep - # Flyte DataCatalog - - name: datacatalog # match images with this name - newTag: v0.2.130 # override the tag - newName: ghcr.io/lyft/datacatalog # override the name - # FlytePropeller - - name: flytepropeller # match images with this name - newTag: v0.5.13 # override the tag - newName: ghcr.io/flyteorg/flytepropeller # override the name - # Override postgres image to use alpine based (rather smaller) docker image - - name: postgres - newTag: 10.16-alpine +# FlyteAdmin +- name: flyteadmin # match images with this name + newTag: v0.3.29 # override the tag + newName: ghcr.io/lyft/flyteadmin # override the name +# FlyteConsole +- name: flyteconsole # match images with this name + newTag: v0.19.1 # override the tag + newName: ghcr.io/lyft/flyteconsole # override the namep +# Flyte DataCatalog +- name: datacatalog # match images with this name + newTag: v0.2.130 # override the tag + newName: ghcr.io/lyft/datacatalog # override the name +# FlytePropeller +- name: flytepropeller # match images with this name + newTag: v0.5.13 # override the tag + newName: ghcr.io/flyteorg/flytepropeller # override the name +# Override postgres image to use alpine based (rather smaller) docker image +- name: postgres + newTag: 10.16-alpine From 2dc9086a04191ed005d72c16e5aa0d52d4088737 Mon Sep 17 00:00:00 2001 From: Ketan Umare <16888709+kumare3@users.noreply.github.com> Date: Mon, 1 Mar 2021 20:23:28 -0800 Subject: [PATCH 12/74] Better index (#785) Signed-off-by: Haytham Abuelfutuh --- rsts/index.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rsts/index.rst b/rsts/index.rst index 55a19ff5ea..6c05db9a19 100644 --- a/rsts/index.rst +++ b/rsts/index.rst @@ -8,7 +8,7 @@ Welcome to the documentation hub for Flyte. .. toctree:: :maxdepth: 1 - :caption: Get started with Flyte + :caption: Tutorials :name: tutorialstoc tutorials/first_run @@ -17,7 +17,7 @@ Welcome to the documentation hub for Flyte. tutorials/learn .. toctree:: - :caption: How to Guides + :caption: Get Answers :maxdepth: 1 :name: howtotoc From 307f31ee3495b430f48745a111c3ac64369db944 Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Tue, 2 Mar 2021 10:35:25 -0800 Subject: [PATCH 13/74] Consolidate meeting notes and demo signup sheet (#787) Signed-off-by: Haytham Abuelfutuh --- README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/README.md b/README.md index 909152c6ff..cfeaf765ec 100644 --- a/README.md +++ b/README.md @@ -59,8 +59,7 @@ Resources that would help you get a better understanding of Flyte. - 📣  Flyte OSS Community Sync Every alternate Tuesday, 9am-10am PDT ([Checkout the events calendar & subscribe](https://calendar.google.com/calendar/embed?src=admin%40flyte.org&ctz=America%2FLos_Angeles) - You can join the [zoom link]( https://us04web.zoom.us/j/71298741279?pwd=TDR1RUppQmxGaDRFdzBOa2lHN1dsZz09). -- Meeting notes are captured in [Doc](https://docs.google.com/document/d/1Jb6eOPOzvTaHjtPEVy7OR2O5qK1MhEs3vv56DX2dacM/edit#heading=h.c5ha25xc546e) -- Demo Signup [Sheet](https://docs.google.com/spreadsheets/d/1Bz-s3SpMm1hGl6Z5y3HcxrNBcg9ZhSgqVgmCxK_eEhs/edit#gid=0) +- Meeting notes and backlog of topics are captured in [Doc](https://docs.google.com/document/d/1Jb6eOPOzvTaHjtPEVy7OR2O5qK1MhEs3vv56DX2dacM/edit#heading=h.c5ha25xc546e) - [Video Recordings](https://www.youtube.com/channel/UCNduEoLOToNo3nFVly-vUTQ) ## Conference Talks From 1d4528698e6a69c43345e2a6b4a9e018b5b68466 Mon Sep 17 00:00:00 2001 From: Jeev B Date: Wed, 3 Mar 2021 10:39:02 -0800 Subject: [PATCH 14/74] Add kubernetes dashboard to sandbox overlay (#789) * Add kubernetes dashboard to sandbox overlay Signed-off-by: Jeev B * Remove task logs from headless kustomize module Signed-off-by: Jeev B * Update first run doc to expose 30082 for kubernetes dashboard Signed-off-by: Jeev B * use kubernetes-template-url for specifying log URL in sandbox overlay Signed-off-by: Jeev B * Update flytepropeller to v0.6.1 to leverage `kubernetes-template-uri` config Signed-off-by: Jeev B Signed-off-by: Haytham Abuelfutuh --- deployment/eks/flyte_generated.yaml | 2 +- deployment/gcp/flyte_generated.yaml | 2 +- deployment/sandbox/flyte_generated.yaml | 295 +++++++++++++++++- deployment/test/flyte_generated.yaml | 12 +- docker/sandbox/Dockerfile | 4 +- .../clusterrolebinding.yaml | 12 + .../kubernetes_dashboard/deployment.yaml | 15 + .../kubernetes_dashboard/kustomization.yaml | 7 + .../addons/kubernetes_dashboard/service.yaml | 12 + .../config/propeller/plugins/task_logs.yaml | 5 - .../headless/kustomization.yaml | 1 - kustomize/overlays/eks/kustomization.yaml | 2 +- kustomize/overlays/gcp/kustomization.yaml | 2 +- .../config/propeller/plugins/task_logs.yaml | 2 +- kustomize/overlays/sandbox/kustomization.yaml | 3 +- kustomize/overlays/test/kustomization.yaml | 2 +- rsts/tutorials/first_run.rst | 2 +- 17 files changed, 351 insertions(+), 29 deletions(-) create mode 100644 kustomize/base/addons/kubernetes_dashboard/clusterrolebinding.yaml create mode 100644 kustomize/base/addons/kubernetes_dashboard/deployment.yaml create mode 100644 kustomize/base/addons/kubernetes_dashboard/kustomization.yaml create mode 100644 kustomize/base/addons/kubernetes_dashboard/service.yaml delete mode 100644 kustomize/base/single_cluster/headless/config/propeller/plugins/task_logs.yaml diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index e47d98fb0e..ef7948aac0 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -8777,7 +8777,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.5.13 + image: ghcr.io/flyteorg/flytepropeller:v0.6.1 imagePullPolicy: IfNotPresent name: flytepropeller ports: diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index 0d9b307eff..a8bd2185e7 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -8809,7 +8809,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.5.13 + image: ghcr.io/flyteorg/flytepropeller:v0.6.1 imagePullPolicy: IfNotPresent name: flytepropeller ports: diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index 3f5300cebd..c03aa0572c 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -5,6 +5,11 @@ metadata: --- apiVersion: v1 kind: Namespace +metadata: + name: kubernetes-dashboard +--- +apiVersion: v1 +kind: Namespace metadata: name: projectcontour --- @@ -1595,6 +1600,14 @@ metadata: --- apiVersion: v1 kind: ServiceAccount +metadata: + labels: + k8s-app: kubernetes-dashboard + name: kubernetes-dashboard + namespace: kubernetes-dashboard +--- +apiVersion: v1 +kind: ServiceAccount metadata: name: contour namespace: projectcontour @@ -1613,6 +1626,57 @@ metadata: --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role +metadata: + labels: + k8s-app: kubernetes-dashboard + name: kubernetes-dashboard + namespace: kubernetes-dashboard +rules: +- apiGroups: + - "" + resourceNames: + - kubernetes-dashboard-key-holder + - kubernetes-dashboard-certs + - kubernetes-dashboard-csrf + resources: + - secrets + verbs: + - get + - update + - delete +- apiGroups: + - "" + resourceNames: + - kubernetes-dashboard-settings + resources: + - configmaps + verbs: + - get + - update +- apiGroups: + - "" + resourceNames: + - heapster + - dashboard-metrics-scraper + resources: + - services + verbs: + - proxy +- apiGroups: + - "" + resourceNames: + - heapster + - 'http:heapster:' + - 'https:heapster:' + - dashboard-metrics-scraper + - http:dashboard-metrics-scraper + resources: + - services/proxy + verbs: + - get +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role metadata: name: contour-certgen namespace: projectcontour @@ -1824,6 +1888,39 @@ rules: - deletecollection --- apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + labels: + k8s-app: kubernetes-dashboard + name: kubernetes-dashboard +rules: +- apiGroups: + - metrics.k8s.io + resources: + - pods + - nodes + verbs: + - get + - list + - watch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + labels: + k8s-app: kubernetes-dashboard + name: kubernetes-dashboard + namespace: kubernetes-dashboard +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: kubernetes-dashboard +subjects: +- kind: ServiceAccount + name: kubernetes-dashboard + namespace: kubernetes-dashboard +--- +apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: contour @@ -1850,6 +1947,32 @@ subjects: name: contour namespace: projectcontour --- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: kubernetes-dashboard +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: kubernetes-dashboard +subjects: +- kind: ServiceAccount + name: kubernetes-dashboard + namespace: kubernetes-dashboard +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: kubernetes-dashboard-admin +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: cluster-admin +subjects: +- kind: ServiceAccount + name: kubernetes-dashboard + namespace: kubernetes-dashboard +--- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: @@ -2157,13 +2280,21 @@ data: plugins: logs: kubernetes-enabled: true - kubernetes-url: "http://localhost:30082" + kubernetes-template-uri: "http://localhost:30082/#/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace }}" kind: ConfigMap metadata: - name: flyte-propeller-config-94gbkmm6cm + name: flyte-propeller-config-492gkfhbgk namespace: flyte --- apiVersion: v1 +kind: ConfigMap +metadata: + labels: + k8s-app: kubernetes-dashboard + name: kubernetes-dashboard-settings + namespace: kubernetes-dashboard +--- +apiVersion: v1 data: contour.yaml: | # @@ -2295,6 +2426,26 @@ metadata: type: Opaque --- apiVersion: v1 +data: + csrf: "" +kind: Secret +metadata: + labels: + k8s-app: kubernetes-dashboard + name: kubernetes-dashboard-csrf + namespace: kubernetes-dashboard +type: Opaque +--- +apiVersion: v1 +kind: Secret +metadata: + labels: + k8s-app: kubernetes-dashboard + name: kubernetes-dashboard-key-holder + namespace: kubernetes-dashboard +type: Opaque +--- +apiVersion: v1 kind: Service metadata: annotations: @@ -2408,6 +2559,38 @@ spec: --- apiVersion: v1 kind: Service +metadata: + labels: + k8s-app: dashboard-metrics-scraper + name: dashboard-metrics-scraper + namespace: kubernetes-dashboard +spec: + ports: + - port: 8000 + targetPort: 8000 + selector: + k8s-app: dashboard-metrics-scraper +--- +apiVersion: v1 +kind: Service +metadata: + labels: + k8s-app: kubernetes-dashboard + name: kubernetes-dashboard + namespace: kubernetes-dashboard +spec: + ports: + - name: http + nodePort: 30082 + port: 80 + protocol: TCP + targetPort: 9090 + selector: + k8s-app: kubernetes-dashboard + type: NodePort +--- +apiVersion: v1 +kind: Service metadata: name: contour namespace: projectcontour @@ -2715,7 +2898,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.5.13 + image: ghcr.io/flyteorg/flytepropeller:v0.6.1 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -2726,7 +2909,7 @@ spec: serviceAccountName: flytepropeller volumes: - configMap: - name: flyte-propeller-config-94gbkmm6cm + name: flyte-propeller-config-492gkfhbgk name: config-volume --- apiVersion: apps/v1 @@ -2796,6 +2979,110 @@ spec: --- apiVersion: apps/v1 kind: Deployment +metadata: + labels: + k8s-app: dashboard-metrics-scraper + name: dashboard-metrics-scraper + namespace: kubernetes-dashboard +spec: + replicas: 1 + revisionHistoryLimit: 10 + selector: + matchLabels: + k8s-app: dashboard-metrics-scraper + template: + metadata: + annotations: + seccomp.security.alpha.kubernetes.io/pod: runtime/default + labels: + k8s-app: dashboard-metrics-scraper + spec: + containers: + - image: kubernetesui/metrics-scraper:v1.0.6 + livenessProbe: + httpGet: + path: / + port: 8000 + scheme: HTTP + initialDelaySeconds: 30 + timeoutSeconds: 30 + name: dashboard-metrics-scraper + ports: + - containerPort: 8000 + protocol: TCP + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + runAsGroup: 2001 + runAsUser: 1001 + volumeMounts: + - mountPath: /tmp + name: tmp-volume + nodeSelector: + kubernetes.io/os: linux + serviceAccountName: kubernetes-dashboard + tolerations: + - effect: NoSchedule + key: node-role.kubernetes.io/master + volumes: + - emptyDir: {} + name: tmp-volume +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + k8s-app: kubernetes-dashboard + name: kubernetes-dashboard + namespace: kubernetes-dashboard +spec: + replicas: 1 + revisionHistoryLimit: 10 + selector: + matchLabels: + k8s-app: kubernetes-dashboard + template: + metadata: + labels: + k8s-app: kubernetes-dashboard + spec: + containers: + - args: + - --namespace=kubernetes-dashboard + - --enable-insecure-login + - --enable-skip-login + - --disable-settings-authorizer + image: kubernetesui/dashboard:v2.2.0 + livenessProbe: + httpGet: + path: / + port: 9090 + initialDelaySeconds: 30 + timeoutSeconds: 30 + name: kubernetes-dashboard + ports: + - containerPort: 9090 + protocol: TCP + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + runAsGroup: 2001 + runAsUser: 1001 + volumeMounts: + - mountPath: /tmp + name: tmp-volume + nodeSelector: + kubernetes.io/os: linux + serviceAccountName: kubernetes-dashboard + tolerations: + - effect: NoSchedule + key: node-role.kubernetes.io/master + volumes: + - emptyDir: {} + name: tmp-volume +--- +apiVersion: apps/v1 +kind: Deployment metadata: labels: app: contour diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index 794dde417f..fff5821755 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -415,15 +415,9 @@ data: region: us-east-1 container: "my-s3-bucket" - task_logs.yaml: | - # TODO this is used to enable log links in the UI. - plugins: - logs: - kubernetes-enabled: true - kubernetes-url: "http://localhost:30082" kind: ConfigMap metadata: - name: flyte-propeller-config-569h4b5459 + name: flyte-propeller-config-ttk5h2hkmg namespace: flyte --- apiVersion: v1 @@ -741,7 +735,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.5.13 + image: ghcr.io/flyteorg/flytepropeller:v0.6.1 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -752,7 +746,7 @@ spec: serviceAccountName: flytepropeller volumes: - configMap: - name: flyte-propeller-config-569h4b5459 + name: flyte-propeller-config-ttk5h2hkmg name: config-volume --- apiVersion: apps/v1 diff --git a/docker/sandbox/Dockerfile b/docker/sandbox/Dockerfile index 7211cb74fd..f8d21909ad 100644 --- a/docker/sandbox/Dockerfile +++ b/docker/sandbox/Dockerfile @@ -50,7 +50,7 @@ VOLUME /var/lib/cni VOLUME /var/log # Expose Flyte ports -EXPOSE 30081 30084 +EXPOSE 30081 30082 30084 ENTRYPOINT ["tini", "flyte-entrypoint.sh"] @@ -76,6 +76,6 @@ VOLUME /var/lib/cni VOLUME /var/log # Expose Flyte ports -EXPOSE 30081 30084 +EXPOSE 30081 30082 30084 ENTRYPOINT ["tini", "flyte-entrypoint.sh"] diff --git a/kustomize/base/addons/kubernetes_dashboard/clusterrolebinding.yaml b/kustomize/base/addons/kubernetes_dashboard/clusterrolebinding.yaml new file mode 100644 index 0000000000..76fd3fc503 --- /dev/null +++ b/kustomize/base/addons/kubernetes_dashboard/clusterrolebinding.yaml @@ -0,0 +1,12 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: kubernetes-dashboard-admin +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: cluster-admin +subjects: +- kind: ServiceAccount + name: kubernetes-dashboard + namespace: kubernetes-dashboard diff --git a/kustomize/base/addons/kubernetes_dashboard/deployment.yaml b/kustomize/base/addons/kubernetes_dashboard/deployment.yaml new file mode 100644 index 0000000000..49299fec08 --- /dev/null +++ b/kustomize/base/addons/kubernetes_dashboard/deployment.yaml @@ -0,0 +1,15 @@ +kind: Deployment +apiVersion: apps/v1 +metadata: + name: kubernetes-dashboard + namespace: kubernetes-dashboard +spec: + template: + spec: + containers: + - name: kubernetes-dashboard + args: + - --namespace=kubernetes-dashboard + - --enable-insecure-login + - --enable-skip-login + - --disable-settings-authorizer diff --git a/kustomize/base/addons/kubernetes_dashboard/kustomization.yaml b/kustomize/base/addons/kubernetes_dashboard/kustomization.yaml new file mode 100644 index 0000000000..22384dfe0f --- /dev/null +++ b/kustomize/base/addons/kubernetes_dashboard/kustomization.yaml @@ -0,0 +1,7 @@ +resources: + # TODO (jeev): Figure out how to rev this automatically +- https://raw.githubusercontent.com/kubernetes/dashboard/v2.2.0/aio/deploy/alternative.yaml +- clusterrolebinding.yaml +patchesStrategicMerge: +- deployment.yaml +- service.yaml diff --git a/kustomize/base/addons/kubernetes_dashboard/service.yaml b/kustomize/base/addons/kubernetes_dashboard/service.yaml new file mode 100644 index 0000000000..c08a9308c8 --- /dev/null +++ b/kustomize/base/addons/kubernetes_dashboard/service.yaml @@ -0,0 +1,12 @@ +apiVersion: v1 +kind: Service +metadata: + name: kubernetes-dashboard + namespace: kubernetes-dashboard +spec: + type: NodePort + ports: + - port: 80 + name: http + protocol: TCP + nodePort: 30082 diff --git a/kustomize/base/single_cluster/headless/config/propeller/plugins/task_logs.yaml b/kustomize/base/single_cluster/headless/config/propeller/plugins/task_logs.yaml deleted file mode 100644 index de402d2ed3..0000000000 --- a/kustomize/base/single_cluster/headless/config/propeller/plugins/task_logs.yaml +++ /dev/null @@ -1,5 +0,0 @@ -# TODO this is used to enable log links in the UI. -plugins: - logs: - kubernetes-enabled: true - kubernetes-url: "http://localhost:30082" diff --git a/kustomize/base/single_cluster/headless/kustomization.yaml b/kustomize/base/single_cluster/headless/kustomization.yaml index 0a863fab06..4373a33e41 100644 --- a/kustomize/base/single_cluster/headless/kustomization.yaml +++ b/kustomize/base/single_cluster/headless/kustomization.yaml @@ -43,7 +43,6 @@ configMapGenerator: - ./config/propeller/enabled_plugins.yaml - ./config/propeller/plugins/copilot.yaml - ./config/propeller/plugins/k8s.yaml - - ./config/propeller/plugins/task_logs.yaml - ./config/common/storage.yaml - ./config/common/logger.yaml diff --git a/kustomize/overlays/eks/kustomization.yaml b/kustomize/overlays/eks/kustomization.yaml index fa35ea88e7..6a9e8fe92b 100644 --- a/kustomize/overlays/eks/kustomization.yaml +++ b/kustomize/overlays/eks/kustomization.yaml @@ -35,5 +35,5 @@ images: newName: ghcr.io/lyft/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.5.13 # override the tag + newTag: v0.6.1 # override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name diff --git a/kustomize/overlays/gcp/kustomization.yaml b/kustomize/overlays/gcp/kustomization.yaml index 5e4de67b08..3ed3f8e8eb 100644 --- a/kustomize/overlays/gcp/kustomization.yaml +++ b/kustomize/overlays/gcp/kustomization.yaml @@ -34,5 +34,5 @@ images: newName: ghcr.io/lyft/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.5.13 # override the tag + newTag: v0.6.1 # override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name diff --git a/kustomize/overlays/sandbox/flyte/config/propeller/plugins/task_logs.yaml b/kustomize/overlays/sandbox/flyte/config/propeller/plugins/task_logs.yaml index 0955b59387..2768f8c95d 100644 --- a/kustomize/overlays/sandbox/flyte/config/propeller/plugins/task_logs.yaml +++ b/kustomize/overlays/sandbox/flyte/config/propeller/plugins/task_logs.yaml @@ -1,4 +1,4 @@ plugins: logs: kubernetes-enabled: true - kubernetes-url: "http://localhost:30082" + kubernetes-template-uri: "http://localhost:30082/#/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace }}" diff --git a/kustomize/overlays/sandbox/kustomization.yaml b/kustomize/overlays/sandbox/kustomization.yaml index b3f1f9a6d7..ada55b8166 100644 --- a/kustomize/overlays/sandbox/kustomization.yaml +++ b/kustomize/overlays/sandbox/kustomization.yaml @@ -13,6 +13,7 @@ bases: # Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's # Add node ports for ease of use locally - ../../base/addons/database +- ../../base/addons/kubernetes_dashboard - ../../base/addons/storage - ../../base/addons/contour_ingress_controller @@ -35,7 +36,7 @@ images: newName: ghcr.io/lyft/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.5.13 # override the tag + newTag: v0.6.1 # override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Override postgres image to use alpine based (rather smaller) docker image - name: postgres diff --git a/kustomize/overlays/test/kustomization.yaml b/kustomize/overlays/test/kustomization.yaml index 2a2b6b805b..e63c217a7f 100644 --- a/kustomize/overlays/test/kustomization.yaml +++ b/kustomize/overlays/test/kustomization.yaml @@ -31,7 +31,7 @@ images: newName: ghcr.io/lyft/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.5.13 # override the tag + newTag: v0.6.1 # override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Override postgres image to use alpine based (rather smaller) docker image - name: postgres diff --git a/rsts/tutorials/first_run.rst b/rsts/tutorials/first_run.rst index d403e5d0a9..d2b9693bc3 100644 --- a/rsts/tutorials/first_run.rst +++ b/rsts/tutorials/first_run.rst @@ -12,7 +12,7 @@ Flyte enables scalable, reproducable and reliable orchestration of massively lar With `docker installed `__, run this command: :: - docker run --rm --privileged -p 30081:30081 -p 30084:30084 ghcr.io/flyteorg/flyte-sandbox + docker run --rm --privileged -p 30081:30081 -p 30082:30082 -p 30084:30084 ghcr.io/flyteorg/flyte-sandbox This creates a local Flyte sandbox. Once the sandbox is ready, you should see the following message: ``Flyte is ready! Flyte UI is available at http://localhost:30081/console``. Go ahead and visit http://localhost:30081/console. From 55e37577771075bee543ae17ad2bd23cc53865b7 Mon Sep 17 00:00:00 2001 From: Ketan Umare <16888709+kumare3@users.noreply.github.com> Date: Thu, 4 Mar 2021 17:03:51 -0800 Subject: [PATCH 15/74] Sandbox diagram added to Documentation (#798) Signed-off-by: Ketan Umare Signed-off-by: Haytham Abuelfutuh --- rsts/howto/sandbox.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/rsts/howto/sandbox.rst b/rsts/howto/sandbox.rst index cd086a8605..02bd0eda0a 100644 --- a/rsts/howto/sandbox.rst +++ b/rsts/howto/sandbox.rst @@ -17,6 +17,10 @@ Flyte can be run using a Kubernetes cluster only. This installs all the dependen #. The sandbox deployment is not suitable for production environments. For an in-depth overview of how to productionize your flyte deployment, checkout the :ref:`howto_productionize`. +.. image:: https://raw.githubusercontent.com/flyteorg/flyte/static-resources/img/core/flyte_sandbox_single_k8s_cluster.png + :alt: Architecture of Sandbox deployment of Flyte. Single K8s cluster + + ********************************************************* Deploy Flyte Sandbox environment locally - on your laptop ********************************************************* From 42dde7978f8ca764ea80adb712439008cb781f93 Mon Sep 17 00:00:00 2001 From: Yee Hing Tong Date: Thu, 4 Mar 2021 19:28:26 -0800 Subject: [PATCH 16/74] Milestone v0.11 (#794) Signed-off-by: Haytham Abuelfutuh --- CHANGELOG/CHANGELOG-v0.11.0.md | 21 ++++++ deployment/eks/flyte_generated.yaml | 14 ++-- deployment/gcp/flyte_generated.yaml | 14 ++-- deployment/sandbox/flyte_generated.yaml | 14 ++-- deployment/test/flyte_generated.yaml | 10 +-- end2end/tests/endtoend.yaml | 2 +- kustomize/overlays/eks/kustomization.yaml | 64 ++++++++--------- kustomize/overlays/gcp/kustomization.yaml | 62 ++++++++--------- kustomize/overlays/sandbox/kustomization.yaml | 68 +++++++++---------- kustomize/overlays/test/kustomization.yaml | 62 ++++++++--------- rsts/conf.py | 2 +- 11 files changed, 177 insertions(+), 156 deletions(-) create mode 100644 CHANGELOG/CHANGELOG-v0.11.0.md diff --git a/CHANGELOG/CHANGELOG-v0.11.0.md b/CHANGELOG/CHANGELOG-v0.11.0.md new file mode 100644 index 0000000000..ab19f1e21a --- /dev/null +++ b/CHANGELOG/CHANGELOG-v0.11.0.md @@ -0,0 +1,21 @@ +# Flyte v0.11.0 + +## Flyte Platform +* New to flyte? https://start.flyte.org takes you through first run experience. (Thanks to @jeevb) +* [Grafana templates](https://docs.flyte.org/en/latest/howto/monitoring/index.html) for monitoring Flyte System and User Workflows. +* [Extend Flyte](https://docs.flyte.org/en/latest/plugins/index.html) docs. +* [FlyteIdl Docs](https://docs.flyte.org/projects/flyteidl/en/latest/) are published! You can learn about the core language that makes it all work. +* [Additional knob](https://github.com/flyteorg/flytepropeller/pull/219/files#diff-91657d6448dfbf87f4cecf126ad02bd668ea233edcf74e860ef4f54bdd4cb552R78) for fine tuning flyte propeller performance that speeds up executions drastically. +* OidC support for Google Idp (And other OidC compliant Idps) +* Various stabilization bugs. + +## Flytekit +Since v0.16.0a2, the last flytekit milestone release, all effort has been towards stabilizing the new API. Please see the individual [releases](https://github.com/flyteorg/flytekit/releases) for detailed information. The highlights are + +* Serialization/registration processes have been firmed up and utilities to ease that process introduced (not having to build a container to serialize for instance). +* Plugins structure revamped (eventually we'll move to a separate new repo entirely) +* User-facing imports have been organized into three top-level subpackages (`flytekit`, `flytekit.testing`, and `flytekit.extend`) +* Retries added to read-only Admin calls in client +* Lots of cleanup and additions to the [cookbook](https://flytecookbook.readthedocs.io/en/latest/) and documentation generally. +* Bug fixes. + diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index ef7948aac0..74d29367ae 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -8607,7 +8607,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -8653,7 +8653,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8671,7 +8671,7 @@ spec: - flytesnacks - flytetester - flyteexamples - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -8686,7 +8686,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -8733,7 +8733,7 @@ spec: - envFrom: - configMapRef: name: flyte-console-config - image: ghcr.io/lyft/flyteconsole:v0.19.1 + image: ghcr.io/flyteorg/flyteconsole:v0.19.6 name: flyteconsole ports: - containerPort: 8080 @@ -8777,7 +8777,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.6.1 + image: ghcr.io/flyteorg/flytepropeller:v0.6.2 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -8965,7 +8965,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index a8bd2185e7..0d9529e1ba 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -8639,7 +8639,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -8685,7 +8685,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8703,7 +8703,7 @@ spec: - flytesnacks - flytetester - flyteexamples - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -8718,7 +8718,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -8765,7 +8765,7 @@ spec: - envFrom: - configMapRef: name: flyte-console-config - image: ghcr.io/lyft/flyteconsole:v0.19.1 + image: ghcr.io/flyteorg/flyteconsole:v0.19.6 name: flyteconsole ports: - containerPort: 8080 @@ -8809,7 +8809,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.6.1 + image: ghcr.io/flyteorg/flytepropeller:v0.6.2 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -8997,7 +8997,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index c03aa0572c..1a40a7ac5a 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -2723,7 +2723,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -2775,7 +2775,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -2792,7 +2792,7 @@ spec: - seed-projects - flytesnacks - flyteexamples - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -2807,7 +2807,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -2854,7 +2854,7 @@ spec: - envFrom: - configMapRef: name: flyte-console-config - image: ghcr.io/lyft/flyteconsole:v0.19.1 + image: ghcr.io/flyteorg/flyteconsole:v0.19.6 name: flyteconsole ports: - containerPort: 8080 @@ -2898,7 +2898,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.6.1 + image: ghcr.io/flyteorg/flytepropeller:v0.6.2 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -3204,7 +3204,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index fff5821755..067d8df2de 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -594,7 +594,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -646,7 +646,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -663,7 +663,7 @@ spec: - seed-projects - flytetester - flytesnacks - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -678,7 +678,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/lyft/flyteadmin:v0.3.29 + image: ghcr.io/flyteorg/flyteadmin:v0.3.37 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -735,7 +735,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.6.1 + image: ghcr.io/flyteorg/flytepropeller:v0.6.2 imagePullPolicy: IfNotPresent name: flytepropeller ports: diff --git a/end2end/tests/endtoend.yaml b/end2end/tests/endtoend.yaml index 82aa687985..dc807ddcb2 100644 --- a/end2end/tests/endtoend.yaml +++ b/end2end/tests/endtoend.yaml @@ -11,7 +11,7 @@ spec: command: - bash - -c - image: ghcr.io/flyteorg/flytetools:85ab24ff1025327a02cbd570ab96bbdded132ce9 + image: ghcr.io/flyteorg/flytetools:1fde462c72029d5f4d73ade8bff51af846598754 imagePullPolicy: IfNotPresent name: flytetester resources: diff --git a/kustomize/overlays/eks/kustomization.yaml b/kustomize/overlays/eks/kustomization.yaml index 6a9e8fe92b..d9ddc376c7 100644 --- a/kustomize/overlays/eks/kustomization.yaml +++ b/kustomize/overlays/eks/kustomization.yaml @@ -2,38 +2,38 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization bases: -# All the resources that make up the deployment -- flyte -######## -# Operators to be enabled -############# -# Dependencies to be configured -# In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc -# This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) -# Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's -# Add node ports for ease of use locally -- ../../base/operators/spark -- ../../base/operators/kfoperators/pytorch -# TODO Fix deployment of alb_ingress controller -#- ../../base/addons/alb_ingress -# Optional dependency -- ../../base/addons/redis + # All the resources that make up the deployment + - flyte + ######## + # Operators to be enabled + ############# + # Dependencies to be configured + # In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc + # This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) + # Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's + # Add node ports for ease of use locally + - ../../base/operators/spark + - ../../base/operators/kfoperators/pytorch + # TODO Fix deployment of alb_ingress controller + #- ../../base/addons/alb_ingress + # Optional dependency + - ../../base/addons/redis # Images that should be used images: -# FlyteAdmin -- name: flyteadmin # match images with this name - newTag: v0.3.29 # override the tag - newName: ghcr.io/lyft/flyteadmin # override the name -# FlyteConsole -- name: flyteconsole # match images with this name - newTag: v0.19.1 # override the tag - newName: ghcr.io/lyft/flyteconsole # override the namep -# Flyte DataCatalog -- name: datacatalog # match images with this name - newTag: v0.2.130 # override the tag - newName: ghcr.io/lyft/datacatalog # override the name -# FlytePropeller -- name: flytepropeller # match images with this name - newTag: v0.6.1 # override the tag - newName: ghcr.io/flyteorg/flytepropeller # override the name + # FlyteAdmin + - name: flyteadmin # match images with this name + newTag: v0.3.37 # override the tag + newName: ghcr.io/flyteorg/flyteadmin # override the name + # FlyteConsole + - name: flyteconsole # match images with this name + newTag: v0.19.6 # override the tag + newName: ghcr.io/flyteorg/flyteconsole # override the namep + # Flyte DataCatalog + - name: datacatalog # match images with this name + newTag: v0.2.130 # override the tag + newName: ghcr.io/lyft/datacatalog # override the name + # FlytePropeller + - name: flytepropeller # match images with this name + newTag: v0.6.2 # override the tag + newName: ghcr.io/flyteorg/flytepropeller # override the name diff --git a/kustomize/overlays/gcp/kustomization.yaml b/kustomize/overlays/gcp/kustomization.yaml index 3ed3f8e8eb..9ac428bf1d 100644 --- a/kustomize/overlays/gcp/kustomization.yaml +++ b/kustomize/overlays/gcp/kustomization.yaml @@ -2,37 +2,37 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization bases: -# All the resources that make up the deployment -- flyte -######## -# Operators to be enabled -############# -# Dependencies to be configured -# In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc -# This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) -# Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's -# Add node ports for ease of use locally -- ../../base/operators/spark -- ../../base/operators/kfoperators/pytorch -- ../../base/addons/cloudsqlproxy -# Optional dependency -- ../../base/addons/redis + # All the resources that make up the deployment + - flyte + ######## + # Operators to be enabled + ############# + # Dependencies to be configured + # In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc + # This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) + # Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's + # Add node ports for ease of use locally + - ../../base/operators/spark + - ../../base/operators/kfoperators/pytorch + - ../../base/addons/cloudsqlproxy + # Optional dependency + - ../../base/addons/redis # Images that should be used images: -# FlyteAdmin -- name: flyteadmin # match images with this name - newTag: v0.3.29 # override the tag - newName: ghcr.io/lyft/flyteadmin # override the name -# FlyteConsole -- name: flyteconsole # match images with this name - newTag: v0.19.1 # override the tag - newName: ghcr.io/lyft/flyteconsole # override the namep -# Flyte DataCatalog -- name: datacatalog # match images with this name - newTag: v0.2.130 # override the tag - newName: ghcr.io/lyft/datacatalog # override the name -# FlytePropeller -- name: flytepropeller # match images with this name - newTag: v0.6.1 # override the tag - newName: ghcr.io/flyteorg/flytepropeller # override the name + # FlyteAdmin + - name: flyteadmin # match images with this name + newTag: v0.3.37 # override the tag + newName: ghcr.io/flyteorg/flyteadmin # override the name + # FlyteConsole + - name: flyteconsole # match images with this name + newTag: v0.19.6 # override the tag + newName: ghcr.io/flyteorg/flyteconsole # override the namep + # Flyte DataCatalog + - name: datacatalog # match images with this name + newTag: v0.2.130 # override the tag + newName: ghcr.io/lyft/datacatalog # override the name + # FlytePropeller + - name: flytepropeller # match images with this name + newTag: v0.6.2 # override the tag + newName: ghcr.io/flyteorg/flytepropeller # override the name diff --git a/kustomize/overlays/sandbox/kustomization.yaml b/kustomize/overlays/sandbox/kustomization.yaml index ada55b8166..c23d6a70e1 100644 --- a/kustomize/overlays/sandbox/kustomization.yaml +++ b/kustomize/overlays/sandbox/kustomization.yaml @@ -2,42 +2,42 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization bases: -# All the resources that make up the deployment -- flyte -######## -# Operators to be enabled -############# -# Dependencies to be configured -# In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc -# This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) -# Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's -# Add node ports for ease of use locally -- ../../base/addons/database -- ../../base/addons/kubernetes_dashboard -- ../../base/addons/storage -- ../../base/addons/contour_ingress_controller + # All the resources that make up the deployment + - flyte + ######## + # Operators to be enabled + ############# + # Dependencies to be configured + # In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc + # This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) + # Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's + # Add node ports for ease of use locally + - ../../base/addons/database + - ../../base/addons/kubernetes_dashboard + - ../../base/addons/storage + - ../../base/addons/contour_ingress_controller resources: -- nodeport-services.yaml + - nodeport-services.yaml # Images that should be used images: -# FlyteAdmin -- name: flyteadmin # match images with this name - newTag: v0.3.29 # override the tag - newName: ghcr.io/lyft/flyteadmin # override the name -# FlyteConsole -- name: flyteconsole # match images with this name - newTag: v0.19.1 # override the tag - newName: ghcr.io/lyft/flyteconsole # override the namep -# Flyte DataCatalog -- name: datacatalog # match images with this name - newTag: v0.2.130 # override the tag - newName: ghcr.io/lyft/datacatalog # override the name -# FlytePropeller -- name: flytepropeller # match images with this name - newTag: v0.6.1 # override the tag - newName: ghcr.io/flyteorg/flytepropeller # override the name -# Override postgres image to use alpine based (rather smaller) docker image -- name: postgres - newTag: 10.16-alpine + # FlyteAdmin + - name: flyteadmin # match images with this name + newTag: v0.3.37 # override the tag + newName: ghcr.io/flyteorg/flyteadmin # override the name + # FlyteConsole + - name: flyteconsole # match images with this name + newTag: v0.19.6 # override the tag + newName: ghcr.io/flyteorg/flyteconsole # override the namep + # Flyte DataCatalog + - name: datacatalog # match images with this name + newTag: v0.2.130 # override the tag + newName: ghcr.io/lyft/datacatalog # override the name + # FlytePropeller + - name: flytepropeller # match images with this name + newTag: v0.6.2 # override the tag + newName: ghcr.io/flyteorg/flytepropeller # override the name + # Override postgres image to use alpine based (rather smaller) docker image + - name: postgres + newTag: 10.16-alpine diff --git a/kustomize/overlays/test/kustomization.yaml b/kustomize/overlays/test/kustomization.yaml index e63c217a7f..9e7fe6f8b4 100644 --- a/kustomize/overlays/test/kustomization.yaml +++ b/kustomize/overlays/test/kustomization.yaml @@ -2,37 +2,37 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization bases: -# All the resources that make up the deployment -- flyte -######## -# Operators to be enabled -############# -# Dependencies to be configured -# In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc -# This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) -# Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's -# Add node ports for ease of use locally -- ../../base/addons/database -- ../../base/addons/storage + # All the resources that make up the deployment + - flyte + ######## + # Operators to be enabled + ############# + # Dependencies to be configured + # In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc + # This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) + # Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's + # Add node ports for ease of use locally + - ../../base/addons/database + - ../../base/addons/storage # Images that should be used images: -# FlyteAdmin -- name: flyteadmin # match images with this name - newTag: v0.3.29 # override the tag - newName: ghcr.io/lyft/flyteadmin # override the name -# FlyteConsole -- name: flyteconsole # match images with this name - newTag: v0.19.1 # override the tag - newName: ghcr.io/lyft/flyteconsole # override the namep -# Flyte DataCatalog -- name: datacatalog # match images with this name - newTag: v0.2.130 # override the tag - newName: ghcr.io/lyft/datacatalog # override the name -# FlytePropeller -- name: flytepropeller # match images with this name - newTag: v0.6.1 # override the tag - newName: ghcr.io/flyteorg/flytepropeller # override the name -# Override postgres image to use alpine based (rather smaller) docker image -- name: postgres - newTag: 10.16-alpine + # FlyteAdmin + - name: flyteadmin # match images with this name + newTag: v0.3.37 # override the tag + newName: ghcr.io/flyteorg/flyteadmin # override the name + # FlyteConsole + - name: flyteconsole # match images with this name + newTag: v0.19.6 # override the tag + newName: ghcr.io/flyteorg/flyteconsole # override the namep + # Flyte DataCatalog + - name: datacatalog # match images with this name + newTag: v0.2.130 # override the tag + newName: ghcr.io/lyft/datacatalog # override the name + # FlytePropeller + - name: flytepropeller # match images with this name + newTag: v0.6.2 # override the tag + newName: ghcr.io/flyteorg/flytepropeller # override the name + # Override postgres image to use alpine based (rather smaller) docker image + - name: postgres + newTag: 10.16-alpine diff --git a/rsts/conf.py b/rsts/conf.py index bf7e909a38..7cc369072e 100644 --- a/rsts/conf.py +++ b/rsts/conf.py @@ -25,7 +25,7 @@ # The short X.Y version version = u'' # The full version, including alpha/beta/rc tags -release = u'0.10.0' +release = u'0.11.0' # -- General configuration --------------------------------------------------- From f91c267c0e3e6b3dba5b6c1a0efd70e7d712c4b8 Mon Sep 17 00:00:00 2001 From: Ketan Umare <16888709+kumare3@users.noreply.github.com> Date: Fri, 5 Mar 2021 08:30:05 -0800 Subject: [PATCH 17/74] AWS EKS Illustration: Production single cluster mode (#799) Signed-off-by: Haytham Abuelfutuh --- rsts/howto/productionize/index.rst | 1 + rsts/howto/productionize/production_eks.rst | 21 +++++++++++++++++++++ 2 files changed, 22 insertions(+) create mode 100644 rsts/howto/productionize/production_eks.rst diff --git a/rsts/howto/productionize/index.rst b/rsts/howto/productionize/index.rst index 4104251076..d08c4a7a2b 100644 --- a/rsts/howto/productionize/index.rst +++ b/rsts/howto/productionize/index.rst @@ -10,3 +10,4 @@ How Do I productionize my Flyte cluster :name: howtoprovguidestoc production + production_eks diff --git a/rsts/howto/productionize/production_eks.rst b/rsts/howto/productionize/production_eks.rst new file mode 100644 index 0000000000..56731f738c --- /dev/null +++ b/rsts/howto/productionize/production_eks.rst @@ -0,0 +1,21 @@ +.. _production-eks: + +Using AWS EKS to host Flyte +------------------------------ + +Illustration +************* + +.. note:: + + - Flyte needs a prefix in an AWS S3 bucket to store all its metadata. This is where the data about executions, workflows, tasks is stored + - this S3 bucket/prefix should be accessible to all FlytePropeller, FlyteAdmin, Datacatalog and running executions (user pods) + - FlyteAdmin can use any RDBMS database but we recommend Postgres. At scale we have used AWS Aurora + - Datacatalog also uses a postgres database similar to admin. They both could share the same physical instance, but prefer to have 2 logically separate databases + - If you want to use AWS IAM role for SeviceAccounts, then you have to manage the provisioning of the service account and providing it to Flyte at the time of execution + - For secrets, you can use Vault, Kube secrets etc, we are working on getting first class support for this + +.. image:: https://raw.githubusercontent.com/flyteorg/flyte/static-resources/img/core/flyte_single_cluster_eks.png + :alt: Illustration of setting up Flyte Cluster in a single AWS EKS (or any K8s cluster on AWS) + + From d3244f56c5369624e6b47555b0d96db01b6efaa5 Mon Sep 17 00:00:00 2001 From: Ketan Umare <16888709+kumare3@users.noreply.github.com> Date: Mon, 8 Mar 2021 10:51:06 -0800 Subject: [PATCH 18/74] Create housekeeping_template.md (#804) * Create housekeeping_template.md * correct the label Signed-off-by: Haytham Abuelfutuh --- .github/ISSUE_TEMPLATE/docs_issue.md | 2 +- .github/ISSUE_TEMPLATE/housekeeping_template.md | 17 +++++++++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-) create mode 100644 .github/ISSUE_TEMPLATE/housekeeping_template.md diff --git a/.github/ISSUE_TEMPLATE/docs_issue.md b/.github/ISSUE_TEMPLATE/docs_issue.md index 42b12ea2e2..22c13edeea 100644 --- a/.github/ISSUE_TEMPLATE/docs_issue.md +++ b/.github/ISSUE_TEMPLATE/docs_issue.md @@ -2,7 +2,7 @@ name: Docs bug/update request about: Suggest an improvement for Flyte docs title: "[Docs]" -labels: docs, untriaged +labels: documentation, untriaged assignees: '' --- diff --git a/.github/ISSUE_TEMPLATE/housekeeping_template.md b/.github/ISSUE_TEMPLATE/housekeeping_template.md new file mode 100644 index 0000000000..e176190c4a --- /dev/null +++ b/.github/ISSUE_TEMPLATE/housekeeping_template.md @@ -0,0 +1,17 @@ +--- +name: Housekeeping +about: Maintenance and cleanup tasks, that need to be performed. This may include tech-debt, library upgrades, CI/CD improvements etc +title: "[Housekeeping] " +labels: housekeeping +assignees: '' + +--- + +**Describe the Issue** +A clear and concise description of what the housekeeping item is. + +**What if we do not do this?** +Short description of potential consequences + +***Related component*** +Either Specific / all From 7ded64d43cb6e8ee0b840ebe0cc9c5eff742ea66 Mon Sep 17 00:00:00 2001 From: Ketan Umare <16888709+kumare3@users.noreply.github.com> Date: Tue, 9 Mar 2021 08:37:11 -0800 Subject: [PATCH 19/74] create_node_execution_event was missing (#795) Signed-off-by: Haytham Abuelfutuh --- .../prometheus/flyteadmin-dashboard.json | 44 +++++++++---------- stats/flyteadmin_dashboard.py | 4 +- 2 files changed, 24 insertions(+), 24 deletions(-) diff --git a/deployment/stats/prometheus/flyteadmin-dashboard.json b/deployment/stats/prometheus/flyteadmin-dashboard.json index 1742e9b2bd..507444c6aa 100644 --- a/deployment/stats/prometheus/flyteadmin-dashboard.json +++ b/deployment/stats/prometheus/flyteadmin-dashboard.json @@ -1619,7 +1619,7 @@ "targets": [ { "datasource": "", - "expr": "sum(irate(flyte:admin:create_node_execution:codes:OK[5m]))", + "expr": "sum(irate(flyte:admin:create_node_execution_event:codes:OK[5m]))", "format": "time_series", "hide": false, "instant": false, @@ -1633,7 +1633,7 @@ }, { "datasource": "", - "expr": "sum(irate(flyte:admin:create_node_execution:codes:InvalidArgument[5m]))", + "expr": "sum(irate(flyte:admin:create_node_execution_event:codes:InvalidArgument[5m]))", "format": "time_series", "hide": false, "instant": false, @@ -1647,7 +1647,7 @@ }, { "datasource": "", - "expr": "sum(irate(flyte:admin:create_node_execution:codes:AlreadyExists[5m]))", + "expr": "sum(irate(flyte:admin:create_node_execution_event:codes:AlreadyExists[5m]))", "format": "time_series", "hide": false, "instant": false, @@ -1661,7 +1661,7 @@ }, { "datasource": "", - "expr": "sum(irate(flyte:admin:create_node_execution:codes:FailedPrecondition[5m]))", + "expr": "sum(irate(flyte:admin:create_node_execution_event:codes:FailedPrecondition[5m]))", "format": "time_series", "hide": false, "instant": false, @@ -1676,7 +1676,7 @@ ], "timeFrom": null, "timeShift": null, - "title": "create_node_execution return codes", + "title": "create_node_execution_event return codes", "tooltip": { "msResolution": true, "shared": true, @@ -1767,7 +1767,7 @@ "targets": [ { "datasource": "", - "expr": "sum(irate(flyte:admin:create_node_execution:errors[5m]))", + "expr": "sum(irate(flyte:admin:create_node_execution_event:errors[5m]))", "format": "time_series", "hide": false, "instant": false, @@ -1781,7 +1781,7 @@ }, { "datasource": "", - "expr": "sum(irate(flyte:admin:create_node_execution:success[5m]))", + "expr": "sum(irate(flyte:admin:create_node_execution_event:success[5m]))", "format": "time_series", "hide": false, "instant": false, @@ -1796,7 +1796,7 @@ ], "timeFrom": null, "timeShift": null, - "title": "create_node_execution success vs errors", + "title": "create_node_execution_event success vs errors", "tooltip": { "msResolution": true, "shared": true, @@ -1887,7 +1887,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:create_node_execution:duration_ms[5m])) by (quantile)", + "expr": "sum(rate(flyte:admin:create_node_execution_event:duration_ms[5m])) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -1902,7 +1902,7 @@ ], "timeFrom": null, "timeShift": null, - "title": "create_node_execution Latency", + "title": "create_node_execution_event Latency", "tooltip": { "msResolution": true, "shared": true, @@ -1938,7 +1938,7 @@ ], "repeat": null, "showTitle": true, - "title": "create_node_execution stats" + "title": "create_node_execution_event stats" }, { "collapse": true, @@ -2003,7 +2003,7 @@ "targets": [ { "datasource": "", - "expr": "sum(irate(flyte:admin:create_task_execution:codes:OK[5m]))", + "expr": "sum(irate(flyte:admin:create_task_execution_event:codes:OK[5m]))", "format": "time_series", "hide": false, "instant": false, @@ -2017,7 +2017,7 @@ }, { "datasource": "", - "expr": "sum(irate(flyte:admin:create_task_execution:codes:InvalidArgument[5m]))", + "expr": "sum(irate(flyte:admin:create_task_execution_event:codes:InvalidArgument[5m]))", "format": "time_series", "hide": false, "instant": false, @@ -2031,7 +2031,7 @@ }, { "datasource": "", - "expr": "sum(irate(flyte:admin:create_task_execution:codes:AlreadyExists[5m]))", + "expr": "sum(irate(flyte:admin:create_task_execution_event:codes:AlreadyExists[5m]))", "format": "time_series", "hide": false, "instant": false, @@ -2045,7 +2045,7 @@ }, { "datasource": "", - "expr": "sum(irate(flyte:admin:create_task_execution:codes:FailedPrecondition[5m]))", + "expr": "sum(irate(flyte:admin:create_task_execution_event:codes:FailedPrecondition[5m]))", "format": "time_series", "hide": false, "instant": false, @@ -2060,7 +2060,7 @@ ], "timeFrom": null, "timeShift": null, - "title": "create_task_execution return codes", + "title": "create_task_execution_event return codes", "tooltip": { "msResolution": true, "shared": true, @@ -2151,7 +2151,7 @@ "targets": [ { "datasource": "", - "expr": "sum(irate(flyte:admin:create_task_execution:errors[5m]))", + "expr": "sum(irate(flyte:admin:create_task_execution_event:errors[5m]))", "format": "time_series", "hide": false, "instant": false, @@ -2165,7 +2165,7 @@ }, { "datasource": "", - "expr": "sum(irate(flyte:admin:create_task_execution:success[5m]))", + "expr": "sum(irate(flyte:admin:create_task_execution_event:success[5m]))", "format": "time_series", "hide": false, "instant": false, @@ -2180,7 +2180,7 @@ ], "timeFrom": null, "timeShift": null, - "title": "create_task_execution success vs errors", + "title": "create_task_execution_event success vs errors", "tooltip": { "msResolution": true, "shared": true, @@ -2271,7 +2271,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:create_task_execution:duration_ms[5m])) by (quantile)", + "expr": "sum(rate(flyte:admin:create_task_execution_event:duration_ms[5m])) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -2286,7 +2286,7 @@ ], "timeFrom": null, "timeShift": null, - "title": "create_task_execution Latency", + "title": "create_task_execution_event Latency", "tooltip": { "msResolution": true, "shared": true, @@ -2322,7 +2322,7 @@ ], "repeat": null, "showTitle": true, - "title": "create_task_execution stats" + "title": "create_task_execution_event stats" }, { "collapse": true, diff --git a/stats/flyteadmin_dashboard.py b/stats/flyteadmin_dashboard.py index c7c660e74b..e87d1d0473 100644 --- a/stats/flyteadmin_dashboard.py +++ b/stats/flyteadmin_dashboard.py @@ -20,8 +20,8 @@ class FlyteAdmin(object): "create_launch_plan", "create_task", "create_workflow", - "create_node_execution", - "create_task_execution", + "create_node_execution_event", + "create_task_execution_event", "get_execution", "get_launch_plan", "get_task", From ac66683a648759a28aae2a9354a4e0a4a07810af Mon Sep 17 00:00:00 2001 From: Yee Hing Tong Date: Tue, 9 Mar 2021 17:37:13 -0800 Subject: [PATCH 20/74] Add a small bashrc to flyte sandbox (#811) Signed-off-by: wild-endeavor Signed-off-by: Haytham Abuelfutuh --- docker/sandbox/Dockerfile | 1 + docker/sandbox/bashrc | 7 +++++++ 2 files changed, 8 insertions(+) create mode 100644 docker/sandbox/bashrc diff --git a/docker/sandbox/Dockerfile b/docker/sandbox/Dockerfile index f8d21909ad..0f70df8eaf 100644 --- a/docker/sandbox/Dockerfile +++ b/docker/sandbox/Dockerfile @@ -39,6 +39,7 @@ RUN apk add --no-cache bash git make tini # Copy entrypoints COPY docker/sandbox/flyte-entrypoint-default.sh /flyteorg/bin/flyte-entrypoint.sh +COPY docker/sandbox/bashrc /root/.bashrc # Update PATH variable ENV PATH "/flyteorg/bin:${PATH}" diff --git a/docker/sandbox/bashrc b/docker/sandbox/bashrc new file mode 100644 index 0000000000..550a1d4e9e --- /dev/null +++ b/docker/sandbox/bashrc @@ -0,0 +1,7 @@ +# Friendlier prompt +export PS1="\e[0;35m\u@flyte-sandbox \w> \e[m" + +# Useful aliases +alias k="kubectl" +alias kf="kubectl -n flyte" +alias l="ls -lthra" From 993a907dcae74b3312b44eb8c44eba08197b3f6e Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Wed, 10 Mar 2021 09:26:10 -0800 Subject: [PATCH 21/74] Allow K3s K8s Api Port to be overriden (#812) Signed-off-by: Haytham Abuelfutuh --- docker/sandbox/flyte-entrypoint-default.sh | 3 ++- docker/sandbox/flyte-entrypoint-dind.sh | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/docker/sandbox/flyte-entrypoint-default.sh b/docker/sandbox/flyte-entrypoint-default.sh index ce1cc8a0c0..5b5c3ceb1d 100755 --- a/docker/sandbox/flyte-entrypoint-default.sh +++ b/docker/sandbox/flyte-entrypoint-default.sh @@ -6,7 +6,8 @@ trap 'pkill -P $$' EXIT # Start k3s echo "Starting k3s cluster..." -k3s server --no-deploy=traefik --no-deploy=servicelb --no-deploy=local-storage --no-deploy=metrics-server &> /var/log/k3s.log & +KUBERNETES_API_PORT=${KUBERNETES_API_PORT:-6443} +k3s server --no-deploy=traefik --no-deploy=servicelb --no-deploy=local-storage --no-deploy=metrics-server --https-listen-port=${KUBERNETES_API_PORT} &> /var/log/k3s.log & K3S_PID=$! timeout 600 sh -c "until k3s kubectl explain deployment &> /dev/null; do sleep 1; done" || ( echo >&2 "Timed out while waiting for the Kubernetes cluster to start"; exit 1 ) echo "Done." diff --git a/docker/sandbox/flyte-entrypoint-dind.sh b/docker/sandbox/flyte-entrypoint-dind.sh index 39b7d7ce48..9db5613c34 100755 --- a/docker/sandbox/flyte-entrypoint-dind.sh +++ b/docker/sandbox/flyte-entrypoint-dind.sh @@ -23,7 +23,8 @@ echo "Done." # Start k3s echo "Starting k3s cluster..." -k3s server --docker --no-deploy=traefik --no-deploy=servicelb --no-deploy=local-storage --no-deploy=metrics-server &> /var/log/k3s.log & +KUBERNETES_API_PORT=${KUBERNETES_API_PORT:-6443} +k3s server --docker --no-deploy=traefik --no-deploy=servicelb --no-deploy=local-storage --no-deploy=metrics-server --https-listen-port=${KUBERNETES_API_PORT} &> /var/log/k3s.log & K3S_PID=$! timeout 600 sh -c "until k3s kubectl explain deployment &> /dev/null; do sleep 1; done" || ( echo >&2 "Timed out while waiting for the Kubernetes cluster to start"; exit 1 ) echo "Done." From 2228c769358c0afe2c2b9ff1612792de46a30a70 Mon Sep 17 00:00:00 2001 From: SandraGH5 <80421934+SandraGH5@users.noreply.github.com> Date: Thu, 11 Mar 2021 14:09:55 -0800 Subject: [PATCH 22/74] Update first_run.rst (#821) Spelling, sentence rewording, "Flyte is read" and "Go ahead" moved to next line. http://localhost:30081/console is a broken link. Signed-off-by: Haytham Abuelfutuh --- rsts/tutorials/first_run.rst | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/rsts/tutorials/first_run.rst b/rsts/tutorials/first_run.rst index d2b9693bc3..498a8a523c 100644 --- a/rsts/tutorials/first_run.rst +++ b/rsts/tutorials/first_run.rst @@ -8,15 +8,19 @@ Getting Started with Flyte .. rubric:: Estimated time to complete: 2 minutes. -Flyte enables scalable, reproducable and reliable orchestration of massively large workflows. In order to get a sense of the product, we have packaged a minimalist version of the Flyte system into a Docker image. +Flyte enables scalable, reproducible and reliable orchestration of massively large workflows. To get a sense of the product, a minimalist version of the Flyte system is packaged into a Docker image. With `docker installed `__, run this command: :: docker run --rm --privileged -p 30081:30081 -p 30082:30082 -p 30084:30084 ghcr.io/flyteorg/flyte-sandbox -This creates a local Flyte sandbox. Once the sandbox is ready, you should see the following message: ``Flyte is ready! Flyte UI is available at http://localhost:30081/console``. Go ahead and visit http://localhost:30081/console. +This creates a local Flyte sandbox. Once the sandbox is ready, you should see the following message: -A quick visual tour for launching your first Workflow: +``Flyte is ready! Flyte UI is available at http://localhost:30081/console``. + +Go ahead and visit http://localhost:30081/console to check it out. + +Below is a quick visual tour for launching your first Workflow: .. image:: https://github.com/flyteorg/flyte/raw/static-resources/img/first-run-console-2.gif :alt: A quick visual tour for launching your first Workflow. From cc21097163ce6da87e2205ed8ca1c6aef46ab4a7 Mon Sep 17 00:00:00 2001 From: v01dXYZ <14996868+v01dXYZ@users.noreply.github.com> Date: Mon, 15 Mar 2021 14:25:35 +0000 Subject: [PATCH 23/74] Update Docs Copyright date (#830) Signed-off-by: v01dxyz Co-authored-by: v01dxyz Signed-off-by: Haytham Abuelfutuh --- rsts/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rsts/conf.py b/rsts/conf.py index 7cc369072e..17b30b2d5f 100644 --- a/rsts/conf.py +++ b/rsts/conf.py @@ -19,7 +19,7 @@ # -- Project information ----------------------------------------------------- project = u'Flyte' -copyright = u'2020, Flyte Authors' +copyright = u'2021, Flyte Authors' author = u'Flyte' # The short X.Y version From 489e182a872ec0a5d7326f76d9243a5e48dc2732 Mon Sep 17 00:00:00 2001 From: SandraGH5 <80421934+SandraGH5@users.noreply.github.com> Date: Mon, 15 Mar 2021 07:25:56 -0700 Subject: [PATCH 24/74] Update first_run.rst (#829) Capitalize With Signed-off-by: Haytham Abuelfutuh --- rsts/tutorials/first_run.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rsts/tutorials/first_run.rst b/rsts/tutorials/first_run.rst index 498a8a523c..1a1ea67e6a 100644 --- a/rsts/tutorials/first_run.rst +++ b/rsts/tutorials/first_run.rst @@ -3,7 +3,7 @@ .. _flyte-tutorials-firstrun: ############################################ -Getting Started with Flyte +Getting Started With Flyte ############################################ .. rubric:: Estimated time to complete: 2 minutes. From 1698f88706734701ddbfa48f85150acfddfff82b Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Mon, 15 Mar 2021 13:14:58 -0700 Subject: [PATCH 25/74] Update EKS Ingress and Instructions (#825) * Update EKS Ingress and Instructions Signed-off-by: Haytham Abuelfutuh * Move alb ingress to addons Signed-off-by: Haytham Abuelfutuh * Update k8s version used in KinD Signed-off-by: Haytham Abuelfutuh * delete unused file Signed-off-by: Haytham Abuelfutuh * PR Comments Signed-off-by: Haytham Abuelfutuh * comments Signed-off-by: Haytham Abuelfutuh --- .github/workflows/tests.yml | 32 +-- deployment/eks/flyte_generated.yaml | 167 ++++++++++++++-- deployment/gcp/flyte_generated.yaml | 150 ++++++++++++-- deployment/sandbox/flyte_generated.yaml | 159 ++++++++++++--- deployment/test/flyte_generated.yaml | 49 +---- kustomize/base/addons/alb_ingress/README.md | 1 - .../base/addons/alb_ingress/ingress.yaml | 47 ----- kustomize/base/addons/storage/storage.yaml | 53 ++--- kustomize/base/ingress/ingress.yaml | 185 ++++++++++++++---- kustomize/base/ingress/kustomization.yaml | 5 +- .../headless/kustomization.yaml | 101 +++++----- .../overlays/eks/flyte/ingress/README.md | 5 + .../overlays/eks/flyte/ingress/ingress.yaml | 137 ++++++++++++- .../eks/flyte/ingress/ingress_grpc.yaml | 31 +++ .../eks/flyte/ingress/kustomization.yaml | 6 + .../overlays/eks/flyte/kustomization.yaml | 90 ++++----- kustomize/overlays/eks/kustomization.yaml | 2 - kustomize/overlays/gcp/kustomization.yaml | 1 + kustomize/overlays/sandbox/kustomization.yaml | 1 + 19 files changed, 892 insertions(+), 330 deletions(-) delete mode 100644 kustomize/base/addons/alb_ingress/README.md delete mode 100644 kustomize/base/addons/alb_ingress/ingress.yaml create mode 100644 kustomize/overlays/eks/flyte/ingress/README.md create mode 100644 kustomize/overlays/eks/flyte/ingress/ingress_grpc.yaml create mode 100644 kustomize/overlays/eks/flyte/ingress/kustomization.yaml diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index b11d184b82..6c690be8be 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -2,23 +2,25 @@ name: tests on: push: branches: - - master + - master pull_request: jobs: end-to-end: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v1 - - name: Kustomize and diff - run: DELTA_CHECK=true make kustomize - - uses: engineerd/setup-kind@v0.5.0 - - name: End2End - env: - DOCKER_USERNAME: ${{ github.actor }} - DOCKER_PASSWORD: "${{ secrets.GITHUB_TOKEN }}" - run: | - kubectl cluster-info - kubectl get pods -n kube-system - echo "current-context:" $(kubectl config current-context) - echo "environment-kubeconfig:" ${KUBECONFIG} - make end2end_execute + - uses: actions/checkout@v1 + - name: Kustomize and diff + run: DELTA_CHECK=true make kustomize + - uses: engineerd/setup-kind@v0.5.0 + with: + version: "v0.10.0" + - name: End2End + env: + DOCKER_USERNAME: ${{ github.actor }} + DOCKER_PASSWORD: "${{ secrets.GITHUB_TOKEN }}" + run: | + kubectl cluster-info + kubectl get pods -n kube-system + echo "current-context:" $(kubectl config current-context) + echo "environment-kubeconfig:" ${KUBECONFIG} + make end2end_execute diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index 74d29367ae..da4a775757 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -9017,14 +9017,19 @@ spec: restartPolicy: Never serviceAccountName: sparkoperator --- -apiVersion: extensions/v1beta1 +apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: + alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' + alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-2:111111111111:certificate/e92fefd8-6197-4249-a524-431d611c9af6 + alb.ingress.kubernetes.io/group.name: flytesystem + alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/tags: service_instance=production kubernetes.io/ingress.class: alb - nginx.ingress.kubernetes.io/ssl-redirect: "false" + labels: + app: flyteadmin name: flytesystem namespace: flyte spec: @@ -9032,30 +9037,150 @@ spec: - http: paths: - backend: - serviceName: flyteconsole - servicePort: 80 + service: + name: ssl-redirect + port: + name: use-annotation + path: /* + pathType: ImplementationSpecific + - backend: + service: + name: flyteconsole + port: + number: 80 path: /console + pathType: ImplementationSpecific - backend: - serviceName: flyteconsole - servicePort: 80 - path: /__webpack_hmr + service: + name: flyteconsole + port: + number: 80 + path: /console/* + pathType: ImplementationSpecific - backend: - serviceName: flyteadmin - servicePort: 80 - path: /api + service: + name: flyteadmin + port: + number: 80 + path: /api/* + pathType: ImplementationSpecific - backend: - serviceName: flyteadmin - servicePort: 80 + service: + name: flyteadmin + port: + number: 80 path: /healthcheck + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /v1/* + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /openapi/* + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /.well-known/* + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /login + pathType: ImplementationSpecific - backend: - serviceName: flyteadmin - servicePort: 80 - path: /v1 + service: + name: flyteadmin + port: + number: 80 + path: /login/* + pathType: ImplementationSpecific - backend: - serviceName: flyteadmin - servicePort: 81 - path: /flyteidl.service.AdminService + service: + name: flyteadmin + port: + number: 80 + path: /logout + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /logout/* + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /callback + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /callback/* + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /me + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /config + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /config/* + pathType: ImplementationSpecific +--- +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + annotations: + alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' + alb.ingress.kubernetes.io/backend-protocol-version: GRPC + alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-2:111111111111:certificate/e92fefd8-6197-4249-a524-431d611c9af6 + alb.ingress.kubernetes.io/group.name: flytesystem + alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]' + alb.ingress.kubernetes.io/scheme: internet-facing + alb.ingress.kubernetes.io/tags: service_instance=production + kubernetes.io/ingress.class: alb + nginx.ingress.kubernetes.io/ssl-redirect: "false" + labels: + app: flyteadmin + name: flytesystem-grpc + namespace: flyte +spec: + rules: + - http: + paths: - backend: - serviceName: flyteadmin - servicePort: 87 - path: /openapi + service: + name: flyteadmin + port: + number: 81 + path: /flyteidl.service.AdminService/* + pathType: ImplementationSpecific diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index 0d9529e1ba..20b741c4a0 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -9049,7 +9049,7 @@ spec: restartPolicy: Never serviceAccountName: sparkoperator --- -apiVersion: extensions/v1beta1 +apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: @@ -9061,30 +9061,142 @@ spec: - http: paths: - backend: - serviceName: flyteconsole - servicePort: 80 + service: + name: flyteconsole + port: + number: 80 + path: /__webpack_hmr + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 81 + path: /flyteidl.service.AdminService + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 87 + path: /openapi + pathType: ImplementationSpecific + - backend: + service: + name: flyteconsole + port: + number: 80 path: /console + pathType: ImplementationSpecific - backend: - serviceName: flyteconsole - servicePort: 80 - path: /__webpack_hmr + service: + name: flyteconsole + port: + number: 80 + path: /console/* + pathType: ImplementationSpecific - backend: - serviceName: flyteadmin - servicePort: 80 + service: + name: flyteadmin + port: + number: 80 path: /api + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /api/* + pathType: ImplementationSpecific - backend: - serviceName: flyteadmin - servicePort: 80 + service: + name: flyteadmin + port: + number: 80 path: /healthcheck + pathType: ImplementationSpecific - backend: - serviceName: flyteadmin - servicePort: 80 - path: /v1 + service: + name: flyteadmin + port: + number: 80 + path: /v1/* + pathType: ImplementationSpecific - backend: - serviceName: flyteadmin - servicePort: 81 - path: /flyteidl.service.AdminService + service: + name: flyteadmin + port: + number: 80 + path: /openapi/* + pathType: ImplementationSpecific - backend: - serviceName: flyteadmin - servicePort: 87 - path: /openapi + service: + name: flyteadmin + port: + number: 80 + path: /.well-known/* + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /login + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /login/* + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /logout + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /logout/* + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /callback + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /callback/* + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /me + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /config + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /config/* + pathType: ImplementationSpecific diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index 1a40a7ac5a..736454efa8 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -3401,7 +3401,7 @@ spec: serviceAccountName: contour-certgen ttlSecondsAfterFinished: 0 --- -apiVersion: extensions/v1beta1 +apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: @@ -3413,35 +3413,147 @@ spec: - http: paths: - backend: - serviceName: flyteconsole - servicePort: 80 + service: + name: flyteconsole + port: + number: 80 + path: /__webpack_hmr + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 81 + path: /flyteidl.service.AdminService + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 87 + path: /openapi + pathType: ImplementationSpecific + - backend: + service: + name: flyteconsole + port: + number: 80 path: /console + pathType: ImplementationSpecific - backend: - serviceName: flyteconsole - servicePort: 80 - path: /__webpack_hmr + service: + name: flyteconsole + port: + number: 80 + path: /console/* + pathType: ImplementationSpecific - backend: - serviceName: flyteadmin - servicePort: 80 + service: + name: flyteadmin + port: + number: 80 path: /api + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /api/* + pathType: ImplementationSpecific - backend: - serviceName: flyteadmin - servicePort: 80 + service: + name: flyteadmin + port: + number: 80 path: /healthcheck + pathType: ImplementationSpecific - backend: - serviceName: flyteadmin - servicePort: 80 - path: /v1 + service: + name: flyteadmin + port: + number: 80 + path: /v1/* + pathType: ImplementationSpecific - backend: - serviceName: flyteadmin - servicePort: 81 - path: /flyteidl.service.AdminService + service: + name: flyteadmin + port: + number: 80 + path: /openapi/* + pathType: ImplementationSpecific - backend: - serviceName: flyteadmin - servicePort: 87 - path: /openapi + service: + name: flyteadmin + port: + number: 80 + path: /.well-known/* + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /login + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /login/* + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /logout + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /logout/* + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /callback + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /callback/* + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /me + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /config + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /config/* + pathType: ImplementationSpecific --- -apiVersion: extensions/v1beta1 +apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: @@ -3453,6 +3565,9 @@ spec: - http: paths: - backend: - serviceName: minio - servicePort: 9000 + service: + name: minio + port: + number: 9000 path: /minio + pathType: ImplementationSpecific diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index 067d8df2de..0efb4347d3 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -814,47 +814,7 @@ spec: - emptyDir: {} name: postgres-storage --- -apiVersion: extensions/v1beta1 -kind: Ingress -metadata: - annotations: - nginx.ingress.kubernetes.io/ssl-redirect: "false" - name: flytesystem - namespace: flyte -spec: - rules: - - http: - paths: - - backend: - serviceName: flyteconsole - servicePort: 80 - path: /console - - backend: - serviceName: flyteconsole - servicePort: 80 - path: /__webpack_hmr - - backend: - serviceName: flyteadmin - servicePort: 80 - path: /api - - backend: - serviceName: flyteadmin - servicePort: 80 - path: /healthcheck - - backend: - serviceName: flyteadmin - servicePort: 80 - path: /v1 - - backend: - serviceName: flyteadmin - servicePort: 81 - path: /flyteidl.service.AdminService - - backend: - serviceName: flyteadmin - servicePort: 87 - path: /openapi ---- -apiVersion: extensions/v1beta1 +apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: @@ -866,6 +826,9 @@ spec: - http: paths: - backend: - serviceName: minio - servicePort: 9000 + service: + name: minio + port: + number: 9000 path: /minio + pathType: ImplementationSpecific diff --git a/kustomize/base/addons/alb_ingress/README.md b/kustomize/base/addons/alb_ingress/README.md deleted file mode 100644 index 4d6403e0fe..0000000000 --- a/kustomize/base/addons/alb_ingress/README.md +++ /dev/null @@ -1 +0,0 @@ -# :construction: Instructions to deploy ALB Ingress controller diff --git a/kustomize/base/addons/alb_ingress/ingress.yaml b/kustomize/base/addons/alb_ingress/ingress.yaml deleted file mode 100644 index 69ed75f223..0000000000 --- a/kustomize/base/addons/alb_ingress/ingress.yaml +++ /dev/null @@ -1,47 +0,0 @@ -apiVersion: extensions/v1beta1 -kind: Ingress -metadata: - name: "flytesystem" - namespace: "flyte" - annotations: - # TODO ALB can only be used for REST non grpc endpoints - kubernetes.io/ingress.class: alb - alb.ingress.kubernetes.io/tags: service_instance=production - alb.ingress.kubernetes.io/scheme: internet-facing - labels: - app: flyteadmin -spec: - rules: - - http: - paths: - - path: /console - backend: - serviceName: flyteconsole - servicePort: 80 - - path: /console/* - backend: - serviceName: flyteconsole - servicePort: 80 - - path: /api/* - backend: - serviceName: flyteadmin - servicePort: 80 - - path: /healthcheck - backend: - serviceName: flyteadmin - servicePort: 80 - - path: /v1/* - backend: - serviceName: flyteadmin - servicePort: 80 - # NOTE: Port 81 in flyteadmin is the GRPC server port for - # FlyteAdmin. - - path: /flyteidl.service.AdminService/* - backend: - serviceName: flyteadmin - servicePort: 81 - # Port 87 in FlyteAdmin maps to the redoc container. - - path: /openapi/* - backend: - serviceName: flyteadmin - servicePort: 87 diff --git a/kustomize/base/addons/storage/storage.yaml b/kustomize/base/addons/storage/storage.yaml index 2f72d3c9a6..b14b37be81 100644 --- a/kustomize/base/addons/storage/storage.yaml +++ b/kustomize/base/addons/storage/storage.yaml @@ -14,25 +14,25 @@ spec: app: minio spec: volumes: - - name: minio-storage - emptyDir: {} + - name: minio-storage + emptyDir: {} containers: - - image: "minio/minio:RELEASE.2020-12-16T05-05-17Z" - name: minio - env: - - name: MINIO_ACCESS_KEY - value: minio - - name: MINIO_SECRET_KEY - value: miniostorage - args: - - server - - /data - ports: - - containerPort: 9000 + - image: "minio/minio:RELEASE.2020-12-16T05-05-17Z" name: minio - volumeMounts: - - name: minio-storage - mountPath: /data + env: + - name: MINIO_ACCESS_KEY + value: minio + - name: MINIO_SECRET_KEY + value: miniostorage + args: + - server + - /data + ports: + - containerPort: 9000 + name: minio + volumeMounts: + - name: minio-storage + mountPath: /data --- apiVersion: v1 kind: Service @@ -42,11 +42,11 @@ metadata: spec: externalName: minio ports: - - port: 9000 + - port: 9000 selector: app: minio --- -apiVersion: extensions/v1beta1 +apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: minio @@ -55,9 +55,12 @@ metadata: nginx.ingress.kubernetes.io/ssl-redirect: "false" spec: rules: - - http: - paths: - - path: /minio - backend: - serviceName: minio - servicePort: 9000 + - http: + paths: + - path: /minio + pathType: ImplementationSpecific + backend: + service: + name: minio + port: + number: 9000 diff --git a/kustomize/base/ingress/ingress.yaml b/kustomize/base/ingress/ingress.yaml index 431cbede0a..041421259f 100644 --- a/kustomize/base/ingress/ingress.yaml +++ b/kustomize/base/ingress/ingress.yaml @@ -1,4 +1,4 @@ -apiVersion: extensions/v1beta1 +apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: flytesystem @@ -7,38 +7,151 @@ metadata: nginx.ingress.kubernetes.io/ssl-redirect: "false" spec: rules: - - http: - paths: - # NOTE: If you change this, you must update the BASE_URL value in flyteconsole.yaml - - path: /console - backend: - serviceName: flyteconsole - servicePort: 80 - # This is useful only for sandbox mode and should be templatized/removed in non-sandbox environments - - path: /__webpack_hmr - backend: - serviceName: flyteconsole - servicePort: 80 - - path: /api - backend: - serviceName: flyteadmin - servicePort: 80 - - path: /healthcheck - backend: - serviceName: flyteadmin - servicePort: 80 - - path: /v1 - backend: - serviceName: flyteadmin - servicePort: 80 - # NOTE: Port 81 in flyteadmin is the GRPC server port for - # FlyteAdmin. - - path: /flyteidl.service.AdminService - backend: - serviceName: flyteadmin - servicePort: 81 - # Port 87 in FlyteAdmin maps to the redoc container. - - path: /openapi - backend: - serviceName: flyteadmin - servicePort: 87 + - http: + paths: + # This is useful only for sandbox mode and should be templatized/removed in non-sandbox environments + - path: /__webpack_hmr + pathType: ImplementationSpecific + backend: + service: + name: flyteconsole + port: + number: 80 + # NOTE: Port 81 in flyteadmin is the GRPC server port for + # FlyteAdmin. + - path: /flyteidl.service.AdminService + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 81 + # Port 87 in FlyteAdmin maps to the redoc container. + - path: /openapi + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 87 + # NOTE: If you change this, you must update the BASE_URL value in flyteconsole.yaml + - path: /console + pathType: ImplementationSpecific + backend: + service: + name: flyteconsole + port: + number: 80 + - path: /console/* + pathType: ImplementationSpecific + backend: + service: + name: flyteconsole + port: + number: 80 + - path: /api + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /api/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /healthcheck + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /v1/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + # Port 87 in FlyteAdmin maps to the redoc container. + - path: /openapi/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /.well-known/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /login + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /login/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /logout + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /logout/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /callback + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /callback/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /me + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /config + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /config/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 diff --git a/kustomize/base/ingress/kustomization.yaml b/kustomize/base/ingress/kustomization.yaml index 36e8c40ae5..14d8f3a54f 100644 --- a/kustomize/base/ingress/kustomization.yaml +++ b/kustomize/base/ingress/kustomization.yaml @@ -1,2 +1,5 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + resources: -- ingress.yaml + - ingress.yaml diff --git a/kustomize/base/single_cluster/headless/kustomization.yaml b/kustomize/base/single_cluster/headless/kustomization.yaml index 4373a33e41..c63c457102 100644 --- a/kustomize/base/single_cluster/headless/kustomization.yaml +++ b/kustomize/base/single_cluster/headless/kustomization.yaml @@ -3,62 +3,61 @@ kind: Kustomization # All the resources that make up the deployment resources: -# global resources -- ../../namespace -- ../../ingress -- ../../admindeployment -- ../../datacatalog -- ../../wf_crd -- ../../propeller -- ../../adminserviceaccount + # global resources + - ../../namespace + - ../../admindeployment + - ../../datacatalog + - ../../wf_crd + - ../../propeller + - ../../adminserviceaccount configMapGenerator: -# the main admin configmap -- name: flyte-admin-config - files: - - ./config/admin/server.yaml - - ./config/admin/domain.yaml - - ./config/admin/db.yaml - - ./config/admin/cluster_resources.yaml - - ./config/admin/remote_data.yaml - - ./config/admin/task_resource_defaults.yaml - - ./config/common/storage.yaml - - ./config/common/logger.yaml + # the main admin configmap + - name: flyte-admin-config + files: + - ./config/admin/server.yaml + - ./config/admin/domain.yaml + - ./config/admin/db.yaml + - ./config/admin/cluster_resources.yaml + - ./config/admin/remote_data.yaml + - ./config/admin/task_resource_defaults.yaml + - ./config/common/storage.yaml + - ./config/common/logger.yaml -# cluster resource templates -- name: clusterresource-template - files: -# Files are read in alphabetical order. To ensure that we create the namespace first, prefix the file name with "aa". - - ./config/clusterresource-templates/aa_namespace.yaml - - ./config/clusterresource-templates/ab_project-resource-quota.yaml - - ./config/clusterresource-templates/ac_project-copilot-dataconfig.yaml + # cluster resource templates + - name: clusterresource-template + files: + # Files are read in alphabetical order. To ensure that we create the namespace first, prefix the file name with "aa". + - ./config/clusterresource-templates/aa_namespace.yaml + - ./config/clusterresource-templates/ab_project-resource-quota.yaml + - ./config/clusterresource-templates/ac_project-copilot-dataconfig.yaml -# Flyte Propeller Configuration -- name: flyte-propeller-config - files: - - ./config/propeller/core.yaml - - ./config/propeller/admin.yaml - - ./config/propeller/catalog.yaml - - ./config/propeller/resource_manager.yaml - - ./config/propeller/enabled_plugins.yaml - - ./config/propeller/plugins/copilot.yaml - - ./config/propeller/plugins/k8s.yaml - - ./config/common/storage.yaml - - ./config/common/logger.yaml + # Flyte Propeller Configuration + - name: flyte-propeller-config + files: + - ./config/propeller/core.yaml + - ./config/propeller/admin.yaml + - ./config/propeller/catalog.yaml + - ./config/propeller/resource_manager.yaml + - ./config/propeller/enabled_plugins.yaml + - ./config/propeller/plugins/copilot.yaml + - ./config/propeller/plugins/k8s.yaml + - ./config/common/storage.yaml + - ./config/common/logger.yaml -# TODO Flyte Console Configuration -#- name: flyte-console-config -# files: -# - ./config/console.yaml + # TODO Flyte Console Configuration + #- name: flyte-console-config + # files: + # - ./config/console.yaml -- name: datacatalog-config - files: - - ./config/datacatalog/server.yaml - - ./config/datacatalog/db.yaml - - ./config/common/storage.yaml - - ./config/common/logger.yaml + - name: datacatalog-config + files: + - ./config/datacatalog/server.yaml + - ./config/datacatalog/db.yaml + - ./config/common/storage.yaml + - ./config/common/logger.yaml secretGenerator: -- name: db-pass - literals: - - pass.txt="awesomesauce" + - name: db-pass + literals: + - pass.txt="awesomesauce" diff --git a/kustomize/overlays/eks/flyte/ingress/README.md b/kustomize/overlays/eks/flyte/ingress/README.md new file mode 100644 index 0000000000..28fa058659 --- /dev/null +++ b/kustomize/overlays/eks/flyte/ingress/README.md @@ -0,0 +1,5 @@ +# :construction: Instructions to deploy ALB Ingress controller + +Follow instructions here to install ALB Ingress Controller: https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html + +Replace `alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-2:111111111111:certificate/e92fefd8-6197-4249-a524-431d611c9af6` in ingress.yaml and ingress_grpc.yaml with your own SSL cert (that you will create by following ALB Instructions above) diff --git a/kustomize/overlays/eks/flyte/ingress/ingress.yaml b/kustomize/overlays/eks/flyte/ingress/ingress.yaml index e6d36dfa8f..d88fb9b52e 100644 --- a/kustomize/overlays/eks/flyte/ingress/ingress.yaml +++ b/kustomize/overlays/eks/flyte/ingress/ingress.yaml @@ -1,10 +1,143 @@ -apiVersion: extensions/v1beta1 +apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: flytesystem namespace: flyte annotations: - # TODO ALB can only be used for REST non grpc endpoints kubernetes.io/ingress.class: alb alb.ingress.kubernetes.io/tags: service_instance=production alb.ingress.kubernetes.io/scheme: internet-facing + alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' + alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' + + # Instruct ALB Controller to not create multiple load balancers (and hence maintain a single endpoint for both GRPC and Http) + alb.ingress.kubernetes.io/group.name: flytesystem + + # Replace certificate Arn with one deployed to your EKS cluster. Follow instructions in README.md + alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-2:111111111111:certificate/e92fefd8-6197-4249-a524-431d611c9af6 + labels: + app: flyteadmin +spec: + rules: + - http: + paths: + - path: /* + pathType: ImplementationSpecific + backend: + service: + name: ssl-redirect + port: + name: use-annotation + - path: /console + pathType: ImplementationSpecific + backend: + service: + name: flyteconsole + port: + number: 80 + - path: /console/* + pathType: ImplementationSpecific + backend: + service: + name: flyteconsole + port: + number: 80 + - path: /api/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /healthcheck + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /v1/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + # Port 87 in FlyteAdmin maps to the redoc container. + - path: /openapi/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /.well-known/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /login + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /login/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /logout + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /logout/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /callback + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /callback/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /me + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /config + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 + - path: /config/* + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 diff --git a/kustomize/overlays/eks/flyte/ingress/ingress_grpc.yaml b/kustomize/overlays/eks/flyte/ingress/ingress_grpc.yaml new file mode 100644 index 0000000000..f7ea333fb8 --- /dev/null +++ b/kustomize/overlays/eks/flyte/ingress/ingress_grpc.yaml @@ -0,0 +1,31 @@ +--- +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + annotations: + alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' + alb.ingress.kubernetes.io/backend-protocol-version: GRPC + alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]' + alb.ingress.kubernetes.io/scheme: internet-facing + alb.ingress.kubernetes.io/tags: service_instance=production + kubernetes.io/ingress.class: alb + nginx.ingress.kubernetes.io/ssl-redirect: "false" + alb.ingress.kubernetes.io/group.name: flytesystem + + # Replace certificate Arn with one deployed to your EKS cluster. Follow instructions in README.md + alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-2:111111111111:certificate/e92fefd8-6197-4249-a524-431d611c9af6 + labels: + app: flyteadmin + name: flytesystem-grpc + namespace: flyte +spec: + rules: + - http: + paths: + - backend: + service: + name: flyteadmin + port: + number: 81 + path: /flyteidl.service.AdminService/* + pathType: ImplementationSpecific diff --git a/kustomize/overlays/eks/flyte/ingress/kustomization.yaml b/kustomize/overlays/eks/flyte/ingress/kustomization.yaml new file mode 100644 index 0000000000..2cb8f8d97a --- /dev/null +++ b/kustomize/overlays/eks/flyte/ingress/kustomization.yaml @@ -0,0 +1,6 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +resources: + - ingress.yaml + - ingress_grpc.yaml diff --git a/kustomize/overlays/eks/flyte/kustomization.yaml b/kustomize/overlays/eks/flyte/kustomization.yaml index b5e9e0173f..0da9ffcee5 100644 --- a/kustomize/overlays/eks/flyte/kustomization.yaml +++ b/kustomize/overlays/eks/flyte/kustomization.yaml @@ -5,60 +5,60 @@ kind: Kustomization namespace: flyte bases: -- ../../../base/single_cluster/complete + - ../../../base/single_cluster/complete + - ingress resources: -- datacatalog/service.yaml -- propeller/service.yaml + - datacatalog/service.yaml + - propeller/service.yaml patchesStrategicMerge: -- admin/deployment.yaml -- admin/service.yaml -- admin/serviceaccount.yaml -- datacatalog/deployment.yaml -- propeller/deployment.yaml -- propeller/serviceaccount.yaml -- ingress/ingress.yaml -- console/service.yaml + - admin/deployment.yaml + - admin/service.yaml + - admin/serviceaccount.yaml + - datacatalog/deployment.yaml + - propeller/deployment.yaml + - propeller/serviceaccount.yaml + - console/service.yaml # Files are read in alphabetical order. To ensure that we create the namespace first, prefix the file name with "aa". configMapGenerator: -- behavior: merge - files: - - ./config/admin/db.yaml - - ./config/admin/remote_data.yaml - - ./config/admin/task_resource_defaults.yaml - - ./config/common/storage.yaml - name: flyte-admin-config -- behavior: merge - files: - - ./config/clusterresource-templates/ac_project-copilot-dataconfig.yaml - - ./config/clusterresource-templates/ad_spark-role.yaml - - ./config/clusterresource-templates/ae_spark-service-account.yaml - - ./config/clusterresource-templates/af_spark-role-binding.yaml - name: clusterresource-template -- behavior: merge - files: - - ./config/propeller/core.yaml - - ./config/propeller/enabled_plugins.yaml - - ./config/propeller/plugins/catalog_cache.yaml - - ./config/propeller/plugins/k8s.yaml - - ./config/propeller/plugins/qubole.yaml - - ./config/propeller/plugins/spark.yaml - - ./config/propeller/plugins/task_logs.yaml - - ./config/common/storage.yaml - name: flyte-propeller-config -- behavior: merge - files: - - ./config/common/storage.yaml - - ./config/datacatalog/db.yaml - name: datacatalog-config + - behavior: merge + files: + - ./config/admin/db.yaml + - ./config/admin/remote_data.yaml + - ./config/admin/task_resource_defaults.yaml + - ./config/common/storage.yaml + name: flyte-admin-config + - behavior: merge + files: + - ./config/clusterresource-templates/ac_project-copilot-dataconfig.yaml + - ./config/clusterresource-templates/ad_spark-role.yaml + - ./config/clusterresource-templates/ae_spark-service-account.yaml + - ./config/clusterresource-templates/af_spark-role-binding.yaml + name: clusterresource-template + - behavior: merge + files: + - ./config/propeller/core.yaml + - ./config/propeller/enabled_plugins.yaml + - ./config/propeller/plugins/catalog_cache.yaml + - ./config/propeller/plugins/k8s.yaml + - ./config/propeller/plugins/qubole.yaml + - ./config/propeller/plugins/spark.yaml + - ./config/propeller/plugins/task_logs.yaml + - ./config/common/storage.yaml + name: flyte-propeller-config + - behavior: merge + files: + - ./config/common/storage.yaml + - ./config/datacatalog/db.yaml + name: datacatalog-config ########### # Use this to record the password for the Google CloudSQL Database secretGenerator: -- name: db-pass - behavior: merge - literals: - - pass.txt="yourpassword" + - name: db-pass + behavior: merge + literals: + - pass.txt="yourpassword" diff --git a/kustomize/overlays/eks/kustomization.yaml b/kustomize/overlays/eks/kustomization.yaml index d9ddc376c7..a0b5600f02 100644 --- a/kustomize/overlays/eks/kustomization.yaml +++ b/kustomize/overlays/eks/kustomization.yaml @@ -14,8 +14,6 @@ bases: # Add node ports for ease of use locally - ../../base/operators/spark - ../../base/operators/kfoperators/pytorch - # TODO Fix deployment of alb_ingress controller - #- ../../base/addons/alb_ingress # Optional dependency - ../../base/addons/redis diff --git a/kustomize/overlays/gcp/kustomization.yaml b/kustomize/overlays/gcp/kustomization.yaml index 9ac428bf1d..abdaad34af 100644 --- a/kustomize/overlays/gcp/kustomization.yaml +++ b/kustomize/overlays/gcp/kustomization.yaml @@ -12,6 +12,7 @@ bases: # This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) # Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's # Add node ports for ease of use locally + - ../../base/ingress - ../../base/operators/spark - ../../base/operators/kfoperators/pytorch - ../../base/addons/cloudsqlproxy diff --git a/kustomize/overlays/sandbox/kustomization.yaml b/kustomize/overlays/sandbox/kustomization.yaml index c23d6a70e1..5316bcdcd8 100644 --- a/kustomize/overlays/sandbox/kustomization.yaml +++ b/kustomize/overlays/sandbox/kustomization.yaml @@ -12,6 +12,7 @@ bases: # This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache) # Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's # Add node ports for ease of use locally + - ../../base/ingress - ../../base/addons/database - ../../base/addons/kubernetes_dashboard - ../../base/addons/storage From 7f0d9059af69ce9e9c6d150215e7994df8b39cde Mon Sep 17 00:00:00 2001 From: Katrina Rogan Date: Mon, 15 Mar 2021 16:20:19 -0700 Subject: [PATCH 26/74] Update deployments with latest k8s library deps (#826) Signed-off-by: Haytham Abuelfutuh --- deployment/eks/flyte_generated.yaml | 29 +++++++------------ deployment/gcp/flyte_generated.yaml | 29 +++++++------------ deployment/sandbox/flyte_generated.yaml | 29 +++++++------------ deployment/test/flyte_generated.yaml | 26 +++++++---------- .../admindeployment/clustersync/cron.yaml | 2 +- .../base/admindeployment/deployment.yaml | 18 ++++++------ kustomize/base/datacatalog/deployment.yaml | 10 +++---- kustomize/base/propeller/deployment.yaml | 4 +-- kustomize/overlays/eks/kustomization.yaml | 8 ++--- kustomize/overlays/gcp/kustomization.yaml | 8 ++--- .../sandbox/flyte/admin/deployment.yaml | 9 ++---- kustomize/overlays/sandbox/kustomization.yaml | 8 ++--- .../overlays/test/flyte/admin/deployment.yaml | 6 ++-- kustomize/overlays/test/kustomization.yaml | 8 ++--- 14 files changed, 82 insertions(+), 112 deletions(-) diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index da4a775757..dce796fcae 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -8525,16 +8525,15 @@ spec: labels: app: datacatalog app.kubernetes.io/name: datacatalog - app.kubernetes.io/version: 0.2.2 + app.kubernetes.io/version: 0.3.0 spec: containers: - command: - datacatalog - - --logtostderr - --config - /etc/datacatalog/config/*.yaml - serve - image: ghcr.io/lyft/datacatalog:v0.2.130 + image: ghcr.io/flyteorg/datacatalog:v0.3.0 imagePullPolicy: IfNotPresent name: datacatalog ports: @@ -8553,12 +8552,11 @@ spec: initContainers: - command: - datacatalog - - --logtostderr - --config - /etc/datacatalog/config/*.yaml - migrate - run - image: ghcr.io/lyft/datacatalog:v0.2.130 + image: ghcr.io/flyteorg/datacatalog:v0.3.0 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8598,16 +8596,15 @@ spec: labels: app: flyteadmin app.kubernetes.io/name: flyteadmin - app.kubernetes.io/version: 0.3.29 + app.kubernetes.io/version: 0.3.38 spec: containers: - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -8648,12 +8645,11 @@ spec: initContainers: - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8663,7 +8659,6 @@ spec: name: db-pass - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - migrate @@ -8671,7 +8666,7 @@ spec: - flytesnacks - flytetester - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -8681,12 +8676,11 @@ spec: name: db-pass - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -8764,7 +8758,7 @@ spec: labels: app: flytepropeller app.kubernetes.io/name: flytepropeller - app.kubernetes.io/version: 0.5.13 + app.kubernetes.io/version: 0.7.1 spec: containers: - args: @@ -8777,7 +8771,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.6.2 + image: ghcr.io/flyteorg/flytepropeller:v0.7.1 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -8960,12 +8954,11 @@ spec: containers: - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index 20b741c4a0..2faab6b203 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -8557,16 +8557,15 @@ spec: labels: app: datacatalog app.kubernetes.io/name: datacatalog - app.kubernetes.io/version: 0.2.2 + app.kubernetes.io/version: 0.3.0 spec: containers: - command: - datacatalog - - --logtostderr - --config - /etc/datacatalog/config/*.yaml - serve - image: ghcr.io/lyft/datacatalog:v0.2.130 + image: ghcr.io/flyteorg/datacatalog:v0.3.0 imagePullPolicy: IfNotPresent name: datacatalog ports: @@ -8585,12 +8584,11 @@ spec: initContainers: - command: - datacatalog - - --logtostderr - --config - /etc/datacatalog/config/*.yaml - migrate - run - image: ghcr.io/lyft/datacatalog:v0.2.130 + image: ghcr.io/flyteorg/datacatalog:v0.3.0 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8630,16 +8628,15 @@ spec: labels: app: flyteadmin app.kubernetes.io/name: flyteadmin - app.kubernetes.io/version: 0.3.29 + app.kubernetes.io/version: 0.3.38 spec: containers: - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -8680,12 +8677,11 @@ spec: initContainers: - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8695,7 +8691,6 @@ spec: name: db-pass - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - migrate @@ -8703,7 +8698,7 @@ spec: - flytesnacks - flytetester - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -8713,12 +8708,11 @@ spec: name: db-pass - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -8796,7 +8790,7 @@ spec: labels: app: flytepropeller app.kubernetes.io/name: flytepropeller - app.kubernetes.io/version: 0.5.13 + app.kubernetes.io/version: 0.7.1 spec: containers: - args: @@ -8809,7 +8803,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.6.2 + image: ghcr.io/flyteorg/flytepropeller:v0.7.1 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -8992,12 +8986,11 @@ spec: containers: - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index 736454efa8..f0dde874e2 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -2646,16 +2646,15 @@ spec: labels: app: datacatalog app.kubernetes.io/name: datacatalog - app.kubernetes.io/version: 0.2.2 + app.kubernetes.io/version: 0.3.0 spec: containers: - command: - datacatalog - - --logtostderr - --config - /etc/datacatalog/config/*.yaml - serve - image: ghcr.io/lyft/datacatalog:v0.2.130 + image: ghcr.io/flyteorg/datacatalog:v0.3.0 imagePullPolicy: IfNotPresent name: datacatalog ports: @@ -2669,12 +2668,11 @@ spec: initContainers: - command: - datacatalog - - --logtostderr - --config - /etc/datacatalog/config/*.yaml - migrate - run - image: ghcr.io/lyft/datacatalog:v0.2.130 + image: ghcr.io/flyteorg/datacatalog:v0.3.0 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -2714,16 +2712,15 @@ spec: labels: app: flyteadmin app.kubernetes.io/name: flyteadmin - app.kubernetes.io/version: 0.3.29 + app.kubernetes.io/version: 0.3.38 spec: containers: - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -2770,12 +2767,11 @@ spec: name: check-db-ready - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -2785,14 +2781,13 @@ spec: name: db-pass - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - migrate - seed-projects - flytesnacks - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -2802,12 +2797,11 @@ spec: name: db-pass - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -2885,7 +2879,7 @@ spec: labels: app: flytepropeller app.kubernetes.io/name: flytepropeller - app.kubernetes.io/version: 0.5.13 + app.kubernetes.io/version: 0.7.1 spec: containers: - args: @@ -2898,7 +2892,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.6.2 + image: ghcr.io/flyteorg/flytepropeller:v0.7.1 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -3199,12 +3193,11 @@ spec: containers: - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index 0efb4347d3..2028979b20 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -517,16 +517,15 @@ spec: labels: app: datacatalog app.kubernetes.io/name: datacatalog - app.kubernetes.io/version: 0.2.2 + app.kubernetes.io/version: 0.3.0 spec: containers: - command: - datacatalog - - --logtostderr - --config - /etc/datacatalog/config/*.yaml - serve - image: ghcr.io/lyft/datacatalog:v0.2.130 + image: ghcr.io/flyteorg/datacatalog:v0.3.0 imagePullPolicy: IfNotPresent name: datacatalog ports: @@ -540,12 +539,11 @@ spec: initContainers: - command: - datacatalog - - --logtostderr - --config - /etc/datacatalog/config/*.yaml - migrate - run - image: ghcr.io/lyft/datacatalog:v0.2.130 + image: ghcr.io/flyteorg/datacatalog:v0.3.0 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -585,16 +583,15 @@ spec: labels: app: flyteadmin app.kubernetes.io/name: flyteadmin - app.kubernetes.io/version: 0.3.29 + app.kubernetes.io/version: 0.3.38 spec: containers: - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -641,12 +638,11 @@ spec: name: check-db-ready - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -656,14 +652,13 @@ spec: name: db-pass - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - migrate - seed-projects - flytetester - flytesnacks - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -673,12 +668,11 @@ spec: name: db-pass - command: - flyteadmin - - --logtostderr - --config - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.3.37 + image: ghcr.io/flyteorg/flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -722,7 +716,7 @@ spec: labels: app: flytepropeller app.kubernetes.io/name: flytepropeller - app.kubernetes.io/version: 0.5.13 + app.kubernetes.io/version: 0.7.1 spec: containers: - args: @@ -735,7 +729,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.6.2 + image: ghcr.io/flyteorg/flytepropeller:v0.7.1 imagePullPolicy: IfNotPresent name: flytepropeller ports: diff --git a/kustomize/base/admindeployment/clustersync/cron.yaml b/kustomize/base/admindeployment/clustersync/cron.yaml index 7651a93e0b..b23ffaf6d5 100644 --- a/kustomize/base/admindeployment/clustersync/cron.yaml +++ b/kustomize/base/admindeployment/clustersync/cron.yaml @@ -14,7 +14,7 @@ spec: - name: sync-cluster-resources image: flyteadmin:v0.3.29 imagePullPolicy: IfNotPresent - command: ["flyteadmin", "--logtostderr", "--config", "/etc/flyte/config/*.yaml", "clusterresource", "sync"] + command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "clusterresource", "sync"] volumeMounts: - name: resource-templates mountPath: /etc/flyte/clusterresource/templates diff --git a/kustomize/base/admindeployment/deployment.yaml b/kustomize/base/admindeployment/deployment.yaml index b7c476ccff..1c455774ab 100644 --- a/kustomize/base/admindeployment/deployment.yaml +++ b/kustomize/base/admindeployment/deployment.yaml @@ -16,7 +16,7 @@ spec: labels: app: flyteadmin app.kubernetes.io/name: flyteadmin - app.kubernetes.io/version: 0.3.29 + app.kubernetes.io/version: 0.3.38 annotations: prometheus.io/scrape: "true" prometheus.io/port: "10254" @@ -37,9 +37,9 @@ spec: secretName: db-pass initContainers: - name: run-migrations - image: flyteadmin:v0.3.29 + image: flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent - command: ["flyteadmin", "--logtostderr", "--config", "/etc/flyte/config/*.yaml", "migrate", "run"] + command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "migrate", "run"] volumeMounts: - name: config-volume mountPath: /etc/flyte/config @@ -47,9 +47,9 @@ spec: mountPath: /etc/db # Optional, These just seed the project - TODO move them to only - name: seed-projects - image: flyteadmin:v0.3.29 + image: flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent - command: ["flyteadmin", "--logtostderr", "--config", "/etc/flyte/config/*.yaml", + command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "migrate", "seed-projects", "flytesnacks", "flytetester", "flyteexamples"] volumeMounts: - name: config-volume @@ -57,9 +57,9 @@ spec: - name: db-pass mountPath: /etc/db - name: sync-cluster-resources - image: flyteadmin:v0.3.29 + image: flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent - command: ["flyteadmin", "--logtostderr", "--config", "/etc/flyte/config/*.yaml", "clusterresource", "sync"] + command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "clusterresource", "sync"] volumeMounts: - name: resource-templates mountPath: /etc/flyte/clusterresource/templates @@ -69,9 +69,9 @@ spec: mountPath: /etc/db containers: - name: flyteadmin - image: flyteadmin:v0.3.29 + image: flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent - command: ["flyteadmin", "--logtostderr", "--config", "/etc/flyte/config/*.yaml", "serve"] + command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "serve"] ports: - containerPort: 8088 - containerPort: 8089 diff --git a/kustomize/base/datacatalog/deployment.yaml b/kustomize/base/datacatalog/deployment.yaml index 11d7febc0c..ebed3d6708 100644 --- a/kustomize/base/datacatalog/deployment.yaml +++ b/kustomize/base/datacatalog/deployment.yaml @@ -15,7 +15,7 @@ spec: labels: app: datacatalog app.kubernetes.io/name: datacatalog - app.kubernetes.io/version: 0.2.2 + app.kubernetes.io/version: 0.3.0 annotations: prometheus.io/scrape: "true" prometheus.io/port: "10254" @@ -33,9 +33,9 @@ spec: secretName: db-pass initContainers: - name: run-migrations - image: datacatalog:v0.2.2 + image: datacatalog:v0.3.0 imagePullPolicy: IfNotPresent - command: ["datacatalog", "--logtostderr", "--config", "/etc/datacatalog/config/*.yaml", "migrate", "run"] + command: ["datacatalog", "--config", "/etc/datacatalog/config/*.yaml", "migrate", "run"] volumeMounts: - name: config-volume mountPath: /etc/datacatalog/config @@ -43,9 +43,9 @@ spec: mountPath: /etc/db containers: - name: datacatalog - image: datacatalog:v0.2.2 + image: datacatalog:v0.3.0 imagePullPolicy: IfNotPresent - command: ["datacatalog", "--logtostderr", "--config", "/etc/datacatalog/config/*.yaml", "serve"] + command: ["datacatalog", "--config", "/etc/datacatalog/config/*.yaml", "serve"] ports: - containerPort: 8088 - containerPort: 8089 diff --git a/kustomize/base/propeller/deployment.yaml b/kustomize/base/propeller/deployment.yaml index da0b1179ed..e3e33240b8 100644 --- a/kustomize/base/propeller/deployment.yaml +++ b/kustomize/base/propeller/deployment.yaml @@ -15,7 +15,7 @@ spec: labels: app: flytepropeller app.kubernetes.io/name: flytepropeller - app.kubernetes.io/version: 0.5.13 + app.kubernetes.io/version: 0.7.1 annotations: prometheus.io/scrape: "true" prometheus.io/port: "10254" @@ -28,7 +28,7 @@ spec: name: flyte-propeller-config containers: - name: flytepropeller - image: flytepropeller:v0.5.13 + image: flytepropeller:v0.7.1 command: - flytepropeller args: diff --git a/kustomize/overlays/eks/kustomization.yaml b/kustomize/overlays/eks/kustomization.yaml index a0b5600f02..e0ab501cdd 100644 --- a/kustomize/overlays/eks/kustomization.yaml +++ b/kustomize/overlays/eks/kustomization.yaml @@ -21,7 +21,7 @@ bases: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.3.37 # override the tag + newTag: v0.3.38 # override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -29,9 +29,9 @@ images: newName: ghcr.io/flyteorg/flyteconsole # override the namep # Flyte DataCatalog - name: datacatalog # match images with this name - newTag: v0.2.130 # override the tag - newName: ghcr.io/lyft/datacatalog # override the name + newTag: v0.3.0 # override the tag + newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.6.2 # override the tag + newTag: v0.7.1 # override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name diff --git a/kustomize/overlays/gcp/kustomization.yaml b/kustomize/overlays/gcp/kustomization.yaml index abdaad34af..acfcec1508 100644 --- a/kustomize/overlays/gcp/kustomization.yaml +++ b/kustomize/overlays/gcp/kustomization.yaml @@ -23,7 +23,7 @@ bases: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.3.37 # override the tag + newTag: v0.3.38 # override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -31,9 +31,9 @@ images: newName: ghcr.io/flyteorg/flyteconsole # override the namep # Flyte DataCatalog - name: datacatalog # match images with this name - newTag: v0.2.130 # override the tag - newName: ghcr.io/lyft/datacatalog # override the name + newTag: v0.3.0 # override the tag + newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.6.2 # override the tag + newTag: v0.7.1 # override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name diff --git a/kustomize/overlays/sandbox/flyte/admin/deployment.yaml b/kustomize/overlays/sandbox/flyte/admin/deployment.yaml index 2f02e350e7..33f39ab46b 100644 --- a/kustomize/overlays/sandbox/flyte/admin/deployment.yaml +++ b/kustomize/overlays/sandbox/flyte/admin/deployment.yaml @@ -20,12 +20,11 @@ spec: do echo waiting for database; sleep 2; done;", ] - name: run-migrations - image: flyteadmin:v0.3.29 + image: flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent command: [ "flyteadmin", - "--logtostderr", "--config", "/etc/flyte/config/*.yaml", "migrate", @@ -35,12 +34,11 @@ spec: - name: config-volume mountPath: /etc/flyte/config - name: seed-projects - image: flyteadmin:v0.3.29 + image: flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent command: [ "flyteadmin", - "--logtostderr", "--config", "/etc/flyte/config/*.yaml", "migrate", @@ -52,12 +50,11 @@ spec: - name: config-volume mountPath: /etc/flyte/config - name: sync-cluster-resources - image: flyteadmin:v0.3.29 + image: flyteadmin:v0.3.38 imagePullPolicy: IfNotPresent command: [ "flyteadmin", - "--logtostderr", "--config", "/etc/flyte/config/*.yaml", "clusterresource", diff --git a/kustomize/overlays/sandbox/kustomization.yaml b/kustomize/overlays/sandbox/kustomization.yaml index 5316bcdcd8..59f2f9e5ca 100644 --- a/kustomize/overlays/sandbox/kustomization.yaml +++ b/kustomize/overlays/sandbox/kustomization.yaml @@ -25,7 +25,7 @@ resources: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.3.37 # override the tag + newTag: v0.3.38 # override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -33,11 +33,11 @@ images: newName: ghcr.io/flyteorg/flyteconsole # override the namep # Flyte DataCatalog - name: datacatalog # match images with this name - newTag: v0.2.130 # override the tag - newName: ghcr.io/lyft/datacatalog # override the name + newTag: v0.3.0 # override the tag + newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.6.2 # override the tag + newTag: v0.7.1 # override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Override postgres image to use alpine based (rather smaller) docker image - name: postgres diff --git a/kustomize/overlays/test/flyte/admin/deployment.yaml b/kustomize/overlays/test/flyte/admin/deployment.yaml index c6f7e6cab4..b4b9d59183 100644 --- a/kustomize/overlays/test/flyte/admin/deployment.yaml +++ b/kustomize/overlays/test/flyte/admin/deployment.yaml @@ -19,7 +19,7 @@ spec: - name: run-migrations image: flyteadmin:v0.3.5 imagePullPolicy: IfNotPresent - command: ["flyteadmin", "--logtostderr", "--config", "/etc/flyte/config/*.yaml", + command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "migrate", "run"] volumeMounts: - name: config-volume @@ -27,7 +27,7 @@ spec: - name: seed-projects image: flyteadmin:v0.3.5 imagePullPolicy: IfNotPresent - command: ["flyteadmin", "--logtostderr", "--config", "/etc/flyte/config/*.yaml", + command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "migrate", "seed-projects", "flytetester", "flytesnacks"] volumeMounts: - name: config-volume @@ -35,7 +35,7 @@ spec: - name: sync-cluster-resources image: flyteadmin:v0.3.5 imagePullPolicy: IfNotPresent - command: ["flyteadmin", "--logtostderr", "--config", "/etc/flyte/config/*.yaml", "clusterresource", "sync"] + command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "clusterresource", "sync"] volumeMounts: - name: resource-templates mountPath: /etc/flyte/clusterresource/templates diff --git a/kustomize/overlays/test/kustomization.yaml b/kustomize/overlays/test/kustomization.yaml index 9e7fe6f8b4..313dffa35e 100644 --- a/kustomize/overlays/test/kustomization.yaml +++ b/kustomize/overlays/test/kustomization.yaml @@ -19,7 +19,7 @@ bases: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.3.37 # override the tag + newTag: v0.3.38 # override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -27,11 +27,11 @@ images: newName: ghcr.io/flyteorg/flyteconsole # override the namep # Flyte DataCatalog - name: datacatalog # match images with this name - newTag: v0.2.130 # override the tag - newName: ghcr.io/lyft/datacatalog # override the name + newTag: v0.3.0 # override the tag + newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.6.2 # override the tag + newTag: v0.7.1 # override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Override postgres image to use alpine based (rather smaller) docker image - name: postgres From 0f577383e8d9e798e4e385a6ff2b382cd9684310 Mon Sep 17 00:00:00 2001 From: Niels Bantilan Date: Wed, 17 Mar 2021 18:13:55 -0400 Subject: [PATCH 27/74] streamline flyte docs (#836) Signed-off-by: cosmicBboy Co-authored-by: cosmicBboy Signed-off-by: Haytham Abuelfutuh --- rsts/community/compare.rst | 3 +++ rsts/conf.py | 6 ++++- rsts/faq/index.rst | 12 ---------- .../first_example.rst | 8 +++---- .../first_run.rst | 8 +++---- rsts/getting_started/learn.rst | 8 +++++++ .../run_on_flyte.rst | 14 ++++++------ rsts/howto/enable_backend_plugin.rst | 2 +- rsts/howto/execute_single_task.rst | 2 +- rsts/howto/execute_workflow.rst | 6 ++++- rsts/{faq => howto}/gcp.rst | 6 ++--- rsts/howto/index.rst | 13 ++++------- rsts/howto/install_sdk.rst | 4 +++- rsts/howto/interruptible.rst | 4 +++- rsts/howto/monitoring/index.rst | 4 ++-- rsts/howto/multi_cluster/index.rst | 6 ++--- rsts/howto/new_project.rst | 9 ++++++-- rsts/howto/performance/index.rst | 4 ++++ rsts/howto/productionize/index.rst | 2 +- rsts/howto/resource_manager/index.rst | 5 ++++- rsts/howto/resource_quota.rst | 4 ++++ rsts/howto/sandbox.rst | 4 ++-- rsts/index.rst | 20 ++++++++--------- rsts/plugins/aws/athena.rst | 6 ++++- rsts/plugins/aws/sagemaker.rst | 6 ++++- rsts/plugins/extend/flyte_backend.rst | 22 ++++++++++++++----- rsts/plugins/hive.rst | 6 ++++- rsts/plugins/index.rst | 11 ++++------ rsts/plugins/kubeflow/pytorch_operator.rst | 6 ++++- rsts/plugins/kubeflow/tensorflow_operator.rst | 6 ++++- rsts/plugins/pandera.rst | 6 ++++- rsts/plugins/papermill.rst | 6 ++++- rsts/plugins/pod.rst | 6 ++++- rsts/plugins/sqlite3.rst | 6 ++++- rsts/tutorials/learn.rst | 10 --------- 35 files changed, 153 insertions(+), 98 deletions(-) delete mode 100644 rsts/faq/index.rst rename rsts/{tutorials => getting_started}/first_example.rst (92%) rename rsts/{tutorials => getting_started}/first_run.rst (86%) create mode 100644 rsts/getting_started/learn.rst rename rsts/{tutorials => getting_started}/run_on_flyte.rst (90%) rename rsts/{faq => howto}/gcp.rst (84%) delete mode 100644 rsts/tutorials/learn.rst diff --git a/rsts/community/compare.rst b/rsts/community/compare.rst index 4323ecaee8..b91540c15f 100644 --- a/rsts/community/compare.rst +++ b/rsts/community/compare.rst @@ -4,3 +4,6 @@ Compare Flyte to other products ################################### +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/conf.py b/rsts/conf.py index 17b30b2d5f..8317a22730 100644 --- a/rsts/conf.py +++ b/rsts/conf.py @@ -119,12 +119,16 @@ "globaltoc_collapse": False, # If True, show hidden TOC entries "globaltoc_includehidden": False, + # don't include home link in breadcrumb bar, since it's included + # in the nav_links key below. + "master_doc": False, + # custom nav in breadcrumb bar "nav_links": [ {"href": "index", "internal": True, "title": "Flyte Docs"}, { "href": "https://flytecookbook.readthedocs.io", "internal": False, - "title": "Flyte Tutorials", + "title": "Flytekit Tutorials", }, ], } diff --git a/rsts/faq/index.rst b/rsts/faq/index.rst deleted file mode 100644 index 2c0141919e..0000000000 --- a/rsts/faq/index.rst +++ /dev/null @@ -1,12 +0,0 @@ -.. _faqs: - -###### -FAQs -###### - - -.. toctree:: - :maxdepth: 2 - :name: faqstoc - - gcp \ No newline at end of file diff --git a/rsts/tutorials/first_example.rst b/rsts/getting_started/first_example.rst similarity index 92% rename from rsts/tutorials/first_example.rst rename to rsts/getting_started/first_example.rst index 2bcc5c5c09..baffaa8211 100644 --- a/rsts/tutorials/first_example.rst +++ b/rsts/getting_started/first_example.rst @@ -1,8 +1,8 @@ -.. _tutorials-getting-started-first-example: +.. _getting-started-first-example: -###################################### -Write your first Flyte workflow -###################################### +############################### +Write your First Flyte Workflow +############################### By the end of this getting started guide you'll be familiar with how easy it is to author a Flyte workflow and run it locally. diff --git a/rsts/tutorials/first_run.rst b/rsts/getting_started/first_run.rst similarity index 86% rename from rsts/tutorials/first_run.rst rename to rsts/getting_started/first_run.rst index 1a1ea67e6a..a8ed0e0627 100644 --- a/rsts/tutorials/first_run.rst +++ b/rsts/getting_started/first_run.rst @@ -1,10 +1,10 @@ .. currentmodule:: firstrun -.. _flyte-tutorials-firstrun: +.. _getting-started-firstrun: -############################################ -Getting Started With Flyte -############################################ +########################### +Start a Local Flyte Backend +########################### .. rubric:: Estimated time to complete: 2 minutes. diff --git a/rsts/getting_started/learn.rst b/rsts/getting_started/learn.rst new file mode 100644 index 0000000000..dad96a1142 --- /dev/null +++ b/rsts/getting_started/learn.rst @@ -0,0 +1,8 @@ +.. _getting-started-learn: + +############################## +Next Steps: Flytekit Tutorials +############################## + +Flytekit provides programmatic access to Flyte's functionality, with SDK support in Python and Java. +The best way to see the capabilities of Flyte is by trying out the `Flytekit Tutorials `__. diff --git a/rsts/tutorials/run_on_flyte.rst b/rsts/getting_started/run_on_flyte.rst similarity index 90% rename from rsts/tutorials/run_on_flyte.rst rename to rsts/getting_started/run_on_flyte.rst index 215c0654b7..6aed571ed3 100644 --- a/rsts/tutorials/run_on_flyte.rst +++ b/rsts/getting_started/run_on_flyte.rst @@ -1,8 +1,8 @@ -.. _tutorials-getting-started-run-on-flyte: +.. _getting-started-run-on-flyte: -###################################### +##################################### Run Your Workflow on a Flyte Backend -###################################### +##################################### ************************ Installing Flyte Locally @@ -16,13 +16,13 @@ workflows against this deployment. (The tabs below have an option to install Fly Prerequisites ============= -Make Sure you have followed :ref:`flyte-tutorials-firstrun` and have a running docker container and can access FlyteConsole on http://localhost:30081/console. +Make Sure you have followed :ref:`getting-started-firstrun` and have a running docker container and can access FlyteConsole on http://localhost:30081/console. -.. tip:: To check if your flyte-sandbox container is running you can run ``docker ps`` and it should show image ``ghcr.io/flyteorg/flyte-sandbox `` running +.. tip:: To check if your flyte-sandbox container is running you can run ``docker ps`` and it should show image ``ghcr.io/flyteorg/flyte-sandbox`` running .. tip:: If you prefer using k3d, Minikube, docker for mac, or a hosted Kubernetes cluster like AWS-EKS, GCP-GKE, Azure Kubernetes refer to :ref:`howto-sandbox`. It is recommended that you use a simple Docker based approach when you are first getting started with Flyte. -.. _tutorials-run-flyte-laptop: +.. _getting-started-run-flyte-laptop: **************************** Running your Flyte Workflows @@ -35,7 +35,7 @@ Registration is the process of shipping your code to Flyte backend. This creates Register your workflows ----------------------- -From within root directory of ``flyteexamples`` you created :ref:`previously ` +From within root directory of ``flyteexamples`` you created :ref:`previously ` feel free to make any changes and then register :: FLYTE_AWS_ENDPOINT=http://localhost:30084/ FLYTE_AWS_ACCESS_KEY_ID=minio \ diff --git a/rsts/howto/enable_backend_plugin.rst b/rsts/howto/enable_backend_plugin.rst index c3b57c59e9..b5f1d054de 100644 --- a/rsts/howto/enable_backend_plugin.rst +++ b/rsts/howto/enable_backend_plugin.rst @@ -20,7 +20,7 @@ Ok, How do I enable the backend plugins? To enable a backend plugin you have to add the ``ID`` of the plugin to the enabled plugins list. The ``enabled-plugins`` is available under the ``tasks > task-plugins`` section of FlytePropeller's configuration. The `plugin configuration structure is defined here `_. An example of the config follows, -.. rli:: https://raw.githubusercontent.com/flyteorg/flyte/master/kustomize/overlays/sandbox/config/propeller/enabled_plugins.yaml +.. rli:: https://raw.githubusercontent.com/flyteorg/flyte/master/kustomize/overlays/sandbox/flyte/config/propeller/enabled_plugins.yaml :language: yaml How do I find the ``ID`` of the backend plugin? diff --git a/rsts/howto/execute_single_task.rst b/rsts/howto/execute_single_task.rst index ffbabf05c6..03b5c876f4 100644 --- a/rsts/howto/execute_single_task.rst +++ b/rsts/howto/execute_single_task.rst @@ -1,7 +1,7 @@ .. _howto_exec_single_task: #################################### -How Do I Execute a single task? +How do I execute a single task? #################################### diff --git a/rsts/howto/execute_workflow.rst b/rsts/howto/execute_workflow.rst index b641192d36..eee7f9179d 100644 --- a/rsts/howto/execute_workflow.rst +++ b/rsts/howto/execute_workflow.rst @@ -1,5 +1,9 @@ .. _howto_exec_workflow: #################################### -How Do I Execute a Workflow? +How do I execute a workflow? #################################### + +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/faq/gcp.rst b/rsts/howto/gcp.rst similarity index 84% rename from rsts/faq/gcp.rst rename to rsts/howto/gcp.rst index f1d627c6d6..a8540fefa0 100644 --- a/rsts/faq/gcp.rst +++ b/rsts/howto/gcp.rst @@ -1,8 +1,8 @@ .. _faq_gcp: -####################### -Google Cloud Platform -####################### +############################################## +How do I Use Flyte with Google Cloud Platform? +############################################## I tried to run examples, but task fails with 401 error? ------------------------------------------------------- diff --git a/rsts/howto/index.rst b/rsts/howto/index.rst index b66cdc53d0..77e85c719a 100644 --- a/rsts/howto/index.rst +++ b/rsts/howto/index.rst @@ -1,8 +1,8 @@ .. _howto: -######################## -How Do I...? -######################## +########################### +Frequently Asked Questions +########################### .. toctree:: :maxdepth: 1 @@ -31,12 +31,7 @@ How Do I...? labels_annotations notifications serviceaccount + gcp .. _howto_extend: - -======================= -How do I Extend flyte? -======================= - -Flyte was designed to be extensible. The section - :ref:`plugins`- dives into more details on extending Flyte and also provides available examples. \ No newline at end of file diff --git a/rsts/howto/install_sdk.rst b/rsts/howto/install_sdk.rst index 996aa0c9cb..111fcc1413 100644 --- a/rsts/howto/install_sdk.rst +++ b/rsts/howto/install_sdk.rst @@ -21,4 +21,6 @@ All Flytekiplugins are also published to pypi as independent libraries and can b How to install Flytekit Java? ################################# -.. todo: coming soon \ No newline at end of file +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/howto/interruptible.rst b/rsts/howto/interruptible.rst index 79b0589b5b..934e75291a 100644 --- a/rsts/howto/interruptible.rst +++ b/rsts/howto/interruptible.rst @@ -49,4 +49,6 @@ Most Flyte workloads should be good candidates for spot instances. If your task How to recover from interruptions? =================================== -.. todo: Intra-task checkpointing coming soon \ No newline at end of file +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/howto/monitoring/index.rst b/rsts/howto/monitoring/index.rst index 044864cbc8..d8fa308e18 100644 --- a/rsts/howto/monitoring/index.rst +++ b/rsts/howto/monitoring/index.rst @@ -4,7 +4,7 @@ How do I monitor my Flyte deployment? ###################################### -.. tip:: The flyte core team publishes a maintains Grafana dashboards built using prometheus data source and can be found `here `_. +.. tip:: The flyte core team publishes a maintains Grafana dashboards built using prometheus data source and can be found `here `__. Flyte Backend is written in Golang and exposes stats using Prometheus. The Stats themselves are labeled with the Workflow, Task, Project & Domain whereever appropriate. @@ -17,4 +17,4 @@ The dashboards are divided into primarily 2 types - DataPlane/FlytePropeller dashboards published @ `13979 `_ - ControlPlane/Flyteadmin dashboards published @ `13981 `_ -These are basic dashboards and do no include all the metrics that are exposed by Flyte. You can contribute to the dashboards and help us improve them - by referring to the build scripts `here `_. +These are basic dashboards and do no include all the metrics that are exposed by Flyte. You can contribute to the dashboards and help us improve them - by referring to the build scripts `here `__. diff --git a/rsts/howto/multi_cluster/index.rst b/rsts/howto/multi_cluster/index.rst index 77cb9cc275..0bf2ff0105 100644 --- a/rsts/howto/multi_cluster/index.rst +++ b/rsts/howto/multi_cluster/index.rst @@ -1,8 +1,8 @@ .. _howto-multi-cluster: -######################################### -How Do I use Multiple Kubernetes Clusters -######################################### +########################################## +How do I use multiple Kubernetes clusters? +########################################## Scaling Beyond Kubernetes ------------------------- diff --git a/rsts/howto/new_project.rst b/rsts/howto/new_project.rst index 294eb21af6..78e766c0f3 100644 --- a/rsts/howto/new_project.rst +++ b/rsts/howto/new_project.rst @@ -7,7 +7,10 @@ How do I create/register a new project? Using flytectl --------------- -.. todo: link here +.. CAUTION:: + + Coming soon 🛠 + Using flyte-cli @@ -24,4 +27,6 @@ If you refresh your console you'll see your new project appear! FlyteAdmin API reference ------------------------- -.. todo: link here \ No newline at end of file +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/howto/performance/index.rst b/rsts/howto/performance/index.rst index ad4f14bfe0..5851f4e059 100644 --- a/rsts/howto/performance/index.rst +++ b/rsts/howto/performance/index.rst @@ -3,3 +3,7 @@ ###################################################### How do I optimize performance of my Flyte Deployment? ###################################################### + +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/howto/productionize/index.rst b/rsts/howto/productionize/index.rst index d08c4a7a2b..7dbde0d04c 100644 --- a/rsts/howto/productionize/index.rst +++ b/rsts/howto/productionize/index.rst @@ -1,7 +1,7 @@ .. _howto_productionize: ############################################## -How Do I productionize my Flyte cluster +How do I productionize my Flyte cluster ############################################## .. toctree:: diff --git a/rsts/howto/resource_manager/index.rst b/rsts/howto/resource_manager/index.rst index 87ec3faa2e..ba96a480f4 100644 --- a/rsts/howto/resource_manager/index.rst +++ b/rsts/howto/resource_manager/index.rst @@ -2,6 +2,9 @@ ################################################# -How Do I enable and configure resource manager? +How do I enable and configure resource manager? ################################################# +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/howto/resource_quota.rst b/rsts/howto/resource_quota.rst index 3d6e13894f..79f7d4c926 100644 --- a/rsts/howto/resource_quota.rst +++ b/rsts/howto/resource_quota.rst @@ -3,3 +3,7 @@ ############################################### How do I limit resources per project/domain? ############################################### + +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/howto/sandbox.rst b/rsts/howto/sandbox.rst index 02bd0eda0a..c0e3a1bd09 100644 --- a/rsts/howto/sandbox.rst +++ b/rsts/howto/sandbox.rst @@ -35,7 +35,7 @@ Ensure ``kubectl`` is installed. Follow `kubectl installation docs ` with deploying the sandbox +We'll proceed like with :ref:`locally hosted flyte ` with deploying the sandbox Flyte configuration on your remote cluster. .. warning:: diff --git a/rsts/index.rst b/rsts/index.rst index 6c05db9a19..e3b1dd146b 100644 --- a/rsts/index.rst +++ b/rsts/index.rst @@ -8,25 +8,24 @@ Welcome to the documentation hub for Flyte. .. toctree:: :maxdepth: 1 - :caption: Tutorials - :name: tutorialstoc + :caption: Getting Started + :name: gettingstartedtoc - tutorials/first_run - tutorials/first_example - tutorials/run_on_flyte - tutorials/learn + getting_started/first_run + getting_started/first_example + getting_started/run_on_flyte + getting_started/learn .. toctree:: - :caption: Get Answers + :caption: How-Tos :maxdepth: 1 :name: howtotoc plugins/index howto/index - faq/index .. toctree:: - :caption: Diving Deep: Architecture, Concepts & References + :caption: Deep Dive :maxdepth: 1 :name: divedeeptoc @@ -34,7 +33,7 @@ Welcome to the documentation hub for Flyte. reference/index .. toctree:: - :caption: Project Plan & Contributor Guide + :caption: Contributor Guide :maxdepth: 1 :name: roadmaptoc @@ -42,4 +41,3 @@ Welcome to the documentation hub for Flyte. community/docs community/roadmap community/compare - diff --git a/rsts/plugins/aws/athena.rst b/rsts/plugins/aws/athena.rst index 541b6313e6..8d5898b674 100644 --- a/rsts/plugins/aws/athena.rst +++ b/rsts/plugins/aws/athena.rst @@ -2,4 +2,8 @@ ############### AWS Athena -############### \ No newline at end of file +############### + +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/plugins/aws/sagemaker.rst b/rsts/plugins/aws/sagemaker.rst index 23ef769ad9..628f0d288a 100644 --- a/rsts/plugins/aws/sagemaker.rst +++ b/rsts/plugins/aws/sagemaker.rst @@ -2,4 +2,8 @@ ############### AWS Sagemaker -############### \ No newline at end of file +############### + +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/plugins/extend/flyte_backend.rst b/rsts/plugins/extend/flyte_backend.rst index a1524db9cf..8791fcf590 100644 --- a/rsts/plugins/extend/flyte_backend.rst +++ b/rsts/plugins/extend/flyte_backend.rst @@ -30,14 +30,14 @@ This makes it possible to extend a task-template beyond the default supported ta The motivation of the Custom field, is to marshal a JSON structure that specifies information beyond what a regular TaskTemplate can capture. The actual structure of the JSON is known only to the implemented backend-plugin and the SDK components. The core Flyte platform, does not understand of look into the specifics of this structure. It is highly recommended to use an interface definition lanugage like Protobuf, OpenAPISpec etc to declare specify the structure of the JSON. From here, on we refer to this as the ``Plugin Specification``. -For Spark we decided to use Protobuf to specify the plugin as can be seen `here `_. Note it is not necessary to have the Plugin structure specified in FlyteIDL, we do it for simplicity, ease of maintenance alongwith the core platform and because of existing tooling to generate code for protobuf. +For Spark we decided to use Protobuf to specify the plugin as can be seen `here `__. Note it is not necessary to have the Plugin structure specified in FlyteIDL, we do it for simplicity, ease of maintenance alongwith the core platform and because of existing tooling to generate code for protobuf. Flytekit Plugin implementation -------------------------------- Now that you have a specification, we have to implement a method to generate this new TaskTemplate, with the special custom field. Also, this is where the UX design comes into play. You want to write the best possible interface in the SDK that users are delighted to use. The end goal is to create the TaskTemplate with the Custom field populated with the actual JSON structure. We will currently refer to Flytekit - python as an example for extending and implementing the SDK. (For java refer to other docs). The SDK task should be implemented as an extension of :py:class:`flytekit.extend.PythonTask`, or more commonly :py:class:`flytekit.PythonFunctionTask`. -In the case of Spark, we extend the :py:class:`flytekit.PythonFunctionTask`, as shown `here `_. +In the case of Spark, we extend the :py:class:`flytekit.PythonFunctionTask`, as shown `here `__. The SparkTask is implemented as a regular flytekit plugin, with one exception, the ``Custom`` field is now actually the ``SparkJob`` protocol buffer. Flytekit base classes when serializing a task, will automatically invoke the `get_custom method `_. @@ -52,12 +52,24 @@ The backend plugin is where the actual logic of the execution is implemented. Th Kubernetes operator Plugin ^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. todo: fill in + +.. CAUTION:: + + Coming soon 🛠 + Web API plugin ^^^^^^^^^^^^^^^ -.. todo: fill in + +.. CAUTION:: + + Coming soon 🛠 + Catch-all - Core Plugin ^^^^^^^^^^^^^^^^^^^^^^^^ -.. todo: fill in \ No newline at end of file + +.. CAUTION:: + + Coming soon 🛠 + \ No newline at end of file diff --git a/rsts/plugins/hive.rst b/rsts/plugins/hive.rst index 1d3d225793..55e078c214 100644 --- a/rsts/plugins/hive.rst +++ b/rsts/plugins/hive.rst @@ -2,4 +2,8 @@ ########## BYOC Hive -########## \ No newline at end of file +########## + +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/plugins/index.rst b/rsts/plugins/index.rst index 8a718e0258..e59f565fb7 100644 --- a/rsts/plugins/index.rst +++ b/rsts/plugins/index.rst @@ -1,14 +1,11 @@ .. _plugins: -######################################## -Extend Flyte and Available Extenstions -######################################## +################ +Extending Flyte +################ .. _plugins_howto: -==================== -How to extend Flyte -==================== Flyte as platform was designed with extensibility as a core primitive. Flyte is essentially an integration framework and hence extensibility is possible through-out the system. The following sections will guide you through writing your own extensions - either private or public (contribute back to the community). @@ -24,7 +21,7 @@ The following sections will guide you through writing your own extensions - eith ==================== Available Extensions ==================== -Following is a list of maintained plugins for Flyte and guides on how to install / use them. +The following is a list of maintained plugins for Flyte and guides on how to install / use them. .. toctree:: :maxdepth: 1 diff --git a/rsts/plugins/kubeflow/pytorch_operator.rst b/rsts/plugins/kubeflow/pytorch_operator.rst index b2b3ba5f4f..6beb4c1226 100644 --- a/rsts/plugins/kubeflow/pytorch_operator.rst +++ b/rsts/plugins/kubeflow/pytorch_operator.rst @@ -2,4 +2,8 @@ ################# Pytorch Operator -################# \ No newline at end of file +################# + +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/plugins/kubeflow/tensorflow_operator.rst b/rsts/plugins/kubeflow/tensorflow_operator.rst index 59db34e2fa..a198557e5e 100644 --- a/rsts/plugins/kubeflow/tensorflow_operator.rst +++ b/rsts/plugins/kubeflow/tensorflow_operator.rst @@ -2,4 +2,8 @@ ############### TF Operator -############### \ No newline at end of file +############### + +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/plugins/pandera.rst b/rsts/plugins/pandera.rst index 687d803e27..004219d100 100644 --- a/rsts/plugins/pandera.rst +++ b/rsts/plugins/pandera.rst @@ -2,4 +2,8 @@ ################################################## Use Pandera to Enforce Type safety in DataFrames -################################################## \ No newline at end of file +################################################## + +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/plugins/papermill.rst b/rsts/plugins/papermill.rst index 364490a686..7edd202a1d 100644 --- a/rsts/plugins/papermill.rst +++ b/rsts/plugins/papermill.rst @@ -2,4 +2,8 @@ ################################## Use Papermill notebooks as tasks -################################## \ No newline at end of file +################################## + +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/plugins/pod.rst b/rsts/plugins/pod.rst index 9d03e56cb4..1aca0fee46 100644 --- a/rsts/plugins/pod.rst +++ b/rsts/plugins/pod.rst @@ -2,4 +2,8 @@ ######## K8s Pods -######## \ No newline at end of file +######## + +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/plugins/sqlite3.rst b/rsts/plugins/sqlite3.rst index 91f778580b..8bda021c19 100644 --- a/rsts/plugins/sqlite3.rst +++ b/rsts/plugins/sqlite3.rst @@ -2,4 +2,8 @@ ################################################## Query SQLite3 databases -> DataFrames -################################################## \ No newline at end of file +################################################## + +.. CAUTION:: + + Coming soon 🛠 diff --git a/rsts/tutorials/learn.rst b/rsts/tutorials/learn.rst deleted file mode 100644 index 1a87c3adff..0000000000 --- a/rsts/tutorials/learn.rst +++ /dev/null @@ -1,10 +0,0 @@ -.. _tutorials-learn: - -########################### -Learn Flytekit by Example -########################### - -Flytekit implements a programming model for Flyte in (python and Java). The best way to see the capabilities of Flyte is to try out Flytekit. - -`Learn by example - Flytekit Python `_ -========================================================================================= From 6418e6a5fca1da86d37b6bc20f5eb5d6f12c1180 Mon Sep 17 00:00:00 2001 From: Niels Bantilan Date: Thu, 18 Mar 2021 13:59:07 -0400 Subject: [PATCH 28/74] add flytekit docs link to top navbar (#840) Signed-off-by: cosmicBboy Signed-off-by: Haytham Abuelfutuh --- rsts/conf.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/rsts/conf.py b/rsts/conf.py index 8317a22730..eac1607abd 100644 --- a/rsts/conf.py +++ b/rsts/conf.py @@ -124,12 +124,17 @@ "master_doc": False, # custom nav in breadcrumb bar "nav_links": [ - {"href": "index", "internal": True, "title": "Flyte Docs"}, + {"href": "index", "internal": True, "title": "Flyte"}, { "href": "https://flytecookbook.readthedocs.io", "internal": False, "title": "Flytekit Tutorials", }, + { + "href": "https://flytekit.readthedocs.io", + "internal": False, + "title": "Flytekit Python Reference" + }, ], } From 4b328f37d9aaf301c19ace5bb5d2333cc4e44fa0 Mon Sep 17 00:00:00 2001 From: SandraGH5 <80421934+SandraGH5@users.noreply.github.com> Date: Thu, 18 Mar 2021 13:49:12 -0700 Subject: [PATCH 29/74] Update first_run.rst (#842) Estimated time edit Signed-off-by: Haytham Abuelfutuh --- rsts/getting_started/first_run.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rsts/getting_started/first_run.rst b/rsts/getting_started/first_run.rst index a8ed0e0627..404a3a7fda 100644 --- a/rsts/getting_started/first_run.rst +++ b/rsts/getting_started/first_run.rst @@ -6,7 +6,7 @@ Start a Local Flyte Backend ########################### -.. rubric:: Estimated time to complete: 2 minutes. +.. rubric:: Estimated time: 2 minutes Flyte enables scalable, reproducible and reliable orchestration of massively large workflows. To get a sense of the product, a minimalist version of the Flyte system is packaged into a Docker image. From f3f384ea4cccfc527283b00bfc19b1c4bb5463e2 Mon Sep 17 00:00:00 2001 From: SandraGH5 <80421934+SandraGH5@users.noreply.github.com> Date: Thu, 18 Mar 2021 13:50:13 -0700 Subject: [PATCH 30/74] Update first_example.rst (#843) Grammar edits. Also, the Prerequisite section is to be removed, but will defer to Niels for confirmation. Signed-off-by: Haytham Abuelfutuh --- rsts/getting_started/first_example.rst | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/rsts/getting_started/first_example.rst b/rsts/getting_started/first_example.rst index baffaa8211..fc6c0e09b7 100644 --- a/rsts/getting_started/first_example.rst +++ b/rsts/getting_started/first_example.rst @@ -1,12 +1,12 @@ .. _getting-started-first-example: ############################### -Write your First Flyte Workflow +Write Your First Flyte Workflow ############################### -By the end of this getting started guide you'll be familiar with how easy it is to author a Flyte workflow and run it locally. +By the end of this guide you will become familiar with how easy it is to author a Flyte workflow and run it locally. -.. rubric:: Estimated time to complete: <3 minutes. +.. rubric:: Estimated time: <3 minutes Prerequisites @@ -32,28 +32,29 @@ Prerequisites Flyte Tasks and Workflows ************************* -Let's take a look at the example workflow found in `myapp/workflows/example.py `__ +Take a look at the example workflow found in `myapp/workflows/example.py `__ .. rli:: https://raw.githubusercontent.com/flyteorg/flytekit-python-template/main/myapp/workflows/example.py :language: python The most basic Flyte primitive is a :std:doc:`task ` + Flyte tasks are units of work that can be composed in a :std:doc:`workflow ` -You can call this task +You can call this task: .. code-block:: python greet(name="world") -and iterate locally before adding it to part of a larger overall workflow. +and iterate it locally before adding it to part of a larger overall workflow. -Similarly, you can call this workflow +Similarly, you can call this workflow: .. code-block:: python hello_world(name=...) -and iterate locally before moving on to register it with Flyte. +and iterate it locally before registering it with Flyte. .. tip:: Every invocation of a Flyte workflow requires specifying keyword arguments as in the example - ``hello_world(name="name")``. Calling the workflow without the keyword ``name`` will raise an ``AssertionError``. From 73aaf0cc6371f77e6ed6c13438a850fcf923ea5c Mon Sep 17 00:00:00 2001 From: Niels Bantilan Date: Thu, 18 Mar 2021 16:53:47 -0400 Subject: [PATCH 31/74] update tutorials header text (#841) * update tutorials header text Signed-off-by: cosmicBboy * add 1 * remove 1 Co-authored-by: SandraGH5 Signed-off-by: Haytham Abuelfutuh --- rsts/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rsts/conf.py b/rsts/conf.py index eac1607abd..3534a33e85 100644 --- a/rsts/conf.py +++ b/rsts/conf.py @@ -128,7 +128,7 @@ { "href": "https://flytecookbook.readthedocs.io", "internal": False, - "title": "Flytekit Tutorials", + "title": "Tutorials", }, { "href": "https://flytekit.readthedocs.io", From 1b84bf574dccac5b6a0d06b0dac387a2293d44c0 Mon Sep 17 00:00:00 2001 From: SandraGH5 <80421934+SandraGH5@users.noreply.github.com> Date: Fri, 19 Mar 2021 14:24:05 -0700 Subject: [PATCH 32/74] Update run_on_flyte.rst (#846) * Update run_on_flyte.rst * Update run_on_flyte.rst Signed-off-by: Haytham Abuelfutuh --- rsts/getting_started/run_on_flyte.rst | 65 ++++++++++++++------------- 1 file changed, 34 insertions(+), 31 deletions(-) diff --git a/rsts/getting_started/run_on_flyte.rst b/rsts/getting_started/run_on_flyte.rst index 6aed571ed3..423e111669 100644 --- a/rsts/getting_started/run_on_flyte.rst +++ b/rsts/getting_started/run_on_flyte.rst @@ -8,15 +8,22 @@ Run Your Workflow on a Flyte Backend Installing Flyte Locally ************************ -This guide will walk you through a quick installation of Flyte on your laptop and then how to register and execute your -workflows against this deployment. (The tabs below have an option to install Flyte on a cloud provider as well) +This guide will walk you through: -.. rubric:: Estimated time to complete: 1 minutes. +* A quick installation of Flyte on your device + +* How to register and execute your workflows against this deployment. + +(The tips below have an option to install Flyte on a cloud provider as well) + +.. rubric:: Estimated time: 1 minute Prerequisites ============= -Make Sure you have followed :ref:`getting-started-firstrun` and have a running docker container and can access FlyteConsole on http://localhost:30081/console. +1. Follow :ref:`getting-started-firstrun` + +2. Access FlyteConsole on http://localhost:30081/console .. tip:: To check if your flyte-sandbox container is running you can run ``docker ps`` and it should show image ``ghcr.io/flyteorg/flyte-sandbox`` running @@ -25,50 +32,46 @@ Make Sure you have followed :ref:`getting-started-firstrun` and have a running d .. _getting-started-run-flyte-laptop: **************************** -Running your Flyte Workflows +Running Your Flyte Workflows **************************** -Registration -============ -Registration is the process of shipping your code to Flyte backend. This creates an immutable, versioned record of your code with FlyteAdmin service. - -Register your workflows ------------------------ +Register Your Workflows +======================= +Registration is the process of shipping your code to the Flyte backend. This creates an immutable, versioned record of your code with the FlyteAdmin service. -From within root directory of ``flyteexamples`` you created :ref:`previously ` -feel free to make any changes and then register :: +From within root directory of ``flyteexamples`` you previously created with the :ref:`Write Your First Workflow tutorial `, +feel free to make any changes and then register: :: FLYTE_AWS_ENDPOINT=http://localhost:30084/ FLYTE_AWS_ACCESS_KEY_ID=minio \ FLYTE_AWS_SECRET_ACCESS_KEY=miniostorage make fast_register -.. tip:: Flyte sandbox uses minio as a substitue for S3/GCS etc, in the first command we port-forwarded it to 30084. If you use s3/gcs or a different port-forward you can drop or change ``FLYTE_AWS_ENDPOINT`` accordingly. +.. tip:: Flyte sandbox uses minio as a substitute for S3/GCS etc. It is port-forwarded in the first command to 30084. If you use S3/GCS or a different port-forward you can drop or change the ``FLYTE_AWS_ENDPOINT`` accordingly. -.. rubric:: Boom! It's that simple. +.. rubric:: It's that simple! -Run your workflows ------------------- +Run Your Workflows +================== -Visit the page housing workflows registered for your project (method if you used k3d): +Visit the page housing workflows registered to your project at: `http://localhost:30081/console/projects/flyteexamples/workflows `__ -else if you used docker-desktop or something else, then copy paste this URL into the browser and fill in the ````:: - - http:///console/projects/flyteexamples/workflows +Once you have accessed your workflows, -Select your workflow, click the bright purple "Launch Workflow" button in the upper right, update the "name" input -argument as you please, proceed to launch and you'll have triggered an execution! +* Select your workflow +* Click the bright purple "Launch Workflow" button in the upper right +* Update the "name" input argument +* Proceed to launch to trigger an execution .. note:: - After registration Flyte Workflows exist in the FlyteAdmin service and can be triggered using the - - console + After registration, Flyte Workflows exist in the FlyteAdmin service and can be triggered using: + - Console - Command line - - directly invoking the REST API - - on a schedule + - Directly invoking the REST API + - On a schedule - More on this later -Optionally you can create a new project ----------------------------------------- -Refer to :ref:`howto_new_project`. \ No newline at end of file +Create a New Project +-------------------- +Visit :ref:`howto_new_project`. From f27fc795936dbb656e3aebdddb20cadc471928b1 Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Tue, 23 Mar 2021 19:39:23 -0700 Subject: [PATCH 33/74] Pod Webhook deployment & Doc (#857) * update tutorials header text (#841) * update tutorials header text Signed-off-by: cosmicBboy * add 1 * remove 1 Co-authored-by: SandraGH5 Signed-off-by: Haytham Abuelfutuh * Update run_on_flyte.rst (#846) * Update run_on_flyte.rst * Update run_on_flyte.rst Signed-off-by: Haytham Abuelfutuh * Add webhook deployment Signed-off-by: Haytham Abuelfutuh * update Signed-off-by: Haytham Abuelfutuh * add webhook deployment Signed-off-by: Haytham Abuelfutuh * Add webhook deployment Signed-off-by: Haytham Abuelfutuh * update Signed-off-by: Haytham Abuelfutuh * regen Signed-off-by: Haytham Abuelfutuh * Deploy sample secret Signed-off-by: Haytham Abuelfutuh * Add doc for secrets Signed-off-by: Haytham Abuelfutuh * Update secrets.rst Signed-off-by: Haytham Abuelfutuh * limit permissions Signed-off-by: Haytham Abuelfutuh * plural Signed-off-by: Haytham Abuelfutuh * Update docs Signed-off-by: Haytham Abuelfutuh Co-authored-by: Niels Bantilan Co-authored-by: SandraGH5 Co-authored-by: SandraGH5 <80421934+SandraGH5@users.noreply.github.com> Signed-off-by: Haytham Abuelfutuh --- deployment/eks/flyte_generated.yaml | 163 ++++++++++++++- deployment/gcp/flyte_generated.yaml | 163 ++++++++++++++- deployment/sandbox/flyte_generated.yaml | 187 +++++++++++++++++- deployment/test/flyte_generated.yaml | 168 +++++++++++++++- .../base/admindeployment/deployment.yaml | 8 +- kustomize/base/propeller/kustomization.yaml | 5 +- kustomize/base/propeller/webhook.yaml | 161 +++++++++++++++ .../headless/config/propeller/core.yaml | 3 + kustomize/overlays/eks/kustomization.yaml | 8 +- kustomize/overlays/gcp/kustomization.yaml | 8 +- .../sandbox/flyte/admin/deployment.yaml | 6 +- .../overlays/sandbox/flyte/kustomization.yaml | 66 ++++--- .../flyte/propeller/webhook-secret.yaml | 10 + .../sandbox/flyte/propeller/webhook.yaml | 18 ++ kustomize/overlays/sandbox/kustomization.yaml | 8 +- kustomize/overlays/test/kustomization.yaml | 8 +- rsts/howto/index.rst | 1 + rsts/howto/secrets.rst | 62 ++++++ 18 files changed, 978 insertions(+), 75 deletions(-) create mode 100644 kustomize/base/propeller/webhook.yaml create mode 100644 kustomize/overlays/sandbox/flyte/propeller/webhook-secret.yaml create mode 100644 kustomize/overlays/sandbox/flyte/propeller/webhook.yaml create mode 100644 rsts/howto/secrets.rst diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index dce796fcae..66576a38a3 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -7610,6 +7610,12 @@ metadata: --- apiVersion: v1 kind: ServiceAccount +metadata: + name: flyte-pod-webhook + namespace: flyte +--- +apiVersion: v1 +kind: ServiceAccount metadata: annotations: eks.amazonaws.com/role-arn: arn:aws:iam::111222333456:role/flyte-operator @@ -7641,6 +7647,24 @@ metadata: --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole +metadata: + name: flyte-pod-webhook + namespace: flyte +rules: +- apiGroups: + - '*' + resources: + - mutatingwebhookconfigurations + - secrets + - pods + verbs: + - get + - create + - update + - patch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole metadata: name: flyteadmin namespace: flyte @@ -7892,6 +7916,20 @@ rules: verbs: - '*' --- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: flyte-pod-webhook + namespace: flyte +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flyte-pod-webhook +subjects: +- kind: ServiceAccount + name: flyte-pod-webhook + namespace: flyte +--- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: @@ -8391,6 +8429,22 @@ spec: --- apiVersion: v1 kind: Service +metadata: + annotations: + projectcontour.io/upstream-protocol.h2c: grpc + name: flyte-pod-webhook + namespace: flyte +spec: + ports: + - name: https + port: 443 + protocol: TCP + targetPort: 9443 + selector: + app: flyte-pod-webhook +--- +apiVersion: v1 +kind: Service metadata: annotations: external-dns.alpha.kubernetes.io/hostname: flyteadmin.subdomain.mydomain.com @@ -8577,6 +8631,63 @@ spec: --- apiVersion: apps/v1 kind: Deployment +metadata: + labels: + app: flyte-pod-webhook + name: flyte-pod-webhook + namespace: flyte +spec: + selector: + matchLabels: + app: flyte-pod-webhook + template: + metadata: + annotations: + prometheus.io/path: /metrics + prometheus.io/port: "10254" + prometheus.io/scrape: "true" + labels: + app: flyte-pod-webhook + app.kubernetes.io/name: flyte-pod-webhook + app.kubernetes.io/version: 0.5.13 + spec: + containers: + - args: + - webhook + - --config + - /etc/flyte/config/*.yaml + command: + - flytepropeller + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + imagePullPolicy: IfNotPresent + name: webhook + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + readOnly: true + - mountPath: /etc/webhook/certs + name: webhook-certs + readOnly: true + serviceAccountName: flyte-pod-webhook + volumes: + - configMap: + name: flyte-propeller-config-ct29cf4cch + name: config-volume + - name: webhook-certs + secret: + secretName: flyte-pod-webhook +--- +apiVersion: apps/v1 +kind: Deployment metadata: labels: app: flyteadmin @@ -8604,7 +8715,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -8649,7 +8760,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8666,7 +8777,7 @@ spec: - flytesnacks - flytetester - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -8680,7 +8791,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -8771,7 +8882,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.7.1 + image: ghcr.io/flyteorg/flytepropeller:v0.7.8 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -8958,7 +9069,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -8984,6 +9095,46 @@ spec: --- apiVersion: batch/v1 kind: Job +metadata: + name: flyte-pod-webhook-secret + namespace: flyte +spec: + backoffLimit: 3 + template: + spec: + containers: + - args: + - webhook + - init-certs + - --config + - /etc/flyte/config/*.yaml + command: + - flytepropeller + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + imagePullPolicy: IfNotPresent + name: webhook + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + restartPolicy: Never + serviceAccountName: flyte-pod-webhook + volumes: + - configMap: + name: flyte-propeller-config-ct29cf4cch + name: config-volume + ttlSecondsAfterFinished: 0 +--- +apiVersion: batch/v1 +kind: Job metadata: labels: app.kubernetes.io/name: sparkoperator diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index 2faab6b203..2480b3640d 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -7610,6 +7610,12 @@ metadata: --- apiVersion: v1 kind: ServiceAccount +metadata: + name: flyte-pod-webhook + namespace: flyte +--- +apiVersion: v1 +kind: ServiceAccount metadata: name: flyteadmin namespace: flyte @@ -7637,6 +7643,24 @@ metadata: --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole +metadata: + name: flyte-pod-webhook + namespace: flyte +rules: +- apiGroups: + - '*' + resources: + - mutatingwebhookconfigurations + - secrets + - pods + verbs: + - get + - create + - update + - patch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole metadata: name: flyteadmin namespace: flyte @@ -7888,6 +7912,20 @@ rules: verbs: - '*' --- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: flyte-pod-webhook + namespace: flyte +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flyte-pod-webhook +subjects: +- kind: ServiceAccount + name: flyte-pod-webhook + namespace: flyte +--- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: @@ -8391,6 +8429,22 @@ spec: --- apiVersion: v1 kind: Service +metadata: + annotations: + projectcontour.io/upstream-protocol.h2c: grpc + name: flyte-pod-webhook + namespace: flyte +spec: + ports: + - name: https + port: 443 + protocol: TCP + targetPort: 9443 + selector: + app: flyte-pod-webhook +--- +apiVersion: v1 +kind: Service metadata: annotations: cloud.google.com/load-balancer-type: Internal @@ -8609,6 +8663,63 @@ spec: --- apiVersion: apps/v1 kind: Deployment +metadata: + labels: + app: flyte-pod-webhook + name: flyte-pod-webhook + namespace: flyte +spec: + selector: + matchLabels: + app: flyte-pod-webhook + template: + metadata: + annotations: + prometheus.io/path: /metrics + prometheus.io/port: "10254" + prometheus.io/scrape: "true" + labels: + app: flyte-pod-webhook + app.kubernetes.io/name: flyte-pod-webhook + app.kubernetes.io/version: 0.5.13 + spec: + containers: + - args: + - webhook + - --config + - /etc/flyte/config/*.yaml + command: + - flytepropeller + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + imagePullPolicy: IfNotPresent + name: webhook + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + readOnly: true + - mountPath: /etc/webhook/certs + name: webhook-certs + readOnly: true + serviceAccountName: flyte-pod-webhook + volumes: + - configMap: + name: flyte-propeller-config-7288ccf2f8 + name: config-volume + - name: webhook-certs + secret: + secretName: flyte-pod-webhook +--- +apiVersion: apps/v1 +kind: Deployment metadata: labels: app: flyteadmin @@ -8636,7 +8747,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -8681,7 +8792,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8698,7 +8809,7 @@ spec: - flytesnacks - flytetester - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -8712,7 +8823,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -8803,7 +8914,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.7.1 + image: ghcr.io/flyteorg/flytepropeller:v0.7.8 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -8990,7 +9101,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -9016,6 +9127,46 @@ spec: --- apiVersion: batch/v1 kind: Job +metadata: + name: flyte-pod-webhook-secret + namespace: flyte +spec: + backoffLimit: 3 + template: + spec: + containers: + - args: + - webhook + - init-certs + - --config + - /etc/flyte/config/*.yaml + command: + - flytepropeller + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + imagePullPolicy: IfNotPresent + name: webhook + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + restartPolicy: Never + serviceAccountName: flyte-pod-webhook + volumes: + - configMap: + name: flyte-propeller-config-7288ccf2f8 + name: config-volume + ttlSecondsAfterFinished: 0 +--- +apiVersion: batch/v1 +kind: Job metadata: labels: app.kubernetes.io/name: sparkoperator diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index f0dde874e2..24c6f4928e 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -1588,6 +1588,12 @@ metadata: --- apiVersion: v1 kind: ServiceAccount +metadata: + name: flyte-pod-webhook + namespace: flyte +--- +apiVersion: v1 +kind: ServiceAccount metadata: name: flyteadmin namespace: flyte @@ -1691,6 +1697,24 @@ rules: --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole +metadata: + name: flyte-pod-webhook + namespace: flyte +rules: +- apiGroups: + - '*' + resources: + - mutatingwebhookconfigurations + - secrets + - pods + verbs: + - get + - create + - update + - patch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole metadata: name: flyteadmin namespace: flyte @@ -1936,6 +1960,20 @@ subjects: --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding +metadata: + name: flyte-pod-webhook + namespace: flyte +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flyte-pod-webhook +subjects: +- kind: ServiceAccount + name: flyte-pod-webhook + namespace: flyte +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding metadata: name: contour roleRef: @@ -2236,6 +2274,9 @@ data: type: bucket rate: 10 capacity: 100 + webhook: + certDir: /etc/webhook/certs + serviceName: flyte-pod-webhook enabled_plugins.yaml: | tasks: task-plugins: @@ -2283,7 +2324,7 @@ data: kubernetes-template-uri: "http://localhost:30082/#/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace }}" kind: ConfigMap metadata: - name: flyte-propeller-config-492gkfhbgk + name: flyte-propeller-config-c8kkg4fth7 namespace: flyte --- apiVersion: v1 @@ -2426,6 +2467,17 @@ metadata: type: Opaque --- apiVersion: v1 +data: + password: bXl1c2Vy + user_secret: bXlzZWNyZXQ= + username: bXl1c2Vy +kind: Secret +metadata: + name: user-info + namespace: flyte +type: Opaque +--- +apiVersion: v1 data: csrf: "" kind: Secret @@ -2467,6 +2519,22 @@ spec: --- apiVersion: v1 kind: Service +metadata: + annotations: + projectcontour.io/upstream-protocol.h2c: grpc + name: flyte-pod-webhook + namespace: flyte +spec: + ports: + - name: https + port: 443 + protocol: TCP + targetPort: 9443 + selector: + app: flyte-pod-webhook +--- +apiVersion: v1 +kind: Service metadata: annotations: projectcontour.io/upstream-protocol.h2c: grpc @@ -2693,6 +2761,69 @@ spec: --- apiVersion: apps/v1 kind: Deployment +metadata: + labels: + app: flyte-pod-webhook + name: flyte-pod-webhook + namespace: flyte +spec: + selector: + matchLabels: + app: flyte-pod-webhook + template: + metadata: + annotations: + prometheus.io/path: /metrics + prometheus.io/port: "10254" + prometheus.io/scrape: "true" + labels: + app: flyte-pod-webhook + app.kubernetes.io/name: flyte-pod-webhook + app.kubernetes.io/version: 0.5.13 + spec: + containers: + - args: + - webhook + - --config + - /etc/flyte/config/*.yaml + command: + - flytepropeller + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + imagePullPolicy: IfNotPresent + name: webhook + volumeMounts: + - mountPath: /etc/secrets/user-info + name: sample-secrets + readOnly: true + - mountPath: /etc/flyte/config + name: config-volume + readOnly: true + - mountPath: /etc/webhook/certs + name: webhook-certs + readOnly: true + serviceAccountName: flyte-pod-webhook + volumes: + - name: sample-secrets + secret: + secretName: user-info + - configMap: + name: flyte-propeller-config-c8kkg4fth7 + name: config-volume + - name: webhook-certs + secret: + secretName: flyte-pod-webhook +--- +apiVersion: apps/v1 +kind: Deployment metadata: labels: app: flyteadmin @@ -2720,7 +2851,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -2771,7 +2902,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -2787,7 +2918,7 @@ spec: - seed-projects - flytesnacks - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -2801,7 +2932,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -2892,7 +3023,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.7.1 + image: ghcr.io/flyteorg/flytepropeller:v0.7.8 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -2903,7 +3034,7 @@ spec: serviceAccountName: flytepropeller volumes: - configMap: - name: flyte-propeller-config-492gkfhbgk + name: flyte-propeller-config-c8kkg4fth7 name: config-volume --- apiVersion: apps/v1 @@ -3197,7 +3328,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -3357,6 +3488,46 @@ spec: --- apiVersion: batch/v1 kind: Job +metadata: + name: flyte-pod-webhook-secret + namespace: flyte +spec: + backoffLimit: 3 + template: + spec: + containers: + - args: + - webhook + - init-certs + - --config + - /etc/flyte/config/*.yaml + command: + - flytepropeller + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + imagePullPolicy: IfNotPresent + name: webhook + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + restartPolicy: Never + serviceAccountName: flyte-pod-webhook + volumes: + - configMap: + name: flyte-propeller-config-c8kkg4fth7 + name: config-volume + ttlSecondsAfterFinished: 0 +--- +apiVersion: batch/v1 +kind: Job metadata: name: contour-certgen-v1.13.1 namespace: projectcontour diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index 2028979b20..59d5e1a0d5 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -26,6 +26,12 @@ metadata: --- apiVersion: v1 kind: ServiceAccount +metadata: + name: flyte-pod-webhook + namespace: flyte +--- +apiVersion: v1 +kind: ServiceAccount metadata: name: flyteadmin namespace: flyte @@ -38,6 +44,24 @@ metadata: --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole +metadata: + name: flyte-pod-webhook + namespace: flyte +rules: +- apiGroups: + - '*' + resources: + - mutatingwebhookconfigurations + - secrets + - pods + verbs: + - get + - create + - update + - patch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole metadata: name: flyteadmin namespace: flyte @@ -121,6 +145,20 @@ rules: - post - deletecollection --- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: flyte-pod-webhook + namespace: flyte +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flyte-pod-webhook +subjects: +- kind: ServiceAccount + name: flyte-pod-webhook + namespace: flyte +--- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: @@ -375,6 +413,9 @@ data: type: bucket rate: 10 capacity: 100 + webhook: + certDir: /etc/webhook/certs + serviceName: flyte-pod-webhook enabled_plugins.yaml: | tasks: task-plugins: @@ -417,7 +458,7 @@ data: kind: ConfigMap metadata: - name: flyte-propeller-config-ttk5h2hkmg + name: flyte-propeller-config-cbbhf652h4 namespace: flyte --- apiVersion: v1 @@ -451,6 +492,22 @@ spec: --- apiVersion: v1 kind: Service +metadata: + annotations: + projectcontour.io/upstream-protocol.h2c: grpc + name: flyte-pod-webhook + namespace: flyte +spec: + ports: + - name: https + port: 443 + protocol: TCP + targetPort: 9443 + selector: + app: flyte-pod-webhook +--- +apiVersion: v1 +kind: Service metadata: annotations: projectcontour.io/upstream-protocol.h2c: grpc @@ -564,6 +621,63 @@ spec: --- apiVersion: apps/v1 kind: Deployment +metadata: + labels: + app: flyte-pod-webhook + name: flyte-pod-webhook + namespace: flyte +spec: + selector: + matchLabels: + app: flyte-pod-webhook + template: + metadata: + annotations: + prometheus.io/path: /metrics + prometheus.io/port: "10254" + prometheus.io/scrape: "true" + labels: + app: flyte-pod-webhook + app.kubernetes.io/name: flyte-pod-webhook + app.kubernetes.io/version: 0.5.13 + spec: + containers: + - args: + - webhook + - --config + - /etc/flyte/config/*.yaml + command: + - flytepropeller + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + imagePullPolicy: IfNotPresent + name: webhook + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + readOnly: true + - mountPath: /etc/webhook/certs + name: webhook-certs + readOnly: true + serviceAccountName: flyte-pod-webhook + volumes: + - configMap: + name: flyte-propeller-config-cbbhf652h4 + name: config-volume + - name: webhook-certs + secret: + secretName: flyte-pod-webhook +--- +apiVersion: apps/v1 +kind: Deployment metadata: labels: app: flyteadmin @@ -591,7 +705,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -642,7 +756,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -658,7 +772,7 @@ spec: - seed-projects - flytetester - flytesnacks - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -672,7 +786,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.3.38 + image: ghcr.io/flyteorg/flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -729,7 +843,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.7.1 + image: ghcr.io/flyteorg/flytepropeller:v0.7.8 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -740,7 +854,7 @@ spec: serviceAccountName: flytepropeller volumes: - configMap: - name: flyte-propeller-config-ttk5h2hkmg + name: flyte-propeller-config-cbbhf652h4 name: config-volume --- apiVersion: apps/v1 @@ -808,6 +922,46 @@ spec: - emptyDir: {} name: postgres-storage --- +apiVersion: batch/v1 +kind: Job +metadata: + name: flyte-pod-webhook-secret + namespace: flyte +spec: + backoffLimit: 3 + template: + spec: + containers: + - args: + - webhook + - init-certs + - --config + - /etc/flyte/config/*.yaml + command: + - flytepropeller + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + imagePullPolicy: IfNotPresent + name: webhook + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + restartPolicy: Never + serviceAccountName: flyte-pod-webhook + volumes: + - configMap: + name: flyte-propeller-config-cbbhf652h4 + name: config-volume + ttlSecondsAfterFinished: 0 +--- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: diff --git a/kustomize/base/admindeployment/deployment.yaml b/kustomize/base/admindeployment/deployment.yaml index 1c455774ab..b77ea096a1 100644 --- a/kustomize/base/admindeployment/deployment.yaml +++ b/kustomize/base/admindeployment/deployment.yaml @@ -37,7 +37,7 @@ spec: secretName: db-pass initContainers: - name: run-migrations - image: flyteadmin:v0.3.38 + image: flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "migrate", "run"] volumeMounts: @@ -47,7 +47,7 @@ spec: mountPath: /etc/db # Optional, These just seed the project - TODO move them to only - name: seed-projects - image: flyteadmin:v0.3.38 + image: flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "migrate", "seed-projects", "flytesnacks", "flytetester", "flyteexamples"] @@ -57,7 +57,7 @@ spec: - name: db-pass mountPath: /etc/db - name: sync-cluster-resources - image: flyteadmin:v0.3.38 + image: flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "clusterresource", "sync"] volumeMounts: @@ -69,7 +69,7 @@ spec: mountPath: /etc/db containers: - name: flyteadmin - image: flyteadmin:v0.3.38 + image: flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "serve"] ports: diff --git a/kustomize/base/propeller/kustomization.yaml b/kustomize/base/propeller/kustomization.yaml index 3f090bdf48..408244dcaf 100644 --- a/kustomize/base/propeller/kustomization.yaml +++ b/kustomize/base/propeller/kustomization.yaml @@ -1,3 +1,4 @@ resources: -- deployment.yaml -- rbac.yaml + - deployment.yaml + - rbac.yaml + - webhook.yaml diff --git a/kustomize/base/propeller/webhook.yaml b/kustomize/base/propeller/webhook.yaml new file mode 100644 index 0000000000..23efff56d2 --- /dev/null +++ b/kustomize/base/propeller/webhook.yaml @@ -0,0 +1,161 @@ +apiVersion: batch/v1 +kind: Job +metadata: + name: flyte-pod-webhook-secret + namespace: flyte +spec: + backoffLimit: 3 + ttlSecondsAfterFinished: 0 + template: + spec: + serviceAccountName: flyte-pod-webhook + restartPolicy: Never + containers: + - name: webhook + image: flytepropeller:v0.5.13 + imagePullPolicy: IfNotPresent + command: + - flytepropeller + args: + - webhook + - init-certs + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + volumes: + - name: config-volume + configMap: + name: flyte-propeller-config +--- +# Create the actual deployment +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyte-pod-webhook + namespace: flyte + labels: + app: flyte-pod-webhook +spec: + selector: + matchLabels: + app: flyte-pod-webhook + template: + metadata: + labels: + app: flyte-pod-webhook + app.kubernetes.io/name: flyte-pod-webhook + app.kubernetes.io/version: 0.5.13 + annotations: + prometheus.io/scrape: "true" + prometheus.io/port: "10254" + prometheus.io/path: "/metrics" + spec: + serviceAccountName: flyte-pod-webhook + containers: + - name: webhook + image: flytepropeller:v0.5.13 + imagePullPolicy: IfNotPresent + command: + - flytepropeller + args: + - webhook + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + readOnly: true + - name: webhook-certs + mountPath: /etc/webhook/certs + readOnly: true + volumes: + - name: config-volume + configMap: + name: flyte-propeller-config + - name: webhook-certs + secret: + secretName: flyte-pod-webhook +--- +# Service +apiVersion: v1 +kind: Service +metadata: + name: flyte-pod-webhook + namespace: flyte + annotations: + # This tells contour to use a H2 connection for the port associated + # with the name 'grpc' under spec/ports. + # For more information, refer to + # https://github.com/heptio/contour/blob/master/docs/annotations.md#contour-specific-service-annotations + # # Following this issue - the annotation was updated https://github.com/projectcontour/contour/issues/2092 + projectcontour.io/upstream-protocol.h2c: "grpc" +spec: + selector: + app: flyte-pod-webhook + ports: + - name: https + protocol: TCP + port: 443 + targetPort: 9443 +--- +# Create a ClusterRole for the webhook +# https://kubernetes.io/docs/admin/authorization/rbac/ +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: flyte-pod-webhook + namespace: flyte +rules: + - apiGroups: + - "*" + resources: + - mutatingwebhookconfigurations + - secrets + - pods + verbs: + - get + - create + - update + - patch +--- +# Create a Service Account for webhook +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-pod-webhook + namespace: flyte +--- +# Create a binding from Role -> ServiceAccount +kind: ClusterRoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: flyte-pod-webhook + namespace: flyte +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flyte-pod-webhook +subjects: + - kind: ServiceAccount + name: flyte-pod-webhook + namespace: flyte diff --git a/kustomize/base/single_cluster/headless/config/propeller/core.yaml b/kustomize/base/single_cluster/headless/config/propeller/core.yaml index e1fce27b72..055e7a8840 100644 --- a/kustomize/base/single_cluster/headless/config/propeller/core.yaml +++ b/kustomize/base/single_cluster/headless/config/propeller/core.yaml @@ -31,3 +31,6 @@ propeller: type: bucket rate: 10 capacity: 100 +webhook: + certDir: /etc/webhook/certs + serviceName: flyte-pod-webhook diff --git a/kustomize/overlays/eks/kustomization.yaml b/kustomize/overlays/eks/kustomization.yaml index e0ab501cdd..21b3f2ef07 100644 --- a/kustomize/overlays/eks/kustomization.yaml +++ b/kustomize/overlays/eks/kustomization.yaml @@ -21,7 +21,7 @@ bases: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.3.38 # override the tag + newTag: v0.4.0 # override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -33,5 +33,9 @@ images: newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.7.1 # override the tag + newTag: v0.7.8 # override the tag + newName: ghcr.io/flyteorg/flytepropeller # override the name + # Webhook + - name: webhook # match images with this name + newTag: v0.7.8 # override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name diff --git a/kustomize/overlays/gcp/kustomization.yaml b/kustomize/overlays/gcp/kustomization.yaml index acfcec1508..a209c3bf86 100644 --- a/kustomize/overlays/gcp/kustomization.yaml +++ b/kustomize/overlays/gcp/kustomization.yaml @@ -23,7 +23,7 @@ bases: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.3.38 # override the tag + newTag: v0.4.0 # override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -35,5 +35,9 @@ images: newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.7.1 # override the tag + newTag: v0.7.8 # override the tag + newName: ghcr.io/flyteorg/flytepropeller # override the name + # Webhook + - name: webhook # match images with this name + newTag: v0.7.8 # override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name diff --git a/kustomize/overlays/sandbox/flyte/admin/deployment.yaml b/kustomize/overlays/sandbox/flyte/admin/deployment.yaml index 33f39ab46b..f3a7c1ef20 100644 --- a/kustomize/overlays/sandbox/flyte/admin/deployment.yaml +++ b/kustomize/overlays/sandbox/flyte/admin/deployment.yaml @@ -20,7 +20,7 @@ spec: do echo waiting for database; sleep 2; done;", ] - name: run-migrations - image: flyteadmin:v0.3.38 + image: flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent command: [ @@ -34,7 +34,7 @@ spec: - name: config-volume mountPath: /etc/flyte/config - name: seed-projects - image: flyteadmin:v0.3.38 + image: flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent command: [ @@ -50,7 +50,7 @@ spec: - name: config-volume mountPath: /etc/flyte/config - name: sync-cluster-resources - image: flyteadmin:v0.3.38 + image: flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent command: [ diff --git a/kustomize/overlays/sandbox/flyte/kustomization.yaml b/kustomize/overlays/sandbox/flyte/kustomization.yaml index d3485ba7b7..c0ad168558 100644 --- a/kustomize/overlays/sandbox/flyte/kustomization.yaml +++ b/kustomize/overlays/sandbox/flyte/kustomization.yaml @@ -5,38 +5,42 @@ kind: Kustomization namespace: flyte bases: -- ../../../base/single_cluster/complete + - ../../../base/single_cluster/complete + +resources: + - propeller/webhook-secret.yaml patchesStrategicMerge: -- admin/deployment.yaml + - admin/deployment.yaml + - propeller/webhook.yaml configMapGenerator: -- behavior: merge - files: - - ./config/admin/db.yaml - - ./config/admin/task_resource_defaults.yaml - - ./config/common/storage.yaml - - ./config/common/logger.yaml - name: flyte-admin-config - -- behavior: merge - files: - - ./config/clusterresource-templates/ac_project-copilot-dataconfig.yaml - name: clusterresource-template - -- behavior: merge - files: - - ./config/propeller/enabled_plugins.yaml - - ./config/propeller/resource_manager.yaml - - ./config/propeller/plugins/k8s.yaml - - ./config/propeller/plugins/task_logs.yaml - - ./config/common/storage.yaml - - ./config/common/logger.yaml - name: flyte-propeller-config - -- behavior: merge - files: - - ./config/common/storage.yaml - - ./config/common/logger.yaml - - ./config/datacatalog/db.yaml - name: datacatalog-config + - behavior: merge + files: + - ./config/admin/db.yaml + - ./config/admin/task_resource_defaults.yaml + - ./config/common/storage.yaml + - ./config/common/logger.yaml + name: flyte-admin-config + + - behavior: merge + files: + - ./config/clusterresource-templates/ac_project-copilot-dataconfig.yaml + name: clusterresource-template + + - behavior: merge + files: + - ./config/propeller/enabled_plugins.yaml + - ./config/propeller/resource_manager.yaml + - ./config/propeller/plugins/k8s.yaml + - ./config/propeller/plugins/task_logs.yaml + - ./config/common/storage.yaml + - ./config/common/logger.yaml + name: flyte-propeller-config + + - behavior: merge + files: + - ./config/common/storage.yaml + - ./config/common/logger.yaml + - ./config/datacatalog/db.yaml + name: datacatalog-config diff --git a/kustomize/overlays/sandbox/flyte/propeller/webhook-secret.yaml b/kustomize/overlays/sandbox/flyte/propeller/webhook-secret.yaml new file mode 100644 index 0000000000..4029be3716 --- /dev/null +++ b/kustomize/overlays/sandbox/flyte/propeller/webhook-secret.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +data: + password: bXl1c2Vy + user_secret: bXlzZWNyZXQ= + username: bXl1c2Vy +kind: Secret +metadata: + name: user-info + namespace: flyte +type: Opaque diff --git a/kustomize/overlays/sandbox/flyte/propeller/webhook.yaml b/kustomize/overlays/sandbox/flyte/propeller/webhook.yaml new file mode 100644 index 0000000000..a659e88141 --- /dev/null +++ b/kustomize/overlays/sandbox/flyte/propeller/webhook.yaml @@ -0,0 +1,18 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyte-pod-webhook + namespace: flyte +spec: + template: + spec: + containers: + - name: webhook + volumeMounts: + - name: sample-secrets + mountPath: /etc/secrets/user-info + readOnly: true + volumes: + - name: sample-secrets + secret: + secretName: user-info diff --git a/kustomize/overlays/sandbox/kustomization.yaml b/kustomize/overlays/sandbox/kustomization.yaml index 59f2f9e5ca..50d2f74c61 100644 --- a/kustomize/overlays/sandbox/kustomization.yaml +++ b/kustomize/overlays/sandbox/kustomization.yaml @@ -25,7 +25,7 @@ resources: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.3.38 # override the tag + newTag: v0.4.0 # override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -37,7 +37,11 @@ images: newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.7.1 # override the tag + newTag: v0.7.8 # override the tag + newName: ghcr.io/flyteorg/flytepropeller # override the name + # Webhook + - name: webhook # match images with this name + newTag: v0.7.8 # override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Override postgres image to use alpine based (rather smaller) docker image - name: postgres diff --git a/kustomize/overlays/test/kustomization.yaml b/kustomize/overlays/test/kustomization.yaml index 313dffa35e..6bb67c1098 100644 --- a/kustomize/overlays/test/kustomization.yaml +++ b/kustomize/overlays/test/kustomization.yaml @@ -19,7 +19,7 @@ bases: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.3.38 # override the tag + newTag: v0.4.0 # override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -31,7 +31,11 @@ images: newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.7.1 # override the tag + newTag: v0.7.8 # override the tag + newName: ghcr.io/flyteorg/flytepropeller # override the name + # Webhook + - name: webhook # match images with this name + newTag: v0.7.8 # override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Override postgres image to use alpine based (rather smaller) docker image - name: postgres diff --git a/rsts/howto/index.rst b/rsts/howto/index.rst index 77e85c719a..1c0bbe5e1c 100644 --- a/rsts/howto/index.rst +++ b/rsts/howto/index.rst @@ -32,6 +32,7 @@ Frequently Asked Questions notifications serviceaccount gcp + secrets .. _howto_extend: diff --git a/rsts/howto/secrets.rst b/rsts/howto/secrets.rst new file mode 100644 index 0000000000..939c00f5d6 --- /dev/null +++ b/rsts/howto/secrets.rst @@ -0,0 +1,62 @@ +.. _howto-secrets: + +################################ +How to Inject Secrets Into Tasks +################################ + + +************************** +What Is Secrets Injection? +************************** + +Flyte supports running a wide variety of tasks; from containers to sql queries and service calls. In order for flyte-run +containers to request and access secrets, flyte now natively supports a Secret construct. + +For a simple task that launches a Pod, the flow will look something like this: + +.. image:: https://mermaid.ink/img/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG4gICAgUHJvcGVsbGVyLT4-K1BsdWdpbnM6IENyZWF0ZSBLOHMgUmVzb3VyY2VcbiAgICBQbHVnaW5zLT4-LVByb3BlbGxlcjogUmVzb3VyY2UgT2JqZWN0XG4gICAgUHJvcGVsbGVyLT4-K1Byb3BlbGxlcjogU2V0IExhYmVscyAmIEFubm90YXRpb25zXG4gICAgUHJvcGVsbGVyLT4-K0FwaVNlcnZlcjogQ3JlYXRlIE9iamVjdCAoZS5nLiBQb2QpXG4gICAgQXBpU2VydmVyLT4-K1BvZCBXZWJob29rOiAvbXV0YXRlXG4gICAgUG9kIFdlYmhvb2stPj4rUG9kIFdlYmhvb2s6IExvb2t1cCBnbG9iYWxzXG4gICAgUG9kIFdlYmhvb2stPj4rUG9kIFdlYmhvb2s6IEluamVjdCBTZWNyZXQgQW5ub3RhdGlvbnMgKGUuZy4gSzhzLCBWYXVsdC4uLiBldGMuKVxuICAgIFBvZCBXZWJob29rLT4-LUFwaVNlcnZlcjogTXV0YXRlZCBQb2RcbiAgICBcbiAgICAgICAgICAgICIsIm1lcm1haWQiOnt9LCJ1cGRhdGVFZGl0b3IiOmZhbHNlfQ + :target: https://mermaid.ink/img/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG4gICAgUHJvcGVsbGVyLT4-K1BsdWdpbnM6IENyZWF0ZSBLOHMgUmVzb3VyY2VcbiAgICBQbHVnaW5zLT4-LVByb3BlbGxlcjogUmVzb3VyY2UgT2JqZWN0XG4gICAgUHJvcGVsbGVyLT4-K1Byb3BlbGxlcjogU2V0IExhYmVscyAmIEFubm90YXRpb25zXG4gICAgUHJvcGVsbGVyLT4-K0FwaVNlcnZlcjogQ3JlYXRlIE9iamVjdCAoZS5nLiBQb2QpXG4gICAgQXBpU2VydmVyLT4-K1BvZCBXZWJob29rOiAvbXV0YXRlXG4gICAgUG9kIFdlYmhvb2stPj4rUG9kIFdlYmhvb2s6IExvb2t1cCBnbG9iYWxzXG4gICAgUG9kIFdlYmhvb2stPj4rUG9kIFdlYmhvb2s6IEluamVjdCBTZWNyZXQgQW5ub3RhdGlvbnMgKGUuZy4gSzhzLCBWYXVsdC4uLiBldGMuKVxuICAgIFBvZCBXZWJob29rLT4-LUFwaVNlcnZlcjogTXV0YXRlZCBQb2RcbiAgICBcbiAgICAgICAgICAgICIsIm1lcm1haWQiOnt9LCJ1cGRhdGVFZGl0b3IiOmZhbHNlfQ + +Where: + +1. Flyte invokes a plugin to create the K8s object. This can be a Pod or a more complex CRD (e.g. Spark, PyTorch, etc.) + + .. tip:: The plugin will ensure that labels and annotations are passed through to any Pod that will be spawned due to the creation of the CRD. + +3. Flyte will apply labels and annotations that are referenced to all secrets the task is requesting access to. +4. Flyte will send a POST request to ApiServer to create the object. +5. Before persisting the Pod, ApiServer will invoke all registered Pod Webhooks. Flyte's Pod Webhook will be called. +6. Flyte Pod Webhook will then lookup globally mounted secrets for each of the requested secrets. +7. If found, Pod Webhook will mount them directly in the Pod. If not found, it will inject the appropriate annotations to load the secrets for K8s (or Vault or Confidant or any other secret management system plugin configured) into the Pod. + +****************************** +How to Enable Secret Injection +****************************** + +This feature is available in Flytekit v0.17.0+. Here is an example of an annotated task: + +The webhook is included in all overlays in this repo. The deployment file creates (mainly) two things; a Job and a Deployment. + +1) flyte-pod-webhook-secrets Job: This job runs ``flytepropeller webhook init-certs`` command that issues self-signed + CA Certificate as well as a derived TLS certificate and its private key. It stores them into a new secret ``flyte-pod-webhook-secret``. +2) flyte-pod-webhook Deployment: This deployment creates the Webhook pod which creates a MutatingWebhookConfiguration + on startup. This serves as the registration contract with the ApiServer to know about the Webhook before it starts serving + traffic. + +******************* +Scaling the Webhook +******************* + +Vertical Scaling +================= + +To scale the Webhook to be able to process the number/rate of pods you need, you may need to configure a vertical `pod +autoscaler `_. + +Horizontal Scaling +================== + +The Webhook does not make any external API Requests in response to Pod mutation requests. It should be able to handle traffic +quickly, but a benchmark is needed. For horizontal scaling, adding additional replicas for the Pod in the +deployment should be sufficient. A single MutatingWebhookConfiguration object will be used, the same TLS certificate +will be shared across the pods and the Service created will automatically load balance traffic across the available pods. From a7fcbfd5876240087d446895623db8dce1707dea Mon Sep 17 00:00:00 2001 From: JakeNeyer Date: Wed, 24 Mar 2021 16:26:34 -0400 Subject: [PATCH 34/74] Add striveworks to users list in README (#865) Signed-off-by: Jake Neyer Signed-off-by: Haytham Abuelfutuh --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index cfeaf765ec..c5afbd6157 100644 --- a/README.md +++ b/README.md @@ -150,6 +150,7 @@ Resources that would help you get a better understanding of Flyte. - [Lyft L5 autonomous](https://self-driving.lyft.com/level5/) - [Spotify](https://www.spotify.com/) - [USU Group](https://www.usu.com/) +- [Striveworks](https://striveworks.us/) # Component Repos Repo | Language | Purpose | Status From 1e44c8d1450159a8a8a3ae9c574cc815fba95fd9 Mon Sep 17 00:00:00 2001 From: Samhita Alla Date: Thu, 25 Mar 2021 11:29:30 +0530 Subject: [PATCH 35/74] Update README.md (#855) * Updated README.md * Updated README.md * Updated README.md * Updated README.md * Updated README.md * Updated Links * Updated text in README * Modified TOC * Fixed README Signed-off-by: samhita-alla * Added Features #853 Signed-off-by: samhita-alla * Updated Features Signed-off-by: samhita-alla Signed-off-by: Haytham Abuelfutuh --- README.md | 423 +++++++++++++++++++++++++++++++----------------------- 1 file changed, 242 insertions(+), 181 deletions(-) diff --git a/README.md b/README.md index c5afbd6157..0bdf0ead81 100644 --- a/README.md +++ b/README.md @@ -1,210 +1,271 @@ -![Flyte Logo](rsts/images/flyte_lockup_gradient_on_light.png "Flyte Logo") - -[![Current Release](https://img.shields.io/github/release/lyft/flyte.svg)](https://github.com/lyft/flyte/releases/latest) -[![Sandbox Build](https://github.com/flyteorg/flyte/actions/workflows/sandbox.yml/badge.svg)](https://github.com/flyteorg/flyte/actions/workflows/sandbox.yml) -[![End to End tests](https://github.com/flyteorg/flyte/actions/workflows/tests.yml/badge.svg)](https://github.com/flyteorg/flyte/actions/workflows/tests.yml) -[![License](https://img.shields.io/badge/LICENSE-Apache2.0-ff69b4.svg)](http://www.apache.org/licenses/LICENSE-2.0.html) -![Commit activity](https://img.shields.io/github/commit-activity/w/lyft/flyte.svg?style=plastic) -![Commit since last release](https://img.shields.io/github/commits-since/lyft/flyte/latest.svg?style=plastic) -![GitHub milestones Completed](https://img.shields.io/github/milestones/closed/lyft/flyte?style=plastic) -![GitHub next milestone percentage](https://img.shields.io/github/milestones/progress-percent/lyft/flyte/11?style=plastic) -[![Docs](https://readthedocs.org/projects/flyte/badge/?version=latest&style=plastic)](https://flyte.rtfd.io) -![Twitter Follow](https://img.shields.io/twitter/follow/flyteorg?label=Follow&style=social) -[![Slack Status](https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&style=social)](https://forms.gle/UVuek9WfBoweiqcJA) - -Flyte is a production-grade, container-native, type-safe workflow and pipelines platform optimized for large scale processing and machine learning written in Golang. Workflows can be written in any language, with out of the box support for [Python](https://github.com/flyteorg/flytekit), [Java and Scala](https://github.com/spotify/flytekit-java). - ---- - -[HomePage](https://flyte.org) | -[Quickstart](#quickstart) | -[Documentation](https://docs.flyte.org/) | -[Features](#features) | -[Community & Resources](#community--resources) | -[Changelogs](CHANGELOG/) | -[Components](#component-repos) - ---- - -# Introduction -Flyte is a fabric that connects disparate computation backends using a type safe data dependency graph. It records all changes to a pipeline, making it possible to rewind time. It also stores + +

+ Flyte Logo +

+ +

+ Flyte +

+ +

+Flyte is a production-grade, container-native, type-safe workflow and pipelines platform optimized for large scale processing and machine learning written in Golang +

+ +

+ + Current Release + + + Sandbox Build + + + End-to-End Tests + + + License + + Commit Activity + Commits since Last Release + GitHub Milestones Completed + GitHub Next Milestone Percentage + + Docs + + Twitter Follow + + Slack Status + +

+ +

+ Home Page + · + Quick Start + · + Documentation + · + Features + · + Community & Resources + · + Changelogs + · + Components +

+ + + +## 💥 Introduction + +Flyte is a structured programming and distributed processing platform that enables highly concurrent, scalable and maintainable workflows for `Machine Learning` and `Data Processing`. It is a fabric that connects disparate computation backends using a type safe data dependency graph. It records all changes to a pipeline, making it possible to rewind time. It also stores a history of all executions and provides an intuitive UI, CLI and REST/gRPC API to interact with the computation. -Flyte is more than a workflow engine, it provides workflows as a core concepts, but it also provides a single unit of execution - tasks, as a top level concept. Multiple tasks arranged in a data -producer-consumer order creates a workflow. Flyte workflows are pure specification and can be created using any language. Every task can also by any language. We do provide first class support for -python, making it perfect for modern Machine Learning and Data processing pipelines. +Flyte is more than a workflow engine -- it provides `workflow` as a core concept and a single unit of execution called `task` as a top level concept. Multiple tasks arranged in a data +producer-consumer order create a workflow. + +`Workflows` and `Tasks` can be written in any language, with out of the box support for [Python](https://github.com/flyteorg/flytekit), [Java and Scala](https://github.com/spotify/flytekit-java). + + +

+ 🚀 Quick Start +

+ -# QuickStart -With [docker installed](https://docs.docker.com/get-docker/), run this command: +With [docker installed](https://docs.docker.com/get-docker/), run the following command: ```bash docker run --rm --privileged -p 30081:30081 -p 30084:30084 ghcr.io/flyteorg/flyte-sandbox ``` -This creates a local Flyte sandbox. Once the sandbox is ready, you should see the following message: ``Flyte is ready! Flyte UI is available at http://localhost:30081/console``. Go ahead and visit http://localhost:30081/console. -A quick visual tour of the console - +This creates a local Flyte sandbox. Once the sandbox is ready, you should see the following message: `Flyte is ready! Flyte UI is available at http://localhost:30081/console`. + +Go ahead and visit http://localhost:30081/console to view the Flyte dashboard. + +Here's a quick visual tour of the console. + ![Flyte console Example](https://github.com/flyteorg/flyte/raw/static-resources/img/first-run-console-2.gif) -Refer to [Docs - Getting Started](https://docs.flyte.org/en/latest/index.html) for complete end to end example. +To dig deeper into Flyte, refer to the [Documentation](https://docs.flyte.org/en/latest/index.html). + +## ⭐️ Current Deployments + +- [Freenome](https://www.freenome.com/) +- [Lyft Rideshare, Mapping](https://www.lyft.com/) +- [Lyft L5 autonomous](https://self-driving.lyft.com/level5/) +- [Spotify](https://www.spotify.com/) +- [USU Group](https://www.usu.com/) +- [Striveworks](https://striveworks.us/) + + +

+ 🔥 Features +

+ + +- Used at _Scale_ in production by **500+** users at Lyft with more than **1 million** executions and **40+ million** container executions per month +- Enables **collaboration across your organization**, as in: + - Execute distributed data pipelines/workflows + - Reuse tasks across projects, users, and workflows + - Backtrace to a specified workflow + - Compare results of training workflows over time and across pipelines + - Share workflows and tasks across your teams +- **[Quick registration](https://docs.flyte.org/projects/cookbook/en/latest/tutorial.html)** -- start locally and scale to the cloud instantly +- **Centralized Inventory** constituting Tasks, Workflows and Executions +- **gRPC / REST** interface to define and execute tasks and workflows +- **Type safe** construction of pipelines -- each task has an interface which is characterized by its input and output; thus, illegal construction of pipelines fails during declaration rather than at runtime +- Supports multiple **[data types](https://docs.flyte.org/projects/cookbook/en/latest/core.html)** for machine learning and data processing pipelines, such as Blobs (images, arbitrary files), Directories, Schema (columnar structured data), collections, maps etc. +- Memoization and Lineage tracking +- Workflow features: + - Start with one task, convert to a pipeline, attach **[multiple schedules](https://docs.flyte.org/projects/cookbook/en/latest/auto_core_remote_flyte/lp_schedules.html)**, trigger using a programmatic API, or on-demand + - Parallel step execution + - Extensible backend to add [customized plugin](https://docs.flyte.org/projects/cookbook/en/latest/auto_core_advanced/custom_task_plugin.html) experience (with simplified user experience) + - **[Branching](https://docs.flyte.org/projects/cookbook/en/latest/auto_core_intermediate/run_conditions.html)** + - Inline **[subworkflows](https://docs.flyte.org/projects/cookbook/en/latest/auto_core_intermediate/subworkflows.html)** (a workflow can be embeded within one node of the top level workflow) + - Distributed **remote child workflows** (a remote workflow can be triggered and statically verified at compile time) + - **[Array Tasks](https://docs.flyte.org/projects/cookbook/en/latest/auto_core_intermediate/map_task.html)** (map a function over a large dataset -- ensures controlled execution of thousands of containers) + - **[Dynamic workflow](https://docs.flyte.org/projects/cookbook/en/latest/auto_core_intermediate/dynamics.html)** creation and execution with runtime type safety + - Container side [plugins](https://docs.flyte.org/projects/cookbook/en/latest/plugins.html) with first class support in Python + - _PreAlpha_: Arbitrary flytekit-less containers supported ([RawContainer](https://docs.flyte.org/projects/cookbook/en/latest/auto_core_intermediate/raw_container.html)) +- Guaranteed **[reproducibility](https://docs.flyte.org/projects/cookbook/en/latest/auto_core_basic/task_cache.html)** of pipelines via: + - Versioned data, code and models + - Automatically tracked executions + - Declarative pipelines +- **Multi cloud support** (AWS, GCP and others) +- Extensible core, modularized, and deep observability +- Automated notifications to Slack, Email, and Pagerduty +- [Multi K8s cluster support](https://docs.flyte.org/projects/cookbook/en/latest/auto_plugins_pod/index.html) +- Out of the box support to run **[Spark jobs on K8s](https://docs.flyte.org/projects/cookbook/en/latest/auto_plugins_k8s_spark/index.html)**, **[Hive queries](https://docs.flyte.org/projects/cookbook/en/latest/auto_plugins_hive/index.html)**, etc. +- Snappy Console +- Python CLI and Golang CLI (flytectl) +- Written in **Golang** and optimized for large running jobs' performance -# Community & Resources -Resources that would help you get a better understanding of Flyte. +### In Progress + +- Grafana templates (user/system observability) +- Helm chart for Flyte +- Performance optimization +- Flink-K8s + +## 🔌 Available Plugins + +- Containers +- [K8s Pods](https://docs.flyte.org/projects/cookbook/en/latest/auto_plugins_pod/index.html) +- AWS Batch Arrays +- K8s Pod Arrays +- K8s Spark (native [Pyspark](https://docs.flyte.org/projects/cookbook/en/latest/auto_plugins_k8s_spark/index.html) and Java/Scala) +- AWS Athena +- [Qubole Hive](https://docs.flyte.org/projects/cookbook/en/latest/auto_plugins_hive/index.html) +- Presto Queries +- Distributed Pytorch (K8s Native) -- [Pytorch Operator](https://docs.flyte.org/projects/cookbook/en/latest/auto_plugins_kfpytorch/index.html) +- Sagemaker([builtin algorithms](https://docs.flyte.org/projects/cookbook/en/latest/auto_plugins_sagemaker_training/sagemaker_builtin_algo_training.html) & [custom models](https://docs.flyte.org/projects/cookbook/en/latest/auto_plugins_sagemaker_training/sagemaker_custom_training.html)) +- Distributed Tensorflow (K8s Native) - TFOperator +- Papermill notebook execution ([Python](https://github.com/lyft/flytekit/tree/master/plugins/papermill) and Spark) +- Type safe and data checking for Pandas dataframe using Pandera + +### In Queue + +- Reactive pipelines +- A lot more integrations! + + +

+ 📦 Component Repos +

+ + +| Repo | Language | Purpose | Status | +| --------------------------------------------------------- | ------------- | ---------------------------------------------- | ---------------- | +| [flyte](https://github.com/lyft/flyte) | Kustomize,RST | deployment, documentation, issues | Production-grade | +| [flyteidl](https://github.com/lyft/flyteidl) | Protobuf | interface definitions | Production-grade | +| [flytepropeller](https://github.com/lyft/flytepropeller) | Go | execution engine | Production-grade | +| [flyteadmin](https://github.com/lyft/flyteadmin) | Go | control plane | Production-grade | +| [flytekit](https://github.com/lyft/flytekit) | Python | python SDK and tools | Production-grade | +| [flyteconsole](https://github.com/lyft/flyteconsole) | Typescript | admin console | Production-grade | +| [datacatalog](https://github.com/lyft/datacatalog) | Go | manage input & output artifacts | Production-grade | +| [flyteplugins](https://github.com/lyft/flyteplugins) | Go | flyte plugins | Production-grade | +| [flytestdlib](https://github.com/lyft/flytestdlib) | Go | standard library | Production-grade | +| [flytesnacks](https://github.com/lyft/flytesnacks) | Python | examples, tips, and tricks | Incubating | +| [flytekit-java](https://github.com/spotify/flytekit-java) | Java/Scala | Java & scala SDK for authoring Flyte workflows | Incubating | +| [flytectl](https://github.com/lyft/flytectl) | Go | A standalone Flyte CLI | Incomplete | + +## 🔩 Production K8s Operators + +| Repo | Language | Purpose | +| --------------------------------------------------------------------- | -------- | ---------------------- | +| [Spark](https://github.com/GoogleCloudPlatform/spark-on-k8s-operator) | Go | Apache Spark batch | +| [Flink](https://github.com/lyft/flinkk8soperator) | Go | Apache Flink streaming | + + +

+ 🤝 Community & Resources +

+ + +Here are the resources that would help you get a better understanding of Flyte. + +### Communication Channels -## Communication channels - [Slack Org](https://forms.gle/UVuek9WfBoweiqcJA) - [Email list](https://groups.google.com/a/flyte.org/g/users) -## Biweekly Community Sync -- 📣  Flyte OSS Community Sync Every alternate Tuesday, 9am-10am PDT ([Checkout the events calendar & subscribe](https://calendar.google.com/calendar/embed?src=admin%40flyte.org&ctz=America%2FLos_Angeles) +### Biweekly Community Sync + +- 📣 **Flyte OSS Community Sync** happens every alternate Tuesday, 9am-10am PDT ([Checkout the events calendar & subscribe](https://calendar.google.com/calendar/embed?src=admin%40flyte.org&ctz=America%2FLos_Angeles)). Here's the [zoom link](https://us04web.zoom.us/j/71298741279?pwd=TDR1RUppQmxGaDRFdzBOa2lHN1dsZz09). +- Meeting notes and backlog of topics are captured in [doc](https://docs.google.com/document/d/1Jb6eOPOzvTaHjtPEVy7OR2O5qK1MhEs3vv56DX2dacM/edit#heading=h.c5ha25xc546e). +- If you'd like to revisit any community sync meeting that has happened, you can access the [video recordings](https://www.youtube.com/channel/UCNduEoLOToNo3nFVly-vUTQ). -- You can join the [zoom link]( https://us04web.zoom.us/j/71298741279?pwd=TDR1RUppQmxGaDRFdzBOa2lHN1dsZz09). -- Meeting notes and backlog of topics are captured in [Doc](https://docs.google.com/document/d/1Jb6eOPOzvTaHjtPEVy7OR2O5qK1MhEs3vv56DX2dacM/edit#heading=h.c5ha25xc546e) -- [Video Recordings](https://www.youtube.com/channel/UCNduEoLOToNo3nFVly-vUTQ) +### Conference Talks -## Conference Talks - Kubecon 2019 - Flyte: Cloud Native Machine Learning and Data Processing Platform [video](https://www.youtube.com/watch?v=KdUJGSP1h9U) | [deck](https://kccncna19.sched.com/event/UaYY/flyte-cloud-native-machine-learning-data-processing-platform-ketan-umare-haytham-abuelfutuh-lyft) - Kubecon 2019 - Running LargeScale Stateful workloads on Kubernetes at Lyft [video](https://www.youtube.com/watch?v=ECeVQoble0g) -- re:invent 2019 - Implementing ML workflows with Kubernetes and Amazon Sagemaker [video](https://youtu.be/G-wzIQQJKaE) +- re:invent 2019 - Implementing ML workflows with Kubernetes and Amazon Sagemaker [video](https://youtu.be/G-wzIQQJKaE) - Cloud-native machine learning at Lyft with AWS Batch and Amazon EKS [video](https://youtu.be/n_rRb8u1GSM) - OSS + ELC NA 2020 [splash](https://ossna2020.sched.com/event/313cec91aa38a430a25f9571039874b8) - Datacouncil [splash](https://docs.google.com/document/d/1ZsCDOZ5ZJBPWzCNc45FhNtYQOxYHz0PAu9lrtDVnUpw/edit) - FB AI@Scale [Making MLOps & DataOps a reality](https://www.facebook.com/atscaleevents/videos/ai-scale-flyte-making-mlops-and-dataops-a-reality/1047312585732459/) - [GAIC 2020](http://www.globalbigdataconference.com/seattle/global-artificial-intelligence-virtual-conference-122/speaker-details/ketan-umare-113746.html) -## Blog Posts - 1. [Introducing Flyte: A Cloud Native Machine Learning and Data Processing Platform](https://eng.lyft.com/introducing-flyte-cloud-native-machine-learning-and-data-processing-platform-fb2bb3046a59) - 2. [Building a Gateway to Flyte](https://eng.lyft.com/building-a-gateway-to-flyte-474b451b32c8) +### Blog Posts + +1. [Introducing Flyte: A Cloud Native Machine Learning and Data Processing Platform](https://eng.lyft.com/introducing-flyte-cloud-native-machine-learning-and-data-processing-platform-fb2bb3046a59) +2. [Building a Gateway to Flyte](https://eng.lyft.com/building-a-gateway-to-flyte-474b451b32c8) -## Podcasts -- TWIML&AI - [Scalable and Maintainable ML Workflows at Lyft - Flyte](https://twimlai.com/twiml-talk-343-scalable-and-maintainable-workflows-at-lyft-with-flyte-w-haytham-abuelfutuh-and-ketan-umare/) +### Podcasts + +- TWIML&AI - [Scalable and Maintainable ML Workflows at Lyft - Flyte](https://twimlai.com/twiml-talk-343-scalable-and-maintainable-workflows-at-lyft-with-flyte-w-haytham-abuelfutuh-and-ketan-umare/) - Software Engineering Daily - [Flyte: Lyft Data Processing Platform](https://softwareengineeringdaily.com/2020/03/12/flyte-lyft-data-processing-platform-with-allyson-gale-and-ketan-umare/) -- MLOps Coffee session - [Flyte: an open-source tool for scalable, extensible , and portable workflows](https://anchor.fm/mlops/episodes/MLOps-Coffee-Sessions-12-Flyte-an-open-source-tool-for-scalable--extensible---and-portable-workflows-eksa5k) - - -# Features - - Used at Scale in production by 500+ users at Lyft with more than *900k* workflow executed a month and more than *30+* million container executions per month - - Fast registration - from local to remote in one second. - - Centralized Inventory of Tasks, Workflows and Executions - - Single Task Execution support - Start executing a task and then convert it to a workflow - - gRPC / REST interface to define and executes tasks and workflows - - Type safe construction of pipelines, each task has an interface which is characterized by its input and outputs. Thus illegal construction of pipelines fails during declaration rather than at - runtime - - Types that help in creating machine learning and data processing pipelines like - Blobs (images, arbitrary files), Directories, Schema (columnar structured data), collections, maps etc - - Memoization and Lineage tracking - - Workflows features - * Multiple Schedules for every workflow - * Parallel step execution - * Extensible Backend to add customized plugin experiences (with simplified User experiences) - * Arbitrary container execution - * Branching - * Inline Subworkflows (a workflow can be embeded within one node of the top level workflow) - * Distributed Remote Child workflows (a remote workflow can be triggered and statically verified at compile time) - * Array Tasks (map some function over a large dataset, controlled execution of 1000's of containers) - * Dynamic Workflow creation and execution - with runtime type safety - * Container side plugins with first class support in python - * PreAlpha: Arbitrary flytekit less containers supported (RawContainer) - - Maintain an inventory of tasks and workflows - - Record history of all executions and executions (as long as they follow convention) are completely repeatable - - Multi Cloud support (AWS, GCP and others) - - Extensible core - - Modularized - - Automated notifications to Slack, Email, Pagerduty - - Deep observability - - Multi K8s cluster support - - Comes with many system supported out of the box on K8s like Spark etc. - - Snappy Console - - Python CLI - - Written in Golang and optimized for performance of large running jobs - - Golang CLI - flytectl - -## Inprogress - - Grafana templates (user/system observability) - - helm chart for Flyte - - Performance optimization - - Flink-K8s - -# Available Plugins - - Containers - - K8s Pods - - AWS Batch Arrays - - K8s Pod arrays - - K8s Spark (native pyspark and java/scala) - - AWS Athena - - Qubole Hive - - Presto Queries - - Distributed Pytorch (K8s Native) - Pytorch Operator - - Sagemaker (builtin algorithms & custom models) - - Distributed Tensorflow (K8s Native) - TFOperator - - Papermill Notebook execution (python and spark) - - Type safe and data checking for Pandas dataframe using Pandera - -## Coming Soon - - Reactive pipelines - - More integrations - - -# Current Usage -- [Freenome](https://www.freenome.com/) -- [Lyft Rideshare, Mapping](https://www.lyft.com/) -- [Lyft L5 autonomous](https://self-driving.lyft.com/level5/) -- [Spotify](https://www.spotify.com/) -- [USU Group](https://www.usu.com/) -- [Striveworks](https://striveworks.us/) +- MLOps Coffee session - [Flyte: an open-source tool for scalable, extensible, and portable workflows](https://anchor.fm/mlops/episodes/MLOps-Coffee-Sessions-12-Flyte-an-open-source-tool-for-scalable--extensible---and-portable-workflows-eksa5k) + +## 💖 Top Contributors -# Component Repos -Repo | Language | Purpose | Status ---- | --- | --- | --- -[flyte](https://github.com/lyft/flyte) | Kustomize,RST | deployment, documentation, issues | Production-grade -[flyteidl](https://github.com/lyft/flyteidl) | Protobuf | interface definitions | Production-grade -[flytepropeller](https://github.com/lyft/flytepropeller) | Go | execution engine | Production-grade -[flyteadmin](https://github.com/lyft/flyteadmin) | Go | control plane | Production-grade -[flytekit](https://github.com/lyft/flytekit) | Python | python SDK and tools | Production-grade -[flyteconsole](https://github.com/lyft/flyteconsole) | Typescript | admin console | Production-grade -[datacatalog](https://github.com/lyft/datacatalog) | Go | manage input & output artifacts | Production-grade -[flyteplugins](https://github.com/lyft/flyteplugins) | Go | flyte plugins | Production-grade -[flytestdlib](https://github.com/lyft/flytestdlib) | Go | standard library | Production-grade -[flytesnacks](https://github.com/lyft/flytesnacks) | Python | examples, tips, and tricks | Incubating -[flytekit-java](https://github.com/spotify/flytekit-java) | Java/Scala | Java & scala SDK for authoring Flyte workflows | Incubating -[flytectl](https://github.com/lyft/flytectl) | Go | A standalone Flyte CLI | Incomplete - - -# Production K8s Operators - -Repo | Language | Purpose ---- | --- | --- -[Spark](https://github.com/GoogleCloudPlatform/spark-on-k8s-operator) | Go | Apache Spark batch -[Flink](https://github.com/lyft/flinkk8soperator) | Go | Apache Flink streaming - -# Top Contributors -Thank you to the community for making Flyte possible. - - [@wild-endeavor](https://github.com/wild-endeavor) - - [@katrogan](https://github.com/katrogan) - - [@EngHabu](https://github.com/EngHabu) - - [@akhurana001](https://github.com/akhurana001) - - [@anandswaminathan](https://github.com/anandswaminathan) - - [@kanterov](https://github.com/kanterov) - - [@honnix](https://github.com/honnix) - - [@jeevb](https://github.com/jeevb) - - [@jonathanburns](https://github.com/jonathanburns) - - [@migueltol22](https://github.com/migueltol22) - - [@varshaparthay](https://github.com/varshaparthay) - - [@pingsutw](https://github.com/pingsutw) - - [@narape](https://github.com/narape) - - [@lu4nm3](https://github.com/lu4nm3) - - [@bnsblue](https://github.com/bnsblue) - - [@RubenBarragan](https://github.com/RubenBarragan) - - [@schottra](https://github.com/schottra) - - [@evalsocket](https://github.com/evalsocket) - - [@matthewphsmith](https://github.com/matthewphsmith) - - [@slai](https://github.com/slai) - - [@derwiki](https://github.com/derwiki) - - [@tnsetting](https://github.com/tnsetting) - - [@jbrambleDC](https://github.com/jbrambleDC) - - [@igorvalko](https://github.com/igorvalko) - - [@chanadian](https://github.com/chanadian) - - [@surindersinghp](https://github.com/surindersinghp) - - [@vsbus](https://github.com/vsbus) - - [@catalinii](https://github.com/catalinii) - - [@kumare3](https://github.com/kumare3) +A big thank you to the community for making Flyte possible! +- [@wild-endeavor](https://github.com/wild-endeavor) +- [@katrogan](https://github.com/katrogan) +- [@EngHabu](https://github.com/EngHabu) +- [@akhurana001](https://github.com/akhurana001) +- [@anandswaminathan](https://github.com/anandswaminathan) +- [@kanterov](https://github.com/kanterov) +- [@honnix](https://github.com/honnix) +- [@jeevb](https://github.com/jeevb) +- [@jonathanburns](https://github.com/jonathanburns) +- [@migueltol22](https://github.com/migueltol22) +- [@varshaparthay](https://github.com/varshaparthay) +- [@pingsutw](https://github.com/pingsutw) +- [@narape](https://github.com/narape) +- [@lu4nm3](https://github.com/lu4nm3) +- [@bnsblue](https://github.com/bnsblue) +- [@RubenBarragan](https://github.com/RubenBarragan) +- [@schottra](https://github.com/schottra) +- [@evalsocket](https://github.com/evalsocket) +- [@matthewphsmith](https://github.com/matthewphsmith) +- [@slai](https://github.com/slai) +- [@derwiki](https://github.com/derwiki) +- [@tnsetting](https://github.com/tnsetting) +- [@jbrambleDC](https://github.com/jbrambleDC) +- [@igorvalko](https://github.com/igorvalko) +- [@chanadian](https://github.com/chanadian) +- [@surindersinghp](https://github.com/surindersinghp) +- [@vsbus](https://github.com/vsbus) +- [@catalinii](https://github.com/catalinii) +- [@kumare3](https://github.com/kumare3) From 0ae5c5fcceb4e44799a9ef71f8a2d7ab23d6819c Mon Sep 17 00:00:00 2001 From: Ketan Umare <16888709+kumare3@users.noreply.github.com> Date: Thu, 25 Mar 2021 11:03:28 -0700 Subject: [PATCH 36/74] Copilot version updated (#866) Signed-off-by: Ketan Umare Signed-off-by: Haytham Abuelfutuh --- deployment/eks/flyte_generated.yaml | 10 +++++----- deployment/gcp/flyte_generated.yaml | 10 +++++----- deployment/sandbox/flyte_generated.yaml | 10 +++++----- deployment/test/flyte_generated.yaml | 10 +++++----- .../headless/config/propeller/plugins/copilot.yaml | 2 +- 5 files changed, 21 insertions(+), 21 deletions(-) diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index 66576a38a3..efe1d383c1 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -8240,7 +8240,7 @@ data: k8s: co-pilot: name: "flyte-copilot-" - image: "ghcr.io/lyft/flyteplugins/flytecopilot:dc4bdbd61cac88a39a5ff43e40f026bdbc2c78a2" + image: "ghcr.io/flyteorg/flytecopilot:v0.5.28" start-timeout: "30s" core.yaml: | propeller: @@ -8379,7 +8379,7 @@ data: cloudwatch-log-group: kind: ConfigMap metadata: - name: flyte-propeller-config-ct29cf4cch + name: flyte-propeller-config-727k54546d namespace: flyte --- apiVersion: v1 @@ -8680,7 +8680,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-ct29cf4cch + name: flyte-propeller-config-727k54546d name: config-volume - name: webhook-certs secret: @@ -8898,7 +8898,7 @@ spec: serviceAccountName: flytepropeller volumes: - configMap: - name: flyte-propeller-config-ct29cf4cch + name: flyte-propeller-config-727k54546d name: config-volume --- apiVersion: apps/v1 @@ -9129,7 +9129,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-ct29cf4cch + name: flyte-propeller-config-727k54546d name: config-volume ttlSecondsAfterFinished: 0 --- diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index 2480b3640d..79f76f71a4 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -8231,7 +8231,7 @@ data: k8s: co-pilot: name: "flyte-copilot-" - image: "ghcr.io/lyft/flyteplugins/flytecopilot:dc4bdbd61cac88a39a5ff43e40f026bdbc2c78a2" + image: "ghcr.io/flyteorg/flytecopilot:v0.5.28" start-timeout: "30s" core.yaml: | propeller: @@ -8366,7 +8366,7 @@ data: stackdriver-logresourcename: k8s_container kind: ConfigMap metadata: - name: flyte-propeller-config-7288ccf2f8 + name: flyte-propeller-config-k8km2mmdgf namespace: flyte --- apiVersion: v1 @@ -8712,7 +8712,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-7288ccf2f8 + name: flyte-propeller-config-k8km2mmdgf name: config-volume - name: webhook-certs secret: @@ -8930,7 +8930,7 @@ spec: serviceAccountName: flytepropeller volumes: - configMap: - name: flyte-propeller-config-7288ccf2f8 + name: flyte-propeller-config-k8km2mmdgf name: config-volume --- apiVersion: apps/v1 @@ -9161,7 +9161,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-7288ccf2f8 + name: flyte-propeller-config-k8km2mmdgf name: config-volume ttlSecondsAfterFinished: 0 --- diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index 24c6f4928e..5f162c1391 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -2238,7 +2238,7 @@ data: k8s: co-pilot: name: "flyte-copilot-" - image: "ghcr.io/lyft/flyteplugins/flytecopilot:dc4bdbd61cac88a39a5ff43e40f026bdbc2c78a2" + image: "ghcr.io/flyteorg/flytecopilot:v0.5.28" start-timeout: "30s" core.yaml: | propeller: @@ -2324,7 +2324,7 @@ data: kubernetes-template-uri: "http://localhost:30082/#/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace }}" kind: ConfigMap metadata: - name: flyte-propeller-config-c8kkg4fth7 + name: flyte-propeller-config-6gd7cgkkdt namespace: flyte --- apiVersion: v1 @@ -2816,7 +2816,7 @@ spec: secret: secretName: user-info - configMap: - name: flyte-propeller-config-c8kkg4fth7 + name: flyte-propeller-config-6gd7cgkkdt name: config-volume - name: webhook-certs secret: @@ -3034,7 +3034,7 @@ spec: serviceAccountName: flytepropeller volumes: - configMap: - name: flyte-propeller-config-c8kkg4fth7 + name: flyte-propeller-config-6gd7cgkkdt name: config-volume --- apiVersion: apps/v1 @@ -3522,7 +3522,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-c8kkg4fth7 + name: flyte-propeller-config-6gd7cgkkdt name: config-volume ttlSecondsAfterFinished: 0 --- diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index 59d5e1a0d5..2296b79bbd 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -377,7 +377,7 @@ data: k8s: co-pilot: name: "flyte-copilot-" - image: "ghcr.io/lyft/flyteplugins/flytecopilot:dc4bdbd61cac88a39a5ff43e40f026bdbc2c78a2" + image: "ghcr.io/flyteorg/flytecopilot:v0.5.28" start-timeout: "30s" core.yaml: | propeller: @@ -458,7 +458,7 @@ data: kind: ConfigMap metadata: - name: flyte-propeller-config-cbbhf652h4 + name: flyte-propeller-config-dtdt5gbgg2 namespace: flyte --- apiVersion: v1 @@ -670,7 +670,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-cbbhf652h4 + name: flyte-propeller-config-dtdt5gbgg2 name: config-volume - name: webhook-certs secret: @@ -854,7 +854,7 @@ spec: serviceAccountName: flytepropeller volumes: - configMap: - name: flyte-propeller-config-cbbhf652h4 + name: flyte-propeller-config-dtdt5gbgg2 name: config-volume --- apiVersion: apps/v1 @@ -958,7 +958,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-cbbhf652h4 + name: flyte-propeller-config-dtdt5gbgg2 name: config-volume ttlSecondsAfterFinished: 0 --- diff --git a/kustomize/base/single_cluster/headless/config/propeller/plugins/copilot.yaml b/kustomize/base/single_cluster/headless/config/propeller/plugins/copilot.yaml index 09abe54a38..de03e858c6 100644 --- a/kustomize/base/single_cluster/headless/config/propeller/plugins/copilot.yaml +++ b/kustomize/base/single_cluster/headless/config/propeller/plugins/copilot.yaml @@ -2,5 +2,5 @@ plugins: k8s: co-pilot: name: "flyte-copilot-" - image: "ghcr.io/lyft/flyteplugins/flytecopilot:dc4bdbd61cac88a39a5ff43e40f026bdbc2c78a2" + image: "ghcr.io/flyteorg/flytecopilot:v0.5.28" start-timeout: "30s" From c98495d88f39d48c3d62c6c37827dc61938f1f05 Mon Sep 17 00:00:00 2001 From: SandraGH5 <80421934+SandraGH5@users.noreply.github.com> Date: Thu, 25 Mar 2021 14:59:49 -0700 Subject: [PATCH 37/74] Update index.rst (#853) * Update index.rst Homepage edits. The last 3 lines will eventually become links to the top row headings Hongxun will add to the website. * Update rsts/index.rst Co-authored-by: Niels Bantilan * Update rsts/index.rst Co-authored-by: Haytham Abuelfutuh * Update rsts/index.rst Co-authored-by: Haytham Abuelfutuh * Update index.rst * Update rsts/index.rst Co-authored-by: Ketan Umare <16888709+kumare3@users.noreply.github.com> * Update rsts/index.rst Co-authored-by: Ketan Umare <16888709+kumare3@users.noreply.github.com> * Update rsts/index.rst Co-authored-by: Ketan Umare <16888709+kumare3@users.noreply.github.com> * Update rsts/index.rst Co-authored-by: Ketan Umare <16888709+kumare3@users.noreply.github.com> Co-authored-by: Niels Bantilan Co-authored-by: Haytham Abuelfutuh Co-authored-by: Ketan Umare <16888709+kumare3@users.noreply.github.com> Signed-off-by: Haytham Abuelfutuh --- rsts/index.rst | 77 +++++++++++++++++++++++++------------------------- 1 file changed, 39 insertions(+), 38 deletions(-) diff --git a/rsts/index.rst b/rsts/index.rst index e3b1dd146b..a8c4f3ecd7 100644 --- a/rsts/index.rst +++ b/rsts/index.rst @@ -3,41 +3,42 @@ Flyte .. image:: images/flyte_lockup_gradient_on_light.png -Flyte is a structured programming and distributed processing platform created at Lyft that enables highly concurrent, scalable and maintainable workflows for machine learning and data processing. -Welcome to the documentation hub for Flyte. - -.. toctree:: - :maxdepth: 1 - :caption: Getting Started - :name: gettingstartedtoc - - getting_started/first_run - getting_started/first_example - getting_started/run_on_flyte - getting_started/learn - -.. toctree:: - :caption: How-Tos - :maxdepth: 1 - :name: howtotoc - - plugins/index - howto/index - -.. toctree:: - :caption: Deep Dive - :maxdepth: 1 - :name: divedeeptoc - - dive_deep/index - reference/index - -.. toctree:: - :caption: Contributor Guide - :maxdepth: 1 - :name: roadmaptoc - - community/index - community/docs - community/roadmap - community/compare +Meet Flyte +========== + +Flyte is an open-source, container-native, structured programming and distributed processing platform. It enables highly concurrent, scalable and maintainable workflows for machine learning and data processing. + +Created at Lyft, Flyte provides first class support for Python, Java, and Scala, and is built directly on Kubernetes for all the benefits containerization provides: portability, scalability, and reliability. + +Flyte provides a single unit of execution (task) as a top-level concept. Multiple tasks arranged in a data producer-consumer order create the workflow, which is pure specification created in any language. + +Why Flyte? +========== + +Flyte's main purpose is to increase the development velocity for data processing and machine learning, enabling large-scale compute execution without the operational overhead. Teams can therefore focus on the business goals and not the infrastructure. + +What makes Flyte different? +--------------------------- + +* Container Native +* Extensible Backend & SDK’s +* Ergonomic SDK’s in Python, Java & Scala +* Versioned & Auditable - all actions are recorded +* Matches your workflow - Start with one task, convert to a pipeline, attach multiple schedules or trigger using a programmatic API or on-demand +* Battle-tested - millions of pipelines executed per month +* Vertically-Integrated Compute - serverless experience +* Deep understanding of data-lineage & provenance +* Operation Visibility - cost, performance, etc. +* Cross-Cloud Portable Pipelines + +At Lyft, Flyte has served production model training and data processing for over four years, becoming the de-facto platform for the Pricing, Locations, ETA, Mapping teams, Growth, Autonomous and other teams + +Whether you will be writing Flyte workflows, deploying the Flyte platform to your k8 cluster, or would like to extend and contribute to the architecture and design of Flyte, we have what you need. + +Welcome to the documentation hub for Flyte! + +Beginners: see :ref:`getting-started-firstrun` + +Intermediate Users: see :ref:`plugins` + +Advanced Users: see :ref:`divedeep` From 59e9b1ef7a0e4f191767f7148505a88072e675ed Mon Sep 17 00:00:00 2001 From: SandraGH5 <80421934+SandraGH5@users.noreply.github.com> Date: Thu, 25 Mar 2021 15:08:39 -0700 Subject: [PATCH 38/74] Update index.rst (#870) * Update index.rst * Update index.rst Signed-off-by: Haytham Abuelfutuh --- rsts/index.rst | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/rsts/index.rst b/rsts/index.rst index a8c4f3ecd7..ee33251783 100644 --- a/rsts/index.rst +++ b/rsts/index.rst @@ -3,6 +3,46 @@ Flyte .. image:: images/flyte_lockup_gradient_on_light.png +.. toctree:: + :maxdepth: 1 + :caption: Getting Started + :name: gettingstartedtoc + :hidden: + + getting_started/first_run + getting_started/first_example + getting_started/run_on_flyte + getting_started/learn + +.. toctree:: + :caption: How-Tos + :maxdepth: 1 + :name: howtotoc + :hidden: + + plugins/index + howto/index + +.. toctree:: + :caption: Deep Dive + :maxdepth: 1 + :name: divedeeptoc + :hidden: + + dive_deep/index + reference/index + +.. toctree:: + :caption: Contributor Guide + :maxdepth: 1 + :name: roadmaptoc + :hidden: + + community/index + community/docs + community/roadmap + community/compare + Meet Flyte ========== From d27e177cb6450ca6f12fc45e244685f1e8feae58 Mon Sep 17 00:00:00 2001 From: Niels Bantilan Date: Thu, 25 Mar 2021 18:36:35 -0400 Subject: [PATCH 39/74] show hidden toc (#871) Signed-off-by: cosmicBboy Signed-off-by: Haytham Abuelfutuh --- rsts/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rsts/conf.py b/rsts/conf.py index 3534a33e85..871f9190d5 100644 --- a/rsts/conf.py +++ b/rsts/conf.py @@ -118,7 +118,7 @@ # If False, expand all TOC entries "globaltoc_collapse": False, # If True, show hidden TOC entries - "globaltoc_includehidden": False, + "globaltoc_includehidden": True, # don't include home link in breadcrumb bar, since it's included # in the nav_links key below. "master_doc": False, From da88980a93fcf4da21ca2b947601cbac26055ac0 Mon Sep 17 00:00:00 2001 From: Yee Hing Tong Date: Thu, 25 Mar 2021 18:15:07 -0700 Subject: [PATCH 40/74] Switch end-to-end test to one that uses flytectl (#869) Signed-off-by: Haytham Abuelfutuh --- end2end/tests/endtoend.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/end2end/tests/endtoend.yaml b/end2end/tests/endtoend.yaml index dc807ddcb2..cb9d16e8fe 100644 --- a/end2end/tests/endtoend.yaml +++ b/end2end/tests/endtoend.yaml @@ -11,7 +11,7 @@ spec: command: - bash - -c - image: ghcr.io/flyteorg/flytetools:1fde462c72029d5f4d73ade8bff51af846598754 + image: ghcr.io/flyteorg/flytetools:78c917fb8d5dab74f5ff5f02d92e43d53ed4e7c3 imagePullPolicy: IfNotPresent name: flytetester resources: From de347138a2fe71418230f09b8d3bb55282ea6fa8 Mon Sep 17 00:00:00 2001 From: Niels Bantilan Date: Tue, 30 Mar 2021 11:39:49 -0400 Subject: [PATCH 41/74] update docs theme (#876) * update docs theme Signed-off-by: cosmicBboy * remove sphinx material and readthedocs search Signed-off-by: cosmicBboy * update flytekit link Signed-off-by: cosmicBboy * update Signed-off-by: cosmicBboy * add readthedocs sphinx search Signed-off-by: cosmicBboy * update community link Signed-off-by: cosmicBboy Signed-off-by: Haytham Abuelfutuh --- doc-requirements.in | 3 +- doc-requirements.txt | 42 ++++++-------- rsts/_static/custom.css | 11 ++++ rsts/conf.py | 66 +++++++++------------- rsts/getting_started.rst | 74 +++++++++++++++++++++++++ rsts/getting_started/first_example.rst | 60 -------------------- rsts/getting_started/first_run.rst | 26 --------- rsts/getting_started/learn.rst | 8 --- rsts/getting_started/run_on_flyte.rst | 77 -------------------------- rsts/index.rst | 67 ++++++++++++++-------- rsts/reference/index.rst | 15 +++-- 11 files changed, 182 insertions(+), 267 deletions(-) create mode 100644 rsts/_static/custom.css create mode 100644 rsts/getting_started.rst delete mode 100644 rsts/getting_started/first_example.rst delete mode 100644 rsts/getting_started/first_run.rst delete mode 100644 rsts/getting_started/learn.rst delete mode 100644 rsts/getting_started/run_on_flyte.rst diff --git a/doc-requirements.in b/doc-requirements.in index d84752e50a..79fbabb86a 100644 --- a/doc-requirements.in +++ b/doc-requirements.in @@ -1,6 +1,7 @@ +furo +readthedocs-sphinx-search sphinx sphinx-prompt -sphinx-material sphinx-code-include sphinx-autoapi sphinx-copybutton diff --git a/doc-requirements.txt b/doc-requirements.txt index b7e13e0c63..a801a17fd3 100644 --- a/doc-requirements.txt +++ b/doc-requirements.txt @@ -6,22 +6,22 @@ # alabaster==0.7.12 # via sphinx -astroid==2.4.2 +astroid==2.5.2 # via sphinx-autoapi babel==2.9.0 # via sphinx beautifulsoup4==4.9.3 # via + # furo # sphinx-code-include - # sphinx-material certifi==2020.12.5 # via requests chardet==4.0.0 # via requests -css-html-js-minify==2.5.5 - # via sphinx-material docutils==0.16 # via sphinx +furo==2021.3.20b30 + # via -r doc-requirements.in idna==2.10 # via requests imagesize==1.2.0 @@ -30,36 +30,34 @@ jinja2==2.11.3 # via # sphinx # sphinx-autoapi -lazy-object-proxy==1.4.3 +lazy-object-proxy==1.6.0 # via astroid -lxml==4.6.2 - # via sphinx-material markupsafe==1.1.1 # via jinja2 packaging==20.9 # via sphinx -pygments==2.7.4 +pygments==2.8.1 # via # sphinx # sphinx-prompt # sphinx-tabs pyparsing==2.4.7 # via packaging -python-slugify[unidecode]==4.0.1 - # via sphinx-material pytz==2021.1 # via babel pyyaml==5.4.1 # via sphinx-autoapi +readthedocs-sphinx-search==0.1.0 + # via -r doc-requirements.in requests==2.25.1 # via sphinx six==1.15.0 # via - # astroid # sphinx-code-include + # sphinxext-remoteliteralinclude snowballstemmer==2.1.0 # via sphinx -soupsieve==2.2 +soupsieve==2.2.1 # via beautifulsoup4 sphinx-autoapi==1.7.0 # via -r doc-requirements.in @@ -69,20 +67,18 @@ sphinx-copybutton==0.3.1 # via -r doc-requirements.in sphinx-issues==1.2.0 # via -r doc-requirements.in -sphinx-material==0.0.32 - # via -r doc-requirements.in -sphinx-prompt==1.3.0 +sphinx-prompt==1.4.0 # via -r doc-requirements.in -sphinx-tabs==2.0.1 +sphinx-tabs==2.1.0 # via -r doc-requirements.in -sphinx==3.4.3 +sphinx==3.5.3 # via # -r doc-requirements.in + # furo # sphinx-autoapi # sphinx-code-include # sphinx-copybutton # sphinx-issues - # sphinx-material # sphinx-prompt # sphinx-tabs # sphinxext-remoteliteralinclude @@ -98,15 +94,11 @@ sphinxcontrib-qthelp==1.0.3 # via sphinx sphinxcontrib-serializinghtml==1.1.4 # via sphinx -sphinxext-remoteliteralinclude==0.1.0 +sphinxext-remoteliteralinclude==0.2.0 # via -r doc-requirements.in -text-unidecode==1.3 - # via python-slugify unidecode==1.2.0 - # via - # python-slugify - # sphinx-autoapi -urllib3==1.26.3 + # via sphinx-autoapi +urllib3==1.26.4 # via requests wrapt==1.12.1 # via astroid diff --git a/rsts/_static/custom.css b/rsts/_static/custom.css new file mode 100644 index 0000000000..13c39dba3f --- /dev/null +++ b/rsts/_static/custom.css @@ -0,0 +1,11 @@ +h1, h2, h3, h4, h5, h6 { + font-weight: bold; +} + +.sidebar-logo { + max-width: 30%; +} + +.sidebar-tree .reference.external:after { + content: none; +} diff --git a/rsts/conf.py b/rsts/conf.py index 871f9190d5..9bfce8e5e9 100644 --- a/rsts/conf.py +++ b/rsts/conf.py @@ -16,6 +16,10 @@ # import sys # sys.path.insert(0, os.path.abspath('.')) +import sphinx.application +import sphinx.errors +sphinx.application.ExtensionError = sphinx.errors.ExtensionError + # -- Project information ----------------------------------------------------- project = u'Flyte' @@ -55,6 +59,7 @@ "sphinx_tabs.tabs", "sphinxext.remoteliteralinclude", "sphinx_issues", + "sphinx_search.extension", ] extlinks = { @@ -97,45 +102,28 @@ # html_favicon = "images/flyte_circle_gradient_1_4x4.png" html_logo = "images/flyte_circle_gradient_1_4x4.png" -html_static_path = [] -html_theme = "sphinx_material" +html_theme = "furo" +html_title = "Flyte Docs" + +html_static_path = ["_static"] +templates_path = ["_templates"] + +pygments_style = "tango" +pygments_dark_style = "paraiso-dark" + +html_css_files = [ + "custom.css", +] + html_theme_options = { - # Set the name of the project to appear in the navigation. - "nav_title": "Flyte", - # Set you GA account ID to enable tracking - "google_analytics_account": "G-YQL24L5CKY", - # Specify a base_url used to generate sitemap.xml. If not - # specified, then no sitemap will be built. - "base_url": "https://github.com/flyteorg/flyte", - # Set the color and the accent color - "color_primary": "deep-purple", - "color_accent": "blue", - # Set the repo location to get a badge with stats - "repo_url": "https://github.com/flyteorg/flyte/", - "repo_name": "flyte", - # Visible levels of the global TOC; -1 means unlimited - "globaltoc_depth": 1, - # If False, expand all TOC entries - "globaltoc_collapse": False, - # If True, show hidden TOC entries - "globaltoc_includehidden": True, - # don't include home link in breadcrumb bar, since it's included - # in the nav_links key below. - "master_doc": False, - # custom nav in breadcrumb bar - "nav_links": [ - {"href": "index", "internal": True, "title": "Flyte"}, - { - "href": "https://flytecookbook.readthedocs.io", - "internal": False, - "title": "Tutorials", - }, - { - "href": "https://flytekit.readthedocs.io", - "internal": False, - "title": "Flytekit Python Reference" - }, - ], + "light_css_variables": { + "color-brand-primary": "#4300c9", + "color-brand-content": "#4300c9", + }, + "dark_css_variables": { + "color-brand-primary": "#9D68E4", + "color-brand-content": "#9D68E4", + }, } # Theme options are theme-specific and customize the look and feel of a theme @@ -157,7 +145,7 @@ # default: ``['localtoc.html', 'relations.html', 'sourcelink.html', # 'searchbox.html']``. # -html_sidebars = {"**": ["logo-text.html", "globaltoc.html", "localtoc.html", "searchbox.html"]} +# html_sidebars = {"**": ["logo-text.html", "globaltoc.html", "localtoc.html", "searchbox.html"]} # -- Options for HTMLHelp output --------------------------------------------- diff --git a/rsts/getting_started.rst b/rsts/getting_started.rst new file mode 100644 index 0000000000..09b8e7c545 --- /dev/null +++ b/rsts/getting_started.rst @@ -0,0 +1,74 @@ +.. _gettingstarted: + +Getting started +--------------- + +.. rubric:: Estimated time: 3 minutes + +Prerequisites +############# + +Make sure you have `docker installed `__ and `git `__ installed, then install flytekit: + +.. prompt:: bash + + pip install flytekit + +Clone the `flytekit-python-template `__ repo to create our own git repository called ``flyteexamples``: + +.. prompt:: bash + + git clone git@github.com:flyteorg/flytekit-python-template.git flyteexamples + cd flyteexamples + + +Write Your First Flyte Workflow +############################### + + +Let's take a look at the example workflow in `myapp/workflows/example.py `__: + +.. rli:: https://raw.githubusercontent.com/flyteorg/flytekit-python-template/main/myapp/workflows/example.py + :language: python + +As you can see, a Flyte :std:doc:`task ` is the most basic unit of work in Flyte, +and you can compose multiple tasks into a :std:doc:`workflow `. Try running and +modifying the ``example.py`` script locally. + +Start a Local Flyte Backend +########################### + +Once you're happy with the ``example.py`` script, run the following command in your terminal: + +.. prompt:: bash + + docker run --rm --privileged -p 30081:30081 -p 30082:30082 -p 30084:30084 ghcr.io/flyteorg/flyte-sandbox + +When you see the message ``Flyte is ready!``, your local sandbox should be ready on http://localhost:30081/console. + +Register Your Workflows +########################### + +Now we're ready to ship your code to the Flyte backend by running the following command: + +.. prompt:: bash + + FLYTE_AWS_ENDPOINT=http://localhost:30084/ FLYTE_AWS_ACCESS_KEY_ID=minio FLYTE_AWS_SECRET_ACCESS_KEY=miniostorage make fast_register + +Run Your Workflows +################## + +To run a workflow, go to http://localhost:30081/console/projects/flyteexamples/workflows and then follow these steps: + +1. Select the ``hello_world`` workflow +2. Click the **Launch Workflow** button in the upper right corner +3. Update the ``name`` input argument +4. Proceed to **Launch** to trigger an execution + +.. rubric:: 🎉 Congratulations, you just ran your first Flyte workflow 🎉 + + +Next Steps: Tutorials +##################### + +To experience the full capabilities of Flyte, try out the `Flytekit Tutorials `__ 🛫 diff --git a/rsts/getting_started/first_example.rst b/rsts/getting_started/first_example.rst deleted file mode 100644 index fc6c0e09b7..0000000000 --- a/rsts/getting_started/first_example.rst +++ /dev/null @@ -1,60 +0,0 @@ -.. _getting-started-first-example: - -############################### -Write Your First Flyte Workflow -############################### - -By the end of this guide you will become familiar with how easy it is to author a Flyte workflow and run it locally. - -.. rubric:: Estimated time: <3 minutes - - -Prerequisites -************* - -#. Ensure that you have `git `__ installed. - -#. Let us make a virutal environment **(recommended)** - and then install flytekit using - ``--pre`` is used because we are currently using the beta version of flytekit 0.16.0, this introduces a completely new SDK for authoring workflows - :: - - pip install --pre flytekit - - -#. Now we will use the ``flytekit-python-template`` repo to create our own git repository called ``flyteexamples`` :: - - git clone git@github.com:flyteorg/flytekit-python-template.git flyteexamples - cd flyteexamples - rm -rf .git - git init - - -Flyte Tasks and Workflows -************************* - -Take a look at the example workflow found in `myapp/workflows/example.py `__ - -.. rli:: https://raw.githubusercontent.com/flyteorg/flytekit-python-template/main/myapp/workflows/example.py - :language: python - -The most basic Flyte primitive is a :std:doc:`task ` - -Flyte tasks are units of work that can be composed in a :std:doc:`workflow ` - -You can call this task: - -.. code-block:: python - - greet(name="world") - -and iterate it locally before adding it to part of a larger overall workflow. - -Similarly, you can call this workflow: - -.. code-block:: python - - hello_world(name=...) - -and iterate it locally before registering it with Flyte. - -.. tip:: Every invocation of a Flyte workflow requires specifying keyword arguments as in the example - ``hello_world(name="name")``. Calling the workflow without the keyword ``name`` will raise an ``AssertionError``. diff --git a/rsts/getting_started/first_run.rst b/rsts/getting_started/first_run.rst deleted file mode 100644 index 404a3a7fda..0000000000 --- a/rsts/getting_started/first_run.rst +++ /dev/null @@ -1,26 +0,0 @@ -.. currentmodule:: firstrun - -.. _getting-started-firstrun: - -########################### -Start a Local Flyte Backend -########################### - -.. rubric:: Estimated time: 2 minutes - -Flyte enables scalable, reproducible and reliable orchestration of massively large workflows. To get a sense of the product, a minimalist version of the Flyte system is packaged into a Docker image. - -With `docker installed `__, run this command: :: - - docker run --rm --privileged -p 30081:30081 -p 30082:30082 -p 30084:30084 ghcr.io/flyteorg/flyte-sandbox - -This creates a local Flyte sandbox. Once the sandbox is ready, you should see the following message: - -``Flyte is ready! Flyte UI is available at http://localhost:30081/console``. - -Go ahead and visit http://localhost:30081/console to check it out. - -Below is a quick visual tour for launching your first Workflow: - -.. image:: https://github.com/flyteorg/flyte/raw/static-resources/img/first-run-console-2.gif - :alt: A quick visual tour for launching your first Workflow. diff --git a/rsts/getting_started/learn.rst b/rsts/getting_started/learn.rst deleted file mode 100644 index dad96a1142..0000000000 --- a/rsts/getting_started/learn.rst +++ /dev/null @@ -1,8 +0,0 @@ -.. _getting-started-learn: - -############################## -Next Steps: Flytekit Tutorials -############################## - -Flytekit provides programmatic access to Flyte's functionality, with SDK support in Python and Java. -The best way to see the capabilities of Flyte is by trying out the `Flytekit Tutorials `__. diff --git a/rsts/getting_started/run_on_flyte.rst b/rsts/getting_started/run_on_flyte.rst deleted file mode 100644 index 423e111669..0000000000 --- a/rsts/getting_started/run_on_flyte.rst +++ /dev/null @@ -1,77 +0,0 @@ -.. _getting-started-run-on-flyte: - -##################################### -Run Your Workflow on a Flyte Backend -##################################### - -************************ -Installing Flyte Locally -************************ - -This guide will walk you through: - -* A quick installation of Flyte on your device - -* How to register and execute your workflows against this deployment. - -(The tips below have an option to install Flyte on a cloud provider as well) - -.. rubric:: Estimated time: 1 minute - -Prerequisites -============= - -1. Follow :ref:`getting-started-firstrun` - -2. Access FlyteConsole on http://localhost:30081/console - -.. tip:: To check if your flyte-sandbox container is running you can run ``docker ps`` and it should show image ``ghcr.io/flyteorg/flyte-sandbox`` running - -.. tip:: If you prefer using k3d, Minikube, docker for mac, or a hosted Kubernetes cluster like AWS-EKS, GCP-GKE, Azure Kubernetes refer to :ref:`howto-sandbox`. It is recommended that you use a simple Docker based approach when you are first getting started with Flyte. - -.. _getting-started-run-flyte-laptop: - -**************************** -Running Your Flyte Workflows -**************************** - -Register Your Workflows -======================= -Registration is the process of shipping your code to the Flyte backend. This creates an immutable, versioned record of your code with the FlyteAdmin service. - -From within root directory of ``flyteexamples`` you previously created with the :ref:`Write Your First Workflow tutorial `, -feel free to make any changes and then register: :: - - FLYTE_AWS_ENDPOINT=http://localhost:30084/ FLYTE_AWS_ACCESS_KEY_ID=minio \ - FLYTE_AWS_SECRET_ACCESS_KEY=miniostorage make fast_register - - -.. tip:: Flyte sandbox uses minio as a substitute for S3/GCS etc. It is port-forwarded in the first command to 30084. If you use S3/GCS or a different port-forward you can drop or change the ``FLYTE_AWS_ENDPOINT`` accordingly. - -.. rubric:: It's that simple! - -Run Your Workflows -================== - -Visit the page housing workflows registered to your project at: -`http://localhost:30081/console/projects/flyteexamples/workflows `__ - -Once you have accessed your workflows, - -* Select your workflow -* Click the bright purple "Launch Workflow" button in the upper right -* Update the "name" input argument -* Proceed to launch to trigger an execution - -.. note:: - - After registration, Flyte Workflows exist in the FlyteAdmin service and can be triggered using: - - Console - - Command line - - Directly invoking the REST API - - On a schedule - - -Create a New Project --------------------- -Visit :ref:`howto_new_project`. diff --git a/rsts/index.rst b/rsts/index.rst index ee33251783..4bd7d9cafd 100644 --- a/rsts/index.rst +++ b/rsts/index.rst @@ -1,18 +1,14 @@ -Flyte -===== -.. image:: images/flyte_lockup_gradient_on_light.png .. toctree:: :maxdepth: 1 - :caption: Getting Started - :name: gettingstartedtoc + :name: mainsections :hidden: - getting_started/first_run - getting_started/first_example - getting_started/run_on_flyte - getting_started/learn + getting_started + Tutorials + reference/index + community/index .. toctree:: :caption: How-Tos @@ -30,15 +26,14 @@ Flyte :hidden: dive_deep/index - reference/index .. toctree:: - :caption: Contributor Guide + :caption: Developers :maxdepth: 1 :name: roadmaptoc :hidden: - community/index + Join the Community community/docs community/roadmap community/compare @@ -48,17 +43,40 @@ Meet Flyte Flyte is an open-source, container-native, structured programming and distributed processing platform. It enables highly concurrent, scalable and maintainable workflows for machine learning and data processing. -Created at Lyft, Flyte provides first class support for Python, Java, and Scala, and is built directly on Kubernetes for all the benefits containerization provides: portability, scalability, and reliability. +Created at `Lyft `__, Flyte provides first class support for Python, Java, and Scala, and is built directly on Kubernetes for all the benefits containerization provides: portability, scalability, and reliability. + +The core unit of execution in Flyte is the ``task``, which you can easily write with the Flytekit Python SDK: + +.. code:: python + + @task + def greet(name: str) -> str: + return f"Welcome, {name}!" + +You can compose one or more tasks to create a ``workflow``: + +.. code:: python + + @task + def add_question(greeting: str) -> str: + return f"{greeting} How are you?" + + @workflow + def welcome(name: str) -> str: + greeting = greet(name=name) + return add_question(greeting=greeting) + + welcome("Traveler") + # Output: "Welcome, Traveler! How are you?" -Flyte provides a single unit of execution (task) as a top-level concept. Multiple tasks arranged in a data producer-consumer order create the workflow, which is pure specification created in any language. Why Flyte? ========== Flyte's main purpose is to increase the development velocity for data processing and machine learning, enabling large-scale compute execution without the operational overhead. Teams can therefore focus on the business goals and not the infrastructure. -What makes Flyte different? ---------------------------- +Core Features +------------- * Container Native * Extensible Backend & SDK’s @@ -71,14 +89,17 @@ What makes Flyte different? * Operation Visibility - cost, performance, etc. * Cross-Cloud Portable Pipelines -At Lyft, Flyte has served production model training and data processing for over four years, becoming the de-facto platform for the Pricing, Locations, ETA, Mapping teams, Growth, Autonomous and other teams - -Whether you will be writing Flyte workflows, deploying the Flyte platform to your k8 cluster, or would like to extend and contribute to the architecture and design of Flyte, we have what you need. +Who's Using Flyte? +------------------ -Welcome to the documentation hub for Flyte! +At `Lyft `__, Flyte has served production model training and data processing for over four years, becoming the de-facto platform for the Pricing, Locations, ETA, Mapping teams, Growth, Autonomous and other teams. -Beginners: see :ref:`getting-started-firstrun` +Next Steps +---------- -Intermediate Users: see :ref:`plugins` +Whether you want to write Flyte workflows, deploy the Flyte platform to your k8 cluster, or extend and contribute to the architecture and design of Flyte, we have what you need. -Advanced Users: see :ref:`divedeep` +* :ref:`Get Started ` +* :ref:`Main Concepts ` +* :ref:`Extend Flyte ` +* :ref:`Join the Community ` diff --git a/rsts/reference/index.rst b/rsts/reference/index.rst index 32281a70c4..c27aeaa579 100644 --- a/rsts/reference/index.rst +++ b/rsts/reference/index.rst @@ -1,8 +1,8 @@ .. _reference: -################################# -API, CLI and Component Reference -################################# +############# +API Reference +############# .. toctree:: @@ -10,10 +10,10 @@ API, CLI and Component Reference :caption: API Reference :name: apitoc - Flyte Core Language Specification (FlyteIDL) - Flytekit Python - Flytekit JAVA - Flytectl CLI reference + Flytekit Python + Flytekit Java + FlyteIDL + Flytectl .. toctree:: @@ -25,4 +25,3 @@ API, CLI and Component Reference FlyteAdmin FlytePlugins DataCatalog - Flytectl \ No newline at end of file From 52a0fb34fd750e5bba69fd11cab88ed7550a8f4f Mon Sep 17 00:00:00 2001 From: Niels Bantilan Date: Wed, 31 Mar 2021 11:59:55 -0400 Subject: [PATCH 42/74] dark theme updates (#883) Signed-off-by: cosmicBboy Signed-off-by: Haytham Abuelfutuh --- rsts/_static/custom.css | 80 +++++++++++++++++++++++++++++++++++++++++ rsts/conf.py | 2 +- 2 files changed, 81 insertions(+), 1 deletion(-) diff --git a/rsts/_static/custom.css b/rsts/_static/custom.css index 13c39dba3f..d9851b7d8f 100644 --- a/rsts/_static/custom.css +++ b/rsts/_static/custom.css @@ -6,6 +6,86 @@ h1, h2, h3, h4, h5, h6 { max-width: 30%; } + .sidebar-tree .reference.external:after { content: none; } + +.sphx-glr-thumbcontainer { + background-color: transparent; + border: transparent; +} + +.sphx-glr-thumbcontainer:hover { + border: transparent; +} + +div.sphx-glr-download a { + color:white; + background-color: #9d68e4cf; + background-image: none; + border: 1px solid #9d68e4cf; +} + +div.sphx-glr-download a:hover { + background-color: #8b48e2cf; + box-shadow: none; +} + +div.sphx-glr-thumbcontainer a.headerlink { + display: none; +} + +div.sphx-glr-thumbcontainer:hover { + border-color: white; + box-shadow: none; +} + +.sphx-glr-script-out .highlight pre { + background-color: #f8f8f8; +} + +p.sphx-glr-script-out { + padding-top: 0em; +} + +.search__outer::-webkit-scrollbar-track { + border-radius: 0px; +} + +@media (prefers-color-scheme: dark) { + .search__outer { + background-color: #131416 !important; + border: 1px solid #131416 !important; + } + .search__outer__input { + background-color: #1a1c1e !important; + } + .search__result__single { + border-bottom: #303335 !important; + } + .outer_div_page_results:hover { + background-color: black; + } + .search__result__title, .rtd_ui_search_subtitle { + color: #9D68E4 !important; + border-bottom: 1px solid #9D68E4 !important; + } + .search__outer .search__result__title span, .search__outer .search__result__content span { + background-color: #9d68e454; + } + .search__result__subheading, .search__result__content { + color: #ffffffd9 !important; + } + .search__outer::-webkit-scrollbar-track { + background-color: #131416 !important; + } + .rtd__search__credits { + background-color: #1a1c1e !important; + border: 1px solid #1a1c1e !important; + color: #81868d !important; + } + .rtd__search__credits a, .search__error__box { + color: #9ca0a5 !important; + } + } diff --git a/rsts/conf.py b/rsts/conf.py index 9bfce8e5e9..f778cbe4de 100644 --- a/rsts/conf.py +++ b/rsts/conf.py @@ -109,7 +109,7 @@ templates_path = ["_templates"] pygments_style = "tango" -pygments_dark_style = "paraiso-dark" +pygments_dark_style = "native" html_css_files = [ "custom.css", From 9a0d0cbfac536520f5f862da4f70a611981f2af2 Mon Sep 17 00:00:00 2001 From: Yuvraj <10830562+evalsocket@users.noreply.github.com> Date: Thu, 1 Apr 2021 23:57:51 +0530 Subject: [PATCH 43/74] Release automation addedd for flyte definition (#835) * wip: added script for release automation Signed-off-by: yuvraj * wip: added manual github workflow Signed-off-by: Yuvraj * wip: added release workflow Signed-off-by: Yuvraj * wip: fix e2e testcase Signed-off-by: Yuvraj * wip: small fixes Signed-off-by: Yuvraj * wip: revert version in kustomize Signed-off-by: Yuvraj * wip: revert kustomize changes Signed-off-by: Yuvraj * wip: added generated kustomize output Signed-off-by: yuvraj * update latest version of component Signed-off-by: Yuvraj * wip: added generated kustomize output Signed-off-by: yuvraj * wip: added message input in workflow Signed-off-by: yuvraj * wip: remove release automation from kustomize command Signed-off-by: yuvraj * wip: generate kustomize output Signed-off-by: yuvraj * wip: revert webhook and build kustomize output Signed-off-by: yuvraj * updated release workflow Signed-off-by: yuvraj * fix typo Signed-off-by: yuvraj Signed-off-by: Haytham Abuelfutuh --- .github/workflows/release-kustomize.yml | 42 +++++++++++++++++++ Makefile | 6 ++- deployment/eks/flyte_generated.yaml | 22 +++++----- deployment/gcp/flyte_generated.yaml | 22 +++++----- deployment/sandbox/flyte_generated.yaml | 22 +++++----- deployment/test/flyte_generated.yaml | 18 ++++---- kustomize/overlays/eks/kustomization.yaml | 12 +++--- kustomize/overlays/gcp/kustomization.yaml | 12 +++--- kustomize/overlays/sandbox/kustomization.yaml | 10 ++--- kustomize/overlays/test/kustomization.yaml | 10 ++--- script/release.sh | 13 ++++++ 11 files changed, 124 insertions(+), 65 deletions(-) create mode 100644 .github/workflows/release-kustomize.yml create mode 100755 script/release.sh diff --git a/.github/workflows/release-kustomize.yml b/.github/workflows/release-kustomize.yml new file mode 100644 index 0000000000..ca0f014afa --- /dev/null +++ b/.github/workflows/release-kustomize.yml @@ -0,0 +1,42 @@ +name: Publish flyte defination file + +on: + workflow_dispatch: + +jobs: + update-flyte-releases: + name: Update Flyte components + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v2 + with: + fetch-depth: "0" + - name: Build kustomize + run: | + make release_automation + make kustomize + - name: Create Pull Request + id: cpr + uses: peter-evans/create-pull-request@v3 + with: + token: ${{ secrets.FLYTE_BOT_PAT }} + commit-message: Update Flyte Components + committer: Flyte-Bot + author: ${{ github.actor }} <${{ github.actor }}@users.noreply.github.com> + signoff: true + branch: flyte-bot-update-releases + delete-branch: true + title: 'Update Flyte components' + body: | + Updated flyte deployment + - Updated GCP Flyte deployment + - Updated EKS Flyte deployment + - Updated Sandbox Flyte deployment + - Updated TEST Flyte deployment + - Auto-generated by [flyte-bot] + labels: | + kustomize + team-reviewers: | + owners + maintainers + draft: false \ No newline at end of file diff --git a/Makefile b/Makefile index 3474016751..19cb24ee95 100644 --- a/Makefile +++ b/Makefile @@ -3,9 +3,13 @@ pip-compile $(1) --upgrade --verbose endef .PHONY: kustomize -kustomize: +kustomize: KUSTOMIZE_VERSION=3.9.2 bash script/generate_kustomize.sh +.PHONY: release_automation +release_automation: + bash script/release.sh + .PHONY: deploy_sandbox deploy_sandbox: bash script/deploy.sh diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index efe1d383c1..31c70bd81b 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -8587,7 +8587,7 @@ spec: - --config - /etc/datacatalog/config/*.yaml - serve - image: ghcr.io/flyteorg/datacatalog:v0.3.0 + image: ghcr.io/flyteorg/datacatalog:v0.3.1 imagePullPolicy: IfNotPresent name: datacatalog ports: @@ -8610,7 +8610,7 @@ spec: - /etc/datacatalog/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/datacatalog:v0.3.0 + image: ghcr.io/flyteorg/datacatalog:v0.3.1 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8667,7 +8667,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + image: ghcr.io/flyteorg/flytepropeller:v0.7.10 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -8715,7 +8715,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -8760,7 +8760,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8777,7 +8777,7 @@ spec: - flytesnacks - flytetester - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -8791,7 +8791,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -8838,7 +8838,7 @@ spec: - envFrom: - configMapRef: name: flyte-console-config - image: ghcr.io/flyteorg/flyteconsole:v0.19.6 + image: ghcr.io/flyteorg/flyteconsole:v0.19.7 name: flyteconsole ports: - containerPort: 8080 @@ -8882,7 +8882,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + image: ghcr.io/flyteorg/flytepropeller:v0.7.10 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -9069,7 +9069,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -9119,7 +9119,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + image: ghcr.io/flyteorg/flytepropeller:v0.7.10 imagePullPolicy: IfNotPresent name: webhook volumeMounts: diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index 79f76f71a4..0c3d03f416 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -8619,7 +8619,7 @@ spec: - --config - /etc/datacatalog/config/*.yaml - serve - image: ghcr.io/flyteorg/datacatalog:v0.3.0 + image: ghcr.io/flyteorg/datacatalog:v0.3.1 imagePullPolicy: IfNotPresent name: datacatalog ports: @@ -8642,7 +8642,7 @@ spec: - /etc/datacatalog/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/datacatalog:v0.3.0 + image: ghcr.io/flyteorg/datacatalog:v0.3.1 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8699,7 +8699,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + image: ghcr.io/flyteorg/flytepropeller:v0.7.10 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -8747,7 +8747,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -8792,7 +8792,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8809,7 +8809,7 @@ spec: - flytesnacks - flytetester - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -8823,7 +8823,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -8870,7 +8870,7 @@ spec: - envFrom: - configMapRef: name: flyte-console-config - image: ghcr.io/flyteorg/flyteconsole:v0.19.6 + image: ghcr.io/flyteorg/flyteconsole:v0.19.7 name: flyteconsole ports: - containerPort: 8080 @@ -8914,7 +8914,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + image: ghcr.io/flyteorg/flytepropeller:v0.7.10 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -9101,7 +9101,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -9151,7 +9151,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + image: ghcr.io/flyteorg/flytepropeller:v0.7.10 imagePullPolicy: IfNotPresent name: webhook volumeMounts: diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index 5f162c1391..58f495184c 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -2722,7 +2722,7 @@ spec: - --config - /etc/datacatalog/config/*.yaml - serve - image: ghcr.io/flyteorg/datacatalog:v0.3.0 + image: ghcr.io/flyteorg/datacatalog:v0.3.1 imagePullPolicy: IfNotPresent name: datacatalog ports: @@ -2740,7 +2740,7 @@ spec: - /etc/datacatalog/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/datacatalog:v0.3.0 + image: ghcr.io/flyteorg/datacatalog:v0.3.1 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -2797,7 +2797,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + image: ghcr.io/flyteorg/flytepropeller:v0.7.10 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -2851,7 +2851,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -2902,7 +2902,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -2918,7 +2918,7 @@ spec: - seed-projects - flytesnacks - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -2932,7 +2932,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -2979,7 +2979,7 @@ spec: - envFrom: - configMapRef: name: flyte-console-config - image: ghcr.io/flyteorg/flyteconsole:v0.19.6 + image: ghcr.io/flyteorg/flyteconsole:v0.19.7 name: flyteconsole ports: - containerPort: 8080 @@ -3023,7 +3023,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + image: ghcr.io/flyteorg/flytepropeller:v0.7.10 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -3328,7 +3328,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -3512,7 +3512,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + image: ghcr.io/flyteorg/flytepropeller:v0.7.10 imagePullPolicy: IfNotPresent name: webhook volumeMounts: diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index 2296b79bbd..2a10bff71e 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -582,7 +582,7 @@ spec: - --config - /etc/datacatalog/config/*.yaml - serve - image: ghcr.io/flyteorg/datacatalog:v0.3.0 + image: ghcr.io/flyteorg/datacatalog:v0.3.1 imagePullPolicy: IfNotPresent name: datacatalog ports: @@ -600,7 +600,7 @@ spec: - /etc/datacatalog/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/datacatalog:v0.3.0 + image: ghcr.io/flyteorg/datacatalog:v0.3.1 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -657,7 +657,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + image: ghcr.io/flyteorg/flytepropeller:v0.7.10 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -705,7 +705,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -756,7 +756,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -772,7 +772,7 @@ spec: - seed-projects - flytetester - flytesnacks - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -786,7 +786,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.0 + image: ghcr.io/flyteorg/flyteadmin:v0.4.2 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -843,7 +843,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + image: ghcr.io/flyteorg/flytepropeller:v0.7.10 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -948,7 +948,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.8 + image: ghcr.io/flyteorg/flytepropeller:v0.7.10 imagePullPolicy: IfNotPresent name: webhook volumeMounts: diff --git a/kustomize/overlays/eks/kustomization.yaml b/kustomize/overlays/eks/kustomization.yaml index 21b3f2ef07..a8322c1299 100644 --- a/kustomize/overlays/eks/kustomization.yaml +++ b/kustomize/overlays/eks/kustomization.yaml @@ -21,21 +21,21 @@ bases: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.4.0 # override the tag + newTag: v0.4.2 # FLYTEADMIN_TAG override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name - newTag: v0.19.6 # override the tag + newTag: v0.19.7 # FLYTECONSOLE_TAG the tag newName: ghcr.io/flyteorg/flyteconsole # override the namep # Flyte DataCatalog - name: datacatalog # match images with this name - newTag: v0.3.0 # override the tag + newTag: v0.3.1 # DATACATALOG_TAG override the tag newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.7.8 # override the tag + newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Webhook - name: webhook # match images with this name - newTag: v0.7.8 # override the tag - newName: ghcr.io/flyteorg/flytepropeller # override the name + newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag + newName: ghcr.io/flyteorg/flytepropeller # override the name \ No newline at end of file diff --git a/kustomize/overlays/gcp/kustomization.yaml b/kustomize/overlays/gcp/kustomization.yaml index a209c3bf86..802209f3b8 100644 --- a/kustomize/overlays/gcp/kustomization.yaml +++ b/kustomize/overlays/gcp/kustomization.yaml @@ -23,21 +23,21 @@ bases: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.4.0 # override the tag + newTag: v0.4.2 # FLYTEADMIN_TAG override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name - newTag: v0.19.6 # override the tag + newTag: v0.19.7 # FLYTECONSOLE_TAG the tag newName: ghcr.io/flyteorg/flyteconsole # override the namep # Flyte DataCatalog - name: datacatalog # match images with this name - newTag: v0.3.0 # override the tag + newTag: v0.3.1 # DATACATALOG_TAG override the tag newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.7.8 # override the tag + newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Webhook - name: webhook # match images with this name - newTag: v0.7.8 # override the tag - newName: ghcr.io/flyteorg/flytepropeller # override the name + newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag + newName: ghcr.io/flyteorg/flytepropeller # override the name \ No newline at end of file diff --git a/kustomize/overlays/sandbox/kustomization.yaml b/kustomize/overlays/sandbox/kustomization.yaml index 50d2f74c61..e0a74e5d53 100644 --- a/kustomize/overlays/sandbox/kustomization.yaml +++ b/kustomize/overlays/sandbox/kustomization.yaml @@ -25,23 +25,23 @@ resources: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.4.0 # override the tag + newTag: v0.4.2 # FLYTEADMIN_TAG override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name - newTag: v0.19.6 # override the tag + newTag: v0.19.7 # FLYTECONSOLE_TAG the tag newName: ghcr.io/flyteorg/flyteconsole # override the namep # Flyte DataCatalog - name: datacatalog # match images with this name - newTag: v0.3.0 # override the tag + newTag: v0.3.1 # DATACATALOG_TAG override the tag newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.7.8 # override the tag + newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Webhook - name: webhook # match images with this name - newTag: v0.7.8 # override the tag + newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Override postgres image to use alpine based (rather smaller) docker image - name: postgres diff --git a/kustomize/overlays/test/kustomization.yaml b/kustomize/overlays/test/kustomization.yaml index 6bb67c1098..943319bab1 100644 --- a/kustomize/overlays/test/kustomization.yaml +++ b/kustomize/overlays/test/kustomization.yaml @@ -19,23 +19,23 @@ bases: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.4.0 # override the tag + newTag: v0.4.2 # FLYTEADMIN_TAG override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name - newTag: v0.19.6 # override the tag + newTag: v0.19.7 # FLYTECONSOLE_TAG the tag newName: ghcr.io/flyteorg/flyteconsole # override the namep # Flyte DataCatalog - name: datacatalog # match images with this name - newTag: v0.3.0 # override the tag + newTag: v0.3.1 # DATACATALOG_TAG override the tag newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.7.8 # override the tag + newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Webhook - name: webhook # match images with this name - newTag: v0.7.8 # override the tag + newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Override postgres image to use alpine based (rather smaller) docker image - name: postgres diff --git a/script/release.sh b/script/release.sh new file mode 100755 index 0000000000..b605cd9b0d --- /dev/null +++ b/script/release.sh @@ -0,0 +1,13 @@ +#!/usr/bin/env bash + +set -ex + +FLYTEADMIN_TAG=$(curl --silent "https://api.github.com/repos/flyteorg/flyteadmin/releases/latest" | jq -r .tag_name) +DATACATALOG_TAG=$(curl --silent "https://api.github.com/repos/flyteorg/datacatalog/releases/latest" | jq -r .tag_name) +FLYTECONSOLE_TAG=$(curl --silent "https://api.github.com/repos/flyteorg/flyteconsole/releases/latest" | jq -r .tag_name) +FLYTEPROPELLER_TAG=$(curl --silent "https://api.github.com/repos/flyteorg/flytepropeller/releases/latest" | jq -r .tag_name) + +grep -rlZ "newTag:[^P]*# FLYTEADMIN_TAG" ./kustomize/overlays | xargs -0 sed -i "s/newTag:[^P]*# FLYTEADMIN_TAG/newTag: ${FLYTEADMIN_TAG} # FLYTEADMIN_TAG/g" +grep -rlZ "newTag:[^P]*# DATACATALOG_TAG" ./kustomize/overlays | xargs -0 sed -i "s/newTag:[^P]*# DATACATALOG_TAG/newTag: ${DATACATALOG_TAG} # DATACATALOG_TAG/g" +grep -rlZ "newTag:[^P]*# FLYTECONSOLE_TAG" ./kustomize/overlays | xargs -0 sed -i "s/newTag:[^P]*# FLYTECONSOLE_TAG/newTag: ${FLYTECONSOLE_TAG} # FLYTECONSOLE_TAG/g" +grep -rlZ "newTag:[^P]*# FLYTEPROPELLER_TAG" ./kustomize/overlays | xargs -0 sed -i "s/newTag:[^P]*# FLYTEPROPELLER_TAG/newTag: ${FLYTEPROPELLER_TAG} # FLYTEPROPELLER_TAG/g" From 4c72b7930be056d99afae4418004d11f152705bf Mon Sep 17 00:00:00 2001 From: Flyte Bot Date: Thu, 1 Apr 2021 14:41:11 -0700 Subject: [PATCH 44/74] Milestone 0.12.0 (#888) * Update Flyte Components Signed-off-by: Flyte-Bot Signed-off-by: Haytham Abuelfutuh * update endtoend to flytekit 0.17 Signed-off-by: wild-endeavor Signed-off-by: Haytham Abuelfutuh * empty changelog Signed-off-by: wild-endeavor Signed-off-by: Haytham Abuelfutuh * Update ChangeLog Signed-off-by: Haytham Abuelfutuh * before pull from flyte-bot-update-releases Signed-off-by: Haytham Abuelfutuh Co-authored-by: EngHabu Co-authored-by: wild-endeavor Co-authored-by: HackMD Signed-off-by: Haytham Abuelfutuh --- CHANGELOG/CHANGELOG-v0.12.0.md | 35 +++++++++++++++++++ deployment/eks/flyte_generated.yaml | 16 ++++----- deployment/gcp/flyte_generated.yaml | 16 ++++----- deployment/sandbox/flyte_generated.yaml | 16 ++++----- deployment/test/flyte_generated.yaml | 14 ++++---- end2end/tests/endtoend.yaml | 2 +- kustomize/overlays/eks/kustomization.yaml | 6 ++-- kustomize/overlays/gcp/kustomization.yaml | 6 ++-- kustomize/overlays/sandbox/kustomization.yaml | 6 ++-- kustomize/overlays/test/kustomization.yaml | 6 ++-- 10 files changed, 79 insertions(+), 44 deletions(-) create mode 100644 CHANGELOG/CHANGELOG-v0.12.0.md diff --git a/CHANGELOG/CHANGELOG-v0.12.0.md b/CHANGELOG/CHANGELOG-v0.12.0.md new file mode 100644 index 0000000000..742cb4a12f --- /dev/null +++ b/CHANGELOG/CHANGELOG-v0.12.0.md @@ -0,0 +1,35 @@ +# General +* Added CoPilot configuration to sandbox manifest +* Updated and streamlined documentation navigation and themes. Better content organization coming soon! + +# Performance +* Improved execution performance including: + - Reduced cache lookups + - Improved GetWorkflowExecution performance + - Capped max number of nodes in each propeller round + - Misc. propeller performance tweaks +* TaskTemplate offloading + +# Housekeeping +* Migrated Datacatalog protobuf definitions to flyteidl [thanks @tnsetting] +* Upgraded stow version used in flytestdlib +* Moved off lyft kubernetes forks and onto official kubernetes library dependencies +* Revamped pod tasks to use official kubernetes python client library for defining PodSpecs + +# Events +* Richer event metadata for task executions +* Better merging of custom info across task events + +# Bug fixes +* Resolved non-backwards protobuf role changes that prevented launching single task executions [thanks @kanterov] +* Better handling of large workflows and errors in flytepropeller + +# Flytekit (Python) +* Access to secrets +* Bug fixes around the 0.16 release. + * Use original FlyteFile/FlyteDirectory + * Fix serialization of pod specs in pod plugin [thanks @jeevb] + * Accept auth role arg in single task execution + * Fixed task resolver in map task + * Requests and limits added to ContainerTask [thanks @migueltol22] + diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index 31c70bd81b..de4d19286b 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -8667,7 +8667,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.10 + image: ghcr.io/flyteorg/flytepropeller:v0.10.0 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -8715,7 +8715,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -8760,7 +8760,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8777,7 +8777,7 @@ spec: - flytesnacks - flytetester - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -8791,7 +8791,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -8882,7 +8882,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.7.10 + image: ghcr.io/flyteorg/flytepropeller:v0.10.0 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -9069,7 +9069,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -9119,7 +9119,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.10 + image: ghcr.io/flyteorg/flytepropeller:v0.10.0 imagePullPolicy: IfNotPresent name: webhook volumeMounts: diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index 0c3d03f416..4e7ce1c0c5 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -8699,7 +8699,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.10 + image: ghcr.io/flyteorg/flytepropeller:v0.10.0 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -8747,7 +8747,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -8792,7 +8792,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8809,7 +8809,7 @@ spec: - flytesnacks - flytetester - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -8823,7 +8823,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -8914,7 +8914,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.7.10 + image: ghcr.io/flyteorg/flytepropeller:v0.10.0 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -9101,7 +9101,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -9151,7 +9151,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.10 + image: ghcr.io/flyteorg/flytepropeller:v0.10.0 imagePullPolicy: IfNotPresent name: webhook volumeMounts: diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index 58f495184c..8073b2a3f5 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -2797,7 +2797,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.10 + image: ghcr.io/flyteorg/flytepropeller:v0.10.0 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -2851,7 +2851,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -2902,7 +2902,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -2918,7 +2918,7 @@ spec: - seed-projects - flytesnacks - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -2932,7 +2932,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -3023,7 +3023,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.7.10 + image: ghcr.io/flyteorg/flytepropeller:v0.10.0 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -3328,7 +3328,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -3512,7 +3512,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.10 + image: ghcr.io/flyteorg/flytepropeller:v0.10.0 imagePullPolicy: IfNotPresent name: webhook volumeMounts: diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index 2a10bff71e..d0f417a8fa 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -657,7 +657,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.10 + image: ghcr.io/flyteorg/flytepropeller:v0.10.0 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -705,7 +705,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -756,7 +756,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -772,7 +772,7 @@ spec: - seed-projects - flytetester - flytesnacks - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -786,7 +786,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.2 + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -843,7 +843,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.7.10 + image: ghcr.io/flyteorg/flytepropeller:v0.10.0 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -948,7 +948,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.7.10 + image: ghcr.io/flyteorg/flytepropeller:v0.10.0 imagePullPolicy: IfNotPresent name: webhook volumeMounts: diff --git a/end2end/tests/endtoend.yaml b/end2end/tests/endtoend.yaml index cb9d16e8fe..7cd2cb6a50 100644 --- a/end2end/tests/endtoend.yaml +++ b/end2end/tests/endtoend.yaml @@ -11,7 +11,7 @@ spec: command: - bash - -c - image: ghcr.io/flyteorg/flytetools:78c917fb8d5dab74f5ff5f02d92e43d53ed4e7c3 + image: ghcr.io/flyteorg/flytetools:2638d4444add18096a688bc458d963cd3997e52e imagePullPolicy: IfNotPresent name: flytetester resources: diff --git a/kustomize/overlays/eks/kustomization.yaml b/kustomize/overlays/eks/kustomization.yaml index a8322c1299..4457715cff 100644 --- a/kustomize/overlays/eks/kustomization.yaml +++ b/kustomize/overlays/eks/kustomization.yaml @@ -21,7 +21,7 @@ bases: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.4.2 # FLYTEADMIN_TAG override the tag + newTag: v0.4.7 # FLYTEADMIN_TAG override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -33,9 +33,9 @@ images: newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Webhook - name: webhook # match images with this name - newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name \ No newline at end of file diff --git a/kustomize/overlays/gcp/kustomization.yaml b/kustomize/overlays/gcp/kustomization.yaml index 802209f3b8..7fcca6183f 100644 --- a/kustomize/overlays/gcp/kustomization.yaml +++ b/kustomize/overlays/gcp/kustomization.yaml @@ -23,7 +23,7 @@ bases: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.4.2 # FLYTEADMIN_TAG override the tag + newTag: v0.4.7 # FLYTEADMIN_TAG override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -35,9 +35,9 @@ images: newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Webhook - name: webhook # match images with this name - newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name \ No newline at end of file diff --git a/kustomize/overlays/sandbox/kustomization.yaml b/kustomize/overlays/sandbox/kustomization.yaml index e0a74e5d53..bb1e00f598 100644 --- a/kustomize/overlays/sandbox/kustomization.yaml +++ b/kustomize/overlays/sandbox/kustomization.yaml @@ -25,7 +25,7 @@ resources: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.4.2 # FLYTEADMIN_TAG override the tag + newTag: v0.4.7 # FLYTEADMIN_TAG override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -37,11 +37,11 @@ images: newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Webhook - name: webhook # match images with this name - newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Override postgres image to use alpine based (rather smaller) docker image - name: postgres diff --git a/kustomize/overlays/test/kustomization.yaml b/kustomize/overlays/test/kustomization.yaml index 943319bab1..1b4e568464 100644 --- a/kustomize/overlays/test/kustomization.yaml +++ b/kustomize/overlays/test/kustomization.yaml @@ -19,7 +19,7 @@ bases: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.4.2 # FLYTEADMIN_TAG override the tag + newTag: v0.4.7 # FLYTEADMIN_TAG override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -31,11 +31,11 @@ images: newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Webhook - name: webhook # match images with this name - newTag: v0.7.10 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Override postgres image to use alpine based (rather smaller) docker image - name: postgres From cbda32cdb2e499c07026c4f199e7096734920f2c Mon Sep 17 00:00:00 2001 From: SandraGH5 <80421934+SandraGH5@users.noreply.github.com> Date: Fri, 2 Apr 2021 14:21:03 -0700 Subject: [PATCH 45/74] Update index.rst (#884) * Update index.rst Add Flyte users to Who's Using Flyte and Meet Flyte. * Update rsts/index.rst Co-authored-by: Ketan Umare <16888709+kumare3@users.noreply.github.com> * Update index.rst Co-authored-by: Ketan Umare <16888709+kumare3@users.noreply.github.com> Signed-off-by: Haytham Abuelfutuh --- rsts/index.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/rsts/index.rst b/rsts/index.rst index 4bd7d9cafd..4fee64cdd9 100644 --- a/rsts/index.rst +++ b/rsts/index.rst @@ -43,7 +43,7 @@ Meet Flyte Flyte is an open-source, container-native, structured programming and distributed processing platform. It enables highly concurrent, scalable and maintainable workflows for machine learning and data processing. -Created at `Lyft `__, Flyte provides first class support for Python, Java, and Scala, and is built directly on Kubernetes for all the benefits containerization provides: portability, scalability, and reliability. +Created at `Lyft `__ in collaboration with Spotify, Freenome and many others, Flyte provides first class support for Python, Java, and Scala, and is built directly on Kubernetes for all the benefits containerization provides: portability, scalability, and reliability. The core unit of execution in Flyte is the ``task``, which you can easily write with the Flytekit Python SDK: @@ -94,6 +94,8 @@ Who's Using Flyte? At `Lyft `__, Flyte has served production model training and data processing for over four years, becoming the de-facto platform for the Pricing, Locations, ETA, Mapping teams, Growth, Autonomous and other teams. +For the most current list of Flyte's deployments, please click `here `_. + Next Steps ---------- From 9789656f5784babeaeb0205e465b7d0563fd58df Mon Sep 17 00:00:00 2001 From: Ketan Umare <16888709+kumare3@users.noreply.github.com> Date: Fri, 2 Apr 2021 21:20:12 -0700 Subject: [PATCH 46/74] Dashboard updates (#886) * Adding additional metrics in Flyteadmin dashboard Signed-off-by: Ketan Umare * Flytepropeller metrics added Signed-off-by: Ketan Umare * updated Signed-off-by: Ketan Umare * updated Signed-off-by: Ketan Umare * Added queue metrics Signed-off-by: Ketan Umare * updated name Signed-off-by: Ketan Umare Signed-off-by: Haytham Abuelfutuh --- .../prometheus/flyteadmin-dashboard.json | 10494 +++++++++++++++- .../prometheus/flytepropeller-dashboard.json | 2246 +++- stats/flyteadmin_dashboard.py | 96 +- stats/flytepropeller_dashboard.py | 333 +- 4 files changed, 13051 insertions(+), 118 deletions(-) diff --git a/deployment/stats/prometheus/flyteadmin-dashboard.json b/deployment/stats/prometheus/flyteadmin-dashboard.json index 507444c6aa..a120ec974c 100644 --- a/deployment/stats/prometheus/flyteadmin-dashboard.json +++ b/deployment/stats/prometheus/flyteadmin-dashboard.json @@ -77,6 +77,10302 @@ "renderer": "flot", "repeat": null, "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:executions:get_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 2, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:executions:list_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 3, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:executions:create_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 4, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:executions:update_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "update Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 5, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:executions:list_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 6, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:executions:list_identifiers_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_identifiers Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 7, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:executions:delete_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "delete Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 8, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:executions:exists_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "exists Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "DB executions ops stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 9, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:executions:get_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 10, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:executions:list_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 11, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:executions:create_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 12, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:executions:update_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "update Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 13, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:executions:list_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 14, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:executions:list_identifiers_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_identifiers Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 15, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:executions:delete_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "delete Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 16, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:executions:exists_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "exists Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "DB executions ops stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 17, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:task_executions:get_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 18, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:task_executions:list_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 19, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:task_executions:create_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 20, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:task_executions:update_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "update Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 21, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:task_executions:list_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 22, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:task_executions:list_identifiers_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_identifiers Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 23, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:task_executions:delete_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "delete Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 24, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:task_executions:exists_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "exists Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "DB task_executions ops stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 25, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:task_executions:get_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 26, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:task_executions:list_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 27, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:task_executions:create_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 28, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:task_executions:update_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "update Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 29, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:task_executions:list_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 30, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:task_executions:list_identifiers_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_identifiers Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 31, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:task_executions:delete_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "delete Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 32, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:task_executions:exists_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "exists Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "DB task_executions ops stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 33, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:node_executions:get_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 34, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:node_executions:list_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 35, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:node_executions:create_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 36, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:node_executions:update_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "update Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 37, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:node_executions:list_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 38, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:node_executions:list_identifiers_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_identifiers Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 39, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:node_executions:delete_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "delete Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 40, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:node_executions:exists_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "exists Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "DB node_executions ops stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 41, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:node_executions:get_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 42, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:node_executions:list_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 43, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:node_executions:create_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 44, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:node_executions:update_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "update Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 45, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:node_executions:list_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 46, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:node_executions:list_identifiers_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_identifiers Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 47, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:node_executions:delete_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "delete Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 48, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:node_executions:exists_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "exists Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "DB node_executions ops stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 49, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:workflows:get_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 50, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:workflows:list_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 51, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:workflows:create_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 52, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:workflows:update_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "update Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 53, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:workflows:list_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 54, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:workflows:list_identifiers_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_identifiers Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 55, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:workflows:delete_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "delete Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 56, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:workflows:exists_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "exists Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "DB workflows ops stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 57, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:workflows:get_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 58, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:workflows:list_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 59, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:workflows:create_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 60, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:workflows:update_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "update Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 61, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:workflows:list_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 62, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:workflows:list_identifiers_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_identifiers Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 63, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:workflows:delete_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "delete Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 64, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:workflows:exists_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "exists Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "DB workflows ops stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 65, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:launch_plans:get_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 66, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:launch_plans:list_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 67, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:launch_plans:create_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 68, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:launch_plans:update_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "update Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 69, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:launch_plans:list_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 70, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:launch_plans:list_identifiers_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_identifiers Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 71, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:launch_plans:delete_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "delete Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 72, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:launch_plans:exists_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "exists Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "DB launch_plans ops stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 73, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:launch_plans:get_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 74, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:launch_plans:list_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 75, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:launch_plans:create_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 76, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:launch_plans:update_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "update Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 77, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:launch_plans:list_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 78, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:launch_plans:list_identifiers_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_identifiers Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 79, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:launch_plans:delete_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "delete Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 80, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:launch_plans:exists_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "exists Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "DB launch_plans ops stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 81, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:project:get_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 82, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:project:list_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 83, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:project:create_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 84, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:project:update_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "update Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 85, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:project:list_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 86, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:project:list_identifiers_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_identifiers Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 87, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:project:delete_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "delete Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 88, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:admin:database:postgres:repositories:project:exists_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "exists Latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "DB project ops stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 89, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:project:get_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "get Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 90, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:project:list_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 91, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:project:create_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "create Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 92, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:project:update_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "update Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 93, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:project:list_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 94, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:project:list_identifiers_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "list_identifiers Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 95, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:project:delete_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "delete Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 96, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:admin:database:postgres:repositories:project:exists_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "exists Count Ops", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ops", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "DB project ops stats" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 97, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], "span": 4, "stack": false, "steppedLine": false, @@ -190,7 +10486,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 2, + "id": 98, "interval": null, "isNew": true, "legend": { @@ -310,7 +10606,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 3, + "id": 99, "interval": null, "isNew": true, "legend": { @@ -351,7 +10647,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:create_execution:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:create_execution:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -381,7 +10677,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -426,7 +10722,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 4, + "id": 100, "interval": null, "isNew": true, "legend": { @@ -574,7 +10870,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 5, + "id": 101, "interval": null, "isNew": true, "legend": { @@ -694,7 +10990,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 6, + "id": 102, "interval": null, "isNew": true, "legend": { @@ -735,7 +11031,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:create_launch_plan:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:create_launch_plan:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -765,7 +11061,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -810,7 +11106,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 7, + "id": 103, "interval": null, "isNew": true, "legend": { @@ -958,7 +11254,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 8, + "id": 104, "interval": null, "isNew": true, "legend": { @@ -1078,7 +11374,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 9, + "id": 105, "interval": null, "isNew": true, "legend": { @@ -1119,7 +11415,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:create_task:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:create_task:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -1149,7 +11445,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -1194,7 +11490,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 10, + "id": 106, "interval": null, "isNew": true, "legend": { @@ -1342,7 +11638,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 11, + "id": 107, "interval": null, "isNew": true, "legend": { @@ -1462,7 +11758,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 12, + "id": 108, "interval": null, "isNew": true, "legend": { @@ -1503,7 +11799,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:create_workflow:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:create_workflow:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -1533,7 +11829,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -1578,7 +11874,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 13, + "id": 109, "interval": null, "isNew": true, "legend": { @@ -1726,7 +12022,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 14, + "id": 110, "interval": null, "isNew": true, "legend": { @@ -1846,7 +12142,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 15, + "id": 111, "interval": null, "isNew": true, "legend": { @@ -1887,7 +12183,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:create_node_execution_event:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:create_node_execution_event:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -1917,7 +12213,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -1962,7 +12258,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 16, + "id": 112, "interval": null, "isNew": true, "legend": { @@ -2110,7 +12406,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 17, + "id": 113, "interval": null, "isNew": true, "legend": { @@ -2230,7 +12526,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 18, + "id": 114, "interval": null, "isNew": true, "legend": { @@ -2271,7 +12567,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:create_task_execution_event:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:create_task_execution_event:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -2301,7 +12597,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -2346,7 +12642,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 19, + "id": 115, "interval": null, "isNew": true, "legend": { @@ -2494,7 +12790,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 20, + "id": 116, "interval": null, "isNew": true, "legend": { @@ -2614,7 +12910,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 21, + "id": 117, "interval": null, "isNew": true, "legend": { @@ -2655,7 +12951,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:get_execution:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:get_execution:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -2685,7 +12981,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -2730,7 +13026,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 22, + "id": 118, "interval": null, "isNew": true, "legend": { @@ -2878,7 +13174,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 23, + "id": 119, "interval": null, "isNew": true, "legend": { @@ -2998,7 +13294,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 24, + "id": 120, "interval": null, "isNew": true, "legend": { @@ -3039,7 +13335,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:get_launch_plan:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:get_launch_plan:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -3069,7 +13365,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -3114,7 +13410,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 25, + "id": 121, "interval": null, "isNew": true, "legend": { @@ -3262,7 +13558,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 26, + "id": 122, "interval": null, "isNew": true, "legend": { @@ -3382,7 +13678,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 27, + "id": 123, "interval": null, "isNew": true, "legend": { @@ -3423,7 +13719,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:get_task:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:get_task:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -3453,7 +13749,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -3498,7 +13794,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 28, + "id": 124, "interval": null, "isNew": true, "legend": { @@ -3646,7 +13942,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 29, + "id": 125, "interval": null, "isNew": true, "legend": { @@ -3766,7 +14062,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 30, + "id": 126, "interval": null, "isNew": true, "legend": { @@ -3807,7 +14103,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:get_workflow:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:get_workflow:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -3837,7 +14133,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -3882,7 +14178,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 31, + "id": 127, "interval": null, "isNew": true, "legend": { @@ -4030,7 +14326,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 32, + "id": 128, "interval": null, "isNew": true, "legend": { @@ -4150,7 +14446,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 33, + "id": 129, "interval": null, "isNew": true, "legend": { @@ -4191,7 +14487,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:get_node_execution:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:get_node_execution:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -4221,7 +14517,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -4266,7 +14562,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 34, + "id": 130, "interval": null, "isNew": true, "legend": { @@ -4414,7 +14710,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 35, + "id": 131, "interval": null, "isNew": true, "legend": { @@ -4534,7 +14830,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 36, + "id": 132, "interval": null, "isNew": true, "legend": { @@ -4575,7 +14871,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:get_task_execution:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:get_task_execution:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -4605,7 +14901,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -4650,7 +14946,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 37, + "id": 133, "interval": null, "isNew": true, "legend": { @@ -4798,7 +15094,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 38, + "id": 134, "interval": null, "isNew": true, "legend": { @@ -4918,7 +15214,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 39, + "id": 135, "interval": null, "isNew": true, "legend": { @@ -4959,7 +15255,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:get_active_launch_plan:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:get_active_launch_plan:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -4989,7 +15285,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -5034,7 +15330,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 40, + "id": 136, "interval": null, "isNew": true, "legend": { @@ -5182,7 +15478,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 41, + "id": 137, "interval": null, "isNew": true, "legend": { @@ -5302,7 +15598,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 42, + "id": 138, "interval": null, "isNew": true, "legend": { @@ -5343,7 +15639,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:list_execution:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:list_execution:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -5373,7 +15669,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -5418,7 +15714,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 43, + "id": 139, "interval": null, "isNew": true, "legend": { @@ -5566,7 +15862,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 44, + "id": 140, "interval": null, "isNew": true, "legend": { @@ -5686,7 +15982,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 45, + "id": 141, "interval": null, "isNew": true, "legend": { @@ -5727,7 +16023,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:list_launch_plan:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:list_launch_plan:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -5757,7 +16053,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -5802,7 +16098,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 46, + "id": 142, "interval": null, "isNew": true, "legend": { @@ -5950,7 +16246,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 47, + "id": 143, "interval": null, "isNew": true, "legend": { @@ -6070,7 +16366,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 48, + "id": 144, "interval": null, "isNew": true, "legend": { @@ -6111,7 +16407,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:list_task:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:list_task:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -6141,7 +16437,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -6186,7 +16482,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 49, + "id": 145, "interval": null, "isNew": true, "legend": { @@ -6334,7 +16630,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 50, + "id": 146, "interval": null, "isNew": true, "legend": { @@ -6454,7 +16750,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 51, + "id": 147, "interval": null, "isNew": true, "legend": { @@ -6495,7 +16791,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:list_workflow:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:list_workflow:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -6525,7 +16821,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -6570,7 +16866,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 52, + "id": 148, "interval": null, "isNew": true, "legend": { @@ -6718,7 +17014,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 53, + "id": 149, "interval": null, "isNew": true, "legend": { @@ -6838,7 +17134,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 54, + "id": 150, "interval": null, "isNew": true, "legend": { @@ -6879,7 +17175,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:list_node_execution:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:list_node_execution:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -6909,7 +17205,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -6954,7 +17250,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 55, + "id": 151, "interval": null, "isNew": true, "legend": { @@ -7102,7 +17398,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 56, + "id": 152, "interval": null, "isNew": true, "legend": { @@ -7222,7 +17518,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 57, + "id": 153, "interval": null, "isNew": true, "legend": { @@ -7263,7 +17559,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:list_task_execution:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:list_task_execution:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -7293,7 +17589,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, @@ -7338,7 +17634,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 58, + "id": 154, "interval": null, "isNew": true, "legend": { @@ -7486,7 +17782,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 59, + "id": 155, "interval": null, "isNew": true, "legend": { @@ -7606,7 +17902,7 @@ }, "height": null, "hideTimeOverride": false, - "id": 60, + "id": 156, "interval": null, "isNew": true, "legend": { @@ -7647,7 +17943,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:admin:list_active_launch_plan:duration_ms[5m])) by (quantile)", + "expr": "sum(flyte:admin:list_active_launch_plan:duration_ms) by (quantile)", "format": "time_series", "hide": false, "instant": false, @@ -7677,7 +17973,7 @@ "yaxes": [ { "decimals": null, - "format": "s", + "format": "ms", "label": null, "logBase": 1, "max": null, diff --git a/deployment/stats/prometheus/flytepropeller-dashboard.json b/deployment/stats/prometheus/flytepropeller-dashboard.json index 3dbd64bc4d..457765c18d 100644 --- a/deployment/stats/prometheus/flytepropeller-dashboard.json +++ b/deployment/stats/prometheus/flytepropeller-dashboard.json @@ -189,7 +189,7 @@ "targets": [ { "datasource": "", - "expr": "sum(rate(flyte:propeller:all:round:abort_error[5m]))*300", + "expr": "sum(rate(flyte:propeller:all:round:abort_error[5m]))", "format": "time_series", "hide": false, "instant": false, @@ -295,7 +295,7 @@ "targets": [ { "datasource": "", - "expr": "sum(deriv(flyte:propeller:all:round:system_error_unlabeled[5m]))*300", + "expr": "sum(deriv(flyte:propeller:all:round:system_error_unlabeled[5m]))", "format": "time_series", "hide": false, "instant": false, @@ -985,7 +985,7 @@ "yaxes": [ { "decimals": null, - "format": "ops", + "format": "percent", "label": null, "logBase": 1, "max": null, @@ -1781,7 +1781,7 @@ "renderer": "flot", "repeat": null, "seriesOverrides": [], - "span": 4, + "span": 3, "stack": false, "steppedLine": false, "targets": [ @@ -1887,7 +1887,7 @@ "renderer": "flot", "repeat": null, "seriesOverrides": [], - "span": 4, + "span": 3, "stack": false, "steppedLine": false, "targets": [ @@ -1993,7 +1993,7 @@ "renderer": "flot", "repeat": null, "seriesOverrides": [], - "span": 4, + "span": 3, "stack": false, "steppedLine": false, "targets": [ @@ -2060,11 +2060,2245 @@ "show": true } ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 19, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 3, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:node:perma_system_error_duration_unlabeled_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "system error", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:node:perma_user_error_duration_unlabeled_ms[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "user error", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:node:perma_unknown_error_duration_unlabeled_ms[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "user error", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "node event recording count", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "none", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] } ], "repeat": null, "showTitle": true, "title": "Node Metrics" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 20, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:workflow:event_recording:success_duration_ms) by (quantile, wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "wf event recording latency success", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 21, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:workflow:event_recording:success_duration_ms_count[5m])) by (wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:workflow:event_recording:failure_duration_ms_count[5m])) by (wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failure", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "wf event recording count", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "none", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 22, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:node:event_recording:success_duration_ms) by (quantile, wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "node event recording latency success", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 23, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:node:event_recording:success_duration_ms_count[5m])) by (wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:node:event_recording:failure_duration_ms_count[5m])) by (wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failure", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "node event recording count", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "none", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 24, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:task:event_recording:success_duration_ms) by (quantile, wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "task event recording latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 25, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:task:event_recording:success_duration_ms_count[5m])) by (wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "success wf", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:task:event_recording:failure_duration_ms_count[5m])) by (wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "failure", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "task event recording count", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "none", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 26, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:node:build_dynamic_workflow_us) by (quantile, wf) / 1000", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Dynamic workflow build latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 27, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:node:build_dynamic_workflow_us_count[5m])) by (wf)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Dynamic workflow build count", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "none", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 28, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:admin_launcher:cache_hit[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "hit", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + }, + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:admin_launcher:cache_miss[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "miss", + "metric": "", + "refId": "B", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Admin Launcher cache", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "Perf metrics" + }, + { + "collapse": false, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 29, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 3, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(flyte:propeller:all:wf_update_latency_ms) by (quantile)", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "wf update etcD latency", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "ms", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 30, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 3, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:wf_update_latency_ms_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "etcD writes", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "none", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 31, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 3, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:wf_update_conflict[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "etcD write conflicts", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "none", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 32, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 3, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:wf_update_failed[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "etcD write fail", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "none", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "etcD write metrics" + }, + { + "collapse": true, + "editable": true, + "height": "250px", + "panels": [ + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 33, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:main_adds[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "WF Adds to main queue", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 34, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:main_depth[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Unprocessed Queue depth", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 35, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:main_retries[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Item retries", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 36, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "flyte:propeller:all:main_unfinished_work_s", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Seconds of unfinished work in progress", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 37, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:main_work_duration_us_sum[5m]) / rate(flyte:propeller:all:main_work_duration_us_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Workqueue work average duration", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + }, + { + "aliasColors": {}, + "bars": false, + "cacheTimeout": null, + "datasource": "${DS_PROM}", + "description": null, + "editable": true, + "error": false, + "fill": 1, + "grid": { + "threshold1": null, + "threshold1Color": "rgba(216, 200, 27, 0.27)", + "threshold2": null, + "threshold2Color": "rgba(234, 112, 112, 0.22)" + }, + "height": null, + "hideTimeOverride": false, + "id": 38, + "interval": null, + "isNew": true, + "legend": { + "alignAsTable": false, + "avg": false, + "current": false, + "hideEmpty": false, + "hideZero": false, + "max": false, + "min": false, + "rightSide": false, + "show": true, + "sideWidth": null, + "sort": null, + "sortDesc": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 2, + "links": [], + "maxDataPoints": 100, + "minSpan": null, + "nullPointMode": "connected", + "options": { + "alertThreshold": true, + "dataLinks": [] + }, + "percentage": false, + "pointradius": 5, + "points": false, + "renderer": "flot", + "repeat": null, + "seriesOverrides": [], + "span": 2, + "stack": false, + "steppedLine": false, + "targets": [ + { + "datasource": "", + "expr": "sum(rate(flyte:propeller:all:main_queue_latency_us_sum[5m]) / rate(flyte:propeller:all:main_queue_latency_us_count[5m]))", + "format": "time_series", + "hide": false, + "instant": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "", + "metric": "", + "refId": "A", + "step": 10, + "target": "" + } + ], + "timeFrom": null, + "timeShift": null, + "title": "Duration for which an item stays in queue - avg", + "tooltip": { + "msResolution": true, + "shared": true, + "sort": 0, + "value_type": "cumulative" + }, + "transparent": false, + "type": "graph", + "xaxis": { + "show": true + }, + "yaxes": [ + { + "decimals": null, + "format": "s", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + }, + { + "decimals": null, + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": 0, + "show": true + } + ] + } + ], + "repeat": null, + "showTitle": true, + "title": "FlytePropeller Queue metrics" } ], "schemaVersion": 12, diff --git a/stats/flyteadmin_dashboard.py b/stats/flyteadmin_dashboard.py index e87d1d0473..c8e8aa819d 100644 --- a/stats/flyteadmin_dashboard.py +++ b/stats/flyteadmin_dashboard.py @@ -2,7 +2,7 @@ from grafanalib.core import ( Alert, AlertCondition, Dashboard, Graph, GreaterThan, OP_AND, OPS_FORMAT, Row, RTYPE_SUM, SECONDS_FORMAT, - SHORT_FORMAT, single_y_axis, Target, TimeRange, YAxes, YAxis, DataSourceInput + SHORT_FORMAT, single_y_axis, Target, TimeRange, YAxes, YAxis, DataSourceInput, MILLISECONDS_FORMAT ) # ------------------------------ @@ -38,6 +38,26 @@ class FlyteAdmin(object): "list_active_launch_plan", ] + ENTITIES = [ + "executions", + "task_executions", + "node_executions", + "workflows", + "launch_plans", + "project", + ] + + DB_OPS = [ + "get", + "list", + "create", + "update", + "list", + "list_identifiers", + "delete", + "exists", + ] + @staticmethod def error_codes(api: str, interval: int = 1) -> Graph: return Graph( @@ -101,11 +121,11 @@ def api_latency(api: str, interval: int = 1) -> Graph: dataSource=DATASOURCE, targets=[ Target( - expr=f'sum(rate(flyte:admin:{api}:duration_ms[{interval}m])) by (quantile)', + expr=f'sum(flyte:admin:{api}:duration_ms) by (quantile)', refId='A', ), ], - yAxes=single_y_axis(format=SECONDS_FORMAT), + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), ) @staticmethod @@ -120,6 +140,67 @@ def create_api_row(api: str, collapse: bool, interval: int = 1) -> Row: ] ) + @staticmethod + def db_latency(entity: str, op: str, interval: int = 1) -> Graph: + return Graph( + title=f"{op} Latency", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(flyte:admin:database:postgres:repositories:{entity}:{op}_ms) by (quantile)', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ) + + @staticmethod + def create_entity_db_row_latency(entity: str, collapse: bool, interval: int = 1) -> Row: + r = Row( + title=f"DB {entity} ops stats", + collapse=collapse, + panels=[], + ) + for op in FlyteAdmin.DB_OPS: + r.panels.append(FlyteAdmin.db_latency(entity, op=op, interval=interval)) + return r + + @staticmethod + def db_count(entity: str, op: str, interval: int = 1) -> Graph: + return Graph( + title=f"{op} Count Ops", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:admin:database:postgres:repositories:{entity}:{op}_ms_count[{interval}m]))', + refId='A', + ), + ], + yAxes=YAxes( + YAxis(format=OPS_FORMAT), + YAxis(format=SHORT_FORMAT), + ), + ) + + @staticmethod + def create_entity_db_count(entity: str, collapse: bool, interval: int = 1) -> Row: + r = Row( + title=f"DB {entity} ops stats", + collapse=collapse, + panels=[], + ) + for op in FlyteAdmin.DB_OPS: + r.panels.append(FlyteAdmin.db_count(entity, op=op, interval=interval)) + return r + + @staticmethod + def create_all_entity_db_rows(collapse: bool, interval: int = 1) -> typing.List[Row]: + rows = [] + for entity in FlyteAdmin.ENTITIES: + rows.append(FlyteAdmin.create_entity_db_row_latency(entity=entity, collapse=collapse, interval=interval)) + rows.append(FlyteAdmin.create_entity_db_count(entity=entity, collapse=collapse, interval=interval)) + return rows + @staticmethod def create_all_apis(interval: int = 5) -> typing.List[Row]: rows = [] @@ -127,6 +208,13 @@ def create_all_apis(interval: int = 5) -> typing.List[Row]: rows.append(FlyteAdmin.create_api_row(api, collapse=True, interval=interval)) return rows + @staticmethod + def create_all_rows(interval: int = 5) -> typing.List[Row]: + rows = [] + rows.extend(FlyteAdmin.create_all_entity_db_rows(collapse=True, interval=interval)) + rows.extend(FlyteAdmin.create_all_apis(interval)) + return rows + dashboard = Dashboard( tags=["flyte", "prometheus", "flyteadmin", "flyte-controlplane"], @@ -141,7 +229,7 @@ def create_all_apis(interval: int = 5) -> typing.List[Row]: ], editable=False, title="Flyte Admin Dashboard (via Prometheus)", - rows=FlyteAdmin.create_all_apis(interval=5), + rows=FlyteAdmin.create_all_rows(), description="Flyte Admin/Control Plane Dashboard. This is great for monitoring FlyteAdmin and the Service API.", ).auto_panel_ids() diff --git a/stats/flytepropeller_dashboard.py b/stats/flytepropeller_dashboard.py index f3cd01618d..bc306fee94 100644 --- a/stats/flytepropeller_dashboard.py +++ b/stats/flytepropeller_dashboard.py @@ -1,8 +1,9 @@ import typing + from grafanalib.core import ( - Alert, AlertCondition, Dashboard, Graph, - GreaterThan, OP_AND, OPS_FORMAT, Row, RTYPE_SUM, SECONDS_FORMAT, - SHORT_FORMAT, single_y_axis, Target, TimeRange, YAxes, YAxis, MILLISECONDS_FORMAT, DataSourceInput + Dashboard, Graph, Gauge, Stat, + OPS_FORMAT, Row, SHORT_FORMAT, single_y_axis, Target, YAxes, YAxis, MILLISECONDS_FORMAT, DataSourceInput, + PERCENT_FORMAT, NO_FORMAT, SECONDS_FORMAT ) # ------------------------------ @@ -102,7 +103,7 @@ def system_errors() -> Graph: dataSource=DATASOURCE, targets=[ Target( - expr='sum(deriv(flyte:propeller:all:round:system_error_unlabeled[5m]))*300', + expr='sum(deriv(flyte:propeller:all:round:system_error_unlabeled[5m]))', refId='A', ), ], @@ -119,7 +120,7 @@ def abort_errors() -> Graph: dataSource=DATASOURCE, targets=[ Target( - expr='sum(rate(flyte:propeller:all:round:abort_error[5m]))*300', + expr='sum(rate(flyte:propeller:all:round:abort_error[5m]))', refId='A', ), ], @@ -269,10 +270,7 @@ def metastore_cache_hit_percentage(interval: int) -> Graph: refId='A', ), ], - yAxes=YAxes( - YAxis(format=OPS_FORMAT), - YAxis(format=SHORT_FORMAT), - ), + yAxes=single_y_axis(format=PERCENT_FORMAT), ) @staticmethod @@ -346,6 +344,219 @@ def metastore_latencies(collapse: bool) -> Row: ] ) + @staticmethod + def admin_launcher_cache() -> Graph: + return Graph( + title="Admin Launcher cache", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:admin_launcher:cache_hit[5m]))', + legendFormat="hit", + refId='A', + ), + + Target( + expr=f'sum(rate(flyte:propeller:all:admin_launcher:cache_miss[5m]))', + legendFormat="miss", + refId='B', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ) + + @staticmethod + def dynamic_wf_build() -> typing.List[Graph]: + return [ + Graph( + title="Dynamic workflow build latency", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(flyte:propeller:all:node:build_dynamic_workflow_us) by (quantile, wf) / 1000', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ), + Graph( + title="Dynamic workflow build count", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:node:build_dynamic_workflow_us_count[5m])) by (wf)', + refId='A', + ), + ], + yAxes=single_y_axis(format=NO_FORMAT), + ), + ] + + @staticmethod + def task_event_recording() -> typing.List[Graph]: + return [ + Graph( + title="task event recording latency", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(flyte:propeller:all:task:event_recording:success_duration_ms) by (quantile, wf)', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ), + Graph( + title="task event recording count", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:task:event_recording:success_duration_ms_count[5m])) by (wf)', + legendFormat="success wf", + refId='A', + ), + Target( + expr=f'sum(rate(flyte:propeller:all:task:event_recording:failure_duration_ms_count[5m])) by (wf)', + legendFormat="failure", + refId='A', + ), + ], + yAxes=single_y_axis(format=NO_FORMAT), + ), + ] + + @staticmethod + def node_event_recording() -> typing.List[Graph]: + return [ + Graph( + title="node event recording latency success", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(flyte:propeller:all:node:event_recording:success_duration_ms) by (quantile, wf)', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ), + Graph( + title="node event recording count", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:node:event_recording:success_duration_ms_count[5m])) by (wf)', + legendFormat="success", + refId='A', + ), + Target( + expr=f'sum(rate(flyte:propeller:all:node:event_recording:failure_duration_ms_count[5m])) by (wf)', + legendFormat="failure", + refId='A', + ), + ], + yAxes=single_y_axis(format=NO_FORMAT), + ), + ] + + @staticmethod + def wf_event_recording() -> typing.List[Graph]: + return [ + Graph( + title="wf event recording latency success", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(flyte:propeller:all:workflow:event_recording:success_duration_ms) by (quantile, wf)', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ), + Graph( + title="wf event recording count", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:workflow:event_recording:success_duration_ms_count[5m])) by (wf)', + legendFormat="success", + refId='A', + ), + Target( + expr=f'sum(rate(flyte:propeller:all:workflow:event_recording:failure_duration_ms_count[5m])) by (wf)', + legendFormat="failure", + refId='A', + ), + ], + yAxes=single_y_axis(format=NO_FORMAT), + ), + ] + + @staticmethod + def wf_store_latency(collapse: bool) -> Row: + return Row( + title="etcD write metrics", + collapse=collapse, + panels=[ + Graph( + title="wf update etcD latency", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(flyte:propeller:all:wf_update_latency_ms) by (quantile)', + refId='A', + ), + ], + yAxes=single_y_axis(format=MILLISECONDS_FORMAT), + ), + Graph( + title="etcD writes", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:wf_update_latency_ms_count[5m]))', + refId='A', + ), + ], + yAxes=single_y_axis(format=NO_FORMAT), + ), + Graph( + title="etcD write conflicts", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:wf_update_conflict[5m]))', + refId='A', + ), + ], + yAxes=single_y_axis(format=NO_FORMAT), + ), + Graph( + title="etcD write fail", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:wf_update_failed[5m]))', + refId='A', + ), + ], + yAxes=single_y_axis(format=NO_FORMAT), + ), + ]) + + @staticmethod + def perf_metrics(collapse: bool) -> Row: + r = Row( + title="Perf metrics", + collapse=collapse, + panels=[], + ) + r.panels.extend(FlytePropeller.wf_event_recording()) + r.panels.extend(FlytePropeller.node_event_recording()) + r.panels.extend(FlytePropeller.task_event_recording()) + r.panels.extend(FlytePropeller.dynamic_wf_build()) + r.panels.append(FlytePropeller.admin_launcher_cache()) + return r + @staticmethod def metastore_metrics(interval: int, collapse: bool) -> Row: return Row( @@ -357,6 +568,106 @@ def metastore_metrics(interval: int, collapse: bool) -> Row: ], ) + @staticmethod + def node_errors() -> Graph: + return Graph( + title="node event recording count", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:node:perma_system_error_duration_unlabeled_ms_count[5m]))', + legendFormat="system error", + refId='A', + ), + Target( + expr=f'sum(rate(flyte:propeller:all:node:perma_user_error_duration_unlabeled_ms[5m]))', + legendFormat="user error", + refId='A', + ), + Target( + expr=f'sum(rate(flyte:propeller:all:node:perma_unknown_error_duration_unlabeled_ms[5m]))', + legendFormat="user error", + refId='A', + ), + ], + yAxes=single_y_axis(format=NO_FORMAT), + ) + + @staticmethod + def queue_metrics(collapse: bool) -> Row: + return Row( + title="FlytePropeller Queue metrics", + collapse=collapse, + panels=[ + Graph( + title="WF Adds to main queue", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:main_adds[5m]))', + refId='A', + ), + ], + yAxes=single_y_axis(format=SHORT_FORMAT), + ), + Graph( + title="Unprocessed Queue depth", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:main_depth[5m]))', + refId='A', + ), + ], + yAxes=single_y_axis(format=SHORT_FORMAT), + ), + Graph( + title="Item retries", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:main_retries[5m]))', + refId='A', + ), + ], + yAxes=single_y_axis(format=SHORT_FORMAT), + ), + Graph( + title="Seconds of unfinished work in progress", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'flyte:propeller:all:main_unfinished_work_s', + refId='A', + ), + ], + yAxes=single_y_axis(format=SECONDS_FORMAT), + ), + Graph( + title="Workqueue work average duration", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:main_work_duration_us_sum[5m]) / rate(flyte:propeller:all:main_work_duration_us_count[5m]))', + refId='A', + ), + ], + yAxes=single_y_axis(format=SECONDS_FORMAT), + ), + Graph( + title="Duration for which an item stays in queue - avg", + dataSource=DATASOURCE, + targets=[ + Target( + expr=f'sum(rate(flyte:propeller:all:main_queue_latency_us_sum[5m]) / rate(flyte:propeller:all:main_queue_latency_us_count[5m]))', + refId='A', + ), + ], + yAxes=single_y_axis(format=SECONDS_FORMAT), + ), + ], + ) + @staticmethod def node_metrics(collapse: bool) -> Row: return Row( @@ -366,6 +677,7 @@ def node_metrics(collapse: bool) -> Row: FlytePropeller.node_exec_latency(), FlytePropeller.node_input_latency(), FlytePropeller.node_event_recording_latency(), + FlytePropeller.node_errors(), ], ) @@ -393,6 +705,9 @@ def create_all_rows(interval: int = 5) -> typing.List[Row]: FlytePropeller.metastore_metrics(interval, True), FlytePropeller.metastore_latencies(True), FlytePropeller.node_metrics(True), + FlytePropeller.perf_metrics(True), + FlytePropeller.wf_store_latency(False), + FlytePropeller.queue_metrics(True), ] From 4efaa7b406736a96b54398605dfd146e0d17b199 Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Sun, 11 Apr 2021 06:59:14 -0700 Subject: [PATCH 47/74] Simplify authentication setup docs Signed-off-by: Haytham Abuelfutuh --- deployment/eks/flyte_generated.yaml | 83 +++++++- deployment/gcp/flyte_generated.yaml | 83 +++++++- deployment/sandbox/flyte_generated.yaml | 93 ++++++++- deployment/test/flyte_generated.yaml | 81 +++++++- .../base/admindeployment/auth_secret.yaml | 6 + .../base/admindeployment/deployment.yaml | 182 ++++++++++-------- .../base/admindeployment/kustomization.yaml | 1 + .../headless/config/admin/server.yaml | 24 ++- kustomize/overlays/sandbox/kustomization.yaml | 2 +- rsts/howto/authentication/index.rst | 102 ++++------ 10 files changed, 494 insertions(+), 163 deletions(-) create mode 100644 kustomize/base/admindeployment/auth_secret.yaml diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index de4d19286b..6d34a7ba16 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -8157,13 +8157,35 @@ data: grpcPort: 8089 security: secure: false - useAuth: false + useAuth: true allowCors: true allowedOrigins: # Accepting all domains for Sandbox installation - "*" allowedHeaders: - "Content-Type" + secrets: + secrets-prefix: /etc/secrets/auth/ + auth: + disableForGrpc: true + httpPublicUri: http://localhost:30081/ + appAuth: + thirdPartyConfig: + flyteClient: + clientId: flytectl + redirectUri: https://localhost:53593/callback + scopes: + - offline + - all + userAuth: + openId: + baseUrl: https://accounts.google.com + scopes: + - profile + - openid + clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com + callbackUrl: "http://localhost:30081/callback" + redirectUrl: "/api/v1/projects" flyteadmin: roleNameKey: "iam.amazonaws.com/role" profilerPort: 10254 @@ -8207,7 +8229,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-t2f9g68k65 + name: flyte-admin-config-7g4m67fbc4 namespace: flyte --- apiVersion: v1 @@ -8392,6 +8414,13 @@ metadata: type: Opaque --- apiVersion: v1 +kind: Secret +metadata: + name: flyte-admin-auth + namespace: flyte +type: Opaque +--- +apiVersion: v1 kind: Service metadata: annotations: @@ -8733,6 +8762,8 @@ spec: name: config-volume - mountPath: /etc/db name: db-pass + - mountPath: /etc/secrets/auth + name: auth - command: - sh - -c @@ -8801,12 +8832,51 @@ spec: name: config-volume - mountPath: /etc/db name: db-pass + - args: + - --localPath + - /etc/secrets/auth-new + command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - secrets + - init + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + imagePullPolicy: IfNotPresent + name: generate-secrets + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + - mountPath: /etc/secrets/auth-new + name: new-auth-secrets + - args: + - --fromPath + - /etc/secrets/auth-new + command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - secrets + - create + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + imagePullPolicy: IfNotPresent + name: save-secrets + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + - mountPath: /etc/secrets/auth-new + name: new-auth-secrets serviceAccountName: flyteadmin volumes: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-t2f9g68k65 + name: flyte-admin-config-7g4m67fbc4 name: config-volume - configMap: name: clusterresource-template-tkdkkt4cb5 @@ -8814,6 +8884,11 @@ spec: - name: db-pass secret: secretName: db-pass-bthd2588cc + - name: auth + secret: + secretName: flyte-admin-auth + - emptyDir: {} + name: new-auth-secrets --- apiVersion: apps/v1 kind: Deployment @@ -9086,7 +9161,7 @@ spec: name: clusterresource-template-tkdkkt4cb5 name: resource-templates - configMap: - name: flyte-admin-config-t2f9g68k65 + name: flyte-admin-config-7g4m67fbc4 name: config-volume - name: db-pass secret: diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index 4e7ce1c0c5..e0cf7c0129 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -8148,13 +8148,35 @@ data: grpcPort: 8089 security: secure: false - useAuth: false + useAuth: true allowCors: true allowedOrigins: # Accepting all domains for Sandbox installation - "*" allowedHeaders: - "Content-Type" + secrets: + secrets-prefix: /etc/secrets/auth/ + auth: + disableForGrpc: true + httpPublicUri: http://localhost:30081/ + appAuth: + thirdPartyConfig: + flyteClient: + clientId: flytectl + redirectUri: https://localhost:53593/callback + scopes: + - offline + - all + userAuth: + openId: + baseUrl: https://accounts.google.com + scopes: + - profile + - openid + clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com + callbackUrl: "http://localhost:30081/callback" + redirectUrl: "/api/v1/projects" flyteadmin: roleNameKey: "iam.amazonaws.com/role" profilerPort: 10254 @@ -8198,7 +8220,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-7k2dmcb4tg + name: flyte-admin-config-27ggcgm724 namespace: flyte --- apiVersion: v1 @@ -8379,6 +8401,13 @@ metadata: type: Opaque --- apiVersion: v1 +kind: Secret +metadata: + name: flyte-admin-auth + namespace: flyte +type: Opaque +--- +apiVersion: v1 kind: Service metadata: name: cloudsqlproxy @@ -8765,6 +8794,8 @@ spec: name: config-volume - mountPath: /etc/db name: db-pass + - mountPath: /etc/secrets/auth + name: auth - command: - sh - -c @@ -8833,12 +8864,51 @@ spec: name: config-volume - mountPath: /etc/db name: db-pass + - args: + - --localPath + - /etc/secrets/auth-new + command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - secrets + - init + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + imagePullPolicy: IfNotPresent + name: generate-secrets + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + - mountPath: /etc/secrets/auth-new + name: new-auth-secrets + - args: + - --fromPath + - /etc/secrets/auth-new + command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - secrets + - create + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + imagePullPolicy: IfNotPresent + name: save-secrets + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + - mountPath: /etc/secrets/auth-new + name: new-auth-secrets serviceAccountName: flyteadmin volumes: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-7k2dmcb4tg + name: flyte-admin-config-27ggcgm724 name: config-volume - configMap: name: clusterresource-template-tkdkkt4cb5 @@ -8846,6 +8916,11 @@ spec: - name: db-pass secret: secretName: db-pass-bthd2588cc + - name: auth + secret: + secretName: flyte-admin-auth + - emptyDir: {} + name: new-auth-secrets --- apiVersion: apps/v1 kind: Deployment @@ -9118,7 +9193,7 @@ spec: name: clusterresource-template-tkdkkt4cb5 name: resource-templates - configMap: - name: flyte-admin-config-7k2dmcb4tg + name: flyte-admin-config-27ggcgm724 name: config-volume - name: db-pass secret: diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index 8073b2a3f5..327c9b05f3 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -2164,13 +2164,35 @@ data: grpcPort: 8089 security: secure: false - useAuth: false + useAuth: true allowCors: true allowedOrigins: # Accepting all domains for Sandbox installation - "*" allowedHeaders: - "Content-Type" + secrets: + secrets-prefix: /etc/secrets/auth/ + auth: + disableForGrpc: true + httpPublicUri: http://localhost:30081/ + appAuth: + thirdPartyConfig: + flyteClient: + clientId: flytectl + redirectUri: https://localhost:53593/callback + scopes: + - offline + - all + userAuth: + openId: + baseUrl: https://accounts.google.com + scopes: + - profile + - openid + clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com + callbackUrl: "http://localhost:30081/callback" + redirectUrl: "/api/v1/projects" flyteadmin: roleNameKey: "iam.amazonaws.com/role" profilerPort: 10254 @@ -2206,7 +2228,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-5b5g7785h8 + name: flyte-admin-config-tkg6c6ft28 namespace: flyte --- apiVersion: v1 @@ -2467,6 +2489,13 @@ metadata: type: Opaque --- apiVersion: v1 +kind: Secret +metadata: + name: flyte-admin-auth + namespace: flyte +type: Opaque +--- +apiVersion: v1 data: password: bXl1c2Vy user_secret: bXlzZWNyZXQ= @@ -2851,7 +2880,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -2869,6 +2898,8 @@ spec: name: config-volume - mountPath: /etc/db name: db-pass + - mountPath: /etc/secrets/auth + name: auth - command: - sh - -c @@ -2902,7 +2933,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -2918,7 +2949,7 @@ spec: - seed-projects - flytesnacks - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -2932,7 +2963,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -2942,6 +2973,45 @@ spec: name: config-volume - mountPath: /etc/db name: db-pass + - args: + - --localPath + - /etc/secrets/auth-new + command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - secrets + - init + image: ghcr.io/flyteorg/flyteadmin:ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 + imagePullPolicy: IfNotPresent + name: generate-secrets + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + - mountPath: /etc/secrets/auth-new + name: new-auth-secrets + - args: + - --fromPath + - /etc/secrets/auth-new + command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - secrets + - create + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: ghcr.io/flyteorg/flyteadmin:ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 + imagePullPolicy: IfNotPresent + name: save-secrets + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + - mountPath: /etc/secrets/auth-new + name: new-auth-secrets serviceAccountName: flyteadmin volumes: - configMap: @@ -2950,11 +3020,16 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-5b5g7785h8 + name: flyte-admin-config-tkg6c6ft28 name: config-volume - name: db-pass secret: secretName: db-pass-9dgchhk2bm + - name: auth + secret: + secretName: flyte-admin-auth + - emptyDir: {} + name: new-auth-secrets --- apiVersion: apps/v1 kind: Deployment @@ -3328,7 +3403,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -3345,7 +3420,7 @@ spec: name: clusterresource-template-dtg8ff28mt name: resource-templates - configMap: - name: flyte-admin-config-5b5g7785h8 + name: flyte-admin-config-tkg6c6ft28 name: config-volume - name: db-pass secret: diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index d0f417a8fa..d8fc2e281a 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -312,13 +312,35 @@ data: grpcPort: 8089 security: secure: false - useAuth: false + useAuth: true allowCors: true allowedOrigins: # Accepting all domains for Sandbox installation - "*" allowedHeaders: - "Content-Type" + secrets: + secrets-prefix: /etc/secrets/auth/ + auth: + disableForGrpc: true + httpPublicUri: http://localhost:30081/ + appAuth: + thirdPartyConfig: + flyteClient: + clientId: flytectl + redirectUri: https://localhost:53593/callback + scopes: + - offline + - all + userAuth: + openId: + baseUrl: https://accounts.google.com + scopes: + - profile + - openid + clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com + callbackUrl: "http://localhost:30081/callback" + redirectUrl: "/api/v1/projects" flyteadmin: roleNameKey: "iam.amazonaws.com/role" profilerPort: 10254 @@ -354,7 +376,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-k7f4k7g76c + name: flyte-admin-config-42thh57bff namespace: flyte --- apiVersion: v1 @@ -471,6 +493,13 @@ metadata: type: Opaque --- apiVersion: v1 +kind: Secret +metadata: + name: flyte-admin-auth + namespace: flyte +type: Opaque +--- +apiVersion: v1 kind: Service metadata: annotations: @@ -723,6 +752,8 @@ spec: name: config-volume - mountPath: /etc/db name: db-pass + - mountPath: /etc/secrets/auth + name: auth - command: - sh - -c @@ -796,6 +827,45 @@ spec: name: config-volume - mountPath: /etc/db name: db-pass + - args: + - --localPath + - /etc/secrets/auth-new + command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - secrets + - init + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + imagePullPolicy: IfNotPresent + name: generate-secrets + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + - mountPath: /etc/secrets/auth-new + name: new-auth-secrets + - args: + - --fromPath + - /etc/secrets/auth-new + command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - secrets + - create + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + imagePullPolicy: IfNotPresent + name: save-secrets + volumeMounts: + - mountPath: /etc/flyte/config + name: config-volume + - mountPath: /etc/secrets/auth-new + name: new-auth-secrets serviceAccountName: flyteadmin volumes: - configMap: @@ -804,11 +874,16 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-k7f4k7g76c + name: flyte-admin-config-42thh57bff name: config-volume - name: db-pass secret: secretName: db-pass-9dgchhk2bm + - name: auth + secret: + secretName: flyte-admin-auth + - emptyDir: {} + name: new-auth-secrets --- apiVersion: apps/v1 kind: Deployment diff --git a/kustomize/base/admindeployment/auth_secret.yaml b/kustomize/base/admindeployment/auth_secret.yaml new file mode 100644 index 0000000000..9a1281d776 --- /dev/null +++ b/kustomize/base/admindeployment/auth_secret.yaml @@ -0,0 +1,6 @@ +apiVersion: v1 +kind: Secret +metadata: + name: flyte-admin-auth + namespace: flyte +type: Opaque diff --git a/kustomize/base/admindeployment/deployment.yaml b/kustomize/base/admindeployment/deployment.yaml index b77ea096a1..846f6f1ac5 100644 --- a/kustomize/base/admindeployment/deployment.yaml +++ b/kustomize/base/admindeployment/deployment.yaml @@ -24,81 +24,113 @@ spec: spec: serviceAccountName: flyteadmin volumes: - - name: shared-data - emptyDir: {} - - name: config-volume - configMap: - name: flyte-admin-config - - name: resource-templates - configMap: - name: clusterresource-template - - name: db-pass - secret: - secretName: db-pass - initContainers: - - name: run-migrations - image: flyteadmin:v0.4.0 - imagePullPolicy: IfNotPresent - command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "migrate", "run"] - volumeMounts: - - name: config-volume - mountPath: /etc/flyte/config - - name: db-pass - mountPath: /etc/db - # Optional, These just seed the project - TODO move them to only - - name: seed-projects - image: flyteadmin:v0.4.0 - imagePullPolicy: IfNotPresent - command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", - "migrate", "seed-projects", "flytesnacks", "flytetester", "flyteexamples"] - volumeMounts: - - name: config-volume - mountPath: /etc/flyte/config - - name: db-pass - mountPath: /etc/db - - name: sync-cluster-resources - image: flyteadmin:v0.4.0 - imagePullPolicy: IfNotPresent - command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "clusterresource", "sync"] - volumeMounts: - - name: resource-templates - mountPath: /etc/flyte/clusterresource/templates - - name: config-volume - mountPath: /etc/flyte/config - - name: db-pass - mountPath: /etc/db - containers: - - name: flyteadmin - image: flyteadmin:v0.4.0 - imagePullPolicy: IfNotPresent - command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "serve"] - ports: - - containerPort: 8088 - - containerPort: 8089 - volumeMounts: - name: shared-data - mountPath: /srv/flyte + emptyDir: { } - name: config-volume - mountPath: /etc/flyte/config + configMap: + name: flyte-admin-config + - name: resource-templates + configMap: + name: clusterresource-template - name: db-pass - mountPath: /etc/db - - name: redoc - image: docker.io/redocly/redoc - imagePullPolicy: IfNotPresent - ports: - - containerPort: 8087 - resources: - limits: - memory: "200Mi" - cpu: "0.1" - command: - - sh - - -c - - ln -s /usr/share/nginx/html /usr/share/nginx/html/openapi && sh /usr/local/bin/docker-run.sh - env: - - name: PAGE_TITLE - value: "Flyte Admin OpenAPI" - - name: SPEC_URL - value: "/api/v1/openapi" - - name: PORT - value: "8087" + secret: + secretName: db-pass + - name: auth + secret: + secretName: flyte-admin-auth + - name: new-auth-secrets + emptyDir: { } + initContainers: + - name: run-migrations + image: flyteadmin:v0.4.0 + imagePullPolicy: IfNotPresent + command: [ "flyteadmin", "--config", "/etc/flyte/config/*.yaml", "migrate", "run" ] + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + - name: db-pass + mountPath: /etc/db + # Optional, These just seed the project - TODO move them to only + - name: seed-projects + image: flyteadmin:v0.4.0 + imagePullPolicy: IfNotPresent + command: [ "flyteadmin", "--config", "/etc/flyte/config/*.yaml", + "migrate", "seed-projects", "flytesnacks", "flytetester", "flyteexamples" ] + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + - name: db-pass + mountPath: /etc/db + - name: sync-cluster-resources + image: flyteadmin:v0.4.0 + imagePullPolicy: IfNotPresent + command: [ "flyteadmin", "--config", "/etc/flyte/config/*.yaml", "clusterresource", "sync" ] + volumeMounts: + - name: resource-templates + mountPath: /etc/flyte/clusterresource/templates + - name: config-volume + mountPath: /etc/flyte/config + - name: db-pass + mountPath: /etc/db + - name: generate-secrets + image: flyteadmin:v0.4.0 + imagePullPolicy: IfNotPresent + command: [ "flyteadmin", "--config", "/etc/flyte/config/*.yaml", "secrets", "init" ] + args: [ "--localPath", "/etc/secrets/auth-new" ] + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + - name: new-auth-secrets + mountPath: /etc/secrets/auth-new + - name: save-secrets + image: flyteadmin:v0.4.0 + imagePullPolicy: IfNotPresent + command: [ "flyteadmin", "--config", "/etc/flyte/config/*.yaml", "secrets", "create" ] + args: [ "--fromPath", "/etc/secrets/auth-new" ] + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + - name: new-auth-secrets + mountPath: /etc/secrets/auth-new + containers: + - name: flyteadmin + image: flyteadmin:v0.4.0 + imagePullPolicy: IfNotPresent + command: [ "flyteadmin", "--config", "/etc/flyte/config/*.yaml", "serve" ] + ports: + - containerPort: 8088 + - containerPort: 8089 + volumeMounts: + - name: shared-data + mountPath: /srv/flyte + - name: config-volume + mountPath: /etc/flyte/config + - name: db-pass + mountPath: /etc/db + - name: auth + mountPath: /etc/secrets/auth + - name: redoc + image: docker.io/redocly/redoc + imagePullPolicy: IfNotPresent + ports: + - containerPort: 8087 + resources: + limits: + memory: "200Mi" + cpu: "0.1" + command: + - sh + - -c + - ln -s /usr/share/nginx/html /usr/share/nginx/html/openapi && sh /usr/local/bin/docker-run.sh + env: + - name: PAGE_TITLE + value: "Flyte Admin OpenAPI" + - name: SPEC_URL + value: "/api/v1/openapi" + - name: PORT + value: "8087" diff --git a/kustomize/base/admindeployment/kustomization.yaml b/kustomize/base/admindeployment/kustomization.yaml index a944d005ca..5c3fedd15b 100644 --- a/kustomize/base/admindeployment/kustomization.yaml +++ b/kustomize/base/admindeployment/kustomization.yaml @@ -1,3 +1,4 @@ resources: +- auth_secret.yaml - deployment.yaml - service.yaml diff --git a/kustomize/base/single_cluster/headless/config/admin/server.yaml b/kustomize/base/single_cluster/headless/config/admin/server.yaml index 253bf8c776..3607631073 100644 --- a/kustomize/base/single_cluster/headless/config/admin/server.yaml +++ b/kustomize/base/single_cluster/headless/config/admin/server.yaml @@ -3,13 +3,35 @@ server: grpcPort: 8089 security: secure: false - useAuth: false + useAuth: true allowCors: true allowedOrigins: # Accepting all domains for Sandbox installation - "*" allowedHeaders: - "Content-Type" +secrets: + secrets-prefix: /etc/secrets/auth/ +auth: + disableForGrpc: true + httpPublicUri: http://localhost:30081/ + appAuth: + thirdPartyConfig: + flyteClient: + clientId: flytectl + redirectUri: https://localhost:53593/callback + scopes: + - offline + - all + userAuth: + openId: + baseUrl: https://accounts.google.com + scopes: + - profile + - openid + clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com + callbackUrl: "http://localhost:30081/callback" + redirectUrl: "/api/v1/projects" flyteadmin: roleNameKey: "iam.amazonaws.com/role" profilerPort: 10254 diff --git a/kustomize/overlays/sandbox/kustomization.yaml b/kustomize/overlays/sandbox/kustomization.yaml index bb1e00f598..bdc0e1e899 100644 --- a/kustomize/overlays/sandbox/kustomization.yaml +++ b/kustomize/overlays/sandbox/kustomization.yaml @@ -25,7 +25,7 @@ resources: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.4.7 # FLYTEADMIN_TAG override the tag + newTag: ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 # FLYTEADMIN_TAG override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name diff --git a/rsts/howto/authentication/index.rst b/rsts/howto/authentication/index.rst index 9ff67eee9e..fd04277add 100644 --- a/rsts/howto/authentication/index.rst +++ b/rsts/howto/authentication/index.rst @@ -62,92 +62,62 @@ Please refer to the `inline documentation `__ on how to configure the IdP for OpenIDConnect. + +.. note:: -Flyte Admin secures cookies using AES encryption. In order to achieve that, follow the steps below: + Make sure to create an OAuth2 Client Credential. The `client_id` and `client_secret` will be needed in the following + steps. -1. Generate secure cookie keys. Run this command to generate new keys: +2. Store the `client_secret` in a k8s secrt as follows: .. prompt:: bash - go test -v github.com/lyft/flyteadmin/pkg/auth -run TestSecureCookieLifecycle + kubectl edit secret -n flyte flyte-admin-auth -2. Create two secrets in the ``flyte`` namespace with the generated keys: +Add a new key under `data`: -.. prompt:: bash +.. code-block:: yaml - kubectl create secret generic flyteadmin-cookie-blockkey -n flyte --from-literal=blockkey= + stringData: + oidc_client_secret: from the previous step -.. prompt:: bash +Save and close your editor. - kubectl create secret generic flyteadmin-cookie-hashkey -n flyte --from-literal=hashkey= - kubectl create secret generic flytepropeller-oauth -n flyte --from-literal=secret=5aATwGcDZmFd3n0mLDbuR1uA - -3. Configure FlyteAdmin deployment to mount them to FlyteAdmin pod. - e.g. Add the following to your existing configuration - - .. code-block:: yaml - - containers: - name: flyteadmin - volumeMounts: - - mountPath: /etc/secrets/oauth - name: oauth - readOnly: true - - mountPath: /etc/secrets/hashkey - name: hashkey - readOnly: true - - mountPath: /etc/secrets/blockkey - name: blockkey - readOnly: true - volumes: - - name: oauth - secret: - defaultMode: 420 - secretName: flyteadmin-oauth-client - - name: hashkey - secret: - defaultMode: 420 - secretName: flyteadmin-cookie-hashkey - - name: blockkey - secret: - defaultMode: 420 - secretName: flyteadmin-cookie-blockkey +3. Edit FlyteAdmin config to add `client_id` as follows: -Google IdP -########## +.. prompt:: bash -1. Follow `Google Docs `__ on how to configure the IdP for OpenIDConnect. + kubectl deploy -n flyte flyteadmin -o yaml | grep "name: flyte-admin-config" -2. Create a secret in the ``flyte`` namespace with the value of the client password. +This will output the name of the config map where the `client_id` need to go. .. prompt:: bash - kubectl create secret generic -n flyte flyteadmin-oauth-client --from-literal=secret= + kubectl edit configmap -n flyte + +Find `client_id` and replace with the copied `client_id` + +.. code-block:: yaml + + clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com -3. Configure Flyte Admin with the following configuration: +Find `useAuth` and enable Auth enforcement: .. code-block:: yaml - server: - security: - useAuth: true - oauth: - scopes: - - profile - - openid - claims: - iss: https://accounts.google.com - aud: - clientId: - clientSecretFile: "/etc/secrets/oauth/secret" - cookieHashKeyFile: "/etc/secrets/hashkey/hashkey" - cookieBlockKeyFile: "/etc/secrets/blockkey/blockkey" - authorizeUrl: "https://accounts.google.com/o/oauth2/v2/auth?access_type=offline" - tokenUrl: "https://oauth2.googleapis.com/token" - callbackUrl: "http://localhost:8088/callback" - redirectUrl: "/api/v1/projects" + useAuth: true + +Save and exit your editor. + +4. Restart `flyteadmin` for the changes to take effect: + +.. prompt:: bash + + kubectl rollout restart deployment/flyteadmin -n flyte ****** CI From b7776dd0dcfc2dc0945561a6170dc49806bf8de2 Mon Sep 17 00:00:00 2001 From: Jeev B Date: Fri, 9 Apr 2021 11:42:07 -0700 Subject: [PATCH 48/74] Add init container that will pre-create buckets in minio (#905) * Add init container that will pre-create buckets in minio Signed-off-by: Jeev B * move minio tag override to overlay Signed-off-by: Jeev B Signed-off-by: Haytham Abuelfutuh --- deployment/sandbox/flyte_generated.yaml | 11 ++++++++++- deployment/test/flyte_generated.yaml | 11 ++++++++++- kustomize/base/addons/storage/storage.yaml | 11 ++++++++++- kustomize/overlays/sandbox/kustomization.yaml | 2 ++ kustomize/overlays/test/kustomization.yaml | 2 ++ 5 files changed, 34 insertions(+), 3 deletions(-) diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index 327c9b05f3..1d4bb10335 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -3135,7 +3135,7 @@ spec: value: minio - name: MINIO_SECRET_KEY value: miniostorage - image: minio/minio:RELEASE.2020-12-16T05-05-17Z + image: minio/minio:RELEASE.2021-04-06T23-11-00Z name: minio ports: - containerPort: 9000 @@ -3143,6 +3143,15 @@ spec: volumeMounts: - mountPath: /data name: minio-storage + initContainers: + - command: + - mkdir + - /data/my-s3-bucket + image: minio/minio:RELEASE.2021-04-06T23-11-00Z + name: create-bucket + volumeMounts: + - mountPath: /data + name: minio-storage volumes: - emptyDir: {} name: minio-storage diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index d8fc2e281a..f673d1cc76 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -955,7 +955,7 @@ spec: value: minio - name: MINIO_SECRET_KEY value: miniostorage - image: minio/minio:RELEASE.2020-12-16T05-05-17Z + image: minio/minio:RELEASE.2021-04-06T23-11-00Z name: minio ports: - containerPort: 9000 @@ -963,6 +963,15 @@ spec: volumeMounts: - mountPath: /data name: minio-storage + initContainers: + - command: + - mkdir + - /data/my-s3-bucket + image: minio/minio:RELEASE.2021-04-06T23-11-00Z + name: create-bucket + volumeMounts: + - mountPath: /data + name: minio-storage volumes: - emptyDir: {} name: minio-storage diff --git a/kustomize/base/addons/storage/storage.yaml b/kustomize/base/addons/storage/storage.yaml index b14b37be81..132a338cb8 100644 --- a/kustomize/base/addons/storage/storage.yaml +++ b/kustomize/base/addons/storage/storage.yaml @@ -16,8 +16,17 @@ spec: volumes: - name: minio-storage emptyDir: {} + initContainers: + - image: minio/minio + name: create-bucket + command: + - mkdir + - /data/my-s3-bucket + volumeMounts: + - name: minio-storage + mountPath: /data containers: - - image: "minio/minio:RELEASE.2020-12-16T05-05-17Z" + - image: minio/minio name: minio env: - name: MINIO_ACCESS_KEY diff --git a/kustomize/overlays/sandbox/kustomization.yaml b/kustomize/overlays/sandbox/kustomization.yaml index bdc0e1e899..10e2c1d300 100644 --- a/kustomize/overlays/sandbox/kustomization.yaml +++ b/kustomize/overlays/sandbox/kustomization.yaml @@ -46,3 +46,5 @@ images: # Override postgres image to use alpine based (rather smaller) docker image - name: postgres newTag: 10.16-alpine + - name: minio/minio + newTag: RELEASE.2021-04-06T23-11-00Z diff --git a/kustomize/overlays/test/kustomization.yaml b/kustomize/overlays/test/kustomization.yaml index 1b4e568464..ddf689f58c 100644 --- a/kustomize/overlays/test/kustomization.yaml +++ b/kustomize/overlays/test/kustomization.yaml @@ -40,3 +40,5 @@ images: # Override postgres image to use alpine based (rather smaller) docker image - name: postgres newTag: 10.16-alpine + - name: minio/minio + newTag: RELEASE.2021-04-06T23-11-00Z From f7f843414603ed0896616717279e513f91d06e15 Mon Sep 17 00:00:00 2001 From: Samhita Alla Date: Sun, 11 Apr 2021 21:00:30 +0530 Subject: [PATCH 49/74] Add LF AI & Data Logo to README (#902) Signed-off-by: Haytham Abuelfutuh --- README.md | 2 +- rsts/images/flyte-and-lf.png | Bin 0 -> 166426 bytes 2 files changed, 1 insertion(+), 1 deletion(-) create mode 100644 rsts/images/flyte-and-lf.png diff --git a/README.md b/README.md index 0bdf0ead81..90dcaac6fe 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@

- Flyte Logo + Flyte and LF AI & Data Logo

diff --git a/rsts/images/flyte-and-lf.png b/rsts/images/flyte-and-lf.png new file mode 100644 index 0000000000000000000000000000000000000000..47c8958805dcbe8a263900d79b02aef53b0d68f7 GIT binary patch literal 166426 zcmY&g2|SeB`|k=xA*4-G*-A-eD?%YcNM%dWLQ~et&M;Rg$(k%7Bq4jq$Tmh4lAY`f zW#7g=82q0zxxf7TeB8>docFxvd7kffcwN#|VPW3GykWxz7S#(%mp5!+nB1`8pN`Fp z@EQA7(#{PV_HR&CI(OCa&1j^99ENaLV$orrvUwh%@&zz_tc5pEg#R) z9k_Lzw1JWPiOjc~3cT$%Ejp6(ygT~3ejlt59uJ8az7U=^ZuO)XFFqYL;UAW9H^rvE zz*0Pu+R<#6=54akTvaSR&`@Pwk$%IyO?QqK-_Eejxt1!+!O5APsee?DQ+xg%&j0J} zp{^`U%|yqeN)h?ISxk-*u8hZ_!*<$NYE^i)A*K7&wEwK!8&h12ZLkd`rmyLKT%~zD zuaR~ksrJQiucATI5-X-@RQBT!ZTtO|<1!yp!g=)foLUO~B>&aBr1~8`OVoD9Px|WH zp6QP6u_=x(U9Be-1_-WU)xEt>=NOiDO2JU6<)6~NUA$QE;&~d!GG=Lh0@qZ+W@v?-au=F#{9UZFLN0gpKVy*H z){t{xqO`2KN<(!{gJtARtMAup4VLk64Hk^K^;5B&1W<9xL1+Gxt0nG85aeN8)-C3uWHUz+MuGIQ8)9`kVQrPU5D2C_})O%su8si zaRHt5Gm{S=#wSeh&CAaH=5vr;@$}?%obvPJ6p@;n=r*P}9ekJkSe`mqy=#OqgYAD0 z1iv`Tb?$!jwxG&R$JMZul$2!8nSE2qFI|;aCZ30`wBlN}l*qSb4W9izB)d~ra>tu9 zjrtEV({jnS>WV9>_7Bbrv`6byvzq8Ay;(NkvcQE@IUG-JbGYZ^JMlo*cNKk*Pr8tf z<{a@1Z4XwAd1b2ii5VMKAZqmc(u`g*a4xAVal!akmc&y2Hd>+4&nCU6`IbUS$g`e@kPkvM&^*{%f(Yygbj=yoo)~A zxry&fFgXA7ux@VeOk!qA!?F3c5Lxc{(YE7G`injlB?EHu&3RK%Js~~K7YHUy+olZUy+AhO@ zJhx%@;wvggsmpfFH|EFg87E(QA%sugv;LVkoH!sZ$35DT^Qzb@hks>I9yc4EGx4Ks z;j>ut>9j|&mWm6E?G@n~n*A-m@6Kr)yW9QycAt52ON@@`D0%piZeHHy!v>bq!D(re z!Rcv}q3NmG-V}Z_FsbrJ)Xx7SLC$Ju6O;M7DDu92jm zDLWmtuZzP(1HIIs)QpV!CP{c?;Rm`5nhDz}?+hu@^ll!dY|Pl0(jq02vGa?|?bUwb z37LFVi56Nm!sip+uLPK2o>tiFz8*DsTA`^M@cZWZ!^i9&^mwJS{gBVgH`5tubkkyS znG{{^9Em!|#qTF6%3Z?*z04=mTAB4f#Dv}?x6WN032D3)!nO%Vnexb=o}Lz6_VxDg zPG^7IxM|%}{QH`-z_iLTX5jsAtaU}HfELED=AvD{(;RQN!@ESMGVT73;oF;Z_d zq0_Rd>AdUKxM73zX7%4n<;^hVtbKXT%t^Kjwwud@Sadgjj_pypajWxb-eYH70^xCR z$}!Wv5#gZBY%a7&x%Ndf-2O;T!SU?P)Fx>L!o6}yHB{Z9g0?4pnXrXSk@x6~`6s6_ zo{f=6Urj=K@D>K!@okk`$Y343mnyxB(i9cZC; zo3SOkdAx`NrnbxiyLrNhY!|5bE63a8EFy-c2C5z`P2W8g{3cPtVx)18?y2MuuIjWq zPqRk!wJ&#ejonv~&z+aTCgr-qWT)Lcv2^kmdm{wM*(SD)o2j)G5j_?%4UtE2@rD%FJ!|z@>K9)oJvU^RgyY*Gl&agMN_Z>FMClMVo-z2)flIl$TeQ}? z=y29Y_fW1`^^9zTJTZG5aV+P2|Yy zy7!}RTp!&ZSDH8G)%?`4Yc6yApRo07`#j1dc;VQK>T`vgWIx#%%~5u=Hwy%}#{Ix2 zzR|Hvx+!6p95kG&qt?`~rkj0w|9CbtHnDWTj?BW0y}t=+VT6Mc$Vw28t5=HP?mz2HUx!tUc6pA;jJhn6(#4`8{`rZ8Jr?tnq(tvI2MtNZ94D}=S3_! zG?8_B%BYfsnmu)({T>T-v%Bca`T7HfuaqsI5Z+6XPfv9r zrsm?r%R$*$Q%YCo;~6 zG4Y7d8B3Z}st@ksqSSk6zAly0AaPrey!i=f9)3!sY4|g=rpQEPx`hcW%ent9*0d9& zV^O8g-5xd*C+bp=?3_DG^bm4eDxD225Xk%WY(1FXKYsjMo`9gB5!Yg~mzP)f_nPmy z-7n1}j0>}S6-rn|le;cMpHtr`HDQw!6i{F-G}$C*`)M<_B0xkm=8ir|okP0bkixNc z_!Ydmy%-1~9;{lVEdnb_?xk!|Rh+Wl`)h8bj>sk7;x3ahSZU<5C^=R3bd+g)VfVyD zt~?qawO#-)Qi)w6dza$>{S~h;VQ}eC54e;C7$V6 z$g&g3B8sG)S>{6J{&re_ZompZWW&~eyt;}tZsDiT@GC2)(fN>Qk1pPGVg5f@*S)~E zik}^`KCj9zu8s^$_;EX>O?ltQe#)wwk>PO~ue$)tQ>*%^rkf>kXk3(?V#X=`j@>Y- zAk?pAibFb+-Oy;GvATlF$=j9YBnJ}(70KI23f)ETlcwqAw7R%0Nh9z(jll=MUFRtH ztP{51a$DkS3cr7=DI{b^|NNNB$!+jZZ6qxxI_%UGFK%!(yHl`POHj4J~yFVe3J>~)v#`YP#XlM@MFpXE+uYO-=`OZ9F{>rbyLQ{EN6zJ~Wb zBe*?dvJC6?XG3^~{?Sga!P|`dIMX?nxRE(6d7Hrt(=R*iQk>OyCwE{!N3NYBA9z^` z7+-9;WEvgr;Q%LvC#?3R8a5W4_amx<7Z}`Lmv)m4T6Pc2rgC3t$Q!>R)Gpaa-!8Fi zF`cPx+Mwt6qv*h>NXRy<|N4{juk+(OZ)6o#o$Cf{a1S3@|4QhhCS^rE-gCG8`WLfD z-?_?kx%IlZcT&d0r1R(rgF6WO&@12-KScJvQ%l^bip_#X`OLzX8QT>g0;k8S$KeB! zm;2O6Szhbn!+qV08OKa#@5!`_r$t%r>waKg_Cr)|xuE8qu}5K_^UVz)>iqA=Tjt~@ zXYH;cDRN==C*8=U(kqu-zegDsn?0&D(9k?>)8D$?MUz3V%bcC^r|n4Ck&GgmM^fi^yG~oFA1mP^NnJoL{Yknq$urV7+9}vB`g2R$iDUS2PAS9H`(ufj zwyQ!LMJAS?&3COI!vAjGb%$yD#esshZqKO^IVPDqg#@(17yilm#X+JYFECM&Q;o~%T^!gWg7n?q z|J$hz8~!~K`QV|Jl1YX>A9_=<(OJJ)5y2ig;xPIqUKeA@X@C@-JcZU%SmzdyZ93srKVPlPxHX z4Vk^a%<%R;o~gV09%=E0$b7GFn2P+EbI46av@MF~sJ;+ByNUDXob& z7W)U*Y9AphV#an!8OnScF_!w?ly^5MqEG8gNv7O~L}T$OB55Qof4aS3UU=Qcy!Gak zyC!2&9$=jkUfB3(`0Ot?t#cAK19}<5B5|Yd3fOpF(Gxzoi|)=0W~OHMZ=r7XD)M<% zcS4w4!jFntabq-W2T=Nr#@+}Ngf|iOWK5C`N#4?wBzi(bD$m$0a}}4C@Ve}kU|yM) zyc2d7ov)qomc3!iuNJL8Zq0QnmwIe{!>4tq0>Rb^gYuQLyz2#@49=Q;lBADZo zi*McDEE&j>BcR`uFKu?`EnznlVd5EL zp;ib>Q?!2<-YXAjChoK&!>Bvn zug+L3$?T{kPLR8Pt=o`)E9t-K(=p`F!zkA~Jsrs=gAWek5ly=zk!$*ckXq0bjF|>< zsw2#g>uojLGoZXj15>l+2%t zdWMCmN8}!5h(t%;PL_KRlDgZ5+JjWr$JPtu#q(#@t>kSMXV(w6sTdVp-}mI}wI3Ub zucHH!5R^M*;ih85VDmdnh#bd4wRl2@yD`H^us|G1uk=uW-ZA$hE7w@UoVgCY)ZFNzIB~V5MAvousbu*S9bMN!olLn0 z8J$s%-2o1u)g+O^?tXbZ73&CcBOD0?wPFg(kAMwobvMLdR9|nu1Ddf&b59 z|8QtA6FRnJC#J}zy-r2Ld~Ajbw@4|J+Dcv5>&!kqGX$Ea$pi z+VJoow(;bVBO}ATedn+Eapi02S;-NvI|a+lOGb3(ranFoIvI^50yY+)>wbU^51eET zE=s|HcE`c;EXn#497 z)Hl(*6has=NtQRe?G&9IwDr@)g6R2_bqm>0x(f=5{lq{sTwCmM#?FK7^&{S-ZW`%S2NR4xIl`gLK{q~C{uS9TrR~ksMzmiB<>yOd4 zBVPnMT5`(H*e~9^^3bmOR{yaNSI<0^{2H8oxhn^HhY@KSkU4w%b4G$6yW#zf#!CS6 z3amxR$uq$3r`)t9+H(S4)z#Gz{e$K+c4bHux$_3hv+A&5hn|ZF#qjOP=wpIcw^|Pm zGi4-xJ6X3&-HN2HVsV;<`ftJfgTCdd`9g~Y7fVc^i{1CBd9&cJ6Q}g&gfK6Rmx?m{ zC!-Fm-^Ev(U*uRRDS=57&gr>F5PRd{Xk}8W=j-H;@4f~b7k^&hfsaRAY4T6w}#%c4xN$5mgwDHx6hN~?z)=IhTBXK~SEc24M)1EBKMG3t?O~E&0 z-g_U}=h`;a7hh+55rn5^dfvmG|BqZ8X99;vcBXk#&bZ4*vxefpymw}_GUU4Y$(|W? zpvep9D_`i1Xe8=Mn3x9=Ls?q)a8b^9S%(v&t=-F>7Kf6w}X;06+yj|m@72}&bB}|R_BK-9^ zdPqAas_%(y6gsbTygNJ*hnIB{!s}4yliMiqel2R!6qsSR5MYJDaM{cH@po+yvH|XO%Lx@KZamL+ z@Et*GcG=F!e)2_pCu@KCkvrozm*l8QBP8!<7IvLqLhZ>-OxXL%Wp9b0cWrGqW9@jb z8N{8PUnY(N>d0t1NRH!(ZY~A|$#e~WNp{g0A2&I&Cj%y3NzD}Qr?9$mUQ5!vf2NbQ zZmJ{IA$?VvQaY5ND!fh~MNX183@>ZUyupV9Fo@iZPYM(vGlWp1N&2Bjl>O$@d zu{FBh$VfbOyFk)RL8u0HgeDXSAi;0s!r%i2!f!?zSZ8$G1be2S;!O-i-5Ak$K!72z zJ~a3rZbLxUZyQCuJJD2KCqyg5!_D#gZjV1^{%XiS-t8dlo-TlN z9`+n^mDw62-+|h+ggQxGngXaj{RyrWriC(ECcHqJIRmUh+{7FljDwS*J}IV^%0G{& zcO~iUC>XlQEx(ww5Q|G0t8$TCw?hxkbL5Rl*PbNHObh;qBI_(@>qgo=~D(p#{#PA@1EwIOX+=5(qr2Bt9PbQ0ioJ>zn?< zWBorwrkkzO%^a4DsscpP-3znb9RJG#h|)M#J>!b<*p$(m+PW^{K!wXoVsmiqOL6o9 z?@G}2hYsO+&=SN=KuZ7weX$FQwW*63`rSo)ll%*0mH>!28pC-2fnd8XqVX4Yig!2j z8&hkkLtB&`sYh0w-t^(VJX(}LGyaW?;ab-g^yrK0%IOcj^Wys^`yg#fUqh$*pZ;=D z%dtzSInp>6P2nNA*|es)wKb{G z9pp3b?j%!hl}0%BdxF056{Jo-Yc-hrutnm@z6i=$a$PWS^OKyv3jzkDcIEgt4nH2- zqH*C*k@j3cC-lxaB_E28Jsk7O{1U0`RLm7C7$LBL1BNgyx`_i&bbtj!kguoj(_*GN zc#oa(h1G2<15ma*NyyriyDV?gu1yy$G`Z9NgQ>T2W2S(0bIvhr1H4on7w0C=G9$JX zGP@Jc--XoLG5IQ0XJ0F7z3hZd%Kz{D6~qg%))Y zc1K;Vj$Ad5&h6fc3t4xRYE3tH8u3^;O(yL8?w|J2`pB)WvEZG3h2e&jcqXbyPVsDI5Q5v zZhIaWa9CV$;5(SqkvDQ!uee89OPj%lD>(ZD7e$$y0$>I;UT!o)&bX(WmjC>K5Ct4b zQBD}9HR|Y)1hTqW(8~=lt|;Y}0-T0^Ex-dp6*9~OY!C7iSl6^puqvOHjI=zKGBe!4 znvMx8DfCTzA{@;uH=Os>dExI?ZMZH;e}z-_`Sa>vi5c0vveseYLwBMcpNu}{9=&rT zv_*7m0Whj4cpf3^Y;6q}BtKQe_tHv;J5z$@95v9dY<2yA1Ec^d4BnlpG|aKs)4zVWO-8nhFm#iyl!_bJP>hc!4-Bpoj;5rh=wX z`}nbM$Z`V1lrG5TV z7dIyb>jc^<40#x(raw5OC1VPjPu}JZtL?ER3m||xroh67H|vA$#`+0;wJE-2cxPLddJ*<=#b^CRr=N2 zFKb^y!`%!B9^tSi8tpl0lB`8)RvIHk97aWFNoRaxWA_K5lufBFFq7*fZy>iwdTnP_ zM;KG7cN)xw&Dc&UatXa#|3_*31N|p~r&I8D#kI~M61~6ccaCF6+Iwmg)-MVjrxhx3 zW6`P1n!=Py)KcaEs0D`sEGN6`kUZ#_`(dVh$d#!8-tL4>xR{IKSo*O`nZ zDXHa!ojo;5>sPiBRu+}F8e`};KD{@o)2O23X`h>x_UTtlkVTM6bQi4}EImisV%-Dn z08URJBLOBwFy#&H=V4L=yhfJRZ;EI+v66`Ot8PTdik-r%Y)vw+9K2wK{-!!Q`WT~w zlVYAfGZqwwTRUR1SAN8>95tsTD$aM$HH!4qD6d;quPSt_*iA{RWV`cOosq_EKZ#OT zNCV%_lGK$g>@c=h{aVb}0g)MI^I#JZ;0CatdB8Astz%w0Dg6p6^V=oT$OI{QJ0)Bi zhTw_|;H*F&zlZMK_WdlW^EKSRO-H&;TN|IG4)T79e!1?;mp&O+QnJgpl#Os`_tzcR zdo#QJ$a2q=kuTBzyh1|TO$8O<2`5GT87I+gH1Xqi;Z;Q6;DoUhxR^hfI8EV?lAMz34~ns&5e|8Y-W0a#6naNgL>?=G6>o(`mti>AK* zhh=HKk%0%>um|-_5U6jK{~TRY(y(5Lq=A(TD#4n>^ap`LlIjwvhWmP1KT8^B3!~IR zJ>%z=$1y_wHM={gWn_%t`h}f?h5eFAUTG;Pv-=X|@?1&!s75);(pQo4_PKNY3u~0>%2ki`N@XUH=3`VR-i*n;dB^D4w+# zf9o7_`a?yi9I+~-;Nev1o8qzqQ-4RxSN6YX%IPKzDPn~0eYrCF%(pj%~;h<(al6D5Y?cGiNF@Ya(|k`Gk6mH z0}x0MK~@Z?2-z%H2@p$m*Z;2BMY`qvY@fUoCSX2}j5)M!cQW3}u)nlf>e2HXwG&RG zUe?VoynF)biN-3RxRBvIO>UzI!=UA)y)_;zFc5JQ?$jqBp3$J<7f7f;e&UZ>BW+SS zLMnqczz=qa^`|E_oXEwFeF7x(@qu=A1~(Zm`HxXmye%KzvjJH0jJjHHobD<>H9+Ck zwIBZ;28{On12_9inF(U0%6zNgbtg|(XL^FR(;ox%HmD8TD4C2X5#bEom z<{<@$<qxNCaMJ8agypT@o<76n9ZT7_a8zX~!L?6~FCgpn2}{AIE;| z*01SjvABBDD1BCBFE&%a^53OCQf#&%xpn;#Z{@MGpPpiyz27x(YT}3NZ?QME6=JVz zUv9<@1*Fz70t=~OrzAbI$fb*JCV3LZ5J{9smC}fY z-22mkc(5f%e?-s@lt~{yZ7}dBWx)LC{L)-kJivt20SL}Sb%$<13R`lH4Q%VK+_6>K z{-{lP#-@_!^#QjF838cFD$gvfTs~l*rK153h(XG}WFib4PrAAe`7FL9IG*|7| zddlhKm~VVkPtDDBFS4z8*NC#xOzkpT=cBaV4~5bL`;(6(-D`SP+XLN>CaBlJzl7Kt zte_H1{CXPh9VA8kT+>aU`X*4i2EaLR$%E)!LyrP|nWp+8ffah*nylgWxm0VSn|P8@ zJV&sw_qIXF@?5@^%U}o@b7I}1b~35Ps&Gy$EmYjDIiTluI9ViRW|E$8G#1f9mewic zw9>>Iq|zesh6T0G#^TWKFM8lvfYe3w2lsvW4Eg&+h4|?h{;M3fjJ>D>7ORNImC{o196Wk1K2|?=Nz@Yw)_f%=d2` z?%yN9B#|1xTSBa-5}dt-!(!jQ|7sHl!&gD&6*Bc=*R)%CGg`!%-}d5~M1hZGK!gf> zAdhOsCv@Oi%x(rk1XW;s5(RL88y2vllLx5{E=s!O$3{2bqkTHo>eq^tC0A#&t~~J{ z9{>FEZ$RonAICO|Cs?bnimbPvvh6)J>VQ72b7Fte79@eNK?SlcYDP$q4?Mq{Cy@q& zLD+cjZmp)Q5(oK%jqaiqgRg;9a3DK3hb*H;s$A1vJSE5rbn*}Cmn4ohF>O?UJbex3i;rwK?5@E}%B6vLw7a7NJKo_6nMAcVnN0yz9< z+-&;O+r4EdZJri=yLoI-y4YWJld&hU`Oy|&H)wd%8t?Kj*3lz6z28VNDy(~MC`wCBeyHppTia<=PZkNgk zNktvCEht0n{~W;cUFr;S2}nr*BvZ97qbZQa#423Cxl|yjyqwQaL%sL{X#{d4&`H`rwC`^F2bTu+Bxb!oY>LojU~!v6Pyg8mqq@qxNYAL@S>^ ze>OAj;Gh3CrQ(KT1q)Ud`4M0P(njt7G+s#)bHMI2-yL#gppTD|%HVZPCV(vmSTwZu z#8I*U7~cqs0$k@;#GhR2v+NuQfe4^L;}=Mi)4=wgOkapI&~Olt5EeS)v4B)7_wMvU zqL!1ugTN(8e0)JidB|2X5x3qPzH~NR$X^-#-(&P}`Q~{Z%kGoExsCP2MI0=S+)8_w z86xSPE*EB3)&TX4)h%d^52^r5+>TU56=s0UR8|g(vWf*9{#glfEDAcL zXTl*^gR>x8{Qn}lK_`E`x2S`7*(lLEZ27W?;mPMByeCb{rJjE*Stbw0U1XLhMlnfgfLkzV;)efM$&ByNo$y0No>5;aR0&oLN5Drkkx{m_sfLGZI@zmjG7Iy$=bbzvg zjVhH=N7IAWm7y*G$33Bw&(-2nA8sfQToONW*wydN_!6;k)AQ#?J-F8ap-_>S4u!Vz zA*na%XLQGq&3W67>9 z1yOSro5-qcOIhOer>3bOxtSmZ5-JqV*q_l7gkUX9aWkYDg-ZMpKv4G`*+DRXsaiVW zLuU*^GSmO8cFX^jiuszN3;jo$;<~483pG?BA<@p+dZ*!upuTHIL(C5yw`~{c>UQ|M zw13qUVm;yTFLg|23GsA}n50>nx@9ZkGkrAAFc8Ms4eJ6dKhb6bP0LpIv;Z+w7Re&O zfc}If!s~|iyQjzfVT`=qNJ~b%B^W&sG_u`D!ps%H0&ItWt~kI_2E9b?)*qpJs6*** zZY%N{oH;yGfbof0KRLq!&MJMfSS%Q**4t~IJ-};~%92gi`R)RBb05tCc{R{X$jxTE zGW*Kmlc0yxR1OXX&V$#m!wC-S8xKtnB1oB1Cy@UaFKBxTNILRRK$wQ}4d{u3au?Yu zQX(;8m*q^jCH|W=s)L(7(X65M$Cu1h!X6ksQ*swcHEt&!1R9@t_k*HxVj1;VaTuqrdD5Uaup@m2OqG~d zQ@!k_&em*|qZ2ZVUB3+7cvfWpzUvLP3Xi7*4?w&_CRInsUUy)h33vx9esD(tiODO{ zpfIQ!MNpMATUmU!ECO?oAK(|9E0St*Dby`%?NV)8&d$(q9 zga~mD_#90j#2(K zzNi1#Q9S`9erwZk4J3WCY26(%S$inqs9GeC(IG4b!ZWf?{v^lQ5g-pWu{EK?c}FpV zm1j;na!5~e7cF?otK}~GeQqHU+dj4Wmn3Ty*?SC+bV z_jGUhyJKgCzQAQsp@VH%X9g;+c#hOaPq8i)Kt_zV7%1I8IXatb;Jsi)c1NCv82JYt zEQ(cag6v6ieKEAK|L&?t#}b=?MEuwLX?;k8mS?<2(`Lp3ELE66or<3EziZk*ZAVy0 zeDUqfucdOW_@0vwR zilo*p`V@x+ufs?~LXDJVgtwmbl=|zOj|ENlHX>T3hRsIwN=lVf385aENevGG-WACE z;LL!*VU1)6S~~`72^wTnDR6j@-AdL%Y8pI9*^U+X)6~($#ZDN()WDIAt zUD{}CR*jU`2ku46-(@{~cH*b+^YX0N;jwsG`+ng=-rFcA*KD8ypK4P+vfePLXRL%a zLljm7vVIzcL7Gp~`Y|hF`VYC#r3w&TKqZ4g9E(=;U;WhW|DV-wn971B5UowboTV~` zYNT=mi)>$L;4+5B$P?^;*R!A1!%9Eo_c)~_<8}>bjEASKat=>IeKHat(gcx@|F9AK z&XR_DP)rRFFVrAx9Okv?niTbiib4yfoq3*22^bkPBtx4rpg~yX(=ZlI7+F0KLdkOG z)sB~KCZ1fx+F4Cyap5+v8m;4~(gBy06C5VGnsxj39-7^ub8=#i`MeFZEtCrab-+5% zzXdWXkhjq3@>WKd_Oi=eDxQl#a!nb!r;+q+b6tqCp(wl*9Gv@ze%f^sc|q?6z#qci zUhbhe!zwocIl$6C8ICwQXh8o(*?8)nBgLB=vE27yD{hoOUqbs#q5CQuff!wvpE!1= zyY#LsPAP1X$}YJO)>f}#J?<)UAo(u@t)28>+@i&s6jA#zS^42W=wc(}&o@E%5}K!V zAZSBqs-n?Lv-)Vph=GC%cs6a^$Vi2e)6+Yiy!L> zPrm7YA-ZqGJ?g)C1A=D<`b9n_5}tx&i(=`hU88>*MU-G59}Q(gb|&l!?K5zKX;+cJ z(1BnTNGDA+W#y#F5opQuNP6Y`BfS>6(%uWLZG-m$LjNEMm;eIatnftug+WZ!gXj;V zfRhpTOhvB8VJsPOj zfNW{BMINF`r%brxkStD1)&nmS4M62yhjJlk!90-n{GTwSf;v51Is{z&hYJ7#KlGot zp8W)*Qdk?X6`H!`)_%HPdMNA~o?<6i3HKoXEuA<^CqsWKa>#a2QfY6|ir!B~Wva1W z=)hXBmAP#CBuyZP2K|TO0Z&GGSk$S( zFcbUFFo!Y&;ByUREmUske+t^ld0T>+MpEBH=~(OS3v@-UVChbrSHugV-VW*!ChZw5 zT@{7k6Xe&~X}8F{=~q5sB3gVf{)fbzC6?~W;p)2w(kf7!pu4@5-)%r9fO6v{r6}!K z&Dwc+F3{TA+I`B{?pwW2F6Ky_h${?9+R6l@1LHx7=6HrGxV>7Ofdc_^T{ zw`RgvU)7;7Gu& zxUCcNjf^ylCg{^MD*FpQ2JYMS`Gp(BJKyWW*le2wQ$fu{_I1U>O?>y-R+z+{CI-Y! zSLw$Wu+8-0S7h8*R=OhDajSi5gHn&2{V42Jc5WA6&0Tz}y3~g2;GB2U%u*K&u$vZp zxWefmK;}L^DbQ6kmQz_zb@WZ_Ggar|?* z;>b?<(>1_P=0}-CC>q#C7h1Oabsb30_mx#Zx(clG^87E~32|-~Y)`;g-vJQVfQ=%| z2!s&*Mc!9X5Qdb(FZ^!leBXmGdxC;?5-7c6Dp|Z+SG(vg3eGq!*K}3bu3maN&O?fI zVLWu|;hA~Dc*4>%;hE@~bT+r(+d{`ue-hcsSH@OO&yXURJE!xCe#uq4yX85&RhVZn z+LPs9DG|d)-BBqpjopzL`e0Im-TB6K1JyHGgTe|D z>vp>|{AT{={8^TfqK|pg48aj>$E{yhfllE26GcCvg=ov5LHYiPwLZGbd5dN@Y9Y1< z)LPIUkrvo)F%+2xFf#mo(1U4>-=9RG1^9cwu4O2DX#Nf)HNc(Z`E#=t$_1_h-{QcT z4W4l*m0z*e3~7qyni;ykyz}6&pPbv)&0f*%E>-WAjPwS_-9&8lj2VRd2BHXZc>{J& z$Fa^DG)HE%Lu2E?Sl94Jrh$qp?CC)H>*xvb9|$nS)!GAOi*s0 zylJCw(6F=;x>dg2arHW(C~;dLMQS$hsV>&eBXG>8UREvNU#99;Y_6}5;OHAps!aZ{ zP05)yNo$2qs?-SU6~<2Ne70@LnEYGr35r~&W7)ju$}Ef2bwwXL?*Pi!ssVAcuDMj+ z;Ykd%O+$3Nqsqxu+}n#sm9fI&F+*!t-Z;PJCNp+ae7Fwu1V#)w*%#Ps$sH zzSO8Mb2;Q(nnfE}fZ7iC8q(G^gE)pN(q&=lXj~k8zzvr`K~_45{=<$;e-)LD(MKjg z>jHWUwgOb4hu;k2O4$@E`d%u3uGp}(DkBN&k5dNHEK(al`9lmI@^cW{yI@VS0Wt`I z8E(o>h7~cuQDQ1Vhik;Ms~!72CYH+_3fWg=8DC!*v{@X%U>aOE?&_R2JJQ+d-%~nW zFr47B>3Cj4rX)q@MPcb`YvIt0XN=GI(8$O~|LHC_I37C6^E1XA)CSw8)w*uM$%(O5 zuFmBo4oc7Y0%J^QaqbA0vTXOw8!Lixcm5fgog0vkxr#X*=fb5nO?jtA70*#PtjQ~K z%FWLxVpg2}MptxnKxt8Gv&g4ji|t+$tQV-0?FS}JJ&g2vdY4}kpD!kN-d@E$$hg)Z}GxWYw>X+-wbZ!M8fKB zTNzpLQR%$px#TCjt#RpW1>MyAi1I>0ihHNLT-K!FWk;Ddp762z# zdm-k+NK60IZbiZO&yWEV^wCBLFn}e>k5bhU!L_?ZdH)7IN?_jBod8eViWDJ;F8B$o z$rf;139ML9h}hk&KOzNX^2~vXrP{i+@w+!(YFo*tO}D2o-009lmVz>i+s~ zX|qe(UN2rpfB)-Kbf){PV@wjF>rc@w)xYYU(pKh)-QDN?0aLTj_H>cw{>iF%4bnVz zTxI4P)#Y&KB!}rKp}L?a=J#(;9#gYxKUC3mR(0-G&{dmffn%6MiTxMJ=~`DZ3&Sqo zXcF%uv4$ID_?+C!Ti^IZtJby9Z68+j*=oRsdk1EmwWyW8pMLAJRAsuJtB|mS>V6bB zmTZy3E<+#@3G{OqpY`K6BN_|_)OJ)qGKMmEcLXWo4+)4Ftd_I{6zV5?jBT|L)x zTl^(!R|$*z+3h-4s~OsE!nc3<#&LatYYR`iT9}Z09@3{Y%n>C85(3wdIN(wc&oN|s zdHyd!2{S>1mI8-cO>m(VL+*M@GI|1pBI z4$`~Do<`=s{MY&)@dkd2g0h{&_!+b7yB6g*Cng+jFkhf*I_{s$-xWY{t_kLwQB+FokeUHVvmEQj^`V2t_Uro>C(?KP{?rAv;>>_SyMj$=Q}ry8ir%F_{s zhf7?Zojju>p5e$H_xa>~kS1q}+cSu=?4u~n{0MI=ajz2 zfTT+>LQHIbe)mMo-j|uG4?*-CuB3Uzn!uL&mk!e2W^&=v zbQJ^*Y{a*=+6N&9C`%{byMyqO5t8_x)#8=frgzgDEWS^d@v^Q)ZM?XFqlk0g1?n+I zenOXxzy)eRqcf(;RXn|Cdt4s%pVR5ez4^P~SD(nA6=Iul+qNf#4y+D7E0uck@1aln zY4Nzb%P)8Ai&VHjw=AAMd9*5Zo&lGy7_&V=p+ZwW?lU!Lc#+a-FugQ{*0Cx*)Gs|a+c#v zar5~kvHcqw&bWnNnMhbrfwOU1Q%snx-ebA)Ugtv3Om*+l(6_Q8Uct%qlxDRHM0xWu zRqvWN?@aPVhh0R!#$8HkP_Z%NQcLPmfv!BrUo=}UIEe|eit&+tdU$ow*XI7!y~P{K0=#s8}9XU~IEr7i!XZ ze%v+nj-hI9&Ih%Zs#s8-+YnGglb97n+0RKw&67S`t# zd$|RUpLFiNA|KB%{*QRpyMP^gWwk1-)@IcXpSHIjEnBp* z)Vkiuvru>682|O1NA+`QjkKY9#T-#TyDMGejMg=>U2y;0l^1-Jk^?ykyR+TG+llvX zO+O6Rd8w$Hrgf-cZu2Z|z-9XiH=iDz4+EonS-)0{lg3f9+to=YjfabeM=EV*Mn9K% zJ(n)FJ_(pD%+&C3>@o61zj^bIVv+CnC|A}j{5m$7m6^wRhe5dk)$h}(qpt_Ja{7Plq*N8ET$T&EWUR~1pOYU$(rBS zK5I{z-*kt#0VzAVMZtUK zbmXy%J#bh);l8JQY_45)kkLTGcFftlEnf=_W3D<3;tM(Kp$vDdDqpzJpCHYpmOg#u zQ5Cz|^pE#?C#~XY**MSIenL2lgX;Ws<^5A0Sq`z~UTZA9)v%Dv_U|928#=y;9;zaE z>+a*$us&Ac$+2X7UP70hJyY*Y7ZIRC`rT9Lvz~w?8>oQ`UAh#7jk0L zzF75!TF0E8WsW;#C^MS9du5N#{K80!yk3%F41d1Dk?ODWf^LI3lJM=>6xEViwRUGS z$0bp3sfSmpOz+w48lDp_s6}ApDBAL6*T}bB>Xe%;n{s|me3<);?(-=`$j3vD4<&B? zq+Y-pzOR5e57&+$VzQ0$j8hsWS-!h5d`m}kS0mS*7xEqdU6+AwIZ9HYU0JC=&<^GG ze^h;SSX5#6?aPIC`*+J#e;=d zfVCub?k!G{`DTRG*Y|9q%q;sA98nARTj zhMQ&ps{kV*?%kr>I9US^e%pV{%sjC3v)L(f{6hvtN-or^S>|jeNX1&bp|ER!+)9 z)Y#8<@j9w!xt)&Um0Efnzw*b_mC@qT{118p&$rN7p`LPk1+bel^ACf;ob|ii&DAqf znsyl74lT2qZZx;T#YgCD26~fW{?{M&*Cf;5G{zS(3>M>DpTE}DAQ&!@P&}a_A4Fi@ zrL|Qlm&TagNe$Qo2eq2g5+N8zGl*24&E`S)yZv+1CW%Dt*@fs@QTZ5 z&yxDA-wP~HRi*fQK7akTsBMv_M<0FCWA5dFaiS^Zv_7yf_~%>Z=!=V=PovBmgSSO9 z3w19j=@2dWnfP!c)r$aG>IO=V9?t8vPdO89bjR6IlAI%T_3K0Ydr@M60K9FDf_2>M zSq+=Xr47ekOUZH^o5gBpNvk>|CO7jgARH@I&)>4AtWvrSrs7Cx9#u)aALTsu!Cf>) z3jWg&0VD&=EeihU2%Vgaaid&||FHuDo4Xb8IRm67KrjDu>H~Dh10?#V0kb18sOiiA zbLxXW1!RUgT4Y`*3Rp06SUgthQHx1c@X+Gg*&T!xs zs6C2+E6P0i-{rKD=U85BG5Z=JYy5FjxKR97c7W#YMJ=$j!Q%x|w0m{Kw;$dBYqqsK z<0YW#acO%;qBNuDOftR&%nNnT624ph9(Z52+s-ThA>f2l)g=s_8{Q-WU>QA*y5{8f z`pH_VXQZknJ~`}g|LpABQx{*`b+pqPo2V&gRZZXX$v)db=t`2^J>3LOHlG^DEL059!nQ9s{on=q=L~`x}ICp9~Aco~s%D#8uzUUDgu-mi$xy;&r&CxT! zvFsN+(cXs&L*w&@w)!U>StlJWtGX{YN%T1L8Unzi&arANDRkq5-0>3ue+{L~br+z8 z`;e~xSGuO>@%bNS@%6ajfKO}wLGuDkS^3u8q7)BPabK5#+0g5FhX;9-^?~nC z2ac)0p(|cAvLGXMIY##y5^Wa{Pw&7Dl>HVdDNgqId}7ItMtH8!RG#>T8g{46^-V&hv_Vf zrW_WccR21B`Lo(^YOk>{Gk?xHPocsVeg(&^;%8{o*{rQ!vZ-%jKRrbiq=_0xUzD^Y zQ)F44&XWAvE$W7kJ9^zF0_R9y3rKh^-TPmiEzgb*d#%V^m^c;O(_P{D?%&sWt*klf z+Aem`UDehfBD(lmhg!j{Jb#Wd*urZMFhrTqwLxevvDpaZu!&WiIodCt>AS(%$W=Tt zRekxL_G{TLj=F0q(sr)jy>}R*G5XB5SSxwqku|jHT`>MT?Rh@TXT*Q#E;^y{^p_vr zo8GxNXGY;2Glg_$7?&5DsA=@C7PUU&A^~Q>4#G8`$C;F+CyV7t+iv7PlIg#G8;kuF zgmS^0sPBS(8#okRh&}o)ccSSpo7l%M!uBKVZ>LKPyB|IG{aS|4vuR8*FrDmuJh_8{ z>|;IrqPa4=c3b0dtphh-C2D8r{8RrNLQX9fo-F%`qAPl1-uo(iWKk?vYT@3-yYzmP zQS#cOQQP~vh?7%@#N@Zk3fGu^ZWEJa0&#jXn*E5goC1`pd*{kjb(*)WY0z9Qnazj$ zXQX!|*OueP_;26kwSzcERIrUSQW_P>7wX`@84Ujv9v{PK@)ZPI#03ikF&y{>!5twL zLJ$}Uh4@H_gFgnsOb#r7N2dvb;xT}oQJ|@D@l~LJR(Xf4%he`}+zp&gFKR1p%iIm# zPB=CZILCMw0s`mgulk`NGE81?EHuUJchFSB9YNnJ&m#Y} z%|X>dDfbk(W)wta)L1|A1#2PwVwPi10#QIbBXdEi%tvZJQuV^+?By+e2&9cKX@a)- zPnKm#yD!uQjre1;efi~t$cU{ia>vBGcu+osn^19_MOJU$C{rPh-+6v)=Y{a!WT|^f zh|=#yHhld>tfRHS{!MT9hgNs84cXJTr|l?c@BDc+83W|lELM@4vX8|zk+g4mBh(n)O`e_2x%=V&Rk8`?8QRoY6}ZK-kLZ1aal&={55rfFj%wkNcgr-UX0$+?673Xkg_B6z#wZt>xk)jJ5=Eid}n%8{~|2yoiEmZx|!NKxWI_c!I#1 zAr9>r$jT50;AN61NJ}7c(2y03P%#iJIXwOe2*%b;SvPi!zS@fT`EWtA+FCv4ViXQX zU5zn%xP6u^I1mNtNqk=+%6PZf^Sg9SU%PGgqcrfZI8~CuH0eLIX0Srwks2XL6n32T zrX~!*Zmb{=tUjjNY7|uxj;b+H=F-IKT^aOj?{z*#WiR}m| zP*JkveWS^1WYGBXl81DN`}l7EUeu9$;0^G;3+%c&ULf&9Mm@gD-xQ6G-83o4dW&v~ zA(_vgD{QKLpQiY)7r;NhfpUVlsDZgOsA|y5qUP7n0L(QXeTOzAIVM3%Sx`l{~PHlwAvfb>x{ftZ9a@HfVi0{gmUb|QC zCggTg8~_VP;!~$(0gE%g z`vZjJV}hn)L4gcVh!ubbf$;_9jMdNsO9s;*0t36=Fws#H9myw!kU~N7fe}5D2q{$L z2pF-5=)3^d9l!PvJBa^%{z>z-Y-1M2!-D6afcd2!F1(}=%a};8$(cS$-O;di`u$bU zOOOm{w{y<_zvNuqPjI`0MwjpuXHdSYGn;1<@NCjx;v{=e)mdVL2(hL9_G=QScu=jq zcl};zU%E0|2jQLBXirNJbDX7Pl?7k){5+F5cJ$@3RQw|r2K^=~p!GrX?eJy5YVW6W z`PXUXlyY)TzaVTKy}f^UA%o6A6K}~zZJy2%n{--k&3`z}{;1rv7n;SHz;t&_h9aCW zfA?*M*gW-=T6mWaKkKjXOlA&x;W?H``{hWm;l`_L09F#O|+fxxmj{0=w(fz0)+B?tK!?pxfzL)p?5P_rZzTvI_7s^Ne zxa)Z8FP;};DsvN>X4ck}#n0^2FtU^dSZheAY4Jt7DM;QQMF&5n;8~HN8)t5m;0o|C zm$PMI4|B;KT)y=Ev&WqGCe)njM(aEnk7DwZgW%&4D-$5W%Qi~m)p(Ftac_shc{{zf zLg769GiE{X3L+>~r?dkEGGzRBHIWB$^o#)!4GQ;B>m-zE-xH}8Tz~^ z8dwbYIgtAK!OkG45G@@LCW?s_sYC&hKyq+!HwA&6{U2_k1A!4T00jg-q0~nZ)Zy#R zR6+NqyOQpW9Cu_oujaicLVhZXi}3a(khp)b%ie&#pdT+TqLQrRy_-fB-YdDkCO3DP z$915l<{$|Be&a?(^;ZZq+__!$qbgC$BGlBLEPN?M>Q^3|9c5R#Pdx;A6fN=U9><3i zw>`OR#L;ZaVHH(rBL=sIDdtrp>&>>2jUf#xB*h)LR^AssP_qiF{#I=&82tr~eVi|Z}E1PRWZ1OoOYS3WmjmWw) z>ABBoW*~FpkGXMuNY3}Ts`B1QwKq-+PQNGaAiSJA@}(wpPaIji;MW2ev|h033Vq{C zRUp*WmJF$G#_#3Bdb$#Fc;%9O4oWu%X}z6E3?9oo3JV{=N*_G9nc?mn z%8h5IpEIxfs|F|O#xau6i){3HU)^B%`-k6guEK(*5<%6xfjDR+0E+o)yAT7|iyL7U zJ|}+i(|J;^&&(SREUbL$q1gNA`vsC!iLx$d%|yoxdL$oE?Sa<^!TH)Q3d7^^yq&QS z56A(3wTgZvkAS;CPrjlYj6MXu9E3LW_Hn$V|GvQ}y(phS6rY8DjHV7e1ZO-2NX}s?^4VB2*xif`#D+ZZ^I{Agp4EjnNz8#?>yqA}y8)L!R3;ZunemS#EO^F?}r&>0=Z{S37&-68=eHe7box=4G{*XD<#w)5@ zF!eeixk0DTzH`ZfcSy4nppaV{oGtv#iX2{i(eJT5c|4Q=!l;tzw#E-rZ^jx5T>4o> z;X+z!Y80I4bBm%fja=4Hn2M2+-GE&oIQ@G)^7%Hns1vLNvw_l#ig*B6why?<%OcmU z#{}5E;qg?Rl%dF*Q6aDd`6B9yvdf{B6<-#lKbP#2BO2I1wvmniAtY+g`R+>!;qeQT zkn8P1FrBr7C+e{TXw^?U7*4t>aq{zmq|<9V6qfR@g6A_XroC6CSirAj8{P~GP=n-* zkxppZI=SemKe}dvL01EM!4^fS1M$39ClrpPvw7@kocnGha+?G%!w2@x&jqHhqRgWcnjUfNxtk|x1PBY zdD8l{=Cj@Pp&e$XP?cP7(6P@GMlpzqMEI64b8aT0*|%ajKyCdIk~oLV3%*|NspRNO zf56YmREnUWT-h&RHn)D8J2p>j@cvf_%Yn=8*zz*&W84`z7U(6$x*9(sXXtK;%N^fR z4ZH4(4a(}#c$dHLcs3`5cPn1@07o!x#?K>XV;%1$vAL;S4j%nEqwe-E zY%y`;z5WK#?nh)r#T8-dGi4>;Kp?2De^9@ibcdXkHZL^kssA+%h_o9X(UJuTv8&Rp z5P1xJweZV;7O1ZeMVZIF4Fh>dSnENy>Sv$Jf{8FQ-mM2Aha1J4-9=^PZ8k*Q*o402{54$&jvlT8;<{=CXl9ps8;kxNTDMu0>6qcs+YL^P7;hM>)I(j(0KGB++M(( zhI1Q_r6<5P4_Va=V^bH=SwEFj&2%c-O!G++%&;G*JK?bQlyI`f#oc$_~oHm1B+@ny0dI0Vq(ucX~XPhS#uwDXQJLO=2bW<8_F z*@}By-~yVC)1tvC{lmH%)%pke6pvBZ>aJC#++mZ4Qh_m5ZV@s^N)dBjrL3ZIf&C+y z5$Z?Uw%4)TN;i(N)a*oX*>!?VTlu)RDyO+gUXDT@8SsqPj&GgIkfU2By^@5)CyZbR z>Wup?d8#}c_4v8S^;W{wHve2AUHn`~hNH;=71HkZhFO8$r@~}@AN%?_csxU;q$1Lk zn?ih;xu_bQVZ^W0hc=*9fIC|use#xIZOBNcp|mLe_mlEgFrk0MecNK>y)PHedW_nx z8&lnVOhKs~Utre<=R%g*S_pqxZ)X&L<6OjnL>)Piyv3B9H6w|iD5f15UNawFpx<6# zLC~U80r6d~&sq|-roo%)bl1tM>WEmLJ}$5T(SO>?5u(aaht@}{#g`PTIcmQf>}@vO zcHJFBv!X}VW!lAKUMX0O@1>fGXM8U_84z!2Tb42Kq}khd5;9wSSG1ImVI`@fUgXB6 zn`!d0XTgq3WNIuf+IO(SRH;vI-ko_6XjzOBE0jSm)YYgnL{8r&1*@Jm=;|3N&y?G7 z{jOj->A@X*ZYr0)!RBs}kvvjtwCk=-pA{`Kd5ld)_nL;|U0wEh)#OC>R&%|L58t6# zNv?lc!{c4^(Vw@calYc!vZKK_4M$k>zCcd}M%DjOf4vdP^DLUA)9IUE(;%FM9K2V2=ZcbZ4 zxhrO`%WKQs)8x4|T?zX}4{gBYs^GmsX2vH5Ue# zM$-(W`^N}Bv3u#YjcK-GqG8jhKQX@Ry3pR33N_bLU!)}0-fu%yO?_YZeSktcel(vq zfB4aPZd%g2XZ;L!w$h6JPPzRrQ;PO);;YAxDOOu+$89M)FJ{nSWDpqcY88#XLfHzB zg?^ib#TFAmQLZLGxMeGqZGhmCeqCr+EO^XDo$yY|mh%pHs%?L_T-$_wp1;AVW(ODD zgQ2MR*+0;4do$%>I7Tr42u zpK6&M?TkhuEOS~Ot-y&X?pEEeK#qB+_!x*^^g)QyOp zrD#lGOy9a%)|&ada0xaclHZQ zsL|X>0V8)V!I)Ryh4yrHDY7&M3x_1m-2JQ4gq3-$p6MxUAc+9HZJ9BdwJFn zH)S~b149I_j(aavu(g>6Hq2-OP(M`Nlw1g+NZY+tC4z^YF&vHgH7D3QipEJ$SI1a$ z%SWu_X>o-54man83a1&&Kz*l)a%=!8DV`@h5qu_oWns&ATP^(6vubQ?MbCJlL;w5W z?Brbo(1}Vgch8Ukh!hmCu(SWR(<#{1N98n3{|mR4VOQdvpKCZW{LXd+GO2xUL5;-L8O5 zV4(4Yl$%3dPYG7VS2X1l{^&Vpj%d40-q^)3>;1C?-Hj7YFf&ZLCh>}l;ToVZ#$9j_ z%Gc7}Gt}L1><4Njl486xw=hl(D}%#Mn0Ffms>`%m2_`tcPN>U;p?B3W1V%RO=UD9Z zny(*l>K)@UScsO6{igiB>N)l)1X>R)R-`oHD+fE+%`yJ`j;8=>>wrBm9a^b;vmRO=6=?*jW!{w-~VyJS)7 z2%MuhGhffDO-oHNUA(kXX;t2`Mi<5Z6_G2 z-K|t?Mj_58iRb~&Kv-)h;&t!-*XfmnzY=QOS=eV{`xBkp^E*9vi;XOM@lHA5bpJU~ z!^(FS#sHX0SO8hm-|3t#*WpgbOfyAd`h!+NelU|PSo10x{kU*OACDEn|K&^$zT&mI z*lq#gW#>>pbtLztU0211Ihah;YuJnUudN`N^D4--#A^z)sHhaS$Wk(4F{4&5k{JU) zD$kVAV1ZNv)4u%;%&P_f1I)HoSR;h25J$ao?YpT7lnfy-a3jg?3s!s$F%Ka4`3vBb zjit!f(S{iVcgvw#G<>p-cHFxuvflKe!}B`=t|RdD#xnyEYkh@LB{_@@DoMw;#7^%? zQZnKUSK^dxM46ryutC;uW?tKB9NTV-I)|Y7xV6X(itPT zyg*bJZ)%Uo3Z?N==!7>%I3-PwSEOUsiahPE9oZ|S8{1}*@&QSny`UI((R)SK4JZB= zb{e2SH8;bG)nN1TGRLb-$1g^%K{t;f9#Fb>?@I0>;_Qa@OD23{)OX5u7oeo#^vQj} z*U7#M?b^z%2D`hFTjIc_rk}Ble-6O)a3Rb<59k=YxPrRvoMm4y^;}6c&9PuJ4fxh` z>$U%wjBQjH04@~bX@+tJ=lY44g9Kb6%Xt7cDJ^SkHa#3Coe&GXq9AUkg*c*IVf+{s zu6=dUQIlPmu1%Xvb-;Q3ftN0(;#<;l#e&EDpiA)Qz)a0(fyZqI8r6w*nQjj$*dHC? zZ<|~9QwVjWdt z9n#ICcfcaAomORj0QdmT4YX3-z3#0uHu$0U;GR|JlSt4m^r7jb6JATuN~;I&n%c); zVB+$Ib`M%gitW!A9U7}4yKcL`UDw=!F}qHlF}N#p<5H+fh4z2OZM)p+Ksw%1o~jVT z8p7TQTXk1`GA$$tN8)O*kaA%=j~@I`1+S&PxSMjWzH{}=Ab!z&BmRc!hPdNiYR9x$ zTQdU4#DB8u+#WIG4K3ie=qYpQ@Kq^ZS^N0 znQBMWW`2H8@w%Ipd*DUm*!HJO(*|kt+^Wa3vJ~RXAlN^k0CxmPGf<9mS;V8dgB6~t zw7cyRWhSHjAI0%Nz3oFDUuA^xPbZvyVT1xDa`Opspk!zO8PzE*%-!zQ4#s6I_PqQ2DTxF zIsT6lQ^5ebNtTcP(32=f5L6#;Y=?Lf7ay=k28Ar9opg?ZCn#ZChx z1^2GdIZgH<`(X})*{_Zl9T8Y}r?^2!CXOnP!pv{hrN6r+d<>2t)I;)gxl*(0i>=1+ zW=8riUc5Rorv5umrczlgm-|i;%}EcFmf@oxIZ%&(*qYzGeqh-YWT^msB+Qy2Y_KzD zL%&binz_F&ArJoTT7q@Ta5rN#@RXT!(4J|wS}O*9k;@}&H{_SMB+@BhDBGiQL0dBt zDN|Q5QZ|{R06wDzCq zaA@e!PLoR;dzk(CEZUKx)!o|mL5ZFS)b!U|dq!0MW;ul5Z{hzOPqjrzdiN!HPlCD}jmsun(wc>bRs_dakpm1FTI5)8k6Hh+$Xz` zmY@+w{=%gVJ*$B>*=e?Qi=U*K@VC!{uN7N-TM+&thi}Z4UgwzDE}Qt&Tf0k}$MFLa zJy{T(%8DD{y7bnXhOl55on90V8WDg?4ccFK@%c;TY{Ipo9QyB+v6r6BOmCE!tA!v} zfG>cdStSH^rFd+xNB)A_xcj!Pj z6w|xJWyW=q*{c16jLX>$2}&;?-GvZmVLCKL|8zl zB3E~t$qG0Ec334@_!cjy^C~>F_Oi;kCKiAA72JW|;P`x#XD@F1^j4Yl>Ncl_X)zke zW`$5iMPcXQFSO`+6W$dGn(^?nyiu+$)hdkEWPk-|R|fYTIPEXD zUeJN6ZGWbRliO5?YFvrT(0M<^kkRg+GfAN51%QlfBQuao40o|t@4(s=S#hHUY9=C9 z;yNl5$nxSJXi7dGO!wlR+IbVxLvXMp1K@I3`SWn&bk_LM;Rax z)PzBEQr>lc9Tg5XM+tKOI+LY1g%?S<96#Oni`Ym;E2`5IuuoWG#8hd|wBDwL^Kbkr z?ugnP%)lNZuh|_ZOTwhh;|bd;_F@7|2OUz`56P=5^-2#Z2lUFD%>?l~rZ)2GIC%3b zTOjc~IWg>nwqvp|>%+)}?lXZ%>8v52lR)a4ey0vLm(PsSPu2p&-Xg22_XjN$5Rd(V zQ+pXfFBN2UwJjZ*b!9;-%vMy7A`kRG zpaxYkNxvqR{(Sr5`>@Yrb&qF!q6(=-VRn^o`~8*_nmSfs{%knwVgT1_-W#XvPy%pN z8y|=J8+a_axe)V~0S!vDWG!+zCKQ*?bbB4bG7LZ2fic&VCDRW)J+R9ebej>c?D2tPcLgVLj#QNk?nqXT=@Tx?MR{QR|~= z9Tz8z=~>Y^jPWblAI_X97JGkR3DFtsL|?7Opn#hIR}eblqH>V!q_DKe|BO}yBWnS! z^NTlvi|5SUF_8Mu#5vDQ4Zq$#H#-zLv{csV1wzKe*4vX@xJCX}S=^@maA1n&nBn{l z0BVQEmU;P~!jn9Hd{)KxaL;f1MB^lGydf;_cmF3w%E;2cq%tBGRYQtSTTcsF9R5gr zK{Z3k_`?v^@Kcys5I>F8fld${`xsE5n}9|ZO4^VCYeP1P{N zN99W15t~f~Y3e!y|IjFEHh`0cNw21m-GCLKT;87llKoj(9a9m6aj|2-skJiY>8tjq zNG68?N!*5cfmnUpYFS%mj;iAI<36~jJuJ$1t)%#B4>gu9^b71N& z?u!TJW|Xn!^Uqjh)D0BfGw0yd=VP?)L8GB!12dkYO#iPAP{1P+>L7u)DiMqXU|3=O zc`m9uGCrW$zOKqzfNha8kgda)&}3pe>(Jh->JJTLAqg z&sTsw%s8Ls-|7x2y2F%DCrftMb8OJ?vUSLiI$Esfk&(k2fyyT+8EUDJ10le^ks|D@WdtDp3_B%1igp@KDe+`B>0CwdPAs)01pxjqD0MLGU=wxV%NENbPlV{DJNPxvJ>X;9~+vtzGY zj&44D&h?bqVE$;v%K_smwhSnIoFLy`O0Rzz%h7|k8-60N_Ckj{2d3dyn70i#sX}eBvGAjQlf7sPr8}LZ{+-1b^d%hX9h7H~4V1JYXPGZ*o_UBzV zBktcJX&@1L_0Saq^aBes&j^=$vm3oZ?_1-2Q-%l{H*TLh=|8gF8D28B?{kUE(znav zH)7PO1W#~UZ&SKoh9G%+&dLfoJGxWKbs+sVJx^HU@RKm*)S;L^nL1Eb8T2~4`~#f2 z%CMWnF~?mmz?iRl;tVtHm_Qf37HiJZi_cJTw$898pS^PWjxXvovAUptdgPC_iq)6T z0+5fdvy&157_$iOS-$?Z>`4318(2hU0{;90pqUvI*C0QMwA zH)*C=TmkM%hcLI8RbO|4U9uka5K#rloR*1|sTfgpn z-V4c*znLLD9-Ymp$jc|uUtpRXuT(S@fpT=4cik?=V_>RgR1_zriC+aYrx9{ixI6iA zQjgARIi~%{@t^cl%RUcg`ZPe90ufAD`j$1M_EMl%)gJcS`ES27QKuJo|BUj3_sz>9 ze*+MDf{@%Wjv8L@v`VSh-tU+WoJ;GW8VzmxV6;O)-G2_$$6%m1z} zNMgpttb{NAO0BNF`VYJcKs^p*lf4-+AjZ^yGI0IE`5w0(+=v@TfTI_OWCsg^Mp0=o zyid<#cM`RX3BvV%lYRXry6{c37Q!o@P*N7K^_BZEkaVC{wQo|Nt`Yv8{~D1hHxfVK zf)PQm=7TS~V`VXWR?KK<>%)@k{OTo=7{^*K@xZ~ex^~M?+0c-qvR8(^b<&xu=)t4Y z^z|vY0Y*3T2g#Z3QU4p< zj&3gQH+M*Wkfceopv5Jc3a?b;(~1qZn2Ani^UBM;`EN9s9;CuBh!-9l$hRiGMKAbTF~CE}(9NB1PxK+#;1I*) zz2U&i02=m}+>>{8DTGcBO0_uQx>uy?jp!6}F6#2`i$;toeCUm|+sp}aTZ7^~fh z=;oa1=4$rB^X$pKFklY|K{^g;GD3BuU?rr`DQMlUNWs)hl>Ten;6~Ma944IwgBtZm zVRyIiU`ZjmHTIDw^CatTjd)SyO^C~)ra$MS<_}PsEwlBLJxt!t5p7evPZB*j_(Vv5 zXOY;~nv5L|9pWw{VAAqZ{r#eb?E-b`J*MjT0GgAvjDrgsu=6{e0@*|WSEMttuul-VcBf@YgCO_MtAsm>0v;vuF*q_;I!8uVon+dg=UC-&=VP@Q-#TmtgkYt(Gg10dk3eS^0K-i13DN1edWx?+AZLJdy8EfY z%NW2xbj3~DtiDxR7bUuYUpH#krFEONlEM?Nj#U7!|l z{uu&ErSO#Ck+*8oRbHe>p1AVjskc?21p(UVgWC>BVekLhq7grxF^<0y@a2&^CoNDA z4YKq|)E|s1%O9bDa8Ip^R=E2kD|UcN|Il0k9Y)mW0ibRtO4QtVvocpa29t&pgolOi zt$}Z8hOL&-%O!2)Q5kF2{stR?r<*or&{7r9_GD8)IC>2^Iwl1`i-twh#HJ5u z*!k278X_I!i`yY#e^oi^V}I>ahOb~Iew;J>6|~!wjW>7DJ*^R2=fY*06*q1yiG8q7F`w{n9NPr`I%@(Q zx-BcnDm-%M+Umr+3QZGjPXIyH@UoWQpn~}U(E=1e)(sm(x%utyTgur!k3M014f8S( zh{Ml{o}Ls)zMuzc`%Q`-5=+dLIB@i@=|m3ale{B>Nv^8%MEOIt<@0YW02Iwxa9NfCfWX=V6^}In*KE{9VRYa z=v_SbkZw2}@Z(XEzznt{i310QOM%it7tC7b@9P-p| z>UJ*gIRql9FFdC2_&#?cJ4k)`O(%UA`Pzkr+N+2!7jKznpQ@Br_Lgm6p<*u`0A*~c z+5dfKyP?;6N@kw;GC2Bm(f1H;Zzp6G$lJ-0>&saet@91~LDivxH@)oizSNYGlPg!v zHpNQL6=73c{d)UQp+*Nq#b`gXzc0ncZzW~?|Htn!`26!Akgej;Bade};g-2{GI)>X zX{f}H2>y#22s>Z=cnz48&g#7iBLpX7TCbj7>hcDI!?!5(D5kzt{H)Ot5N!lz543Z> zY|7W1d%l@w6OF!Yfr!Rek8O`*IAP7V)Q|7+1i`xipF>cG!$%TczxE^}fgiCc9r}-# zr#DIqK!Cr14FYQfl%`2mP||2#5o2x_X80@oYS-jrdp`PK-v%!H^j>r6-VRkM%3H90 zSL8Mf|0Y49leZnh(O>0236$uxu)7IAKxO**mGauP zP`yczx7QnVsty;}BKBxiq$>Y<$7pCa1Wb`pncY3~D{y5?{}C^6t5LSh4(b7C#@sw? z)vP*PA14o`f z9Xs52en-tNK#MP-YAA5+jaZ)Z_TlduMlXDYKjC^2r1zp1LF@VY#m2R_l_$p(g=>Ai zVD-b~fT5az=lqKw8f}J-G(H(uz&h+fEtQ6fk9t=y=)DWV= zy`jZaC0va?DX79zCEKs=B%~lnV5qy4O#{b%4qEyjk~Zl)>GY6|B^Dt+{Yyj zq&N;8C`tOo9rG$x#SbBVVE>eTIN8xv+l;WQJ^x7t%4#D{VWL*p7hKUJnr9dcRJn`Wtp?1UFzGoX>1D0fD85P5nI|tOK1Sd_7U$*6N!e z@l=XEQpMAY@{rRD=3~9dP~EV=A^p?3UN%d4?xPPSFie8CvQ!2MLFo2p8D74lVdmXr zcb9gB)R!K7s_4?8Q4-ykBR5AxElU22RW z3TJAOX&W7QaEa8UlD?o-(C&1!i9wQ;r+!05NlS4nyRFjf3Dz2V{o_+CK`?8KhshAa z^(GVqf6@ENp8Ch)w7bMGYCx|G28Mh9GqD%Wu?gs5%?GikCyCVD2EaAZkd}@o0roZc zYKk;`Hfu`OEY|%M^|vN0ho)8q4|JojhO)^H&2FTgPf~96#HC}QRg0FINMYQoH;DzR z{x991x8N-OiWlQZuPHcm5{{VS#Dt7%+D@kU6` zO+HRy;nE(AM&bo!@e7qy^kT^bO&}zI>Gff;|im&a`N>ZTR3_*wabd?*y@v zIFFNmMtl^C`FQI^C3@l_G4y!d+@}BZx_WKmqeI*0kdJN{UBj89?Q_jqv_>2rqG3Bi ziS~SNe+K8?*kjU5o&wwuKVPn$_pfuWZzD~zxo)O<1){asLrewB<-HwB_tY{w23dI5 z&pppA9*+tmHq*!L zJ`5Ir++(nw;7N$c!kE8W0Lt^fi9{WS8-0*@5HiNRVJsqd>KXLDQGmBk^5thR^Z(;w zBvQuiCo{aye@%NI*KvB^)s=W#Ow0yMX5YKx-bwk;T*cDQ_3R8#u2)WbeZ3(+E@bcb zU=D7?Bs^j@n6~KNw*Y~41UTDz=g9qu4RpKY=|2UTuwYsJ(!elG`dcu%0y6>d?JcRC zQ}H2@EZC)H(ydCcR%?%1fl^E^yP3X9-5bQqbQk7vkJaG9m((%@sl`q1VfKeKX>09L zDQwO&&Sv+8g#MDkA3mH1g!&(0e&f~0fy$;YsUH~!Y-yMIeK#Fqh5{dbb^^l~YZi40 zS)TeLfAs50&5ziLJ+j)!DaWWgWz({AHbKO3XV{MvBljYMh@}NVlM}p#%*G`Eu)1KZ z;T%Iy;L@UhNjCnmb?uu(#g(|8NF8E5Ms3EofJdbU!c+a$>6uvLb>1jO0uqDyeAwD}jcO}cDduaHzk&K+30CxhgM9__ zS#2us>PYOmmHe8hn2s+rzfb2s*xG6sQ5Ylfd!4xB3ksC38%B&I?l3e^ezN|oBQOGP z%yyM=`U_X6a9Ct4K3r*mZ)1V*ZfE9FNj1a%Tm5r@ud)A@WCR3Hw|cRTYQKAP3K!mf z$z74uxD7S466%$dQY1^+hiPKMbw9j^nN`5A_-|k}z>V9{^eJOPBYv?znJ@QIB2#Av zdV@0Hoj}oC##tp~sFxTAM4GHyk`@3Oe`O3<+*gIb@K3H|x%*DYVh}erJ>EYP7Tj_u z*37@C9gn`Iz1*0NyACFH|M3oUUA0-pL;$-2U}5;4;iX;KNSB$Cr3}9~`Q2_9c#P#l z9t6n|Hh2k;t0eOi%c3RaSqx_0hM4vNPHa^Bk$~`r znWPw|j_z$dIMe9@pTg!>dB^w9{1RvA`!(X$-hN6BUigR}BNV~Rx+z+MJBjvho`7Lr z;{UMq-r-co|NHnsoa_~n>_}z^*;^u!vS)bKPbo%6DaroWtD#3j(&w*NF^wl|O~2TevDL;`vM&^;eP zOQLmbZUk~b_J)NI*6MqZizE_ckE{=?8iT|&uab&ujlZviz`RLAujkdtO{`oxfmq=k zipEv|6Z>QxuC1T*ELgvT!t%)TOW;x~VC8-eiK@R+e@mG!r;9HdbT7jD?16ycT$Pl>LCUd9%%hb>ZM*=B1+=^Pd~u$oIM~O z{x$13h6M%Ga+>CFlqkU#Bh)|+auxD{}WJyVx7G-*V8%{fxcLBbk$koV+9^BSxd zrIOP-(lA%al*_OrD(^K1PyF(uF(KZsww=RczeA>*Xu&-sBpN7D$picug+sZ&tPC|G zID18mb3cmE!Ei_*6bx6Sp4_w=jmI|{sND}IFF9Rq1!TdcpFxE-H_urDp$FL7_F!)* z$tmB-gUk)iJg$&!zD}A*c)D&2E4VYT&4>P6Z`MGKFoXf?g7S{V&N0nB4;uM_FHqh#k~Ayi9}PxgJ4eC zXSp>RjgPOO??`!PfHU#dhUq)fSbLc_euQJ@L}DWN0!H!HeG)7+p&Pzo(n-zHi+`=P{PR0M(tlG3=!Skml3 zN9wuqc(|DFG3=LV<&_nsW+B~?qr4BvyQ=T8HkTN9Ev6~l7WQVcG6jV3cjkP1smO1~ zrAMICR+M>K{z^s(3vpZQwIH@es%gyQUODFqY_L&e>l;A*;>Q=jeQqneJ(exW&{VNrbAB$EtanWEiq4B z|N1I_;=w~jB?t~8Y8!n(yp$gHCx~>*Js*v{BiRX7<|==GSNVwKbLVbDIWjPPlNf)x zeWg))YEf@RJyTf|IRUpUP{|W3{HK%#LJAxpt86==1Xt>{U-aQ5&QP|*4YDJ@^!a-~ zS@GA64iwkz2uiYPy+0hnxzZ@e=3?_bO~3Dyt51OS;$XTP?T)*%%=vG~;qnheUrnFI z)aG1&4fb#TwiKgv(rnc8PCwhWK@dZUxgMea3zO+f07KtY;KIW9VNzd1)@|$_ueZCW zx8+vigY2F9SWt_e!4=<}S|ODn%S1IS-wobKcf38w^D*^g%}q15e5@J_$9n69=&-L2 z{6VPZdp1pVG~3)!3P4@a?)e0oV(`oYbld|h_tSWD_%mdDSnRc`kLS$(ZXP=O^+mT* z-5&f!F60x5=vC+uh#p5d(FUz3QD!|Hz*?BaRbQNn)OsneL@0SMah5thOJc@Y(wg!C zeAhJA6HIAwAPl~=2=`$#QQBgp;vgBC{iN0f9O|RstvrY2wL9m$FfQu_%|F6_J@>ry z{oe{@;t~B)Gm{*(g%~Re5%kTwe|`cj7jmonr(?r85T#vsxIW}bw`4T)C8w!aPmFh) zPjlT_K>DI$PLS)y+Tcb7AM%nT07=f^H!!TElv$Sd`guB!pYq`KmvEooDrbJ5KdW2N zy#M}-FhFuruE;qx4fXy16Da@=Z0ZjLu($sY7r-*NMwnHkq9~w@wlKe{Sua}RtYjq* zneia~YuY$L!O!0JyL!hYt)m!MsAG(O0y&+H1}xy0VY&PP1G~mVyR#z6)Hp^HpqfRXglSNMe|OS@ z51lerD$E}Tsmi{_#zF4zgmtw!$j`_f`5kcQ$hMN zy47YfWSdkDi=1xc2GzyWRGQHr zdl0RVeg1&f?|^Gv^H0LmhGhg`#UQi_q!k+>IarTh)aW9@9&e?*y%V6rP$+$Nw`HS&J9)+QL2#*6ex&B9&j7rqYI z**&UG^H1jMll#g4BH?LN&>4_stMxZx)r*YZT0gfoY#$Oku(jkmvds$`U%$_u-JCTIB^m9mP6W2MHtOh5=wQ zfDfxPmo+o)eCU+^q&D{30I?81>~XSm=64FcvQT4mepY?qR$}ybA$Jjt_6lTuR+Ld#@%`*$Hp^wNn#qFrn3M`lT4pWW^ki(L4W z<))aW{JfE{y}f>URvb@`iY&ndqu%Rg5mh46cwcJaudqM)!9#S~bL7Q<%Zu0LDueGb zFnmOTU%~(QA>bU$E(akg+a$BnWvSwvH($J9O-!@C2?v;&c2k8x0D2dK&YZgO$|z;@ z10s)IowA(fJ`K4s8==wPERX|oNBJ8D)%T>-B8KF{P7B>1PFKuS+uQFa)%gnsMn>!G zRr+t}!S>jw*B%}`H7lQemaiopn_^8^Y1ocA$S*6J0=R*CyDI`{QxX*)+Zm^S3{W5K zB{apdy{{vcrf^5(sQt!lgXY~VE#%4s&PA=(!A-CK`g$g7{elQ`TimBH3yc-uo%|0r zP2WRegzn&s@F5}fw8-K66x12m+am*Ha4#68lPW&|KKzOBYjr!US0v_p`Y<%#2mNJ) z`ZN2W8M)j2&$|JH?QYpt3O-PSg!sVwZSh&TTB@Y{#?hTZ!cLX()V;{XIbx3MdPmj34uyUv50&tnmfAP<4gczsL-dhbUpIeTuz3#_|$sFmphxF++;7H2ir?577+jGi-JHmA`=ZZqefUy|rQx-r+{C6Kj=3@y3K&(O@3;V7|4 z5bK}V_k1lqj7$)dRc-7(4a18aWV`T^Zq*In97?OSh}=sg!uZS5E>B z2i!Vf?!nz=G*?B)_NB8tiBB1tG+ZSu)VjHKFv$bD{{8=8-dtee!3RWU0}X_-v0izZ z^&^fY_Y$qCA7~^k_J<5&NrlSzyv>?VA0-}^r_evjWJ9dl_?zHyJeEG7|86UKv%<P3zke=Ta(&<%y!|?dZ9#_OhM_S5#k!-A!0_X8j1__ejSMmFyGUv>16uKjU zOs8InU$2rqs`}+tk3~ds==FU^PpX{9pI}ks$q`9v!vVy}*oK8wt(*EYKRLeqxTl$x z+3$9@?F1onvt|0$W*_r^JB4_3Qo5{#Np+7)-D8Hpf@5u7vtX-@!V!Tss_Nkb;kNbJ zD_0Sq_N7-NF)B1E|5iCX@9J4&3y)Pd-`KR#eSWBf6&v&ON#KQEjk9GYZ4c=79`-*L zs^2tM%52BDL{OWDU-ruJ#z!THam2v&-WJfC_VvWMj*@Ovt0%%GH>}Q7bOpv_(iBc_ z|1b7Q3qd8<;GnU>Er0qePn%(}PwTW#Yc+XY904lf_ zXE-!f)$Rok)wg)CqrYZg_J(~x-7xzPhJ9gGB7Tc@I#Kkm6}8<=UXQMC;(Kg_JR-iLX{C^sBMIS!IBAYrirB}Yqeof^LXA`qW+%=1+b#P zKm#|FCPGwvgYhiC<8*>I8xMn&ll3YcSB`AXy@8kAvH|;)2*B(C<=#u2!V;5sDR~{4 zORt$?IqSo0n`V*iyZxuA3DL+S@yfZ@RmV_#*T6m}pZb2!2}>K5?ve{XOC7o%yu#s^ zBXX5SpV<~^_-*4?M~haR*vIB`^=gmlq~b#TsKQu8?ii|k3k7|i;v}WWwoLOOFii_U z-dp+(+?ar-5eFRsy)&A7O*z`3g=aaQ-J}0x zHXzeO_?qW3*!)7%Oz5XTJqL6UcLZf@O&BJ_wAP#;Fvli7S+fP!oQ^|#siFu3VD|*w zIuKO=0@$fu>CwY@0R6!YUc{qnLeZ_{j zLwbjaTmZcG*Msw)yPv&6`d(mJ*_PUt?vDOAwzSnYy{$Y2KS^nwxte zQja{ypZUWoQu9-q8u$Y^2L?m0^d%wiA#(>VU$4Hw8cR*3G^>J@pZbYcSFRy0s}2Gb zYZI?7qmnW#sILe=3`yED=6u0Ps^ek*dQ>j@RMJO>_@a&kn6wymVIipkcs7IR83K5u zF3hl$rUV8Bho(Y_5e@U?-wT*XkpA)vez%sqHE%72iXyV$&ZO#nPm$+>4r2&!a;kM^ zuX%g)u83a=eipA;XNw{ipA@)y`W@)u{^H^6+ubG1wl)lmk@DGp@MdA&|2kvJ=Ro7Z zw41bTv0QUh;7pse*YJ6DSjbFU=s2ZAd#!qFw+_>o>GE`~2isN|Q91iwbhwm{C9Wx# z6>2c{l`loF598ftE(B6j_U=MF3Yd@?s`=3T;hDda4%p&sJ&m~{T_OqRS&57gy$>~G>)EE|~YkB?gw}bWb z;35Ii&+Az9$FP2{h4E+SQz`7-6w}-o z3iFv3gbX!5VS;yVIIP)t{(@&LNocs8(5*Mc?hEaqZ)sb-2QY#`!5e)y=10ENzuDOC zN_k`!9wITTE~7=H^*XGw-8Y$e{*<;XO@N@YSx=6pbD@O?axLIVe^G~;KPS->BK9Gr zp>_8#+M#Ve0q#zi`^gdpGRn||p}0V^Uw^)GyqEU>;U2W-z1=N07T>f@HFg_uVnhm~ z4@75D1fmyjAB1>!F_&Ku=#F9&Eu_e?uM)h>PpZ=P@NoXBRX}vK7j!)S!55&W0EKZ*N4*MIk?_qT7OR+$JUSDVHar@HcV{#Joa{nY#>t_r0?YjHf zk(=f-57#c=BIY=0Cb82^MIKISq~V;tb+F7QsJNI4fJ+O?oLo?vOD9?_UU!l2OnlF+Y*inDaRc`INy^eW&LVu zAe)i7tDJ1PLPz@Galz<&rsOVy9c#+g7%uNR?TKG#%tgKFmMAurs%B_8emiAH{ zqAg&(`f-Wht#{hLbMxCxFM7h#Q#T51peW$!Q26b(Yu2E{PWN}ou3Qz{5Q^0|eX>6O z;wZD5N(1LWNS0NO9Fdh3i>@$Av|g)35zxl)4FrtuLE*NGztAipv17GRFxxij7$S<6;FO3 z*wJOL%3y^yFi`9NWUi;b(DeRs*y}zNiN%0>eMRqa5(vT2^{3>nj zTSGeOL%KX~_uyZduevba+*(prPGP3^L4_*vNV4D_)XEs63kWZg0_oE_Jm{PLYS@%p z2J;#Hii=E(pU4mo@Ww$7+jx#slYJPU^tlAzV98#u2xPs43hf4af&E?jrX-P@05P?T z`}63XCNe<%!H}k@6I1(u$&ZlNDYWOKFMaDs=AV47RSq3fr&Qv_7YZVO-Wk05ECHrI zH^V~dyZB(q{;p|v^rie!NR=~k_q*cRpi?@0YQqJ$Eja)Un5^fRD3))iQDIn?cS-iQ zScA6Jp0Zr#v>D*p8vtZ|XXSh16fjTw&5YeTj1ZLu>zDnk6=8jf%_b7?$)CXNqn6O&zDORp0q1HC%38b zg$KcZzg96z!m%p07tSHQ_K`%KOqUg`_9vouG0}WrPeO-=Lwp(;W+x~`0wL%RlTfQWhX>bEQ%EC&_s1iy(5baF|jX6Q@A{#tOUbfmr83jF^=2uz$~ zmUqXmgJno@n+StJD-^HKe!wB7wbVU(Td#GQ!S7k5a!BsOy_S=6ZJ#H!Ve${G3i5dQ z6uHe%Z+i=gW{FkLp%D9WKPdJ1%i$35sA1fUMEUdsvZQZ*v?CFYzpL@ z6a>jlWdfyyLRC10D`>Mz(*H8cLVrK@w-QJdt)Pg2(he%h55gsU4YT+WO2*O;fqX_K zcv;ojirkfNf6Pl_Zs@adE25V!hWdyZ^KygnC6A0-b2;2hwkzF#M_2QMnqw0Lm>fhg7ne*Hv@`XJHSF55!DYib4>RUZR zBC%?1Osw{zA2}~yhrYRoA3>qcHFtpq1unk9S^U+3X?}86JVBktX4lVOrVI4b^&0Dz*H~=+t6vn8v1!CcCB-1*bK$(^vrdWyhn~-`V%+ml6AuO z>Rvz@E)Q{V2mO-sy0dG?HA8!GXxA^9S=ON>qe9mN6X~-{h=}{5V&Pc~ou^RT&U&^$ zwQZRyzI$23^$+nw;)#<@Qsfn~J6>`>akd1k)=;wK6Ql!cZ%|G*B; z=49L|iUnVM#9+bUXHS>BTsU=RNY&I@z!;85Gh;IR4fWi=`U3vq&8O0O3UufM^~R70 zvu|CyqPNz@<~Ao(I7B|4WZ(W{ebnimB8%Azp^@$guKR69@4sZ=|Ee*~3_C94M$J$m zH9mCv{80G_A4^o*x7tKu75BBF z&O_?ek7roLn?cWFIH3@kd@>L@vpKgBU=|q+m%4BO@B-D;u(dyt*_G#p4d<8%C62AP~F2YqD@ z@tZF^vBnnJf+DkT#G1I<-oGFq+gVjxwMGq`J=1MYlwZJoMI1m+MeoZCik1ef?~o+g z+T_!lxqzv3?YWbfYnkkJuUkvc7~l+x`ouKn_%`2q52l2uHG(J2Rj^3VR__*@J|7;~ zJVtFmo)PwwK@@v&S0Vz-%EUkXd(kPvC44hwnUC8)cT-|I2-r*XytCM!56mL|1&6m+^6#nc@jaaY;3)prJ@W7`d$E zo5pW)7x{ds_Q2Nny+xw#X8YJ9{z=OOu#jkiHg~XY2i+=YQl;zYxLka0q1Q*yNtw!> zo{9IR@DD~D-ky{vkBu%8HvSD6SrBbYE^bTyXBG~4U`{Fl=-K-#Hh5UD!Ts;mVFmou zEC~XAQ#J+F&KSf8>w53kW$-Ch#~stq9?c#5Li6u7(tfWd@cwu)Yp%oQTn7!{c^ycv z79d$vz@l;|zz>7P$C%~Fn^Nhs`Bzn~JEKDMTCZ$FO;!);#-GR(Q3by8 z(J11}lJ;`t&7?k6{1+tkiV!r<%NF7^p8aPgj8KwJtoMz0KBb9)O27bXo{S(J6}ram z7lTTJuKVWhE2LYJ$Klar^D4hNO9T8Hf;c6siob{0uy8nkw#2WP))OO-jaBahl=5|y z;c%<9U5xxS5i!vf7Ea8Sr13AcpwpLPZ$qHa;{O2AR>kK zS7`E$5{qoopW>vL?W4VRODdgxkftvk2IybzS_E?Du7vBeJY-d6yp=;!_&JJbj#jgc z{)|PUgFw2T-ZKoBq@W?~LX~VTs(agVN|xeXc7FI$lVWu&ftIuw9wy|Se)r=x39R!ZA6ttuy5^A;!nR~UrL7u@#Xvg&TP#Wu&>;Z% zgb~Ko=X~pTPd6Eun38a9i#K$`u1_a?L|9GonExRrmk=~`l6rFpha9Y4z8-!LpF0+K zGu~{hJ}eh1NHG-c$vV?gh`q3{Z=6Gm&zT@F_=4(0%VD;L8Kr@~pTyFHvcxV>saESE z;DG)S9lC*h!2p&Y#%=onw7N5Fo=zAnkvt5$=Yk`?#(6cHL?XGI%(L#UcJrrlb2NlG z5_Fg>?OtrwZ-qIr)1g+|(4O4QYyLIezwFDspan zpA4g(y@lLzmD7A*7bpAB=Yp9eD<0BC@40F*w3Vpn^3)7?pbhEXeP2phBEe7nD$jo} z&)%09B7RV{WN7nqx=ZVK1kazsD1tFo99?I|iA)+ox!?C5N8Hl5@<8Y-OU((eER_}M z>DG}|dWLBqY&t*|O@EU7i=wqaLw^}f72E``CCX%7*^4>WcjIJdE#f#*#^(xCp~BC3 z7>ZKxC#Ru{QV84x=`WMf)_fFS2Pnh^Lp3aqd}zIiu&Ft4kAjGrslM10PK6 zrSsdUc-AR4M;VMsR!X%IIij`hnA%vk$f%k;Ff-*f+x`6?F2GwxGy3l@$2(Q*LVrH} zv2gFNY)1;apkUl_5Si`?JH6F5cqtCRM=tcq0I@5$mH;n-!WDsQciXzI=jhTEOU=?} znm)EU!^vZX|F;(ceWXXl;e6b@A}4$3#}fv?ih3j;+!(sC)dduw}DB#@V$BGiBDxOQ8J@qV&X@sgPyD=7|$U8ZT z$3>*2>d8opSJ{!VTAN@56CpudMiL>gqJm9hkZb*l+bs*10{xpEuwr#xxmH7#9;H*YPZ;E{>|aq#+VOHFLJLu2Ey#6zj&pX$SG| z#>C&%exFk!Y(8@s8`WE)zxP3MS$S&0qb#YezefGc1>ZL9H{}xa7u3J%7UBeg`ps~4 zi)i6tBu{p*BbSP3UBJ?OrTWB}4e zfBe!;=b_Rr1Vh2fee(y$hu=Hm3ZYNd3;;2(kU&Kr&4)B`POB(Q?NqLwK%G<5k%(d* z6tMj5->2IT{dEtA-FKGVBPvW@pDuz{DBWXacOByv5omKf+X}Dcono=Af_l=FeH>2f zdhx`>kBlz-+->)8AwX29opq^c9=^v?5QFay4x(RqFf|8BP&yz4+UkhBl@>5hEp#C(L15 z5&QVdV|Q zE5}QE$oPWz)WE)BT5g@M)WyYF!k3hTXuklAZU8XGM1|DZhh#dzIoI6;63#y-SH~8_ zhsh@hbdQmlb3iUmee{fTZ29{Wd~3pC7kbUc(@BpFmK$|@(@jCuqU=p!A-rkxf$o?! z403_H1Zkz7{|@oVP*%QgBz0hl$F^s{u_?ipeYvf?`JH*P2oBsBWZyC*t(M__WE80= z2-lR@&+fPP!a!XTRwf?YTy${+w&EX7_$Wo}ZgvK(!0l)z-fU;&YdOFDv2NeFoLe4y z;D62T_eri}UvYZHyhHUU3Kr__$Es}be{7e%esy?Bk$xBg(SB%u!SX~{>uIZQST8^K zFD}9s1?0z#4ZTl^<40T~sd@n(ejBQ!_TsovVW1t-SQd@dR@8QK5zWo>2jwZ_K`xz# zXRm>_ryEo&?9nU2a)D}Czr`n=`GDH>iM+`G;4&6~{BwH02YE5Rtw!_f0!=9lH7^?x zY9C-)n_-mWxEd@cc?X7fGW=r)LDeMI_x(1`uc^~}O5T!|-ZsD|;RQvzCK&GNc=c^c zQ(i5clhv}HbLr3R&jORYHn{8cv}_Dq7a5-4Ss5!EaIIn)91`2ZQGSGKwOg8mMdLo$Z&xp%xn2)ZLBEko>nSZ7qO=}S|S zY(6seh}JGWa({yChuIrVyxj+@dO2!y;iXjM+0~@=w>hdd(u^P4JWU=jdbsF3xc8ov z_@JD&L9FX4JoPqv|$7XWnuuutP(pED)zsJ{>KJyRzN zIC&VXtc*8Cz7mO4J&K8bF69$L0oas`@JWcjt@3Q^j8s>(56poLKmn8XnnJ_Tr)zZ- z06p@X31t-(P;yr-0W(j3}3Y zBN7@HB?^?6RQR-yVC5cF2}xo)UEI4yD)?F4v_Fsn-v_Z2{x^%^VZtQiP#f<(bc-zO)J!~+8{9Bl%rWewAQn75wKQiT2k+QeABszV_sMv9)O}g)ebtpmmG2|TC zu_SANO!7GvT-)^~-yWouYL1d)B!(Uc<0qprp@wN_;`-%%Hdl8T&l`b(nIj_gKI>C` z>;oZ|C#*PD{1LxCs6O^u-M_%xC_M=*gy!bjb;jkr{-mqf-cre^PFj3}LEhcKqeiT- zE=HMM6B?De2Ad8HT+xjvtD@%D_e|A`K-kLO6?1+Na2v7zlEgpXspMKkXe7;W$Rwb%t(WLL!)RYQ$a&C zw7nE*Y@9tpUYEFe(A9ixCo!+~szJu*2C8aYCn`M3lCrk-wx398kpG+?$6E6AfU#9$ z661mrHrU^7IzgQ1@tEtV!0GZB0~ZSC)-iEW0pK2^2I5Hl8w;~_e@>7Lnp z1l1r$y-=h*u}`Ja#h92#Mv8K>q`>RTBZbUDSL!}|Sn56MrshfMXM$^CGaDY(Im`HM zR6r^gf`(3@NUrZ|#nO$CWF?<=xb~)ZYAi-zTB!V5-9{n2!oEl~QfK{-|7i zlYm^H4DNf1HS6Eb)b;bS*{v#CvlaAo;4;E8&kc7Rv-ORGQKApy;)Lb^g^J%e8juhZyI`R6TW9dXette#fwdw2 zhhGeJ<^pg1;C!F$jySK)8FA&&5*;z!&*qHiFp^|b*%!wGHmBTFdhou^iJdZgUai9l zmh*c{P7yi0@$VCBXHDc{$@^k{VGJFhYD8syFft`CQ03Wt88Xc(b4tyVT zc6^>JjMsF)|K3B9BUZ}forYC5jSYY|W305(PnANa$t8FHbIWS3nEEVi$T!W)+nZ*| z@>nktPdV$53}T|Y!jMHQeO;a*;eD@pB&a-lot^qtMzcHx&XChhR~yYwe`WIHTFsoI5d zYov*p?Z>(}`z5ri6UdxwhhTJv*j<6GSMT0vN}0X44#^h)yEUB$$>YZ6T@~5X&N68U zC{D5+%|e2u!<60yIWO#p=~)Ym+gY7c14D1e7qb$4%*vaauJKm=lYZ3UFZA7-KEp?? zey}mKb$JH?Zsee8_Rkm@dHHZfJ)|HB2Ag`!cG@p$sFYROmuFgugM`2%iEnOO-3P+( z3B_|Q>J1bte`2Z~*(;BLsnv zeL9t)Vxt|&s3n}HEmBh(r`clsRmJ`m9XJU&?WGsO14WtLoZRw*LVSpplB*-G4CKB+ z`4aP>_phF7A6i?583aa^fP0(Nx&CLjR?fYlWS0 zxd$St*-0M!0o5MHpVajOJD*7)5dHN+vwwR0NlrwGCD8Z}C@CB!RlW@_aK66V{CIcY z&X;2AKbw3DnGVTc z!8eW7@)s&h*0NpO$MF*%tXCvQ{boAwkLVDA?stoEbFZ`^Zs0)6rWZL zCOIi1x~})gHZ|n*(Kk#n5YJKj?N7MS2gH&)vXstYds zfN>IgIvjj2(3OIND}Y*ri4HHMgteL&oBr@_IceVW+9&Y-FN6Xp<6~5c+pe#+F{m~s z$ICC3TT*G$$L?T+t%LIp>#{@*PVlAn(S{0jRSNW;#?WUCLJ!|XWr#=}UX!=% z1zm&W)=R`z4CA~^?y*2ehms69qjbe^b^qv)2%AwR)-#d$GuNFVYEQWi%`;o&&O^Sch$6?|2TqkFzkK?68K zCjzDsPm`KmODbrjw2r(&`P-4rGkIDKWX0;gU$O1Xw;65t!eRWO%F{4=Rm#!IQrPe= z5-PRvMj@0QvA%|q7;X(5@xd11;1D6MiS(9fF5T|d+NQLRr*>A@D=p1mx)-T)^&|kL z^J$muXls015TdA`^O_=)Gus+kUKNL$2NyH#IZ$TeD3%X4;<_6i?#DErT6@?B=|wKgz7Qpm2G(Raj9;vLxP1K!f%3E3v_0*m_mH?N8s zD>a^}^+af|Mfoc1rS~&KNoVL>wY|!yj2BK*h78{;%(gs8pgV(e=7wmq?Exvxz(Fh|} z&vc)y%X=Wzk1cx6Scwv=VSSk2;Wn-SuE_-b`qP)vxWBn(WX#(yTAx5? z=rRKma#;%rULIIi9yJj6vL2S$y_ZqqjR-kUcAcUTr_FH)LU*`VoQ(hYWcVtk!RMg9 zqT}taqNq69CoP130d)DHB>53v!*Omb8U#q5$6>?Ex>?; zz{Ex39}*b-sPv7$ID+cAhp``fUIO2GN(&Bpc3qFnH9U-y%IZ{djQ4yT7-Do@YJk4c?Je!F$*lHyRVW77{tzVSuG_+a$F+Q0^m-^ju2q|*|Nck( z@;-FP5gb0FSh6tt{Q*pV6)RE%aS;W`KWfy%#10vorgQvgqO9ii%h(E7?NKkskoF(X zY<5>d2fs_lvv?1c1D>A1dyB0tRcHg7a*ApdM;U7to5nikogFIB*ywlgZvso6%`f}y%wC9?t3>0=i zc>?`Nd@W&e=q6)fR`6r(M-HyNCdEBolsNZo@jG4mT5PFVLlde7$BIYORZD%|-0Jh- zVc)s&1cu&Os6T249Ma_4Az-1ce7X>!z0sht_|W;iQ`Q#uC-)#88ev zqYLN6Msi3r*sV=MvO>|`T6Z^;YZFtqJZJ8kd`g6+o-{6gj3=_Gc)e!htSkV(g%||7 z4>5eOWg87cn3$x~I-|-K&UzQE00%5D;kl;L`Ovcf9`bh_RyoT;Z9J&kD1rUi=i=ek zsjud)wcvVEj{B6KoPslsOV?!_X!JrCKK4BQZaCmDbaezYNYf0nQl1M?rk)9KXD%-! znxoC=j)qdWzt-c!EQg_)z&96vkV2A6Uc`jf|5Y@=uEfC6h!0A1!6>LNaVsNKN^eh@ z-9n4pN`)XvXmAtfp?Wy=uNhuKsqUz(z1s)a4uTF-JlTAPD>3!E=#dm{#45icM{D5o zgKLbT#$zhEbX9bCylJd4$6{AhuzmH+hMYQMM3k1B=|;Gum%a^2*9a_%wh&Ey zL-G;!+svj~O_zR8!Q0-+lD=$-Tm4C$`|8Eb__an}H$Jxb^TZ1aF5lEqtPQ!v2+q=DT;1D49&?V4IvQZq$6(;O?b`5ODBtQj zoNRo2BPk^f$O2hk$C)!hxLZb)QhU?9cJvT>%1ot9Jt`!`W~zWu@S;*6=!WH?5rVXY z_(z^VjKr%7iWH`w3yi$%C3Op@)s*73ry9x>Fw2*8j^DDc(f{!RD1thUy!Mv2YxXq2 z);$`6HW3slgBY>*)Nx?IX%JKxXwZ%-L|hxR4|wOM7+ezYs$TlFKZS^+z2AHMSKbHY z&_bmYdLyGhPe=MVa>kVXf3C@_gN1->o~1z=)FGQ!I7fhdY>gbP+)D4N-DwUk6?N32 zFA&|ang&52mA1W$3JWX?s5t1Y_i$L?5fO)}qnop?SD$J7yFF4@vSzWRe_~F_P9Bt> zt68}GtMf~q{Bmmngw(l(Kcjj7Z_G(EccznGk!WaoZrnQF`C*h zcTafLMev%xC%I~fbivm-%{u2VasN9p@0K9<8ngR%)+R@fPu-O6+ZT*k+Ag-(z)HdH z_6@8Q%s)0TE+c?X?i)bh#jZ5NfA;l4DTgLWrMI`s-W2BoXmq)FhiR&v%qO?Xn9 zFabKO%^Ljq?>ZYbd6-6se~`>H@ZKT5_^SMVGaxQNdxO&n3wd<-c%n{waiT{`4Rm`;1A)cJB9r^%@pD1nO_k?5&Jk}rbh_!UNo$*mvBjp9x@r)L% zgm+#(?%}3~9XMZcE z61GlX@4CdhmxdP>5;c_a;37?h^Nknff&667PmP7ELeb$ke@2+-$Iv{#5V>=Wy}72T zc zTPEswu#BJJ&d@#G2tfkHxe)}OKnyp6NUBQoXx>`Ldig7_+@&$$BqZpejY03p3=77a z`NH@4qmKkdnqR4A*>7f(=urpw2SCy1TF!1F$;XKeyQXZEtL7M(}_o`i3vU@k=bg3hR zv#fZVf-vpAGb5nL82tM3XX=$6KuB7i@vo*+<#`2!LT`o|~ zuH=hb2s@NC&#tC=J6%}`3vU{?Z4L3<*o;+WY^kDTZ}C3*0(s?%?YSYoQ4b-}q(~!$ zAvHVDcO|ZM@-@bupDlhP1k^tg%DvC+_M6Bf1~F(1j5^F#BT8*G0(+)vClYw=AC7!WPsz>Po1khhb^tu}i~6e} z8W>MWXcrsb8v44NH__PonA)akulJ^%dg6K=1>Z!Y(jz)m63I*=pVA05nPfPzVu%7+vHi7cF*ym>D2=p%?JNd?2 zW_L9a)rf6M;RZC9eTjv?tRG%Hx{m;2raqmG`&Lkpqf+tfTD3*j?QwVX7e`CVwYbnURd<8BZYA6=~UT&PRwhBPM9sd~ec2f!d; zjxw___;Cc3`wYJ9T_jQ`a{pwN4?0%op(LT*e4jySQOZR1tIZ~J{?QoiuTVXg{a%d0S@+Loa?Odc*WCQM8uyzij!mczOgr17WR)*yF zBh~CNE=Gx`k4u~W4_9v)73CN84G-NV9Rm!hG$J6~AqWUcNQ2TPT?0d>q;yDIbV$Qc ziULxiNSCxQG($g!|9ju-dDi;@<%5gxI@j5I|7y1~8pr!3zxz5QZ(~{8A;}VicU|nC zTXn~GK5(%Nrz6Dx&fpmDHOimav>5R!9vbAt{djF)=?!284=ZgPhjX;49Gik5ag?&*XMgh*8cQo^!#PP>v<;65nq)m1z zqZH&+YoZgPi24lg23U_4&eZNcmh|#*?WBZyM)HZ7$Mjarg$unvsF?Jw@+uqrsky{J3<& zKhc`Du<%tfLz^Sx$=An}J5(QU{~15)g5pAu!H=s=?L8-2p3Zy{zWm`2fGRc6$QgRR z{SCwbW8Z3_!D4K{-#Su+pm6v7L#ml}3!9gi`V13aJ^3d^I(?pz-$?2`dZJQRDmODA zvhXP1JX?^DHEkyMZZP~_rT@odvCH@TU73|Vw1-`%l>cow0`hB6Kr28E?*iyG5sP5S z*;O2);gMT0_sxTC=wzV7AK*H;orW*)_Ad^|<+6|u&)p;M!lA$TAXGY^)&g^i9Bn zDC7Atc?0Cg*qiY|RQ+g{gyjN_x62wU>F+Ec<`pn~G`D2#0^} zj`iaW=jfLhIzzhyI64>O`n;m8NUT7uJ<_EK(GrDWSg=pKKy~S4J+UU1mH9nBAwNP=rL=3f+-u&Cj_4h=7zT zX#4aA#eeuO)*h#e*EGf+7K1e}^-EqreiNcbI9A6VX_4Bn)KF}{RH9ryyk0OQrTQ8A zO8_KIL2eVjT{!wb8@rw=R%k|E@i+Z``kP5)C(EsD=dh!G%z4nK9FWZ( z<}2<8aTwPXiNz{iHn($?W>CE2ay(lgOWarK>qcuWbP|=#cZV_%I6>1x6^g?ZOt0p8zRC4(%oi^8Gm0~^?Li<$}1OM#`Y1)5>Cumb?XHflK`I|gX{c|o3 z8j1`i4MpJh3C6O+cs)E_OM1_Sslx5Tbnu5nc@pCF@pC!+mZnNB?RKaWaw;(bGfeem zupt9IunRqbGZPisVK4jEM6@8LcF)v#Uck;X3ceEn>5Bas@s;_dZhiW5!z6CEepB!b z&>JY2sK-e&Y z-E&7>WD$|W*ZQV2P2=hplUW~HYu?Uuk8WwREz4i~s`1>rQleasl$-QSc5(7s7_R_e zu0~rX)syV9(2JRToFuazGp&W$!Ui&;yY6}V6RK(geFlI;n;`_QJor5pI2nzk^`2~E z^y!Zsv)xlMlG>l|U_QelH|PZ0-2CB(v^{pS6;}2$mxRV=?Im9{`qjuTKkpFK9)n-0 z5z_*9e$aS>cn>-c3Uo8(B~DW2SQx_a;kt~0@?e3pj(+)lIhsm+-R)5(}2ao zm9>22mndGt6w6G)2AgD~9!ulLtg`i_ zU2lZ#a|L8XCm>|uemjykfF0xE<#9FQ%5%&x(tGNn^Mxnb-x}zh+g(d4&R&?fdqE#> zC9!ohlJ_Au&oC#oxmUsZJ5qX`%m=+APww;zO41Fd{p=P!^YYRR0R$HW%;FZm#!sri z;@r<%pUNG!mft!cx2mN!->iy}PHXBf2G)#z&vPkqsjp~o$w5+gi2KiZTQqEVBVggR-+VSL9od0DX!ceufs_^6LVF^Gp#Xk?Fb3sv9+%8h?}J* z;(TFl`&%|H?;7*;G0eK@-HGPdRMzoDm&pAv0u8|F_}DI`dGO6JC{=H|PH# zjDzUgHB#WYmcehCqNlO`N9kfo!*{ zW>p%#tp)oPU1@W~cpGAZ9WN@C+p`s%Z+$ZKiH-IEZ-7E$9RwN9??okx1+U@{C@ywbdXfHM+r9U&>U5&! znH^B+BZT7R;c_z<+R~D(q(Sr5N(-!RL8=S#3uDI{QUcT=P4d#1`$`&@aM61{5A@3O zn0w6gN2l?({_Q|Iqn~u@;3s=^&jX|93zY?F@(=?!2GOc0R+RP?drSbfb1HY@ zCIExaYB*{PYXQjS|G!T_Pfq`m(K4hUS?#^onU|!ljz! ze;u0`(-r9E?A0f|2gr3}3_wgXZV+iHeCrqkAviIlZr)GUoa>#FE19|$#C~dI*EeSf z|I|1s7tfU|;Z2N>`S3Bcj52n$l<`Gb9{~>NnuvffF%$)cyMyl2jZlr43Yx-U97D%1|}_V!?Fk*Xcf~p_)9rGQ03W&Z`luIpSqGCcsH_tyP_w zE{yyrL4my)afka!@|SxPSGvB6ZxB=rPBWiWiac~dGz{;j{)0L23KGuwYw!6n-@mXb zouoSa6?&&XAEg|iPs!yBoojbtodDIWFSd!j+k@Wm4R+rTC&d6xt09A7ZEiYP)#Bvk z>_K#)XmBjH6jnNZo@3RyXx*rBsVCPRB7#Bf-7oR&v|bOxK!hG`U;d1p-8D|^m$T6r zY^&Dc7KhbIVTofL$hC|K;gy`7H=BJbsPA7G;oKQ=@*gwE8N*A5bm;8C-Di%*4gznK|G}iY@@<$?a4I?*sy_g zUl>Gi6HI>Jj_Xvfz8gI-dXla9V(qJ`trq{e&SF*dtDDY^gn;8M@0rh6lP7=wwW+pT zc+gq=CxZnF{cEt@K=3jIH~|@<^tVHm_nDVQrVi+nHY3E6I_F$C?bvltir9)oHMO5= z5)++ zjfs)-wfN6S9iABXI!5`pniT^ej|TM{Pp1!SNLgZ(j!TAFUDjr|~D)a}8Or zV7Z0MarRkOri=S|vd*ps9J*I*0&jnP6T^iOsb^4(eQ$%F-%OOaYB!|HGUQNb9?WuH zV;A+|Srec(gBRRHm~-iX*`0X*;rWO<)Tb8eI>Gu@ z&A*CNUrT$J$v4 z$gslU3rq$gg%_?f>+UT~eJmaNIIMov7Dd)xUurSby-b%Dc*vV<^m2{Hnn;_S5)`a; zvF6P#xFD>O>PeFpDy*{aD=JJ1_2#0!6jZA^(hv5^?n@XNcCiVZaxl~1h#=RYATZE| zUr7~5;hrOGEn+}fU5L5AgKUnvF1@EGIH}-J4Sl&9GW0l#Z2{Xn{i70jq$}|2WTiJ_vYV_b>b000Z9oS$?Z*bD=YhL;EBH)1y1I5PkSQnNFv{7Y`Hb_t$kiM-Jh9A6Q&X` zj`-cV#n}${(G(msSlx(G3AAmubSXvnsUBp(m+X?p54l)-KW8DyiRe82=WPD~FXUE* zeMqXWCVz>|Xyw^RQL=vBI^|ka9vY;k2rr5|7Q_-SL}B-ou(OlPnKI(--veI9XMpX1 z1`%a2&kS^I31HNvpJ}PzV~1z8G7zea)NDLmqMKjXqp7QsDjLeftpOc)!?*{rfixkj z5J1hr7Ib-MABF~lJ3q{y09)b)E8ms1-C>wtjmC#>3ts+rmn*qC5A>{=Vjm>L0vtPX ze2;U@c~zc24DENN3$|}P<3h@=yK_vt%o?Rs^!T!sH4soza%1c@NoEK71j3K1i|(5% zv`13!1cF{CLa}F!vL16U+l+Tf!rL|n(_z(A{#EYEE%oxekJhv&91GFyMsGT4VXp*V zEXqVFIu9Ebhc*Ipnj?KYeVZW${*E2dQj6u!+V7*1>JOeh2eR1ux+l4Z`x}g>2PU|a zS=)RR2+ySF6o)m}Ph`#JGNBLuo0@eSto>aT7jw`wjVGg?m; z&CKU@31T?N5r?^v4cXCWjOMVwi)#>k=r-d-tQ_hgW`&8CnzQc?VC(q$t z2ND(15_j#V<6exgiCw5jKindN3&s?fEalyFf^4~)8Np@2v61BwCES+JawPN-NRmKO z1j32+TH+=5`D;#T$pIa5=1{c44Di;%IU!|}vS@a_0goav4MU>xBVc7UT=_z9mD5Dx zO>j8(VHK&YI;DU2m}A*+UavOWTp=sW#*Oc!bW+0s zBUr0;S+BBo?AR{JeN@07aWOG^b`#!L%=UrArQn`1-#LR2d=4MdExsnHiD~~Z)7vrZ z>_hgC`M}S>suPJ021SuEzY6#EiSHbDfdhaOgV4*>vG}t7)LS>7Tb9=A4(n0N4yj~~ z20$`G0A^qSdT$M&CbTKm$)NjwZ8jP1Xwi3{k3!RH=?$Sqf$j1?uj!|eU}dO??W>y% z$zO{7&p6GNV+Faa<<+&8g>=6Y2D& zEi9-PI>DQ7^K0gK4lR+x)i#uP4?yf~kWS?71PZLyO}&b0q>Ce< zlk&VsA7Acws}s4P=Wd=i>D3O6LiZ?r{Q8Wn>{>d_-;O-@Le$zb?7hp?F7ddAz~VSQ~CsoON#Cmm=P7f)Fmcj6_`me;1 zpAlE1P*^9V)cmb5)AP@Ox$Cnk9_V7LuVX;Km)#&?jIKnlPKrLXk!N<$eCPCL>dQ1D zcXS03AG2!E zukP<6ZvqwYQk1BdDUX$_$C|9pXc;)DQ_ z#n!ED6m(8W%yNBI$mp5R$+q=KSw>dp+|=cvbLqbhL*OSY10xyMt-S0Oa{Mnw;d#sm z7QB`RUX=f2RrdrI=y})XE@<8}Pib#KgF48~6Ez6DbH{-0!?^AU%81q{F7wW?ds#=Q zteNg8ll?YX*TQd5S^0w`@ea!48N%PYnVzittK!k1GaPlU{z(Dk+^TAT+`*o@U8zS1 zd?=eFyX#+XsK(C?nW!#Yg@ZZ{C4c?$C})2C^Kh0mCmS*|W}iUEJ^JU|l`v$hM`>eOWfef&-#rk3%PVUM+bm{7yUPaz=h% zoH~7vk$kPkmv?v*FadtkRVPkXK9 z>Q*9)hd+54g%x^n+M<4TC^WMS#cjr?9o+xU<5(0RP|}J~q7Upe2vcvh7{RW%?~H-$ zMa_hc;a<2(#u7_c{4eGji3|vmPxQaI6%A=Av>$Ulv-aVv6Z%!Js z@U{cmcxsuN&E+?v#s@7e1|FR@g}jzEo8vflmv@h6*i9iN9TtJ)4mK)J=%;ZvJyynI z(*^?Ck!vK7C0*#O^a4kF((3^(cO4(frmW)HX(8kN$qbxKR^7WGbW!0zM{qv+uXV>p z7Uf(1MtyOsya1+W(zx~f9tN4ODWD(i#9f3Xi>WSw@ z`+iJp)y|034Ax#5P$2@TCqM$ioI8M*?LVLzinFKVc(~atnWW9h@Znun^;f&F$VLs# z5cC=M;O3}g$LW6^F|8lc7w6P-Ith8i zU5qhEqM&}^4Vhw$4mDwBV{q{V#)H`}Z7*^^{p>2`xfzj7r*d;~EJ#CTD`8{7D!C!| z3h70->rV(j_qG0!eH9}}UY+Mu_13fI&#PeZ;`bnqa`E;CszG_mp3_@bnaQ{7&*~|^ z@BpH-yVW}iM-wa&eOXWM1}KKU_4ECH^@P5z=yOdmkzy6_5dP!JfS+l5j(R+NR;=-{ zO%dK5?4$?euYyto)Q0MS0yq({@y<$n*y?YJX1BWVZ4ywn73F2HFBf(%)O8s6x;C+J zup)%6mXIJ2I?LX;NY=Ia8$P3AS$7-pgj+6GFyKZn-On6HZb9eC{(_6Oq?LlFRvMGC z*XGAFy8RKmU7`{@BG)06gmeB>!fFqqU&TbN+NJ6BS*`g*`lLxOPR;xDiB7W6zx!A$ z=6owRraPGirV0iFRzn4C0cn8(Q6e(8z#+;817u?-Hto%Wg&M*{`A?Y_uO7L`=8yjU z`Ka=3XUt&9CFK{UH$*tV;a|L?CmPf^rXKw}5J*mNZ|!Jcf0SVe(zh)31pcy=J-70i zZ@!=k=|L6cd1Yf=tPgKRLr`L5-P##G3J9+48Y52?HM5Gxr zd5X1TY^pqPTJhr-8i-YabK?h=1=taXJO7`m9;htAk9nP((de!YbAg8+4q3?j(av4q zEAAa|{B+N=^*Eoki121B)4O4p+MN*!2ypR`1!@4fgU$^iCl78}?5P3$#Su@Jph0`{ z1wtS50*{=T_w5^%7hKnLi@VikK?P*Zl6RKzoZ^4DD6V-c-snDsd{er-30!?~1Xc6% zN-GHc3@)kze$cuo`?fbCxuSiX!x7V7;g=PX#3WU<1T5mN}>d!L$ zOu0@)i3`su5aUlz1TFYaoB#$yAb$a<6F7YZw_S_V9PFCFMKbzxk>yxJIW{TMBtM8sV$_u@XVv>wt<-iR3;+`? z*qbwA(203(O;vro7KJ&s@h@3dplA8T7&(oPZ?x=5srMF!U_vnVxQ4GnTp9fx5<=gm z=E@TMkG2@8Bx;LS^fGNv#2sh61Y;I@TLk5dkAx~Z(?}eCtB&4f+CET$CK!Ya-Ir;H z?l&wzSPEAU1NM%@VeijLInwZl6mnyYDee!rb!XNhutKVSQS-Tlht@BIefCaB?xL&V zaiPRWtrWNP@Aia@x;HA=g`*FAF1)Tilg$F3?q_MG<;HKoJ?)CD5 zaOIya>Jzb#%M#xyhdi5s8nvKYAR7#Xyl#}cM$dmR{kX_b`u#QDM8F?!zXJzN$P7;+ zgo;86R=UXux(w-FmgDb1y*bLT$@)x{c4{@KRBC&b;4PGBTP;fFn6Z^z8%}_8i^rfM z75nGZ1^xb43qY_>yXAi941LS1|O{NW**d!*!+#or`OFR9Bgx3Z&ft!UGvm>8zb(kb;>=6js-%ZL*6=h(+Yni{Ei? zE}kvsz5nu6fo#1K6_B{22aC}mX9;1IRk++`JPW#Zn7#Y$;~Pn(=U6mvjxbBJC=Bv$ zaLtKt9OMi2)DqgwS0+^ye5iSR2D#`tSX1SY>QQt&ht4LYC2(8fccN$l!IqC~AdU{Wz#=y$nA$NQl_Jd9F-xs*w+Rqzr zSqnqW(-5>w9yPtc_|VOk8+nj0+5N$s?voY`ktz*JDW03MmGqm(I zmhWf;Sw0XP+wKV@UimxdfG|({J^IB%CNvOS1;Phk%^bhsx9{NjVMhLsIYUI*U1na3 zcGprnf`n4G!Ey_kh9T>l>>>n_RVrD*-IEA+{sp;AU$PlCK2ALH+?R8qlWJzt`Mb~; zl{>CFm~EO6Z{Z>}nt`()go&>9U_vt`kkT$9Kg@@s>Q(zq@Hz=Y1QjkF^QPZxWL-R1 z&Ly)|ZF&2m?HT?uWgi|ZI624*+*-V&-5V6S_r8<5aLGZ4NjKlG`&n?}dy5;& ziZ)jat=Kh7 z5e#zGGCG$_cI8jqJ$PYW_*uzAq9^Y2IMihnXX!KX-%REl?Ce!Gj=Ev@M2x0QYA*Dk zCz;QtYV7aLsrq$a3`k;?L~G?MI|B1r_^(}}_EXy!e$Aqi2Yrn=^LP$1J2{Ffc%3<< z@Kac$3x@1q=m;j-f}46}sn7aSvIUz=*{nB}ofG8-vvjhouD2nncpx% z?00`bc(D;{#4Ne(z==<*+0%ylD~Z?zF={4Yj~=q%hS$lWAq!0m z?~f`QM>YbX{U@!>KaQDZ$`r{9SY;s!@G$h8$Uz`{aW<2+X;+McDn1U_y1m(68e~~n z{=Jc4p7BPS4s^Zqy+K6(qCI)EYai(fPtXqA8q}_c1l!T~TLPTsJlM#@jBJcnO~J2u0>-%YqSmJ1G~^!Rg~GpdOm$&#sd-JpUT1GM`v$^w zhYoy(N{G`ZH-(|HmeJ~kBuQdr;%BW~eQr}~DJj0)yr>&&Rl837SWaDXi_*vjz3vs2 ze2#E@0%NJt=PEIsb-+%35glUIm4Y(PPLrVcJy9b!l1hvTew*Z{PLZ_zvJ*Dm+6R(a zaY2N7nobG~?|wXKPx}LFtD;vOx)|N>8g+{w#@=7uk(Ts%_;*6A9(C|DKKTioqN+Fb zQ2g%dNpls|nY5_o-;!io{%8)UiAh!=wWV1Hr1!!2X?@i!9}Ex5-PF*~5a?FW8Fcx0 zUZTDA^XP88q8jyWGA+&{%*OFj#pRFdheO2OVa z!{2E%upT8S!tvvYVyxK`W;A%p*qv|bk7`C1%fX?65X?qIBk&B&>F_?lN}@J^no|g0*^uPDrdI z#(igMA-fwSD_@-xNvZ+KzTCeo@`bDAxTAI2UId{g*%kCo{94b};evcrB}VE|>WVqR zA~&NlxbTwAdF227dLd;02-yXl`GW-s?BXrB!cINzhm92$R*Z`()e*o>W+l-peTl;_0!j(;Jr{DYMZ<1YiduTBvxsnsq~%Iu1%iM6QPQcqr76+|);ePc9DWA-Yr zS=HxQn@M^<$tIfDYn@17`B!CR7;rI_jS*0FjU1CK^7`+|@Z2VkN=ItP5r$N!Y?c3E zYi}rjP)e?^6^E?q&ev=(_g%unTy!)0&|i6Y-2CDrg%ck%kgkS!aN9Yd*(L>NQibkS zI9|*M4qAix62wE|ENjl8X&%#C_4TlelS_5yGgr~ld%bsiu|~)E74gy?)dxDJz}>EO zpQR?sF2gY3^1rFP&)s0LiFWI}>(w0Y><<8s+isax0nS1tX&N#@|8nrzh94$6=##r~ zfw7gf8{=>wuJ_30>5*LPTc@_i|4%7X!40cUM9UdHEsIvJJdmG3vtf8RGFtxmF?PzY?f3JeN!rPZhl4OTGbod-GiI*_TB)R7>c`X^3>ktF zC#Eh)4bvTzt-9kKrUZM2oq{+24?@d_nK_KLZ`N_*`?-Z(@jB2CeQGJ+hhjbGa!NLB z$6t;oRml0wtT%)Sa!M6GY9QZmGL}jS>hWQgZs>HCw$nmKpjkoCU37cm+zn4@+FY3O z*Ez_+;=#WhTRM7Cd`Robdi!PD&3X?0w~y%p<8{M9pb4*Ya)iN`DIpM8LGRde)=oGn z)5D#6Mka=2IjN1bqN`Hnb>;trcpe3SehHOTm9n5S=#vz3M5r)+^jShKCb^W7wdDx6 z&n1u zI+{|mrt!#U1OHQ8*05PWw2FLTMgR^J?e#>6B4+WCUrW~yMdm-aD)uE3m8mEwcMB|M zSBf^<<}r&uKKut130;O*duFWHIkmP!%ywU23u-mHxYLWxdXXQHw*L6r>{#~cwN|sg z73WuyM$2U(yl(*H2OS$T6FL|Ro`?k|(To2gn@o1ek={5Ud1bR}-O)=CvX|bv2m0&x z&GMI#Bi3_r!7`^flb$Y_1h<6@ayx_hBEO+fDX}G1W@aMx8fh|r9cU;FEZXsP5PR{i zs}AYi9+y)b>b2^SjI$O;a@MIB^1x$ac<18HSwjN;a+7no%8YsF(ZGH6yEIgSdG_Gl zG**d02H+dy`;(lXBJm($a@dWl+C&8NG#f0D#4C#Gqg$-z+85>z zi0+OLR0!C-r!J=9OIySIrnE_u7We4A)mkIcJ6oaY6KgPkPd&SuhOe}8Fq!;3E=GlV zrUwF52a~lX^Sk>m8i-PFy831jU7tiO0F=qqN0T4rwau?CMo>8R=+*qnlNTM>Xc1>j z4n|<2xUe8W+d04dYc>kH9GFrrCIeLQ{DBKtP>Y@It6Wn}K2gec(fQszN~0-)A|je3z$? zQBi$KkH4os2Ul(w>&jmjKA4q!4sc2g*wwTnKG}_S<~$dKziNKJmn-S+MJuMDb0>%S zocUc388i>1bDY&Ai5XC#zm|Bgh3pmW*0{T`H4F}gjHZ)I16`)BizkZAj%^l7h^!J5clnDCam z{#@ZJPs$vD`CY0xw8!67=T`P*@s{H|r8iHu2FQ523`5Ye81O_qFqMs9p!pIKk&p%I zWjx|V&!|72ZX?MSW?+r~{h%N!*CGo2^n4&_H)OU*&#~!@&HvLh3C742 zE^y0*p_kc{trt|G@p56)osS8>z@%kdHn{JSbudF2hx zpmfb15Xc9)VQ|>IL{Cu&vWnLGv~>fhyKbJ*8C+8w@~PKUrH*ppRw<`ap=w0-WX}Q~ zE;)i{yZ4vzhM-&riIu&kmF&=l@m=-1?{EOw_x#`c#u$W-0==Wn7}~5%zBZ83=kZLm zywkGglDM}OPEm1R0d4=iM8T-(B*{4D=5(wVcK<*CKfb-qj5XMA*VnQy2G!$KeTCm+ z-a*Dp(8hATe|^(Kd#__LeuCC&_~aD!ctxk|Z{@v%O~Y=^ktaLpG*qCwh8&!CSgFHB+EfEp3pn zPAKr8cCiDTKOF!n?9I=Gvq2d@bBm9&Qg4l-3xLVkEUD6v%*MBTt@p&V~+S2i;uf&N*IPf*|;k(>+gN?f+ zbv6=v&u>eqMb(im*DO?@Vym)5R^Z}!;DK0Gwq@_e>5r_3rjb}T>^F-YNJ41AVh5Nt zGO*;538A-l-r_~!C_xk!A|npg_?S$VXkG*(SX6t{EP=A%u2asc?d+sMQMXM_~N1WHTh_r{dMgQuW{UM56Fy=1?S^wPA6 z_TufyV}B0Vx1_CjC}@4vlUE8}AaK5)e@=FNlc-SUY710|FW*4!aZ&b}C5ERjNRxr? zxcD~|OHhYLGoJk1#VSu-3Krvo%0A)A#n%{mBmLs`Y{3GXpkvz}^3*Nku zT?nyWC!b6xVQu>=5cIpLvw1OVJH_0bNHFLi?Wo!HRjt9inC58^#G#REv%d6K=NrLd z8-v-1DgSCrdHd)DAKVy0L^T7A6!!1mq9N}}cYM;L9un1bUegd8ALu|fn3YWPQT-K$ ztR4R{R!(CZbBdI2r<`0?HYa?D*NzK&V1UlweC_tS9R~Axc-hzsJXyR&-he%sySJU4 zX=EZev`9$jX}EI>1nG^}`3{?mQ{CI>Y1lxm$&U+Pzr|U`?SS$EYX;lyMLrMzA7l2EJ01B=0p5{cD!2j#Jx@WnS3q!;@f@&$@~Gr;L87;j^Ee^TX|cc4s?xcxL0A3{!QHe{&&e`ig88OwznsJ`v}%4+CODr;6r@!~ zE&KL>`xllg!E%$bD)dlbD-|cTR%Be05vGE?6Of3ke(L)-kZ8Y5z&s*r zFd#-2L0mkXACh&rpm@{*`8DMRJOB5{$iI%<;4M$YDmNi)SA&2fcPRN~jbdsMgWh9t z*wh3ucm>8aan$SalO1W8gv@ta4P{AZty&roPx&zWsA-h6Hx>PYhI=6jTYIYiFu?0a zr}tQm^^sL!rK(*t2c_a{dIvf6F-q3#XzO*#Rnu09d)V}p;Fp0K-1!qOXV8jYbJmcOUB%MwW{G%eBZr8gL-_ZPXP@&N@WFa8gg*OK2 z9m;%YxAElErNfh6()eY;#XKEjRlJZ8wWKV@B|$LcQrLEgf%l$wr^d&$jHing>Uggg zA!k3d*xn8y6Th=p(20Bt%PBH=j8?{2##1Qj-RW6}z-jtG2yL6f<&(Q!Mi2|Ip0D?I z0tT6CNI|T{HpNTX&p#*wp1V)gHw)iCu$(+t0>C{$)X3z)ISoFICC*sylg9o|Q;%It zXHmCWN+U&2VBhZ6OLVoS>h+U8*T;YA(T@=#$%&(I`8x%2 zoF{X|`s)?B9$$NC?VTrD!bHig9i;9m(XjIao|t<~#FQTAiZ@oLy(27Vp^VaCkdje@ z(*_^r7Id{yAj79-ZZL9EDUyq|qMCbFlC9fu91I+cK7I|qF%Qro08D`Vzx3+G`o2l3 zXZOazRLF)P`2NmIotHWb+I=9&MMaeS)C`XZ`4%@x`T*-y$KGi6vP_@as6B<=5L;=w zhA%sIs~-PuIv-GH_^LjP@{kl5P{+DQeL_*sFv54xtPK zw6}{i+X0Tz)4=$tNI~zbY1n7ekgVQuR0LkzBI;ZtrE$#HQZJbJr2!k|2d*t;b+MRa zeb?~JXt@o)m-h3XQ0VgDdV3SsQPlCV_qdrxW!v5{*iZS#$|WD)!gd5BWcN5QyP-

8O{{6`|(Wc>xZ3ia*?q-GS_Iwgtuog_klzqk_Pem~NBC>)a!hitVs`rjH8 zd6u*B+I9W(Bz)*{fAYj-F@}7q;4Wwpy8`<2?T#|oL=3QNulftHh91QkHy&1n1~GCQ zH+BYO1)zlr6fi=hNeHGFL=Q>?kZq^MmiE<-?8lg)wTdC)n*z$0XPuM=@#de@e`33H zK)&Y<2u(7v>S3GW{__+HDs|woGzwS@hq((PP#Oey2)h_GSUAfw@Giv}B@YSW)dwYf zm<%#SsMvETebsM^YV=m=RZ8$n#m%#snA^!|C(lI}7b{*){Ilml-!2v{3C(b`DX%Umd&mpXM zO`{tMCYZM8QQ{i38{WGccF7UWC;~YMwrX^q2q&)m%3{+2 z61#|fZ84EkA$Jcb_kjrBBkY+eZqh7B{GbXe*&!l~xe1NGpepxS`+>TSvJAr1)t(xD zkV+W*O-{wGtXa8^Nu5KqtkM2W(>Qt+Wum%U%x?QL2Yw(m@{=R8pQ_FOlYXTB=q()6 z7CBpYh0t7vSc>4O6G6Yo>@+V)8Di<7=P4$aw2y2#-ASO|33gpt8CK7!I?sn1h`=qz z_-AWHE1*Dnx1U8RbV5DEB6VeSIq=9da;SkjY4i<$$<{^h>NDt0vZ2`ib%?C&uBKcK z=S3b_*Un>L-@w%unG>^TPX-lSU8i;6mq3~39(5(z)QiO!j$l7ETFIW zJUk4+?E0b2Kwg#&Mvok)vF{H)q7=@x#CTiRMm78Nt5gMQHtBvh4qNp4aQC57p~1aH zqZJ%?<54#E)6yQc|0c!qpw`g;dNKdw(wzC=DC*>ciw=Nz0ObRSaWI;1;yv!ZVASd< z&M=V0busp@)vK1B;!una%cbFKU!HqAI9P>ZzBl>2`?L+=+en^Zg=YX1zL(B?qUYBh z270!9@fwD)BOEE}Eqr>^x$yGaf$KCR-TKo02u3P-NnUx<3avg> zL=Q&T6dhmFJrvEanzu}0@kja(Aw2pmKb51uKG4-?igh~N3i>wlR^{(=O?YXWN`$s! z9rK6q+qOYn7Z`B7(`A9I087~-HpP2DXh*Ld6_k164pWN#>$TCypa{n*G9Y--2Q~l zwQoL+kWR4BQ!IZpH(`229=LVkQW$uNT$w~$arJyC z0IvPcm6(Te3de>?Kteh%?8wLm4FEGJNs$z1ckw4Q+)z$lSTua-34D*1V$}%HSg?w8 zZ1;Z%f>&5mgA3=(pTRKlvd}RuVT8{Q{qU24!- zqe1NWJEK?N`;&$y-t1MMOi%?e_Jv$S1rQp&TEk-fFa(NRhbeh>tq`yc4I!Jd>uZoTNK0706#4uLXyH3^S zas1ZNlDNr{VL$Du`!J4l)Tt${E)yOP-zTR-#>gG*%?FQkpfb|oXkQ@ZT9NPN6 zDnOVw&gPTs`7RX681HirC1z2vLqN8@qy9wn&Rju%qgy#*`pqSM7W_n2);tb@*;9aZ zzDg2)fpG1#C=IY@8&iNi{HiOmm%Tqe@am6=Rbn{|l4Tpw=`Cn#5SO!Hfim6^bQz zmyfqNrnU(&FQ}A&P;CHhCBzDAuW+$3p!}CGXCbt zt>C{}Vhi9M9UrYJtEO1!S5}nrhLpBucWypKsx`=sFDM1QbGRwX{uf95I#pokJkFnV z;sHUbY!CJ>)R`^k)T+g255k47-1G|`xGD`B#<~@)x>*7xoe``4Jc+Da#qmu-kSeEilsEIhc#9VYGcNEsU32&csf<6e}t=Bc2BDCT;Oj zJ7c)#O7$+RH%g;(eJb3|A#OTz=Z2WCV(756@H%?hv>1E(rhO(DDS;&@?I`WM7zz=F zU$ZN#F&DSY#Dv=0{@WxK`+TB@Zbf^|heC95adrY22ndqsEo_7iCuNvPg^Ntdy39kN1$_%sqaw> z=!0eNdsHv(_kp~j0qYa5#pV_27_2ZK7{3*YjIHzvYF}u7QwM`0y@sN|;ix z2MkhO+4lE`h{vA>^ND{rZox#eBe8jOL!3imEW@o01)C-lpPFo4pQ^zUjf2bPiEgsH z;KygJP&!!h>9fS{rNuc!&hZ?58`Yl^EV@+hMpy`b5a*$4%d+Wf>|9kUm*%hF5>hwz zEXa`RXLxMuqS!lq>1!eYc z@+J&Gum2>jpL^b?K!f@Q;w<3!q>*HP!a!QtuyF-%UX$E>I&3U^hh&%CqAq>f`^QUn z5d7*DeW`FSc8ftR2Q%>?Z?ofab;#PA-}B|pyJyE_`T{|cnNomMoP-R&4lO)lxiA=DgRyKp9 zWATJxx&FHmW^QR-FoDY2?BD#dYy0c~?jQW8d_musk&(oI7hSUTPwjBo;s*&LtqsJy zT+CHDTYI%^oF>D2>u^~66P`sHL@C~-WXg`2l$+r%JEv-nAm5YGgp3geFQi2niJjuk zSjHdB9jOlYfgE;6yX+PVFTeS2ThG!byZSY4TH2wz?c4z)JqxKlqD|EKE+e+Dn>$1K z-%b`Z&yy_=?(3JfpVwU_yWNyDf({Y?L>tJcDq*SiO?yhnS31$L1MxY3pmc_&HzpZ>NLE%EiW_ zx#@ME+X-zdOL}aR&upzd2)Xo3!kyDS1syCI8c2+`r%7kk1M`1+$>8bWwb!SnLp(2i z#YV4MvfXCgLSTyPA)XPKzQ&=_RD7p>#gtUMa7PoQ2ev(6_~ zvO=_pv*N%*Z=OMujgHR4>HK@F179~MhG#pVtpo~4FAJJt<9E>5KEDI(qzkNP0zx!6 zSn#Ol8R=5Wl$ip#_|roH(~F5EL{bJ}ELkv_BX1B`?)mzTFArZKnA*fKbNqH=;=UtK z*AC`ufMM%8?;rEQo_K*~kKxU0?W0PO;0?SZ*`-Ty_g#DOpqz70sL}GJGb^=VI`K`> zGr-=UgQF>H9syr*;0f1AfCQ%-Ka*nFt_<2us= z$-3S)y#55y_OYMg8Ec8D%7$f0Dxs|*;hhVog=C0EWta-kxx~icVZYu7YOK9&%YjWy z@mm+W3R?b??E7#@5dJY*VnQQ0;f!G$;+xF2=dx+pJZr=ORAAs7zq zU-hr?CWjV(x3<+Crt;j1Rs^aWN5`15q2aND{Hl}dnUwCx z&Ez(@S2I7_H{QFjys28+x7oj-;++qLSviEjsyW)%C*DT6Yur8jl<0nw@oBNTI;l~M zQni#$k9ebMou2IbDmFXdoKJc(J>}a-agHc^V>9r32eNu$C&zn!?$9E4ukjwSDwq2Sj8(g(?-uWkGj|&4{fkGq`XJ28nMup@)itbaTpHWR_7`H`r#%@=@L%YbFTcV&P6BAa2&V+q>R^ zp7o$Z7`kQM1I?J-I=m)%FGLc)+9fla7lP;#F(K{o6Vtgx&;#a5#u#w@vHaqcb3gNGtyIDUIcOe)gKAIhl$loX+HYx$6&KQ>R!`3(rEp zFQ4p--WvwZQb?i=^7Ad}Wc=!%;hO?~mw5EfWZM1%7@z(2d2lC?AkiJ>*>DrB7%W6D zAGf|fEAA-7U7Zx*wfQluvJRh7EFsuuR!409#iZ9~I14xPg0ZyhGef1MatvoFTq`_0 zcf~M~3(@x>F-5HH3zuj0#z|Jm|BPP=wda{XxlYkUV>zq}Gw<*25oO^O)pYG5*&5|& zt!l+)GMYVEw3A&!YrkZk6Xq(bWVWn1*{83z#E;dg9=n9criZ^>u1LkZkJZ??RN8+x z<(&*XNUGB1(%rCe{gColyqY8Uh`1Ht!U$3@#Aj(Xit8~0bAVJS_3DxEB_ zd^NDmDkV5D5%4#b=rAEEU!g=N^jh2E1wG+yYeX{e$aRmdf4^}sci;cHnP9Rlr~R^K zllSY;7+^^MG6dW0|jknC&)UBSwIbzb{0q{G23(rHmh}5v_74&r zbY)_~O>{h@<2E+5KP``&cfa$jQ(1FV*2kgViL2dF65TBz-}Mz$8U7Whewpx`c$ZUc zRG5ri?wWC$Qb@wENQHdShntsgV4Lbq7G}y`u5O1)ke!6ggBNa=Y{K^jXmJ)usdZH$ zb>AW1Ew`ZY8PINQ`0H{<>LnFf$sBaW_IdQ`Mc45KPWtzig7Q^K0W%-y;rwk{3DD9} z(#9&IA@ViQol_AJOR*pP_C5$DzT1Y2WvjR;&`ESz z?~C7LyG5&fYwl0~dJkJ~raV*gDJ4q2G>GeZz0|enR6l0Kv%xZV)u!6`{`@yW;e20TMT#Xd`=Pe+lqSo-kd6y;xE9E&sifUF~QM0-I7s&6p4I{)fMor1I^}ga%4n+jDMAcEy+e z6#$oW)8i%f)swW-*5kpy-8~iCr+26rkUO}$e{nFx8N6i)yedo)(8_Ij<`wu#CfTI@Nx^tw{9 zOkd~tsz0UnGW|V>y8c>ya7z{qhDwyEoJEpkg0T@2k?f@ZiTZ?X$!|0A-K8a|feq5xLn`SCT8dG8;$|wuga?zIV?h4jT)KP0T_$7R zv(2q3ziQ|-9DP;eSQ<;dIQJ>08$UwyEobWf&>mM|=2?{dLi6`@dW$wk z4*q6rtW`^<)sF-qj;4-%qX+wM$toGGoFecy(zy7qo8xl$n&I#M-ke~_={Ry+p2B~c zOk}5^vXmCEDy6+a9=oUpQ*OBFkZb>()LF#3KV)PcucO)EZo%JI_RJ?wIWElZUybLX zUEn(e;3v)<1J3c-2>Yvo zZ;<;XZ4cmobx27Khie!HkLK+HG-3fC1F9g17mls1cI<$eP_B_KpcMYEqxC?pr|x$1 zYla5b_M3Zs3K2u==*fGBllk4fdhh*#ssp+^LG%C+WDxHLHBaTr?0B&H41eMU?wK^5 zERZhSNmvQHfWSDz6&p3Usn(V44=eG_iO1uJ`uNxBV(q@8Qr8gfJ$jN&6GVkSI}2CO z2=~3pdUZz2DY0MAHqisT^b6jKrDp4J?U@~0OLNZ#XL30=l|fU;=KwLo4lO+nIc`TyJtl zxShL`Bawp__9O&G%$tm#@6S#@maJZ@=vTt00$V!s`Bpq-yGxaF-PIxm{~EoL%C0ce z%XEFv*D(uCPO|As>lC|1n4xnFNpR^HM~Nw!B|Xtt5o8;u}A{2o*8EnUt(E6=iKX!yG=U zM7Z?u-%K-yOC*y&dY`xZ;gX z5xI$)5xQ!;F)F>KXZ00EAcNVKbT}jqIt+CegYJK?EqjA{dmoDUy`A3oy2Z1D=00 zG}wtaykyuxW}w}37ZkQLkbAQm{IE=la) zs{{|o!}H0$jke0$aQE>>OgGuEH~6kw0TcMD1^;mj=wdS2M`^mF`0!AgfQpMC9aYY# zu&jK_1!7HyHOA5`vGywCt0ZX+mINk^cgSd!p;>B}NPm)c<`z|U`CUKAm}gg}Wx?NIS*iJODC@)b#DSe#@TBXXsT!dc{WP$I!R*#K{bzWT}O6 z+P+1p>we@*xcMc6TAZ!6$ArFNd8T!D!ZCeyTXO#dbzJJuA zpC3tDzFkO@;!hzCiNb$ch!0xH*OP4yc^vO=efNGN)<{%Z(yBAg*A6M=Q$1tZP$l_d z+Fh-Kzj6aLBZ?EYQuv-)&Fo^>*cDM5)KcND3dt}~U87!K8k@aLq}*K#HmrR7B7-9F ztDd5X7r`EmAKdCtbD-Ie8zIa%eE{dfa?W2gAaS5AYOx=~p8GKtxj<}vGe0D!>X=J< zoFXY%sL^fe+6S+qXYW#Sl4wA5j99yv9r9DCj6>1#Vm3w0m8%q9CppL+St+5-Xt_k5 z##dB)f?T|yh^`&#*z{rgbmF+U2IV(D7yfk2uHv+D{7R&7Pfj=O4(+2k6Of2h7dV}8 zAaJPse?)zEG#p>t_v+me1kt-hv?O|q5<-ONC2F+j8@;SvgQ!s=x`+4X z^zOQV+f6p}PcGaJFA4q*p))pt*qs7%Kt!eTbXpb2G3sgEnuS>}U6BQzVy&N=qpThd z0C@P@+shgNgLc%TItH<*e9<_O@7T`4BT{gLRm zhW{Fn-32rY)FI=mOKbbpereeU@@i*Loig+$WmonyJ^ZVS-qKIaE-4*1fHwfO122Fq zvfDBK#*9GwiUt2bLghb=3RlaQ|Gi=%oEi$0l0b_EEdRCxdfZ9UA~KEOREc(bL}Oam zIDgU17ei~^KON&#@w!WOr6d-_)hL6U9Ee`@_q&mv(v%A$3`ZRXNC2%-vRP%F{is_N z+inIWd@`l`=@F%VU_7&xAK&hCr=0L{z}^nXNv+K38be*mSY3}ds8k&OlxvsKY*NCf zRZ`vCs6j8LmS`%G_^4#CjhA@VbAZ+o5X^7VxgK(o0_-fcqxE8TBOT#dvyFt+g`Ax~ z$vGbk7$#3RJojU=AnASB??ooPBCYLP4>!fhkC@;nzs)n=lFXdBri{JVw7u;&qTFhE zX2yJvCC0xJ|K6Xs91jOfu0?`>Je`H^LS8m;M$qvx#VDYlQm69~<0S+-?HMHSH4K4t zb_W>oI{=^0+Y$7XCmtM>pWnTLuWbQktf4A_UQsG6-vmscolRYPNQ(Rg1~viBJ)4jv z7p#$EHmkXB?K9hk>Wbm5rC2d_X@{i#QDf7vMGbi~D{hbe8e$_wB~Ox>ztw6J$L~qLdYr zOdIaWJ3Rd;yxy+3l*Rrd4P2@ozsDa0n3*#Ib1)Be)3%d7$Ki$NP(jAk@@Z)}yat82 zoCPi6z58=Sbrd#09Y9@3YJzg7{J{oRSwjd_T^CRWrblD8i=0rHUk&Xzg7NZ8$ITGb zQwYYd^ao2}jOM*hekb0yLs$lVec%q>bnokwqLTuyb~3{d?Z@(@5eU^<$An9(Yh2)5N<3)GyXvr;Zb#G*tu#>qi#|%E84_`k2XBA(sqehwcfSr0`0eLmg)jHm> zG@L)tirArCMeCdz$JZB((i()^Ipg*!c=zxthfJUy<(`T9w-`Z7>}@xXcQ&S;Oyd^@ zIr8-nT&#oiw-qh?9B9TbjDJQ>O~Kw(UlTo&EJzWAS1(SKm2F1V<9K#G+QWPIxkmWW z^;-D+kD<+;5B*pJaNp04wW2Tt0`>8-uvytAF4(;BhG+;OG~UB(0~HUcm3jB3j2)!l z?b%Wv|97*r?K=LI?d+D-Nwi*iycDQ+fT;}>+IU;`QMhMSOfogRTp4-?K2y^-gNJ#< z0UU$=y*12yrXY!vyQ z@LQmmw4UG1`+4IzxgflV78PY8AF`0H9=+PVkYUCF4AbRBaZdHj@m%g552kj}g2=*xh&WsvxQC;DW}`fe1n zzX`>i&X;uq6S+AYrys^$5SvNG5X^G6Kt+dkWIROjY0je_+UQTVL&>2X#i8m_r*h{p zCXp0*MdnV2%VIU`k?p78KDWf!Ch+PgLT2J6kwy^%t8fZjpl>CE^HEn^hpWTO?1H+_ z?rz#7jT3RTN186{CFI~3*~rLsf0g{2_}cwg4BJ8M;jxJdDf=fKb!mB9Qam!ri3u^^ zDyZd9hy61t;=|zVAh`jKf*yLlN>~L$Hm-iYWoMO#9N<1H|#7kTJpiZQ*0mE=o#v zknd4d`6TR)4crE@Z_=xGp+Q|wDQbCIoAY&;r0CBT6W(ErL_K_EE3cQQcb^Xuh%Wwm za{1{D*x=)VH-9+VKffX+8WZ)I>5I5LFiL~E-JaDEGsFZ3I~l5m*ZiOBUaQHp!!WIYU!GP#z=9T@WT;0W}qdXl!^ZZ@!?i9I_?t z>_3(^E>FRxBJ1XptK`A6V6rRlt}OWbM*BQF<}*Gt_g%73GUv+-5X>W0c#J^FEWG}2 z1P9q69X=*QAB1Y=4DDja5QAV!$2tGT?f z)cVL1lcnsO2hUnw4A%!PK8mkr?|&aSx`Og&qp!dsDYe?%27`ycByFAX0lHz?uoZLC z-ajMA#^cb~6&IQ`BdF@6)5#}4ltxFL>fs%-ymD8O@P45Z`!OyH`MTMk{sKG<*U5#= zMB_mtYmp{(>E4o;9IQ=-0l!-khO_Y>?6Ko>LFnl|kJ!F8j|x8RagwS3 zcH&!gr9Ts4|E7Otcvw_Rd_Y&@uuHEYc+COzXy^X8K<`R0$6UWUzSu!tp*xLr_dd~a zlPx-oG&u<7kUBBd+y`3?0y!VnzS2ya45>rNl01kTwdb!gTI(Rrx= zWVRuG?0`qBh8_v2?^)48|B<>0U4g<6`C5CwbB5L%c*bEEbvtgA?T|T4W$ktg6Y6r= z<#jy}EztBh{cuNSjM`~Qv7(x*6Bft&zD6N7X-lX?afVJmEO@ZCxgkL3E?u@I7D8Q5 z7>7d)i=_5Dm!8R3#UCKzF54OTntQgd#C_6<>SR>{rABAGzey0P9yzLq@VAV+0gnRF z+?o_Sg%y40|BpRDIp^aQE=0FssA|_Y4MHBxr&R8OCZ!~v!x;l5b$=@#kG5V>~2^8%A^wzr8 zB|<4QX>s>8zyFx7a(pdN5KwYJRYBH30x$ze$&3T40F(=0MJnV+m(o%a)LrG4N8ub) zhbfxZKT1oKj{m``)el(fySh#JLgi(fIAKuv=ucb*ZTNLR6^T|2@bL(4)iB7}eo&yX zK+yt|U>6yUwZi_0Cw1HsR(T-2-hd%nic&G)PRodA@fY##h45%uTgb{!O6ib;*OV4& zv)$Z|H49PRaSZtJJ#=-~OGsyLy{?VuZ@&?eZPT~bqz{bG?lwgN0hWSxRh}yhVClcP7<1aflWCIoJB_n~%LyAsDuS z>!tli$@|_#izA|W!(@0AORO(*x7kiZ>3!y@>UFwCZ`?n;-_*KHbfqh~%l~2IaajG?!8pQ<#ypil+TSCV`|u)9awNoO~T%+{R$G zn}WVO3<|c#dUS7~w2%aZ+4j#fmFR{-E3e)qB~j?TxTcIQnp`h1EY#rQEGfE>utpb3T#vWzIkVOH+O ztyLMic64=lK8AP_ZI;)PazO&rP|9TI&+0stJ{u+|Un-WLG&VMd;x4%6E*C9v{Gy)c zHNsgWieu^mTmeAcXh*obLp1mMzE6SHt>qo_f6zbDT~dEk;x;1ciBJZ%#i|sIWxc_| z;hwjR)?0a)YN`EK@v0sc2ZXWuP9sozdj=t zWDQ+qzQr7ib#%i9&kxbR2JGr(TKn2)Rol7ekl6Y6A&Ic%NYNyf)@Y%bD<#gINWXac z!uMrM?PfG89L>6>J~Dnigs=@IX=}6GLal$*eX(!)9!Rq!>v1t{lJ(DV+V*vQ#A9Oe zInj@=H}2gWYl;F(s#-?e3SnD&n^-cX%bB&8G_Q{F{762IE7QMXSIEgW@rX`SkRs%4 z4u9S)`8O}|Mb`%X?ta}&20F&}^wtme)Trej=J;WfDwlqp!~!HHSBg~e_jT8#Rc|G@ z;7I0(qTWyVipCopbZTaka7HJ)>{egiN4b6!4-2il?6smi#A~K{ruyr55GmDAW$_lZ zKT{ns*`F-8$NMKND>XpP*IL=C0zH)?elDc8CAIL62Eh|SL6gL!K=k;s;l))anRga^ z3KB_vXcT-$aWcVKNXoW1@;J!;Rs-9%RWlj1B6D6ze}UfQACJ8$zi(|H$^1gb;^Hx~ z3RfyQfZNctc<_tV^x)p93{$$bfGSk$SA737I8n0gaJ2Ti0p%(I8+zlP*&t*(r1lV& ziiE_8hb-OIHurJYh)_%5!>QllABXfAq1%5?6xZDMdqxvk$mKmxxSmlNviTq$(fa8z zEr!uE@8Y5~PBD08Kjl+CdjB>s!&ai;Qm=|L9>HgTtz^g>-!UQufT2Wy31YaJh5wOX z&i)to!*f%EK=WDPh!Nbqc9NZ+Z-3nOtI3%2-GP@__{UZM^ zpa1v{FrT~mqysev1Axi_MaJ|l%}d}+9Sj&*kdiYnQo}ij4(S?x7UNyo3;Vp5lx=1v zDQhk*oA_+)HDL%6-1%O18S>oe&OmmGCcEMDTuD4Xgf%d&P!GORMs*YPqJw7_NSrnv zLsfo@d6}~(>?rCH8=a8f(RiWq<-pAT`LH1|eI-jfRopw_m{t8M1@Fr2il0~Rdmdzi zJJdXXdYd^^Ew*E5df4vSTY`!=!~@syd_Mr&G1ODpJ+cNrX8Bi1(2zdjYI7XJ<@4q3 zEn6V5!vlk_`dLR`F|z|}su1?!j0+-hX+WLud2uM~4w-j7?v`0#`^X_*kFFEX07+5J z?F-_gtT2x8AhJg|o8~)MK^Fcx@M-LyyBbv6mX{;4-Q52&6 zC$72%*a*K?!s28b`vR))FK6#lADKFW)s_?TA;c2*5;D)C>apZmd=7hPsDCR*_IUR# z6(mI&e&O`D=3XO2EResdT*IId{RyaDSxt?*1vl~3x`pUDj$PF#^8sjLK)L*a)jr!z z;|;;!!zF>f4K7HGqDm;tWvQPNc2DIuuxcnCWhpV+m3?p{a5xR6Tr7A&)w}?^6Q0!2tkcjbrg23OpDs1)F6!$5T9b3eWH~z&}wO%9;I>|Jw0((Rx~~ zMPym^+W$GlIl5Y4zTcAX@W%=4fLn9chSkp&ep#^kOHfs*4kYY zBcK4am9Ahgx+QJRB8||~Dpzo$2bu-{h>NRMXUL>>O)Sscv zRt_9rr?aU(e|PqK@i9wHGhw!M4*|s$BX?R=fggv3hm6*At6sLhTo<>S6dZ>PfbO5b;|hsCY-G*jH5|^ zw7?dp5&aOwB_b^NI?Q<00IUL4o8A6#7jN@Q}igMWJ`#L#tp| z;cSy2v?@IQAZ0lPn7awoXzwFBNdE*>hWbn2=i*WQ1xvmIPGk8&4bCdGd!|L zoIlfM?(~9y;;Al>!#Qw`feb!kc@4UYB>CR3Iv>D)PcNz zBFtCLm-paO`Y{}MC=QFe1ed~zXQ`ehk1`M;2Vydz4W(<>HR0%lNxP7$L(=w;3&lpw zesvAt%^;lnEsx3|#>lc`I*fqnx6rXP<314J#sEUZzXL#tlt>JJ%ceV=fCO(utWLRxo*rumc`GOO;lbN{v<#(S`Mij<{XuF}rV)im@CU!&X zl-Xnk|Ji!0n8!%`Sww_PgGr3x7E;J-g;JiHizzmrJX{TOEhdd%%wfsVvF2xwo7`?+ zB41}vUe&F|PFewE0E)a`6wy6%qW6q*jKQ35nS90Z2T=Rco_g7}v zI^dW`R>5~3ky`B0LKH8~MJ;t9n!i-~ zv1MmN_k=`Mdbc%R8OjUoX*|X^IVD*W{s2x~x2VYcUg@Ufods z%C>sbyf5ul^ z=^H#KkEARu$eMBW4-}OFP`($)9sEoC5R@~>nqpO8HL9Kf<*eMYfe+P6bq7YeGsh)q zIPT}Y_KWLfp&oACrO@!*piM)o*T5Q-{pC0*;EIr)vvauhyCd@@bZD(9sV&DJ-^=! zz*yF(J2X-fG(Dw#8Wj~WgV^S82ctBTD|EdiXOaG%z42QHa>9I4)IG9Q*6VoRq67Wf z?t~ILp`_jJMB$Iq#nQMuxjs^VpBch)H=v06DMi*2$f>(xMOX-&VBuRX<}g>}IdnqY5feA2 zimbtk#yX@VwNc;AI&!n{UwUCGe*j-OxMN&nZ0QXog-CY4kSSUtUI3_*?1}SK=MRE) z=M1dMY&4AYwzcP=Jlq|~anr9i6Z|A6JcY-=oN0Qw$?dO0Rx_pBXBMVnSE9LW(kt4# zkEP{zpw{G+ZrTKp(}I_dDG^i0xv#*VC1l1pLp}fN?am9O4_`@lTk9QgF|2~`TM)zB z6h6#{QWj*zmCXBdz3$4GYWWD(YM6qNalJ@1OH@~@-PUweY4OIU<%cMG)pfB>k7OfV zUjNBqGMEQjv$S;t<5_?dUROnqeyN)h^dR;BaPn2i7HI)dI=Vfk3wO1=#@Hnx`2n=Td$=`&_0HX2j=>K{&@Fh!9fAIel zzw`k>D@b;%qb-}`|RYP=Sk)7AA$;z6z z!+t%}Zu;q}d@rI3q(wjOaxcNv1UViirsQE zE-})fY}gWFH2~+Y9X<3@^zc=Z8#j{mDqmMdEs{PvC81F%@yjWYqSW~JC~s=!Ygjqg zAi<#m;dm^LyZ&p>7ERbde4?1N6vYP}>XzI)kYxcR>d{>W%pD=6MccM*-4e*UJLqCi z+VNcK=vmW?7pJ16@27b4>Zz!eB9NgKR~`P@lgL(0idL11(!Ah$%0_oQvWHbYWl*5g zSK|uBA~GJCvOeUjxq8~p@#rzkMS=Ep%OWg2u(c z*=3U&o4v%t5n&#ar~lpx+dk7;e;9Z+1*t<&xHuYOXP-niSrJ-eiKCI`NgsU8eDKI0 z&xy`oB_vO*6~vJ_h|DU0`}f_GEu&W``uCS&N~VtAGk7h{8~{w;L4x2rWQtUm5$-6& zI+byMUN{&3hR*=XY)t}y>0nzThq8s&yB3!kMATEDX84?LI@8bQKngMxh}xq<3hYN9 zrbv|7ZRzcymdkgqF8oOX_VqPqe9>BmRiKiceL|y?Vl=@>ET4N)BQqd#mMFF`FEh zGgIp)^17ch;?S>rb6!I;~h)T?~p5T52#k z2uFIpwYB2~7o8Uo?rdB4%aq&o4WyoM!gOOFr~qKPMGK*wR=wJI4R6bw38&MyDJDL{ zqDQ6#iYQ@IA&w_{r}Z{yD7VRmKFRU6H(TsO!9HUw<%8pZw1Bo}j=#Q|ev%%r}~ooku+e^fmQqDXkJK03(_kl1%t{{HQ4 zgC>N>)t8Fq)P{(G>MRVTF*$FBdOiv3zO%XNM0=_IXASs z+SFfh?Jiqtbym3-_hX+xuEV(N-9m5ey*voVY_Ex_TM!1HmqG|%)x_J-^q5q(s$NRC zj`#nNzyUr;=2^eTmv{#0a=7SEJUo9@q<5sk!K`{0cCvdKC_>f(B~{d7VxCZ)daO4$ zzfPAlxeEXuj*eUu*SLu_ChQ@h%BHS`_?zzf{GjIhK7Eb4JqNz_6TX~^;iA=F#M#Wu z7?F(*J&`n%q&euPH24DI3K&aD(UG`8FAobET(JOgI9fQOc0b1+cStTPry_ z+(y$f{~W0zKts0{tR{WQjX|0;wEM&N4`10Aux0%&ev!y?g5B8~ZlHYfja8r~a|XU~ zB^(Xcgye=uw?9A_g@Fu4rttz1Ag-kaY5)w%e)5}?%ef3-N^$2ass}L0jL6ZdW#QW( z0O~Pb5g@$jpuYm$1_0d9DB<0-%XAkmA-0Se{_F#mU;uk z(@E{~ScgJ7mI$tE$m+_TrmyYID!-LyzsOzOcH{cE`GMjlHOS8F-zWSVjuI%>01lIW zK#BtS@pUmB)#1CB|Nh}a^Z)aWYoFM&^HbS<@4UIijOZ+}2YAvfoS_=)zuG={+oGko z0H%J9#yX4g!;mOM1$AXcY%boFw8u`CXB|x=OVznaFzXUV+#4pBkR_sVmJ~Sr<)wHV z>H+0P=w0G>cM_8>-!j>as_Ily5RKMH(pXTq4&sU`wiQqjbjZtjhAIFRR}o7p$xRW{ zJPed*swxMbczZHlgQ0xnYDWY9Z3+22g>?Cxvq$~r$J~TCM+j_v*k>I64y-%a&}Bh+ zN8LsM-!%>lvQw+*$yBT`L}$gi`jlvOrOwfH0xx)UI?@?-KPd zBC7=4(LZ;F@3josYNiqc&4P5dE?nnJ^$OG$qKKb}b${f+mK#14(3l=3Dau!Asx zbBu{R1HPn)ytyVkUjHxv5fT@daWf5rEWd;ko!w>lY|d%s@h6}M0KV_Erm9dC;-q2YX(|>$7`myx4 z5K#i@anZ=*D~dpeYKENbf2aT0fSEEpFlI2jcNMFVlqm(Y!~8eQe^pHlN8YuPUU)5K zbh#9b{n)iK7QxTme8rTe#OjAurHRzYBn!=qetV^|1#n&sT&@@E$VCakN(8>^sSX5I4<{+hsh(c*cVy~1 zaMw1Moj?&5U9{Tev;Q!y&*b6eC~Vp%yw5NJZyZU~i!rABJ~DvpB}h$s!}Z{S$UrUKf}CjVJ`*+990Zc1K7~u5!F^CPekX@g-1|<=LWvw4t0($vCh(aofXrKe)vEKP@RD@}ZWMw#p zEh?&{@dL6uEK5kkE|T#}$ayo(dpHTW-wG>AZ`?)pa-E`KYB#+3a!AnhoM~4A2R_SX zYcRf|KAZ1RCEdk4$mXsxXs$v$LG_C5Hsh4{@ue=Wy$!@TU#R@ur4!>h;jUq)?b>M1 zMt@()K<9&52{x}7$$=2X^w@IS+G|>JW5w9W^_1e*>tLJ8E185UD>c2rmkq&+gdDQ` z;E}5@$uF5Hfwtq~tO&lT>bxcMty6eZVFF^4I~iZj9`;ZvsF>U|;&f*CC=Uo`2F zi9#yxfCEjB>Jj13XfjpX`fef>)I*1K0pp)l71}SfXjFj2nm4FxTuH@|rpd#U>5-}HVE8PQBJRUNVdl%NztKV#1X|C;9 zQi*;HE$TPk{5^FO&@z^cpV3XS0JO8x9Z0y_>=~YtW?Kv^-D@uF(?8{| zSx3Z{1`$Xj-XWnCel&J6?As*c50mQNTw=5}qko#fxIG z=-9Cp23zlEkSUID!=pza-lNN5wB!E2LqXG1rs$t!4(aCg^&&kpZE$A@%u=u_fN^>o z0oTU@{8C}b>nS_I`#3JQ{lb3pEAIQ5o>9jklp;Ux>EdJFU)O(La%56jqD2(|X;KeH zCLde!+-TSmhpCi<%#7!V4wQ5Xf-jQ8GV;;89`9z58x><=WiZx51v9JkOb(tTmYt~Rf53zJH^d~BlKUUjRzeUl5I*@Y{ktT(4v zca21&zzGQr88TAX?*AQkagS)wsWTonn0kC*itxiSb{TI_daaUeYLZ2!Q*y z0<*5I*AScNKnhdENAVYXOuNn|%cD`x@W%!E%=}qfwm0=m6@=X1k)#i~vRos334~@$9fEa-D8HM1j$}Q?f9{-B$Ti0w_l@Mn7 zC`$5y-Xm>^eF$F6R8y)pM3Bo1=k(Q^D1pVi=r--^Q?PWQpsp9OP6V0*pJTS(jRa1j12+?B zjT1*)6^P{(i`4)rlJ}*U-=4l%jl=v3Nh?@f)ABRv$4k23RdaR8((~H+v7Kyeel|rM zS{W7LxIQ=8BPF-&dZaKE$BR1Umc}_ldL`Qr*yCFfRXWhbhG6t#(xs@wmS0EBTg7jH zuQ0^gL90K+b?~aJ-c70PU`|eny2oe2Nq-S#HKYOzbIB}{8{&mWxup_d_pl3xFxR%fPxM^n+7w@g960N9 zU3;V1`J}vs?T;-8{u9380T}_^R)EY%B~BOM*-ihc;aw8SqYT`Onx(5t&h_%#R z07D+cjuGd&7XO}q=-U?Rvj98i>9M|ZF7it21u*nUWFIT992qr`z+xp|YcULvO!guj z64@GN06|renUxBQL_?CqW10my0D@5)!qtjsrbuvcnw&tcGmu@PMX$eSHee>+ZN2k| zHAFrNCL(b?{{xxMf6oPTT-W1GrR&9V3rBD#&+%}dsXaYGCZH2(B5~0#x^FXzGN)g@ zVTr_4&XLw}wHH$T4>4Sz!< z8=qhlv+RAhl_1UCe9iG-@@&ur!OE`LPK>p!zoO+A7IZHBC-a9>@u#@5?2pyR{#H44 zc4A;8D|fT3j`kS5I19Ik&?xqUqeewU$yR5*&N%sc_5#}k zeG&Z-L(PBRqG&ga8=XoUk9aBRV4)W#CKC@9RBm-37qDHl>i{~|799_nDjY~Xd`JcL^m1|2*06F8qc_5dMi&vr^$1^>@<=ZK zAs68-2^f3@a>Qq5ex34B{cUI&tZB==Ko`7gc+Pr=9Pb z){Xq7@Zr|GcV4GI+wPttqrh-0EG?!Z`7a>h%-0IC8>*uQmOg(%>R#h!x-#Fj#!^0K7-Lc`UN1-EVCf+6xt1?P2T3XTd1|$QSA;l zU6E~@jz`K`dUm!_Sv*$LJJ(PkKBO>_=dSlVZ1Ix?yEk+`d0&H_Rf`8#Y^JCBXy-VS zSO;_L7K9CF7I`!+Jd(YySIuw^e?uQ<6*Cq>*T|ZkyUWZHC`^3IV;DSQS48Lr|tBd^u37AKsVz=Dmu#XaE5}4cG*h5&##64>In0d8!%b;bszgq&`L-T;c)cHh^Y%l z2azzb1F*{UP}t<#^>4qukzGN=_a#z*5{72Hh_qASbT7`*%oa`r)K1nwo#9HlOv5kx zP}@~$k~=mrHYL0}=wSfPLpqY%O*~QRYt@)Ua%sr=D;Br?8Jk*DLe39Y*$17rGTD*f zUA+so`K$HiFEO_MiGm)pDS8I{LYs;*3+3O``QqDyXm^`+BPbq?W&nabb;qS_wwYfm z8P%t3`}~jswg!=xN&3wb^8`ZdJX6$%Ln3+!YO?5uYb`q*4?CEky`-^d~x% z-;0SJ`EA~#CY^XbxIkE01Qa6A2KQSgV zO+SB8HBJHAX^MP%JV5k+Abi7=3fL59kb|Ki;bGb*4y?CB%;NkehTHtPxp?3Cpdb8w zsJAWiBTtNKkZNcJHJo)n&w6*Z%N=4V@M;$R0Q8OX8rV^bCv47or6!vM#D-uY-eg#D zc|C}Hw#kH9w&ydMG0nbRQgo(-l;dz6_=a78PbKEm86M=4+26^$$6;?7PB!Y)Z{E$; zlG_swE{N=}zu)BhEWhg4>Ovc7!#e7FlmXWy@iLC~{*-PwIFn1xlUF-pR=q6t)yBK% z+A4fMG%zbp*r1;^bo4hCb~obkdjEco2v+12P27ZfMxmD?x?Xj5u{hSszaSRqpAe1P z`FPY+;;Zzbg-ESz-n7{yoikb|;+PGziIc0Dn&@AFvBZpKzxpZ$y9)SAwD<1aR)gty zjQc5%mr+TIm-ez4YlK3B@Q-IDzb9BLhcd^qi|m)kq+id#1zyrf@AUNX&H&1xZ}mO% z#7S?K;#Ykw>s$MUOsdT9DXOY}V_G*@vUuBbBrd%J5V|dOn{Yr`yTr(pD#a8IWE9gP zGGo))`^9RZWUR|LWz8j}l^Qv|%~Z_m$0E1$e&d_bPPAFOfxCPHmLsRSiA?J9V*p)3 zB)QEG1CoK(_e$l_ni_?6y$Iq&>w%HeOi%~OYHNLL!#m=GFd``D;wc;Zp-3UjYCv$5 z02IVAFfqzP>4bU)QPffzpi}=YOIw=Hf_DNyi7(HE+XT@m=Eas>`*xJ0>|{k)uF!9j zgAmE+tvy&{N>>Wp2q(2hYBGF~#f$z@%BH327&QhyL{OQOEq3XOJP$_2BxebNi2U$Snp8x@ANF%$pbaM1>~$A^kL-)2(t!IT z{A@zVmdY)1y7p=hrr!F6McdK76bY2%Xtc zk6$z?u#hFPK$_3G_(!5SQiYDMjwG4{ew&yiioeff%(LE~kmUy)w7-1K@GkcXzVuEjwJ5z&KeSKUwcXI3b?6S)4`H_Tg3b$PA7lKp(& znxc6BR0Lb*A&GWvAhC{fHM<*b=%Y48KK;lBL#l1%L$q742bsKPr#|o?<&p`1XHaVo zWs`jHb!;WW=6E=YY)-0&idvWMzL967w-R-EJuAVH7tej8AeYCf)6|*XY}vhYNW6oq zyN;4wB(8gk(mM`fPToS9E8GeU)MT#b!Qhgqt~__NXen7a&G9K&vEWg zZX814hkI!`FY$3x0stW&f4{JP)nEq#d2_)!z%JMajhrl_QukwMprDv)9APK+nRHY+ zJx18i-v6to8dk=CbfSGqxLz`?@X)TI1D9AblD$Iip{_Rw<+OV}5Ik^sTvt$EQiZiS z^;H-G?q2W+77^od&b}iVP|WJRycz=`#sNUV&e7@T?$I5DO3QKX{*n?~<>u5E8&Cp) zD11A#bO$!mMeNl7k%KlcZpUZ!kbpM-fMpyOuvxJi20K9 zWuMQ~W%<*xs`0@NpaCL>?Ne#z?Ig?At?~6V-2v1iD|zwlJEOhr9IAR#9~*){B5Pb@ zigZXXD8F)ODfMlqEr3m)M$^bsy6*MZ)&{ZHHMbVonB5g=x3|$+05^wxH64GlaAvX$h^-*kV}|`us$S4r+5FHFso{l6Qi25+xGR!KYa<%uJ_~&ct{I@)J=0(>& z)d`0j!DIAhkl3K8%2V_6$RoT7w+l>E_Wrd+&|%%gr#jR!2wonD+=-dphGTD}n5bn# zpp@;B0p@(2{}yVo&!!!7?ovXbYvL0OIZ7 z9S}kbd=Mb&sln$t{pS0}H>fD0s2NHdAd&Qil~tJZ<-^p@S0el-Ki)1IPsBF6;ISIe z3p)I}ME93v%Zb%`vGdmyEUkCC8#Re@xyVudhFyf?$4oWV^}Pe5n|AifEk~}-v@|`N^k!z0fP&z6Sq5_X zD``g7B4Hd_L+~C>B4Otm&jd*^>k3vS&5Lf@r;mOi6fH|T{=!rdd|D1aPw8J-CK<g2#qr*(+*(;g5;! z%O7&2``_vr)G*09<6kgRtCGA=&ikafD|>uNBzTf(^I&hzRFUNg!?C!-+YXCv^w*(> zUANb2b9K9^V}NhVZ5awZK>y9`k%S@k+V5niJP93yYFn~j$#CjCk4K(0QT{(5eDmi(-+RvZlvh6uIB=I1h zl~|70XV%N7-yUK$_(=Ox-rw(|PIl(&^0Oi))fJuRbHfT?zJ2DL##bVrf6k{)HHibG z;b>EL_ruknG+gjz$*0du86%x%s=$bNSCV_qWcLs}?%O&UVigu9`OR99!$1z9RPAIp z(%9Z$EEmz`c`~e+l~oXRq)y%bY}3{y^BdOf+)ev#VPfSogEzt@iIflS+lgHtoeNfV z5&KQ}+XB@M^#KA{-3bqrsGTYp#%-iK_DF}_-_C1zP3K3Cg^==5arCZS1FMKqT&W`u zxk=4{X&tHACDGD!s9K$Djy*@Dcp2afS_HX{Wi)hJ$IxI1&*ph(BFlkNPL=e@iU0OZ zMMt!Gbt~_H4FGh*g-{^3WpjH9BG5&z+h``R6pA-D5*mzbe(nmh`zGl;Xkp5tCC-iG z{xcp?RY6v;cnqu_0D9>~Z{!gRY)=P_t6pXdn6)4Tc_px>=-OaV*t}2AT3`nW$bIn} zr90~q3WlAPBMDeid!Tn46OQnp1ltn3n$;?~lh{^V{7R0oLaQsw=dX5upvUxJ9Bj_G z@QD^dIAZDQ`)>BZoT_2GZqrZA+sGZWe`?B?g#X|4NKJ%fA!~eo`+p1nI?e@pz`|o9 zl}2LnC~Zg;NLsK`RM%l==mfd&ckpU1o9k+>F|wqjE?BVHwtTFOgUs_84($inb~u!Lup2RS3ansYR;ZRw`=X!PcLPOCRtcBVVq#w6RQA z?Yudca{Yq*NFm+`k4?H;3QV(jB_zJxdvjZ}^}yL|Ov5Pia-+WZu_t<(MasB#3!3{X z5F!+4wp;pEaFV z?i?Chl-+-JdcGxp*KVBmxjgpQ0{w)qm}}2zXnFbjD!;|Hl7+8;V*oG{=>gWW$g)*( zZE#0#peNn|f?8t_Y*QVGpLMi3-&!)_WLterwp(z}Z{#m}UqZ_EaLh=s@~&cvhn#VmgyaarPeyx3+pjiJ zrGDaCk&2Mz1uhInbm>``O>O9{tdUKd+JM`@6>X{mh?D1m(T){KVA%)m>towR%M_%h zAi~o^mYM01U{%V@)gTp{`(J=WTUot)D|o*!hS}Gsp$0}9iL*jQU~Q|`Rinw>dx}^_c!m<5k)KiQ}-XcJY~jg;rXeLD4E2Vc#cJ)%3O#Y zr)%*|nuwue>ZHCedjmNDS!P^AFboVlQRqKKKn)gNO;ud;hM8Y9H~G!<+hHUA61jrv z<)!h{Gs&d8p|o>_(tGD)odp3Ne}~r2<&-D{0Q)yUaZCOG{`H>xjb5>v2Fdc}rzYvm z2!CyeR00&_@@GyMOVEYvyw^IwtxGW85rxYBJuCw5*~Qok;Z%5PJKQ1U?>h0o^yX#qGx zXa4T(U+u_OBoU$jUYv920nLYuSQ|*_ZQ`}{SN{emE##d^z3vlmnb>lzK6K8%#x2*3 zM`aIR?Zp9fAmmF3sYsqYd4GA*P|X!+?^O`vfRu)U(hEru zz~{UlWhHied}__zb0$Rx zd177WcfHAYg2G0^(sHm;fs25&ZF0bYiDd<1w8+%gz(er2f5p7?B_G^Z6?fd`1H#SM zCwM!zMc92!5Z`_%>ia*vs;!&hd|r_A1>D=Po5qwQ$DY0Ru6K|w_SNHMo3-^29+xNT zp6wj(Dq34@wGXScsdF^FpLcU_<{zYq7Ia)Gv2osn3Viu>{sfTocSy=A07lg+)Zeok z$@QE6Jc)pAVGPjm2@D$(oiEfat6GI(Xt-_>U=9J1gV+KO_Q{dWIi6SWmSUr`jtdZ- z_5aGc5^$*3_B{wmND>l?BuR55git6gmLuYem`+5pO!jSTNt8WFwj4_oN%n}Lk|o(g z)*1Vr8S7y1e|}Rs-}(M?UCz}xM>Frdzh}Ac`+1)CW!}h!L2VO6|LD9&u>ggTKT!~c z^lw9Fx3#oUtz6AxboI_MVdgVNe5*H}@p0)1x9&@DF3K8LxN&j)@8g9jvx+nC3d7bl zQA(PcmT_=!q`FF{$w8E7b?mced-!gv6!bjo@Aht9^T?Il!^iOIkG~*(N zDCBqy=0u#`2+T=(a!@9}^`Ydgl%arZjoNJ7WgLtNGb3ZdcD_kBg<#Sb=Fc&m=f&W_ zgcJ^#waA8Q7R*;_`=VD)gcrcigP2ecGbPj?zD-s4-I$pypAY&|G&ME7E><5@=HlXt zIyMCTU*k5@V~H0PjMpYLy>!&^@@v@@nfA?V;gN^!{n*Bv+$w;V3>EMq^R9A>;Y z$+VvuTT?X|Qa=2^emFWSZ&qmA-$H*0960>%SNiY^gCUhYtGDWMil2HD zq7ls8dS5?V1eOc^IC03S7&`MGfXW-jZ6}Ew*kF)|t7VXguk8@^tR?t(CG1iS;{9>< zW`9o_g?L%`U{kOA{oc`UMJ*3`h;7H59>dH5GB!g!0{2r@!X%`n?PHo42;*Rf5KHtt z5q`n%Yg?Mdg4nVK9*&dwpOQ4r1*QQ<~mRBE3NBK-{Ewy9%OHa+g+8KoWscvWKIxGrt9pk;Yl7DXoc$i~Ip8g2Cvq z5Dp9j9IO8*%q1g}Npw_ZgOicWaHm9|fo!BC>v9C9*FK^k@u_)`&r^S#v#{Ch8`-?Y zNd~x>Y?^{2@k4*ThcXhn6lg*y(O-vwVu3W=LkY|lyN#ltOgT>n(@D2aom&0&P(J4L z3JHss^y+XvJxVWl_4{T*_ChyAQwVGP2t>D7){*4K|O*T zB#^^zZVd+(cs{d@R~$M}A{*1Iv;Csj{j}u;&ya`zeocft-ErgI<$ZyhF7*Qy%Fa_7 z!(EEX95s4=NuUuMF(PLiWbdHb0g60Ia)#UIIPlky00((Em60@ywzeY(UhmRWb{~84 zHShEDLla73g*!Hv2`bvMdx|4M_rRb*l@1LBoPtX$q7aKC!&4}Q(R5Fu9!=ba{!)E9 zm_x{Yt7xWT#oM-I*z>kJ@_xigdL}Xi@qfLpys56Pc0^THj6at!Et&mRmtnVZ!Q)3* zlV~b!2xuD#0#-8$4?FyGj1sC#u*q$-X4c$h_jji^4)h%H)yYTh>EJkqJ#3tFM>sri zDN>X=``@p=1KibAkNV^_)7~87?cW$K&X^?sxIliaj9y_V$?8YMthb9;yjWOnkENa`$`N>kJIUDY_ zTF2b4W5{B!h5jIa8)p-^&RBv?WBgCx$W>}Ak~@r`uJRv?!v#0?X4(_wYm09z2AG4 z_u%85#Z68T=!*{A8+B&8-3K3h8a5bH&J%~6co78`2SB2YTKtfK6dMU}CR`0O0nUbj z0!D5NOm#y5VEI0N0sl2MP1bsLt;D~c8FfMIen8g=seiwm-|HWlkDA)dp^PXlCs99V zzpu?N_B#He712!!QFyDp3R4UINPlHBu^bBT#{s%Bz=eng{QWR^vjZALz%jYavn-Rj zSY}g;UNo5Zv^Na5RV3O4_F85h60LoD;XZZAv!*t?@#%3m=9mK~4K4^8F*oJmf5yk% zh;6j-bLiIOzG;2vXBme!;9%^BWiN}N!z<43VhKGcR-U@NhXF;$faLmry_628Z0l17 zzPpIFvpL*kLwPOv7Y&E7{_6OKhA`6zI0}-&iArK3Lu3cnTtvD=A^kvDjQP^DeyhQ} zKE-10r6*e6^3rt z&BZBGy7wxahtuofiXu8V&nd>eKr*8J67^cm8)npA^ZI|s!m4kdgdb&VTUj-%Y|d8u zsH}g>PKvO^ZkC0@((;>24!ZnGI1psxkPe<2F7tR*q*WcNTZmC^g% zWg&{~Ag{;Pboa(GU$pIr(TRE5k-%{v@#=u|zb!>-k^a@e;v`EVe~2adjiGX-Fm`OJG9QJ4Tq&l|hLJvmR$ZeRtKR1ZZ) z+4qE^YKmVb#uw~*n60s3@ zMvpA|^Q3OWaOwV5oYm%+y2x3k|Ai#!rhg-6DxgT~SbQO+b9ui)(a@;Qr>5T30P`xE zGWb(?bR$kZ_&y1Y${kS>paDcX2pT|YwwKAlL3(h&0{_j+P#t8+ec?{j__*d0R8bIX zW74W#bnK4s?B>$8gX-+1GgFV%-PcO59SUc}A1+o%l|UwBS`HRHLAp}_ooF!kBcB2B zmRKJ9vqz=sDY%%b4g6l|!6NcUPdHy84M&n@59kbe8kQ}%zA;$qO8?)n2KlqzY-XC^K3oG(gu z6~W>m$P}UbE-SCU-|fWzgZ-|-yv>O0!`_C4o z(@_uk+j#Cqz;S#6#|EcVg!7}<`&aruMb6Uy|Ghsfyf&2Km0_o`sbXX6xn9S=dHH$i z467~&LUv&^AIt^x7C@J|wEQ3`M0LwTvZi)~Z7T${^J09ykm{#UIOnK!a(rE?jAYs^ z*Z7^E)^47|f1@K~Jczs{?Om#}jbTMC7nzXmZ}nqNw`uk+g7`X6hpZhHJ^%Q+9?>s{ zLJTg9U8^n49bU})%V(P7K1EClV zZAA}I$U5P$tWkf0B?|wPHtR}jU&oeTTB@m7o3of3nqrSI!_;rXOqy*_GP#@i6}2Pc zZKP26^w6-2D3vegGpv!u5#(4TAa>xG^QRGjX+~>~9aW_cJ%${AEnds$4KI00994PG z=6^H%>VsAk@ke)3&gv(Vj2@R>s(nwJoR7HZ@UKBg8lrcmQ=* zWi4B2SiJTW4peBs9#)jye`vrUfVj*Vkz<4^UteHxh?h7Y1TB9Ek7FjK0w~^*8%o30 z5bvRO*BOBH77C!LKTwqZ5H;H=ji{mkz6vAmuBHEt98!x800F;z`LY@v9liM8c2TEj zeQs&*+;GGD&rcvIf%`$YgHy)(t=)*$E5!I2KZtlk9UOV!aBT4VNU)tY$s`)<b)uPa2Oz)h#@f8|jP2NFuhLcqQyumeZNYS%W z2T;YeUk46acmk;)YS2+Xd)9Si4bOzdY7G~Pg>$Oz2_jrRi+`z4^r5T*t}5C&F4ri_ zPD(uhIFg{wn#eB>W7Pp+#}ObTQ_t`dT0&}oYqok!P=qA+g6zYo0lj}br@lEh6~~H7 z<0@&e{5O@jd2{meQ3;7DW|Z|8p)ad_eHkjf_H7yFHIhleCxbTg`C(T)SSn|Cl3`9G z{Q+$f@GUarC*wr|+ya204n?iCtrsp4-b@?|=Th~sl))ddBt7OJoYMMHic^7dZMY3k zR52@LstP$=>0dkqlBPbtfFsJACn!n57wsX<5mlI47s$%wqau-){r?vm^>{#geZ)9p z_T5Upnoi$BfcbX@1nR&scDHu2bcV{FOAA4N9 zws%%^`S@d{xy@B73twEIXT*2c$E9eYa`(92k{c=|<%de?k*WA$xVj6Oibvk%o{h7u zlM_HiW&)h-a4R*6w=#SNNj->x+NDOhYtG#gkU#0zT%4g%ME(~hLH@{|@DD<@RS_JADA6$--eSH&C4HVF zrVe9Ni@6U%+PhU1=*>MZJys7&b0cofyJpmyu)Hz%!J(C& zNGV?eNsA9Syfs-9-3Gk&gW7`#2T1+xDRO59By1}4PTFAr&Jv`YRutvQ4xm>Re|Ej; zz5D~yud2LJZ>4%cBfYHruTL)jr$iT@jO60bos5#0q$ zzD0w3Mafi^o=UklkNez%CL=SZu4^oE?iw8bnxf9THxC z(H==5T)q+stRCN@VXVZ*T-TjD{7R<8x{)D5`^6Fb|Vn)+sz8t$Wd&&1fa z)8J8q=I-Nk!k5nEAHC^&HzxS{{hXKN-3~ez=vj8X8N6E|Zm4^fGiM~PbmoWAd(YB} zK0;29UY1Kwrl(7B7WW_b4@pO9S{%OqICcf^<8#Q5kAlM|&Dw4soU+#rvupbCJYRC< zhkTRwss{rKUX_dKYm|Yn7Ypkudfk3RnMq8Je}7aO@xvD_Z*DC^tY_~sVxyGt3G~o! z+~UMwD|u#!)>`BLVWXTE#=r7K$2ONxxi^aVvYsTXv{)+%pxE{huKFS z;v)%s_%m7L4EL*rIur3X!{byHP}Z=72fEWNc8DDo&8W&^G=%1 z#)ggIU*Xu=S>+o&qruHCIfN2DmAj_&A%rIWqupSmMk2zP(37KM5j#;J?0=2d)0(vkMw z2+3Km&C{FA{ShG)?u+jFAh(BfF3`qy2PH+VC{B@L0PlC+67yz5t;m1@AE{PE-a+Y8 zF)sC%)wW9s?^pDc8qBia4kxA&98YNGTwT4rcww2FY}zS-<5HyL`J$D~C1P75nQ(On zjrF7zJ{53ub#s#%gmSs}=e2+TAh(+|6Hc+c5D6C_r6tpy%8dHGm&7owOGpD z@#kig8|y37)WVF3+qQ`wHeFU%ry?Y=3`soX7i3eX+Nb$`P4O*K+_R!w3g-vQZ!;6# zu;D}kZD9LnnedS;0xbuP%lRo~EQ#a36{3@tQ%-WWi9gcWxZ=hQ(TwQEzVCbIQa6@y zD;E`KyvhQe*SWMkxF}oYsaq*0o%|_y`8++|TvcP=tSG!CmqM8vx~Ed^M6w7QWrB&l zYi_W77+H*p9hxNIeblKyS8LyhbTuz~7yYlVfKwDbQT=^w3O5zr(a{pwt(R<8KY3Mx z4x^B4CPl0n3d)8h(u_==O98ffmT}|w=WZtNG6rwWs-)3$$ui|>KJEuTFs10FyN02#>X`eY{Qw*p5I5cA|u!kdi-!i_6Rsd zOYUycIW*}79VX8oD@jbc8ob}{&q$FA+*Sq3tHx|w8%{W({-l!k@{N$(3gfo9xt4AP zkF3$Gu%zSJQQG6YER+-hT=uW^X4?@jvv)12SH=kD6;9B;2iTf3uzJmu~>!3{Hk_h_J{-J)yYCz*Uk)uNM6x}-p!1yb^8jQGg+hP zWsG`nT4ouA30yPMP|23(9tG25$G7{U70o@hm~kS~3OeH|aD9xgP&gaTsxrogh!G^2 zd8e0Ht#6Z4T7Da-ycsjk^9i?{UjDpuxv!;S!$;NWZIy=gdH4-Cj_e^D;ymp;%Q@6` z+^OUd$ZJ?Q>$QU<~L!1N9pThpnWVXWJ@Ygwt{b8J1Zgb==Qo)+I= z&6UOf=?uon$|xHf~Tq4F>RT>t&BlSCfg zQ#a!+sqlXoFiZ3pFh_n$3ZFnZ+4OX><48zYzZ?2_bIW#$USGk)hsFxKXW!MQs^wak z-}}dmIv+Zyvif-g8igZkGRca(ft1y+GGz1Vuaz{P0{*H~Tm*cUs@ zh(APhDGu6Bz4|phBtG^rjBO)nL>MzpT8&=GI(*;HW69b{v9Y~f@WKhPQuPO4y+}^Ur z@Oq{V>Zxrwele#M5&RpL#CrOTba4|E4wuHM*8)B2#_8_k>xpy-T0>|eu8zX1$7e6kx_!Q5J}41pio zQ`I<``l*M8Z$7VmS1an~M{HxChE1pqk^kW`c{?Q_Ojd@_qAJ=r|2)t8Lv9P<`qoyX zL>}{+O_vw44TYiS6FeQs7d_smE2r?0&M8tLF+esqSFO2I3AT(MX51*`C538^KVzZf zu?Xl^68^}Y|GZv(QLvlxcc84c6lt~JoEGcinx%G=UT`KY(-SUK#!M42^>mm8h(^d0 zf&@z`tHcKHIfc`Ix?9;W%EkNoLh;_M^;Cu&s^;~~A64%scaPRiT@@RjiIe91aX-w@ z7v24#*pmSgVielLH!W?e7eK|_vyhh6;Bm1eJ`2JM6KgzGExE*qHR{wizCQM{HMO`f z0BmI}0%~+g#r|09$_fX>Rgh( z^?Lj>uUViq{_9%Li|4FZ{e+zrCSKyhK4xH(-1vvSknY1DQV!E^JU?Ya3^pC;DfB}> zqQKVkJ2Bx}B&b+pBo@2mw- zpFAiV$4YFiz*iJfZt2Z7*pt2cjaz%k79+1niougtQx%c-JvrFpRMk<-j$h!$A45Xu zi$EJux+J+&Gr)hAP9DO@s&xKPP6bE;pv{Q(mk=vtMiwKOSALo}C2wBO0y&DnK!is!9#ucDb(e9bf#Wv&6OabB<#nmIrM_COW}2+KIJ zS&Ey4U=?9}B3+Ve?Nc5E2idYx6xb0k3hscoDFZw|z|H8M4`VbcQO1M7bEn#lbYHCK zpsP@>vd27~+lwygd`-QZu6S57EhcXuN zv0_}U!EOZEfYTj8XvKrWPMX_rko(GfvCgVB@R)uOt!ef;s8(dZ6LF<1UId5#&R)^J znBi0WrV+iWqT*xm5==}r8a|4@ZTQUt^=jTVt8+sXk@kju!}`ruzB$QlxU4{%7l@&7 zag###=Y?8U@|?r<(Bs?LaA&JJ-}}KsM799zH_+ymv_h+IQ_qKDabmEAlMO57-meVO zvj?#^+wRI@K~nsyuU(^(0^{c@nSbqouV7Ryq@lf?Bm1+L-eJVo9{0b@3@>YpZxN3X z$}5ya0OFhrDVqt`#g2D6ud$ny{ew7954(UV^uY*Gd4I9Ziih z&B+X}vY{e?zKZTDRI)N0T6t$n#GdEsVAE+>P~xG70aNY1P4Ug*GIwibH|REB8$B7M zA@gG+y@3p?Uj_E8Yi-+F!8?m1j~;`K=vW+I?^g^q4#S(8>IClX2!u6EyS5fYj8juD=QXn#jfj;K z$|BDyU8k)_BW0)@P&WUmn^|)G=$9EW^OFHIY?|Ok;3GAS16adX`c$2%l#@!%7qR0~ z&{_rR7nHrgh&N||+#|$4t7KU1S;uMoX|W#`vTrSC&v#$6vbnX(Ccc^TZ()r;t~~2K&utZ1uJ+(-nfxmMKhLp2IOYD2 z3YF5TiXWtp9v+1Y)*iM4u|cPa)-R@|YIz+-hY7=?EJ<( z%lmeiP?vHq6MkcTUu&aI>u-?H6|Z>WiM7I*@xnX>kL&{F*Osapt2;2@=8y#F@qY~d z3?Et$i!^KkIFXca5_#C&pfb>+KIFH4H9+kM#7_71S&_h%|4LDa84s^ zQ;ELB;h#Asgx`7&p{LUA<*t6l?2*k@I7k*5&%92aryT78cl?P$pBG@KotHRv1|bx zKrDc<3g%gINDKrrBh6Xyy=CvwyOfuiLhFA~ma}Fd=jSdlokzzAu$=M7_5auQEm;Z{)Kl zxLf?s;HhE57-+M|gwrFNA`gY6WrJX`3|NQwg{791lOC?`^1DCqnmudUT8QsdSjrLD zkkK!SR6`#_fo`vt3n$BsaB-%Rwm}O{2&i&lL?~1mE#wL06_)yezE|&wxHWvyY2X+< zlypRVL4wbDga&`6(RPG00;KoFo*d*+ycek&n6yLSY^LCa_wx?yn^zZ#cWylqVrkpD za8l52c0>0|9NcmR_$y9vrs7uHyWlHuleHLe!T$<=5C=U1fFX1`*UE;`^qB)RLiQ7Y zYa#+8pkqW$f~11?J>5%x7ao04Sm3#gU*l)mTEN{t;qxpZx!bCF zC3q`5zrGJ{-}I8&lu-EdI*)?>x0jO&<`OIvbd4DUW;fXr=&I?!6l^`?uPiW4oTtfq z;ICU7ZB*f!YlVMw35h#Du$n-vwpSUmr=9o7 z+S&~i-ubv-+0`%D%(&I+u%Tepg~5AVPH*2GOeyrkPTgm>WWWFg(i*=YOL7OT4IxCp zHR}h~J^CQltvGh(^hz=X^KeL zJ4(bXvLxmYP>1&(wRic`dEdIl(|N;BrOFpsPTgSLTFBklH~z&)cI^2|=q+1`Z5KIBb!3FpvZo_ra%(VMM=h_a069(Gk z7*%%K*LHjf0mLB0MYi2e`7CZLptG44c~oz_SEMfKN38s?oSduIBaZ=J$*ooRGNoLm zv+W!bQlaLHI?cUc#I${2wEP%HM`%Q1=$Js>OeB0dH~us2F_ictO1KMRr~`lx1Q7Yn zqqc~d9zP8r3&IHU?R(z(V~E=y%tv_@6sRQPPS5CYV5mz6`nPC4YP^kmE(*T6hs8vq z=X2^{pIU&ahg41&`>0H&i{|)ru|hIXagg|a%AQ|LiHNV_E0d0?AZ7^KiWSX3VdwYE zLUIEI84+T;E9JmsL5R-gQa~sWE6%j66F?hi-WsBf>|Y#Z%i`THPb8WoC8Uk}%>RZd zsD-HpUNh|shoNNZgIj@@@A0r7yPX!ZH{s#_TI(z5a0s5%cRJ%=+ryCU_wXa6B21hB zVk}Ljy(a*n{Fkf**x8h5ICaXrCyGwxFp`oR$}3f}_R7;o3TyC7YQA9oTxVOKpgmGO zj{O}wQP0wjT)MeTsQ8c5&FOxziq!=oJ;#jKoP>vXg;W|h=~kc(L@a*y07T4!gkIza zEdD2|fG;1%rND%^fGr33g~ZR$4NM2{2>^o&^mv0Rv`az2+~Yp|>D#HNE!#5AHJ0X< zVSn2p>Oajg&Mm=G)qBpBkh34LDux|TBjUxhTm2YA1uTHVh!mv+Aq5NY!etsQ7C}Vb zt`Cs5-4&6<(xzbp>)6K+6%4!$1AOiH?Fhb1XG3Zy@BMQ4NP680dk=Zu);-FY)hev` zIao&aGdRG>q5{RC`gqr~J zhj6dXUIZ6{+yl5U-E_ba$-tn=I0}RDVfF_4`nV%@eQw8ozq_V+@t4|OE>1i_yNG`w z^Scp#V|moLJdfpxj?)WGLj;KoAFqkJ^WBTz4HrIgg&u^ozD2$WjbP&dKP{coKuTdp zBy-4jeF#IJ7xvafqC1#rSU)pWxM?_3&lvcjX=9+sCl7pN5L_9gYCeE|~)w3+GuRp637q)2Hizz}P; zQBMix%H#nu>t}T($$%Bdb|JsxSWE`Q%%#vmE47t#DI@@WLn0gxBW{Jsdu*Wl9@yxF zhM1r;Rikk5w71!U+?GY4P;Yd1=0RYp;=gpdh0;;4Q`pQ}5?U~p+g339Fg@s3 za=|(?5CCdy>3KaM_k+T+MMZ(y_xm<^`uSutQ$ZuttCX9g2Swq%i1(ONSUAFq?VbKi zmL_%hDmyZtTVE2J!~d2IPrPD?_oDYpJ#l+z3DfZ6ahH{(RJs?uTu13P%3HE#VY{K zLV0TGWKNePoeC#fLiBQBQgg2}J?eVp#x)g%1h1*l4`l?a-kB z11;wyzDRm`&h zS03uW4CGRE3_G}M7RncvWyQ{_fuMbDr^wn8F?^7SBc-9BMb#hQe#O>DL1CoGwVLc5 zxW!^nM1^lMxlvGSiJpiiT$7!M`7)59=v^@@#WCagO-oVeRcc?j%sD3=L~I5b;we(P zsJR3JN+pO!;Lf3lHjTtn5F-Jlh&dfXL^;CXeRedl2OtG2<#7<26?rNb{Ms}=H|Fb4 zpVo+ewXi1dwUP7qx#>#c8%$r?xcjkxc4tG>c~gskk0tb+TFZYl7Oj_oanDS2QF3_T zqSU&pd;PZFeI6tW5}x^P{#`Ws8F|o?ujR-dW1hW z-gsF%k?}=8g;9HULM#DubzvrcMX7)G=&C0Qc@z*EsLUjjdj08@ahDn^N z41OADe2bM{uwIYCoPg<32d0;&3Ky$-hcP+z6REv>7A4}&guMMHsh~ofKYs+&T;F(P zAMR9=OZlAoJG+`s8MiZ9&v@=}{SLY+WUlxAu)lC-b{spUOp+=HqT8>|2D~}5&o!M{ z1kwXQ7Dj5&65A;Q!Wl{M;hC3GOS`=bruyS~T!gZS^tKO=E5d!IVL#$uQ9z2*16 zqXg=}9xv)ssp?A>WzyqkOxM!;!aaJHjx-dQHClOiYx|d4o9ca^rs+MsNY;Q<;lhjy zh7uAFr3*~J6tdl;p9EV2sVI}A2X~RqablM5Yah7&WYCZ`QBwc0!}CiCJl%WQLTus` ziG62vD;!Q4{e#k~#WgRQ>&CAXmQ*;}J%`hn3?165dB&15#zqs8XYzFdT^i=^J2Qe1 zzEQXz2@?pCc=juP0vc1V1XN9{p8wo_C?2|=r`~vX>V-VWESI9K)PvXb%uMeL^q(3IP4=cJGOf*Xm;K|JgdOD<6^hed#M7)H)1yOL{HHq)$>)Md*!WSnPZVLcf&YPWFq7x z0cbHmOz=B`OJ68b?jdp#B6WfG`j7%NUbkxYp#UhIpuBh~QqC40lH@&pfqdnf5wqmw z41)G*i(mI^<6xhUYkj{15-Nya_)_hcFOz-i7=mM3y&-!tR$iWzJR@p#_UI+s7q=!B<|n*13`#P>9gLS} zH4F?4u)>#!o3j5bw2Fmu@ln{ywQJW1CQ4&EU5bJ(6}jsJ)l21$O|4Q4qlePER<)DI&MALhBt{MHI z|77yT;{t=saiQcv?~SFr4L>({SmO{rctu9e%r{jGwaJgytQ43TekH~jL| zyz|>;Ke>1RauVjskut05&VpO#1!kVLeAPpIqN#>siayEmhAo+Aw~-VT(3r6B+%`FDZT7>O%i1r`XmH(F1e>R$)=RNky1GRF8xfN|8@1y(j zNDcPgwBZ3iUJJ+HrcC5dnUb0N{8HZPZu|WaF6CaAaj8|O`2~+|g;c+y#@lH#ZAz0% z6N2qaB1J%@&O@RFNt>U-wDpI_zO)F26Vb34I`O+%D)5)lbgR64x?ug{vA6QkOY>J| zZmcfk4y)L{pJJBbO4$e!+x02f^Y>CHOd=`UXk87LCpT*Hn2x+^YFKROms_T`m`F2B zF7ZKJJ1=b;+i}J?L*`Yykb_2%)_kT*S>^8sGt20$kJN3-FCkqDpzdV>9`zMkW=V7s z2(&Sy_x{u5yUH%{iCM$Tt7|WDmGVbteV*=R-~K9ked6}#O42l4@sU@$KCh#YcD}Io z-%zO1S>)&_rnk{>&&UI-;vDUs^|#nM>8Opde3Se}lkf6V`)%X* zhM&bu4-lEP$E!d}17MCc5++_%I*p`z;H@!$k`Ee#p6OD_k$G#~e*lilyKAq)5zTSx z@>PbbnfuaA{}bUcm_NCb@R+=Am?8dXw^D3WklSxIh3YJS6PqYuabfSR&3UijYHCtU zhYDrH2T%Qj(WPl;%c^*W)%ugR0(&F1wT_8~fo#3<%zmU9&$F=?UiIH8Mi$gC}yjdQl9NL|t^xN8fi+-$A>z8V~l)TdWTrB5?brsW-`rg@=Zl+4= z8Cy+#Gw43it}!Jh)BNBfTY>$}sM5b4$>kwGRyXtSC?KKdp)b~56{(-1X$+MWK{heM z-e@yF)4jr3A zofHovm1Nz%KR&eO+B-P9tq+d|O-E#(CORU_=qDrlS$6gVWP@ny1|md5Hg@&B>^tt) z2T~NiFv(`UVCd6--mlv3ub;`MIwMo8=XkaP?Q>1JR_{a!!7qds$Cqj{4Lkjerh9)Y zcBtCc`aSqg%P2I&cctM-f0vGmLWm+MMOif{a$3v`}=R=q%3O&j){`0x7% z&LPFBLFknr_LTZJAq0WIMd#BJaVG8N-2>O7Ik!i1%w1VIfR1K!XnVXf<%QZS_XdH5 z*%B)uq5XZ98lMaB=-wK6&G@@rNu0ke4mB>W{$!U-ITx3$nE5ekOJ7)Tq#o08pF5j7 zyUT4I#y^c^w3aLm#lQbpUOX>Wqt!I}&dMaraE5AS&>3p0i~In6axFtzU)}whf|m^) zRWqt5?`xlzxO!{bx6BYEn`D4_f+Xy`9H|Fa5bkR7yf3OU`V1 zj;Gs(M0{-=h24lhn00vL)@u62v7$FZW9~j)+K%-=*}6|EU*IqGelF0X*%V@xW?6l* zC9XwqF*diK)SdA3t+mr-^O^Cz&hT^98j6$va-fk3QYgpoHEaWvsv;4}L=;vl*>)3fYZA`;fG zr?{H+C~kW5)VuKwmGuuY9mW-2!i9FRd#ng2lr^WSSFPF^77G;K`x#g!C135*n#^R5 zQ|K_4b3Lt?F1qmb*;7(Cvyrg1;B8&#Vjq%1YNnpYh->X}iqyz+uB)HgU02nvx~?i5 z{Xufl@BML=$Ks^!xd-uI0#CLM_B6HTDF)q{{d;L7_8UCGmrXjG(;V&-Ndm_jPF<7o zsci9FU7A@df3vCnIeODb%J7O|K~2#=v7&M@-P%PgS%Tgg-*1r6F1NRT{X=cW0|9p` z`7e#-tKqA&%dR@<1lyzwn3-p1@U~n*MWJFw@qL;S8j0!Asg2EF))ajVLhekvjru)Z zYUe0$Ivv%w+bFSbx2aLE73{BT?b4=kD@irNE7Mw)@*{`85QLt2wjxjU-1e%e8)4i@*K(!t#rx zwcdh3Z>-zM%crC-x_HuLB)*pRv!>u<&^x7v-h>vVbDTcvtdasPna7bIKJN+3E>cHr z-avNbi`b3YR!yHYi~B0qrORZ>PUVZ1jccPNOWQ67&n~nMDwY+Dv!k!FE&kcwyw^x6 z;r61*x7?#qlLp~`tCapoU;c7JTYz_c$g;y?)7*>9pkgX#_0cvy@F?h#+cwLVRbT$| zjZtc)m}!lfspY-v%ZN19xbN>YcE`qjPgkO@r3S9w?tfh2$N$8>tzwj&SzE62aRU?aOOcpD*N$PI3nm6sm>VU?`g**Z6H&`9}o zYtc>h;i|c>&0-P{0yqC!B-LKdIx*1|9qV^5*+&W6(X5<*p5y(kj<>IkpL~1$`sRR& zPDpOQuHw=u)qnUWYH30_ZpxvgG(58(n%2LqCh6`G&DBXth(~-e&S5HI_?ys9{<}R| zu56N}W0oRC!}lEx!{?MrEYfm37GHK48idZxm`JrQcVv3Tt6X!(&ezn>T5a5JSCULZ zS2Xc-Al0L$#O9JLnKGw$v-6@NR3XXo`b~1)duq}#{sB+xy1RsaQP)Czp82~R538b^ZR#dF%~Z&YvG`# Date: Mon, 12 Apr 2021 11:50:11 -0700 Subject: [PATCH 50/74] Explaining the state machine used by FlytePropeller and Flyte (#903) * Explaining the state machine used by FlytePropeller and Flyte - this document helps to explain the various states a workflow, node and task transitions through. - TODO add a small table that helps users understand what a state in the UI represents Signed-off-by: Ketan Umare * updated docs (addressed comments) Signed-off-by: Ketan Umare * comments addressed - image for nodes improved Signed-off-by: Ketan Umare Signed-off-by: Haytham Abuelfutuh --- rsts/dive_deep/index.rst | 1 + rsts/dive_deep/state_machine.rst | 78 ++++++++++++++++++++++++++++++++ 2 files changed, 79 insertions(+) create mode 100644 rsts/dive_deep/state_machine.rst diff --git a/rsts/dive_deep/index.rst b/rsts/dive_deep/index.rst index 6e01c1dcff..9efea9094f 100644 --- a/rsts/dive_deep/index.rst +++ b/rsts/dive_deep/index.rst @@ -40,6 +40,7 @@ Execution Time Details :maxdepth: 1 executions + state_machine execution_timeline observability dynamic_spec diff --git a/rsts/dive_deep/state_machine.rst b/rsts/dive_deep/state_machine.rst new file mode 100644 index 0000000000..e04872db39 --- /dev/null +++ b/rsts/dive_deep/state_machine.rst @@ -0,0 +1,78 @@ +.. _divedeep-state-machine: + +################################################# +Understanding the State Transition in a workflow +################################################# + +High Level Overview of how a Workflow progresses to Success +============================================================ + +.. image:: https://mermaid.ink/img/eyJjb2RlIjoic3RhdGVEaWFncmFtLXYyXG4gICAgWypdIC0tPiBSZWFkeVxuICAgIFJlYWR5IC0tPiBSdW5uaW5nXG4gICAgUnVubmluZyAtLT4gU3VjY2Vzc1xuXG4gICAgc3RhdGUgUnVubmluZyB7XG4gICAgICBbKl0gLS0-IE5vZGVRdWV1ZWRcbiAgICAgIE5vZGVRdWV1ZWQgLS0-IE5vZGVSdW5uaW5nXG4gICAgICBOb2RlUnVubmluZyAtLT4gTm9kZVN1Y2Nlc3NcblxuICAgICAgc3RhdGUgTm9kZVJ1bm5pbmcge1xuICAgICAgICBbKl0gLS0-IFRhc2tRdWV1ZWRcbiAgICAgICAgVGFza1F1ZXVlZCAtLT4gVGFza1J1bm5pbmdcbiAgICAgICAgVGFza1J1bm5pbmcgLS0-IFRhc2tTdWNjZXNzXG4gICAgICB9XG4gICAgfVxuXG4iLCJtZXJtYWlkIjp7fSwidXBkYXRlRWRpdG9yIjpmYWxzZX0 + :alt: Happy case for a workflow with one node and one task. + +This State diagram illustrates an extremely high level, simplistic view of the state transitions that a Workflow, with a single node and one task will go through as the observer observes success. + +The following section explains in detail the various observable (and some hidden) states for a workflow, node and tasks state transitions. + + +Workflow States +================ + +.. image:: https://mermaid.ink/img/eyJjb2RlIjoic3RhdGVEaWFncmFtLXYyXG4gICAgWypdIC0tPiBBYm9ydGVkIDogT24gc3lzdGVtIGVycm9ycyBtb3JlIHRoYW4gdGhyZXNob2xkXG4gICAgWypdIC0tPiBSZWFkeVxuICAgIFJlYWR5IC0tPiBSdW5uaW5nIDogV3JpdGUgaW5wdXRzIHRvIHdvcmtmbG93XG4gICAgUnVubmluZyAtLT4gUnVubmluZyA6IE9uIHN5c3RlbSBlcnJvclxuICAgIFJ1bm5pbmcgLS0-IFN1Y2NlZWRpbmcgOiBPbiBhbGwgTm9kZXMgU3VjY2Vzc1xuICAgIFN1Y2NlZWRpbmcgLS0-IFN1Y2NlZWRlZCA6IE9uIHN1Y2Nlc3NmdWwgZXZlbnQgc2VuZCB0byBBZG1pblxuICAgIFN1Y2NlZWRpbmcgLS0-IFN1Y2NlZWRpbmcgOiBPbiBzeXN0ZW0gZXJyb3JcbiAgICBSZWFkeSAtLT4gRmFpbGluZyA6IE9uIHByZWNvbmRpdGlvbiBmYWlsdXJlXG4gICAgUnVubmluZyAtLT4gRmFpbGluZyA6IE9uIGFueSBOb2RlIEZhaWx1cmVcbiAgICBSZWFkeSAtLT4gQWJvcnRlZCA6IE9uIHVzZXIgaW5pdGlhdGVkIGFib3J0XG4gICAgUnVubmluZyAtLT4gQWJvcnRlZCA6IE9uIHVzZXIgaW5pdGlhdGVkIGFib3J0XG4gICAgU3VjY2VlZGluZyAtLT4gQWJvcnRlZCA6IE9uIHVzZXIgaW5pdGlhdGVkIGFib3J0XG5cbiAgICBGYWlsaW5nIC0tPiBIYW5kbGVGYWlsdXJlTm9kZSA6IElmIEZhaWx1cmUgbm9kZSBleGlzdHNcbiAgICBGYWlsaW5nIC0tPiBBYm9ydGVkIDogT24gdXNlciBpbml0aWF0ZWQgYWJvcnRcbiAgICBIYW5kbGVGYWlsdXJlTm9kZSAtLT4gRmFpbGVkIDogT24gY29tcGxldGluZyBmYWlsdXJlIG5vZGVcbiAgICBIYW5kbGVGYWlsdXJlTm9kZSAtLT4gQWJvcnRlZCA6IE9uIHVzZXIgaW5pdGlhdGVkIGFib3J0XG4gICAgRmFpbGluZyAtLT4gRmFpbGVkIDogT24gc3VjY2Vzc2Z1bCBzZW5kIG9mIEZhaWx1cmUgbm9kZVxuICAgICIsIm1lcm1haWQiOnt9LCJ1cGRhdGVFZGl0b3IiOmZhbHNlfQ + :alt: The State diagram above illustrates the various states through which a workflow transitions. This is the core finite state machine (FSM) of a Workflow. + +The State diagram above illustrates the various states through which a Workflow transitions. This is the core Finite state machine of a Workflow. + +A Workflow always starts in the Ready State and ends either in Failed, Succeeded or Aborted state. +Any system error within a state causes a retry on that state. These retries are capped by system retries and will eventually lead to an Aborted state. + +Every transition between states is recorded in Flyteadmin using :std:ref:`gen/pb-protodoc/flyteidl/event/event.proto:flyteidl.event.workflowexecutionevent` + +The phases in the above state diagram are captured in the Admin database as specified here :std:ref:`api_enum_flyteidl.core.workflowexecution.phase` and are sent as part of the Execution Event. + +The state machine specification for the illustration can be found `here `_ + + +Node States +================ + +.. image:: https://mermaid.ink/img/eyJjb2RlIjoic3RhdGVEaWFncmFtLXYyXG4gICAgWypdIC0tPiBOb3RZZXRTdGFydGVkXG4gICAgWypdIC0tPiBBYm9ydGVkIDogV2lsbCBzdG9wIHRoZSBub2RlIGV4ZWN1dGlvblxuICAgIE5vdFlldFN0YXJ0ZWQgLS0-IFF1ZXVlZCA6IElmIGFsbCB1cHN0cmVhbSBub2RlcyBhcmUgcmVhZHkgaS5lLCBpbnB1dHMgYXJlIHJlYWR5XG4gICAgTm90WWV0U3RhcnRlZCAtLT4gU2tpcHBlZCA6IElmIHRoZSBicmFuY2ggd2FzIG5vdCB0YWtlblxuICAgIFF1ZXVlZCAtLT4gUnVubmluZyA6IFN0YXJ0IHRhc2sgZXhlY3V0aW9uIC0gYXR0ZW1wdCAwXG4gICAgUnVubmluZyAtLT4gVGltaW5nT3V0IDogSWYgdGFzayB0aW1lb3V0IGhhcyBlbGFwc2VkIGFuZCByZXRyeV9hdHRlbXB0cyA-PSBtYXhfcmV0cmllc1xuICAgIFRpbWluZ091dCAtLT4gVGltZWRPdXQgOiBJdCB0b3RhbCBub2RlIHRpbWVvdXQgaGFzIGVsYXBzZWRcbiAgICBSdW5uaW5nIC0tPiBSZXRyeWFibGVGYWlsdXJlIDogb24gcmV0cnlhYmxlIGZhaWx1cmVcbiAgICBSZXRyeWFibGVGYWlsdXJlIC0tPiBSdW5uaW5nIDogaWYgcmV0cnlfYXR0ZW1wdHMgPCBtYXhfcmV0cmllc1xuICAgIFJldHJ5YWJsZUZhaWx1cmUgLS0-IEZhaWxpbmcgOiByZXRyeV9hdHRlbXB0cyA-PSBtYXhfcmV0cmllc1xuICAgIEZhaWxpbmcgLS0-IEZhaWxlZFxuICAgIFJ1bm5pbmcgLS0-IFN1Y2NlZWRpbmcgOiBJbnRlcm5hbCBzdGF0ZVxuICAgIFN1Y2NlZWRpbmcgLS0-IFN1Y2NlZWRlZCA6IFVzZXIgb2JzZXJ2ZXMgdGhlIHRhc2sgYXMgc3VjY2VlZGVkXG4gICAgU3VjY2VlZGVkIC0tPiBbKl1cbiAgICBGYWlsZWQgLS0-IFsqXVxuIiwibWVybWFpZCI6e30sInVwZGF0ZUVkaXRvciI6ZmFsc2V9 + :alt: The State diagram above illustrates the various states through which a Node transitions. This is the core FSM for a Node. + +The state diagram above illustrates the various states through which a Node transitions. This is the core FSM for a Node. +From a user's point of view, a Workflow simply consists of a sequence of tasks. But to Flyte, a Workflow internally creates a meta entity called a + +Once a Workflow enters a ``Running`` state, it triggers the phantom ``start node`` of the workflow. The Start node is always the entry node of any workflow. The start node starts executing all its child-nodes using a modified DepthFirst Search algorithm recursively. + +Nodes can be of different types, as follows, but all the nodes traverse through the same transitions + +#. Start Node - Only exists during the execution and is not modeled in the core spec +#. :std:ref:`gen/pb-protodoc/flyteidl/core/workflow.proto:flyteidl.core.tasknode` +#. :std:ref:`gen/pb-protodoc/flyteidl/core/workflow.proto:flyteidl.core.branchnode` +#. :std:ref:`gen/pb-protodoc/flyteidl/core/workflow.proto:flyteidl.core.workflownode` +#. Dynamic node - which is just a task node that does not return outputs, but futures. +#. End Node - only exists during the execution and is not modeled in the core spec + +Every transition between states is recorded in Flyteadmin using :std:ref:`gen/pb-protodoc/flyteidl/event/event.proto:flyteidl.event.nodeexecutionevent` + +Every NodeExecutionEvent can have one of the :std:ref:`api_enum_flyteidl.core.nodeexecution.phase` + +.. note:: TODO add explanation for each phase + +The state machine specification for the illustration can be found `here `_ + +Task States +================ + +.. image:: https://mermaid.ink/img/eyJjb2RlIjoic3RhdGVEaWFncmFtLXYyXG4gICAgWypdIC0tPiBOb3RSZWFkeVxuICAgIFsqXSAtLT4gQWJvcnRlZCA6IEFib3J0ZWQgYnkgTm9kZUhhbmRsZXIgLSB0aW1lb3V0cywgZXh0cmVuYWwgYWJvcnQsIGV0Y1xuICAgIE5vdFJlYWR5IC0tPiBXYWl0aW5nRm9yUmVzb3VyY2VzIDogQmxvY2tlZCBvbiByZXNvdXJjZSBxdW90YSBvciByZXNvdXJjZSBwb29sIChvcHRpb25hbClcbiAgICBXYWl0aW5nRm9yUmVzb3VyY2VzIC0tPiBRdWV1ZWQgOiBIYXMgYmVlbiBzdWJtaXR0ZWQsIGJ1dCBoYXMgbm90IHN0YXJ0ZWQgKG9wdGlvbmFsKVxuICAgIFF1ZXVlZCAtLT4gSW5pdGlhbGl6aW5nIDogUHJlc3RhcnQgaW5pdGlhbGl6YXRpb24gKG9wdGlvbmFsKVxuICAgIEluaXRpYWxpemluZyAtLT4gUnVubmluZyA6IEFjdHVhbCBleGVjdXRpb24gb2YgdXNlciBjb2RlIGhhcyBzdGFydGVkXG4gICAgUnVubmluZyAtLT4gU3VjY2VzcyA6IFN1Y2Nlc3NmdWwgZXhlY3V0aW9uXG4gICAgUnVubmluZyAtLT4gUmV0cnlhYmxlRmFpbHVyZSA6IEZhaWxlZCB3aXRoIGEgcmV0cnlhYmxlIGVycm9yXG4gICAgUnVubmluZyAtLT4gUGVybWFuZW50RmFpbHVyZSA6IFVucmVjb3ZlcmFibGUgZmFpbHVyZSwgd2lsbCBzdG9wIGFsbCBleGVjdXRpb25cbiAgICBTdWNjZXNzIC0tPiBbKl1cbiAgICBSZXRyeWFibGVGYWlsdXJlIC0tPiBbKl1cbiAgICBQZXJtYW5lbnRGYWlsdXJlIC0tPiBbKl1cbiIsIm1lcm1haWQiOnt9LCJ1cGRhdGVFZGl0b3IiOmZhbHNlfQ + :alt: The State diagram above illustrates the various states through which a Task transitions. This is the core FSM for any Task in Flyte. + +The State diagram above illustrates the various states through which a Task transitions. + +Every transition between states is recorded in Flyteadmin using :std:ref:`gen/pb-protodoc/flyteidl/event/event.proto:flyteidl.event.taskexecutionevent` + +Every TaskExecutionEvent can have one of the :std:ref:`api_enum_flyteidl.core.taskexecution.phase` + +.. note:: TODO add explanation for each phase + +The state machine specification for the illustration can be found `here `_ From 430e2927ecd5eebc7aa5de82ec446eba14addb2f Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Tue, 13 Apr 2021 09:57:57 -0700 Subject: [PATCH 51/74] updated headings Signed-off-by: Haytham Abuelfutuh --- rsts/howto/authentication/index.rst | 139 +++++++++++++++++++++++----- 1 file changed, 117 insertions(+), 22 deletions(-) diff --git a/rsts/howto/authentication/index.rst b/rsts/howto/authentication/index.rst index fd04277add..c80824befe 100644 --- a/rsts/howto/authentication/index.rst +++ b/rsts/howto/authentication/index.rst @@ -1,14 +1,14 @@ .. _howto_authentication: -####################################### +############################# How to setup Authentication? -####################################### +############################# Flyte Admin ships with a canonical implementation of OAuth2, integrating seamlessly into an organization's existing identity provider. At Lyft, we use Okta as the IDP, but if you have issues integrating with another implementation of the OAuth server, please open an issue. -*********************** +*********** Components -*********************** +*********** While the most obvious interaction with the Flyte control plane is through the web based UI, there are other critical components of Flyte that also need to be considered. These components should be thought of as third-party services even though the Flyte codebase provides them. @@ -18,12 +18,10 @@ Principal amongst these is the Flyte CLI. This is the command-line entrypoint to The IDP application corresponding to the CLI will need to support PKCE. - Direct Client Access ==================== The gRPC client provided by the Flyte IDL, or direct calls to the HTTP endpoints on Admin from within a running script are ways that we have seen users hit the control plane directly. We generally discourage this behavior as it leads to a possible self-imposed DOS vector, as they are generally made from within a running workflow itself. For instance, a Flyte task can fetch the definition for a launch plan associated with a completely different workflow, and then launch an execution of it. This is not the correct way to launch one workflow from another but for the time being remains possible. - ***************** Swimlane Diagrams ***************** @@ -59,35 +57,59 @@ Flyte Admin Configuration ========================= Please refer to the `inline documentation `_ on the ``Config`` object in the ``auth`` package for a discussion on the settings required. +*********************** Example Configurations -====================== +*********************** + +OpenID Connect +=============== -Google IdP -########## +OpenID Connect allows users to authenticate to flyte in their browser using a familiar authentication provider (perhaps an organization-wide configured IdP). +Flyte supports connecting with external OIdC providers. Here are some examples for how to set these up: -1. Follow `Google Docs `__ on how to configure the IdP for OpenIDConnect. +Google OpenID Connect +---------------------- + +Follow `Google Docs `__ on how to configure the IdP for OpenIDConnect. .. note:: Make sure to create an OAuth2 Client Credential. The `client_id` and `client_secret` will be needed in the following steps. -2. Store the `client_secret` in a k8s secrt as follows: +Okta OpenID Connect +------------------- + +Okta supports OpenIDConnect protocol as well as allows the creation of custom OAuth2 Authorization Servers allowing it to act as both the user and apps IdP. +It offers better fine grained control on access policies, users' consent, and app management. + +1. If you don't already have an Okta account, sign up for one `here `__. +2. Create an app (choose Web for the platform) and OpenID Connect for the sign on method. +3. Add Login redirect URIs (e.g. http://localhost:30081/callback for sandbox or https:///callback) +4. OPTIONAL: Add logout redirect URIs (e.g. http://localhost:30081/logout for sandbox) +5. Note down the Client ID and Client Secret + +Apply configuration +=================== + +1. Store the `client_secret` in a k8s secrt as follows: .. prompt:: bash kubectl edit secret -n flyte flyte-admin-auth -Add a new key under `data`: +Add a new key under `stringData`: .. code-block:: yaml stringData: oidc_client_secret: from the previous step + data: + ... Save and close your editor. -3. Edit FlyteAdmin config to add `client_id` as follows: +2. Edit FlyteAdmin config to add `client_id`, `client_secret` and configure auth as follows: .. prompt:: bash @@ -99,26 +121,99 @@ This will output the name of the config map where the `client_id` need to go. kubectl edit configmap -n flyte -Find `client_id` and replace with the copied `client_id` +Follow the inline comments to make the necessary changes: .. code-block:: yaml - clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com - -Find `useAuth` and enable Auth enforcement: - -.. code-block:: yaml - - useAuth: true + server: + httpPort: 8088 + grpcPort: 8089 + grpcServerReflection: true + kube-config: /Users/haythamabuelfutuh/.kube/config + security: + secure: false + # 1. Enable Auth by turning this to true + useAuth: true + allowCors: true + allowedOrigins: + # Accepting all domains for Sandbox installation + - "*" + allowedHeaders: + - "Content-Type" + auth: + # 2. Update with the public facing url of flyte admin (e.g. https://flyte.mycompany.io/) + httpPublicUri: http://localhost:8088/ + userAuth: + openId: + # 3. Put the URL of the OpenID Connect provider. + # baseUrl: https://accounts.google.com/ # Uncomment for Google + baseUrl: https://dev-14186422.okta.com/oauth2/default # Okta with a custom Authorization Server + scopes: + - profile + - openid + # - offline_access # Uncomment if OIdC supports issuing refresh tokens. + # 4. Replace with the client id created for Flyte. + clientId: 0oakkheteNjCMERst5d6 + # 5. Replace with the public facing URL of flyte admin (e.g. https://flyte.mycompany.io/callback) + callbackUrl: "http://localhost:8088/callback" + # 6. Replace with the flyte console's URL (e.g. https://flyte.mycompany.io/console) + redirectUrl: "/api/v1/projects" Save and exit your editor. -4. Restart `flyteadmin` for the changes to take effect: +3. Restart `flyteadmin` for the changes to take effect: .. prompt:: bash kubectl rollout restart deployment/flyteadmin -n flyte +OAuth2 Authorization Server +=========================== + +Flyte Admin comes with a built-in authorization server that can be statically configured with a set of clients to request and act on behalf of the user. + +Okta IdP +-------- + +1. Under security -> API, click `Add Authorization Server`. Set the audience to the public URL of flyte admin (e.g. https://flyte.mycompany.io/). +2. Under `Access Policies`, click `Add New Access Policy` and walk through the wizard to allow access to the authorization server. +3. Under `Scopes`, click `Add Scope`. Set the name to `all` (required) and check `Require user consent for this scope` (recommended). + +Apply Configurations +-------------------- + +It's possible to direct flyte admin to use an external authorization server, however. To do so, edit the same config map once more and follow these changes: + +.. code-block:: yaml + + auth: + # 1. Update with the public facing url of flyte admin (e.g. https://flyte.mycompany.io/) + httpPublicUri: http://localhost:8088/ + appAuth: + # 1. Choose External if you will use an external Authorization Server (e.g. a Custom Authorization server in Okta) + # Choose Self (or omit the value) to use Flyte Admin's internal (albeit limited) Authorization Server. + authServerType: External + thirdPartyConfig: + flyteClient: + # 2. Replace with a new native client id provisioned in the custom authorization server + clientId: flytectl + redirectUri: https://localhost:53593/callback + # 3. "all" is a required scope and must be configured in the custom authorization server + scopes: + - offline + - all + userAuth: + openId: + # 4. Use the URL of your custom authorization server created above: + baseUrl: https://dev-14186422.okta.com/oauth2/auskngnn7uBViQq6b5d6 # Okta with a custom Authorization Server + scopes: + - profile + - openid + # - offline_access # Uncomment if OIdC supports issuing refresh tokens. + clientId: 0oakkheteNjCMERst5d6 + callbackUrl: "http://localhost:8088/callback" + redirectUrl: "/api/v1/projects" + ****** CI ****** From b90b878d0fe374fbe63363762da137fe5d55be52 Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Tue, 13 Apr 2021 14:11:04 -0700 Subject: [PATCH 52/74] fix google url Signed-off-by: Haytham Abuelfutuh --- rsts/howto/authentication/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rsts/howto/authentication/index.rst b/rsts/howto/authentication/index.rst index c80824befe..9a6b54a526 100644 --- a/rsts/howto/authentication/index.rst +++ b/rsts/howto/authentication/index.rst @@ -146,7 +146,7 @@ Follow the inline comments to make the necessary changes: userAuth: openId: # 3. Put the URL of the OpenID Connect provider. - # baseUrl: https://accounts.google.com/ # Uncomment for Google + # baseUrl: https://accounts.google.com # Uncomment for Google baseUrl: https://dev-14186422.okta.com/oauth2/default # Okta with a custom Authorization Server scopes: - profile From 19d3133ff95b4a41b02ad29309cc65454709b402 Mon Sep 17 00:00:00 2001 From: SandraGH5 <80421934+SandraGH5@users.noreply.github.com> Date: Tue, 13 Apr 2021 14:10:29 -0700 Subject: [PATCH 53/74] Update index.rst (#911) Signed-off-by: Haytham Abuelfutuh --- rsts/howto/authentication/index.rst | 42 ++++++++++++++++------------- 1 file changed, 23 insertions(+), 19 deletions(-) diff --git a/rsts/howto/authentication/index.rst b/rsts/howto/authentication/index.rst index 9a6b54a526..4b5927f090 100644 --- a/rsts/howto/authentication/index.rst +++ b/rsts/howto/authentication/index.rst @@ -1,8 +1,8 @@ .. _howto_authentication: -############################# -How to setup Authentication? -############################# +############################ +How to Set Up Authentication +############################ Flyte Admin ships with a canonical implementation of OAuth2, integrating seamlessly into an organization's existing identity provider. At Lyft, we use Okta as the IDP, but if you have issues integrating with another implementation of the OAuth server, please open an issue. @@ -14,13 +14,15 @@ While the most obvious interaction with the Flyte control plane is through the w Flyte CLI ========= -Principal amongst these is the Flyte CLI. This is the command-line entrypoint to Flyte Admin and is used by both administrators and users more comfortable in the command line, or are running in a headless OS. +This is the primary component. Flyte CLI is the command-line entrypoint to Flyte Admin and is used by both administrators and users more comfortable in the command line, or are running in a headless OS. The IDP application corresponding to the CLI will need to support PKCE. Direct Client Access ==================== -The gRPC client provided by the Flyte IDL, or direct calls to the HTTP endpoints on Admin from within a running script are ways that we have seen users hit the control plane directly. We generally discourage this behavior as it leads to a possible self-imposed DOS vector, as they are generally made from within a running workflow itself. For instance, a Flyte task can fetch the definition for a launch plan associated with a completely different workflow, and then launch an execution of it. This is not the correct way to launch one workflow from another but for the time being remains possible. +The gRPC client provided by the Flyte IDL, or direct calls to the HTTP endpoints on Admin from within a running script are ways that we have seen users hit the control plane directly. We generally discourage this course of action as it leads to a possible self-imposed DOS vector, which is generally made from within a running workflow itself. + +For instance, a Flyte task can fetch the definition for a launch plan associated with a completely different workflow, and then launch an execution of it. This is not the correct way to launch one workflow from another but for the time being remains possible. ***************** Swimlane Diagrams @@ -80,8 +82,8 @@ Follow `Google Docs `__. 2. Create an app (choose Web for the platform) and OpenID Connect for the sign on method. @@ -115,7 +117,7 @@ Save and close your editor. kubectl deploy -n flyte flyteadmin -o yaml | grep "name: flyte-admin-config" -This will output the name of the config map where the `client_id` need to go. +This will output the name of the config map where the `client_id` needs to go. .. prompt:: bash @@ -152,7 +154,7 @@ Follow the inline comments to make the necessary changes: - profile - openid # - offline_access # Uncomment if OIdC supports issuing refresh tokens. - # 4. Replace with the client id created for Flyte. + # 4. Replace with the client ID created for Flyte. clientId: 0oakkheteNjCMERst5d6 # 5. Replace with the public facing URL of flyte admin (e.g. https://flyte.mycompany.io/callback) callbackUrl: "http://localhost:8088/callback" @@ -182,12 +184,12 @@ Okta IdP Apply Configurations -------------------- -It's possible to direct flyte admin to use an external authorization server, however. To do so, edit the same config map once more and follow these changes: +It is possible to direct flyte admin to use an external authorization server. To do so, edit the same config map once more and follow these changes: .. code-block:: yaml auth: - # 1. Update with the public facing url of flyte admin (e.g. https://flyte.mycompany.io/) + # 1. Update with the public facing URL of flyte admin (e.g. https://flyte.mycompany.io/) httpPublicUri: http://localhost:8088/ appAuth: # 1. Choose External if you will use an external Authorization Server (e.g. a Custom Authorization server in Okta) @@ -195,7 +197,7 @@ It's possible to direct flyte admin to use an external authorization server, how authServerType: External thirdPartyConfig: flyteClient: - # 2. Replace with a new native client id provisioned in the custom authorization server + # 2. Replace with a new native client ID provisioned in the custom authorization server clientId: flytectl redirectUri: https://localhost:53593/callback # 3. "all" is a required scope and must be configured in the custom authorization server @@ -214,24 +216,26 @@ It's possible to direct flyte admin to use an external authorization server, how callbackUrl: "http://localhost:8088/callback" redirectUrl: "/api/v1/projects" -****** -CI -****** +*************************** +Continuous Integration - CI +*************************** If your organization does any automated registration, then you'll need to authenticate with the `basic authentication `_ flow (username and password effectively) as CI systems are generally not suitable OAuth resource owners. After retrieving an access token from the IDP, you can send it along to Flyte Admin as usual. -Flytekit configuration variables are automatically designed to look up values from relevant environment variables. To aid with continuous integration use-cases however, Flytekit configuration can also reference other environment variables. For instance, if your CI system is not capable of setting custom environment variables like ``FLYTE_CREDENTIALS_CLIENT_SECRET`` but does set the necessary settings under a different variable, you may use ``export FLYTE_CREDENTIALS_CLIENT_SECRET_FROM_ENV_VAR=OTHER_ENV_VARIABLE`` to redirect the lookup. A ``FLYTE_CREDENTIALS_CLIENT_SECRET_FROM_FILE`` redirect is available as well, where the value should be the full path to the file containing the value for the configuration setting, in this case, the client secret. We found this redirect behavior necessary when setting up registration within our own CI pipelines. +Flytekit configuration variables are automatically designed to look up values from relevant environment variables. However, to aid with continuous integration use-cases, Flytekit configuration can also reference other environment variables. + +For instance, if your CI system is not capable of setting custom environment variables like ``FLYTE_CREDENTIALS_CLIENT_SECRET`` but does set the necessary settings under a different variable, you may use ``export FLYTE_CREDENTIALS_CLIENT_SECRET_FROM_ENV_VAR=OTHER_ENV_VARIABLE`` to redirect the lookup. A ``FLYTE_CREDENTIALS_CLIENT_SECRET_FROM_FILE`` redirect is available as well, where the value should be the full path to the file containing the value for the configuration setting, in this case, the client secret. We found this redirect behavior necessary when setting up registration within our own CI pipelines. -The following is a listing of the Flytekit configuration values we set in CI, along with a brief explanation where relevant. +The following is a listing of the Flytekit configuration values we set in CI, along with a brief explanation. * ``FLYTE_CREDENTIALS_CLIENT_ID`` and ``FLYTE_CREDENTIALS_CLIENT_SECRET`` - When using basic authentication, this is the username and password + When using basic authentication, this is the username and password. * ``export FLYTE_CREDENTIALS_AUTH_MODE=basic`` This tells the SDK to use basic authentication. If not set, Flytekit will assume you want to use the standard OAuth based three-legged flow. * ``export FLYTE_CREDENTIALS_AUTHORIZATION_METADATA_KEY=text`` At Lyft, we set this to conform to this `header config `_ on the Admin side. * ``export FLYTE_CREDENTIALS_SCOPE=text`` - When using basic authentication, you'll need to specify a scope to the IDP (instead of ``openid``, as that's only for OAuth). Set that here. + When using basic authentication, you'll need to specify a scope to the IDP (instead of ``openid``, which is only for OAuth). Set that here. * ``export FLYTE_PLATFORM_AUTH=True`` Set this to force Flytekit to use authentication, even if not required by Admin. This is useful as you're rolling out the requirement. From bf2dbcfabd4a54187b4ecb357ea739ec1910ee24 Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Tue, 13 Apr 2021 14:44:05 -0700 Subject: [PATCH 54/74] wip Signed-off-by: Haytham Abuelfutuh --- deployment/eks/flyte_generated.yaml | 9 +++++---- deployment/gcp/flyte_generated.yaml | 9 +++++---- deployment/sandbox/flyte_generated.yaml | 9 +++++---- deployment/test/flyte_generated.yaml | 7 ++++--- .../single_cluster/headless/config/admin/server.yaml | 3 ++- rsts/howto/authentication/index.rst | 9 +++++---- 6 files changed, 26 insertions(+), 20 deletions(-) diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index 6d34a7ba16..c0f350bd96 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -8157,13 +8157,14 @@ data: grpcPort: 8089 security: secure: false - useAuth: true + useAuth: false allowCors: true allowedOrigins: # Accepting all domains for Sandbox installation - "*" allowedHeaders: - "Content-Type" + - "flyte-authorization" secrets: secrets-prefix: /etc/secrets/auth/ auth: @@ -8229,7 +8230,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-7g4m67fbc4 + name: flyte-admin-config-6dg952hkfh namespace: flyte --- apiVersion: v1 @@ -8876,7 +8877,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-7g4m67fbc4 + name: flyte-admin-config-6dg952hkfh name: config-volume - configMap: name: clusterresource-template-tkdkkt4cb5 @@ -9161,7 +9162,7 @@ spec: name: clusterresource-template-tkdkkt4cb5 name: resource-templates - configMap: - name: flyte-admin-config-7g4m67fbc4 + name: flyte-admin-config-6dg952hkfh name: config-volume - name: db-pass secret: diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index e0cf7c0129..251e681e50 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -8148,13 +8148,14 @@ data: grpcPort: 8089 security: secure: false - useAuth: true + useAuth: false allowCors: true allowedOrigins: # Accepting all domains for Sandbox installation - "*" allowedHeaders: - "Content-Type" + - "flyte-authorization" secrets: secrets-prefix: /etc/secrets/auth/ auth: @@ -8220,7 +8221,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-27ggcgm724 + name: flyte-admin-config-mchkdmtm5k namespace: flyte --- apiVersion: v1 @@ -8908,7 +8909,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-27ggcgm724 + name: flyte-admin-config-mchkdmtm5k name: config-volume - configMap: name: clusterresource-template-tkdkkt4cb5 @@ -9193,7 +9194,7 @@ spec: name: clusterresource-template-tkdkkt4cb5 name: resource-templates - configMap: - name: flyte-admin-config-27ggcgm724 + name: flyte-admin-config-mchkdmtm5k name: config-volume - name: db-pass secret: diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index 1d4bb10335..59768f9b39 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -2164,13 +2164,14 @@ data: grpcPort: 8089 security: secure: false - useAuth: true + useAuth: false allowCors: true allowedOrigins: # Accepting all domains for Sandbox installation - "*" allowedHeaders: - "Content-Type" + - "flyte-authorization" secrets: secrets-prefix: /etc/secrets/auth/ auth: @@ -2228,7 +2229,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-tkg6c6ft28 + name: flyte-admin-config-bkfh292hd6 namespace: flyte --- apiVersion: v1 @@ -3020,7 +3021,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-tkg6c6ft28 + name: flyte-admin-config-bkfh292hd6 name: config-volume - name: db-pass secret: @@ -3429,7 +3430,7 @@ spec: name: clusterresource-template-dtg8ff28mt name: resource-templates - configMap: - name: flyte-admin-config-tkg6c6ft28 + name: flyte-admin-config-bkfh292hd6 name: config-volume - name: db-pass secret: diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index f673d1cc76..72c260d5cf 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -312,13 +312,14 @@ data: grpcPort: 8089 security: secure: false - useAuth: true + useAuth: false allowCors: true allowedOrigins: # Accepting all domains for Sandbox installation - "*" allowedHeaders: - "Content-Type" + - "flyte-authorization" secrets: secrets-prefix: /etc/secrets/auth/ auth: @@ -376,7 +377,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-42thh57bff + name: flyte-admin-config-f9cbgdhcbh namespace: flyte --- apiVersion: v1 @@ -874,7 +875,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-42thh57bff + name: flyte-admin-config-f9cbgdhcbh name: config-volume - name: db-pass secret: diff --git a/kustomize/base/single_cluster/headless/config/admin/server.yaml b/kustomize/base/single_cluster/headless/config/admin/server.yaml index 3607631073..77948fd719 100644 --- a/kustomize/base/single_cluster/headless/config/admin/server.yaml +++ b/kustomize/base/single_cluster/headless/config/admin/server.yaml @@ -3,13 +3,14 @@ server: grpcPort: 8089 security: secure: false - useAuth: true + useAuth: false allowCors: true allowedOrigins: # Accepting all domains for Sandbox installation - "*" allowedHeaders: - "Content-Type" + - "flyte-authorization" secrets: secrets-prefix: /etc/secrets/auth/ auth: diff --git a/rsts/howto/authentication/index.rst b/rsts/howto/authentication/index.rst index 4b5927f090..54b765a0d4 100644 --- a/rsts/howto/authentication/index.rst +++ b/rsts/howto/authentication/index.rst @@ -142,9 +142,10 @@ Follow the inline comments to make the necessary changes: - "*" allowedHeaders: - "Content-Type" + - "flyte-athorization" auth: # 2. Update with the public facing url of flyte admin (e.g. https://flyte.mycompany.io/) - httpPublicUri: http://localhost:8088/ + httpPublicUri: http://localhost:30081/ userAuth: openId: # 3. Put the URL of the OpenID Connect provider. @@ -157,7 +158,7 @@ Follow the inline comments to make the necessary changes: # 4. Replace with the client ID created for Flyte. clientId: 0oakkheteNjCMERst5d6 # 5. Replace with the public facing URL of flyte admin (e.g. https://flyte.mycompany.io/callback) - callbackUrl: "http://localhost:8088/callback" + callbackUrl: "http://localhost:30081/callback" # 6. Replace with the flyte console's URL (e.g. https://flyte.mycompany.io/console) redirectUrl: "/api/v1/projects" @@ -190,7 +191,7 @@ It is possible to direct flyte admin to use an external authorization server. To auth: # 1. Update with the public facing URL of flyte admin (e.g. https://flyte.mycompany.io/) - httpPublicUri: http://localhost:8088/ + httpPublicUri: http://localhost:30081/ appAuth: # 1. Choose External if you will use an external Authorization Server (e.g. a Custom Authorization server in Okta) # Choose Self (or omit the value) to use Flyte Admin's internal (albeit limited) Authorization Server. @@ -213,7 +214,7 @@ It is possible to direct flyte admin to use an external authorization server. To - openid # - offline_access # Uncomment if OIdC supports issuing refresh tokens. clientId: 0oakkheteNjCMERst5d6 - callbackUrl: "http://localhost:8088/callback" + callbackUrl: "http://localhost:30081/callback" redirectUrl: "/api/v1/projects" *************************** From bf43edf556decc025e8391da357aeda45f812ebf Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Thu, 15 Apr 2021 15:30:55 -0700 Subject: [PATCH 55/74] Break up Auth article Signed-off-by: Haytham Abuelfutuh --- rsts/howto/authentication/index.rst | 210 ++---------------- rsts/howto/authentication/migration.rst | 7 + rsts/howto/authentication/setup.rst | 276 ++++++++++++++++++++++++ 3 files changed, 295 insertions(+), 198 deletions(-) create mode 100644 rsts/howto/authentication/migration.rst create mode 100644 rsts/howto/authentication/setup.rst diff --git a/rsts/howto/authentication/index.rst b/rsts/howto/authentication/index.rst index 54b765a0d4..8e61066b44 100644 --- a/rsts/howto/authentication/index.rst +++ b/rsts/howto/authentication/index.rst @@ -1,10 +1,18 @@ .. _howto_authentication: -############################ -How to Set Up Authentication -############################ +####################### +Authentication in Flyte +####################### -Flyte Admin ships with a canonical implementation of OAuth2, integrating seamlessly into an organization's existing identity provider. At Lyft, we use Okta as the IDP, but if you have issues integrating with another implementation of the OAuth server, please open an issue. +Flyte ships with a canonical implementation of OpenIDConnect client and OAuth2 Server, integrating seamlessly into an organization's existing identity provider. + +.. toctree:: + :maxdepth: 1 + :caption: Setting up Flyte Authentication + :name: howtoprovguidestoc + + setup + migration *********** Components @@ -47,200 +55,6 @@ https://swimlanes.io/d/q64OxuoxT :width: 600 :alt: Flyte CLI Swimlane -************* -Configuration -************* - -IDP Configuration -================= -Flyte Admin will require that the application in your identity provider be configured without PKCE and with a client secret. It should also be configured with a refresh token. - -Flyte Admin Configuration -========================= -Please refer to the `inline documentation `_ on the ``Config`` object in the ``auth`` package for a discussion on the settings required. - -*********************** -Example Configurations -*********************** - -OpenID Connect -=============== - -OpenID Connect allows users to authenticate to flyte in their browser using a familiar authentication provider (perhaps an organization-wide configured IdP). -Flyte supports connecting with external OIdC providers. Here are some examples for how to set these up: - -Google OpenID Connect ----------------------- - -Follow `Google Docs `__ on how to configure the IdP for OpenIDConnect. - -.. note:: - - Make sure to create an OAuth2 Client Credential. The `client_id` and `client_secret` will be needed in the following - steps. - -Okta OpenID Connect -------------------- - -Okta supports OpenIDConnect protocol and the creation of custom OAuth2 Authorization Servers, allowing it to act as both the user and apps IdP. -It offers more detailed control on access policies, users' consent, and app management. - -1. If you don't already have an Okta account, sign up for one `here `__. -2. Create an app (choose Web for the platform) and OpenID Connect for the sign on method. -3. Add Login redirect URIs (e.g. http://localhost:30081/callback for sandbox or https:///callback) -4. OPTIONAL: Add logout redirect URIs (e.g. http://localhost:30081/logout for sandbox) -5. Note down the Client ID and Client Secret - -Apply configuration -=================== - -1. Store the `client_secret` in a k8s secrt as follows: - -.. prompt:: bash - - kubectl edit secret -n flyte flyte-admin-auth - -Add a new key under `stringData`: - -.. code-block:: yaml - - stringData: - oidc_client_secret: from the previous step - data: - ... - -Save and close your editor. - -2. Edit FlyteAdmin config to add `client_id`, `client_secret` and configure auth as follows: - -.. prompt:: bash - - kubectl deploy -n flyte flyteadmin -o yaml | grep "name: flyte-admin-config" - -This will output the name of the config map where the `client_id` needs to go. - -.. prompt:: bash - - kubectl edit configmap -n flyte - -Follow the inline comments to make the necessary changes: - -.. code-block:: yaml - - server: - httpPort: 8088 - grpcPort: 8089 - grpcServerReflection: true - kube-config: /Users/haythamabuelfutuh/.kube/config - security: - secure: false - # 1. Enable Auth by turning this to true - useAuth: true - allowCors: true - allowedOrigins: - # Accepting all domains for Sandbox installation - - "*" - allowedHeaders: - - "Content-Type" - - "flyte-athorization" - auth: - # 2. Update with the public facing url of flyte admin (e.g. https://flyte.mycompany.io/) - httpPublicUri: http://localhost:30081/ - userAuth: - openId: - # 3. Put the URL of the OpenID Connect provider. - # baseUrl: https://accounts.google.com # Uncomment for Google - baseUrl: https://dev-14186422.okta.com/oauth2/default # Okta with a custom Authorization Server - scopes: - - profile - - openid - # - offline_access # Uncomment if OIdC supports issuing refresh tokens. - # 4. Replace with the client ID created for Flyte. - clientId: 0oakkheteNjCMERst5d6 - # 5. Replace with the public facing URL of flyte admin (e.g. https://flyte.mycompany.io/callback) - callbackUrl: "http://localhost:30081/callback" - # 6. Replace with the flyte console's URL (e.g. https://flyte.mycompany.io/console) - redirectUrl: "/api/v1/projects" - -Save and exit your editor. - -3. Restart `flyteadmin` for the changes to take effect: - -.. prompt:: bash - - kubectl rollout restart deployment/flyteadmin -n flyte - -OAuth2 Authorization Server -=========================== - -Flyte Admin comes with a built-in authorization server that can be statically configured with a set of clients to request and act on behalf of the user. - -Okta IdP --------- - -1. Under security -> API, click `Add Authorization Server`. Set the audience to the public URL of flyte admin (e.g. https://flyte.mycompany.io/). -2. Under `Access Policies`, click `Add New Access Policy` and walk through the wizard to allow access to the authorization server. -3. Under `Scopes`, click `Add Scope`. Set the name to `all` (required) and check `Require user consent for this scope` (recommended). - -Apply Configurations --------------------- - -It is possible to direct flyte admin to use an external authorization server. To do so, edit the same config map once more and follow these changes: - -.. code-block:: yaml - - auth: - # 1. Update with the public facing URL of flyte admin (e.g. https://flyte.mycompany.io/) - httpPublicUri: http://localhost:30081/ - appAuth: - # 1. Choose External if you will use an external Authorization Server (e.g. a Custom Authorization server in Okta) - # Choose Self (or omit the value) to use Flyte Admin's internal (albeit limited) Authorization Server. - authServerType: External - thirdPartyConfig: - flyteClient: - # 2. Replace with a new native client ID provisioned in the custom authorization server - clientId: flytectl - redirectUri: https://localhost:53593/callback - # 3. "all" is a required scope and must be configured in the custom authorization server - scopes: - - offline - - all - userAuth: - openId: - # 4. Use the URL of your custom authorization server created above: - baseUrl: https://dev-14186422.okta.com/oauth2/auskngnn7uBViQq6b5d6 # Okta with a custom Authorization Server - scopes: - - profile - - openid - # - offline_access # Uncomment if OIdC supports issuing refresh tokens. - clientId: 0oakkheteNjCMERst5d6 - callbackUrl: "http://localhost:30081/callback" - redirectUrl: "/api/v1/projects" - -*************************** -Continuous Integration - CI -*************************** - -If your organization does any automated registration, then you'll need to authenticate with the `basic authentication `_ flow (username and password effectively) as CI systems are generally not suitable OAuth resource owners. After retrieving an access token from the IDP, you can send it along to Flyte Admin as usual. - -Flytekit configuration variables are automatically designed to look up values from relevant environment variables. However, to aid with continuous integration use-cases, Flytekit configuration can also reference other environment variables. - -For instance, if your CI system is not capable of setting custom environment variables like ``FLYTE_CREDENTIALS_CLIENT_SECRET`` but does set the necessary settings under a different variable, you may use ``export FLYTE_CREDENTIALS_CLIENT_SECRET_FROM_ENV_VAR=OTHER_ENV_VARIABLE`` to redirect the lookup. A ``FLYTE_CREDENTIALS_CLIENT_SECRET_FROM_FILE`` redirect is available as well, where the value should be the full path to the file containing the value for the configuration setting, in this case, the client secret. We found this redirect behavior necessary when setting up registration within our own CI pipelines. - -The following is a listing of the Flytekit configuration values we set in CI, along with a brief explanation. - -* ``FLYTE_CREDENTIALS_CLIENT_ID`` and ``FLYTE_CREDENTIALS_CLIENT_SECRET`` - When using basic authentication, this is the username and password. -* ``export FLYTE_CREDENTIALS_AUTH_MODE=basic`` - This tells the SDK to use basic authentication. If not set, Flytekit will assume you want to use the standard OAuth based three-legged flow. -* ``export FLYTE_CREDENTIALS_AUTHORIZATION_METADATA_KEY=text`` - At Lyft, we set this to conform to this `header config `_ on the Admin side. -* ``export FLYTE_CREDENTIALS_SCOPE=text`` - When using basic authentication, you'll need to specify a scope to the IDP (instead of ``openid``, which is only for OAuth). Set that here. -* ``export FLYTE_PLATFORM_AUTH=True`` - Set this to force Flytekit to use authentication, even if not required by Admin. This is useful as you're rolling out the requirement. - - ********** References ********** diff --git a/rsts/howto/authentication/migration.rst b/rsts/howto/authentication/migration.rst new file mode 100644 index 0000000000..27bd22aeba --- /dev/null +++ b/rsts/howto/authentication/migration.rst @@ -0,0 +1,7 @@ +.. _howto_authentication_migrate: + +################################################### +How to migrate your authentication config (pre 0.X) +################################################### + + diff --git a/rsts/howto/authentication/setup.rst b/rsts/howto/authentication/setup.rst new file mode 100644 index 0000000000..9a1f0461fd --- /dev/null +++ b/rsts/howto/authentication/setup.rst @@ -0,0 +1,276 @@ +.. _howto_authentication_setup: + +############################ +How to Set Up Authentication +############################ + +***************** +IdP Configuration +***************** +Flyte Admin requires that the application in your identity provider be configured as a web client (i.e. with a client secret). We recommend allowing the application to be issued a refresh token to avoid interrupting user's flow by redirecting to the IdP frequently. + +************************* +Flyte Admin Configuration +************************* +Please refer to the `inline documentation `_ on the ``Config`` object in the ``auth`` package for a discussion on the settings required. + +********************** +Example Configurations +********************** + +Here you can find canonical examples for how to set up some of the common IdPs to secure your flyte services. OpenID Connect enables users to authenticate, in the +browser, with an existing IdP. Flyte also allows connecting to an external OAuth2 Authorization Server to allow centerally managed 3rd party app access. + +OpenID Connect +=============== + +OpenID Connect allows users to authenticate to flyte in their browser using a familiar authentication provider (perhaps an organization-wide configured IdP). +Flyte supports connecting with external OIdC providers. Here are some examples for how to set these up: + +Google OpenID Connect +--------------------- + +Follow `Google Docs `__ on how to configure the IdP for OpenIDConnect. + +.. note:: + + Make sure to create an OAuth2 Client Credential. The `client_id` and `client_secret` will be needed in the following + steps. + +Github OpenID Connect +--------------------- + + + +Okta OpenID Connect +------------------- + +Okta supports OpenID Connect protocol and the creation of custom OAuth2 Authorization Servers, allowing it to act as both the user and apps IdP. +It offers more detailed control on access policies, users' consent, and app management. + +1. If you don't already have an Okta account, sign up for one `here `__. +2. Create an app (choose Web for the platform) and OpenID Connect for the sign on method. +3. Add Login redirect URIs (e.g. http://localhost:30081/callback for sandbox or https:///callback) +4. OPTIONAL: Add logout redirect URIs (e.g. http://localhost:30081/logout for sandbox) +5. Note down the Client ID and Client Secret + +KeyCloak OpenID Connect +------------------------ + +`KeyCloak `__ is an open source solution for authentication, it supports both OpenID Connect and OAuth2 protocols (among others). +KeyCloak can be configured to be both the OpenID Connect and OAuth2 Authorization Server provider for flyte. + + + +Apply configuration +------------------- + +1. Store the `client_secret` in a k8s secrt as follows: + +.. prompt:: bash + + kubectl edit secret -n flyte flyte-admin-auth + +Add a new key under `stringData`: + +.. code-block:: yaml + + stringData: + oidc_client_secret: from the previous step + data: + ... + +Save and close your editor. + +2. Edit FlyteAdmin config to add `client_id` and configure auth as follows: + +.. prompt:: bash + + kubectl deploy -n flyte flyteadmin -o yaml | grep "name: flyte-admin-config" + +This will output the name of the config map where the `client_id` needs to go. + +.. prompt:: bash + + kubectl edit configmap -n flyte + +Follow the inline comments to make the necessary changes: + +.. code-block:: yaml + + server: + httpPort: 8088 + grpcPort: 8089 + grpcServerReflection: true + kube-config: /Users/haythamabuelfutuh/.kube/config + security: + secure: false + # 1. Enable Auth by turning this to true + useAuth: true + allowCors: true + allowedOrigins: + # Accepting all domains for Sandbox installation + - "*" + allowedHeaders: + - "Content-Type" + - "flyte-athorization" + auth: + # 2. Update with the public facing url of flyte admin (e.g. https://flyte.mycompany.io/) + httpPublicUri: http://localhost:30081/ + userAuth: + openId: + # 3. Put the URL of the OpenID Connect provider. + # baseUrl: https://accounts.google.com # Uncomment for Google + baseUrl: https://dev-14186422.okta.com/oauth2/default # Okta with a custom Authorization Server + scopes: + - profile + - openid + # - offline_access # Uncomment if OIdC supports issuing refresh tokens. + # 4. Replace with the client ID created for Flyte. + clientId: 0oakkheteNjCMERst5d6 + # 5. Replace with the public facing URL of flyte admin (e.g. https://flyte.mycompany.io/callback) + callbackUrl: "http://localhost:30081/callback" + # 6. Replace with the flyte console's URL (e.g. https://flyte.mycompany.io/console) + redirectUrl: "/api/v1/projects" + +Save and exit your editor. + +3. Restart `flyteadmin` for the changes to take effect: + +.. prompt:: bash + + kubectl rollout restart deployment/flyteadmin -n flyte + +OAuth2 Authorization Server +=========================== + +An OAuth2 Authorization Server allows external clients to request to authenticate and act on behalf of users (or as their own identities). Having +an OAuth2 Authorization Server enables flyte administrators control over which apps can be installed and what scopes (i.e. what privileges can they assume), +they are allowed to request or be granted. + +Flyte comes with a built-in authorization server that can be statically configured with a set of clients to request and act on behalf of the user. +The default clients are defined `here `__ +and the corresponding section can be modified through configs. + +To set up an external OAuth2 Authorization Server, please follow the instructions below: + +Okta IdP +-------- + +1. Under security -> API, click `Add Authorization Server`. Set the audience to the public URL of flyte admin (e.g. https://flyte.mycompany.io/). +2. Under `Access Policies`, click `Add New Access Policy` and walk through the wizard to allow access to the authorization server. +3. Under `Scopes`, click `Add Scope`. Set the name to `all` (required) and check `Require user consent for this scope` (recommended). +4. Create 2 apps (for fltyectl and flytepropeller) to enable these clients to communicate with the service. + Flytectl should be created as a `native client`. + FlytePropeller should be created as an `OAuth Service` and note the client ID and client Secrets provided. + +KeyCloak IdP +------------ + +`KeyCloak `__ is an open source solution for authentication, it supports both OpenID Connect and OAuth2 protocols (among others). +KeyCloak can be configured to be both the OpenID Connect and OAuth2 Authorization Server provider for flyte. + + + +Apply Configurations +-------------------- + +1. It is possible to direct flyte admin to use an external authorization server. To do so, edit the same config map once more and follow these changes: + + .. code-block:: yaml + + auth: + # 1. Update with the public facing URL of flyte admin (e.g. https://flyte.mycompany.io/) + httpPublicUri: http://localhost:30081/ + appAuth: + # 1. Choose External if you will use an external Authorization Server (e.g. a Custom Authorization server in Okta) + # Choose Self (or omit the value) to use Flyte Admin's internal (albeit limited) Authorization Server. + authServerType: External + thirdPartyConfig: + flyteClient: + # 2. Replace with a new native client ID provisioned in the custom authorization server + clientId: flytectl + redirectUri: https://localhost:53593/callback + # 3. "all" is a required scope and must be configured in the custom authorization server + scopes: + - offline + - all + userAuth: + openId: + # 4. Use the URL of your custom authorization server created above: + baseUrl: https://dev-14186422.okta.com/oauth2/auskngnn7uBViQq6b5d6 # Okta with a custom Authorization Server + scopes: + - profile + - openid + # - offline_access # Uncomment if OIdC supports issuing refresh tokens. + clientId: 0oakkheteNjCMERst5d6 + callbackUrl: "http://localhost:30081/callback" + redirectUrl: "/api/v1/projects" + +1. Store flyte propeller's `client_secret` in a k8s secrt as follows: + +.. prompt:: bash + + kubectl edit secret -n flyte flyte-propeller-auth + +Add a new key under `stringData`: + +.. code-block:: yaml + + stringData: + client_secret: from the previous step + data: + ... + +Save and close your editor. + +2. Edit FlytePropeller config to add `client_id` and configure auth as follows: + +.. prompt:: bash + + kubectl deploy -n flyte flytepropeller -o yaml | grep "name: flyte-propeller-config" + +This will output the name of the config map where the `client_id` needs to go. + +.. prompt:: bash + + kubectl edit configmap -n flyte + +Follow the inline comments to make the necessary changes: + +.. code-block:: yaml + +admin: + # 1. Turn to true + useAuth: true + # 2. Replace with the client_id provided by the OAuth2 Authorization Server above. + clientId: flytepropeller + # 3. Replace with the OAuth2 authorization server url provided in the previous step. + authorizationServerUrl: http://localhost:30081/ + +Close the editor + +*************************** +Continuous Integration - CI +*************************** + +If your organization does any automated registration, then you'll need to authenticate with the `basic authentication `_ flow (username and password effectively) as CI systems are generally not suitable OAuth resource owners. After retrieving an access token from the IDP, you can send it along to Flyte Admin as usual. + +Flytekit configuration variables are automatically designed to look up values from relevant environment variables. However, to aid with continuous integration use-cases, Flytekit configuration can also reference other environment variables. + +For instance, if your CI system is not capable of setting custom environment variables like ``FLYTE_CREDENTIALS_CLIENT_SECRET`` but does set the necessary settings under a different variable, you may use ``export FLYTE_CREDENTIALS_CLIENT_SECRET_FROM_ENV_VAR=OTHER_ENV_VARIABLE`` to redirect the lookup. A ``FLYTE_CREDENTIALS_CLIENT_SECRET_FROM_FILE`` redirect is available as well, where the value should be the full path to the file containing the value for the configuration setting, in this case, the client secret. We found this redirect behavior necessary when setting up registration within our own CI pipelines. + +The following is a listing of the Flytekit configuration values we set in CI, along with a brief explanation. + +* ``FLYTE_CREDENTIALS_CLIENT_ID`` and ``FLYTE_CREDENTIALS_CLIENT_SECRET`` + When using basic authentication, this is the username and password. +* ``export FLYTE_CREDENTIALS_AUTH_MODE=basic`` + This tells the SDK to use basic authentication. If not set, Flytekit will assume you want to use the standard OAuth based three-legged flow. +* ``export FLYTE_CREDENTIALS_AUTHORIZATION_METADATA_KEY=text`` + At Lyft, we set this to conform to this `header config `_ on the Admin side. +* ``export FLYTE_CREDENTIALS_SCOPE=text`` + When using basic authentication, you'll need to specify a scope to the IDP (instead of ``openid``, which is only for OAuth). Set that here. +* ``export FLYTE_PLATFORM_AUTH=True`` + Set this to force Flytekit to use authentication, even if not required by Admin. This is useful as you're rolling out the requirement. + + From bdd242ad0d1f5306f2b9c2d88a2c0741a2178797 Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Mon, 19 Apr 2021 17:24:25 -0700 Subject: [PATCH 56/74] doc updates Signed-off-by: Haytham Abuelfutuh --- kustomize/base/propeller/auth_secret.yaml | 6 ++ kustomize/base/propeller/deployment.yaml | 47 ++++++++------ kustomize/base/propeller/kustomization.yaml | 1 + rsts/howto/authentication/setup.rst | 71 ++++++++++----------- 4 files changed, 66 insertions(+), 59 deletions(-) create mode 100644 kustomize/base/propeller/auth_secret.yaml diff --git a/kustomize/base/propeller/auth_secret.yaml b/kustomize/base/propeller/auth_secret.yaml new file mode 100644 index 0000000000..7bf797efa1 --- /dev/null +++ b/kustomize/base/propeller/auth_secret.yaml @@ -0,0 +1,6 @@ +apiVersion: v1 +kind: Secret +metadata: + name: flyte-propeller-auth + namespace: flyte +type: Opaque diff --git a/kustomize/base/propeller/deployment.yaml b/kustomize/base/propeller/deployment.yaml index e3e33240b8..c216295b60 100644 --- a/kustomize/base/propeller/deployment.yaml +++ b/kustomize/base/propeller/deployment.yaml @@ -23,25 +23,30 @@ spec: spec: serviceAccountName: flytepropeller volumes: - - name: config-volume - configMap: - name: flyte-propeller-config - containers: - - name: flytepropeller - image: flytepropeller:v0.7.1 - command: - - flytepropeller - args: - - --config - - /etc/flyte/config/*.yaml - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - imagePullPolicy: IfNotPresent - ports: - - containerPort: 10254 - volumeMounts: - name: config-volume - mountPath: /etc/flyte/config + configMap: + name: flyte-propeller-config + - name: auth + secret: + secretName: flyte-propeller-auth + containers: + - name: flytepropeller + image: flytepropeller:v0.7.1 + command: + - flytepropeller + args: + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + imagePullPolicy: IfNotPresent + ports: + - containerPort: 10254 + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + - name: auth + mountPath: /etc/secrets/ diff --git a/kustomize/base/propeller/kustomization.yaml b/kustomize/base/propeller/kustomization.yaml index 408244dcaf..d21d0b929c 100644 --- a/kustomize/base/propeller/kustomization.yaml +++ b/kustomize/base/propeller/kustomization.yaml @@ -1,4 +1,5 @@ resources: + - auth_secret.yaml - deployment.yaml - rbac.yaml - webhook.yaml diff --git a/rsts/howto/authentication/setup.rst b/rsts/howto/authentication/setup.rst index 9a1f0461fd..23700c6290 100644 --- a/rsts/howto/authentication/setup.rst +++ b/rsts/howto/authentication/setup.rst @@ -105,7 +105,7 @@ Follow the inline comments to make the necessary changes: kube-config: /Users/haythamabuelfutuh/.kube/config security: secure: false - # 1. Enable Auth by turning this to true + # 1. Enable Auth by turning useAuth to true useAuth: true allowCors: true allowedOrigins: @@ -115,23 +115,17 @@ Follow the inline comments to make the necessary changes: - "Content-Type" - "flyte-athorization" auth: - # 2. Update with the public facing url of flyte admin (e.g. https://flyte.mycompany.io/) - httpPublicUri: http://localhost:30081/ userAuth: openId: - # 3. Put the URL of the OpenID Connect provider. + # 2. Put the URL of the OpenID Connect provider. # baseUrl: https://accounts.google.com # Uncomment for Google baseUrl: https://dev-14186422.okta.com/oauth2/default # Okta with a custom Authorization Server scopes: - profile - openid # - offline_access # Uncomment if OIdC supports issuing refresh tokens. - # 4. Replace with the client ID created for Flyte. + # 3. Replace with the client ID created for Flyte. clientId: 0oakkheteNjCMERst5d6 - # 5. Replace with the public facing URL of flyte admin (e.g. https://flyte.mycompany.io/callback) - callbackUrl: "http://localhost:30081/callback" - # 6. Replace with the flyte console's URL (e.g. https://flyte.mycompany.io/console) - redirectUrl: "/api/v1/projects" Save and exit your editor. @@ -177,35 +171,36 @@ Apply Configurations 1. It is possible to direct flyte admin to use an external authorization server. To do so, edit the same config map once more and follow these changes: - .. code-block:: yaml +.. code-block:: yaml auth: - # 1. Update with the public facing URL of flyte admin (e.g. https://flyte.mycompany.io/) - httpPublicUri: http://localhost:30081/ appAuth: - # 1. Choose External if you will use an external Authorization Server (e.g. a Custom Authorization server in Okta) - # Choose Self (or omit the value) to use Flyte Admin's internal (albeit limited) Authorization Server. - authServerType: External + # 1. Choose External if you will use an external Authorization Server (e.g. a Custom Authorization server in Okta) + # Choose Self (or omit the value) to use Flyte Admin's internal (albeit limited) Authorization Server. + authServerType: External + + # 2. Optional: Set external auth server baseUrl if different from OpenId baseUrl. + externalAuthServer: + baseUrl: https://dev-14186422.okta.com/oauth2/auskngnn7uBViQq6b5d6 thirdPartyConfig: flyteClient: - # 2. Replace with a new native client ID provisioned in the custom authorization server - clientId: flytectl - redirectUri: https://localhost:53593/callback - # 3. "all" is a required scope and must be configured in the custom authorization server - scopes: + # 3. Replace with a new Native Client ID provisioned in the custom authorization server + clientId: flytectl + + redirectUri: https://localhost:53593/callback + + # 4. "all" is a required scope and must be configured in the custom authorization server + scopes: - offline - all userAuth: - openId: - # 4. Use the URL of your custom authorization server created above: - baseUrl: https://dev-14186422.okta.com/oauth2/auskngnn7uBViQq6b5d6 # Okta with a custom Authorization Server - scopes: - - profile - - openid - # - offline_access # Uncomment if OIdC supports issuing refresh tokens. - clientId: 0oakkheteNjCMERst5d6 - callbackUrl: "http://localhost:30081/callback" - redirectUrl: "/api/v1/projects" + openId: + baseUrl: https://dev-14186422.okta.com/oauth2/auskngnn7uBViQq6b5d6 # Okta with a custom Authorization Server + scopes: + - profile + - openid + # - offline_access # Uncomment if OIdC supports issuing refresh tokens. + clientId: 0oakkheteNjCMERst5d6 1. Store flyte propeller's `client_secret` in a k8s secrt as follows: @@ -240,13 +235,13 @@ Follow the inline comments to make the necessary changes: .. code-block:: yaml -admin: - # 1. Turn to true - useAuth: true - # 2. Replace with the client_id provided by the OAuth2 Authorization Server above. - clientId: flytepropeller - # 3. Replace with the OAuth2 authorization server url provided in the previous step. - authorizationServerUrl: http://localhost:30081/ + admin: + # 1. Turn to true + useAuth: true + # 2. Replace with the client_id provided by the OAuth2 Authorization Server above. + clientId: flytepropeller + # 3. Replace with the OAuth2 authorization server url provided in the previous step. + authorizationServerUrl: http://localhost:30081/ Close the editor @@ -254,7 +249,7 @@ Close the editor Continuous Integration - CI *************************** -If your organization does any automated registration, then you'll need to authenticate with the `basic authentication `_ flow (username and password effectively) as CI systems are generally not suitable OAuth resource owners. After retrieving an access token from the IDP, you can send it along to Flyte Admin as usual. +If your organization does any automated registration, then you'll need to authenticate with the `basic authentication `_ flow (username and password effectively). After retrieving an access token from the IDP, you can send it along to Flyte Admin as usual. Flytekit configuration variables are automatically designed to look up values from relevant environment variables. However, to aid with continuous integration use-cases, Flytekit configuration can also reference other environment variables. From 9e38bf99aceba66b0100207a3be1bbfe3c8df549 Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Wed, 21 Apr 2021 13:53:00 -0700 Subject: [PATCH 57/74] Update docs Signed-off-by: Haytham Abuelfutuh --- deployment/eks/flyte_generated.yaml | 70 +++++-------- deployment/gcp/flyte_generated.yaml | 70 +++++-------- deployment/sandbox/flyte_generated.yaml | 88 +++++++---------- deployment/test/flyte_generated.yaml | 68 +++++-------- .../base/admindeployment/deployment.yaml | 59 ++++++----- kustomize/base/propeller/auth_secret.yaml | 2 + .../headless/config/admin/server.yaml | 6 -- .../headless/config/propeller/admin.yaml | 1 + kustomize/overlays/sandbox/kustomization.yaml | 6 +- rsts/howto/authentication/index.rst | 98 +++++++++++++------ rsts/howto/authentication/setup.rst | 51 +++++++--- 11 files changed, 264 insertions(+), 255 deletions(-) diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index c0f350bd96..1608eec426 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -8165,11 +8165,7 @@ data: allowedHeaders: - "Content-Type" - "flyte-authorization" - secrets: - secrets-prefix: /etc/secrets/auth/ auth: - disableForGrpc: true - httpPublicUri: http://localhost:30081/ appAuth: thirdPartyConfig: flyteClient: @@ -8185,8 +8181,6 @@ data: - profile - openid clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com - callbackUrl: "http://localhost:30081/callback" - redirectUrl: "/api/v1/projects" flyteadmin: roleNameKey: "iam.amazonaws.com/role" profilerPort: 10254 @@ -8230,7 +8224,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-6dg952hkfh + name: flyte-admin-config-k79c9bh7t4 namespace: flyte --- apiVersion: v1 @@ -8252,6 +8246,7 @@ data: admin: endpoint: flyteadmin:81 insecure: true + useAuth: false catalog.yaml: | catalog-cache: endpoint: datacatalog:89 @@ -8402,7 +8397,7 @@ data: cloudwatch-log-group: kind: ConfigMap metadata: - name: flyte-propeller-config-727k54546d + name: flyte-propeller-config-c2hh7hhkh4 namespace: flyte --- apiVersion: v1 @@ -8422,6 +8417,15 @@ metadata: type: Opaque --- apiVersion: v1 +kind: Secret +metadata: + name: flyte-propeller-auth + namespace: flyte +stringData: + client_secret: foobar +type: Opaque +--- +apiVersion: v1 kind: Service metadata: annotations: @@ -8710,7 +8714,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-727k54546d + name: flyte-propeller-config-c2hh7hhkh4 name: config-volume - name: webhook-certs secret: @@ -8763,7 +8767,7 @@ spec: name: config-volume - mountPath: /etc/db name: db-pass - - mountPath: /etc/secrets/auth + - mountPath: /etc/secrets/ name: auth - command: - sh @@ -8834,31 +8838,10 @@ spec: - mountPath: /etc/db name: db-pass - args: - - --localPath - - /etc/secrets/auth-new - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - secrets - - init - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 - imagePullPolicy: IfNotPresent - name: generate-secrets - volumeMounts: - - mountPath: /etc/flyte/config - name: config-volume - - mountPath: /etc/secrets/auth-new - name: new-auth-secrets - - args: - - --fromPath - - /etc/secrets/auth-new + - flyteadmin --config=etc/flyte/config/*.yaml secrets init --localPath /etc/secrets/auth && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --fromPath /etc/secrets/auth command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - secrets - - create + - /bin/sh + - -c env: - name: POD_NAMESPACE valueFrom: @@ -8866,18 +8849,16 @@ spec: fieldPath: metadata.namespace image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent - name: save-secrets + name: generate-secrets volumeMounts: - mountPath: /etc/flyte/config name: config-volume - - mountPath: /etc/secrets/auth-new - name: new-auth-secrets serviceAccountName: flyteadmin volumes: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-6dg952hkfh + name: flyte-admin-config-k79c9bh7t4 name: config-volume - configMap: name: clusterresource-template-tkdkkt4cb5 @@ -8888,8 +8869,6 @@ spec: - name: auth secret: secretName: flyte-admin-auth - - emptyDir: {} - name: new-auth-secrets --- apiVersion: apps/v1 kind: Deployment @@ -8971,11 +8950,16 @@ spec: volumeMounts: - mountPath: /etc/flyte/config name: config-volume + - mountPath: /etc/secrets/ + name: auth serviceAccountName: flytepropeller volumes: - configMap: - name: flyte-propeller-config-727k54546d + name: flyte-propeller-config-c2hh7hhkh4 name: config-volume + - name: auth + secret: + secretName: flyte-propeller-auth --- apiVersion: apps/v1 kind: Deployment @@ -9162,7 +9146,7 @@ spec: name: clusterresource-template-tkdkkt4cb5 name: resource-templates - configMap: - name: flyte-admin-config-6dg952hkfh + name: flyte-admin-config-k79c9bh7t4 name: config-volume - name: db-pass secret: @@ -9205,7 +9189,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-727k54546d + name: flyte-propeller-config-c2hh7hhkh4 name: config-volume ttlSecondsAfterFinished: 0 --- diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index 251e681e50..ee31cb45ee 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -8156,11 +8156,7 @@ data: allowedHeaders: - "Content-Type" - "flyte-authorization" - secrets: - secrets-prefix: /etc/secrets/auth/ auth: - disableForGrpc: true - httpPublicUri: http://localhost:30081/ appAuth: thirdPartyConfig: flyteClient: @@ -8176,8 +8172,6 @@ data: - profile - openid clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com - callbackUrl: "http://localhost:30081/callback" - redirectUrl: "/api/v1/projects" flyteadmin: roleNameKey: "iam.amazonaws.com/role" profilerPort: 10254 @@ -8221,7 +8215,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-mchkdmtm5k + name: flyte-admin-config-95g5286ffc namespace: flyte --- apiVersion: v1 @@ -8243,6 +8237,7 @@ data: admin: endpoint: flyteadmin:81 insecure: true + useAuth: false catalog.yaml: | catalog-cache: endpoint: datacatalog:89 @@ -8389,7 +8384,7 @@ data: stackdriver-logresourcename: k8s_container kind: ConfigMap metadata: - name: flyte-propeller-config-k8km2mmdgf + name: flyte-propeller-config-b5bccdkh58 namespace: flyte --- apiVersion: v1 @@ -8409,6 +8404,15 @@ metadata: type: Opaque --- apiVersion: v1 +kind: Secret +metadata: + name: flyte-propeller-auth + namespace: flyte +stringData: + client_secret: foobar +type: Opaque +--- +apiVersion: v1 kind: Service metadata: name: cloudsqlproxy @@ -8742,7 +8746,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-k8km2mmdgf + name: flyte-propeller-config-b5bccdkh58 name: config-volume - name: webhook-certs secret: @@ -8795,7 +8799,7 @@ spec: name: config-volume - mountPath: /etc/db name: db-pass - - mountPath: /etc/secrets/auth + - mountPath: /etc/secrets/ name: auth - command: - sh @@ -8866,31 +8870,10 @@ spec: - mountPath: /etc/db name: db-pass - args: - - --localPath - - /etc/secrets/auth-new - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - secrets - - init - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 - imagePullPolicy: IfNotPresent - name: generate-secrets - volumeMounts: - - mountPath: /etc/flyte/config - name: config-volume - - mountPath: /etc/secrets/auth-new - name: new-auth-secrets - - args: - - --fromPath - - /etc/secrets/auth-new + - flyteadmin --config=etc/flyte/config/*.yaml secrets init --localPath /etc/secrets/auth && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --fromPath /etc/secrets/auth command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - secrets - - create + - /bin/sh + - -c env: - name: POD_NAMESPACE valueFrom: @@ -8898,18 +8881,16 @@ spec: fieldPath: metadata.namespace image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent - name: save-secrets + name: generate-secrets volumeMounts: - mountPath: /etc/flyte/config name: config-volume - - mountPath: /etc/secrets/auth-new - name: new-auth-secrets serviceAccountName: flyteadmin volumes: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-mchkdmtm5k + name: flyte-admin-config-95g5286ffc name: config-volume - configMap: name: clusterresource-template-tkdkkt4cb5 @@ -8920,8 +8901,6 @@ spec: - name: auth secret: secretName: flyte-admin-auth - - emptyDir: {} - name: new-auth-secrets --- apiVersion: apps/v1 kind: Deployment @@ -9003,11 +8982,16 @@ spec: volumeMounts: - mountPath: /etc/flyte/config name: config-volume + - mountPath: /etc/secrets/ + name: auth serviceAccountName: flytepropeller volumes: - configMap: - name: flyte-propeller-config-k8km2mmdgf + name: flyte-propeller-config-b5bccdkh58 name: config-volume + - name: auth + secret: + secretName: flyte-propeller-auth --- apiVersion: apps/v1 kind: Deployment @@ -9194,7 +9178,7 @@ spec: name: clusterresource-template-tkdkkt4cb5 name: resource-templates - configMap: - name: flyte-admin-config-mchkdmtm5k + name: flyte-admin-config-95g5286ffc name: config-volume - name: db-pass secret: @@ -9237,7 +9221,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-k8km2mmdgf + name: flyte-propeller-config-b5bccdkh58 name: config-volume ttlSecondsAfterFinished: 0 --- diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index 59768f9b39..59d97b8f1f 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -2172,11 +2172,7 @@ data: allowedHeaders: - "Content-Type" - "flyte-authorization" - secrets: - secrets-prefix: /etc/secrets/auth/ auth: - disableForGrpc: true - httpPublicUri: http://localhost:30081/ appAuth: thirdPartyConfig: flyteClient: @@ -2192,8 +2188,6 @@ data: - profile - openid clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com - callbackUrl: "http://localhost:30081/callback" - redirectUrl: "/api/v1/projects" flyteadmin: roleNameKey: "iam.amazonaws.com/role" profilerPort: 10254 @@ -2229,7 +2223,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-bkfh292hd6 + name: flyte-admin-config-kg9mm798hc namespace: flyte --- apiVersion: v1 @@ -2251,6 +2245,7 @@ data: admin: endpoint: flyteadmin:81 insecure: true + useAuth: false catalog.yaml: | catalog-cache: endpoint: datacatalog:89 @@ -2347,7 +2342,7 @@ data: kubernetes-template-uri: "http://localhost:30082/#/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace }}" kind: ConfigMap metadata: - name: flyte-propeller-config-6gd7cgkkdt + name: flyte-propeller-config-4b26tfd275 namespace: flyte --- apiVersion: v1 @@ -2497,6 +2492,15 @@ metadata: type: Opaque --- apiVersion: v1 +kind: Secret +metadata: + name: flyte-propeller-auth + namespace: flyte +stringData: + client_secret: foobar +type: Opaque +--- +apiVersion: v1 data: password: bXl1c2Vy user_secret: bXlzZWNyZXQ= @@ -2827,7 +2831,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.10.0 + image: ghcr.io/flyteorg/flytepropeller:796c99dd00b0c073c0c48b6a7da413a00417d525 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -2846,7 +2850,7 @@ spec: secret: secretName: user-info - configMap: - name: flyte-propeller-config-6gd7cgkkdt + name: flyte-propeller-config-4b26tfd275 name: config-volume - name: webhook-certs secret: @@ -2881,7 +2885,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 + image: ghcr.io/flyteorg/flyteadmin:dc28c2b9e1c3372e62ab05ab775eae74963347ff imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -2899,7 +2903,7 @@ spec: name: config-volume - mountPath: /etc/db name: db-pass - - mountPath: /etc/secrets/auth + - mountPath: /etc/secrets/ name: auth - command: - sh @@ -2934,7 +2938,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 + image: ghcr.io/flyteorg/flyteadmin:dc28c2b9e1c3372e62ab05ab775eae74963347ff imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -2950,7 +2954,7 @@ spec: - seed-projects - flytesnacks - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 + image: ghcr.io/flyteorg/flyteadmin:dc28c2b9e1c3372e62ab05ab775eae74963347ff imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -2964,7 +2968,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 + image: ghcr.io/flyteorg/flyteadmin:dc28c2b9e1c3372e62ab05ab775eae74963347ff imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -2975,44 +2979,21 @@ spec: - mountPath: /etc/db name: db-pass - args: - - --localPath - - /etc/secrets/auth-new + - flyteadmin --config=etc/flyte/config/*.yaml secrets init --localPath /etc/secrets/auth && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --fromPath /etc/secrets/auth command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - secrets - - init - image: ghcr.io/flyteorg/flyteadmin:ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 - imagePullPolicy: IfNotPresent - name: generate-secrets - volumeMounts: - - mountPath: /etc/flyte/config - name: config-volume - - mountPath: /etc/secrets/auth-new - name: new-auth-secrets - - args: - - --fromPath - - /etc/secrets/auth-new - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - secrets - - create + - /bin/sh + - -c env: - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flyteadmin:ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 + image: ghcr.io/flyteorg/flyteadmin:dc28c2b9e1c3372e62ab05ab775eae74963347ff imagePullPolicy: IfNotPresent - name: save-secrets + name: generate-secrets volumeMounts: - mountPath: /etc/flyte/config name: config-volume - - mountPath: /etc/secrets/auth-new - name: new-auth-secrets serviceAccountName: flyteadmin volumes: - configMap: @@ -3021,7 +3002,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-bkfh292hd6 + name: flyte-admin-config-kg9mm798hc name: config-volume - name: db-pass secret: @@ -3029,8 +3010,6 @@ spec: - name: auth secret: secretName: flyte-admin-auth - - emptyDir: {} - name: new-auth-secrets --- apiVersion: apps/v1 kind: Deployment @@ -3099,7 +3078,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.10.0 + image: ghcr.io/flyteorg/flytepropeller:796c99dd00b0c073c0c48b6a7da413a00417d525 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -3107,11 +3086,16 @@ spec: volumeMounts: - mountPath: /etc/flyte/config name: config-volume + - mountPath: /etc/secrets/ + name: auth serviceAccountName: flytepropeller volumes: - configMap: - name: flyte-propeller-config-6gd7cgkkdt + name: flyte-propeller-config-4b26tfd275 name: config-volume + - name: auth + secret: + secretName: flyte-propeller-auth --- apiVersion: apps/v1 kind: Deployment @@ -3413,7 +3397,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 + image: ghcr.io/flyteorg/flyteadmin:dc28c2b9e1c3372e62ab05ab775eae74963347ff imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -3430,7 +3414,7 @@ spec: name: clusterresource-template-dtg8ff28mt name: resource-templates - configMap: - name: flyte-admin-config-bkfh292hd6 + name: flyte-admin-config-kg9mm798hc name: config-volume - name: db-pass secret: @@ -3597,7 +3581,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.10.0 + image: ghcr.io/flyteorg/flytepropeller:796c99dd00b0c073c0c48b6a7da413a00417d525 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -3607,7 +3591,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-6gd7cgkkdt + name: flyte-propeller-config-4b26tfd275 name: config-volume ttlSecondsAfterFinished: 0 --- diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index 72c260d5cf..2b205ed941 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -320,11 +320,7 @@ data: allowedHeaders: - "Content-Type" - "flyte-authorization" - secrets: - secrets-prefix: /etc/secrets/auth/ auth: - disableForGrpc: true - httpPublicUri: http://localhost:30081/ appAuth: thirdPartyConfig: flyteClient: @@ -340,8 +336,6 @@ data: - profile - openid clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com - callbackUrl: "http://localhost:30081/callback" - redirectUrl: "/api/v1/projects" flyteadmin: roleNameKey: "iam.amazonaws.com/role" profilerPort: 10254 @@ -377,7 +371,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-f9cbgdhcbh + name: flyte-admin-config-hb74mc6784 namespace: flyte --- apiVersion: v1 @@ -390,6 +384,7 @@ data: admin: endpoint: flyteadmin:81 insecure: true + useAuth: false catalog.yaml: | catalog-cache: endpoint: datacatalog:89 @@ -481,7 +476,7 @@ data: kind: ConfigMap metadata: - name: flyte-propeller-config-dtdt5gbgg2 + name: flyte-propeller-config-68bh2f6c6f namespace: flyte --- apiVersion: v1 @@ -501,6 +496,15 @@ metadata: type: Opaque --- apiVersion: v1 +kind: Secret +metadata: + name: flyte-propeller-auth + namespace: flyte +stringData: + client_secret: foobar +type: Opaque +--- +apiVersion: v1 kind: Service metadata: annotations: @@ -700,7 +704,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-dtdt5gbgg2 + name: flyte-propeller-config-68bh2f6c6f name: config-volume - name: webhook-certs secret: @@ -753,7 +757,7 @@ spec: name: config-volume - mountPath: /etc/db name: db-pass - - mountPath: /etc/secrets/auth + - mountPath: /etc/secrets/ name: auth - command: - sh @@ -829,31 +833,10 @@ spec: - mountPath: /etc/db name: db-pass - args: - - --localPath - - /etc/secrets/auth-new - command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - secrets - - init - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 - imagePullPolicy: IfNotPresent - name: generate-secrets - volumeMounts: - - mountPath: /etc/flyte/config - name: config-volume - - mountPath: /etc/secrets/auth-new - name: new-auth-secrets - - args: - - --fromPath - - /etc/secrets/auth-new + - flyteadmin --config=etc/flyte/config/*.yaml secrets init --localPath /etc/secrets/auth && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --fromPath /etc/secrets/auth command: - - flyteadmin - - --config - - /etc/flyte/config/*.yaml - - secrets - - create + - /bin/sh + - -c env: - name: POD_NAMESPACE valueFrom: @@ -861,12 +844,10 @@ spec: fieldPath: metadata.namespace image: ghcr.io/flyteorg/flyteadmin:v0.4.7 imagePullPolicy: IfNotPresent - name: save-secrets + name: generate-secrets volumeMounts: - mountPath: /etc/flyte/config name: config-volume - - mountPath: /etc/secrets/auth-new - name: new-auth-secrets serviceAccountName: flyteadmin volumes: - configMap: @@ -875,7 +856,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-f9cbgdhcbh + name: flyte-admin-config-hb74mc6784 name: config-volume - name: db-pass secret: @@ -883,8 +864,6 @@ spec: - name: auth secret: secretName: flyte-admin-auth - - emptyDir: {} - name: new-auth-secrets --- apiVersion: apps/v1 kind: Deployment @@ -927,11 +906,16 @@ spec: volumeMounts: - mountPath: /etc/flyte/config name: config-volume + - mountPath: /etc/secrets/ + name: auth serviceAccountName: flytepropeller volumes: - configMap: - name: flyte-propeller-config-dtdt5gbgg2 + name: flyte-propeller-config-68bh2f6c6f name: config-volume + - name: auth + secret: + secretName: flyte-propeller-auth --- apiVersion: apps/v1 kind: Deployment @@ -1043,7 +1027,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-dtdt5gbgg2 + name: flyte-propeller-config-68bh2f6c6f name: config-volume ttlSecondsAfterFinished: 0 --- diff --git a/kustomize/base/admindeployment/deployment.yaml b/kustomize/base/admindeployment/deployment.yaml index 846f6f1ac5..f8df687efc 100644 --- a/kustomize/base/admindeployment/deployment.yaml +++ b/kustomize/base/admindeployment/deployment.yaml @@ -25,7 +25,7 @@ spec: serviceAccountName: flyteadmin volumes: - name: shared-data - emptyDir: { } + emptyDir: {} - name: config-volume configMap: name: flyte-admin-config @@ -38,13 +38,18 @@ spec: - name: auth secret: secretName: flyte-admin-auth - - name: new-auth-secrets - emptyDir: { } initContainers: - name: run-migrations image: flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent - command: [ "flyteadmin", "--config", "/etc/flyte/config/*.yaml", "migrate", "run" ] + command: + [ + "flyteadmin", + "--config", + "/etc/flyte/config/*.yaml", + "migrate", + "run", + ] volumeMounts: - name: config-volume mountPath: /etc/flyte/config @@ -54,8 +59,17 @@ spec: - name: seed-projects image: flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent - command: [ "flyteadmin", "--config", "/etc/flyte/config/*.yaml", - "migrate", "seed-projects", "flytesnacks", "flytetester", "flyteexamples" ] + command: + [ + "flyteadmin", + "--config", + "/etc/flyte/config/*.yaml", + "migrate", + "seed-projects", + "flytesnacks", + "flytetester", + "flyteexamples", + ] volumeMounts: - name: config-volume mountPath: /etc/flyte/config @@ -64,7 +78,14 @@ spec: - name: sync-cluster-resources image: flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent - command: [ "flyteadmin", "--config", "/etc/flyte/config/*.yaml", "clusterresource", "sync" ] + command: + [ + "flyteadmin", + "--config", + "/etc/flyte/config/*.yaml", + "clusterresource", + "sync", + ] volumeMounts: - name: resource-templates mountPath: /etc/flyte/clusterresource/templates @@ -75,33 +96,25 @@ spec: - name: generate-secrets image: flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent - command: [ "flyteadmin", "--config", "/etc/flyte/config/*.yaml", "secrets", "init" ] - args: [ "--localPath", "/etc/secrets/auth-new" ] + command: ["/bin/sh", "-c"] + args: + [ + "flyteadmin --config=etc/flyte/config/*.yaml secrets init --localPath /etc/secrets/auth && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --fromPath /etc/secrets/auth", + ] volumeMounts: - name: config-volume mountPath: /etc/flyte/config - - name: new-auth-secrets - mountPath: /etc/secrets/auth-new - - name: save-secrets - image: flyteadmin:v0.4.0 - imagePullPolicy: IfNotPresent - command: [ "flyteadmin", "--config", "/etc/flyte/config/*.yaml", "secrets", "create" ] - args: [ "--fromPath", "/etc/secrets/auth-new" ] env: - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - volumeMounts: - - name: config-volume - mountPath: /etc/flyte/config - - name: new-auth-secrets - mountPath: /etc/secrets/auth-new containers: - name: flyteadmin image: flyteadmin:v0.4.0 imagePullPolicy: IfNotPresent - command: [ "flyteadmin", "--config", "/etc/flyte/config/*.yaml", "serve" ] + command: + ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "serve"] ports: - containerPort: 8088 - containerPort: 8089 @@ -113,7 +126,7 @@ spec: - name: db-pass mountPath: /etc/db - name: auth - mountPath: /etc/secrets/auth + mountPath: /etc/secrets/ - name: redoc image: docker.io/redocly/redoc imagePullPolicy: IfNotPresent diff --git a/kustomize/base/propeller/auth_secret.yaml b/kustomize/base/propeller/auth_secret.yaml index 7bf797efa1..b6bbbe4a8c 100644 --- a/kustomize/base/propeller/auth_secret.yaml +++ b/kustomize/base/propeller/auth_secret.yaml @@ -4,3 +4,5 @@ metadata: name: flyte-propeller-auth namespace: flyte type: Opaque +stringData: + client_secret: foobar diff --git a/kustomize/base/single_cluster/headless/config/admin/server.yaml b/kustomize/base/single_cluster/headless/config/admin/server.yaml index 77948fd719..096b9fe8e6 100644 --- a/kustomize/base/single_cluster/headless/config/admin/server.yaml +++ b/kustomize/base/single_cluster/headless/config/admin/server.yaml @@ -11,11 +11,7 @@ server: allowedHeaders: - "Content-Type" - "flyte-authorization" -secrets: - secrets-prefix: /etc/secrets/auth/ auth: - disableForGrpc: true - httpPublicUri: http://localhost:30081/ appAuth: thirdPartyConfig: flyteClient: @@ -31,8 +27,6 @@ auth: - profile - openid clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com - callbackUrl: "http://localhost:30081/callback" - redirectUrl: "/api/v1/projects" flyteadmin: roleNameKey: "iam.amazonaws.com/role" profilerPort: 10254 diff --git a/kustomize/base/single_cluster/headless/config/propeller/admin.yaml b/kustomize/base/single_cluster/headless/config/propeller/admin.yaml index 55cf01b83b..63ee5c63fc 100644 --- a/kustomize/base/single_cluster/headless/config/propeller/admin.yaml +++ b/kustomize/base/single_cluster/headless/config/propeller/admin.yaml @@ -5,3 +5,4 @@ event: admin: endpoint: flyteadmin:81 insecure: true + useAuth: false diff --git a/kustomize/overlays/sandbox/kustomization.yaml b/kustomize/overlays/sandbox/kustomization.yaml index 10e2c1d300..2af25fb495 100644 --- a/kustomize/overlays/sandbox/kustomization.yaml +++ b/kustomize/overlays/sandbox/kustomization.yaml @@ -25,7 +25,7 @@ resources: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: ee6db644c1ad1495ccd3b2b6fac059133cfb4293-2 # FLYTEADMIN_TAG override the tag + newTag: dc28c2b9e1c3372e62ab05ab775eae74963347ff # FLYTEADMIN_TAG override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -37,11 +37,11 @@ images: newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag + newTag: 796c99dd00b0c073c0c48b6a7da413a00417d525 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Webhook - name: webhook # match images with this name - newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag + newTag: 796c99dd00b0c073c0c48b6a7da413a00417d525 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Override postgres image to use alpine based (rather smaller) docker image - name: postgres diff --git a/rsts/howto/authentication/index.rst b/rsts/howto/authentication/index.rst index 8e61066b44..251a2489aa 100644 --- a/rsts/howto/authentication/index.rst +++ b/rsts/howto/authentication/index.rst @@ -14,53 +14,93 @@ Flyte ships with a canonical implementation of OpenIDConnect client and OAuth2 S setup migration -*********** -Components -*********** +******** +Overview +******** -While the most obvious interaction with the Flyte control plane is through the web based UI, there are other critical components of Flyte that also need to be considered. These components should be thought of as third-party services even though the Flyte codebase provides them. +Flyte system consists of multiple components. For the purposes of this document, let's categorize them into server-side and client-side components: -Flyte CLI -========= -This is the primary component. Flyte CLI is the command-line entrypoint to Flyte Admin and is used by both administrators and users more comfortable in the command line, or are running in a headless OS. +- **Admin**: A server-side control plane component accessible from console, cli and other backends. +- **Catalog**: A server-side control plane component accessible from console, cli and other backends. +- **Console**: A client-side single page react app. +- **flyte-cli**: A python-based client-side command line interface that interacts with Admin and Catalog. +- **flytectl**: A go-based client-side command line interface that interacts with Admin and Catalog. +- **Propeller**: A server-side data plane component that interacts with both admin and catalog services. -The IDP application corresponding to the CLI will need to support PKCE. +************** +OpenID Connect +************** -Direct Client Access -==================== -The gRPC client provided by the Flyte IDL, or direct calls to the HTTP endpoints on Admin from within a running script are ways that we have seen users hit the control plane directly. We generally discourage this course of action as it leads to a possible self-imposed DOS vector, which is generally made from within a running workflow itself. +Flyte supports OpenID Connect. A defacto standard for user authentication. After configuring OpenID Connect, users accessing flyte console or flytectl +(or other 3rd party apps) will be prompted to authenticate using the configured provider. -For instance, a Flyte task can fetch the definition for a launch plan associated with a completely different workflow, and then launch an execution of it. This is not the correct way to launch one workflow from another but for the time being remains possible. +.. image:: https://mermaid.ink/img/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG4lJXtjb25maWc6IHsgJ2ZvbnRGYW1pbHknOiAnTWVubG8nLCAnZm9udFNpemUnOiAxMCwgJ2ZvbnRXZWlnaHQnOiAxMDB9IH0lJVxuICAgIGF1dG9udW1iZXJcbiAgICBVc2VyLT4-K0Jyb3dzZXI6IC9ob21lXG4gICAgQnJvd3Nlci0-PitDb25zb2xlOiAvaG9tZVxuICAgIENvbnNvbGUtPj4tQnJvd3NlcjogMzAyIC9sb2dpblxuICAgIEJyb3dzZXItPj4rQWRtaW46IC9sb2dpblxuICAgIEFkbWluLT4-LUJyb3dzZXI6IElkcC5jb20vb2lkY1xuICAgIEJyb3dzZXItPj4rSWRwOiBJZHAuY29tL29pZGNcbiAgICBJZHAtPj4tQnJvd3NlcjogMzAyIC9sb2dpblxuICAgIEJyb3dzZXItPj4tVXNlcjogRW50ZXIgdXNlci9wYXNzXG4gICAgVXNlci0-PitCcm93c2VyOiBsb2dpblxuICAgIEJyb3dzZXItPj4rSWRwOiBTdWJtaXQgdXNlcm5hbWUvcGFzc1xuICAgIElkcC0-Pi1Ccm93c2VyOiBhZG1pbi8_YXV0aENvZGU9PGFiYz5cbiAgICBCcm93c2VyLT4-K0FkbWluOiBhZG1pbi9hdXRoQ29kZT08YWJjPlxuICAgIEFkbWluLT4-K0lkcDogRXhjaGFuZ2UgVG9rZW5zXG4gICAgSWRwLT4-LUFkbWluOiBpZHQsIGF0LCBydFxuICAgIEFkbWluLT4-K0Jyb3dzZXI6IFdyaXRlIENvb2tpZXMgJiBSZWRpcmVjdCB0byAvY29uc29sZVxuICAgIEJyb3dzZXItPj4rQ29uc29sZTogL2hvbWVcbiAgICBCcm93c2VyLT4-LVVzZXI6IFJlbmRlciAvaG9tZVxuIiwibWVybWFpZCI6eyJ0aGVtZSI6Im5ldXRyYWwifSwidXBkYXRlRWRpdG9yIjpmYWxzZX0 + :target: https://mermaid-js.github.io/mermaid-live-editor/#/edit/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG4lJXtjb25maWc6IHsgJ2ZvbnRGYW1pbHknOiAnTWVubG8nLCAnZm9udFNpemUnOiAxMCwgJ2ZvbnRXZWlnaHQnOiAxMDB9IH0lJVxuICAgIGF1dG9udW1iZXJcbiAgICBVc2VyLT4-K0Jyb3dzZXI6IC9ob21lXG4gICAgQnJvd3Nlci0-PitDb25zb2xlOiAvaG9tZVxuICAgIENvbnNvbGUtPj4tQnJvd3NlcjogMzAyIC9sb2dpblxuICAgIEJyb3dzZXItPj4rQWRtaW46IC9sb2dpblxuICAgIEFkbWluLT4-LUJyb3dzZXI6IElkcC5jb20vb2lkY1xuICAgIEJyb3dzZXItPj4rSWRwOiBJZHAuY29tL29pZGNcbiAgICBJZHAtPj4tQnJvd3NlcjogMzAyIC9sb2dpblxuICAgIEJyb3dzZXItPj4tVXNlcjogRW50ZXIgdXNlci9wYXNzXG4gICAgVXNlci0-PitCcm93c2VyOiBsb2dpblxuICAgIEJyb3dzZXItPj4rSWRwOiBTdWJtaXQgdXNlcm5hbWUvcGFzc1xuICAgIElkcC0-Pi1Ccm93c2VyOiBhZG1pbi8_YXV0aENvZGU9PGFiYz5cbiAgICBCcm93c2VyLT4-K0FkbWluOiBhZG1pbi9hdXRoQ29kZT08YWJjPlxuICAgIEFkbWluLT4-K0lkcDogRXhjaGFuZ2UgVG9rZW5zXG4gICAgSWRwLT4-LUFkbWluOiBpZHQsIGF0LCBydFxuICAgIEFkbWluLT4-K0Jyb3dzZXI6IFdyaXRlIENvb2tpZXMgJiBSZWRpcmVjdCB0byAvY29uc29sZVxuICAgIEJyb3dzZXItPj4rQ29uc29sZTogL2hvbWVcbiAgICBCcm93c2VyLT4-LVVzZXI6IFJlbmRlciAvaG9tZVxuIiwibWVybWFpZCI6eyJ0aGVtZSI6Im5ldXRyYWwifSwidXBkYXRlRWRpdG9yIjpmYWxzZX0 + :width: 600 + :alt: Flyte UI Swimlane + +****** +OAuth2 +****** + +Flyte supports OAuth2 to control access to 3rd party and native apps. FlyteAdmin comes with a built in Authorization Server that can perform 3-legged +and 2-legged OAuth2 flows. It also supports delegating these responsibilities to an external Authorization Server. -***************** -Swimlane Diagrams -***************** +Service Authentication using OAuth2 +=================================== -Flyte Admin authentication is implemented using the authorization code flow. +Propeller (and potentially other non-user facing services) can also authenticate using client_credentials to the Idp and be granted an +access_token valid to be used with admin and other backend services. -Flyte UI Flow -============= -https://swimlanes.io/d/OmV4ybCkx +Using FlyteAdmin's builtin Authorization Server: -.. image:: flyte_ui_flow.png +.. image:: https://mermaid.ink/img/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG4gICAgUHJvcGVsbGVyLT4-K0FkbWluOiAvdG9rZW4_Y2xpZW50X2NyZWRzJnNjb3BlPWh0dHBzOi8vYWRtaW4vXG4gICAgQWRtaW4tPj4tUHJvcGVsbGVyOiBhY2Nlc3NfdG9rZW5cbiAgICBQcm9wZWxsZXItPj4rQWRtaW46IC9saXN0X3Byb2plY3RzP3Rva2VuPWFjY2Vzc190b2tlbiIsIm1lcm1haWQiOnsidGhlbWUiOiJuZXV0cmFsIn0sInVwZGF0ZUVkaXRvciI6ZmFsc2V9 + :target: https://mermaid-js.github.io/mermaid-live-editor/#/edit/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG4gICAgUHJvcGVsbGVyLT4-K0FkbWluOiAvdG9rZW4_Y2xpZW50X2NyZWRzJnNjb3BlPWh0dHBzOi8vYWRtaW4vXG4gICAgQWRtaW4tPj4tUHJvcGVsbGVyOiBhY2Nlc3NfdG9rZW5cbiAgICBQcm9wZWxsZXItPj4rQWRtaW46IC9saXN0X3Byb2plY3RzP3Rva2VuPWFjY2Vzc190b2tlbiIsIm1lcm1haWQiOnsidGhlbWUiOiJuZXV0cmFsIn0sInVwZGF0ZUVkaXRvciI6ZmFsc2V9 :width: 600 - :alt: Flyte UI Swimlane + :alt: Service Authentication Swimlane + +Using an External Authorization Server: + +.. image:: https://mermaid.ink/img/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG4gICAgUHJvcGVsbGVyLT4-K0V4dGVybmFsIEF1dGhvcml6YXRpb24gU2VydmVyOiAvdG9rZW4_Y2xpZW50X2NyZWRzJnNjb3BlPWh0dHBzOi8vYWRtaW4vXG4gICAgRXh0ZXJuYWwgQXV0aG9yaXphdGlvbiBTZXJ2ZXItPj4tUHJvcGVsbGVyOiBhY2Nlc3NfdG9rZW5cbiAgICBQcm9wZWxsZXItPj4rQWRtaW46IC9saXN0X3Byb2plY3RzP3Rva2VuPWFjY2Vzc190b2tlbiIsIm1lcm1haWQiOnsidGhlbWUiOiJuZXV0cmFsIn0sInVwZGF0ZUVkaXRvciI6ZmFsc2V9 + :target: https://mermaid-js.github.io/mermaid-live-editor/#/edit/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG4gICAgUHJvcGVsbGVyLT4-K0V4dGVybmFsIEF1dGhvcml6YXRpb24gU2VydmVyOiAvdG9rZW4_Y2xpZW50X2NyZWRzJnNjb3BlPWh0dHBzOi8vYWRtaW4vXG4gICAgRXh0ZXJuYWwgQXV0aG9yaXphdGlvbiBTZXJ2ZXItPj4tUHJvcGVsbGVyOiBhY2Nlc3NfdG9rZW5cbiAgICBQcm9wZWxsZXItPj4rQWRtaW46IC9saXN0X3Byb2plY3RzP3Rva2VuPWFjY2Vzc190b2tlbiIsIm1lcm1haWQiOnsidGhlbWUiOiJuZXV0cmFsIn0sInVwZGF0ZUVkaXRvciI6ZmFsc2V9 + :width: 600 + :alt: Service Authentication Swimlane +User Authentication in other clients (e.g. Cli) using OAuth2-Pkce +================================================================== + +Users accessing backend services through Cli should be able to use OAuth2-Pkce flow to authenticate (in a browser) to the Idp and be issued +an access_token valid to communicate with the intended backend service on behalf of the user. + +Using FlyteAdmin's builtin Authorization Server: + +.. image:: https://mermaid.ink/img/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG4lJXtjb25maWc6IHsgJ2ZvbnRGYW1pbHknOiAnTWVubG8nLCAnZm9udFNpemUnOiAxMCwgJ2ZvbnRXZWlnaHQnOiAxMDB9IH0lJVxuICAgIGF1dG9udW1iZXJcbiAgICBVc2VyLT4-K0NsaTogZmx5dGVjdGwgbGlzdC1wcm9qZWN0c1xuICAgIENsaS0-PitBZG1pbjogYWRtaW4vbGlzdC1wcm9qZWN0cy9cbiAgICBBZG1pbi0-Pi1DbGk6IDQwMSBVbmF1dGhvcml6ZWRcbiAgICBDbGktPj4rQWRtaW46IGFkbWluL2NsaWVudC1jb25maWdcbiAgICBBZG1pbi0-Pi1DbGk6IENsaWVudF9pZD08YWJjPiwgLi4uXG4gICAgQ2xpLT4-K0Jyb3dzZXI6IC9hdXRoP3BrY2UmY29kZV9jaGFsbGVuZ2UsY2xpZW50X2lkLHNjb3BlXG4gICAgQnJvd3Nlci0-PitBZG1pbjogYWRtaW4vb2F1dGgyP3BrY2UuLi5cbiAgICBBZG1pbi0-Pi1Ccm93c2VyOiAzMDIgaWRwLmNvbS9sb2dpblxuICAgIE5vdGUgb3ZlciBCcm93c2VyLEFkbWluOiBUaGUgcHJpb3IgT3BlbklEIENvbm5lY3QgZmxvd1xuICAgIEJyb3dzZXItPj4rQWRtaW46IGFkbWluL2xvZ2dlZF9pblxuICAgIE5vdGUgb3ZlciBCcm93c2VyLEFkbWluOiBQb3RlbnRpYWxseSBzaG93IGN1c3RvbSBjb25zZW50IHNjcmVlblxuICAgIEFkbWluLT4-LUJyb3dzZXI6IGxvY2FsaG9zdC8_YXV0aENvZGU9PGFiYz5cbiAgICBCcm93c2VyLT4-K0NsaTogbG9jYWxob3N0L2F1dGhDb2RlPTxhYmM-XG4gICAgQ2xpLT4-K0FkbWluOiAvdG9rZW4_Y29kZSxjb2RlX3ZlcmlmaWVyXG4gICAgQWRtaW4tPj4tQ2xpOiBhY2Nlc3NfdG9rZW5cbiAgICBDbGktPj4rQWRtaW46IC9wcm9qZWN0cy8gKyBhY2Nlc3NfdG9rZW5cbiAgICBBZG1pbi0-Pi1DbGk6IHByb2plY3QxLCBwcm9qZWN0MlxuIiwibWVybWFpZCI6eyJ0aGVtZSI6Im5ldXRyYWwifSwidXBkYXRlRWRpdG9yIjpmYWxzZX0 + :target: https://mermaid-js.github.io/mermaid-live-editor/#/edit/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG4lJXtjb25maWc6IHsgJ2ZvbnRGYW1pbHknOiAnTWVubG8nLCAnZm9udFNpemUnOiAxMCwgJ2ZvbnRXZWlnaHQnOiAxMDB9IH0lJVxuICAgIGF1dG9udW1iZXJcbiAgICBVc2VyLT4-K0NsaTogZmx5dGVjdGwgbGlzdC1wcm9qZWN0c1xuICAgIENsaS0-PitBZG1pbjogYWRtaW4vbGlzdC1wcm9qZWN0cy9cbiAgICBBZG1pbi0-Pi1DbGk6IDQwMSBVbmF1dGhvcml6ZWRcbiAgICBDbGktPj4rQWRtaW46IGFkbWluL2NsaWVudC1jb25maWdcbiAgICBBZG1pbi0-Pi1DbGk6IENsaWVudF9pZD08YWJjPiwgLi4uXG4gICAgQ2xpLT4-K0Jyb3dzZXI6IC9hdXRoP3BrY2UmY29kZV9jaGFsbGVuZ2UsY2xpZW50X2lkLHNjb3BlXG4gICAgQnJvd3Nlci0-PitBZG1pbjogYWRtaW4vb2F1dGgyP3BrY2UuLi5cbiAgICBBZG1pbi0-Pi1Ccm93c2VyOiAzMDIgaWRwLmNvbS9sb2dpblxuICAgIE5vdGUgb3ZlciBCcm93c2VyLEFkbWluOiBUaGUgcHJpb3IgT3BlbklEIENvbm5lY3QgZmxvd1xuICAgIEJyb3dzZXItPj4rQWRtaW46IGFkbWluL2xvZ2dlZF9pblxuICAgIE5vdGUgb3ZlciBCcm93c2VyLEFkbWluOiBQb3RlbnRpYWxseSBzaG93IGN1c3RvbSBjb25zZW50IHNjcmVlblxuICAgIEFkbWluLT4-LUJyb3dzZXI6IGxvY2FsaG9zdC8_YXV0aENvZGU9PGFiYz5cbiAgICBCcm93c2VyLT4-K0NsaTogbG9jYWxob3N0L2F1dGhDb2RlPTxhYmM-XG4gICAgQ2xpLT4-K0FkbWluOiAvdG9rZW4_Y29kZSxjb2RlX3ZlcmlmaWVyXG4gICAgQWRtaW4tPj4tQ2xpOiBhY2Nlc3NfdG9rZW5cbiAgICBDbGktPj4rQWRtaW46IC9wcm9qZWN0cy8gKyBhY2Nlc3NfdG9rZW5cbiAgICBBZG1pbi0-Pi1DbGk6IHByb2plY3QxLCBwcm9qZWN0MlxuIiwibWVybWFpZCI6eyJ0aGVtZSI6Im5ldXRyYWwifSwidXBkYXRlRWRpdG9yIjpmYWxzZX0 + :width: 600 + :alt: CLI Authentication with Admin's own Authorization Server + +Using an External Authorization Server: + +.. image:: https://mermaid.ink/img/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG4lJXtjb25maWc6IHsgJ2ZvbnRGYW1pbHknOiAnTWVubG8nLCAnZm9udFNpemUnOiAxMCwgJ2ZvbnRXZWlnaHQnOiAxMDB9IH0lJVxuICAgIGF1dG9udW1iZXJcbiAgICBVc2VyLT4-K0NsaTogZmx5dGVjdGwgbGlzdC1wcm9qZWN0c1xuICAgIENsaS0-PitBZG1pbjogYWRtaW4vbGlzdC1wcm9qZWN0cy9cbiAgICBBZG1pbi0-Pi1DbGk6IDQwMSBVbmF1dGhvcml6ZWRcbiAgICBDbGktPj4rQWRtaW46IGFkbWluL2NsaWVudC1jb25maWdcbiAgICBBZG1pbi0-Pi1DbGk6IENsaWVudF9pZD08YWJjPiwgLi4uXG4gICAgQ2xpLT4-K0Jyb3dzZXI6IC9hdXRoP3BrY2UmY29kZV9jaGFsbGVuZ2UsY2xpZW50X2lkLHNjb3BlXG4gICAgQnJvd3Nlci0-PitFeHRlcm5hbElkcDogYWRtaW4vb2F1dGgyP3BrY2UuLi5cbiAgICBFeHRlcm5hbElkcC0-Pi1Ccm93c2VyOiAzMDIgaWRwLmNvbS9sb2dpblxuICAgIE5vdGUgb3ZlciBCcm93c2VyLEV4dGVybmFsSWRwOiBUaGUgcHJpb3IgT3BlbklEIENvbm5lY3QgZmxvd1xuICAgIEJyb3dzZXItPj4rRXh0ZXJuYWxJZHA6IC9sb2dnZWRfaW5cbiAgICBOb3RlIG92ZXIgQnJvd3NlcixFeHRlcm5hbElkcDogUG90ZW50aWFsbHkgc2hvdyBjdXN0b20gY29uc2VudCBzY3JlZW5cbiAgICBFeHRlcm5hbElkcC0-Pi1Ccm93c2VyOiBsb2NhbGhvc3QvP2F1dGhDb2RlPTxhYmM-XG4gICAgQnJvd3Nlci0-PitDbGk6IGxvY2FsaG9zdC9hdXRoQ29kZT08YWJjPlxuICAgIENsaS0-PitFeHRlcm5hbElkcDogL3Rva2VuP2NvZGUsY29kZV92ZXJpZmllclxuICAgIEV4dGVybmFsSWRwLT4-LUNsaTogYWNjZXNzX3Rva2VuXG4gICAgQ2xpLT4-K0FkbWluOiAvcHJvamVjdHMvICsgYWNjZXNzX3Rva2VuXG4gICAgQWRtaW4tPj4tQ2xpOiBwcm9qZWN0MSwgcHJvamVjdDJcbiIsIm1lcm1haWQiOnsidGhlbWUiOiJuZXV0cmFsIn0sInVwZGF0ZUVkaXRvciI6ZmFsc2V9 + :target: https://mermaid-js.github.io/mermaid-live-editor/#/edit/eyJjb2RlIjoic2VxdWVuY2VEaWFncmFtXG4lJXtjb25maWc6IHsgJ2ZvbnRGYW1pbHknOiAnTWVubG8nLCAnZm9udFNpemUnOiAxMCwgJ2ZvbnRXZWlnaHQnOiAxMDB9IH0lJVxuICAgIGF1dG9udW1iZXJcbiAgICBVc2VyLT4-K0NsaTogZmx5dGVjdGwgbGlzdC1wcm9qZWN0c1xuICAgIENsaS0-PitBZG1pbjogYWRtaW4vbGlzdC1wcm9qZWN0cy9cbiAgICBBZG1pbi0-Pi1DbGk6IDQwMSBVbmF1dGhvcml6ZWRcbiAgICBDbGktPj4rQWRtaW46IGFkbWluL2NsaWVudC1jb25maWdcbiAgICBBZG1pbi0-Pi1DbGk6IENsaWVudF9pZD08YWJjPiwgLi4uXG4gICAgQ2xpLT4-K0Jyb3dzZXI6IC9hdXRoP3BrY2UmY29kZV9jaGFsbGVuZ2UsY2xpZW50X2lkLHNjb3BlXG4gICAgQnJvd3Nlci0-PitFeHRlcm5hbElkcDogYWRtaW4vb2F1dGgyP3BrY2UuLi5cbiAgICBFeHRlcm5hbElkcC0-Pi1Ccm93c2VyOiAzMDIgaWRwLmNvbS9sb2dpblxuICAgIE5vdGUgb3ZlciBCcm93c2VyLEV4dGVybmFsSWRwOiBUaGUgcHJpb3IgT3BlbklEIENvbm5lY3QgZmxvd1xuICAgIEJyb3dzZXItPj4rRXh0ZXJuYWxJZHA6IC9sb2dnZWRfaW5cbiAgICBOb3RlIG92ZXIgQnJvd3NlcixFeHRlcm5hbElkcDogUG90ZW50aWFsbHkgc2hvdyBjdXN0b20gY29uc2VudCBzY3JlZW5cbiAgICBFeHRlcm5hbElkcC0-Pi1Ccm93c2VyOiBsb2NhbGhvc3QvP2F1dGhDb2RlPTxhYmM-XG4gICAgQnJvd3Nlci0-PitDbGk6IGxvY2FsaG9zdC9hdXRoQ29kZT08YWJjPlxuICAgIENsaS0-PitFeHRlcm5hbElkcDogL3Rva2VuP2NvZGUsY29kZV92ZXJpZmllclxuICAgIEV4dGVybmFsSWRwLT4-LUNsaTogYWNjZXNzX3Rva2VuXG4gICAgQ2xpLT4-K0FkbWluOiAvcHJvamVjdHMvICsgYWNjZXNzX3Rva2VuXG4gICAgQWRtaW4tPj4tQ2xpOiBwcm9qZWN0MSwgcHJvamVjdDJcbiIsIm1lcm1haWQiOnsidGhlbWUiOiJuZXV0cmFsIn0sInVwZGF0ZUVkaXRvciI6ZmFsc2V9 + :width: 600 + :alt: CLI Authentication with an external Authorization Server -Flyte CLI Flow -============== -https://swimlanes.io/d/q64OxuoxT +Identity Providers Support +========================== -.. image:: flyte_cli_flow.png - :width: 600 - :alt: Flyte CLI Swimlane ++-----------------+--------+-------------+---------------------+----------+-------+----------+ +| Feature | Okta | Google free | GC Identity Service | Azure AD | Auth0 | KeyCloak | ++=================+========+=============+=====================+==========+=======+==========+ +| OpenIdConnect | Yes | Yes | Yes | Yes | Yes | Yes | ++-----------------+--------+-------------+---------------------+----------+-------+----------+ +| Custom RP | Yes | No | Yes | Yes | ? | Yes | ++-----------------+--------+-------------+---------------------+----------+-------+----------+ ********** References ********** -RFCs -====== This collection of RFCs may be helpful to those who wish to investigate the implementation in more depth. * `OAuth2 RFC 6749 `_ diff --git a/rsts/howto/authentication/setup.rst b/rsts/howto/authentication/setup.rst index 23700c6290..8f517346fd 100644 --- a/rsts/howto/authentication/setup.rst +++ b/rsts/howto/authentication/setup.rst @@ -37,11 +37,6 @@ Follow `Google Docs - Okta OpenID Connect ------------------- @@ -86,7 +81,7 @@ Save and close your editor. .. prompt:: bash - kubectl deploy -n flyte flyteadmin -o yaml | grep "name: flyte-admin-config" + kubectl get deploy -n flyte flyteadmin -o yaml | grep "name: flyte-admin-config" This will output the name of the config map where the `client_id` needs to go. @@ -135,6 +130,34 @@ Save and exit your editor. kubectl rollout restart deployment/flyteadmin -n flyte +4. Edit FlytePropeller config enable auth as follows: + +.. prompt:: bash + + kubectl get deploy -n flyte flytepropeller -o yaml | grep "name: flyte-propeller-config" + +This will output the name of the config map where the `client_id` needs to go. + +.. prompt:: bash + + kubectl edit configmap -n flyte + +Follow the inline comments to make the necessary changes: + +.. code-block:: yaml + + admin: + # 1. Turn to true + useAuth: true + +Close the editor + +5. Restart `flytepropeller` for the changes to take effect: + +.. prompt:: bash + + kubectl rollout restart deployment/flytepropeller -n flyte + OAuth2 Authorization Server =========================== @@ -223,7 +246,7 @@ Save and close your editor. .. prompt:: bash - kubectl deploy -n flyte flytepropeller -o yaml | grep "name: flyte-propeller-config" + kubectl get deploy -n flyte flytepropeller -o yaml | grep "name: flyte-propeller-config" This will output the name of the config map where the `client_id` needs to go. @@ -236,15 +259,17 @@ Follow the inline comments to make the necessary changes: .. code-block:: yaml admin: - # 1. Turn to true - useAuth: true - # 2. Replace with the client_id provided by the OAuth2 Authorization Server above. + # 1. Replace with the client_id provided by the OAuth2 Authorization Server above. clientId: flytepropeller - # 3. Replace with the OAuth2 authorization server url provided in the previous step. - authorizationServerUrl: http://localhost:30081/ Close the editor +3. Restart `flytepropeller` for the changes to take effect: + +.. prompt:: bash + + kubectl rollout restart deployment/flytepropeller -n flyte + *************************** Continuous Integration - CI *************************** @@ -267,5 +292,3 @@ The following is a listing of the Flytekit configuration values we set in CI, al When using basic authentication, you'll need to specify a scope to the IDP (instead of ``openid``, which is only for OAuth). Set that here. * ``export FLYTE_PLATFORM_AUTH=True`` Set this to force Flytekit to use authentication, even if not required by Admin. This is useful as you're rolling out the requirement. - - From 16de21b50319d93480242203a59066d3ccaec135 Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Wed, 21 Apr 2021 13:54:06 -0700 Subject: [PATCH 58/74] Add github Signed-off-by: Haytham Abuelfutuh --- rsts/howto/authentication/index.rst | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/rsts/howto/authentication/index.rst b/rsts/howto/authentication/index.rst index 251a2489aa..2c1e99fe64 100644 --- a/rsts/howto/authentication/index.rst +++ b/rsts/howto/authentication/index.rst @@ -89,13 +89,13 @@ Using an External Authorization Server: Identity Providers Support ========================== -+-----------------+--------+-------------+---------------------+----------+-------+----------+ -| Feature | Okta | Google free | GC Identity Service | Azure AD | Auth0 | KeyCloak | -+=================+========+=============+=====================+==========+=======+==========+ -| OpenIdConnect | Yes | Yes | Yes | Yes | Yes | Yes | -+-----------------+--------+-------------+---------------------+----------+-------+----------+ -| Custom RP | Yes | No | Yes | Yes | ? | Yes | -+-----------------+--------+-------------+---------------------+----------+-------+----------+ ++-----------------+--------+-------------+---------------------+----------+-------+----------+--------+ +| Feature | Okta | Google free | GC Identity Service | Azure AD | Auth0 | KeyCloak | Github | ++=================+========+=============+=====================+==========+=======+==========+========+ +| OpenIdConnect | Yes | Yes | Yes | Yes | Yes | Yes | No | ++-----------------+--------+-------------+---------------------+----------+-------+----------+--------+ +| Custom RP | Yes | No | Yes | Yes | ? | Yes | No | ++-----------------+--------+-------------+---------------------+----------+-------+----------+--------+ ********** References From c2258faee372c7207172698471b8704fad4b2c5f Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Tue, 27 Apr 2021 13:08:02 -0700 Subject: [PATCH 59/74] Simplify setup Signed-off-by: Haytham Abuelfutuh --- deployment/eks/flyte_generated.yaml | 21 ++++ deployment/gcp/flyte_generated.yaml | 21 ++++ deployment/sandbox/flyte_generated.yaml | 39 ++++-- kustomize/base/ingress/ingress.yaml | 21 ++++ .../overlays/eks/flyte/ingress/ingress.yaml | 7 ++ .../eks/flyte/ingress/ingress_grpc.yaml | 14 +++ kustomize/overlays/sandbox/kustomization.yaml | 6 +- rsts/howto/authentication/migration.rst | 117 +++++++++++++++++- rsts/howto/authentication/setup.rst | 28 ----- 9 files changed, 231 insertions(+), 43 deletions(-) diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index 1608eec426..b9150faece 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -9296,6 +9296,13 @@ spec: number: 80 path: /.well-known/* pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /.well-known + pathType: ImplementationSpecific - backend: service: name: flyteadmin @@ -9388,3 +9395,17 @@ spec: number: 81 path: /flyteidl.service.AdminService/* pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 81 + path: /flyteidl.service.AuthMetadataService/* + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 81 + path: /flyteidl.service.IdentityService/* + pathType: ImplementationSpecific diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index ee31cb45ee..45e466b858 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -9278,6 +9278,20 @@ spec: number: 81 path: /flyteidl.service.AdminService pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 81 + path: /flyteidl.service.AuthMetadataService + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 81 + path: /flyteidl.service.IdentityService + pathType: ImplementationSpecific - backend: service: name: flyteadmin @@ -9334,6 +9348,13 @@ spec: number: 80 path: /openapi/* pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /.well-known + pathType: ImplementationSpecific - backend: service: name: flyteadmin diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index 59d97b8f1f..32f2958640 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -2831,7 +2831,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:796c99dd00b0c073c0c48b6a7da413a00417d525 + image: ghcr.io/flyteorg/flytepropeller:6519864dab8db1c8f183aec34fb3d03f97c92c05 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -2885,7 +2885,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:dc28c2b9e1c3372e62ab05ab775eae74963347ff + image: ghcr.io/flyteorg/flyteadmin:fe739e7a7d13e954ea269a3ef1a077bc39077a1a imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -2938,7 +2938,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:dc28c2b9e1c3372e62ab05ab775eae74963347ff + image: ghcr.io/flyteorg/flyteadmin:fe739e7a7d13e954ea269a3ef1a077bc39077a1a imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -2954,7 +2954,7 @@ spec: - seed-projects - flytesnacks - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:dc28c2b9e1c3372e62ab05ab775eae74963347ff + image: ghcr.io/flyteorg/flyteadmin:fe739e7a7d13e954ea269a3ef1a077bc39077a1a imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -2968,7 +2968,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:dc28c2b9e1c3372e62ab05ab775eae74963347ff + image: ghcr.io/flyteorg/flyteadmin:fe739e7a7d13e954ea269a3ef1a077bc39077a1a imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -2988,7 +2988,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flyteadmin:dc28c2b9e1c3372e62ab05ab775eae74963347ff + image: ghcr.io/flyteorg/flyteadmin:fe739e7a7d13e954ea269a3ef1a077bc39077a1a imagePullPolicy: IfNotPresent name: generate-secrets volumeMounts: @@ -3078,7 +3078,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:796c99dd00b0c073c0c48b6a7da413a00417d525 + image: ghcr.io/flyteorg/flytepropeller:6519864dab8db1c8f183aec34fb3d03f97c92c05 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -3397,7 +3397,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:dc28c2b9e1c3372e62ab05ab775eae74963347ff + image: ghcr.io/flyteorg/flyteadmin:fe739e7a7d13e954ea269a3ef1a077bc39077a1a imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -3581,7 +3581,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:796c99dd00b0c073c0c48b6a7da413a00417d525 + image: ghcr.io/flyteorg/flytepropeller:6519864dab8db1c8f183aec34fb3d03f97c92c05 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -3659,6 +3659,20 @@ spec: number: 81 path: /flyteidl.service.AdminService pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 81 + path: /flyteidl.service.AuthMetadataService + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 81 + path: /flyteidl.service.IdentityService + pathType: ImplementationSpecific - backend: service: name: flyteadmin @@ -3715,6 +3729,13 @@ spec: number: 80 path: /openapi/* pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 80 + path: /.well-known + pathType: ImplementationSpecific - backend: service: name: flyteadmin diff --git a/kustomize/base/ingress/ingress.yaml b/kustomize/base/ingress/ingress.yaml index 041421259f..9b0f257f0d 100644 --- a/kustomize/base/ingress/ingress.yaml +++ b/kustomize/base/ingress/ingress.yaml @@ -26,6 +26,20 @@ spec: name: flyteadmin port: number: 81 + - path: /flyteidl.service.AuthMetadataService + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 81 + - path: /flyteidl.service.IdentityService + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 81 # Port 87 in FlyteAdmin maps to the redoc container. - path: /openapi pathType: ImplementationSpecific @@ -85,6 +99,13 @@ spec: name: flyteadmin port: number: 80 + - path: /.well-known + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 - path: /.well-known/* pathType: ImplementationSpecific backend: diff --git a/kustomize/overlays/eks/flyte/ingress/ingress.yaml b/kustomize/overlays/eks/flyte/ingress/ingress.yaml index d88fb9b52e..6d2f06a235 100644 --- a/kustomize/overlays/eks/flyte/ingress/ingress.yaml +++ b/kustomize/overlays/eks/flyte/ingress/ingress.yaml @@ -78,6 +78,13 @@ spec: name: flyteadmin port: number: 80 + - path: /.well-known + pathType: ImplementationSpecific + backend: + service: + name: flyteadmin + port: + number: 80 - path: /login pathType: ImplementationSpecific backend: diff --git a/kustomize/overlays/eks/flyte/ingress/ingress_grpc.yaml b/kustomize/overlays/eks/flyte/ingress/ingress_grpc.yaml index f7ea333fb8..ce9c2e8892 100644 --- a/kustomize/overlays/eks/flyte/ingress/ingress_grpc.yaml +++ b/kustomize/overlays/eks/flyte/ingress/ingress_grpc.yaml @@ -29,3 +29,17 @@ spec: number: 81 path: /flyteidl.service.AdminService/* pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 81 + path: /flyteidl.service.AuthMetadataService/* + pathType: ImplementationSpecific + - backend: + service: + name: flyteadmin + port: + number: 81 + path: /flyteidl.service.IdentityService/* + pathType: ImplementationSpecific diff --git a/kustomize/overlays/sandbox/kustomization.yaml b/kustomize/overlays/sandbox/kustomization.yaml index 2af25fb495..c96414ffa1 100644 --- a/kustomize/overlays/sandbox/kustomization.yaml +++ b/kustomize/overlays/sandbox/kustomization.yaml @@ -25,7 +25,7 @@ resources: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: dc28c2b9e1c3372e62ab05ab775eae74963347ff # FLYTEADMIN_TAG override the tag + newTag: fe739e7a7d13e954ea269a3ef1a077bc39077a1a # FLYTEADMIN_TAG override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -37,11 +37,11 @@ images: newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: 796c99dd00b0c073c0c48b6a7da413a00417d525 # FLYTEPROPELLER_TAG override the tag + newTag: 6519864dab8db1c8f183aec34fb3d03f97c92c05 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Webhook - name: webhook # match images with this name - newTag: 796c99dd00b0c073c0c48b6a7da413a00417d525 # FLYTEPROPELLER_TAG override the tag + newTag: 6519864dab8db1c8f183aec34fb3d03f97c92c05 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Override postgres image to use alpine based (rather smaller) docker image - name: postgres diff --git a/rsts/howto/authentication/migration.rst b/rsts/howto/authentication/migration.rst index 27bd22aeba..ed1faff6ef 100644 --- a/rsts/howto/authentication/migration.rst +++ b/rsts/howto/authentication/migration.rst @@ -1,7 +1,118 @@ .. _howto_authentication_migrate: -################################################### -How to migrate your authentication config (pre 0.X) -################################################### +###################################################### +How to migrate your authentication config (pre 0.13.0) +###################################################### +Using Okta as an example, previously, you would've had something like the following. + +On the Okta side, you'd have: + +* An Application (OpenID Connect Web) for Flyte Admin itself (e.g. **0oal5rch46pVhCGF45d6**). +* An Application (OpenID Native app) for flyte-cli/flytectl (e.g. **0oal62nxuD6OSFSRq5d6**). +* These two applications would be assigned to the relevant users. +* An Application (Web) for Flyte Propeller (e.g. **0abc5rch46pVhCGF9876**). +* These applications would either use the default Authorization server, or you'd create a new one. + +On the Admin side, you'd have the following configuration + +.. code-block:: yaml + + server: + # ... other settings + security: + secure: false + useAuth: true + allowCors: true + allowedOrigins: + - "*" + allowedHeaders: + - "Content-Type" + oauth: + baseUrl: https://dev-62129345.okta.com/oauth2/default/ + scopes: + - profile + - openid + - email + claims: + iss: https://dev-62129345.okta.com/oauth2/default + aud: 0oal5rch46pVhCGF45d6 + clientId: 0oal5rch46pVhCGF45d6 + clientSecretFile: "/Users/ytong/etc/secrets/oauth/secret" + authorizeUrl: "https://dev-62129345.okta.com/oauth2/default/v1/authorize" + tokenUrl: "https://dev-62129345.okta.com/oauth2/default/v1/token" + callbackUrl: "http://localhost:8088/callback" + cookieHashKeyFile: "/Users/ytong/etc/secrets/hashkey/hashkey" + cookieBlockKeyFile: "/Users/ytong/etc/secrets/blockkey/blockkey" + redirectUrl: "/api/v1/projects" + thirdPartyConfig: + flyteClient: + clientId: 0oal62nxuD6OSFSRq5d6 + redirectUri: http://localhost:12345/callback + +From the flyte-cli side, you needed these two settings + +.. code-block:: bash + + FLYTE_PLATFORM_HTTP_URL=http://localhost:8088 FLYTE_CREDENTIALS_CLIENT_ID=0oal62nxuD6OSFSRq5d6 flyte-cli ... + +**FLYTE_PLATFORM_HTTP_URL** is used because **flyte-cli** uses only gRPC to communicate with Admin. It needs to know the HTTP port (which Admin hosts on a different port because of limitations of the +grpc-gateway library). **flyte-cli** uses this setting to talk to **/.well-known/oauth-authorization-server** to retrieve information regarding the auth endpoints. Previously this redirected to the +Okta Authorization Server's metadata endpoint. With this change, Admin now hosts its own (even if still using the external Authorization Service). + +After version `0.13.0 `__ of the platfform, you can still use the IDP as the Authorization Server if you so choose. That configuration would become: + +.. code-block:: yaml + + server: + # ... other settings + security: + secure: false + useAuth: true + allowCors: true + allowedOrigins: + - "*" + allowedHeaders: + - "Content-Type" + auth: + # This should point at your public http Uri. + httpPublicUri: http://localhost:8088/ + userAuth: + openId: + # Put the URL of the OpenID Connect provider. + baseUrl: https://dev-62129345.okta.com/oauth2/default # Okta with a custom Authorization Server + scopes: + - profile + - openid + - offline_access # Uncomment if OIdC supports issuing refresh tokens. + # Replace with the client id created for Flyte. + clientId: 0oal5rch46pVhCGF45d6 + appAuth: + # External delegates app auth responsibilities to an external authorization server, Internal means Flyte Admin does it itself + authServerType: External + thirdPartyConfig: + flyteClient: + clientId: 0oal62nxuD6OSFSRq5d6 + redirectUri: http://localhost:12345/callback + scopes: + - all + - offline + +Specifically, + +* The original **oauth** section has been moved two levels higher into its own section and renamed **auth** but enabling/disabling of authentication remains in the old place. +* Secrets by default will now be looked up in **/etc/secrets**. Use the following command to generate them. + +.. code-block:: bash + + flyteadmin secrets init -p /etc/secrets + +This will generate the new cookie hash/block keys, as well as other secrets Admin needs to run the Authorization server. + +* The **clientSecretFile** has been moved to **/etc/secrets/oidc_client_secret** so move that there. +* **claims** has been removed, just delete that. +* **authorizeUrl** and **tokenUrl** are no longer necessary. +* The **baseUrl** for the external Authorization Server is now in the **appAuth** section. +* The **thirdPartyConfig** has been moved to **appAuth** as well. +* **redirectUrl** has been defaulted to **/console**. If that's the value you want, then you no longer need this setting. diff --git a/rsts/howto/authentication/setup.rst b/rsts/howto/authentication/setup.rst index 8f517346fd..dfe00a3378 100644 --- a/rsts/howto/authentication/setup.rst +++ b/rsts/howto/authentication/setup.rst @@ -130,34 +130,6 @@ Save and exit your editor. kubectl rollout restart deployment/flyteadmin -n flyte -4. Edit FlytePropeller config enable auth as follows: - -.. prompt:: bash - - kubectl get deploy -n flyte flytepropeller -o yaml | grep "name: flyte-propeller-config" - -This will output the name of the config map where the `client_id` needs to go. - -.. prompt:: bash - - kubectl edit configmap -n flyte - -Follow the inline comments to make the necessary changes: - -.. code-block:: yaml - - admin: - # 1. Turn to true - useAuth: true - -Close the editor - -5. Restart `flytepropeller` for the changes to take effect: - -.. prompt:: bash - - kubectl rollout restart deployment/flytepropeller -n flyte - OAuth2 Authorization Server =========================== From 7cd7994b2551897d5d720a378d19575cf6f55064 Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Wed, 28 Apr 2021 12:36:38 -0700 Subject: [PATCH 60/74] Update docs for migration to include Propeller's config Signed-off-by: Haytham Abuelfutuh --- rsts/howto/authentication/migration.rst | 28 +++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/rsts/howto/authentication/migration.rst b/rsts/howto/authentication/migration.rst index ed1faff6ef..f27a07040e 100644 --- a/rsts/howto/authentication/migration.rst +++ b/rsts/howto/authentication/migration.rst @@ -116,3 +116,31 @@ This will generate the new cookie hash/block keys, as well as other secrets Admi * The **thirdPartyConfig** has been moved to **appAuth** as well. * **redirectUrl** has been defaulted to **/console**. If that's the value you want, then you no longer need this setting. +From propeller side, you might have a configuration section that looks like this: + +.. code-block:: yaml + + admin: + endpoint: dns:///mycompany.domain.com + useAuth: true + clientId: flytepropeller + clientSecretLocation: /etc/secrets/client_secret + tokenUrl: https://demo.nuclyde.io/oauth2/token + scopes: + - all + +This can now be simplified to: + +.. code-block:: yaml + + admin: + endpoint: dns:///mycompany.domain.com + # If you are using the built-in authorization server, you can delete these two lines: + clientId: flytepropeller + clientSecretLocation: /etc/secrets/client_secret + +Specifically, + +* **useAuth** is deprecated. Auth requirement will be discovered through an anonymous admin discovery call. +* **tokenUrl** and **scopes** will also be discovered through a metadata call. +* **clientId** and **clientSecretLocation** have defaults that work out of the box with the built-in authorization server (e.g. if you setup Google OpenID Connect). \ No newline at end of file From 08a9deee1dc559c61eb1d4b706b38b5f855bbdeb Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Fri, 30 Apr 2021 16:50:03 -0700 Subject: [PATCH 61/74] Update versions Signed-off-by: Haytham Abuelfutuh --- deployment/eks/flyte_generated.yaml | 40 ++++++++++++------- deployment/gcp/flyte_generated.yaml | 40 ++++++++++++------- deployment/sandbox/flyte_generated.yaml | 40 ++++++++++++------- deployment/test/flyte_generated.yaml | 36 +++++++++++------ .../headless/config/admin/server.yaml | 12 ++++++ kustomize/overlays/eks/kustomization.yaml | 8 ++-- kustomize/overlays/gcp/kustomization.yaml | 8 ++-- kustomize/overlays/sandbox/kustomization.yaml | 8 ++-- kustomize/overlays/test/kustomization.yaml | 8 ++-- rsts/howto/authentication/migration.rst | 15 ++++--- rsts/howto/authentication/setup.rst | 13 +----- 11 files changed, 142 insertions(+), 86 deletions(-) diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index b9150faece..8db369765f 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -8156,7 +8156,9 @@ data: httpPort: 8088 grpcPort: 8089 security: + # Controls whether to serve requests over SSL/TLS. secure: false + # Controls whether to enforce authentication. Follow the guide in https://docs.flyte.org/ on how to setup authentication. useAuth: false allowCors: true allowedOrigins: @@ -8166,6 +8168,15 @@ data: - "Content-Type" - "flyte-authorization" auth: + authorizedUris: + # This should point at your public http Uri. + - https://localhost:30081 + # This will be used by internal services in the same namespace as flyteadmin + - http://flyteadmin:80 + # This will be used by internal services in the same cluster but different namespaces + - http://flyteadmin.flyte.svc.cluster.local:80 + + # Controls app authentication config appAuth: thirdPartyConfig: flyteClient: @@ -8174,6 +8185,7 @@ data: scopes: - offline - all + # Controls user authentication userAuth: openId: baseUrl: https://accounts.google.com @@ -8224,7 +8236,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-k79c9bh7t4 + name: flyte-admin-config-49kdc8f554 namespace: flyte --- apiVersion: v1 @@ -8621,7 +8633,7 @@ spec: - --config - /etc/datacatalog/config/*.yaml - serve - image: ghcr.io/flyteorg/datacatalog:v0.3.1 + image: ghcr.io/flyteorg/datacatalog:v0.3.2 imagePullPolicy: IfNotPresent name: datacatalog ports: @@ -8644,7 +8656,7 @@ spec: - /etc/datacatalog/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/datacatalog:v0.3.1 + image: ghcr.io/flyteorg/datacatalog:v0.3.2 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8701,7 +8713,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.10.0 + image: ghcr.io/flyteorg/flytepropeller:v0.10.6 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -8749,7 +8761,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -8796,7 +8808,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8813,7 +8825,7 @@ spec: - flytesnacks - flytetester - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -8827,7 +8839,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -8847,7 +8859,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: generate-secrets volumeMounts: @@ -8858,7 +8870,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-k79c9bh7t4 + name: flyte-admin-config-49kdc8f554 name: config-volume - configMap: name: clusterresource-template-tkdkkt4cb5 @@ -8937,7 +8949,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.10.0 + image: ghcr.io/flyteorg/flytepropeller:v0.10.6 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -9129,7 +9141,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -9146,7 +9158,7 @@ spec: name: clusterresource-template-tkdkkt4cb5 name: resource-templates - configMap: - name: flyte-admin-config-k79c9bh7t4 + name: flyte-admin-config-49kdc8f554 name: config-volume - name: db-pass secret: @@ -9179,7 +9191,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.10.0 + image: ghcr.io/flyteorg/flytepropeller:v0.10.6 imagePullPolicy: IfNotPresent name: webhook volumeMounts: diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index 45e466b858..70759ada81 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -8147,7 +8147,9 @@ data: httpPort: 8088 grpcPort: 8089 security: + # Controls whether to serve requests over SSL/TLS. secure: false + # Controls whether to enforce authentication. Follow the guide in https://docs.flyte.org/ on how to setup authentication. useAuth: false allowCors: true allowedOrigins: @@ -8157,6 +8159,15 @@ data: - "Content-Type" - "flyte-authorization" auth: + authorizedUris: + # This should point at your public http Uri. + - https://localhost:30081 + # This will be used by internal services in the same namespace as flyteadmin + - http://flyteadmin:80 + # This will be used by internal services in the same cluster but different namespaces + - http://flyteadmin.flyte.svc.cluster.local:80 + + # Controls app authentication config appAuth: thirdPartyConfig: flyteClient: @@ -8165,6 +8176,7 @@ data: scopes: - offline - all + # Controls user authentication userAuth: openId: baseUrl: https://accounts.google.com @@ -8215,7 +8227,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-95g5286ffc + name: flyte-admin-config-7bk9bgt47c namespace: flyte --- apiVersion: v1 @@ -8653,7 +8665,7 @@ spec: - --config - /etc/datacatalog/config/*.yaml - serve - image: ghcr.io/flyteorg/datacatalog:v0.3.1 + image: ghcr.io/flyteorg/datacatalog:v0.3.2 imagePullPolicy: IfNotPresent name: datacatalog ports: @@ -8676,7 +8688,7 @@ spec: - /etc/datacatalog/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/datacatalog:v0.3.1 + image: ghcr.io/flyteorg/datacatalog:v0.3.2 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8733,7 +8745,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.10.0 + image: ghcr.io/flyteorg/flytepropeller:v0.10.6 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -8781,7 +8793,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -8828,7 +8840,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -8845,7 +8857,7 @@ spec: - flytesnacks - flytetester - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -8859,7 +8871,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -8879,7 +8891,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: generate-secrets volumeMounts: @@ -8890,7 +8902,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-95g5286ffc + name: flyte-admin-config-7bk9bgt47c name: config-volume - configMap: name: clusterresource-template-tkdkkt4cb5 @@ -8969,7 +8981,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.10.0 + image: ghcr.io/flyteorg/flytepropeller:v0.10.6 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -9161,7 +9173,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -9178,7 +9190,7 @@ spec: name: clusterresource-template-tkdkkt4cb5 name: resource-templates - configMap: - name: flyte-admin-config-95g5286ffc + name: flyte-admin-config-7bk9bgt47c name: config-volume - name: db-pass secret: @@ -9211,7 +9223,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.10.0 + image: ghcr.io/flyteorg/flytepropeller:v0.10.6 imagePullPolicy: IfNotPresent name: webhook volumeMounts: diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index 32f2958640..0a65369a90 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -2163,7 +2163,9 @@ data: httpPort: 8088 grpcPort: 8089 security: + # Controls whether to serve requests over SSL/TLS. secure: false + # Controls whether to enforce authentication. Follow the guide in https://docs.flyte.org/ on how to setup authentication. useAuth: false allowCors: true allowedOrigins: @@ -2173,6 +2175,15 @@ data: - "Content-Type" - "flyte-authorization" auth: + authorizedUris: + # This should point at your public http Uri. + - https://localhost:30081 + # This will be used by internal services in the same namespace as flyteadmin + - http://flyteadmin:80 + # This will be used by internal services in the same cluster but different namespaces + - http://flyteadmin.flyte.svc.cluster.local:80 + + # Controls app authentication config appAuth: thirdPartyConfig: flyteClient: @@ -2181,6 +2192,7 @@ data: scopes: - offline - all + # Controls user authentication userAuth: openId: baseUrl: https://accounts.google.com @@ -2223,7 +2235,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-kg9mm798hc + name: flyte-admin-config-d8gthbmdg5 namespace: flyte --- apiVersion: v1 @@ -2756,7 +2768,7 @@ spec: - --config - /etc/datacatalog/config/*.yaml - serve - image: ghcr.io/flyteorg/datacatalog:v0.3.1 + image: ghcr.io/flyteorg/datacatalog:v0.3.2 imagePullPolicy: IfNotPresent name: datacatalog ports: @@ -2774,7 +2786,7 @@ spec: - /etc/datacatalog/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/datacatalog:v0.3.1 + image: ghcr.io/flyteorg/datacatalog:v0.3.2 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -2831,7 +2843,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:6519864dab8db1c8f183aec34fb3d03f97c92c05 + image: ghcr.io/flyteorg/flytepropeller:v0.10.6 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -2885,7 +2897,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:fe739e7a7d13e954ea269a3ef1a077bc39077a1a + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -2938,7 +2950,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:fe739e7a7d13e954ea269a3ef1a077bc39077a1a + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -2954,7 +2966,7 @@ spec: - seed-projects - flytesnacks - flyteexamples - image: ghcr.io/flyteorg/flyteadmin:fe739e7a7d13e954ea269a3ef1a077bc39077a1a + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -2968,7 +2980,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:fe739e7a7d13e954ea269a3ef1a077bc39077a1a + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -2988,7 +3000,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flyteadmin:fe739e7a7d13e954ea269a3ef1a077bc39077a1a + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: generate-secrets volumeMounts: @@ -3002,7 +3014,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-kg9mm798hc + name: flyte-admin-config-d8gthbmdg5 name: config-volume - name: db-pass secret: @@ -3078,7 +3090,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:6519864dab8db1c8f183aec34fb3d03f97c92c05 + image: ghcr.io/flyteorg/flytepropeller:v0.10.6 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -3397,7 +3409,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:fe739e7a7d13e954ea269a3ef1a077bc39077a1a + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -3414,7 +3426,7 @@ spec: name: clusterresource-template-dtg8ff28mt name: resource-templates - configMap: - name: flyte-admin-config-kg9mm798hc + name: flyte-admin-config-d8gthbmdg5 name: config-volume - name: db-pass secret: @@ -3581,7 +3593,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:6519864dab8db1c8f183aec34fb3d03f97c92c05 + image: ghcr.io/flyteorg/flytepropeller:v0.10.6 imagePullPolicy: IfNotPresent name: webhook volumeMounts: diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index 2b205ed941..8c6fc282b9 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -311,7 +311,9 @@ data: httpPort: 8088 grpcPort: 8089 security: + # Controls whether to serve requests over SSL/TLS. secure: false + # Controls whether to enforce authentication. Follow the guide in https://docs.flyte.org/ on how to setup authentication. useAuth: false allowCors: true allowedOrigins: @@ -321,6 +323,15 @@ data: - "Content-Type" - "flyte-authorization" auth: + authorizedUris: + # This should point at your public http Uri. + - https://localhost:30081 + # This will be used by internal services in the same namespace as flyteadmin + - http://flyteadmin:80 + # This will be used by internal services in the same cluster but different namespaces + - http://flyteadmin.flyte.svc.cluster.local:80 + + # Controls app authentication config appAuth: thirdPartyConfig: flyteClient: @@ -329,6 +340,7 @@ data: scopes: - offline - all + # Controls user authentication userAuth: openId: baseUrl: https://accounts.google.com @@ -371,7 +383,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-hb74mc6784 + name: flyte-admin-config-cm5bh86k8m namespace: flyte --- apiVersion: v1 @@ -616,7 +628,7 @@ spec: - --config - /etc/datacatalog/config/*.yaml - serve - image: ghcr.io/flyteorg/datacatalog:v0.3.1 + image: ghcr.io/flyteorg/datacatalog:v0.3.2 imagePullPolicy: IfNotPresent name: datacatalog ports: @@ -634,7 +646,7 @@ spec: - /etc/datacatalog/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/datacatalog:v0.3.1 + image: ghcr.io/flyteorg/datacatalog:v0.3.2 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -691,7 +703,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.10.0 + image: ghcr.io/flyteorg/flytepropeller:v0.10.6 imagePullPolicy: IfNotPresent name: webhook volumeMounts: @@ -739,7 +751,7 @@ spec: - --config - /etc/flyte/config/*.yaml - serve - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: flyteadmin ports: @@ -792,7 +804,7 @@ spec: - /etc/flyte/config/*.yaml - migrate - run - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: run-migrations volumeMounts: @@ -808,7 +820,7 @@ spec: - seed-projects - flytetester - flytesnacks - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: seed-projects volumeMounts: @@ -822,7 +834,7 @@ spec: - /etc/flyte/config/*.yaml - clusterresource - sync - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: sync-cluster-resources volumeMounts: @@ -842,7 +854,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flyteadmin:v0.4.7 + image: ghcr.io/flyteorg/flyteadmin:v0.5.0 imagePullPolicy: IfNotPresent name: generate-secrets volumeMounts: @@ -856,7 +868,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-hb74mc6784 + name: flyte-admin-config-cm5bh86k8m name: config-volume - name: db-pass secret: @@ -898,7 +910,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.name - image: ghcr.io/flyteorg/flytepropeller:v0.10.0 + image: ghcr.io/flyteorg/flytepropeller:v0.10.6 imagePullPolicy: IfNotPresent name: flytepropeller ports: @@ -1017,7 +1029,7 @@ spec: valueFrom: fieldRef: fieldPath: metadata.namespace - image: ghcr.io/flyteorg/flytepropeller:v0.10.0 + image: ghcr.io/flyteorg/flytepropeller:v0.10.6 imagePullPolicy: IfNotPresent name: webhook volumeMounts: diff --git a/kustomize/base/single_cluster/headless/config/admin/server.yaml b/kustomize/base/single_cluster/headless/config/admin/server.yaml index 096b9fe8e6..d78ed29f87 100644 --- a/kustomize/base/single_cluster/headless/config/admin/server.yaml +++ b/kustomize/base/single_cluster/headless/config/admin/server.yaml @@ -2,7 +2,9 @@ server: httpPort: 8088 grpcPort: 8089 security: + # Controls whether to serve requests over SSL/TLS. secure: false + # Controls whether to enforce authentication. Follow the guide in https://docs.flyte.org/ on how to setup authentication. useAuth: false allowCors: true allowedOrigins: @@ -12,6 +14,15 @@ server: - "Content-Type" - "flyte-authorization" auth: + authorizedUris: + # This should point at your public http Uri. + - https://localhost:30081 + # This will be used by internal services in the same namespace as flyteadmin + - http://flyteadmin:80 + # This will be used by internal services in the same cluster but different namespaces + - http://flyteadmin.flyte.svc.cluster.local:80 + + # Controls app authentication config appAuth: thirdPartyConfig: flyteClient: @@ -20,6 +31,7 @@ auth: scopes: - offline - all + # Controls user authentication userAuth: openId: baseUrl: https://accounts.google.com diff --git a/kustomize/overlays/eks/kustomization.yaml b/kustomize/overlays/eks/kustomization.yaml index 4457715cff..0df31c0358 100644 --- a/kustomize/overlays/eks/kustomization.yaml +++ b/kustomize/overlays/eks/kustomization.yaml @@ -21,7 +21,7 @@ bases: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.4.7 # FLYTEADMIN_TAG override the tag + newTag: v0.5.0 # FLYTEADMIN_TAG override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -29,13 +29,13 @@ images: newName: ghcr.io/flyteorg/flyteconsole # override the namep # Flyte DataCatalog - name: datacatalog # match images with this name - newTag: v0.3.1 # DATACATALOG_TAG override the tag + newTag: v0.3.2 # DATACATALOG_TAG override the tag newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.6 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Webhook - name: webhook # match images with this name - newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.6 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name \ No newline at end of file diff --git a/kustomize/overlays/gcp/kustomization.yaml b/kustomize/overlays/gcp/kustomization.yaml index 7fcca6183f..c9359d5dc7 100644 --- a/kustomize/overlays/gcp/kustomization.yaml +++ b/kustomize/overlays/gcp/kustomization.yaml @@ -23,7 +23,7 @@ bases: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.4.7 # FLYTEADMIN_TAG override the tag + newTag: v0.5.0 # FLYTEADMIN_TAG override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -31,13 +31,13 @@ images: newName: ghcr.io/flyteorg/flyteconsole # override the namep # Flyte DataCatalog - name: datacatalog # match images with this name - newTag: v0.3.1 # DATACATALOG_TAG override the tag + newTag: v0.3.2 # DATACATALOG_TAG override the tag newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.6 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Webhook - name: webhook # match images with this name - newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.6 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name \ No newline at end of file diff --git a/kustomize/overlays/sandbox/kustomization.yaml b/kustomize/overlays/sandbox/kustomization.yaml index c96414ffa1..67317484f3 100644 --- a/kustomize/overlays/sandbox/kustomization.yaml +++ b/kustomize/overlays/sandbox/kustomization.yaml @@ -25,7 +25,7 @@ resources: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: fe739e7a7d13e954ea269a3ef1a077bc39077a1a # FLYTEADMIN_TAG override the tag + newTag: v0.5.0 # FLYTEADMIN_TAG override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -33,15 +33,15 @@ images: newName: ghcr.io/flyteorg/flyteconsole # override the namep # Flyte DataCatalog - name: datacatalog # match images with this name - newTag: v0.3.1 # DATACATALOG_TAG override the tag + newTag: v0.3.2 # DATACATALOG_TAG override the tag newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: 6519864dab8db1c8f183aec34fb3d03f97c92c05 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.6 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Webhook - name: webhook # match images with this name - newTag: 6519864dab8db1c8f183aec34fb3d03f97c92c05 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.6 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Override postgres image to use alpine based (rather smaller) docker image - name: postgres diff --git a/kustomize/overlays/test/kustomization.yaml b/kustomize/overlays/test/kustomization.yaml index ddf689f58c..644726e20f 100644 --- a/kustomize/overlays/test/kustomization.yaml +++ b/kustomize/overlays/test/kustomization.yaml @@ -19,7 +19,7 @@ bases: images: # FlyteAdmin - name: flyteadmin # match images with this name - newTag: v0.4.7 # FLYTEADMIN_TAG override the tag + newTag: v0.5.0 # FLYTEADMIN_TAG override the tag newName: ghcr.io/flyteorg/flyteadmin # override the name # FlyteConsole - name: flyteconsole # match images with this name @@ -27,15 +27,15 @@ images: newName: ghcr.io/flyteorg/flyteconsole # override the namep # Flyte DataCatalog - name: datacatalog # match images with this name - newTag: v0.3.1 # DATACATALOG_TAG override the tag + newTag: v0.3.2 # DATACATALOG_TAG override the tag newName: ghcr.io/flyteorg/datacatalog # override the name # FlytePropeller - name: flytepropeller # match images with this name - newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.6 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Webhook - name: webhook # match images with this name - newTag: v0.10.0 # FLYTEPROPELLER_TAG override the tag + newTag: v0.10.6 # FLYTEPROPELLER_TAG override the tag newName: ghcr.io/flyteorg/flytepropeller # override the name # Override postgres image to use alpine based (rather smaller) docker image - name: postgres diff --git a/rsts/howto/authentication/migration.rst b/rsts/howto/authentication/migration.rst index f27a07040e..cc919c5040 100644 --- a/rsts/howto/authentication/migration.rst +++ b/rsts/howto/authentication/migration.rst @@ -60,7 +60,7 @@ From the flyte-cli side, you needed these two settings grpc-gateway library). **flyte-cli** uses this setting to talk to **/.well-known/oauth-authorization-server** to retrieve information regarding the auth endpoints. Previously this redirected to the Okta Authorization Server's metadata endpoint. With this change, Admin now hosts its own (even if still using the external Authorization Service). -After version `0.13.0 `__ of the platfform, you can still use the IDP as the Authorization Server if you so choose. That configuration would become: +After version `0.13.0 `__ of the platfform, you can still use the IdP as the Authorization Server if you so choose. That configuration would become: .. code-block:: yaml @@ -75,8 +75,13 @@ After version `0.13.0 `__ of the allowedHeaders: - "Content-Type" auth: - # This should point at your public http Uri. - httpPublicUri: http://localhost:8088/ + authorizedUris: + # This should point at your public http Uri. + - https://flyte.mycompany.com + # This will be used by internal services in the same namespace as flyteadmin + - http://flyteadmin:80 + # This will be used by internal services in the same cluster but different namespaces + - http://flyteadmin.flyte.svc.cluster.local:80 userAuth: openId: # Put the URL of the OpenID Connect provider. @@ -135,12 +140,12 @@ This can now be simplified to: admin: endpoint: dns:///mycompany.domain.com - # If you are using the built-in authorization server, you can delete these two lines: + # If you are using the built-in authorization server, you can delete the following two lines: clientId: flytepropeller clientSecretLocation: /etc/secrets/client_secret Specifically, -* **useAuth** is deprecated. Auth requirement will be discovered through an anonymous admin discovery call. +* **useAuth** is deprecated and will be removed in a future version. Auth requirement will be discovered through an anonymous admin discovery call. * **tokenUrl** and **scopes** will also be discovered through a metadata call. * **clientId** and **clientSecretLocation** have defaults that work out of the box with the built-in authorization server (e.g. if you setup Google OpenID Connect). \ No newline at end of file diff --git a/rsts/howto/authentication/setup.rst b/rsts/howto/authentication/setup.rst index dfe00a3378..4cab5e1b0f 100644 --- a/rsts/howto/authentication/setup.rst +++ b/rsts/howto/authentication/setup.rst @@ -94,21 +94,12 @@ Follow the inline comments to make the necessary changes: .. code-block:: yaml server: - httpPort: 8088 - grpcPort: 8089 - grpcServerReflection: true - kube-config: /Users/haythamabuelfutuh/.kube/config + ... security: secure: false # 1. Enable Auth by turning useAuth to true useAuth: true - allowCors: true - allowedOrigins: - # Accepting all domains for Sandbox installation - - "*" - allowedHeaders: - - "Content-Type" - - "flyte-athorization" + ... auth: userAuth: openId: From 0acbea42b1a8497cb1d8f6b6e558a6266b33442e Mon Sep 17 00:00:00 2001 From: Haytham Abuelfutuh Date: Tue, 13 Apr 2021 13:58:54 -0700 Subject: [PATCH 62/74] Fix default eks storage config (#907) * Fix default eks storage config Signed-off-by: Haytham Abuelfutuh * kustomize Signed-off-by: Haytham Abuelfutuh * bump Signed-off-by: Haytham Abuelfutuh --- deployment/eks/flyte_generated.yaml | 54 +++++++++---------- .../eks/flyte/config/common/storage.yaml | 12 ++--- 2 files changed, 29 insertions(+), 37 deletions(-) diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index 8db369765f..1c0b20cd3a 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -8080,14 +8080,12 @@ data: storage: type: stow stow: - kind: google + kind: s3 config: - json: "" - # TODO: replace with the GCP project ID - project_id: - scopes: https://www.googleapis.com/auth/devstorage.read_write - # TODO replace with the container (bucket) in GCS used by Flyte as intermediate store - container: "flyte" + auth_type: iam + region: us-east-2 + # TODO replace with the container (bucket) in s3 used by Flyte as intermediate store + container: "flyte-demo" # NOTE this cache configuration is purely for propeller. But since we are having a common storage # config, we are configuring this value. In production create a separate storage config for # propeller and increase the cache size @@ -8098,7 +8096,7 @@ data: maxDownloadMBs: 10 kind: ConfigMap metadata: - name: datacatalog-config-2bb94tc5b5 + name: datacatalog-config-fktmhdgfk8 namespace: flyte --- apiVersion: v1 @@ -8207,14 +8205,12 @@ data: storage: type: stow stow: - kind: google + kind: s3 config: - json: "" - # TODO: replace with the GCP project ID - project_id: - scopes: https://www.googleapis.com/auth/devstorage.read_write - # TODO replace with the container (bucket) in GCS used by Flyte as intermediate store - container: "flyte" + auth_type: iam + region: us-east-2 + # TODO replace with the container (bucket) in s3 used by Flyte as intermediate store + container: "flyte-demo" # NOTE this cache configuration is purely for propeller. But since we are having a common storage # config, we are configuring this value. In production create a separate storage config for # propeller and increase the cache size @@ -8236,7 +8232,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-49kdc8f554 + name: flyte-admin-config-4fd96h5bd6 namespace: flyte --- apiVersion: v1 @@ -8380,14 +8376,12 @@ data: storage: type: stow stow: - kind: google + kind: s3 config: - json: "" - # TODO: replace with the GCP project ID - project_id: - scopes: https://www.googleapis.com/auth/devstorage.read_write - # TODO replace with the container (bucket) in GCS used by Flyte as intermediate store - container: "flyte" + auth_type: iam + region: us-east-2 + # TODO replace with the container (bucket) in s3 used by Flyte as intermediate store + container: "flyte-demo" # NOTE this cache configuration is purely for propeller. But since we are having a common storage # config, we are configuring this value. In production create a separate storage config for # propeller and increase the cache size @@ -8409,7 +8403,7 @@ data: cloudwatch-log-group: kind: ConfigMap metadata: - name: flyte-propeller-config-c2hh7hhkh4 + name: flyte-propeller-config-bk6dmfc8tm namespace: flyte --- apiVersion: v1 @@ -8669,7 +8663,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: datacatalog-config-2bb94tc5b5 + name: datacatalog-config-fktmhdgfk8 name: config-volume - name: db-pass secret: @@ -8726,7 +8720,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-c2hh7hhkh4 + name: flyte-propeller-config-bk6dmfc8tm name: config-volume - name: webhook-certs secret: @@ -8870,7 +8864,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-49kdc8f554 + name: flyte-admin-config-4fd96h5bd6 name: config-volume - configMap: name: clusterresource-template-tkdkkt4cb5 @@ -8967,7 +8961,7 @@ spec: serviceAccountName: flytepropeller volumes: - configMap: - name: flyte-propeller-config-c2hh7hhkh4 + name: flyte-propeller-config-bk6dmfc8tm name: config-volume - name: auth secret: @@ -9158,7 +9152,7 @@ spec: name: clusterresource-template-tkdkkt4cb5 name: resource-templates - configMap: - name: flyte-admin-config-49kdc8f554 + name: flyte-admin-config-4fd96h5bd6 name: config-volume - name: db-pass secret: @@ -9201,7 +9195,7 @@ spec: serviceAccountName: flyte-pod-webhook volumes: - configMap: - name: flyte-propeller-config-c2hh7hhkh4 + name: flyte-propeller-config-bk6dmfc8tm name: config-volume ttlSecondsAfterFinished: 0 --- diff --git a/kustomize/overlays/eks/flyte/config/common/storage.yaml b/kustomize/overlays/eks/flyte/config/common/storage.yaml index 6d678d0bbc..39dd393112 100644 --- a/kustomize/overlays/eks/flyte/config/common/storage.yaml +++ b/kustomize/overlays/eks/flyte/config/common/storage.yaml @@ -1,14 +1,12 @@ storage: type: stow stow: - kind: google + kind: s3 config: - json: "" - # TODO: replace with the GCP project ID - project_id: - scopes: https://www.googleapis.com/auth/devstorage.read_write - # TODO replace with the container (bucket) in GCS used by Flyte as intermediate store - container: "flyte" + auth_type: iam + region: us-east-2 + # TODO replace with the container (bucket) in s3 used by Flyte as intermediate store + container: "flyte-demo" # NOTE this cache configuration is purely for propeller. But since we are having a common storage # config, we are configuring this value. In production create a separate storage config for # propeller and increase the cache size From e0dcb4dd9d6bb74013d2b70c6dd6a278a5c930c9 Mon Sep 17 00:00:00 2001 From: Katrina Rogan Date: Thu, 15 Apr 2021 15:13:07 -0700 Subject: [PATCH 63/74] Remove flyteadmin config options which now have defaults (#917) --- deployment/eks/flyte_generated.yaml | 23 ++++--------------- deployment/gcp/flyte_generated.yaml | 19 ++++----------- deployment/sandbox/flyte_generated.yaml | 22 ++++-------------- deployment/test/flyte_generated.yaml | 20 +++------------- .../admindeployment/clustersync/cron.yaml | 2 +- .../base/admindeployment/deployment.yaml | 2 +- .../headless/config/admin/remote_data.yaml | 5 ---- .../headless/config/admin/server.yaml | 8 ------- .../headless/kustomization.yaml | 1 - .../eks/flyte/config/admin/remote_data.yaml | 6 ----- .../overlays/eks/flyte/kustomization.yaml | 1 - kustomize/overlays/eks/kustomization.yaml | 2 +- .../gcp/flyte/config/admin/remote_data.yaml | 2 -- .../overlays/gcp/flyte/kustomization.yaml | 1 - kustomize/overlays/gcp/kustomization.yaml | 2 +- .../sandbox/flyte/admin/deployment.yaml | 6 ++--- .../overlays/test/flyte/admin/deployment.yaml | 6 ++--- 17 files changed, 25 insertions(+), 103 deletions(-) delete mode 100644 kustomize/base/single_cluster/headless/config/admin/remote_data.yaml delete mode 100644 kustomize/overlays/eks/flyte/config/admin/remote_data.yaml delete mode 100644 kustomize/overlays/gcp/flyte/config/admin/remote_data.yaml diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index 1c0b20cd3a..c2dbb0c94d 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -8142,13 +8142,6 @@ data: logger: show-source: true level: 2 - remote_data.yaml: | - remoteData: - # TODO change this to match the region of the s3 bucket - region: "us-west-2" - scheme: aws - signedUrls: - durationMinutes: 3 server.yaml: | server: httpPort: 8088 @@ -8193,14 +8186,6 @@ data: clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com flyteadmin: roleNameKey: "iam.amazonaws.com/role" - profilerPort: 10254 - metricsScope: "flyte:" - metadataStoragePrefix: - - "metadata" - - "admin" - eventVersion: 1 - testing: - host: http://flyteadmin storage.yaml: | storage: type: stow @@ -8232,7 +8217,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-4fd96h5bd6 + name: flyte-admin-config-fh8cdtgk68 namespace: flyte --- apiVersion: v1 @@ -8747,7 +8732,7 @@ spec: labels: app: flyteadmin app.kubernetes.io/name: flyteadmin - app.kubernetes.io/version: 0.3.38 + app.kubernetes.io/version: 0.4.13 spec: containers: - command: @@ -8864,7 +8849,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-4fd96h5bd6 + name: flyte-admin-config-fh8cdtgk68 name: config-volume - configMap: name: clusterresource-template-tkdkkt4cb5 @@ -9152,7 +9137,7 @@ spec: name: clusterresource-template-tkdkkt4cb5 name: resource-templates - configMap: - name: flyte-admin-config-4fd96h5bd6 + name: flyte-admin-config-fh8cdtgk68 name: config-volume - name: db-pass secret: diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index 70759ada81..afe8f3d876 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -8139,9 +8139,6 @@ data: logger: show-source: true level: 2 - remote_data.yaml: | - remoteData: - scheme: "gcp" server.yaml: | server: httpPort: 8088 @@ -8186,14 +8183,6 @@ data: clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com flyteadmin: roleNameKey: "iam.amazonaws.com/role" - profilerPort: 10254 - metricsScope: "flyte:" - metadataStoragePrefix: - - "metadata" - - "admin" - eventVersion: 1 - testing: - host: http://flyteadmin storage.yaml: | storage: type: stow @@ -8227,7 +8216,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-7bk9bgt47c + name: flyte-admin-config-2gh2k52tb8 namespace: flyte --- apiVersion: v1 @@ -8785,7 +8774,7 @@ spec: labels: app: flyteadmin app.kubernetes.io/name: flyteadmin - app.kubernetes.io/version: 0.3.38 + app.kubernetes.io/version: 0.4.13 spec: containers: - command: @@ -8902,7 +8891,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-7bk9bgt47c + name: flyte-admin-config-2gh2k52tb8 name: config-volume - configMap: name: clusterresource-template-tkdkkt4cb5 @@ -9190,7 +9179,7 @@ spec: name: clusterresource-template-tkdkkt4cb5 name: resource-templates - configMap: - name: flyte-admin-config-7bk9bgt47c + name: flyte-admin-config-2gh2k52tb8 name: config-volume - name: db-pass secret: diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index 0a65369a90..0212b406db 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -2152,12 +2152,6 @@ data: logger: show-source: true level: 2 - remote_data.yaml: | - remoteData: - region: "us-east-1" - scheme: "local" - signedUrls: - durationMinutes: 3 server.yaml: | server: httpPort: 8088 @@ -2202,14 +2196,6 @@ data: clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com flyteadmin: roleNameKey: "iam.amazonaws.com/role" - profilerPort: 10254 - metricsScope: "flyte:" - metadataStoragePrefix: - - "metadata" - - "admin" - eventVersion: 1 - testing: - host: http://flyteadmin storage.yaml: |+ storage: type: minio @@ -2235,7 +2221,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-d8gthbmdg5 + name: flyte-admin-config-cbf8chmd2t namespace: flyte --- apiVersion: v1 @@ -2889,7 +2875,7 @@ spec: labels: app: flyteadmin app.kubernetes.io/name: flyteadmin - app.kubernetes.io/version: 0.3.38 + app.kubernetes.io/version: 0.4.13 spec: containers: - command: @@ -3014,7 +3000,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-d8gthbmdg5 + name: flyte-admin-config-cbf8chmd2t name: config-volume - name: db-pass secret: @@ -3426,7 +3412,7 @@ spec: name: clusterresource-template-dtg8ff28mt name: resource-templates - configMap: - name: flyte-admin-config-d8gthbmdg5 + name: flyte-admin-config-cbf8chmd2t name: config-volume - name: db-pass secret: diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index 8c6fc282b9..ca72e1f974 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -300,12 +300,6 @@ data: logger: show-source: true level: 2 - remote_data.yaml: | - remoteData: - region: "us-east-1" - scheme: "local" - signedUrls: - durationMinutes: 3 server.yaml: | server: httpPort: 8088 @@ -350,14 +344,6 @@ data: clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com flyteadmin: roleNameKey: "iam.amazonaws.com/role" - profilerPort: 10254 - metricsScope: "flyte:" - metadataStoragePrefix: - - "metadata" - - "admin" - eventVersion: 1 - testing: - host: http://flyteadmin storage.yaml: |+ storage: type: minio @@ -383,7 +369,7 @@ data: gpu: 1 kind: ConfigMap metadata: - name: flyte-admin-config-cm5bh86k8m + name: flyte-admin-config-t8ct89hdm5 namespace: flyte --- apiVersion: v1 @@ -743,7 +729,7 @@ spec: labels: app: flyteadmin app.kubernetes.io/name: flyteadmin - app.kubernetes.io/version: 0.3.38 + app.kubernetes.io/version: 0.4.13 spec: containers: - command: @@ -868,7 +854,7 @@ spec: - emptyDir: {} name: shared-data - configMap: - name: flyte-admin-config-cm5bh86k8m + name: flyte-admin-config-t8ct89hdm5 name: config-volume - name: db-pass secret: diff --git a/kustomize/base/admindeployment/clustersync/cron.yaml b/kustomize/base/admindeployment/clustersync/cron.yaml index b23ffaf6d5..e3fbfdfa69 100644 --- a/kustomize/base/admindeployment/clustersync/cron.yaml +++ b/kustomize/base/admindeployment/clustersync/cron.yaml @@ -12,7 +12,7 @@ spec: serviceAccountName: flyteadmin containers: - name: sync-cluster-resources - image: flyteadmin:v0.3.29 + image: flyteadmin:v0.4.13 imagePullPolicy: IfNotPresent command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "clusterresource", "sync"] volumeMounts: diff --git a/kustomize/base/admindeployment/deployment.yaml b/kustomize/base/admindeployment/deployment.yaml index f8df687efc..1c21394c91 100644 --- a/kustomize/base/admindeployment/deployment.yaml +++ b/kustomize/base/admindeployment/deployment.yaml @@ -16,7 +16,7 @@ spec: labels: app: flyteadmin app.kubernetes.io/name: flyteadmin - app.kubernetes.io/version: 0.3.38 + app.kubernetes.io/version: 0.4.13 annotations: prometheus.io/scrape: "true" prometheus.io/port: "10254" diff --git a/kustomize/base/single_cluster/headless/config/admin/remote_data.yaml b/kustomize/base/single_cluster/headless/config/admin/remote_data.yaml deleted file mode 100644 index 41719a0e66..0000000000 --- a/kustomize/base/single_cluster/headless/config/admin/remote_data.yaml +++ /dev/null @@ -1,5 +0,0 @@ -remoteData: - region: "us-east-1" - scheme: "local" - signedUrls: - durationMinutes: 3 diff --git a/kustomize/base/single_cluster/headless/config/admin/server.yaml b/kustomize/base/single_cluster/headless/config/admin/server.yaml index d78ed29f87..e13f3ad156 100644 --- a/kustomize/base/single_cluster/headless/config/admin/server.yaml +++ b/kustomize/base/single_cluster/headless/config/admin/server.yaml @@ -41,11 +41,3 @@ auth: clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com flyteadmin: roleNameKey: "iam.amazonaws.com/role" - profilerPort: 10254 - metricsScope: "flyte:" - metadataStoragePrefix: - - "metadata" - - "admin" - eventVersion: 1 - testing: - host: http://flyteadmin diff --git a/kustomize/base/single_cluster/headless/kustomization.yaml b/kustomize/base/single_cluster/headless/kustomization.yaml index c63c457102..d4cf4a7520 100644 --- a/kustomize/base/single_cluster/headless/kustomization.yaml +++ b/kustomize/base/single_cluster/headless/kustomization.yaml @@ -19,7 +19,6 @@ configMapGenerator: - ./config/admin/domain.yaml - ./config/admin/db.yaml - ./config/admin/cluster_resources.yaml - - ./config/admin/remote_data.yaml - ./config/admin/task_resource_defaults.yaml - ./config/common/storage.yaml - ./config/common/logger.yaml diff --git a/kustomize/overlays/eks/flyte/config/admin/remote_data.yaml b/kustomize/overlays/eks/flyte/config/admin/remote_data.yaml deleted file mode 100644 index 5e56e0740e..0000000000 --- a/kustomize/overlays/eks/flyte/config/admin/remote_data.yaml +++ /dev/null @@ -1,6 +0,0 @@ -remoteData: - # TODO change this to match the region of the s3 bucket - region: "us-west-2" - scheme: aws - signedUrls: - durationMinutes: 3 diff --git a/kustomize/overlays/eks/flyte/kustomization.yaml b/kustomize/overlays/eks/flyte/kustomization.yaml index 0da9ffcee5..ebc66e3425 100644 --- a/kustomize/overlays/eks/flyte/kustomization.yaml +++ b/kustomize/overlays/eks/flyte/kustomization.yaml @@ -27,7 +27,6 @@ configMapGenerator: - behavior: merge files: - ./config/admin/db.yaml - - ./config/admin/remote_data.yaml - ./config/admin/task_resource_defaults.yaml - ./config/common/storage.yaml name: flyte-admin-config diff --git a/kustomize/overlays/eks/kustomization.yaml b/kustomize/overlays/eks/kustomization.yaml index 0df31c0358..358eaefa80 100644 --- a/kustomize/overlays/eks/kustomization.yaml +++ b/kustomize/overlays/eks/kustomization.yaml @@ -38,4 +38,4 @@ images: # Webhook - name: webhook # match images with this name newTag: v0.10.6 # FLYTEPROPELLER_TAG override the tag - newName: ghcr.io/flyteorg/flytepropeller # override the name \ No newline at end of file + newName: ghcr.io/flyteorg/flytepropeller # override the name diff --git a/kustomize/overlays/gcp/flyte/config/admin/remote_data.yaml b/kustomize/overlays/gcp/flyte/config/admin/remote_data.yaml deleted file mode 100644 index 8af409c9a4..0000000000 --- a/kustomize/overlays/gcp/flyte/config/admin/remote_data.yaml +++ /dev/null @@ -1,2 +0,0 @@ -remoteData: - scheme: "gcp" diff --git a/kustomize/overlays/gcp/flyte/kustomization.yaml b/kustomize/overlays/gcp/flyte/kustomization.yaml index b369942368..bb08dcf05a 100644 --- a/kustomize/overlays/gcp/flyte/kustomization.yaml +++ b/kustomize/overlays/gcp/flyte/kustomization.yaml @@ -24,7 +24,6 @@ configMapGenerator: - behavior: merge files: - ./config/admin/db.yaml - - ./config/admin/remote_data.yaml - ./config/admin/task_resource_defaults.yaml - ./config/common/storage.yaml name: flyte-admin-config diff --git a/kustomize/overlays/gcp/kustomization.yaml b/kustomize/overlays/gcp/kustomization.yaml index c9359d5dc7..60d76f7a65 100644 --- a/kustomize/overlays/gcp/kustomization.yaml +++ b/kustomize/overlays/gcp/kustomization.yaml @@ -40,4 +40,4 @@ images: # Webhook - name: webhook # match images with this name newTag: v0.10.6 # FLYTEPROPELLER_TAG override the tag - newName: ghcr.io/flyteorg/flytepropeller # override the name \ No newline at end of file + newName: ghcr.io/flyteorg/flytepropeller # override the name diff --git a/kustomize/overlays/sandbox/flyte/admin/deployment.yaml b/kustomize/overlays/sandbox/flyte/admin/deployment.yaml index f3a7c1ef20..5a30adeb68 100644 --- a/kustomize/overlays/sandbox/flyte/admin/deployment.yaml +++ b/kustomize/overlays/sandbox/flyte/admin/deployment.yaml @@ -20,7 +20,7 @@ spec: do echo waiting for database; sleep 2; done;", ] - name: run-migrations - image: flyteadmin:v0.4.0 + image: flyteadmin:v0.4.13 imagePullPolicy: IfNotPresent command: [ @@ -34,7 +34,7 @@ spec: - name: config-volume mountPath: /etc/flyte/config - name: seed-projects - image: flyteadmin:v0.4.0 + image: flyteadmin:v0.4.13 imagePullPolicy: IfNotPresent command: [ @@ -50,7 +50,7 @@ spec: - name: config-volume mountPath: /etc/flyte/config - name: sync-cluster-resources - image: flyteadmin:v0.4.0 + image: flyteadmin:v0.4.13 imagePullPolicy: IfNotPresent command: [ diff --git a/kustomize/overlays/test/flyte/admin/deployment.yaml b/kustomize/overlays/test/flyte/admin/deployment.yaml index b4b9d59183..70db6afa6c 100644 --- a/kustomize/overlays/test/flyte/admin/deployment.yaml +++ b/kustomize/overlays/test/flyte/admin/deployment.yaml @@ -17,7 +17,7 @@ spec: 'until pg_isready -h postgres -p 5432; do echo waiting for database; sleep 2; done;'] - name: run-migrations - image: flyteadmin:v0.3.5 + image: flyteadmin:v0.4.13 imagePullPolicy: IfNotPresent command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "migrate", "run"] @@ -25,7 +25,7 @@ spec: - name: config-volume mountPath: /etc/flyte/config - name: seed-projects - image: flyteadmin:v0.3.5 + image: flyteadmin:v0.4.13 imagePullPolicy: IfNotPresent command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "migrate", "seed-projects", "flytetester", "flytesnacks"] @@ -33,7 +33,7 @@ spec: - name: config-volume mountPath: /etc/flyte/config - name: sync-cluster-resources - image: flyteadmin:v0.3.5 + image: flyteadmin:v0.4.13 imagePullPolicy: IfNotPresent command: ["flyteadmin", "--config", "/etc/flyte/config/*.yaml", "clusterresource", "sync"] volumeMounts: From e17cfcca5505d21c636d4aed0d05a9e7b5920f60 Mon Sep 17 00:00:00 2001 From: Samhita Alla Date: Tue, 20 Apr 2021 20:41:07 +0530 Subject: [PATCH 64/74] Contribution Guide (#922) Signed-off-by: Haytham Abuelfutuh --- CONTRIBUTING.md | 3 + rsts/community/contribute.rst | 255 ++++++++++++++++++++++++++++++++++ rsts/community/docs.rst | 144 ------------------- rsts/index.rst | 2 +- 4 files changed, 259 insertions(+), 145 deletions(-) create mode 100644 CONTRIBUTING.md create mode 100644 rsts/community/contribute.rst delete mode 100644 rsts/community/docs.rst diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000000..f774b84d05 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,3 @@ +# Contributing to Flyte + +For information related to contributing to Flyte, please check out the [Contributing to Flyte](https://docs.flyte.org/en/latest/community/contribute.html) section of the documentation at the Flyte site. diff --git a/rsts/community/contribute.rst b/rsts/community/contribute.rst new file mode 100644 index 0000000000..bf1212adc1 --- /dev/null +++ b/rsts/community/contribute.rst @@ -0,0 +1,255 @@ +###################### +Contributing to Flyte +###################### + +Thank you for taking the time to contribute to Flyte! Here are some guidelines for you to follow, which will make your first and follow-up contributions easier. + +.. note:: + Please read our `Code of Conduct `_ before contributing to Flyte. + +Code +==== +An issue tagged with ``good first issue`` is the best place to start for first-time contributors. Look into them `here `_. + +To take a step ahead, check out the repositories available under `flyteorg `_. + +**Appetizer (for every repo): Fork and clone the concerned repository. Create a new branch on your fork and make the required changes. Create a pull request once your work is ready for review.** + +.. note:: + Note: To open a pull request, follow this `guide `_. + +*A piece of good news -- You can be added as a committer to any ``flyteorg`` repo as you become more involved with the project.* + +Example PR for your reference: `GitHub PR `_. A couple of checks are introduced to help in maintaining the robustness of the project. + +#. To get through DCO, sign off on every commit. (`Reference `_) +#. To improve code coverage, write unit tests to test your code. + +.. note:: + Format your Go code with ``golangci-lint`` followed by ``goimports`` (we used the same in the `Makefile `_), and Python code with ``black`` (use ``make fmt`` command which contains both black and isort). + +Environment Setup +***************** + +.. figure:: https://raw.githubusercontent.com/flyteorg/flyte/static-resources/img/contribution_guide/dependency_graph.png + :alt: Dependency Graph between various flyteorg repos + :align: center + :figclass: align-center + + The dependency graph between various flyteorg repos + +#. `flyte `_ + | Purpose: Deployment, Documentation, and Issues + | Languages: Kustomize & RST +#. `flyteidl `_ + | Purpose: The Flyte Workflow specification in `protocol buffers `_ which forms the core of Flyte + | Language: Protobuf + | Setup: Refer to the `README `_ +#. `flytepropeller `_ + | Purpose: Kubernetes native execution engine for Flyte Workflows and Tasks + | Language: Go + + Setup: + * Check for the Makefile in the root repo + * Run the following commands: + * ``make generate`` + * ``make test_unit`` + * ``make link`` + * To compile, run ``make compile`` +#. `flyteadmin `_ + | Purpose: Control Plane + | Language: Go + + Setup: + * Check for the Makefile in the root repo + * If the service code has to be tested, run it locally: + * ``make compile`` + * ``make server`` + * To seed data locally: + * ``make compile`` + * ``make seed_projects`` + * ``make migrate`` + * To run integration tests locally: + * ``make integration`` + * (or, to run in containerized dockernetes): ``make k8s_integration`` +#. `flytekit `_ + | Purpose: Python SDK & Tools + | Language: Python + | Setup: Refer to the `Flytekit Contribution Guide `_ +#. `flyteconsole `_ + | Purpose: Admin Console + | Language: Typescript + | Setup: Refer to the `README `_ +#. `datacatalog `_ + | Purpose: Manage Input & Output Artifacts + | Language: Go +#. `flyteplugins `_ + | Purpose: Flyte Plugins + | Language: Go + + Setup: + * Check for the Makefile in the root repo + * Run the following commands: + * ``make generate`` + * ``make test_unit`` + * ``make link`` +#. `flytestdlib `_ + | Purpose: Standard Library for Shared Components + | Language: Go +#. `flytesnacks `_ + | Purpose: Examples, Tips, and Tricks to use Flytekit SDKs + | Language: Python (In future, Java shall be added) + + Setup: + * If the Python code has to be tested, run it locally + * If the Python code has to be tested in a cluster: + * Run the ``make start`` command in the root directory of the flytesnacks repo + * Visit https://localhost:30081 to view the Flyte console consisting of the examples present in ``flytesnacks/cookbook/core`` directory + * To fetch the new dependencies and rebuild the image, run ``make register`` +#. `flytectl `_ + | Purpose: A Standalone Flyte CLI + | Language: Go + + Setup: + * Check for the Makefile in the root repo + * Run the following commands: + * ``make generate`` + * ``make test_unit`` + * ``make link`` + +Issues +====== +`GitHub Issues `_ is used for issue tracking. There are a variety of issue types available that you could use while filing an issue. + +* `Plugin Request `_ +* `Bug Report `_ +* `Documentation Bug/Update Request `_ +* `Core Feature Request `_ +* `Flytectl Feature Request `_ +* `Housekeeping `_ +* `UI Feature Request `_ + +If none of the above fits your requirements, file a `blank `_ issue. + +Documentation +============= +Flyte uses Sphinx for documentation and ``godocs`` for Golang. ``godocs`` is quite simple -- comment your code and you are good to go! + +Sphinx spans across multiple repositories under the `flyteorg `_ repository. It uses reStructured Text (rst) files to store the documentation content. For both the API and code-related content, it extracts docstrings from the code files. + +To get started, look into `reStructuredText reference `_. + +Environment Setup +***************** +Install all the requirements from the `docs-requirements.txt` file present in the root of a repository. + +.. code-block:: console + + pip install -r docs-requirements.txt + +From the ``docs`` directory present in the repository root (for ``flytesnacks``, ``docs`` is present in ``flytesnacks/cookbook``), run the command: + +.. code-block:: console + + make html + +.. note:: + For implicit targets, run ``make -C docs html``. + +You can then view the HTML pages in the ``docs/_build`` directory. + +.. note:: + For ``flyte`` repo, there is no ``docs`` directory. Instead, consider the ``rsts`` directory. To generate HTML files, run the following command in the root of the repo. + + .. code-block:: console + + make -C rsts html + +For minor edits that don’t require a local setup, you can edit the GitHub page in the documentation to propose the improvements. + +The edit option is found at the bottom of a page, as shown below. + +.. figure:: https://raw.githubusercontent.com/flyteorg/flyte/static-resources/img/contribution_guide/docs_edit.png + :alt: GitHub edit option for Documentation + :align: center + :figclass: align-center + +Intersphinx +*********** +`Intersphinx `_ can generate automatic links to the documentation of objects in other projects. + +To establish a reference to any other documentation from Flyte or within it, use intersphinx. + +To do so, create an ``intersphinx_mapping`` in the ``conf.py`` file present in the ``docs/source`` directory. + +For example: + +.. code-block:: python + + intersphinx_mapping = { + "python": ("https://docs.python.org/3", None), + "flytekit": ("https://flyte.readthedocs.io/projects/flytekit/en/master/", None), + } + +.. note:: + ``docs/source`` is present in the repository root. Click `here `_ to view the intersphinx configuration. + +The key refers to the name used to refer to the file (while referencing the documentation), and the URL denotes the precise location. + +Here are a couple of examples that you can refer to: + +.. code-block:: text + + Task: :std:doc:`generated/flytekit.task` + +Output: + +Task: :std:doc:`generated/flytekit.task` + +.. code-block:: text + + :std:doc:`Using custom words ` + +Output: + +:std:doc:`Using custom words ` + +| + +Linking to Python elements changes based on what you're linking to. Check out this `section `_ to learn more. + +| + +For instance, linking to the `task` decorator in flytekit uses the ``func`` role. + +.. code-block:: text + + Link to flytekit code :py:func:`flytekit:flytekit.task` + +Output: + +Link to flytekit code :py:func:`flytekit:flytekit.task` + +| + +Here are a couple more examples. + +.. code-block:: text + + :py:mod:`Module ` + :py:class:`Class ` + :py:data:`Data ` + :py:func:`Function ` + :py:meth:`Method ` + +Output: + +:py:mod:`Module ` + +:py:class:`Class ` + +:py:data:`Data ` + +:py:func:`Function ` + +:py:meth:`Method ` \ No newline at end of file diff --git a/rsts/community/docs.rst b/rsts/community/docs.rst deleted file mode 100644 index 2e33df8440..0000000000 --- a/rsts/community/docs.rst +++ /dev/null @@ -1,144 +0,0 @@ -.. _contribute-docs: - -###################### -Contributing to Docs -###################### - -************************ -Docs for various repos -************************ -Flyte is a large project and all the docs span multiple repositories. The core of the documention is in the `flyteorg/flyte `_ repository. -Flyte uses `Sphinx `_ to compile it docs. Docs are automatically pushed on merge to master and docs are hosted using `readthedocs.org `_ - -*************** -Sphinx and RST -*************** - -Style -========= - -Headers --------- -Typically, we try to follow these characters in this order for heading separation. - -.. code-block:: text - - # with overline - * with overline - = - - - ^ - -Intersphinx -============= -`Intersphinx `__ is a plugin that all Flyte repos that build Sphinx documentation use for cross-referencing with each other. There's some good background information on it on these `slides `__. - -Inventory File ----------------- -When Sphinx runs, an inventory file gets created and is available alongside each repo's HTML pages. For example at ``https://readthedocs.org/projects/flytecookbook/objects.inv``. This file is a compressed inventory of all the sections, tags, etc. in the flyte cookbook documentation. This inventory file is what allows the intersphinx plugin to cross-link between projects. - -There is an open-source tool called ``sphobjinv`` that has managed to `reverse engineer these files `__, and offers a CLI to help search for things inside them. - -Setup -------- -Installing ``sphobjinv`` and simple usage :: - - $ pip install sphobjinv - - # Using the CLI to query a hosted inventory file, note the -u switch - $ sphobjinv suggest https://flytekit.readthedocs.io/en/latest/ -u task - - No inventory at provided URL. - Attempting "https://flytekit.readthedocs.io/en/latest/objects.inv" ... - Remote inventory found. - - :py:function:`flytekit.task` - :std:doc:`tasks` - :std:doc:`tasks.extend` - :std:label:`tasks:tasks` - - # Using the CLI to query a local file, useful when iterating locally - $ sphobjinv suggest ~/go/src/github.com/flyteorg/flytekit/docs/build/html/objects.inv task - - :py:function:`flytekit.task` - :std:doc:`tasks` - :std:doc:`tasks.extend` - :std:label:`tasks:tasks` - -.. note:: - - Even though the ``sphobjinv`` CLI returns ``:py:function:...``, when actually creating a link you should just use ``:py:func:...``. See `this `__. Here is a quick list of mappings - - .. list-table:: Conversion table for - ``sphobjinv`` - :widths: 50 50 - :header-rows: 1 - - * - What the tool returns? - - What you should use instead? - * - :py:module: - - :py:mod: - * - :py:function: - - :py:func: - * - :std:label: - - :std:ref: - * - :py:method: - - :py:meth: - - - -Linking Examples ------------------- -In the ``conf.py`` file of each repo, there is an intersphinx mapping argument that looks something like this :: - - intersphinx_mapping = { - "python": ("https://docs.python.org/3", None), - "flytekit": ("https://flyte.readthedocs.io/projects/flytekit/en/master/", None), - ... - } - -This file is what tells the plugin where to look for these inventory files, and what project name to refer to each inventory file as. The project name is important because they're used when actually referencing something from the inventory. - -Here are some examples, first the code and then the link - -.. code-block:: text - - Task: :std:doc:`generated/flytekit.task` - -Task: :std:doc:`generated/flytekit.task` - ------ - -.. code-block:: text - - :std:doc:`Using custom words ` - -:std:doc:`Using custom words ` - -Python -^^^^^^^ -Linking to Python elements changes based on what you're linking to. Check out this `section `__. For instance linking to the ``task`` decorator in flytekit uses the ``func`` role. - -.. code-block:: text - - Link to flytekit code :py:func:`flytekit:flytekit.task` - -Link to flytekit code :py:func:`flytekit:flytekit.task` - -Other elements use different Sphinx roles, here are some examples using Python core docs. :: - - :py:mod:`Module ` - :py:class:`Class ` - :py:data:`Data ` - :py:func:`Function ` - :py:meth:`Method ` - - -:py:mod:`Module ` - -:py:class:`Class ` - -:py:data:`Data ` - -:py:func:`Function ` - -:py:meth:`Method ` diff --git a/rsts/index.rst b/rsts/index.rst index 4fee64cdd9..b4b8bae45f 100644 --- a/rsts/index.rst +++ b/rsts/index.rst @@ -34,7 +34,7 @@ :hidden: Join the Community - community/docs + community/contribute community/roadmap community/compare From 5878595213710a92509f1f6e29c8866f709314f8 Mon Sep 17 00:00:00 2001 From: Jan Kumor Date: Thu, 22 Apr 2021 15:54:53 -0700 Subject: [PATCH 65/74] explicitly grant flyteworkflow finalizer permissions to flytepropeller (#930) * explicitly grant flyteworkflow finalizer permissions to flytepropeller Signed-off-by: akumor * add kustomize output after adding flyteworkflows/finalizers rbac Signed-off-by: akumor Signed-off-by: Haytham Abuelfutuh --- deployment/eks/flyte_generated.yaml | 1 + deployment/gcp/flyte_generated.yaml | 1 + deployment/sandbox/flyte_generated.yaml | 1 + deployment/test/flyte_generated.yaml | 1 + kustomize/base/propeller/rbac.yaml | 1 + 5 files changed, 5 insertions(+) diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml index c2dbb0c94d..4f65a1985e 100644 --- a/deployment/eks/flyte_generated.yaml +++ b/deployment/eks/flyte_generated.yaml @@ -7737,6 +7737,7 @@ rules: - flyte.lyft.com resources: - flyteworkflows + - flyteworkflows/finalizers verbs: - get - list diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml index afe8f3d876..a45962aaa4 100644 --- a/deployment/gcp/flyte_generated.yaml +++ b/deployment/gcp/flyte_generated.yaml @@ -7733,6 +7733,7 @@ rules: - flyte.lyft.com resources: - flyteworkflows + - flyteworkflows/finalizers verbs: - get - list diff --git a/deployment/sandbox/flyte_generated.yaml b/deployment/sandbox/flyte_generated.yaml index 0212b406db..71b2f7422b 100644 --- a/deployment/sandbox/flyte_generated.yaml +++ b/deployment/sandbox/flyte_generated.yaml @@ -1900,6 +1900,7 @@ rules: - flyte.lyft.com resources: - flyteworkflows + - flyteworkflows/finalizers verbs: - get - list diff --git a/deployment/test/flyte_generated.yaml b/deployment/test/flyte_generated.yaml index ca72e1f974..069cb37355 100644 --- a/deployment/test/flyte_generated.yaml +++ b/deployment/test/flyte_generated.yaml @@ -134,6 +134,7 @@ rules: - flyte.lyft.com resources: - flyteworkflows + - flyteworkflows/finalizers verbs: - get - list diff --git a/kustomize/base/propeller/rbac.yaml b/kustomize/base/propeller/rbac.yaml index 524740005e..951a48cc02 100644 --- a/kustomize/base/propeller/rbac.yaml +++ b/kustomize/base/propeller/rbac.yaml @@ -54,6 +54,7 @@ rules: - flyte.lyft.com resources: - flyteworkflows + - flyteworkflows/finalizers verbs: - get - list From 9d171d3b7dcf0005aecb814744768209bfd81ee9 Mon Sep 17 00:00:00 2001 From: Katrina Rogan Date: Thu, 22 Apr 2021 16:07:09 -0700 Subject: [PATCH 66/74] Add dynamic running state to node state machine (#929) Signed-off-by: Haytham Abuelfutuh --- rsts/dive_deep/state_machine.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/rsts/dive_deep/state_machine.rst b/rsts/dive_deep/state_machine.rst index e04872db39..47457f4c15 100644 --- a/rsts/dive_deep/state_machine.rst +++ b/rsts/dive_deep/state_machine.rst @@ -36,7 +36,7 @@ The state machine specification for the illustration can be found `here `_ +The state machine specification for the illustration can be found `here `_ Task States ================ From 7525a3173ccfcfe296a75045f80cdb8f8d58bec2 Mon Sep 17 00:00:00 2001 From: Samhita Alla Date: Wed, 28 Apr 2021 00:02:07 +0530 Subject: [PATCH 67/74] Updated Boilerplate Code (#932) Signed-off-by: Samhita Alla Signed-off-by: Haytham Abuelfutuh --- .github/config.yml | 15 ++++++++ Makefile | 6 +++ boilerplate/flyte/Readme.rst | 8 ++++ boilerplate/flyte/config.yml | 15 ++++++++ boilerplate/flyte/update.sh | 14 +++++++ boilerplate/update.cfg | 1 + boilerplate/update.sh | 75 ++++++++++++++++++++++++++++++++++++ 7 files changed, 134 insertions(+) create mode 100644 .github/config.yml create mode 100644 boilerplate/flyte/Readme.rst create mode 100644 boilerplate/flyte/config.yml create mode 100755 boilerplate/flyte/update.sh create mode 100644 boilerplate/update.cfg create mode 100755 boilerplate/update.sh diff --git a/.github/config.yml b/.github/config.yml new file mode 100644 index 0000000000..f99bcd78f1 --- /dev/null +++ b/.github/config.yml @@ -0,0 +1,15 @@ +# Comment to be posted on PRs from first-time contributors in your repository +newPRWelcomeComment: > + Thank you for opening this pull request! 🙌 + + These tips will help get your PR across the finish line: + - Most of the repos have a PR template; if not, fill it out to the best of your knowledge. + - Sign off your commits (Reference: [DCO Guide](https://github.com/src-d/guide/blob/master/developer-community/fix-DCO.md)). + +# Comment to be posted to on pull requests merged by a first time user +firstPRMergeComment: > + Congrats on merging your first pull request! 🎉 + +# Comment to be posted on first-time issues +newIssueWelcomeComment: > + Thank you for opening your first issue here! 🛠 diff --git a/Makefile b/Makefile index 19cb24ee95..f343018352 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,13 @@ +export REPOSITORY=flyteidl + define PIP_COMPILE pip-compile $(1) --upgrade --verbose endef +.PHONY: update_boilerplate +update_boilerplate: + @boilerplate/update.sh + .PHONY: kustomize kustomize: KUSTOMIZE_VERSION=3.9.2 bash script/generate_kustomize.sh diff --git a/boilerplate/flyte/Readme.rst b/boilerplate/flyte/Readme.rst new file mode 100644 index 0000000000..ea18781185 --- /dev/null +++ b/boilerplate/flyte/Readme.rst @@ -0,0 +1,8 @@ +Config File -- Welcome Bot +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Provides a ``config.yml`` file. + +**To Enable:** + +Add ``flyte/config.yml`` to your ``boilerplate/update.cfg`` file. \ No newline at end of file diff --git a/boilerplate/flyte/config.yml b/boilerplate/flyte/config.yml new file mode 100644 index 0000000000..f99bcd78f1 --- /dev/null +++ b/boilerplate/flyte/config.yml @@ -0,0 +1,15 @@ +# Comment to be posted on PRs from first-time contributors in your repository +newPRWelcomeComment: > + Thank you for opening this pull request! 🙌 + + These tips will help get your PR across the finish line: + - Most of the repos have a PR template; if not, fill it out to the best of your knowledge. + - Sign off your commits (Reference: [DCO Guide](https://github.com/src-d/guide/blob/master/developer-community/fix-DCO.md)). + +# Comment to be posted to on pull requests merged by a first time user +firstPRMergeComment: > + Congrats on merging your first pull request! 🎉 + +# Comment to be posted on first-time issues +newIssueWelcomeComment: > + Thank you for opening your first issue here! 🛠 diff --git a/boilerplate/flyte/update.sh b/boilerplate/flyte/update.sh new file mode 100755 index 0000000000..506c3221b7 --- /dev/null +++ b/boilerplate/flyte/update.sh @@ -0,0 +1,14 @@ +#!/usr/bin/env bash + +# WARNING: THIS FILE IS MANAGED IN THE 'BOILERPLATE' REPO AND COPIED TO OTHER REPOSITORIES. +# ONLY EDIT THIS FILE FROM WITHIN THE 'LYFT/BOILERPLATE' REPOSITORY: +# +# TO OPT OUT OF UPDATES, SEE https://github.com/lyft/boilerplate/blob/master/Readme.rst + +set -e + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" + +# Clone the config.yml file +echo " - copying ${DIR}/config.yml to the root directory." +cp ${DIR}/config.yml ${DIR}/../../.github/config.yml diff --git a/boilerplate/update.cfg b/boilerplate/update.cfg new file mode 100644 index 0000000000..90e8b6daac --- /dev/null +++ b/boilerplate/update.cfg @@ -0,0 +1 @@ +flyte/ diff --git a/boilerplate/update.sh b/boilerplate/update.sh new file mode 100755 index 0000000000..374e4b930e --- /dev/null +++ b/boilerplate/update.sh @@ -0,0 +1,75 @@ +#!/usr/bin/env bash + +# WARNING: THIS FILE IS MANAGED IN THE 'BOILERPLATE' REPO AND COPIED TO OTHER REPOSITORIES. +# ONLY EDIT THIS FILE FROM WITHIN THE 'LYFT/BOILERPLATE' REPOSITORY: +# +# TO OPT OUT OF UPDATES, SEE https://github.com/lyft/boilerplate/blob/master/Readme.rst + +set -e + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" + +OUT="$(mktemp -d)" +trap "rm -fr $OUT" EXIT + +git clone git@github.com:flyteorg/boilerplate.git "${OUT}" + +echo "Updating the update.sh script." +cp "${OUT}/boilerplate/update.sh" "${DIR}/update.sh" +echo "" + + +CONFIG_FILE="${DIR}/update.cfg" +README="https://github.com/flyteorg/boilerplate/blob/master/Readme.rst" + +if [ ! -f "$CONFIG_FILE" ]; then + echo "$CONFIG_FILE not found." + echo "This file is required in order to select which features to include." + echo "See $README for more details." + exit 1 +fi + +if [ -z "$REPOSITORY" ]; then + echo '$REPOSITORY is required to run this script' + echo "See $README for more details." + exit 1 +fi + +while read directory junk; do + # Skip comment lines (which can have leading whitespace) + if [[ "$directory" == '#'* ]]; then + continue + fi + # Skip blank or whitespace-only lines + if [[ "$directory" == "" ]]; then + continue + fi + # Lines like + # valid/path other_junk + # are not acceptable, unless `other_junk` is a comment + if [[ "$junk" != "" ]] && [[ "$junk" != '#'* ]]; then + echo "Invalid config! Only one directory is allowed per line. Found '$junk'" + exit 1 + fi + + dir_path="${OUT}/boilerplate/${directory}" + # Make sure the directory exists + if ! [[ -d "$dir_path" ]]; then + echo "Invalid boilerplate directory: '$directory'" + exit 1 + fi + + echo "***********************************************************************************" + echo "$directory is configured in update.cfg." + echo "-----------------------------------------------------------------------------------" + echo "syncing files from source." + rm -rf "${DIR}/${directory}" + mkdir -p $(dirname "${DIR}/${directory}") + cp -r "$dir_path" "${DIR}/${directory}" + if [ -f "${DIR}/${directory}/update.sh" ]; then + echo "executing ${DIR}/${directory}/update.sh" + "${DIR}/${directory}/update.sh" + fi + echo "***********************************************************************************" + echo "" +done < "$CONFIG_FILE" \ No newline at end of file From 673e300992ab25131af0cad25aa9a28d8d84dc9f Mon Sep 17 00:00:00 2001 From: Yuvraj <10830562+evalsocket@users.noreply.github.com> Date: Thu, 29 Apr 2021 00:34:48 +0530 Subject: [PATCH 68/74] Added goreleaser for publishing menifest (#927) * added goreleaser for publishing menifest Signed-off-by: yuvraj * Rename release file name Signed-off-by: yuvraj * Added script for artifacts Signed-off-by: yuvraj * More changes Signed-off-by: yuvraj * More changes Signed-off-by: yuvraj * fix typo Signed-off-by: yuvraj * minor changes Signed-off-by: yuvraj Signed-off-by: Haytham Abuelfutuh --- .github/workflows/publish-menifest.yml | 40 ++++++++++++++++++++++++++ .gitignore | 1 + .goreleaser.yml | 14 +++++++++ Makefile | 4 +++ script/prepare_artifacts.sh | 11 +++++++ 5 files changed, 70 insertions(+) create mode 100644 .github/workflows/publish-menifest.yml create mode 100644 .goreleaser.yml create mode 100644 script/prepare_artifacts.sh diff --git a/.github/workflows/publish-menifest.yml b/.github/workflows/publish-menifest.yml new file mode 100644 index 0000000000..834e3f51d6 --- /dev/null +++ b/.github/workflows/publish-menifest.yml @@ -0,0 +1,40 @@ +name: Create a flyte release +on: + workflow_dispatch: + inputs: + version: + description: 'version name. example v0.1.1' + required: true + +jobs: + goreleaser: + name: Goreleaser + runs-on: ubuntu-latest + outputs: + version: ${{ steps.bump-version.outputs.tag }} + steps: + - uses: actions/checkout@v2 + with: + fetch-depth: "0" + - name: Bump version and push tag + id: bump-version + uses: anothrNick/github-tag-action@1.17.2 + env: + GITHUB_TOKEN: ${{ secrets.FLYTE_BOT_PAT }} + WITH_V: true + CUSTOM_TAG: ${{ github.event.inputs.version }} + RELEASE_BRANCHES: master + - uses: actions/checkout@v2 + with: + fetch-depth: "0" + ref: ${{ github.event.inputs.version }} + - name: build Release Menifest + run: | + make prepare_artifacts + - name: Run GoReleaser + uses: goreleaser/goreleaser-action@v2 + with: + version: latest + args: release --rm-dist + env: + GITHUB_TOKEN: ${{ secrets.FLYTE_BOT_PAT }} \ No newline at end of file diff --git a/.gitignore b/.gitignore index bdde967a1d..bd8ac5ca2c 100644 --- a/.gitignore +++ b/.gitignore @@ -14,4 +14,5 @@ docs/_sources/ docs/flytekit/flytekit.interfaces.html docs/searchindex.js docs/ +release/ __pycache__/ diff --git a/.goreleaser.yml b/.goreleaser.yml new file mode 100644 index 0000000000..a69fd22b81 --- /dev/null +++ b/.goreleaser.yml @@ -0,0 +1,14 @@ +project_name: flyte +builds: + - skip: true +release: + github: + owner: flyteorg + name: flyte + draft: false + prerelease: auto + discussion_category_name: General + name_template: "{{.ProjectName}}-v{{.Version}}" + disable: false + extra_files: + - glob: ./release/* \ No newline at end of file diff --git a/Makefile b/Makefile index f343018352..f88733e9e5 100644 --- a/Makefile +++ b/Makefile @@ -47,3 +47,7 @@ stats: @generate-dashboard -o deployment/stats/prometheus/flytepropeller-dashboard.json stats/flytepropeller_dashboard.py @generate-dashboard -o deployment/stats/prometheus/flyteadmin-dashboard.json stats/flyteadmin_dashboard.py @generate-dashboard -o deployment/stats/prometheus/flyteuser-dashboard.json stats/flyteuser_dashboard.py + +.PHONY: prepare_artifacts +prepare_artifacts: + bash script/prepare_artifacts.sh \ No newline at end of file diff --git a/script/prepare_artifacts.sh b/script/prepare_artifacts.sh new file mode 100644 index 0000000000..e2d60e7f8d --- /dev/null +++ b/script/prepare_artifacts.sh @@ -0,0 +1,11 @@ +#!/usr/bin/env bash + +set -ex + +for file in ./deployment/**/flyte_generated.yaml; do + if [ -f "$file" ]; then + result=${file/#"./deployment/"} + result=${result/%"/flyte_generated.yaml"} + cp $file "./release/flyte_${result}_manifest.yaml" + fi +done From 49e4fba09eeb1a38e5d5bfb79443efbe53e0ce2a Mon Sep 17 00:00:00 2001 From: Yuvraj <10830562+evalsocket@users.noreply.github.com> Date: Thu, 29 Apr 2021 00:38:45 +0530 Subject: [PATCH 69/74] Added flytectl in sandbox (#952) * Added sandbox changes for new release process Signed-off-by: yuvraj * curl added in sandbox build image Signed-off-by: yuvraj Signed-off-by: Haytham Abuelfutuh --- docker/sandbox/Dockerfile | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/docker/sandbox/Dockerfile b/docker/sandbox/Dockerfile index 0f70df8eaf..3d64453538 100644 --- a/docker/sandbox/Dockerfile +++ b/docker/sandbox/Dockerfile @@ -14,6 +14,9 @@ RUN git clone -b ${BUILDKIT_CLI_FOR_KUBECTL_VERSION} --single-branch --depth 1 h FROM alpine:3.13.2 AS base_ +# Install dependencies +RUN apk add --no-cache curl + # Make directory to store artifacts RUN mkdir -p /flyteorg/bin /flyteorg/share @@ -25,13 +28,16 @@ RUN wget -q -O /flyteorg/bin/k3s https://github.com/k3s-io/k3s/releases/download # Install buildkit-cli-for-kubectl COPY --from=go_builder_ /install/linux/ /flyteorg/bin/ -# Download flyte manifest +# Install flytectl +RUN curl https://raw.githubusercontent.com/flyteorg/flytectl/master/install.sh | sh + +# Copy flyte manifest COPY deployment/sandbox/flyte_generated.yaml /flyteorg/share/flyte_generated.yaml # Copy scripts +RUN cp /bin/flytectl /flyteorg/bin/ COPY docker/sandbox/kubectl docker/sandbox/wait-for-flyte.sh /flyteorg/bin/ - FROM base_ AS default # Install dependencies From 6c25396ea45e8a45aa25aafe2f6441cb2ae6eb86 Mon Sep 17 00:00:00 2001 From: Samhita Alla Date: Thu, 29 Apr 2021 23:58:43 +0530 Subject: [PATCH 70/74] Modify repo name in Makefile (#953) * Updated Boilerplate Code Signed-off-by: Samhita Alla * modified export line Signed-off-by: Samhita Alla Signed-off-by: Haytham Abuelfutuh --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index f88733e9e5..6b38fd26ff 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -export REPOSITORY=flyteidl +export REPOSITORY=flyte define PIP_COMPILE pip-compile $(1) --upgrade --verbose From 870ac8f6bbed38e77138ddd36dc96937c1e6758b Mon Sep 17 00:00:00 2001 From: Niels Bantilan Date: Thu, 29 Apr 2021 20:59:26 -0400 Subject: [PATCH 71/74] Signed-off-by: Samhita Alla aallasamhita@gmail.com (#955) Co-authored-by: Niels Bantilan niels.bantilan@gmail.com Documentation revamp restructuring according to the RFC: https://docs.google.com/document/d/1Yp5cYujKT6UMv17Y6r1djMchaMZWAlPNUTZnbw1sZvc/edit#heading=h.vozt1qaes3ur Signed-off-by: cosmicBboy Co-authored-by: Samhita Alla Signed-off-by: Haytham Abuelfutuh --- doc-requirements.in | 2 +- doc-requirements.txt | 10 +- rsts/_static/custom.css | 91 ------------- rsts/community/compare.rst | 9 -- rsts/{dive_deep => concepts}/admin.rst | 0 .../{dive_deep => concepts}/admin_service.rst | 0 rsts/{dive_deep => concepts}/architecture.rst | 0 rsts/concepts/basics.rst | 18 +++ rsts/{dive_deep => concepts}/catalog.rst | 0 rsts/{dive_deep => concepts}/console.rst | 0 rsts/concepts/control_plane.rst | 13 ++ rsts/concepts/core.rst | 15 ++ .../customizable_resources.rst | 0 rsts/concepts/deployment_options.rst | 7 + rsts/{dive_deep => concepts}/domains.rst | 0 rsts/{dive_deep => concepts}/dynamic_spec.rst | 0 rsts/concepts/execution_time.rst | 14 ++ .../execution_timeline.rst | 0 rsts/{dive_deep => concepts}/executions.rst | 0 rsts/concepts/flyte_cli.rst | 7 + rsts/concepts/flyte_ui.rst | 7 + .../flyte_wf_tasks_high_level.png | Bin rsts/concepts/glossary.rst | 12 ++ .../launchplans_schedules.rst | 0 .../{dive_deep => concepts}/observability.rst | 0 rsts/{dive_deep => concepts}/overview.rst | 0 rsts/{dive_deep => concepts}/projects.rst | 0 rsts/{dive_deep => concepts}/registration.rst | 0 .../{dive_deep => concepts}/state_machine.rst | 0 rsts/{dive_deep => concepts}/tasks.rst | 0 .../workflows_nodes.rst | 0 rsts/conf.py | 4 - rsts/dive_deep/index.rst | 48 ------- rsts/howto/enable_and_use_schedules.rst | 2 +- rsts/howto/execute_workflow.rst | 2 +- rsts/howto/fast_registration.rst | 2 +- rsts/howto/install_sdk.rst | 2 +- rsts/howto/interruptible.rst | 2 +- rsts/howto/new_project.rst | 4 +- rsts/howto/performance/index.rst | 2 +- rsts/howto/resource_manager/index.rst | 2 +- rsts/howto/resource_quota.rst | 2 +- rsts/howto/sandbox.rst | 2 +- rsts/index.rst | 47 ++++--- rsts/plugins/aws/athena.rst | 9 -- rsts/plugins/aws/index.rst | 14 -- rsts/plugins/aws/sagemaker.rst | 9 -- rsts/plugins/extend/flyte_backend.rst | 75 ---------- rsts/plugins/extend/flytekit_python.rst | 10 -- rsts/plugins/extend/intro.rst | 128 ------------------ rsts/plugins/hive.rst | 9 -- rsts/plugins/index.rst | 27 +--- rsts/plugins/kubeflow/index.rst | 13 -- rsts/plugins/kubeflow/pytorch_operator.rst | 9 -- rsts/plugins/kubeflow/tensorflow_operator.rst | 9 -- rsts/plugins/pandera.rst | 9 -- rsts/plugins/papermill.rst | 9 -- rsts/plugins/pod.rst | 9 -- rsts/plugins/spark_k8s.rst | 2 +- rsts/plugins/sqlite3.rst | 9 -- 60 files changed, 142 insertions(+), 534 deletions(-) delete mode 100644 rsts/_static/custom.css delete mode 100644 rsts/community/compare.rst rename rsts/{dive_deep => concepts}/admin.rst (100%) rename rsts/{dive_deep => concepts}/admin_service.rst (100%) rename rsts/{dive_deep => concepts}/architecture.rst (100%) create mode 100644 rsts/concepts/basics.rst rename rsts/{dive_deep => concepts}/catalog.rst (100%) rename rsts/{dive_deep => concepts}/console.rst (100%) create mode 100644 rsts/concepts/control_plane.rst create mode 100644 rsts/concepts/core.rst rename rsts/{dive_deep => concepts}/customizable_resources.rst (100%) create mode 100644 rsts/concepts/deployment_options.rst rename rsts/{dive_deep => concepts}/domains.rst (100%) rename rsts/{dive_deep => concepts}/dynamic_spec.rst (100%) create mode 100644 rsts/concepts/execution_time.rst rename rsts/{dive_deep => concepts}/execution_timeline.rst (100%) rename rsts/{dive_deep => concepts}/executions.rst (100%) create mode 100644 rsts/concepts/flyte_cli.rst create mode 100644 rsts/concepts/flyte_ui.rst rename rsts/{dive_deep => concepts}/flyte_wf_tasks_high_level.png (100%) create mode 100644 rsts/concepts/glossary.rst rename rsts/{dive_deep => concepts}/launchplans_schedules.rst (100%) rename rsts/{dive_deep => concepts}/observability.rst (100%) rename rsts/{dive_deep => concepts}/overview.rst (100%) rename rsts/{dive_deep => concepts}/projects.rst (100%) rename rsts/{dive_deep => concepts}/registration.rst (100%) rename rsts/{dive_deep => concepts}/state_machine.rst (100%) rename rsts/{dive_deep => concepts}/tasks.rst (100%) rename rsts/{dive_deep => concepts}/workflows_nodes.rst (100%) delete mode 100644 rsts/dive_deep/index.rst delete mode 100644 rsts/plugins/aws/athena.rst delete mode 100644 rsts/plugins/aws/index.rst delete mode 100644 rsts/plugins/aws/sagemaker.rst delete mode 100644 rsts/plugins/extend/flyte_backend.rst delete mode 100644 rsts/plugins/extend/flytekit_python.rst delete mode 100644 rsts/plugins/extend/intro.rst delete mode 100644 rsts/plugins/hive.rst delete mode 100644 rsts/plugins/kubeflow/index.rst delete mode 100644 rsts/plugins/kubeflow/pytorch_operator.rst delete mode 100644 rsts/plugins/kubeflow/tensorflow_operator.rst delete mode 100644 rsts/plugins/pandera.rst delete mode 100644 rsts/plugins/papermill.rst delete mode 100644 rsts/plugins/pod.rst delete mode 100644 rsts/plugins/sqlite3.rst diff --git a/doc-requirements.in b/doc-requirements.in index 79fbabb86a..6d288ac2e7 100644 --- a/doc-requirements.in +++ b/doc-requirements.in @@ -1,4 +1,4 @@ -furo +git+git://github.com/flyteorg/furo@main readthedocs-sphinx-search sphinx sphinx-prompt diff --git a/doc-requirements.txt b/doc-requirements.txt index a801a17fd3..cd18b68296 100644 --- a/doc-requirements.txt +++ b/doc-requirements.txt @@ -6,9 +6,9 @@ # alabaster==0.7.12 # via sphinx -astroid==2.5.2 +astroid==2.5.6 # via sphinx-autoapi -babel==2.9.0 +babel==2.9.1 # via sphinx beautifulsoup4==4.9.3 # via @@ -20,7 +20,7 @@ chardet==4.0.0 # via requests docutils==0.16 # via sphinx -furo==2021.3.20b30 +git+git://github.com/flyteorg/furo@main # via -r doc-requirements.in idna==2.10 # via requests @@ -59,7 +59,7 @@ snowballstemmer==2.1.0 # via sphinx soupsieve==2.2.1 # via beautifulsoup4 -sphinx-autoapi==1.7.0 +sphinx-autoapi==1.8.1 # via -r doc-requirements.in sphinx-code-include==1.1.1 # via -r doc-requirements.in @@ -71,7 +71,7 @@ sphinx-prompt==1.4.0 # via -r doc-requirements.in sphinx-tabs==2.1.0 # via -r doc-requirements.in -sphinx==3.5.3 +sphinx==3.5.4 # via # -r doc-requirements.in # furo diff --git a/rsts/_static/custom.css b/rsts/_static/custom.css deleted file mode 100644 index d9851b7d8f..0000000000 --- a/rsts/_static/custom.css +++ /dev/null @@ -1,91 +0,0 @@ -h1, h2, h3, h4, h5, h6 { - font-weight: bold; -} - -.sidebar-logo { - max-width: 30%; -} - - -.sidebar-tree .reference.external:after { - content: none; -} - -.sphx-glr-thumbcontainer { - background-color: transparent; - border: transparent; -} - -.sphx-glr-thumbcontainer:hover { - border: transparent; -} - -div.sphx-glr-download a { - color:white; - background-color: #9d68e4cf; - background-image: none; - border: 1px solid #9d68e4cf; -} - -div.sphx-glr-download a:hover { - background-color: #8b48e2cf; - box-shadow: none; -} - -div.sphx-glr-thumbcontainer a.headerlink { - display: none; -} - -div.sphx-glr-thumbcontainer:hover { - border-color: white; - box-shadow: none; -} - -.sphx-glr-script-out .highlight pre { - background-color: #f8f8f8; -} - -p.sphx-glr-script-out { - padding-top: 0em; -} - -.search__outer::-webkit-scrollbar-track { - border-radius: 0px; -} - -@media (prefers-color-scheme: dark) { - .search__outer { - background-color: #131416 !important; - border: 1px solid #131416 !important; - } - .search__outer__input { - background-color: #1a1c1e !important; - } - .search__result__single { - border-bottom: #303335 !important; - } - .outer_div_page_results:hover { - background-color: black; - } - .search__result__title, .rtd_ui_search_subtitle { - color: #9D68E4 !important; - border-bottom: 1px solid #9D68E4 !important; - } - .search__outer .search__result__title span, .search__outer .search__result__content span { - background-color: #9d68e454; - } - .search__result__subheading, .search__result__content { - color: #ffffffd9 !important; - } - .search__outer::-webkit-scrollbar-track { - background-color: #131416 !important; - } - .rtd__search__credits { - background-color: #1a1c1e !important; - border: 1px solid #1a1c1e !important; - color: #81868d !important; - } - .rtd__search__credits a, .search__error__box { - color: #9ca0a5 !important; - } - } diff --git a/rsts/community/compare.rst b/rsts/community/compare.rst deleted file mode 100644 index b91540c15f..0000000000 --- a/rsts/community/compare.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _community_compare: - -################################### -Compare Flyte to other products -################################### - -.. CAUTION:: - - Coming soon 🛠 diff --git a/rsts/dive_deep/admin.rst b/rsts/concepts/admin.rst similarity index 100% rename from rsts/dive_deep/admin.rst rename to rsts/concepts/admin.rst diff --git a/rsts/dive_deep/admin_service.rst b/rsts/concepts/admin_service.rst similarity index 100% rename from rsts/dive_deep/admin_service.rst rename to rsts/concepts/admin_service.rst diff --git a/rsts/dive_deep/architecture.rst b/rsts/concepts/architecture.rst similarity index 100% rename from rsts/dive_deep/architecture.rst rename to rsts/concepts/architecture.rst diff --git a/rsts/concepts/basics.rst b/rsts/concepts/basics.rst new file mode 100644 index 0000000000..8f970b2a5c --- /dev/null +++ b/rsts/concepts/basics.rst @@ -0,0 +1,18 @@ +.. _basics: + +###### +Basics +###### + +.. NOTE:: + + Coming soon 🛠 + +.. toctree:: + :maxdepth: 1 + :name: Basics + + flyte_ui + flyte_cli + deployment_options + glossary diff --git a/rsts/dive_deep/catalog.rst b/rsts/concepts/catalog.rst similarity index 100% rename from rsts/dive_deep/catalog.rst rename to rsts/concepts/catalog.rst diff --git a/rsts/dive_deep/console.rst b/rsts/concepts/console.rst similarity index 100% rename from rsts/dive_deep/console.rst rename to rsts/concepts/console.rst diff --git a/rsts/concepts/control_plane.rst b/rsts/concepts/control_plane.rst new file mode 100644 index 0000000000..83ebec2d36 --- /dev/null +++ b/rsts/concepts/control_plane.rst @@ -0,0 +1,13 @@ +################ +Control Plane +################ + +.. toctree:: + :maxdepth: 1 + + projects + domains + admin + admin_service + registration + console diff --git a/rsts/concepts/core.rst b/rsts/concepts/core.rst new file mode 100644 index 0000000000..d6eb0fb9c8 --- /dev/null +++ b/rsts/concepts/core.rst @@ -0,0 +1,15 @@ +.. _divedeep: + +############################ +Core Concepts & Architecture +############################ + +.. toctree:: + :maxdepth: 1 + :name: Concepts & Architecture + + overview + tasks + workflows_nodes + launchplans_schedules + architecture diff --git a/rsts/dive_deep/customizable_resources.rst b/rsts/concepts/customizable_resources.rst similarity index 100% rename from rsts/dive_deep/customizable_resources.rst rename to rsts/concepts/customizable_resources.rst diff --git a/rsts/concepts/deployment_options.rst b/rsts/concepts/deployment_options.rst new file mode 100644 index 0000000000..0ae2e5b1d1 --- /dev/null +++ b/rsts/concepts/deployment_options.rst @@ -0,0 +1,7 @@ +################################### +Deployment options (Local & Remote) +################################### + +.. NOTE:: + + Coming soon 🛠 \ No newline at end of file diff --git a/rsts/dive_deep/domains.rst b/rsts/concepts/domains.rst similarity index 100% rename from rsts/dive_deep/domains.rst rename to rsts/concepts/domains.rst diff --git a/rsts/dive_deep/dynamic_spec.rst b/rsts/concepts/dynamic_spec.rst similarity index 100% rename from rsts/dive_deep/dynamic_spec.rst rename to rsts/concepts/dynamic_spec.rst diff --git a/rsts/concepts/execution_time.rst b/rsts/concepts/execution_time.rst new file mode 100644 index 0000000000..c19980680b --- /dev/null +++ b/rsts/concepts/execution_time.rst @@ -0,0 +1,14 @@ +###################### +Execution Time Details +###################### + +.. toctree:: + :maxdepth: 1 + + executions + state_machine + execution_timeline + observability + dynamic_spec + catalog + customizable_resources \ No newline at end of file diff --git a/rsts/dive_deep/execution_timeline.rst b/rsts/concepts/execution_timeline.rst similarity index 100% rename from rsts/dive_deep/execution_timeline.rst rename to rsts/concepts/execution_timeline.rst diff --git a/rsts/dive_deep/executions.rst b/rsts/concepts/executions.rst similarity index 100% rename from rsts/dive_deep/executions.rst rename to rsts/concepts/executions.rst diff --git a/rsts/concepts/flyte_cli.rst b/rsts/concepts/flyte_cli.rst new file mode 100644 index 0000000000..da251e189e --- /dev/null +++ b/rsts/concepts/flyte_cli.rst @@ -0,0 +1,7 @@ +############## +Flyte CLI +############## + +.. NOTE:: + + Coming soon 🛠 diff --git a/rsts/concepts/flyte_ui.rst b/rsts/concepts/flyte_ui.rst new file mode 100644 index 0000000000..e26edf382a --- /dev/null +++ b/rsts/concepts/flyte_ui.rst @@ -0,0 +1,7 @@ +################# +Flyte UI +################# + +.. NOTE:: + + Coming soon 🛠 \ No newline at end of file diff --git a/rsts/dive_deep/flyte_wf_tasks_high_level.png b/rsts/concepts/flyte_wf_tasks_high_level.png similarity index 100% rename from rsts/dive_deep/flyte_wf_tasks_high_level.png rename to rsts/concepts/flyte_wf_tasks_high_level.png diff --git a/rsts/concepts/glossary.rst b/rsts/concepts/glossary.rst new file mode 100644 index 0000000000..0be14da2ad --- /dev/null +++ b/rsts/concepts/glossary.rst @@ -0,0 +1,12 @@ +############ +Glossary +############ + +.. glossary:: + + Memoization + Memoization ensures that a method doesn't run for the same inputs more than once by keeping a record of the results for the given inputs. + +.. NOTE:: + + Coming soon 🛠 \ No newline at end of file diff --git a/rsts/dive_deep/launchplans_schedules.rst b/rsts/concepts/launchplans_schedules.rst similarity index 100% rename from rsts/dive_deep/launchplans_schedules.rst rename to rsts/concepts/launchplans_schedules.rst diff --git a/rsts/dive_deep/observability.rst b/rsts/concepts/observability.rst similarity index 100% rename from rsts/dive_deep/observability.rst rename to rsts/concepts/observability.rst diff --git a/rsts/dive_deep/overview.rst b/rsts/concepts/overview.rst similarity index 100% rename from rsts/dive_deep/overview.rst rename to rsts/concepts/overview.rst diff --git a/rsts/dive_deep/projects.rst b/rsts/concepts/projects.rst similarity index 100% rename from rsts/dive_deep/projects.rst rename to rsts/concepts/projects.rst diff --git a/rsts/dive_deep/registration.rst b/rsts/concepts/registration.rst similarity index 100% rename from rsts/dive_deep/registration.rst rename to rsts/concepts/registration.rst diff --git a/rsts/dive_deep/state_machine.rst b/rsts/concepts/state_machine.rst similarity index 100% rename from rsts/dive_deep/state_machine.rst rename to rsts/concepts/state_machine.rst diff --git a/rsts/dive_deep/tasks.rst b/rsts/concepts/tasks.rst similarity index 100% rename from rsts/dive_deep/tasks.rst rename to rsts/concepts/tasks.rst diff --git a/rsts/dive_deep/workflows_nodes.rst b/rsts/concepts/workflows_nodes.rst similarity index 100% rename from rsts/dive_deep/workflows_nodes.rst rename to rsts/concepts/workflows_nodes.rst diff --git a/rsts/conf.py b/rsts/conf.py index f778cbe4de..feaf81a969 100644 --- a/rsts/conf.py +++ b/rsts/conf.py @@ -111,10 +111,6 @@ pygments_style = "tango" pygments_dark_style = "native" -html_css_files = [ - "custom.css", -] - html_theme_options = { "light_css_variables": { "color-brand-primary": "#4300c9", diff --git a/rsts/dive_deep/index.rst b/rsts/dive_deep/index.rst deleted file mode 100644 index 9efea9094f..0000000000 --- a/rsts/dive_deep/index.rst +++ /dev/null @@ -1,48 +0,0 @@ -.. _divedeep: - -########################################## -Architecture and Concepts -########################################## - -******************************** -Core Concepts & Architecture -******************************** - -.. toctree:: - :maxdepth: 1 - :name: Concepts & Architecture - - overview - tasks - workflows_nodes - launchplans_schedules - architecture - -******************************** -Control Plane Details -******************************** - -.. toctree:: - :maxdepth: 1 - - projects - domains - admin - admin_service - registration - console - -******************************** -Execution Time Details -******************************** - -.. toctree:: - :maxdepth: 1 - - executions - state_machine - execution_timeline - observability - dynamic_spec - catalog - customizable_resources diff --git a/rsts/howto/enable_and_use_schedules.rst b/rsts/howto/enable_and_use_schedules.rst index 6bea7a0b65..9bc719282f 100644 --- a/rsts/howto/enable_and_use_schedules.rst +++ b/rsts/howto/enable_and_use_schedules.rst @@ -165,7 +165,7 @@ Workflow Executor ----------------- Scheduled events which trigger need to be handled by the workflow executor, which subscribes to triggered events from the SQS queue you've configured above. -.. CAUTION:: +.. NOTE:: Failure to configure a workflow executor will result in all your scheduled events piling up silently without ever kicking off workflow executions. diff --git a/rsts/howto/execute_workflow.rst b/rsts/howto/execute_workflow.rst index eee7f9179d..3e4234831a 100644 --- a/rsts/howto/execute_workflow.rst +++ b/rsts/howto/execute_workflow.rst @@ -4,6 +4,6 @@ How do I execute a workflow? #################################### -.. CAUTION:: +.. NOTE:: Coming soon 🛠 diff --git a/rsts/howto/fast_registration.rst b/rsts/howto/fast_registration.rst index cd3ef6e5e9..60b124dba4 100644 --- a/rsts/howto/fast_registration.rst +++ b/rsts/howto/fast_registration.rst @@ -4,7 +4,7 @@ How do I use Fast Registration? ******************************** -.. caution:: Experimental feature (beta) +.. NOTE:: Experimental feature (beta) Are you frustrated by having to wait for an image build in order to test out simple code changes to your Flyte workflows? If you're interested in reducing to your iteration cycle to mere seconds, read on below. diff --git a/rsts/howto/install_sdk.rst b/rsts/howto/install_sdk.rst index 111fcc1413..0d2cb45a18 100644 --- a/rsts/howto/install_sdk.rst +++ b/rsts/howto/install_sdk.rst @@ -21,6 +21,6 @@ All Flytekiplugins are also published to pypi as independent libraries and can b How to install Flytekit Java? ################################# -.. CAUTION:: +.. NOTE:: Coming soon 🛠 diff --git a/rsts/howto/interruptible.rst b/rsts/howto/interruptible.rst index 934e75291a..1abcb5f8ea 100644 --- a/rsts/howto/interruptible.rst +++ b/rsts/howto/interruptible.rst @@ -49,6 +49,6 @@ Most Flyte workloads should be good candidates for spot instances. If your task How to recover from interruptions? =================================== -.. CAUTION:: +.. NOTE:: Coming soon 🛠 diff --git a/rsts/howto/new_project.rst b/rsts/howto/new_project.rst index 78e766c0f3..9ccd0e32b2 100644 --- a/rsts/howto/new_project.rst +++ b/rsts/howto/new_project.rst @@ -7,7 +7,7 @@ How do I create/register a new project? Using flytectl --------------- -.. CAUTION:: +.. NOTE:: Coming soon 🛠 @@ -27,6 +27,6 @@ If you refresh your console you'll see your new project appear! FlyteAdmin API reference ------------------------- -.. CAUTION:: +.. NOTE:: Coming soon 🛠 diff --git a/rsts/howto/performance/index.rst b/rsts/howto/performance/index.rst index 5851f4e059..9c862102e5 100644 --- a/rsts/howto/performance/index.rst +++ b/rsts/howto/performance/index.rst @@ -4,6 +4,6 @@ How do I optimize performance of my Flyte Deployment? ###################################################### -.. CAUTION:: +.. NOTE:: Coming soon 🛠 diff --git a/rsts/howto/resource_manager/index.rst b/rsts/howto/resource_manager/index.rst index ba96a480f4..3ed54ad535 100644 --- a/rsts/howto/resource_manager/index.rst +++ b/rsts/howto/resource_manager/index.rst @@ -5,6 +5,6 @@ How do I enable and configure resource manager? ################################################# -.. CAUTION:: +.. NOTE:: Coming soon 🛠 diff --git a/rsts/howto/resource_quota.rst b/rsts/howto/resource_quota.rst index 79f7d4c926..ac609818c2 100644 --- a/rsts/howto/resource_quota.rst +++ b/rsts/howto/resource_quota.rst @@ -4,6 +4,6 @@ How do I limit resources per project/domain? ############################################### -.. CAUTION:: +.. NOTE:: Coming soon 🛠 diff --git a/rsts/howto/sandbox.rst b/rsts/howto/sandbox.rst index c0e3a1bd09..d8305fca72 100644 --- a/rsts/howto/sandbox.rst +++ b/rsts/howto/sandbox.rst @@ -154,4 +154,4 @@ Deploy Flyte Sandbox environment to a shared kubernetes cluster The goal here is to deploy to an existing Kubernetes cluster - within one namespace only. This would allow multiple Flyte clusters to run within one K8s cluster. -.. caution:: coming soon! +.. NOTE:: coming soon! diff --git a/rsts/index.rst b/rsts/index.rst index b4b8bae45f..bdd1765d67 100644 --- a/rsts/index.rst +++ b/rsts/index.rst @@ -3,41 +3,54 @@ .. toctree:: :maxdepth: 1 :name: mainsections + :titlesonly: :hidden: getting_started - Tutorials - reference/index + User Guide + Tutorials + Concepts + API Reference community/index .. toctree:: - :caption: How-Tos - :maxdepth: 1 - :name: howtotoc - :hidden: - - plugins/index - howto/index - -.. toctree:: - :caption: Deep Dive - :maxdepth: 1 + :caption: Concepts + :maxdepth: -1 :name: divedeeptoc :hidden: - dive_deep/index + concepts/basics + concepts/core + concepts/control_plane + concepts/execution_time .. toctree:: - :caption: Developers - :maxdepth: 1 + :caption: Community + :maxdepth: -1 :name: roadmaptoc :hidden: Join the Community community/contribute community/roadmap - community/compare +.. toctree:: + :caption: API Reference + :maxdepth: -1 + :name: apireference + :hidden: + + References + +.. toctree:: + :caption: How-Tos + :maxdepth: 1 + :name: howtotoc + :hidden: + + plugins/index + howto/index + Meet Flyte ========== diff --git a/rsts/plugins/aws/athena.rst b/rsts/plugins/aws/athena.rst deleted file mode 100644 index 8d5898b674..0000000000 --- a/rsts/plugins/aws/athena.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _plugins-aws-athena: - -############### -AWS Athena -############### - -.. CAUTION:: - - Coming soon 🛠 diff --git a/rsts/plugins/aws/index.rst b/rsts/plugins/aws/index.rst deleted file mode 100644 index d9197ae8de..0000000000 --- a/rsts/plugins/aws/index.rst +++ /dev/null @@ -1,14 +0,0 @@ -.. _plugins-aws: - -############# -AWS Plugins -############# - -.. toctree:: - :maxdepth: 1 - :caption: Available Plugins - :name: pluginsawstoc - - athena - sagemaker - diff --git a/rsts/plugins/aws/sagemaker.rst b/rsts/plugins/aws/sagemaker.rst deleted file mode 100644 index 628f0d288a..0000000000 --- a/rsts/plugins/aws/sagemaker.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _plugins-aws-sagemaker: - -############### -AWS Sagemaker -############### - -.. CAUTION:: - - Coming soon 🛠 diff --git a/rsts/plugins/extend/flyte_backend.rst b/rsts/plugins/extend/flyte_backend.rst deleted file mode 100644 index 8791fcf590..0000000000 --- a/rsts/plugins/extend/flyte_backend.rst +++ /dev/null @@ -1,75 +0,0 @@ -.. _extend-plugin-flyte-backend: - -######################################## -Implement Backend Extensions (advanced) -######################################## - -Now that you have landed here, we can assume that you have exhausted your options of extending and want to extend Flyte in a way that adds new capabilities to the platform. - -Let us try to recap, why we should implement a backend plugin, - -#. We want to add a new capability to the Flyte Platform, for e.g. - * ability to talk to a new service like AWS Sagemaker, Snowflake, Redshift, Athena, BigQuery etc - * ability to orchestrate a set of containers in a new way like Spark, Flink, Distributed training on Kubernetes (usually using a Kubernetes operator) - * use a new container orchestration engine like AWS Batch/ECS, Hashicorp' Nomad - * use a completely new runtime like AWS Lambda, KNative etc -#. For the case of talking to a new service like in 1.a - this can be done using flytekit extensions and usually is the better way to get started. But, once matured most of these extensions are better to be migrated to the backend. For the rest of the cases, it is possible to extend flytekit to achieve these scenarios, but this is less desirable, because of the associated overhead of first launching a container that launches these jobs downstream. -#. You want to retain the capability to updating the plugin implementation and roll out new changes, fixes without affecting the users code, or requiring them to update versions of their plugins etc -#. You want the same plugin to be accessible across multiple language SDK's - build a universal plugin - -Basics -======= -We will try to understand the components of a backend plugin using an example plugin - :ref:`plugins-spark-k8s` A Flyte backend extension consists of 3 parts - -Interface specification ------------------------- -Usually Flyte extensions need information that is not covered by a :std:ref:`Flyte TaskTemplate `. The TaskTemplate consists of a -the interface, task_type identifier, some metadata and other fields. An important field to note here is - :std:ref:`api_field_flyteidl.core.tasktemplate.custom`. The custom field is essentially an unstructured JSON. -This makes it possible to extend a task-template beyond the default supported targets -- :std:ref:`api_field_flyteidl.core.tasktemplate.container` (WIP, sql etc). - -The motivation of the Custom field, is to marshal a JSON structure that specifies information beyond what a regular TaskTemplate can capture. The actual structure of the JSON is known only to the implemented backend-plugin and the SDK components. The core Flyte platform, does not understand of look into the specifics of this structure. -It is highly recommended to use an interface definition lanugage like Protobuf, OpenAPISpec etc to declare specify the structure of the JSON. From here, on we refer to this as the ``Plugin Specification``. - -For Spark we decided to use Protobuf to specify the plugin as can be seen `here `__. Note it is not necessary to have the Plugin structure specified in FlyteIDL, we do it for simplicity, ease of maintenance alongwith the core platform and because of existing tooling to generate code for protobuf. - -Flytekit Plugin implementation --------------------------------- -Now that you have a specification, we have to implement a method to generate this new TaskTemplate, with the special custom field. Also, this is where the UX design comes into play. You want to write the best possible interface in the SDK that users are delighted to use. The end goal is to create the TaskTemplate with the Custom field populated with the actual JSON structure. -We will currently refer to Flytekit - python as an example for extending and implementing the SDK. (For java refer to other docs). -The SDK task should be implemented as an extension of :py:class:`flytekit.extend.PythonTask`, or more commonly :py:class:`flytekit.PythonFunctionTask`. -In the case of Spark, we extend the :py:class:`flytekit.PythonFunctionTask`, as shown `here `__. - -The SparkTask is implemented as a regular flytekit plugin, with one exception, the ``Custom`` field is now actually the ``SparkJob`` protocol buffer. Flytekit base classes when serializing a task, will automatically invoke the `get_custom method `_. - - -FlytePropeller backend Plugin ------------------------------- -The backend plugin is where the actual logic of the execution is implemented. The backend plugin uses Flyte - PluginMachinery inteface to implement a plugin which can be one of the following supported types - -#. A `Kubernetes operator Plugin `_ -#. A Web API plugin - `Async `_ or `Sync `_. -#. Or if none of the above fits then - a `Core Plugin `_ - -Kubernetes operator Plugin -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. CAUTION:: - - Coming soon 🛠 - - -Web API plugin -^^^^^^^^^^^^^^^ - -.. CAUTION:: - - Coming soon 🛠 - - -Catch-all - Core Plugin -^^^^^^^^^^^^^^^^^^^^^^^^ - -.. CAUTION:: - - Coming soon 🛠 - \ No newline at end of file diff --git a/rsts/plugins/extend/flytekit_python.rst b/rsts/plugins/extend/flytekit_python.rst deleted file mode 100644 index 2bd1f1de4f..0000000000 --- a/rsts/plugins/extend/flytekit_python.rst +++ /dev/null @@ -1,10 +0,0 @@ -.. _extend-plugin-flytekit-python: - -################################## -Extend flytekit (python) -################################## - -Extending Flytekit is desirable whether you are writing a backend plugins or a Flytekit only plugin. In this section we will cover the basics of how to extend Flytekit to add new plugins - task-types. -In the next section of :ref:`extend-plugin-flyte-backend`, we will talk about backend plugins, which includes extending flytekit - but specifically for the backend plugin. - -.. caution:: Work in progress. For a simple example of how to write a flytekit plugin refer to :std:ref:`advanced_custom_task_plugin`. But, remember flytekit can be extended beyond this simple method - so feel free to ask us a question in the slack channel. \ No newline at end of file diff --git a/rsts/plugins/extend/intro.rst b/rsts/plugins/extend/intro.rst deleted file mode 100644 index 18b7896393..0000000000 --- a/rsts/plugins/extend/intro.rst +++ /dev/null @@ -1,128 +0,0 @@ -.. _plugins_extend_intro: - -########################### -When & How to Extend Flyte -########################### - -.. caution:: These docs are still work in progress. Please read through and if you have any questions don't shy away from either filing a github issue or ping us in the Slack channel. The community loves plugins and would love to help you in any way. - -The Core of Flyte is a container execution engine, where you can write one or more tasks and string them together to form a data dependency DAG - called a ``workflow``. -If your work involves writing simple python or java tasks that can either perform operations on their own or can call out to external services - then there is **NO NEED to extend FLYTE**. - -But, in that case you can almost do everything using python / java or a container - So why should you even have to extend Flyte? - -================= -But First - Why? -================= - -Case 1: I want to use my special Types - e.g. my own DataFrame format -========================================================================== -Flyte, just like a programming language has a core type-system, but just like most languages, this type system can be extended by allowing users to add ``User defined Data types``. -A User defined data type can be something that Flyte does not really understand, but is extremely useful for a users specific needs. For example it can be a custom user structure or a grouping of images in a specific encoding. - -Flytekit natively supports handling of structured data like User defined structures like DataClasses using JSON as the representation format. An example of this is available in FlyteCookbook - :std:doc:`auto_core_intermediate/custom_objects`. - -For types that are not simply representable as JSON documents, Flytekit allows users to extends Flyte's type system and implement these types in Python. The user has to essentially implement a :py:class:`flytekit.extend.TypeTransformer` class to enable translation of the type from Users type to flyte understood types. As an example, -instead of using :py:class:`pandas.DataFrame` directly, you may want to use `Pandera `_ to perform validation of an input or output dataframe. an example can be found `here `_. - -To extend the type system in flytekit refer to an illustrative example found at - :std:ref:`advanced_custom_types`. - - -Case 2: Add a new Task Type - Flyte capability -=============================================== -So often times you want to interact with a service like, - - - a Database (Postgres, MySQL, etc) - - a DataWarehouse like (Snowflake, BigQuery, Redshift etc) - - a computation platform like (AWS EMR, Databricks etc) - -and you want this to be available like a template for all other users - open source or within your organization. This can be done by creating a task plugin. -A Task-plugin makes it possible for you or other users to use your idea natively within Flyte as this capability was built into the flyte platform. - -Thus for example, if you want users to write code simply using the ``@task`` decorator, but you want to provide a capability of running the function as a spark job or a sagemaker training job - then you can extend Flyte's task system - we will refer to this as the plugin and it could be possible to do the following - -.. code-block:: python - - @task(task_config=MyContainerExecutionTask( - plugin_specific_config_a=..., - plugin_specific_config_b=..., - ... - )) - def foo(...) -> ...: - ... - - -OR provide an interface like this - -.. code-block:: python - - query_task = SnowflakeQuery(query="Select * from x where x.time < {{.inputs.time}}", inputs=(time=datetime), results=pandas.DataFrame) - - @workflow - def my_wf(t: datetime) -> ...: - df = query_task(time=t) - return process(df=df) - - - -=========================================================== -I want to write a Task Plugin or add a new TaskType -=========================================================== - -Interestingly there are 2 options here. You can write a task plugin simply as an extension in flytekit, or you can go deeper and write a Plugin in the Flyte backend itself. - -Flytekit only plugin -====================== -An illustrative example of writing a flytekit plugin can be found at - :std:ref:`advanced_custom_task_plugin`. Flytekit plugins are simple to write and should invariably be -the first place you start at. Here - -**Pros** - -#. Simple to write, just implement in python. Flyte will treat it like a container execution and blindly pass control to the plugin -#. Simple to publish - flytekitplugins can be published as independent libraries and they follow a simple api. -#. Simple to perform testing - just test locally in flytekit - -**Cons** - -#. Limited ways of providing additional visibility in progress, or external links etc -#. Has to be implemented again in every language as these are SDK side plugins only -#. In case of side-effects, potentially of causing resource leaks. For example if the plugins runs a BigQuery Job, it is possible that the plugin may crash after running the Job and Flyte cannot guarantee that the BigQuery job wil be successfully terminated. -#. Potentially expensive - In cases where the plugin just runs a remote job - e.g how Airflow does, then running a new pod for every task execution causes severe strain on k8s and the task itself uses almost no CPUs. Also because of stateful natute, using spot-instances is not trivial. -#. A bug fix to the runtime, needs a new library version of the plugin -#. Not trivial to implement resource controls - e.g. throttling, resource pooling etc - -Backend Plugin -=============== - -Doc on how to writed a backend plugins is coming soon. A backend plugin essentially makes it possible for users to write extensions for FlytePropeller (Flytes scheduling engine). This enables complete control on the visualization and availability of the plugin. - -**Pros** - -#. Service oriented way of deploying new plugins - strong contracts. Maintainers can deploy new versions of the backend plugin, fix bugs, without needing the users to upgrade Libraries etc -#. Drastically cheaper and more efficient to execute. FlytePropeller is written in Golang and uses an event loop model. Each process of FlytePropeller can execute 1000's of tasks concurrently. -#. Flyte will guarantee resource cleanup -#. Flyteconsole plugins (capability coming soon) can be added to customize visualization and progress tracking of the execution -#. Resource controls and backpressure management is available -#. Implement once, use in any SDK or language - -**Cons** - -#. Need to be implemented in golang -#. Needs a FlytePropeller build - *currently* -#. Need to implement contract in some spec language like protobf, openAPI etc -#. Development cycle can be much slower than flytekit only plugins - - -=============================================== -How do I decide which path to take? -=============================================== - -.. image:: https://raw.githubusercontent.com/flyteorg/flyte/static-resources/img/core/extend_flyte_flowchart.png - :alt: Ok you want to add a plugin, but which type? Follow the flowchart and then select the right next steps. - - -Use the conclusion of the flow-chart to refer to the right doc -================================================================ - -- :ref:`extend-plugin-flytekit-python` -- :ref:`extend-plugin-flyte-backend` diff --git a/rsts/plugins/hive.rst b/rsts/plugins/hive.rst deleted file mode 100644 index 55e078c214..0000000000 --- a/rsts/plugins/hive.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _plugins-hive: - -########## -BYOC Hive -########## - -.. CAUTION:: - - Coming soon 🛠 diff --git a/rsts/plugins/index.rst b/rsts/plugins/index.rst index e59f565fb7..d502c7a060 100644 --- a/rsts/plugins/index.rst +++ b/rsts/plugins/index.rst @@ -1,23 +1,5 @@ .. _plugins: -################ -Extending Flyte -################ - -.. _plugins_howto: - -Flyte as platform was designed with extensibility as a core primitive. Flyte is essentially an integration framework and hence extensibility is possible through-out the system. -The following sections will guide you through writing your own extensions - either private or public (contribute back to the community). - -.. toctree:: - :maxdepth: 1 - :name: howtoextendtoc - - extend/intro - extend/flytekit_python - extend/flyte_backend - - ==================== Available Extensions ==================== @@ -27,11 +9,4 @@ The following is a list of maintained plugins for Flyte and guides on how to ins :maxdepth: 1 :name: pluginstoc - spark_k8s - pod - sqlite3 - pandera - papermill - hive - aws/index - kubeflow/index + spark_k8s \ No newline at end of file diff --git a/rsts/plugins/kubeflow/index.rst b/rsts/plugins/kubeflow/index.rst deleted file mode 100644 index c770acda03..0000000000 --- a/rsts/plugins/kubeflow/index.rst +++ /dev/null @@ -1,13 +0,0 @@ -.. _plugins-kubeflow: - -################ -Kubeflow Plugins -################ - -.. toctree:: - :maxdepth: 1 - :caption: Available Plugins - :name: pluginskftoc - - tensorflow_operator - pytorch_operator \ No newline at end of file diff --git a/rsts/plugins/kubeflow/pytorch_operator.rst b/rsts/plugins/kubeflow/pytorch_operator.rst deleted file mode 100644 index 6beb4c1226..0000000000 --- a/rsts/plugins/kubeflow/pytorch_operator.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _plugins-kf-pytorch-operator: - -################# -Pytorch Operator -################# - -.. CAUTION:: - - Coming soon 🛠 diff --git a/rsts/plugins/kubeflow/tensorflow_operator.rst b/rsts/plugins/kubeflow/tensorflow_operator.rst deleted file mode 100644 index a198557e5e..0000000000 --- a/rsts/plugins/kubeflow/tensorflow_operator.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _plugins-kf-tf-operator: - -############### -TF Operator -############### - -.. CAUTION:: - - Coming soon 🛠 diff --git a/rsts/plugins/pandera.rst b/rsts/plugins/pandera.rst deleted file mode 100644 index 004219d100..0000000000 --- a/rsts/plugins/pandera.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _plugins-pandera: - -################################################## -Use Pandera to Enforce Type safety in DataFrames -################################################## - -.. CAUTION:: - - Coming soon 🛠 diff --git a/rsts/plugins/papermill.rst b/rsts/plugins/papermill.rst deleted file mode 100644 index 7edd202a1d..0000000000 --- a/rsts/plugins/papermill.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _plugins-papermill: - -################################## -Use Papermill notebooks as tasks -################################## - -.. CAUTION:: - - Coming soon 🛠 diff --git a/rsts/plugins/pod.rst b/rsts/plugins/pod.rst deleted file mode 100644 index 1aca0fee46..0000000000 --- a/rsts/plugins/pod.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _plugins-pod: - -######## -K8s Pods -######## - -.. CAUTION:: - - Coming soon 🛠 diff --git a/rsts/plugins/spark_k8s.rst b/rsts/plugins/spark_k8s.rst index 0b4aa016fc..772da4e598 100644 --- a/rsts/plugins/spark_k8s.rst +++ b/rsts/plugins/spark_k8s.rst @@ -11,7 +11,7 @@ Flyte has an optional plugin that makes it possible to run `Apache Spark DataFrames -################################################## - -.. CAUTION:: - - Coming soon 🛠 From 7e81123967523fc93a74171d28421b363d7cd874 Mon Sep 17 00:00:00 2001 From: Samhita Alla Date: Fri, 30 Apr 2021 22:12:04 +0530 Subject: [PATCH 72/74] added tagline (#956) Signed-off-by: Samhita Alla Signed-off-by: Haytham Abuelfutuh --- rsts/index.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/rsts/index.rst b/rsts/index.rst index bdd1765d67..b3b0f6af9e 100644 --- a/rsts/index.rst +++ b/rsts/index.rst @@ -54,6 +54,11 @@ Meet Flyte ========== +.. raw:: html + +

The workflow automation platform for complex, mission-critical data and ML processes at scale

+ + Flyte is an open-source, container-native, structured programming and distributed processing platform. It enables highly concurrent, scalable and maintainable workflows for machine learning and data processing. Created at `Lyft `__ in collaboration with Spotify, Freenome and many others, Flyte provides first class support for Python, Java, and Scala, and is built directly on Kubernetes for all the benefits containerization provides: portability, scalability, and reliability. From b4a5199c1226ecb331e3b425b492876a3abf5fc6 Mon Sep 17 00:00:00 2001 From: Tom Forbes Date: Fri, 30 Apr 2021 17:42:43 +0100 Subject: [PATCH 73/74] Fix broken Kustomize link in docs (#944) Signed-off-by: Tom Forbes Signed-off-by: Haytham Abuelfutuh --- kustomize/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kustomize/README.md b/kustomize/README.md index 4189f9a146..94f0f6ec56 100644 --- a/kustomize/README.md +++ b/kustomize/README.md @@ -1,5 +1,5 @@ # Install Flyte using Kustomize -Flyte can be deployed to a kubernetes cluster using a generated deployment yaml file. This file is generated using [Kustomize](https://kubectl.docs.kubernetes.io/pages/reference/kustomize.html). +Flyte can be deployed to a kubernetes cluster using a generated deployment yaml file. This file is generated using [Kustomize](https://kubectl.docs.kubernetes.io/guides/introduction/kustomize/). Please refer to Kustomize documentation to see how it works. In brief, Kustomize allows composing a deployment yaml using multiple components. In Flyte all the core components are listed under [Base Components](./base). The Base components also consist of From d451a01ec4ed6f43487f973c7df8fb31e9efb4ef Mon Sep 17 00:00:00 2001 From: Niels Bantilan Date: Fri, 30 Apr 2021 16:50:25 -0400 Subject: [PATCH 74/74] update getting started guide (#958) since we're consolidating our docs, we want to use the getting started example originally from flytesnacks. Here we're moving it into the flyte repo since the flytesnacks example should be the canonical getting started guide Signed-off-by: cosmicBboy Signed-off-by: Haytham Abuelfutuh --- rsts/getting_started.rst | 99 ++++++++++++++++++++++++++-------------- 1 file changed, 64 insertions(+), 35 deletions(-) diff --git a/rsts/getting_started.rst b/rsts/getting_started.rst index 09b8e7c545..f21ddc9d7c 100644 --- a/rsts/getting_started.rst +++ b/rsts/getting_started.rst @@ -3,72 +3,101 @@ Getting started --------------- -.. rubric:: Estimated time: 3 minutes +.. rubric:: Estimated time to complete: 3 minutes. Prerequisites -############# +*************** Make sure you have `docker installed `__ and `git `__ installed, then install flytekit: +Steps +***** + +1. First install the python Flytekit SDK and clone the ``flytesnacks`` repo: + .. prompt:: bash - pip install flytekit + pip install --pre flytekit + git clone git@github.com:flyteorg/flytesnacks.git flytesnacks + cd flytesnacks -Clone the `flytekit-python-template `__ repo to create our own git repository called ``flyteexamples``: +2. The repo comes with some useful Make targets to make your experimentation workflow easier. Run ``make help`` to get the supported commands. + Let's start a sandbox cluster: .. prompt:: bash - git clone git@github.com:flyteorg/flytekit-python-template.git flyteexamples - cd flyteexamples + make start +3. Take a minute to explore Flyte Console through the provided URL. -Write Your First Flyte Workflow -############################### +.. image:: https://github.com/flyteorg/flyte/raw/static-resources/img/first-run-console-2.gif + :alt: A quick visual tour for launching your first Workflow. +4. Open ``hello_world.py`` in your favorite editor. -Let's take a look at the example workflow in `myapp/workflows/example.py `__: +.. code-block:: -.. rli:: https://raw.githubusercontent.com/flyteorg/flytekit-python-template/main/myapp/workflows/example.py - :language: python + cookbook/core/basic/hello_world.py -As you can see, a Flyte :std:doc:`task ` is the most basic unit of work in Flyte, -and you can compose multiple tasks into a :std:doc:`workflow `. Try running and -modifying the ``example.py`` script locally. +5. Add ``name: str`` as an argument to both ``my_wf`` and ``say_hello`` functions. Then update the body of ``say_hello`` to consume that argument. -Start a Local Flyte Backend -########################### +.. tip:: -Once you're happy with the ``example.py`` script, run the following command in your terminal: + .. code-block:: python -.. prompt:: bash + @task + def say_hello(name: str) -> str: + return f"hello world, {name}" + +.. tip:: + + .. code-block:: python + + @workflow + def my_wf(name: str) -> str: + res = say_hello(name=name) + return res + +6. Update the simple test at the bottom of the file to pass in a name. E.g. - docker run --rm --privileged -p 30081:30081 -p 30082:30082 -p 30084:30084 ghcr.io/flyteorg/flyte-sandbox +.. tip:: -When you see the message ``Flyte is ready!``, your local sandbox should be ready on http://localhost:30081/console. + .. code-block:: python -Register Your Workflows -########################### + print(f"Running my_wf(name='adam') {my_wf(name='adam')}") -Now we're ready to ship your code to the Flyte backend by running the following command: +7. When you run this file locally, it should output ``hello world, adam``. Run this command in your terminal: .. prompt:: bash - FLYTE_AWS_ENDPOINT=http://localhost:30084/ FLYTE_AWS_ACCESS_KEY_ID=minio FLYTE_AWS_SECRET_ACCESS_KEY=miniostorage make fast_register + python cookbook/core/basic/hello_world.py + +*Congratulations!* You have just run your first workflow. Now, let's run it on the sandbox cluster deployed earlier. + +8. Run: + +.. prompt:: bash + + REGISTRY=ghcr.io/flyteorg make fast_register + +9. Visit `the console `__, click launch, and enter your name as the input. -Run Your Workflows -################## +10. Give it a minute and once it's done, check out "Inputs/Outputs" on the top right corner to see your updated greeting. -To run a workflow, go to http://localhost:30081/console/projects/flyteexamples/workflows and then follow these steps: +.. image:: https://raw.githubusercontent.com/flyteorg/flyte/static-resources/img/flytesnacks/tutorial/exercise.gif + :alt: A quick visual tour for launching a workflow and checking the outputs when they're done. -1. Select the ``hello_world`` workflow -2. Click the **Launch Workflow** button in the upper right corner -3. Update the ``name`` input argument -4. Proceed to **Launch** to trigger an execution +.. admonition:: Recap -.. rubric:: 🎉 Congratulations, you just ran your first Flyte workflow 🎉 + You have successfully: + 1. Run a flyte sandbox cluster, + 2. Run a flyte workflow locally, + 3. Run a flyte workflow on a cluster. -Next Steps: Tutorials -##################### + .. rubric:: 🎉 Congratulations, you just ran your first Flyte workflow 🎉 -To experience the full capabilities of Flyte, try out the `Flytekit Tutorials `__ 🛫 + Next Steps: User Guide + ####################### + + To experience the full capabilities of Flyte, take a look at the `User Guide `__ 🛫

3n7q1E7jshN918}Dg_kKa( zyR0S|SPMG0$a|3Yq6p;LaDRLxy!MM9-_MXH$K2Ay&BXpsx>OW`sE<7^M1S(p^ALEM z0Gy_4;l4$Z$=a6tj3QQsc?T!Il3H8Wl2Kmqi%Iu6Wvr@oiQBKvN!BS2Sr~D4B z#$EIT@piX=AL34VJdorIkvofZ%$t4xn|Z&PWc{?aUIxKded>Du>DLiYQf|9v?Bl3N5g5rE}RS=+XVjNAhV`pgGHn#c zJ>_5dC&-N)R-~K^d40y!N4wsg)gUDQ%qM?P;6(TlB7)lPR59S=f$%%`uGN?MBy0az zemmf?dL z&Gy)J*O!&=z%^!;32&1cfPt?U+(>6FzUq;=Dd-QYpY7-K zVlOP58hnhNa`r@cuassJz*-CE#$ds&RY(30Y*wtlRVbh2JCPngi>kymKX0f$bf4oq z5I+4{`koc_n*?myL3G;3<-+O}BfedwK&?M`2pyh_RLhzAS4_jjZlukp^zQnR>`?q_P~px!11xbU>T* zTc3Zsh1@8MY6le?#Q4od5xrFfF$Ku$S06?WG06V!_*}qx64LQLn7CYkDVmBB(<#fg zSewDV?VVo4#~JJVBoJchtGVzac?TC-&dE+eO3=0AM#Fi<_V>KF95iL1@IAVb@s8^ z>dR(t&A~qkqmv!_Q7W@=Vv~CuxeTlw&$foNCy4k*HZkT0=YD^r!O$C$jOnPMi>kU7 zY4AGdRhzz%v0MzZgBATHE;Yt}R9ON!c$0u>-w;`IFu^-Y-?slt=`zu5+#U8XDTO24 z3wE!UOR;z!lXnCa(lGvA$ZqoIp8TJV;T1`v3$4!W7kd#mp?#^=>SD3~mR19#IC9C0 zM=Ejf+ZbEV*9mxIWEP8P5rjFZPm`VH`@iHGpd>1xdPDfahGbo8ELUpZjs&6f%Y!qe zs09R0PU>Om+ZbI*V6mN;7W6qGgdFZ~+qvx3uJ_xZ_R$vuB4+W!(Phk&9T7iH{{0c&wsS+Mu^{pga< zA}>j;J_MY!*Sy`;cN&Z=g1IO18nj;enHRMsV+waqd5Rg^Dc>}cOw*3-andtBc6+%5Lw!JPbL5g%3%{ix!M8#BS8qwz_R!O z?q@}Ay+|wzhvZ6b;C1UdT?CrWeQr5jUP;!@s}zg&Yp>fwm_)n}l&gxWE%*c~dhaa^ ztD2p(YObU3M@XRjl~QD2c})nnTHNTx8!kLJiR&@FZFFZr?B2JTTqeys@R*xi*++Xv znQTYQzh?!u|2!`r*|XgcF^zcJTZbVJ?YsFdSP<69?y87d!aQQ;YWR7dXs@rI?b1b& zL;-HUS;Itoqx0>PLkOBJ^h%-#WO6Bo%x;S>o*^PzWE!PZeM%70TXy6@XC~_PF^N^b z8)vkijuU@0pr8rL$G1xHy};i0bW2lEtgVo6yts5IW8*uQ{IdV8>a1*DX8Z)PS%Sw} zYcI*MDJUP{FvzOQnYA#UE%;DA7TfqIU+Ro>$1}{!x_6u%&-6ap=jBhr7g{d!VkIe3 zUt^YL&PQzU2Ex7wQGwxnL}j5j30~{GR@KDXYx`=}y%4(*@T)aS#+&Xjvqz;{^u$Hl zYSHLN0V&T9)=QpNmQc$-4NG00=QA^X#7oVpYltzHsjWEi<#9~w0X=bS0H*mXh-Szl z;_bkU?{W^JJLi+F`o!ms6`t=)8zUd=LmVXBB$%hJhLa8&8S1TQDk=f$J-^A`F(NX! zZTa%w0&uIY_a&0-LuuPq7Q$c0NhN>q$}AWro%-d2eqcO)QeMS;J*GhyFfn#K z-hWT+48FL~NTdNZ<6DF*u(Rh+5Ss&w{tSf-CO=2P_=+udw8prxotP0FSiA|?ELo%^v6O+umJzu z$piADl&cTk`RlMgfx*K$rwTZmGpxIQ>B=ugFFS-$Z_QTf?6;CDoS9+C9eMF*c|x#{xO#AOF$vueQ4Epojr5Vqc#Bq!SO*r~9E06kX$T>T^)2#A~1)s=HlU5hS5YhITDt~0ATZB>U*kE0CO=itoA*B({Z-E6OrR4 zjn9aBCSPn2LB;;!W@9H=m#ZhW#3515C+X@Ay+5ryK&Ig6B$&blSA{GRBlXw@;Oa$E z3r__?jSSH|-fi4z>WB;he05qJ>CLC9#JjC&FH0bsja;Kt1))^EchGDdED;i|Ilqu@ z0bMZ^3z4hb{opMj;c&zmO?#kOwQEpz&pCq1Nud8LJF3_@FgfN?y}{mo1NHTPnL(6nT@n33lat){nAJXR+-kMO zUC4Wqckh%8vHtIs+XxXqK=RxcP`#f-E?9rPjP6@*oH2Mhj?It{y+1IlgqjSl+6sGW z7=25Otjr@qHrX5w zNi?j?GETCRRko~SSIP_-Imcdcj+JBo@6YM^{{GMNeZF2V@p^jXbARsZzOMK6zTVe$ zcU~N!W-H-*q02A%V6v=!70bM_fZFspU?>;<7Jq9%)GFSoQQ}!T&o}2NCV_$at$o9q zamg*U7W;;VUFUP4SWO8f@8yj0pmnTNO$+$c8X{zN-#QCabT zweJ^HHy(B<*p8_hX?ce_j&fS$9IJ^n;m_f zeZeuTRXw2pE9KhH?Ym0^mj-vG0d7G0lS~>Uz`9F(@LAtzCaKGXUEue-)|ZE+bTr$6 z%9sS+qhhfl-VP(0i>AUnYv#tOrHkE6@m9t(y&;ze+}JC@?P6P19E%EX*n&bViS^;P znYMxsX0(A0m5nRZrP1!j!yTQXhPCEUKH99khQr|x6~i+~Bh(qBQQJ)00P$5ffl!r2 zg|G7grC90!k3jy~=4sQ28r^*fW9Z-K=Cqq76<`!k7BX^#S?E1x1 zeIJ9ZW4b2{wW_K8^40>J`s~Zi!oCl%)58MDPy6=MdvEwIsw_K-?~kX3oKr1XKptOg zx3nT39r_mU?DTl+cJO%l81Lu+^~+>nlRb}9RS@q7uU;I{JJ6+>i+8S8cP?xVs#edF z6CM)2T5k5psL064-tW4E%R`?Px9c@S>S_i|?N${aSKzAsG5%tl}dhV?60!y;`i^5s@PcRHs-!4ueL%ndY>sSWU}e3&I4?pY0Xha z-17!1>yT}<%~GM&`}{Flq31i6&Q4Of$z#0r?@n5?ZHCPxtnOPU@*Xvi8uAuQ(te)L zm+5|K>-PAUxq|4C`M7&GKeu<=f+Fq8vM|rkmjD%u)p%#O$IrEir+fK{h@AGIT1-kv z1#P(t2!IaU-i6Ex#bwp4-LOyETs7{uISr^<`@vkz8?Qh4L+SYNSgD84j(Y%%ZTwNMU#H<@c#vwha(?|z#coXp(b zmVJvnnfOaYBU!bvE}9e!n4Pxuo+28Dk~{ zyi5p~=pfJ8bwlU=D!x z+z*AB?1kVn53S=GrMS2_uP_|pdTYj~PR0+fn<~M=1;}gAAczFHVgdrTGN1Ihd|Gku zik2P6r#2tbYy}})>m@eNVQ+>tscW^-9c?ZMr_@n(ewds>9uk{mJ`ekJc*cfnX7lc3 z%PCNBKXJ#`T@ij|$DgRr*JJ88KV2k(&IHLtT(P4mS%ZEUamMB-dmVaooc&n0wMyBh z<>lW60Fn!)dB-;*=D9iOsA%;cMJr;F^&%ZiPmh|8l_bS>PF=5g9y$9$mp-KMY`Jhc z?+~A_w!qRHt#gZp?#x-EFK?OM40LLmCSo#kSu)=x8R?v9XJH4_iuH9F@rQHh26BnG z0||l3?VUqDf6FG>)sgsIoyU1yzgq`p>X}^DN9uWwT9pHWi(mE5^=VpnqLns~@#aZ% z&*FZq*T$|I=)__l&!N{IJ`!QSyWOF9Cq$X>_Pf@V|LMRAvt?Us|I&uB$B&T09pPjocmjPQm;eMjRrt$Ws2$XlEvL+WRLxZeqJ zkhuA|+kIy)$ou;^eXy5=sN9#y(C)a^$9V$&txroNh_3D^&(Bi5%sCD2sq7B6NobX^ z2y$h)mN`m=>M^HHk)jv@ONC($wEZQ}c5Cwcdr3Gdvc|DAVOu4^rI1(g&Mrt{Mp$+ep1v z41ETdCEr8`ROGWldF#g~(=)mz6|1Ll#w_yYPrf@XioT!hIc7K?vdFDiKoQQ_5V6DKVt< z3n9l3UCHxz^S)L_`TUnl92Q!gdPo!9?KScMjCH{4bVfx_i2$tfp*Rt|IV6>C&9}jQ z|C)TMdNWL3op^!@6w|N66xUAmUqX{weF*-Xy7g<;|H zl=u!Lb_ld0W<2=4OCDKLESoRgY*q*(-wti~JwuLSWdzY>dCv2!rRpDb!x)QBY+G$# z6MF8viy%4AdID6Ltwvd^72sF2Xk7m3 z6=Ku>D(ddyfl8Z}6xZVgm0ht0#jWP4s=F#t#N3Aziu4x_*S^20_Czm$UWxB78>V-- zYF5%=TM;8#JCE~p5cJC$G=Z}r_|Eua3aoDM=}CM>a;#bq{p{yu7o2S`Yw99PI&- zu+JS@xI3tVOm0GnBoLFtrqaS$C%cKzRkWQcHpx>dCxjG;Jd>4hA3aBo^J$WPQwJQ@*^@8dib%KOC)08qdwbCCb7#Wij7jGGXt z*cs{A$6m5qK6!?jyPkFV8%t8H_3TBf`zT*SVb6|q=I8rYmpkc_)S~84bZ?%`#K6qZm8B=1nrh!!bV+Zs}f7Lg}4SO62O=4j&YML$f_NwnEs^0x_?PQ(S^5vq57 z{L~BJ7G(*c5^U4ID%=^PriBMZo3gOdfHZPdS`mL0<4j z0m#i1DWi969@X5O-X()MI*6vt4NL^xWWl(Ld|)Z&wDqKs;9GJaGCZIGB*ZE$Zpa6- z3FT*PtI#`56tmn~^DX>N*+c;aqCER}ReEhmIJ3X#x^;FAGsYuJ zU>CQeX>p6boz{y%-I{ze&jJXt`%6ecDFJ}kA|Ui2K6q~6@D<%!c8v;G7 zL8w%)LG*C2@l29+l^V1!ll8l^5zW5>VQ+hPz_zGm-xBQkWOMy-5m0Zrq7bNfGP+ir zlYZ;hpJlBFcIq~t`wiaXJ~5`%Be=eftFX+kAt^R@dLwoz99!Px%-F1dkg)lkp!yy^ z>mn&X&l-L6NUA|*J+-d}yYn;4BJ>-E)bb&olVRFFmj0G8`$K$-mAL%;oux_oW4h*} zHi_+h(Z}AqV)RHd923Q8(5*#06;x?si3UUdumz;BLMk*MPDoJxCdJJD5+a(1HAA#& ztnyc${vgqEDe?D&RWj6weFqU(l?d-`A6?d!^U{96C5r8ef+NYv+K@sf?@sm%E9h31 z&k>NNyQ;dcd60JAK3m|)0vyRCdgO3%qR(8%U{yau!L*mk?*3gr5z-t3VARX$P z8+PpqHbXKM+L)x+)F|#ye9gz7`Fm99rO~N1j88==n!vA}fwXAe&?qOmcv56RKd>Kf zX?evkZm2|BRjXqDY~Li+#%d|EkHJ9e9wxY;iIuLv+;au4B@5yTh=1GUnBJk8`8smI zh;==ycbTwOFk<+{_ml*vnW}&TDgulyRM}!@3_Fe_lhQPk+%=RcMcTHFZ$JEAH)*We z)Ey)}!HB!1sdNE;`*An=z3nediaQ(u$S_F3H1wx!;Cm#uc39c#{`FHBz>op@ztB`# z@0YQF-lCaU&o8&K<8N`?M{fV$6uAc3ri(&cO|fCQkk{Z(+FjB|>aiJBd&7L8bB?;x-WAYSKZDLh{FQ`F}~gO%;Qh{+!%pSau==4&6rYhl{X9ESEBj17L{dYk3QXb$Wv1tfdx zqlN2p;+J4|Bv-u)2eiy2ooZP(9a8RwL|*pDF7lgEILAlnw-o1JN)@1pCGVsWYD@Oe zkyP0Wk!kSICvcg71OntIv2p<^as{;z0L2B6lWPI>J``|<3wWzbFpZWBaU){_I}Jj# zhz3mmEhJXxKTC&vkIz67*E)E1V|m9y&KKUE_g;TK%lqu5}BShh9Eyn1uh7N0uFGzO!$@p4A;WZXGdd zN0X-TjVe+vm;??2RS8j(NW1?2#6#Ac;xt_Lsb2SH@e)TW1>PY8t)?A zvl0=Z7Si-5-2Ju512?<_H|OZY87&ienk1m2u@@L3NSbSFB?fsiZ15d;8O2>OYEbJR1f=yVxR^XiO7VV~sZ;A21h9wdCTyzCLK^Zrio_u}o2LeK@uPUpnZj_Sw^RVr+T?}(VBr+?IvZ8}%FBJ?zx<@e9lTzwJG z4vW0nWlk;AP?u=X$=-yf%w*XGRYd|b2-XLP65+72*_NHxQ0LfBo=|<$XjnvI&Ze)?Fbc5m~$vWvARG(=f!h_jZXlJpt zxZ$Z-4UIbwM19(<=SCWg#~Mk-$&oA#p1Pc) z2CYd>sR_=JRoU!R{1CuQ2ksmpJ7xN=^Whq}qpb)Vhd4TSPtNlbpOf zl4y&lYdFqdV#FCt--*LK6x!N~N-?{Bh zxB8j z(Pa^U1raiAhXIl7JOEpH6y4?QIc?{ABU>45_ZfqY z^0*Kpe2WR7!2U>|xxW!Xb4swng}+ONr`P8A11`5c`WBPz_3cyN=C6Fp2}P}fMW~V6 zS7E-E!I@UH{w>7sp!BGUWP%Zj^FY^vZ=ANSzJ!!vX1}76hOQLLFLi?>bB{ z8ILxx*F|VBTKwK$4I4nfEz3U_k9YZ;8MlxFDBCRto}B_fHV=5Ts3n}SUwqXPMt3rYIAMCqfW0^-`Ok+9gBnNafffbTVG znnm$OKdKsufti%uNn*(9S+F#gyf78yqg<2^Ns&}oV;>CGEW0D5sIu1E>tfayk&b!tmW!U%A#w{x z0aWf;Ih(YTB0@eBJWUKEu_JGOTR6Qf`C%xGmGsqE*h5y81EJ!b48vs~5 zz!U^N+!0Tf9{`FY2TR$}$0H(~Z0AUJqZe^=3bAqQLw<_YN8}f%OWMgKM$bd=2vnJB zaapniq}&~Gte8W3EXM@JKLb)MBpWaSNFNKzq_SbhcXV#xn)EVo_i6Ot#Yf}+#t0d` z5+8lsb4aHF;05AQW`KR6%f~=rSeXc1()NSgBD=y-Lub{h5U8xQ-{Y^xoVilX{qvfl z)vYOpX<$cWv;s$+OF^O{e$-WNbn(A_IOzJHnD{X9)J4FDKCJ9JG)<7ZkQ!&D8hQ|Q z;lRPlimG<4(>!zdgKW(~UHoIe* z%wNAxG3(Q#BS>N7c(R1sETNM|N?dy4ZJic=0+(qVX5~mUv{t;_9ube|Uz+;(8BPHN zCjfbxg6=YYh}^&vp&tq%%?&m`kkyD6COw`w-R_vb?N@mD^3*VsZ|!pCgNz+DA@SfG zYSV^7T7nmxg_<>hLj`YLlYLRfWW$8ap+1@?0tMO<#DKW{#Ae21}>{@U+ojx?G~xor{5wm=ZPbT=aWS*1Os; zRh=nF={2g{(Z%k$IV}(Q&xK|i;8+3{T1?3x&~*shB^wxS0B}qhC1G(Gh=~M?8xO#Z zlYl0XzXR`E@L2PJGEIoZaBK132nBZFku{kj$eFI2~?OwMhra}>K$jXT~G@^!7=tHRe zljIHlWS~*+ADIT8ql{5LgBC81>RmU?ZX?{pev6VbX5p`63?xGU2EHb0qH0H1Sw<QpzQyY))U6}nuyGB{k z?i^~zFTSJ~ViQi$Lum#;N8U{G&X6xbN2qnNhw0NJ|DHBJ?FG2J7heeKir`g@-y-Px zK;s$I=K-}|0_#;sUG`i64&-g*1F`>T0qzvHh4hmbOHgfi-s#$8>#^)=C^VnAgnrG1 zQQ$eszYhF=e%H?;Q9x)5z3mnwT`GbN+4ThET>j+8hlqCrjksh4VupBHK<=oN)w!c6 zDi6$~$((2Oo3-oq$!5rzENkyL7VWYp{Xwq+M5i?EI}{kuviWBvi3C3ab`gvRID>K& z^f@{G4{W)vz{!Rw@&a!Q$hhdp2`9Tmvr8bX*9%ia3>0HkHwTtEZq(S!WMZmrb)(8V zC&EB~jF>n)nFzMl-kVw6RrbPFkkYmfj^BtR?w%V5u5h-+pVJn2AK+WT{SDw6Ho#@x z_h}%(i}R=)ouhKl7i{}j{+spk#LII}2ZG;B=yVy3Y?iZ7+iy; z+^G2~(z2-_%zu{a8c%4&$5T6rCvII>P&0{)P)8Z}I zWOVwxnPhr@B({VC$VS{yijepcP!JD6LtE&yWZ?_oJbs5zt?3u5wuoC<^2YbZ3SRdW zoFVVo*J2e%RB)ueCBi=i2;{_nTpvqlG%>^@gFYp0;-gwE=l@=gSko0;9VcEipzH*W z^m4li1JtmwergihT8t9tCjsyj`a6H|p{ZE@+in)GHGZCQy5;&FQ)I)ke$SfBJ>y-Q z*yi(_0ljSZL`V+~%*ZFM81B`tjkBwone6VaO)z@GEMK0$72r?(K?PXSh6N#?0g{qF zCoZm)sb3^H3~k9aXU z4ED|&5QTkeUFx{pdQ@w|`QLABS{=S)_7-qr;%yQLL#lLx=7OzS-=j}XOXN7>n53D zPC4(q?jB#VlnEonf#(PG57hB02x)e}Nf$o}P1vBubt&EaFOLv+>XBL$q`(bDG0KY* zAtC4m*@S2qwt^HIC_;&Xi4OJf%TIi~`fUGA%pKD)p7l;5k_W8Q7UDnKbrZ-Z+WQs- zg?uuk2g3plw`;xY_yw&ddsM5VN_B`}pgEct(*M-{CAp{v*&cC|!&@flw&Jr2KOT;o z8r?AGwB7CbG;Ztv%U2@|3)F8QMyf%_Mk{BA4ocTS-`*eOn2p3gn5@e!a+V533G!I9= z`0Q~uQ}0^sR5iBVJA);<$sNUCmZH-^^jAoDp(y|^^uCb;-T=r*K-n5BX(XJR`d?nT z`BbG^ltOz7(0jB58>_n>`eM8l@yfm!twL_w6>==o;~&6th(?#4z*@O<*($le%xN>+ zog>aq8=wnL42v>!O_SA)0*Ak#FM;@|^CkI&q;cE`?MR(c0s7f|tS10N1ZB*GvB5EDu8B~y*_XO?=sBE-@ToP<)6FZ1!e>5EYXRs)8%oZVFA z7Z0kVP7T_CT3|WolnGNHKzYa%IOaz?fD-`Xa^sfZ9*7o*8V^V7bP&Fif=GxiCCg`a z|Lq*4Cw(v!q?mnBrUZ@+qVgf~SK=>R4PxQg)dP*7iKgpu3d}2BhPljTzshJOzb4u2 z7RDsk-hE0d+k2PvfP@^?i*!Q8NS%*{W!IlU*}n*#eCyiD*{yvs*z%dCW64w1>TDw5 zz*WH1W`Lc~U=X(8_~ap26h1vCLZvCI<+ZVFMCD` zf`=q0Ag-d0Kq1{AR04o>E)q0D!f4vL0i*Me_^PrFoC**d5j6oE3Q)l-acI*C>Qhid zfKH?kIFSdO)M3pb30$BAF;fkeC=ZM4tQg@x%SuEfqJHpVX8UFcEgi-q!yWZd?t*Ir z6A#<3#aRz`dYq>@e<|wBik!(rinWApJ+8$uyzzs)nv<;+SZNY#V zhUUy18x$IGr%o^XmjrC%#A@tbD_m1<($>2?{lc`{vye{R6{Js-Ao+y02eej!<*-wS zfjkmaZh$*hAU^njxDfphmZqZ%Krv-NI}?f(fJy}s)-r~c0@^n}z{KbF1JM=PZ8unM z*EesUTF2y5D___+?+xg`72=bTK3N-_%(|Js1n{wJkLUNB=StJ>p*HKCNv2ynAdH@_b0^zQMCGMjZeVoNh;bip64A2m`uISh6LI^= zGiw9vU$|RA5@q@w+SXSf=Lc6Bf`qW$l<42U)MKgeW17{_cdqvWEPQgg zs2s{6TUQy34wQOc>r~BJW-3#s6gPG;$@?)U(?j;)%)d z+}g>w53^n959Q?S9?zm}N62;`sw?bMS)Q)|$ zn@gC=Jnw(!N?%8}JJfD5$`k55)CBT>1H)}A=p)Yu>A6AbH3`b%P_t}dwOF5BizsK- zk6UXTw^kB|JJ-`Ff4WUDLY_^D^q_%NbK**O^ojs0&eK^=<|i{e)Qz>&gq7WEP*G(5 zrewJiL|1~!!!05ev6F;UoDkC|=$96j{fqJ6V+y z4d`k#W6LBWh8RK6w1{chc?*@50Yn2G-V`ABf(ES2aJ^&sAAE$L`scjC70Pv2_YaEz znS*{_VrU`)D5|!cwH@yScN;+)1{yR0W(0k{ZouvNHgVN_9R%!P5LAHL951@X2ndD! zAG&THj9%R&iCWP}S@mU9s*!w9Lu~;wL&ypyh1)yl9czuac*ifVET~0@?kvhQyMK!0 zAOZX)K+aQ90Jz*mlQ4l_=T5QRx+-EliRh*Y@{j!VPH<)D!gDSo-as4$o&p!bWm^Qw z6BsCchj6L%V zyK|{O;OGKhwV+o+LD4|mxUb&?H!Rkr6Hksi+y=F>Bf(=xuMI@rCCx-CEoV^v#Jf-@ zi=U^^D+%e^Riqwg6zOq~FY#b=ap$f?(p0%Do{f_Wf>9|1+7aZqfOaIq-;C)=JOJ0S zlfoZEv)6iu7CnhW#bN_L5wCNfiUhq-s(RXh!E8qfvAHRXyZW`25_O~Ya-E6N?_20qD7p=0&Cxc8_TgfWW}cJ z?2`U3Xg3`{5Qh>L@6=w!5Ty882x6*y%lJv*4C?{z(hhM_wxvL!49Y})phpB0**Iw- z(^Lk}OoRkY!weXOkGPq~Nz==o;OLj4R+3{6$IMilX0D7E)x~aR)dCCN<6XNbyRnB50J;IRBCj>fB$$#YhWe4Aj9ZT2cdB& z(GzdqISsYu0qh495sFYU3wZl@Ap3-eNsOdGssMXb$!5$Q`-XELZhG(*7uVFecey?L zmL_Y(=DG5&FM$`JfHS46MO}uw#?fzY=gr^?(C%+Li~A}6>XU8)Q8>+`Wprd|Ly0raw6k=miG|jhzrU zq){G4eM`|0UfLuMrE2FsEEp>WcH98h`)bH#PnCa$CcoV3>LOWmU(+KmP#1 zg#kZVei_Q^x}j?%0Y(5*g-mn$&AxKqTj>uk14Dzd?BbnEi9u(s*TN;K;gokHZ+@g- zJGf9l;w`Y^HS@KaNX6qLgX~Qy+cic~X2iL|;D$*2CF-WiW-jjwXT5%&Pc#y~Oi-w6 z7LWp+IwZXXRTn$u`0w%}ljYN;Z_QoZ<2^EQM`tPRXR{kjo0}qZ(D7Ff2Syqx@RPLV zZu*2OLk`r~+3SnhSwDl>+4zNu!vRQT=mQo0%sKW8g6?i1oxLX7stkaEQ}EM0`bnii zSx>iy-8aPddmqO={riXTd0zw?*&5fn7D3UvMzd`?_j>l%UhDY_(50&oTDl6--<&p3 zZv&(NflLJAMPMjw(alcPTZZJj(k7>7gURl49_KG!S8|90!whquoLV2zmzf@pWG=1> zLL^)rUbUQrMUBqH+}>#fijNcPp!Q}AfisoYTyWYyR1O6Us+zs0No_O`4nOGZQN_*K zeGO=0$WeLQ9(njv395$^$X7t@v0n@8-6W7J%$Op8^N8UHB|JcckMM|69iXC6ey!R- zMMRVpq(Hp~Z=l|zF(=ZeHM{nAGtgjqu`GL^cxyL+RdL%dZG30A^aa&Qm7RWxSo@$H zZ8V68z^efF2S{?@>}4TIj~LY?Aap?V1Kf@P)ZA4B+zqIV3;I+ShV1|_x`y&6g4fCP zN@GAlVWO(_xvWe2Po4t46TT>X`LqHcoU-%Ipz}x}EAhUt=;iU8MuWkC5g#^k3P<<7 zhUmc01Z4k5XZ_=m>v={mbIgV9k~+-v(vAalK7cDsLFsFIaOpk}CCkZx zp?DBo0{F{uS|HK^PL!H$kQD{9Ukhdc@jLxLaYcI^D+C5G@j3{DYwpQ^B`^-6M8>P&v9L+f|M zOG*1jMxW1p?a^jidwW+bdC^?mBDwoGk;^5)Ijvq(V9v1c1!wZ*H%wmv)#?Ur851SN zXwjA5-P)IVeu#S9O`bH}GgNiogSl~RVU;s5aQVw>+D0Ig_#&)l4dtOlDAC!dy8@ye z#K~re%$WgQ*Wl7R5Xy+q7SSqez0%x|_Tb!opPH$+StS`-HOHrx^x3CgB1ZrF1?nTl zPTtK~?Q=w3tHG97@sE`xN9f&ywGEB}1w~%pDKgX!<@7$Gi;w_NSqU|8Q!uR&Tw@c4 z*3NEw_wf3XBUB#wo@CobL;Hy(uY6ns)UEjEpQknehD*i&>51(aM~}czao6=k_TI_X z237}4_9?S$aLQ?;B@Sh!=5M~Y={Z5=5SMBbGN0;j^SJd95FMsnYK{Kf_B?H`u)pw& znN-g-{N|dG$h@A2uSp9BJM-77yDbx!MAIxD?|NH0R~xZgiInGU9H|5^aH5SGkJjmW z(A)0>`2_hceGye~Godol$fpj!t-17R+2NDV>i~)J9eU5~@~AUqBT8n|4eqQ1s)NoOc_@N=!`bXFh0xl`R;Py~;?;$XHj_SMD7&8QZCi^y{tY%i4vnX1L|A zF4b*mYl=6%aL-@ZuG@OZn_AtD%+WP`aDOm~BglPB4*DEBjfkzjkdfmiZ#~(3|I#4U z4TqqXdwys3Uy#7~C}4tWU5P04l@2xb3!BHy&c!c}>Pw?tjn}3dK6v$*35sDB^D3z- z1EPH%#iGFvsA*nuV%D#sM}dG*fom z-u3Q{<^e~ehardw#BiwX&J7Em6HU48ET2%R=WrblaHN^x5;C7;1i{m2YPrU{8PelD z#J}XqBIglY@M+PIk3%=-I%pFl`0i$^J0Bmsd*s)vtV6-#Frt0XRdN6Lp=i+NOAcX) z2WNvAo?w%T%#Q1?QDl9}{^?zQ9}N!|hf7?-`S;Gb^iFV$DJi7)e$emDPfe^?Bn!FV zrax7`v7?fp<8d^uc5IfZ#qFl7k8Jh{kNqW{i&PfOhd!H|8n4Nzq#E`}%o)L(UVWi- zGcvKsDOXEJU#eTaq{Y8GWMqCzH zv;juI9Axd7?BwCn`KIc<7(Llm9am2M&>RltcFbUiE-KH zC4-&h0`D{>T;k#<6G0HC!zCDVDsJ`gjBC>4-A}7rC4uRYH@WCyLlg4-CCb13vUZ4D zJ1|3cDo;`*9qyNr5mEPdsXHl>mmG;)J4;vbkBt>EiMeflXnKn+?rya5PDp=KIBH5;AJO28&zhZ9qmN)QWuM_p1&uhlW!0gBIl z_w?q*)!gC#C478t+!!N4pSrXC5w~z}FiWnX7QKC6rtwvRMp>@Iz_ZSqLDJ{aCl6AA z`Co{OT@!kZrDN*nhNaw|-8!sIigPjnMSk%*`@*_HudVJKvx+EQsU;XS=!6#kx0T zYs5i&#rVd7C~P!+O?{l9tuMGWVo^F&^8XItpKpc)hrnOlSFkqoIO3+_?v&<3q$ywp%|z*Q+pC;nCmjcp1LLWUmbGi1t}Xbu}P8M&5}9$pBe z_}pzg{-k0kB|V^~J{A3AMJ0b_vras!WK7Sp%3t5y%)MZMN=EpKscv1f>Fs$srme&5 z+k4KA7dg$%t_E`}=(7tf3_MP=9Fr7EmC>7@JIXJly^D?Ck-m*yKYl=V^!PKgxyolX zb`HMdpF5(FJ?e&o|1;{9acUq?fk)yGH~Y^2*36#;38y zB^^B}>Bu#{&K^>~!nol3D3&8!;sr=gso_sJ;0Us&ThH}!sNld41JgL7JP2ZtCpmo_ zQ-E(qH)BLY$~6?vp;-&PU9n_ihl``H^bQVbpLi&RnB6IrmmiQqG+d)VDr(g)+#5NA z4_+7Sh?=x^%|E9g(Fe{!U*s=sVTtO5lG_ufy`>*cSoOh9eNR3- z`1rjv?mN~5yom=wS{(~+N&CV4hkLvHdmBT1L5{m*K^H%(S@<#siBH~4&wu4~*luag zlDV>oksZu;av0-q%+?(gtrRj*)a)|Glp*hBZ(lrUHj^8>xt~>gwPRtIEZKbW>eNK% zJK<>BD`ZD^xXB=I0c^jS1a^J8{uKO!YBh}*t}z}zNeM@s^LcclDXo1+TrH~XhhX*ZEKJW?ee`kK?OvJph+W0UH2n;GwRs-b z2-IywD3%v^;xqUMwm#=iG-aIS6bhFRT93Shi{Qo0`Ah}V<4da$EoOVOa#U90MQB~O z1^ls`Z)L6w)$tS2u^OIdlYzfEj49iPzEgiVTtCfuCrv$g{$9Pt;xDUs-F>cZDCR6x zJ%@Zx;%}uCb*$*c!3Bxg!Ov}ZKK+(U2B?vf*sH$1vcA_o?ltZiD;OAL#q_|vU+Ohj zYag2#UJqu@C|Dibyr15pwsPT>(_EURxzyy%6+3+F!MDL;X*Jj_jKcY&y|YKXrm+_X zi=;L`KX~^dI@)ze6)qP2Y5qs&)826q-bs2%U_6=yykt#3cq{-D9&#Fgn-$NXN{>Hs zz;q|zL8O;`f-Z@6@DrIDsAuk*$g1}ESZPq{`fElhqQyq?EHydd%X5*WNg(Ur92xOO z%l#h1`q=I`rGt$U)$C#kmv|W=0sI7YIn$RG5XbY#M=9qp`yx1gZyY}uQLKY$Zi+99 z(6{ez+I_k8*_&CoEHMFlVD<{8>mkvT-MA^Wgi184bkw`RIrXEzn(jZ>qX*%QeDn(w9-jOIdZigzTY4+QjmM)ux`O|_%)j2LsUT(K^d5S$ z+7)`ETid0HLoej;T(gZ}t6B&yfHx4JA(QJEs*c6$UxLB!34gJ3n&5;cHw`Dd8Ss-1 zm}jTp1r3OhyB9bu(=-4S>R-`IXzsY3coAkg+uYo9GkD&9phcj+D$)RH@c-AbYb;nsxDF)krlf8HhM$PG`>WS3$A;D_RR0_j=X2cGf!CGiPjiS zz7&=}+_slrcMuABUl;Ay^R1xj%-2fSYvLtA?KfR=#420z*b$lP(`D(Os1BuPL@OO0 zM(Q;fo9urNhf8qAC2#65xk*{Rm2Ec>Rs+bcM1>RYzF`f%p$NoYz=g!`+-8v1-{ZrQ z{s@;ikL@_oq_Sb{{wUTEN=zqOQ$$}$8+*ZFSBFDoo zquBX239K00;u|heK#CR!mte}CDMO`3_c5^F8^6VgS0g#bw2I2(?=d&78!@kDB5!c3 zIA(Xp-mNLE+dy**=Z0)@CdgBjzm+T~*Bp?qFRcg#u9TBOK6u_?a80Y?{zx&vndrK$ z!F`P+MD;`A?w=NwnGp^zYd!8b=QhPGhvYnrPCe|{l_}3CexpJQ%a21!ycuzrA6)+6 zU@p~jv#4vc%Ap~0SX1HJQLdu7Lcex5cXb~{`ZJeF7lg;x+R2mn-I?ynyAPC=6wU{b zz`$GTI>`y*!L>)R>Fqlj7@cd2Cn`zNQ71Wh!zDnMP8o?xjs=7LmSCawiuu*epuWMF zkPOHFJG{W&ydpObk8U2Wn;z$S_+=JvSzvSsAfZEL4)uhj~{V zXXVn{P_easfy&B{w%!+94R<73a2RLbpEU~3u!)Fq(rEOrDv~Svh}1<)t9PZH1M}+3 zL|}mPeG3v<1rzA*75fF4Q-tF&ZS9Saa6t|QEu+BCdi=XG@`1lDh3wg0Y?K#4%5<4I zg*=Zmc`h^f)PUJsszrN6z|3jr4%KR4aaeId(FB+J?+0F0DsjKWa*XXZ?=M9wn$Ll1 zZ9m5W59NW)Jq*PqHLcs2q6i)cw6LjT>a`Dmu?P>=Gxilv^6zyj_elN6g+ZsTwzW>h zb#_rI>gm$rZ&=a8gm8(J2&@}RlRlJj2>gKFwVNH}6R{B}gdra(T6ens(>|&G_uMA(e_h;uh?9FeJ2=^4dlelMQ z)RP>s!ed2;HTy86se-W7!xq->TwCdb=WaPD}Q;SJvPBIdvyf&#k#n#Ex{yKWLi|L_#Vy zE3Ix~bhP0r0Cq4Rv5;JprxC*O^EX|kZ32G&5(T0VNk}R$X=-U>Bp8|T6kH34wU_BP zxZNwie+p1=EI30U-q&O1C>~}w#ROv~@lOa70 z1z)*Emhdf?f*gl5g6uVw#SlN#^tEOO>Wjn&G%EHR$odOBclQScgctJ|m5-ZB8+J6I z9;tNiy`sH9s=GU-)Z$GP| z=gl%HEb|;k=q`)=`k0fRA4QW6dDDH`rkjBhAXB=q4&|U)lh0_&w|TJq3OK9T##x?m zBAO4$XR>F6ag8r_V#kgHm^lMpu(#d7zUN8Sywc+;d!4iYWpU^w%VK2i=GN-C{r2Mv^-38ty&&t@ z8vT9ST^Y<>OD72g{)+eh#*X?-`WwAS7c40Ln7R z6Eoqr6>uskP`0Qh0FrQm6UBNwicLbLKJP0$hs*TE0Q4gzkT|*LPs?Ocv}o(j&pGZL z9v$72A(*Gq{2+BF<6F;zwd0;2yNV&DACLTQ88{1UB8|5OwNe*T^}7paAI4lsc$<}| zC31g%i^TlWcWn{Pk>2JVX*I2z)4?LRNO{~bzg@gr#3}ei=n(G6s%c&S>zGr~l=~*& zm#nm1g)(~w8*c5XdIkH`|9h3YuKkW9X3Fp2htE<&-^?iJn{=PX1A_)|ZybIG|HA4I z#{v0|?k9Z-zgko2@u6Ca(f2S^fJeZ~>h)urjWzCe#GV1(s{mGAb z^T$NuoO`}CF<*4HWtlCKGarPOaDrvMJg?AYdlk~FZ~d=`4>3&^QF}0O-@c3 zS(rZTU}^V^f?ofK!cp;-7KXwnxtjv2##xAcGjI zL!SM)j(V&;NIN*-AO!^v&hZ4Bh~lQlQ^P->&AdtpCw?Mv9@hctko-6o{!jyZ&D3W} z6LVXeMD#RAce?*SW&}&%O$qTaydcKd?|b1+JxZ904)ZX za1T2lVDb#5B+a4EV}bA212hFHbubcJfMQ|4CvpJ?0*Hb3$icha>}yXI995QritYa% zPFvz*)Us>2i1PK#Y~*+ScsWOp>JL{&^v0poLI@CP4J_R)J0=(%+7HQR?}-3F11fu8 zP^o7i;~2Ih4<%B-fm|a`9v3;=n1(`2@Zel#8*_fEbZRX(RGd%!LRsW;Pi)2`_UESG zEzigQSdyw?|&MzfiNE_}K9jQi5yv9hQD-y0}SK#L8^U|E;r+pq>} zqt>bfhD zxbh+8@1cTRUFI3H*hgBTOSa6^a+|{)ZzNy`$xWeimlrBfsh?U{p7tf2!_5JjARfgM z-4_V!}O{EgGfRLC0#&D^$DzxBjZA8d`t zGHp%G+Xnd#la0Mo&@bw;SQ?G`T6r%m(j@l7!1B6@x!jnVz@*>|qTo@E)h5-_C~vGD zoEs1$O!yanl5r;dk>=vblkj!g1DlfvdjKV$Xu3lY`uarZYd)a(Yt%WFIl@1NbCeN$; z$`hE)FOI1daeo+ib;`t?-7U{J?vB&wb*hlVAhU0FX~)I+Vo!D};kf~{O(O(S7NEZx z5{S!%K{PBlbfjmqrC?Vf*TrxeZw+315(;}k6z6h2^t$r5B~1iP1NQS;X_xswk`3xn zdIKq;oVg(>xDyL4_^?G$l&9dNkh&7CXWs>&fnR;>4Kj;E$NjYcc>i^BnmmH$*7 z4t%7#D{w3!-Ka7N@av%$4)wSJMvA}b+H*ataO7lO^niToj~DW0OY4(9ioDMI$HezF z5B|$w+hjA$K*ANU_0z*YBkx>UR%M;mVt`6JNTtAu1ZCOC-G4DuUMKPc@n6qljn33~(Z+vGD!c*y~8wc3U?Ji3IALH_hc77IQy zJf?4H{*d0nKOA$5edA_p9K zp=x!E1g`N}UxEN`PHVCD2HXP>uaZ!7eRcy5Bb?A0aLr3Na@C~X(Q%Od*#jY!iZ`0= zmr89Esq-2wzw~>+k_PwLcf%_!P6H?oR?L1kduXP(QoK`p<-*2BXiONy;Tq3maN^fU z&~>Xu*5!lY8Z1uh^dco3B78Uv-~`kEatx`zPyn%ZFQZu4Y(<8{vFeXv`{*so!2iLX zeU6vFArKN2y(dVn2MOsvlkelL6l80;>{$O1AmN=ahw4LnehW3P)tH2C4`6)`h6aB> zX2ib$^foKxc&>r&E)j@81yRNQ^j|p>7+Ut41i=LZl+9pEvH3>WHSr46N@Vk?kWoxXijwDMt{3>j-;faAlokLyyNup8EOGp z{Cij8p6Os`Avn89!AW|OWWZO@r2+gx3d^NJ7eG}M2_T1zCY}1IgJRF0VoD=QN9025@%q(cyp?nX*V zKu{!9kVd2hhLUa-5u`_?MMAne1%{CB9CB!e{_pU;-~WH_`qyHyV69m*&-2W^=kBxj zKIcGxG4lNjH&+?ydbfVKUSoGESIo5xSM(6Mx^$LkD6Rf9rdsUV zv7o1iF)xBS5By^Nf)bT!O^rpFxwytQ$KnVJw_w3Nb_*13meNiNAP8XPL@~|5kq)GN ztU!blPUAedNz5k01?s29YT#%Y!A_n|i0rJoFs@u8lab~yec=<0He_$ zP`ot91EIZDWq3GfSXP3^0QRRqAY&##SR|;Z1Q0y{3!hHkVfdT*1~z#rpwtZY`G#j| z%E~4p@kp)DwPp3SU*IdHeQqDD@f*$@m>t2SykXE^ev@3ND8q7*r*ny82OGOFkzA6e z%h-0ZTapHek)rV|B<)1n@AEg3!K?&}x61J7uOKLD2W!hq?O`T>yW0x_8L=>+oKWNv zNHzpl8ZMY#&WBEn_4+jGNl{Jbh8gH>`_B8~qG64N4E|rN$0|=YB%(aJU6ha^cQYI7 zAbBvX9*3f>A^y0N8UzqIfOxQgiU1I+zd9LKWArU~0kXMB7&#fzXYIiM)Xtm3_j>ru z_{CpozbuxqMQ!D;XBOhvN zm5eIGCnO4*V#Kd+^vI1ye4AQv!BWI+{MpP%j}^#26gYnmmCwk%BA0CVi93mdLZFvq zMiKs>bHWUCXME*kbS|rAlG^`^H6g#(z3+Z=Dh7$aZu@O=@j9&i)CiS)J8J&tC8(V1yEH%=w$Fn(|7@X7?UG^S4?Kw2u;Agk9h+cQpfgcukyLypI55v%{5#r+>Nx6+ zJ}kFQPB4rU8z}&^kl2tSUG8(6wmXDHDVBhU6vP5-&%ma01r72yh^4<2`*f7u&Xw6^ zr2cZ>LnWfCL}dxvt)!otJ>yXnPT6e$2T&a_T>x?3fi72}C@%oaMie8tvPT{#-`8^- z_w3fhMZ<_1e*J&JEICs*^Zt1M+}^&pECGPM~%UU-(OBk9jYCJ7J)z1d->;&zuIH^ z2}f7{d_(D7eBwXrzmI7ybX zBb>JnM9{yH8dK!ydcKiTB|}|QjevOr zxfL2d1RQ{(<;}&^S_iPl(%Ab~##hlYu_yPS5w-$f?{u z34kqEWRd7?A~jHk;VrQvUGzab(J)SEr1QbusXKjLtD{5(X>3f0RXj)p(Ix8Kw@bZ+ zv;fDRFYmo&p#T*i?Ps26lcL5lyVAUL%n$c`0bfmne1|a)SYALQfa{?`$}0bd%)Za! zw$=-je|k$EGAQeVJyXG3WrhPGMc8w}qJc2z&Gz7i9u2UE-67~Wj`5ZLuf6F^b+{Az z>#eGpf)oP7_LLn3;`%@Za$)nlOv=q42B9)|ll?CPvv%wHlEuo# z<-awa*<~}@Ue*QhsPV{+j<3>F!u^W&97(p0<=;iMi*{;@d+GqihLUh1Vf$QrsxNGg zKk%fvQ6bG1Ovc61d4F^${VIR+74e^7hhm-po)Jfh6H)GqxN85?$PdHZ?w0fbGA5u^ z`Szk7G`QNj+jZ#)x|c`0$>a$BUoQ3)77AiV0EOZgfPLL7 zU}!A&AP%r|F|d8RG=@T-n&2$zF9bIGN`}S)#%wCq!KjZHkcc@QI$jZ6fu6BG&I_iNd%3n8k7ArOh!gCtR_~a$8&c3K5lTZ}@Ww&UK1;EtIOy z;FP^e`qCzaIb-0;c=b8+fuwfNF1+<-Sq;0$d1aA5zPlwcdT3t()IGHuYA#)T2?D%N z128T!-Ch)M=;$pF5|yPfGwsu9FHq}ifOV`FhT&cQXXkmMXyiAvc;tV9*^hYSrmiQt z$LBhgml~>O;?Gur;te7n06)NOgzp73ME?@wOiXjdx!cWyz@!CR2+;r*wqtOtG7d_@ z1c<_QX0_pu$6!y+`0yV%d>3O$+-gsTg{8*TzxXG=Id6`y&dNw=L`}K(w#3vO`<#B~ z*y_}LkdI`cf+1i+e1-y>K;-%+;{Yu>KXkh;wqq$^#AL(nx}o*0&JIqev4m1(VaB7- z5awMd<*L`GqT4uXX1(yF`yxOEuGZ}XP9DJXF(Eo`-2s@1f#y8>`jU6`HPmbq@|9#| z#$v`O1-u4r-jIo4G9uVaF6TiA?@S0Bm5tdMzef~$&*7%UCbDD)3{w6dYK187Z}^25 z0{i^Uw9~T~4pj1Mp$x}Ek)SA82;^@7RRPuqa0*HfF$!$jdfMnQ@HZz&=X-wHOy9$V zCK`9C-H32*KIfw?>(!&qCBmbRJs=TX8Lq{zIWkH8o*pGOvSzzx?1e%Iw^|rTyP6z+ zCREZ4=+C_fYfe0?tW(+8Hq%wR@eUYlquy#ZX93?VFon1;Tz~-0wbyXZlJ3#}5Yo3+ z1xe6=0mEEx$%l1M>aM|iFSRr)qiLXINJYp9|V#i zm(TqFxK)RoNI#X&rbbf5PX{6(rhx6fGJCvbaRAbOSwgvc|4Zy@F8B!CQonjjgty5n zg#=RT#lm%v&oYw|f#auFqiQFj}17oI6M+X!Zh-45|Y-(=UO zu>x!RnW;+bsmh|+;u;Oe>7abuV*h%Xz^S{IJ>6b<=T)@GjK?VQBA#bC>1o|BZIkgu zZ@29Ude4xVh!*FanSqGZf3==Ggytuv3oae&pzmirOxUk0 zY$lKI&d|GX;;2@&i8ovbIGzzYI=gnKRSO+iDXW^lbeaq*)e=4NaanDsioun51HSiT z-m>}ZV%fP%A>XqlniSRl(E(IYOhp$QOJ$+13yufsFF~%^&orG$0N@D33qvl&W1tsm zBviK1bsAYepxl(qbJlVHf4u;uuDdREzhEl<_xW-?jqVPwO_&%nq6}NfwywXvr>Ar# z>d%eGqrw9KgEE#2#m;z_>`J`lhwJN#XqZZM=zprsvL5d5vcx>wG*^4VIac41tF=3o)9=j(cj5bcL2yw!!PaFx^3O+i3?g{QUwY_GZw0G})E<tL1dKa`|I`xq}K3M1-6VwINo0jqdAMm3ZjRUdfzVnNE!&}eF-pS|Pg?-k^}Us3ZQ zWy0wf;AuFxh8`XvT7b$0@O5>c51=!%S={cAk5VJ!dhWeJC4?M%z)9G^@aDqCi9d9L z0H|H+mmBXGkkng!J`v27?`V)e1SH*8Wz2ihA`6D4z74Hh#7dK*MeTV;4%Zx7&73x4 z+K9ZzGbZ-ND=JsA+l=jQv61t>XMFWXQXYjELqs!%xuFT za?iJGu2T+znU*J!vHp8e=6$Z!J>xpQ`OGlbJD3vc9lual5FEJFn8;V&$j!L>b9rk= z26AlBaJ@NYez?!WwWOo?_K{M&36_=%wt-6B%Ll|BdGXv(ZSFvh9)hL zd!t((N%mZ}U$>l577wqXr4HBimup|juFAzwn{w5Dz&8ttm!uH1epae`?vF~dNMh|^1gys^(g z&WJ0eA;W*ug~yK2%|JuU_9_G)E3Q1WdaiiffMDIH(q>4+rtl2c_z3@TG5!z?epgYz zk^1X?h9dc?l)*W@%@@K<^#?({HbG^(e*HFK92bKF8~`s`3nzt-`)V4NtDUfJr?Yv^tl(RkzsJm$90B}PYtvyaS2t^Pw{M1W7& zzj<6KxtCSth_TGRRbB!x#mWjg}kIesE_S1DGB8mfS;O|{GB z2w^n1RHe-BC{f`J4e^PyMYC>sjU2{K*7+L;O-?uXB<*`hTZfk!S+pxAsPbDrqc#ot zX6n?Ew&*+Lxp{}{I0_m*8@3nB4-LPd3vJ(qT^I`(#t3bObYA z)M(!`E=0u^f9K`#EFmW)T^uf3g+V0Y{k%5r`Tvvcqr@$XQ|9PWc-^Lx%i$&^ziB(`( z0gwg7V_q@=fCsXeyj#L%G7#|=rRJIgoi4UHV`t76VJu~}0{cC=p0{W6EC$xNSxQ*E zzERw_(X>Ulv1-6mMB^DNs@a=e6k#<*fdJ7%1L6cYMSRE&i<@EGpW#1Be~%VjA6pi# z8I5I}jy<>`bagEESn@KKWQqGGW5S7|Y)KCRic-AwlZYy!R9TX;1b8#bi+t+~0{Iam z>WVn;VKk{MB~HapQBSQ0ljYW0yygzDinW0Mi#fEs9*a$bj`{~S>c{?Y+9t2s6k2`M zsu!De%s?vX*K_eMTh#HL;%S(92+RJxb4>{oMCnE6eTAlcRdUHQ9!($7mQqdP5#m)y zdb)mweA9#loN}uXa%B;h>2+94bRZF`;(8Q(QUxaKb}q(GSPnxmD1l8hjLzO{%C75VA zHpZ2^_d51_wc>m{9ZBv>KOL#alIlTnDtFiB^l=}g`;MrG=s737(tQ4LV)5zMEEq>q z+*-rC(_LnUY1nKPNs!=bHC*)mUb6R{t2a%`H%O)M(VfnK(HPwgtE_dnGoEiX=`K+N zjr;loVbKBKe%RCHA)afl^YC9UGxEcRJg1zMY12I}WX{C-%B5zb8cP{75u;g`7Gmue zl+Bw$$B*vpcahOH7%Uwx+|+jkcMb8IeYn__I^$x|n^OqiG-Nc_9~3{eR_=df-!BKV z`gs>~*jj0?B=I~g`mZV^QwAio7U-=M z*J0PC7nxe=F~3<6U5>s^N|~#L7w71{UB&V@NQ)oS`WX|fcsk;tW&%2|rz^MUwuRu3 zbg2t?rtp~D+R1m9d)y<=GqZ@NXe|;sXF+zzB3+LAir{$-Dt&x*MSn5eSc-V_YFL{{ z<3-GMsXF|s`_at5BAJCxXMxp=i;xDTGv(%Nb;cL@GV5~r)r&dK!7Gih zstrCZ zABQJ=lXHM(`8=^-3cPHPSiZcPZ>-mh!U4>?&HSwe8?>N6<)A3)*Swst35uerQMPr! z_n>?a&A|5nYKZ0tZl#gt^~J#rO)t{&*-ST?P_uvHk56QvzK?9y*ljfF%OB8}_f%x7 z7jye{&a@1CCcZ^Mef-?fiW9z97taHmo8t{9}#5)1-gpc zeA%p@e*k&RX0F&3+iySCw74){F~33_{&yjh>bY!8MB75m`cQ3r;(X(K2PMyWNh-|s zgA6LDqx5sC&wmx(E|I77xBdbmSd$v zDZ|y=#~ka*IK#K}U`DR+TK0LoLCgi?D$KBg1g#`50x}Idn^IPpi_P0u&r~n)zduZt zE&N`WGeIf8Pvv>5a~`ExaPOyZ5^_`1Il1|H)pZGF>NX4_&DG4~@+&O&j8+L*sYIyg z3$pt+LI$;-&{1Bcj3j59k;A#<2Z&0@uPb|- zl^lFYusdO#vXmNCUnjeJif+)^1@j3f0Bo_5a&VaIS!)}jwKF*qYNX0{*oXs13*eBc?t zoEAO@S0-t4Is4N;r`i#92@(GC2_+1)SI^xTdL-+T`g#r*$cFXUhD$ z!@W-}P@TW{%->Zc{JoAecj#F`|W)a^N`YGopZZsn%^xzvqjY(G)dy2 z{9!15Q}y;p27WGJt6xE-q#-VQrmSuz*w+IuB+6yjQ)Fk{(zjF7#CJFCJF2E5#9!`s zJBvzDtr(2#U$Gy#6eNgtCEG5qm`u_4MP2aj^9@Pd$MSmwbX)AkT%t?-`et*1PL26` z-yX*+%mYamWyCM8AO6IZ!(~s6fQbii%7V9fD;=QTqGNxeb~R9%5_?;yfi4Lg{nfH`YKj5T<_miK1mcQK0 zIee)`d+2f6qtVIKA_g#uW&`mQUwJRwzi0v1N zJmPG;ktW=KU)60U6Pm>Ls$PzIy}mc)$pJ1wm$qQmTqlT_n-w<(TJn_LuQ}zcr-scJiKFx~)uoE}Ge48dLlWKGE1eSZX*C$a(1opAzOuJITsNNaZfUK@y>T(_;V&E4! z#i1NMBt`p;T`k6lC zAfX*$59Ykaim~N6?~;xd&3cC+()dav9($qG0JA)gZRYYxT{{!eOz1izI57ozWb?cG9v2 zMJ%9fO@nJ(yqESyV^>#Gogs2tNRm9J@gmLfch^AS{oJoM|Ne;KWK%2TyQV9Tr5h9w z{|ir2udX}$fOf#{huH7R)MCza7aipMtwMsSUt~ikK_tHy7A?W<2Uc#T0QqQ+UDey+ zO-J2avL76=*koK0gXUy&Kdb2$N351xzRMj@o2rb*(vxzYkW6NdQu==>F21V$u!H@0 z=W%T=A(j>eZpF1&-jOS;Xqv&#(_vigQuYs-Kz!z}Dn5_AXsilo9@vG5qzc$iGU4;u z=`02fhQ+||HD&DA4OQAI?GLa2P(1HpQdJrCpes1?u!$E}&RJy@w&JHqB(fW|NlP9C zg~*v&t{OP|$4|E$VM%zA?X&J>k*m5q`xP8QHxnGCgGM8~6{WPvmWmnqKIm#sF0Tb% zLe=k1Gy+I0+1gCwk8QBTZ|p`4T#h(9BD0wyd|^}X36j}JUPtJCRLZ1#I#`1kJaXrU z%cy^v@}*R87(o(fs-g&}s&`qP2*v>4yh#TA`7dkCf8C1+N z??o~s<9DA$6Dy``C-?REznVr`*0At$?%*!}pzg^(LuK|UnZ#%A{Y!{*-FxNd)S05` zLEPVi^wxK-+nz1C>|T%*pEB}wX=5^uQ-1yuvS*tD^T?wRQBp~HWf51{9;aD~=jZWN zQgKvMG3q`gdN-ooi1Wpwbg1yjK1|>CNYq9T-}lY%>{v?`^|!=RC{*lK^N|1yKL1$x zAUioPb&5_iy{d0&XXpwtJMHS8t+ZAcgN>*20XuEi=!36C|Fb31Wap1fjrL0jsBXTy zsEs~e!;iS@A~snGZE?}TIl0A)G_QmV$FulG?`!@#qgzkA)&k+%1*cE^2uvE3?gMIo zbWr9uRFM#{PXSvDu;&AqS=5{ejH|bpp9<&>>-S&P!W!<-_ygR|= z3Q6|KPm&N}plsRH8uJDUhZJ=yO2Q(wF% z&tu*PVZZjK%h=-fh8ok+E&d&F^ShD~T%};q?<4ze92)+WTlFW_I+<9k_Wd|&TKZ9S zQ7+=U(W!_0L^|~&#p9{!5c~)yBDb(Fsrwj(Fn? zp}oqwbnGYCvByzrgUWp*rVMdxo*CP4HU~P47=iK6Q4X&0?PY~IKF^FFN!}FQJi~S` zV`$(+c+{tBsVzxi`4y{Se+^9wS9`EpC*0d`D8TxKX0dJZH6nDE)3__mDuNjam7pK^;A3!*dnY!>~h@sIj*Hi zT$(@lbF^pQ^6{6BR5>NU z0z=kq56exAwu%>6{rD2|S*e!7bfvzfvV zBAiN;9Lp8#3*-meB}kz3^(haCAMB?wSXVu+2(vq|viZ_WPyMa6IO5N=s-xSAObP*X z?1mtF*Ox12H&k$u-G+!K^B;duXZf9iU@g3q6WVhf9|m&uFsOi~LYjeDyf2U6keyO> zy3%NLly19R$Twc6{TC}j5r`K{QepeHmk-x1xMuk-+nq4->4nHy^kr2G{d{SqjjcZ~ z3(dSdWGc7Krl9C~YyPT)nWcjEsw)}o1J8XEXN}7N7{^Pfs041-ly5QHb#AXsm&SEFPB7-OYq{-BYDF3mO_hFYtMPQI<1-Xxpx3FxT$W0Og!;6&!IE7+HTo#l6D@7H4nrtU1% zL+J$_gj*@(mg&gchA&97x31fs<5gS>kDeE23f6~|681*zCHReGT+T;amLl1rQ0bzW4YE>J$#4F@0GG6nuc|dWJj#50L{BC zCekj^`!1}b5jZwqw3XhLdrcUoiutBGhrZRN6M~9DLQC2)b=#f7G^*~>j_BFM^kt3~ zg|hfbwJx71T3n*hUNP#@MAA?H2}^N^wZ{O-aKgUn4D2`V&l9=wL)i*};a3st zB&^0Jl#3lTmqG#ffv%AXjrd2kTF{_OMypX0U)$YZ`3H{mXe!DgybWC}FbS-N1}!vwwxHr%G)%?Mr1 zo}DM#whX-wpjR6=lCjDDaq*a$`Rb5d_-dw>RLG~mtB>6DblOXYTYH0sMNqW~BFAC& z#CJmIUbZV_-{AKgvTukZCop06?#eSrkvDYZ4}lbSW-f=}{gWp%E`1F0(|6_j>QDk+ z`LmOA+b|# z_=^!43fVVgi{Rzb*Gk`y$y+I8l|Qsn19qI`^}fHNV;PfRc7Y1qCz{~>43Qq&cB{=VrTO9NWG&pinAYVrgADWw1W=A1U(6C{3QwF%eK0Mx(! zQcv73lzoLoS^!r=>FG|0T?3)_$KK~LB_9Y|mj|K#CckwCH-7nzxRCu8%37~ZJWKqe zt_HczvCOOFZ>5wCmh6~Z+*HWh`fzslvmQzIt}55kse5 zy+1%UGgr#+&XMlp%UyWrrL{ucJBB!YLqBnKjYm$zT8)OD~|hiNg$ZhFK92A@_^^s zKA8P7pd?r>Q%xMRVkok^T+l}I(z)s+r)tmR7ud7{3DEHsukhGLg|Ys}wh@Dj{KBDq zK+TFP2`+|4pE}DhP~E*DZl%zc|GdX@$C;BnJt-*s>l<*MxVtojJg!7x3cZhN%UJdAUQ%EjlR^Z?(RfX(t7XoNqRZ|7+ngJ~_) zt5Dl!Xw3}BWkT<4M~G)TxieD*@4VJu${d!%etan#yww;c+QClSq2yVihp0Y--r$(t zzyDNfr91*QL1CU>1vHxI@Nx|094b*4s#mbTEWujZOGYhWR}Y(<8cC8Z8HCk288FPZ zr#hVVA4CLcI4OLXxzJ7drj+tEykvqac!n`@da4T7xB_RYa$#`X>0Jo;x6AjcPKhR0 z!Sd3QYw0eFi6@!H?1Q{7CQc>z41?k}`rB}^)n_v_e$@3fpYwE0NMrUXkK%-s$P<^x z&%#;eoJqdwqE+NIuU9p_H$`t%k=IQGUJVcz}%fh@zdRl%Wb=JyROGmIYJ}AX5h^ynpnw}WLx;#aP9)s+JN#F z3Zy_fHZadqlOyf3ipt%RLgtKJKKDLBit#L9WFU8%-*KKFANG|)3}ks}b{|utWT7{o ziY%jJgHd$@)OT*X%@zO~>;JY*Vn8eND`{AJL?y zUB+rs#n4>DM4)}DucQXC{u=&fR0}S2Ory+}AhF~ctOElXbL%nv{iM&>3GbxN*)y4rc+aOu=D|5Mj2zWrAxbWH6Af5ZbbxHdQ#YZRz_8L zlCT@y_r=3Q561Y=T~_^}!_OPlh`>cOX~%eC*8Q}Lu=)$tA2+A#Dp-3DNEJ2U|hV&wN=O zW0Qf!{D!W)l-2id?&-57O2*f(N>~N2gl4A8V)`U;z7@V$;vuFNRu8_8H&cSMw!scR zhYh*Ts7?N}|BL6wD0R1?Q%fOEm4aNbyoZR>H1+ zUi4t*csqG@u1;d|AT(L(46!rU6_`<=Lu9(8pF{T`>W)7)uUj!CJh-} z_cG$eh%T5#Fv0JOkDWRDY>f4x_7?tn-IKj>jyA5&9J2cxe<-k2$_b4$ z5;vpwx_G`W##QFrtWicrN7kf=n8`Cd;`ZV7!E$--ELg3u8 zKFoeaw2{vI6-@#T$)=^^O+?C&s{?^|?BOB3jRCiJY@d9c!Vuc^{lm@FPs^Cs6D!YV zW1EuNd`l}D+o{2nY;Fv>ObK8|1?D%DITF-^bp5AcPi!2@KjuS$Ebi}yf}RhslX3^i z@==XM2EFQhmCcE*8O!dgqK_nqhd1H$HE}z^i1})Yj2rILfM~^X#zzlqN+rR{Ka2enHf#YEUIl?b zS<(Syj~c})8R{9YHz`n^zajS|I>;AcR+ z#!2|%HW)OQ2-P5zUFL9Fofga5{56^V+yJYIrW@mOryGBy^+g?-BsYD~h#pxwl3Z9l z0gsyMEH?FQ#lnVe>W6}v&${(5R|5k+mevG1V20gu&7im=S%Qne*k(bT_Zx zM>rHT$>bA@=4pB;)1E**5&k&ZODXgea#v!?F?JhB-gS>n^M8$2Vhbm+{mwcWeBqk+ zuq)mP8Juf!Nkcq@L+$e7xADe{RDs1$)<3@8%gklO*E4IryrJ432RGsOwE&p7xE;-R zL2S3|azrabR^{nHvMv_*p*|5=(KJrfKRrwG_%E6APe-6rg*-|_UhGJ7{$raa2Xq7j z)lRZzxweG{Bqk6xulteHun;1oRCkjM;9^ zSTqyC@Ru}~0=ty({9*6am5XjiiCi_GKB|w{Vh)D)gGtRTjun`pDWBuWKFSmZYPSN3 zAYy^@c%Z7`>~y)P-0oKbC{czDUA?^&Q!y%&V3-sA`XV5=hVg*Ww42d(#7cD~K_Vxe}O0K`#biSa!wn(~$8p*UaB}QuFg7 zlB;5Qm2!cV?%GUPD*^#`5txK(NXM=xvk@ zksp5NMt3(P@9}ucGg0KnL(eq1X|7skhh7z%c)p00cT>>e=|`_rsJuyDM8mJfd%fRr z?1N$)4IS9eG?kU^X?x|!Ch*QC15pa`8{Wa%(eghxMDpSbmEQ*Ib-jlD)tD(@ZApdzu^B>j)k*CUs!mccsxYzH-yS2PWB5{b|O`4?P4&)y);sR+z^W zyI#Li5!l@oBMb0+Gx{=u@$)|JRM^}ql6T(8aFlxbcd?i`bHlV^+N*ZVk((7KG(wV} zYuF&DI+PP4;3w%lf#FTgKf+()C2W+b1`f`@{~gM);Tx=#5)54Nd9&uo^GW9>uWI1= zg8!(&{rrG~NK=NSsO$NouCu2q?21{5EZmznA5VCnN(kF=Hl}B4@OC1r{bJwNbng85 z_k4B4Uadc14GfsrxS5nXL~`|B#Q9@dd0b3SfDAypOS;S5f(IY;!Q(n966u{NMd1Zo@(@!;dqk?3UK)^NfRy@4apK4qq zkzo^M{VCzzfhLzIVv=!&Y!dQ~aE(0b% z6Et%rC?#l9jfONPNIQ=5jQ3eXaL|22FyDCZV9g4vlQ7+#-pD{Sc|^{%=)!~1Acm+} zs)g|Yqh z)awrf@riOF7#CEd%lUr@&Fm$NU;036YsEt2dX>ETDNB%3tE;aAAXQjTt`hIPc=J_` zsdlwn9s&Z61RnRegOZ=RzgJFs(VqjD>diK`{qcvSO+Ui;sG_39RP^T%3JzXB3VFm& zS(68!-qttZ)mwYwCgI~K)rVzTP8M<*>k&R_vwac860$2)7Jqgy46G2j+7>YM{j;rC z_CFX39)>&RO@?l7yAm&rTsfo`-Mf2gNCVHqg#3|0t6NVjIm_`x8&~!$=7#;OsG#mN zwN>@%e^PRWL9Rd#7Q_DiwiRr5$@zm}!hITsnE|xz7zzUFYpg8dM0SWz>K57jobo4& zT73C!r`0@vf37`I$($fc*-s)o55vbu3Nv19&-kDpMQ)nVx^rG9$2sqRf}(wKy~Z;$ zUyTV)PBDE{OH#lin99Qa5)bDao&aAWz;rJCh5Ly%+^K*w&%_6765GJD>g|(^ zub7w5aT5a`CK*ZU?#gl?Qhsqz(^8o3exS=q<_XI5WkFbMtUr5p95}fg`mgTEm{!QV z&dXr5zv)5?--<`Pc%vHbg!_BVDL%j4Bx|zcZ8>WbB9kM3#>-d&>CxS_V5nf44n~I%-%c(fT(HNQ?7W?P5sZE*GKF^YokQ{}(;4?{XiC)OQkMd2 z%UgrMy*_G(V)D4h_8{eg^ix+zaW`w>RJTQiRL^p&c3Dqp{HE(0Y$P|C97h0Sv7##W z!n}+p^(1`eNoB)fMZL_I%beiRd|vOf->fTjl`<>83(M>FEWpY}+dJlRD@SWI{^Y29 zyYtps;xb`Fn0q2Y4)N%FG-PsEV5H9ECN46)p0z#RC(pN^>nZ(+Qw}y-!RKb&wm(ok z;(ZAtw0c$IU>Mz6*+26)lkU&({aSbM9sZmB$Sby+iRls=;1L>GBHn<0x)F&T%&UB(bRd7W3q7id@H9aV5M8DhAKEVk0GnBqDCgP>V zh`JH7%A{MzYu@DKGO17Zb0C1KN7l?x=b* zU4MG2wCWf?a~OtEW^c!*vN&uFr_+xP#T=js=+}94Qn3D}nmP-pUUWDX!)4|z1sFGX zLSp~<!d)STj zqpNyJb>-5JRB{2*H}D6m$pV(;qU}vwnERA^%8ht`A0|tz{7LNbDVD8HaG0!?$pquF znEc8~AiM#^oZX^k0bsiKAGF4FrfD+NpPU2>TSIUgG>8^%BM9wlEeMRg4BIJ%blVw) z^aDTj%IoXrxPNzczkI~7@9; zr*Qz|vrmp6Zx=cn>N*%5KRNzI%UEsEA!{B7Lo31LJcY_3k41Zo%W%}F?Ln#gWG<&) zwg|g@9xD~oV4z{yQW*X-O^^)dFBG2XYWdedGT=_}YxnWoV<(ebq`3;S99cyDYwi2( zxnc{6(KH_qlB+DMIHv7;YLwp(jK5Czcualc>!RWHs(&@NWpf^12s1eun>b)DdUee%tdmTC%slNuI~zcI*pL2o zpf+$I^-(x~plb#W!i!3!OJPk!i$=Lb?3y_7Tl8ZV5j>jkjJzM?dv$*PG#ZKZ=yN;* z15bnx1D#}pC#kXzB9&#wM#z^i(=gR*Ja}?`%iyposdv<12>+bDBE56Rq`);JJ44PV zjc_%VvZQ)~mO-2BDPMS!>+E5Yxr{9Q^7D*dZ4!P@Yx40kuEI0h!!H1lMbM@g_C412uU&8O#+&yX5-l z5KpjIA&vA#YgGBf+ZR-fyHJQdsWKf>jK3Gebj|;6Qfo@*$?R?i>2BiYTfoc@L2_^g{_~`#%=r~iy2ksw{@1QL!FS&D_@wf7LMCTY&7P0Q-h-HAW6YOr z>U)}qLcDnW;Os@hI|FIMMa6GNj(0JY`h!Z4Y&d_vF;2)>zcSrLC!19=$63m~zr=Il zwqIYda1~6>_C$(-XFOfG$>e3XbkFlxh~M4q7|z!&p&n%iic;cYfu*~T|E7#nqdgk) zpmK`ddduZbC}UuvhAolT0X(=W(CJ{0zDzRrnmk2k}`RUVD5kdLLAH5lC>E z-NlZedi~s;|KN~c^WKLZ`OEf%gEMTQv6d;W9_GYFC(#!#m81Dh9}eE}`aVpxG}5bR zIvbMIc*6fkQcb-18~TrqLn@rO`_lq(h7V?YDT&)&@82+NccP#B)~g0TMG6^K-5CPy zkj|uwO^hOo^Si!^;J$LfG7!S}P=z&c#&I4)H|mI)@T*t;@vY$AGw!Am13Zdm`8VQ~})E$BZe?L`Js>IifHAk<`qoWNJB$K21$L)lyx%_A3 zhrAr;H1l0x-6xIn<+Jrz;Oq`$)cWLDj%E&qj*(jyP6ie)HjjfjO3gkMV8Rn77(g}@ zdPW;n<&OFx?6xhYcIfZy6*PnEwmrWj0 zO_sdD^)kU5UK_PXyENc(drkrYi|^xU3))R%NL^2wtHow-`79a>Rd%oZ`iF=zEL$iA~A_5U#S9pF&F|Nm#6 zO?I~I6)JlZvPp&PkP4BJbk}atNRfvAP7%wWnzYwA(xhw2nq3)EiXohk>L`TwSt(xh^N0UE#j7<-)yU&P z1nCC;2bY%Dte%CNZKO)d|tz}dA?f_0EgK>v9H{Z7~{h-I)YF$ z=1bAhU);R3g?ZCJ()@=)`;HR1=81%m3LUwhHtPF^=kKq zG~Hr+{BRUXTHbc8;L!0w4Ws)#2Mq2F^g*z9%0gpQ$6HI>I14{F*vnDs*E z(pvqmNl&)BLr3{@P+oiT+Pq%p0=WBkpwex2lma|^^E1mH`4{g8sjMe1qv)qKcg8zB z%{0F zQPQZEnE#DTC4HOvH^F_sgG{os~m#eBxc6tiN)L9JuOq=79^A=w-VcHCWNxkC0kbqS7XQ$0^@p~o|7 zr|(GOn&8{>mNsL+uSZy`9kKMKA^cvL-L30UsZYSe8kDKgp z?zF&yDRS=3h{bncEGcSg6Q>-vIK2mB)N=E2&=OY*qo61!f{9P!$T*pbXGs?mRtEx|fJrtp{op6^-4?-KK zc(Lch`H(;(RW^W@ci%(DIKSN%_xeH1_>hoCn2q>biIPUFud{p0;(w&19?TQUz@VkwXNXt@@?{XVa7-DklD@?m(`G(nMyRqHFpEwaGTroVCx|S(HrF&RJL|U+ELE#`du^|@%NKCThn$_DB_H)%fPWvf4Ym`$6#bBoH_ zRbXW*IJg}J3k-X69G;r7yP$-#kL#zuv;9>qX-+V3^1dmIZ^?o7ntR*Ln$C6C?)p?6 z9PT6U9_6#&552q&k0EXWGcy~P7YFdYNww^7m9s?%Oe5X}2oZ6re7uN?TrBl@sDQy% zQn2+Bs+6b72k`9u)w{&#_iI`^6noQN)(Mu`KRHJ4s@iMaxJ%|qefBp3cQG(qNjJ+a zP@T12;Y^Kk0CEo-CXA2&rf`yV;^+Id++;j3|X zd*s@rwH^xzbdBRrSl^MSO6|DQ5KfdC~node+4FnyEAojwKHXM+q{ zqo|mTkct?EazMq(VhR%!UJ*ia+=7Vd#|u%@c`R-dhdB2`e|)9?-Vca|lovn0<}5#GQ~ffoKFFmtj@; zH2O01g>ox-7Sa@o!bFdQ&4`d=KV!e_e|{uH7gDZ34sgW?j~ZWYUW48@8j&E6w_=U| zY(I(BnvdE%?sqA(CK>eCRV3`F5a3k_Em#x#u3H&8&|SXD0Z$Fdxu7S3g(oiuxlZE_ zeqI=V{kBpvL!&p39@4T9&I$@A#(~MghgdL8G4Dv@B_UWCmsCC}JmmFH73MH;68kFG zU2ptUV(COg?JPskx4fqB72*>c9vOnlzd|`0HlAk*u4ADvlGNv(xwY(Gp8J@EUm5HRSk}Fh&ekiut0sq|t{ESQ*FE zM+jKp$VZDR!Jk-%0F}V*U3^n=DYIJa52k<987LkO(Z7HG zo!h)oSsxfSp~vtMKjq*-q*9~1i1UvdQ#;#NY`)9~XLR)Qc3wdN88_O50uVV;X0 z>=t=>Bfg|aWZNEe&tgToi-8-RMYAee^5>4+u7lUyM*>RZdNSi%HovR|d1~$CK=#ls zrTF(LdE@!7>V_+Ql!8-VDcmB}E-DJ`*I9>r{AYC&Z}*UX<|U8L*vJZv#=VA#>xf!T z#H*2Vf_PbncX^RQR=fDl7ZWHG4Hd}MG+nFVHOi{nuW$&2{~3Y zA0!`L&W$O_pAZ0YG;|kxX`|r*KH&15?cPQ@`k*CHExgD!bVZ?8f7YpPlyzu{g`goe<`JHC-tUhyE zGI1s-JDO^W3fig>xpm|j`e*i;oitkQj_f=ndo<`$WFN;J4{yPG=;vxB`P+SK)Ofs`N|rayw|{Fs2Vn+mZmAXQqYni4|Z|@;|2DB`{I#0=H?ya(@eUlLg+jfJyywWvXKQpSj4TCN%CN_z&k*_N6IH+6`Lp>BDRYUxXwCRlzgrOBhKlG> zfCmo*)k!%79kK@8hpN=d;kxND*A2ll15~5GXslEAAfR8OBR}(^Y+aAN??zhJ?Y;6X zuV!DczBQ@C30GiV@W|?{_xxy-CA62r07*(Z&~)h36qB9MBY#>*1_aD=pR(3(AS;u8Gzjp82aLXQXUHvWQr+cpQ z@V9iv<9@bIZlSX!QZk0~M>)T4bFD?a3&MOBDfo;2vSH=7IKy`oFU3B~`wb60DvM5} zeo{hj&#aQ0PmXT;d4SSr5a}59r`Q)=3xOwr+Dr)t z$}9jWk$70BY=gc0W3h5A6y8MvWPnWn(0cc{dojnM!aeZw1eDX0xY8N~#+|QqDu9is zH~47wGymrq0U?P$L5Qb2j*Q*~0_7$Zi3Gc(6~}C)@YiB2qWH*T+{&A|+VKy({)Ikz z)Q#sf{uwwEkMl$0!7b_+dD%hQ@_gm*!*pBQ&5(>liW`<+4(Mpoix@0pITODlw=dsb*M0HY=-K1p8p`AqgZLHW z)0Vslj8OSjC`Osy{f9|T>9TdD&xGuaeB-X9TBx}1D+$D(qm8S2YH1-&_HSAAk3P&q z&TFBfSuJa0lf3TXwzYf1hOO%p>D?u}Zwr^y=llCh4Zq{O9N5_&-4E;g@Ot?;g9aC+ zK&|a$)%i1*q417QzLVD?^LtanhGpqfKki5|V*F-Dp{i> z?hJ4;5>_Cb67PbojNq#83;}g>6$kTJrEV7&M&D(?h-{OT#LA}Q?kH~pjKS&BOPB%k z@q}JeZ*P9mw#E$SEKl^I&D3?^3>+o?D9>CRPLr&fXi0Q_Z4?m8iySvxeFkJroU`}6 z1ej8esl(Bx-WikOz{~axKDoMw+BmK9DEGahdy=Ko1~0RCOFW-|!>>}8My2!^`>b&v zuiOBz^BIdvfoi(dmY3WdS9gy6)*?k%a9Y?toit`tX_H2>|9)d| zPqbbtGeRqBN7=!cqcqXf7yY-HI@(J~9j>Uv#Y8&4~70 z6Swkm5?rPf=hv)3uLwuQYi>av1|8o=clhtMQkVapx1jE`3{~G^{4JKiCwuC z{@9fHVaB$5rQ^igh^2;KTaVKSau^0`m7pt61-!UZGHy8QZS{!X#HS*5TSH)+_`Bw+ z3Jjnr1L=auSsx2{=f{4DAL>oJdfQ)v=6%Jz0{}E$5M(?sOOL$;4U#sI6DG%}Da5Sl zGW90`mM-(!We_N2VcAowih{Y*yC(LIN2ATLUSuQ>YJLTXZ~iFOXZ<0z;RX3s@R_rw zYUMEO+Yv>Rx%umNQ)dz411V|#jbvTp)_o_|OE2#>aP*R^&2votQZ;mL%oPE~kgG}t zZN1_!Sp53jlNul0*@(`^$D~pH=XGd4TE%$c!o0V|d(*wOiqgMEBT!Q^K~+Q7!MtGF6)hWAurQ#{=HxY7s3M=a{Vho`(nam zHf0bu9Xkk3;jo9jtfx>oy=B%ysve4&vAfVcV4o&B{oDnTfKYR$vi3DE{aCC-)*nb1 zl}zWO|M1nnPq+EYTd$9fw%FmagL+$kUvbW2i*UtwkK<$I`CQ{R*`CMK;nX|uYCxzV zKPlhqu_ISk(;QTxhAuJG?uEaotjp|LfGcb0qp1DioTNEI%!a*1A-tlyqmd-@jKdd$iY9XMfFx=c zLn@gm{GK>)z)yZ>OV(PRj&WJXOG!F23=sk<&^`_v!eD2xWZCkHodiRzDUl|mRWYmW zaSW*HJF+ne*oE1b0)foFlj9m$ZaW;Jb9GI4dm4Db#GH%!ZJFGn)AbSZk4H4v7+$~Y zwtSG84}sO*JOj2UT_igVa{K`0gm-orC3k)_6nF%YX*T&cS6F$fSkIjhuiSJOepaTo z`01^}VyuiTb`y5CGGd6NN)K$p{*Pv^>I}#Ua9QthVymbE))rL~yoEI3Xq7rDT4nWp zhuQCPG7dP`i_+$ctdKY5&y2ErdNbskw{kdUUisb8J$}nWZ^jR?Z-^uLs299XJos{} zmZHx%MhWO;@G3FHydoF@M5)L2cS*e6QBJr+zvO|a7Fn0czeAh%51gDt4~BN)7Dn|d zPygL&)Zmd(7K7-VdqcQ8ceIlWeTI2WVPuAD)JEHM;tqq48wa8zo-Cy>iVYI_ciK+Y z4j7uIC{UM59C9ieQGhZ!l=;{-HSf1e*5{ATw=cp<2#W0!S zOf~)$lQEUypyDLJ~ zqPVhRO=Nc-g_^!E?Z#l=qPtHK&2ozn@$VwL9O#N42;T489A=VG-en+)4vrLXntCl{PC}Xq`zE+&UAv0amD0JUBl}-XYa+E z9M8o2HkJTsqR`qKzkbs>7;`ZBAv=&$+oXAr&E{Q&%h+;-;jctXkF6wk=39U{19rhf zX8||OC%_ELqVtVb@$t?IVS9@}j*;_@ADWGtj;T1i#Bze>A??YK?uCVp?+@n{n`l-~Wq@CO<(rG7o_W8wy)Uqf$)82V+c^2O8n&TmBD*50c zu{49qxKtj{eKOK>V53t-pBw7`e$^>?H`G7D5EEM9n+^*gg-+44w=WV@^vPUHAe4L( zfY$WegJohQsuZfyQfVqZLA^V2vAOf|GgLS z!7e>uXXoZq+cQfbk7^V60$l8?D?^%LQ*5gD_`7yU4lWPZ#rLMAUMFZi0$-l4k}Yh) zf$J^czy~<%qLv_2CE&n{PU_XQzN+$=Dj-Hu4vP8&jEKAEJ4iKnl*?ibqnL55f5a^H z-@6xZA3IzO6G>)04iFiE&RJod%Voe72esg$Ok6fvJYHX4-~UgM)QmJpj>7HZ{PZMg z42~ZaF>n7`i@$Cp&D_Ut%n;Qf^6uPH7LiwHsomiKQQ6Gqw;l#r$V!O?+ zd~Ff(>^z%)1rXUL_e1@%vKB)_HylC+t6;6=Tt+$X8}5RRox&DTdyUpX-t55>JiUWk z9GC@KThF&Cr&~v2^ToyS6c=~jy9>Yn@jAqHwUO+tK`O$#4bh>>^zZYdw^M*ENQE4Q zshRi*YZY0H#Ns;LvCMr_l7+kb-tKte+P5MLZ{_pGpUPsk|2!a7ziNyV%Cp;lGVlmp zkj3t5;IN`@h=IP*wQF+0e?u?_x2u0jxk~#CZP++G@+?|7>lS}m&Wuou^el!Gu(g%5wb^RouhDR2)Kk__ zann}tE^VF_0Q{t{lS3+8CtFdYhuQH`Qm`>LQe+%&kEHE??U(-r`d7iX z`EFo4)5>B45qPk0)$R)1-4LACWPOABsh*J-x!&ZR?7V3cvap~2ib{G`4UX&+i_=RB)T7`(n%(dD~i@a0mmK51?jH3;#o zxqvh{tXsKvJJ+f5xj~2@|p99Wu+92O#IjW zm@+k0byAhfUnG#3twue#1Bc>Ns*^hvn!L%V#uWMSxJ2xg75Voe%L4yf!WCiP#_?wl zMN%>;k(*}%+D@}LD-(QZT)e(a%~d465YcZn!`97i<-_Cak?aQwZxEQihTVk> zv!ci2gJVCXfe^>dD6#b~-uG7-ZO`rJV7`)9WG=xiIbz|MDU_T2NNUOyk5%NH3_@ym zT?RGnpUV05X%Jx!v*nIml9#+fYu+ZVLNaG72rKPVCy72I;52hs4t`4zpJT;&Lp>3W=U&zaRpcgytH^Ir^BQiw72=TuClTeMTP0oN)z z^GW!q2UItLn2BHaNhxSWG_Tt0Z-Ab*+-@d*^QCm3Q-+w8@ta=+BZkthFOKv* z3Uhqgb#>De#qIrz-n(ZqJ+{|anzWmDB_|{oTQVtw#w)J$dKQ7>9Cu^yN6&8{9bfDl z?^Vo|U7Grx=WYcRalkXM2bz62h+Pyp{`0>)S^x)2XhFgUaB5*_D?Kvsm(Jx&yBUhT zb+uD1RPnL!ZD%kmT?JXODWi&o_PA9ffW%JH=)VLY zwKO;h$ny{nVu#K528!{E^~v{Vdld^&#hMT>XpPXw?z3k&M!S2Qr<1UA!B=bhgDt{V z1%E{@tjgD^NRd-JC)qTU0!Swx^eUyks7@0RsJB80^%(hFPdzpcr^&h;p^j;p_!%2y z(Y4w6&Hs!UAGxZT{U|*7G@9M<;PSf9jy^i;93@}a?tj;``AK2%Cn4@@xRscH2%0R- z?e`-p_bQ4Y(X;nCMt;-rCdvH^@%A1T@3BmwA97@Tn8vVYUSvZ5bf^{LP%;Zy;hpbp z(E(bgt@a35{ilSZeB}A}NIEjRiHXjMzjCM18YCpOH;6w}>_hIE5RLm<30l=BSVb`6 zw1BJC>&MFnDc!$PQ;rCMNbZ0v$wA>VK2nT)tC1js>Lx6zgBHkQq;B0&N zaow%gL#lnY^j5t{WYM$dnyoI6dg`<)?#~XahU`tw1xFpEOSHx{HZF#ZmpokOP|Wo4 zmWz%JL|U{Fe412hH$Clsnb~L3Mo@h!nj*cg0hv6)>8stU(z$ev3MLCn4Ecz=y02b_Qn#NH*1ks1ks2|C7?B zgaf3|8Nx1wJgUY44Q3~oS?Em;gX77b*fMJxRifj~7MjHZ^P_hWA9gs>0kNu{D4;-_ zF=hJTw0_XwWqBC;nx%JX-r5FjqTiK(m+c%El{YFp)nQ#$NV+bPjkv zAPQ;>6oV!-pnI$k>)Rm4!CB`A@HbOtg63M1|3S3lqIlJ(^ccU56a)x4s|;=-jl!oI1#M=E4}%xqOLgYWy4 z0r{%{8EB$GwWEhKiSpKtE9H4UHr{XU->V|06B)lT4kth?%6xn;*8QtMs&nDto{(&E z?;>@(6jjoXSy~D7{SRIpn;SMRwAZa*C+KIcFS_qVu`3_R^Z&aOsbo{bPA9T?KDkbu z<5m#LJT6L+$~U8JR<1?r8b`FdHNM>`?W@dhM#a85P{ZiTN~w>LQz-8oe1C9hCVy&U zCLy&h^=!}~(v|uy+A8=L0khbzTf|(t-+IIuzRAZ_2x`s`d}j1tx`9^3lb$zXd~r!M z=tb6S{$LGR*hUU&Ci;T>Aq;1$QJ=Oed5+}EN!#7|;3Y;04>pa#l(^~C{e`r>LJ=fz ztZj?EI+mNRy&f{=H#_7rZT4UCa)7`bAs{Q*dn^-8!SLAp^lm`QHUhKkKS4Do5agl7 z@6Fn|`JM%Fpb*F(!e3AlSW2U}wYY9C?GGAWV&xN~@;Ux5XC`WoBJ}t}8)u zTG#~w1BZ)viXIDz&!KDUfqF~*hh7F^&jEMC(*C4&8)-^NdFp-R<914e?aP^yxPm#Q zO3UB*Ki?3~I)Sth+UiFTh-1<$BTaU(vUw56#gXoqv0~n#-Y_^e6L4h^l-e&N#(CJ0cPq&rv%0;s0ob%d~M<kmSRO94Mm6@gt?tA zZ$(47DFY(yNBt+X6 z<8916sK14$Lh?3JOkxYc&?{iU^&@oMzgDH^G;fdhR5Vta(-JBWEC_Y)r4&2aIvGw> zDBKv>hLAM)?pdrI=aip4O)POmeCtQJsZh;{-8=>dSDuh};4_dh@Sk$Ba1XaXCvM&# zUtA)OTmPd_$k}R)yK;lS;#^AiP)W^qOGj+1rO5x*?+e@|FVPOuQyoSVKw}>6gm`@Q zYMuBkuG~k60j7KHDco5*!MXpe`7O_-lFCo~Va7k^3>)_@ZA_1^ z4GeCHHBokKzO&gCL@S=b`xcXs{_1WWP_eQ4_{g4#YEIO)JX zX@+DI><%&gx548o2t>n&jMK=aA;U&HhV-k2S1FRG z@h-A>*sK#1mf_vjRziBm$!p8YbWd$)Zt`qsB$&*E>#fw@&||Y0$sHt&tneIjWOg@d zJaYkU9Zk2#c{&y+;)>V58DVGoE~GQTar6x%F!V!#2?{Uco40OLwK=N51lkiPk9y*} z4kdJBB~vomf?LXo|8A+1DY?T6SnPN{l-PP^6OI!Qy;4Q^g=X&Ll$Qto=D26}Jy%Us z!|cZn(!bH~pB$8^R1YflrARkRyy4W?APB>n{m_Hihmu(`4I45ZsTcMSmpAZJED~x< zwH+q+Z2E}PWm0b7VM2ynq#J5q-^t6lU-v~P6r|J%swZ4K4nYK z`Y${?-D$lP_qFSMtyM;94P+(`cn9Qy^$i{d=r-W;0}kBPaROQs4S<-y;>OlIKuucx z9ak2?%?hj=PO7$-j(%5iCO@t0M)D@_kyT4nsdZKQ*=~fU*6H)f*_#CSCXaWTK5$=i z_jE-}D+IDPcxiY-lqNZ*QgE=RRTl3|ERtOWIS%rbNqE?F679kU6k{ahslmomp7Hmc zM;A^NZ*t8L`w6oqT0+!V+MENvf;Y>5`s7bKrI^Bp%*-fPuI&Et(DzzF7&u+z+-}+# zlwY_RUzuyO0Gonx0-mQs9;PE!7n}%khSlX0__tY8h6tN>#@p8Bx3at^; z52>x4IodoY3glQ0eQkP9{r$)DEwrhodEdpIe|_|SnBSa9s+#|KG=+pn@57gZY%q1*!$xTlRRGlaWAgB+u_^jyM+ zQ^u-&A+&9Oq*CA;eM>eAvI-lL8rzWi2s27|0o z?&^f