Skip to content
This repository has been archived by the owner on Jan 9, 2025. It is now read-only.

feat: set component versions to 0.1.0-alpha #59

Merged
merged 3 commits into from
Feb 9, 2024

Conversation

jvallesm
Copy link
Collaborator

@jvallesm jvallesm commented Feb 7, 2024

Because

  • We need a way to introduce breaking changes in a sensible way.
  • Component page will need the version and release stage of components.

This commit

  • Adds a version field to the component definition. Initial version is
    0.1.0-alpha as we don't have a way to migrate components at the moment and
    thus we can't guarantee that we won't introduce more breaking changes.
  • Moves each component package to a v0 directory so more than one major
    version can coexist.
  • Introduces the source_url field in the component definition. This field is
    required by the component page and will be returned in the connector /
    operator definition list.

Notes

Adding the source_url field in the definition isn't great as it references a
particular branch (in the blob/main bit) and because writing it by hand
shouldn't be required. It would be great if we could just build it from the
component ID but several connectors (e.g. gcp or instill-model) don't match
the package and the ID. Also, there's no clean way to build this value and
inject it into the Load[Connector|Operator]Definitions function.

To make the definitions file easier to build and less error-prone, we should
introduce a linter and perhaps a boilerplate builder in the future.

🤖 Script to migrate to v0

#!/usr/bin/env bash

set -eo pipefail

# This script modifies the directory of each component and sets their version
# to 0.1.0-alpha. It automatically updates the references to that package.
#
# Usage: sh newversion.sh

for p in $(fd definitions.json | cut -d '/' -f1-2 | sort | uniq); do

  # Move component into v0 subdirectory
  mkdir v0
  mv $p/* v0
  mv v0 $p/v0

  # Update references
  escaped=$(echo $p | sed 's/\//\\\//g')
  grep -rl github.com/instill-ai/operator/$p | sort | uniq | xargs -I '{}' sed -i '' "s/\(github.com\/instill-ai\/operator\/$escaped\)/\1\/v0/g" {}

  # Add version and source URL to definition
  v=0.1.0-alpha
  url=github.com/instill-ai/operator/blob/main/$p/v0

  jq ".[0] += {version: \"$v\", source_url: \"$url\"}" $p/v0/config/definitions.json > def.json
  mv def.json $p/v0/config/definitions.json

done;

@jvallesm jvallesm self-assigned this Feb 7, 2024
Copy link

linear bot commented Feb 7, 2024

@jvallesm jvallesm marked this pull request as ready for review February 7, 2024 10:33
Copy link

codecov bot commented Feb 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (ae894f0) 67.93% compared to head (bf9c37f) 67.93%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #59   +/-   ##
=======================================
  Coverage   67.93%   67.93%           
=======================================
  Files           6        6           
  Lines         658      658           
=======================================
  Hits          447      447           
  Misses        168      168           
  Partials       43       43           
Flag Coverage Δ
unittests 67.93% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jvallesm
Copy link
Collaborator Author

jvallesm commented Feb 7, 2024

✅ QA

  • Created some pipelines with different operators & tested existing pipelines
  • Checked operator definition list:
curl --request GET \
     --url 'localhost:8080/vdp/v1beta/operator-definitions?view=VIEW_BASIC' \
     --header 'accept: application/json' \
  --header 'Authorization: Bearer $INSTILL_KEY'  | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3841    0  3841    0     0   136k      0 --:--:-- --:--:-- --:--:--  138k
{
  "operator_definitions": [
    {
      "name": "operator-definitions/base64",
      "uid": "3a836447-c211-4134-9cc5-ad45e1cc467e",
      "id": "base64",
      "title": "Base64",
      "documentation_url": "https://www.instill.tech/docs/latest/vdp/operators/base64",
      "icon": "Instill AI/base64.svg",
      "spec": null,
      "tombstone": false,
      "public": true,
      "custom": false,
      "source_url": "github.com/instill-ai/operator/blob/main/pkg/base64/v1",
      "version": "1.0.0-alpha",
      "tasks": [
        {
          "name": "TASK_ENCODE",
          "title": "Encode",
          "description": "Encode data into base64 string"
        },
        {
          "name": "TASK_DECODE",
          "title": "Decode",
          "description": "Decode the base64 string."
        }
      ]
    },
    {
      "name": "operator-definitions/start",
      "uid": "2ac8be70-0f7a-4b61-a33d-098b8acfa6f3",
      "id": "start",
      "title": "Start",
      "documentation_url": "https://www.instill.tech/docs/latest/vdp/operators/op-start",
      "icon": "Instill AI/start.svg",
      "spec": null,
      "tombstone": false,
      "public": true,
      "custom": false,
      "source_url": "github.com/instill-ai/operator/blob/main/pkg/start/v1",
      "version": "1.0.0-alpha",
      "tasks": [
        {
          "name": "TASK_START",
          "title": "Start",
          "description": ""
        }
      ]
    },
    ...
}

Include changes to display new definition fields in the VIEW_BASIC view
in the component definition list
@jvallesm jvallesm force-pushed the jvalles/ins-3620-add-version-to-existing-components branch from ec6c8d5 to 37a2346 Compare February 7, 2024 13:33
@jvallesm jvallesm changed the title feat: set component versions to 1.0.0-alpha feat: set component versions to 0.1.0-alpha Feb 8, 2024
Copy link
Member

@donch1989 donch1989 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jvallesm jvallesm merged commit 80df32e into main Feb 9, 2024
12 checks passed
@jvallesm jvallesm deleted the jvalles/ins-3620-add-version-to-existing-components branch February 9, 2024 08:47
donch1989 pushed a commit that referenced this pull request Feb 16, 2024
🤖 I have created a release *beep* *boop*
---


##
[0.8.0-beta](v0.7.0-beta...v0.8.0-beta)
(2024-02-16)


### Features

* implement jq task in JSON operator
([#61](#61))
([f38cd90](f38cd90))
* **json:** use `semi-structured/json` in TASK_MARSHAL and
TASK_UNMARSHAL ([#62](#62))
([acbe2db](acbe2db))
* set component versions to 0.1.0-alpha
([#59](#59))
([80df32e](80df32e))
* store icons next to the component definition
([#58](#58))
([9fea343](9fea343))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants