Skip to content
This repository has been archived by the owner on Jan 9, 2025. It is now read-only.

Commit

Permalink
feat: add read task for gcs (#155)
Browse files Browse the repository at this point in the history
Because

- we do not have read objects and create bucket task for gcs

This commit

- add read task for gcs
- add create bucket task for gcs
  • Loading branch information
chuang8511 authored Jun 24, 2024
1 parent ab77d10 commit 77fe2fc
Show file tree
Hide file tree
Showing 8 changed files with 736 additions and 25 deletions.
46 changes: 45 additions & 1 deletion data/googlecloudstorage/v0/README.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ The Google Cloud Storage component is a data component that allows users to uplo
It can carry out the following tasks:

- [Upload](#upload)
- [Read Objects](#read-objects)
- [Create Bucket](#create-bucket)

## Release Stage

Expand All @@ -22,7 +24,6 @@ The component configuration is defined and maintained [here](https://github.com/

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| Bucket Name (required) | `bucket-name` | string | Name of the bucket to be used for object storage. |
| JSON Key File contents (required) | `json-key` | string | Contents of the JSON key file with access to the bucket. |

## Supported Tasks
Expand All @@ -34,6 +35,7 @@ Upload data to Google Cloud Storage.
| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_UPLOAD` |
| Bucket Name (required) | `bucket-name` | string | Name of the bucket to be used for object storage. |
| Object Name (required) | `object-name` | string | The name of the object to be created |
| Data (required) | `data` | string | The data to be saved in the object |

Expand All @@ -44,3 +46,45 @@ Upload data to Google Cloud Storage.
| Public Access (optional) | `public-access` | boolean | Whether the object is publicly accessible |
| Public URL (optional) | `public-url` | string | Anyone with this link can access the object on the public Internet |
| Upload Status | `status` | string | Status of the upload operation |

### Read Objects

Read objects from Google Cloud Storage.

| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_READ_OBJECTS` |
| Bucket Name (required) | `bucket-name` | string | Name of the bucket to be used for object storage. |
| Delimiter | `delimiter` | string | The delimiter to use when listing objects |
| Prefix | `prefix` | string | The prefix to use when listing objects |
| Versions | `versions` | boolean | Whether to list all versions of an object |
| Start Offset | `start-offset` | string | The offset to start listing objects from |
| End Offset | `end-offset` | string | The offset to stop listing objects at |
| Include Trailing Delimiter | `include-trailing-delimiter` | boolean | Whether to include the delimiter in the object name |
| Match Glob | `match-glob` | string | Whether to match objects using a glob pattern |
| Include Folders As Prefixes | `include-folders-as-prefixes` | boolean | Whether to include folders as prefixes |

| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Audio Objects (optional) | `audio-objects` | array[object] | The audio objects in the bucket |
| Document Objects (optional) | `document-objects` | array[object] | The document objects in the bucket |
| Image Objects (optional) | `image-objects` | array[object] | The image objects in the bucket |
| Text Objects (optional) | `text-objects` | array[object] | The text objects in the bucket |
| Video Objects (optional) | `video-objects` | array[object] | The video objects in the bucket |

### Create Bucket

Read objects from Google Cloud Storage.

| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_CREATE_BUCKET` |
| Bucket Name (required) | `bucket-name` | string | Name of the bucket to be used for object storage. |
| Project ID (required) | `project-id` | string | The project ID to create the bucket in |
| Location | `location` | string | The location to create the bucket in |

| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Result (optional) | `result` | string | The result of the operation |
| Bucket Name (optional) | `name` | string | The name of the bucket |
| Location (optional) | `location` | string | The location of the bucket |
4 changes: 3 additions & 1 deletion data/googlecloudstorage/v0/config/definition.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
{
"availableTasks": [
"TASK_UPLOAD"
"TASK_UPLOAD",
"TASK_READ_OBJECTS",
"TASK_CREATE_BUCKET"
],
"custom": false,
"documentationUrl": "https://www.instill.tech/docs/component/data/googlecloudstorage",
Expand Down
19 changes: 2 additions & 17 deletions data/googlecloudstorage/v0/config/setup.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,6 @@
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"properties": {
"bucket-name": {
"description": "Name of the bucket to be used for object storage.",
"instillUpstreamTypes": [
"value"
],
"instillAcceptFormats": [
"string"
],
"instillSecret": false,
"instillUIOrder": 0,
"title": "Bucket Name",
"type": "string"
},
"json-key": {
"description": "Contents of the JSON key file with access to the bucket.",
"instillUpstreamTypes": [
Expand All @@ -31,12 +18,10 @@
}
},
"required": [
"json-key",
"bucket-name"
"json-key"
],
"instillEditOnNodeFields": [
"json-key",
"bucket-name"
"json-key"
],
"title": "Google Cloud Storage Connection",
"type": "object"
Expand Down
Loading

0 comments on commit 77fe2fc

Please sign in to comment.