New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add WASM Transform Proto and Endpoints for Programmatic API #1011

Merged

weeco merged 44 commits into master from add-wasm-transform-proto

Feb 21, 2024

Contributor

gene-redpanda commented Jan 18, 2024 •

edited

Loading

Adds the following:

ability to deploy WASM binaries to RP using the standard api endpoints with multipart form
grpc defined list/get/delete endpoints for managing transforms on the RP cluster
tests for the above
enabled admin api in integration test
added support for enabling transforms to testcontainers-go
openapi docs integration

gene-redpanda changed the title ~~Add wasm transform proto~~ DRAFT: Add WASM Transform Proto and Endpoints for Programmatic API

gene-redpanda force-pushed the add-wasm-transform-proto branch from fe682b9 to ea33d33 Compare

January 20, 2024 00:02

bojand reviewed

View reviewed changes

backend/pkg/api/connect/service/transform/mapper.go Outdated Show resolved Hide resolved

bojand reviewed

View reviewed changes

backend/pkg/api/connect/service/transform/service.go Show resolved Hide resolved

bojand reviewed

View reviewed changes

backend/pkg/api/routes.go Outdated Show resolved Hide resolved

bojand reviewed

View reviewed changes

proto/redpanda/api/dataplane/v1alpha1/transform.proto Show resolved Hide resolved

bojand reviewed

View reviewed changes

proto/redpanda/api/dataplane/v1alpha1/transform.proto Outdated Show resolved Hide resolved

rockwotj reviewed

View reviewed changes

Contributor

rockwotj left a comment

Looks pretty good to me. Couple of small bits

backend/pkg/api/connect/service/transform/mapper.go Outdated Show resolved Hide resolved

backend/pkg/api/handle_transforms.go Outdated Show resolved Hide resolved

gene-redpanda mentioned this pull request

Check for valid magic bytes in wasm file uploads in RPK redpanda-data/redpanda#16230

Closed

gene-redpanda force-pushed the add-wasm-transform-proto branch 2 times, most recently from 3029295 to ecf691b Compare

January 25, 2024 15:23

bojand reviewed

View reviewed changes

backend/pkg/api/connect/integration/transform_test.go Outdated Show resolved Hide resolved

bojand reviewed

View reviewed changes

backend/pkg/api/connect/integration/transform_test.go Outdated Show resolved Hide resolved

bojand reviewed

View reviewed changes

backend/pkg/api/connect/integration/transform_test.go Show resolved Hide resolved

bojand reviewed

View reviewed changes

proto/redpanda/api/dataplane/v1alpha1/transform.proto Outdated Show resolved Hide resolved

gene-redpanda force-pushed the add-wasm-transform-proto branch 2 times, most recently from 675c72f to 3614f74 Compare

January 31, 2024 09:00

gene-redpanda added a commit that referenced this pull request


          fix: remove unneeded code

9d28fc1

see #1011 (comment)

gene-redpanda force-pushed the add-wasm-transform-proto branch 6 times, most recently from 173c106 to 47ebad5 Compare

February 5, 2024 21:58

gene-redpanda changed the title ~~DRAFT: Add WASM Transform Proto and Endpoints for Programmatic API~~ Add WASM Transform Proto and Endpoints for Programmatic API

gene-redpanda marked this pull request as ready for review

February 5, 2024 22:14

gene-redpanda force-pushed the add-wasm-transform-proto branch 3 times, most recently from e3a8ad4 to 8d59fd8 Compare

February 5, 2024 23:19

gene-redpanda requested a review from bojand

February 6, 2024 16:16

gene-redpanda and others added 25 commits

February 20, 2024 22:32


          fix: verify and rebase

8594e69


          backend: move http handler to transform pkg

d9b85b7


          proto: add message for deploy transform request

fb07600

We'll use the prototype to achieve better
maintainability and strict validation.


          chore: compile protos

118091c


          chore: fix proto validation for env key

bbb665a


          chore: compile protos

b3398cf


          proto: rename fields for consistency

05ddac6


          chore: compile protos

8859ccd


          backend: add const admin api not configured err

45e43e9


          backend: update integration tests

3581be1


          backend: use proto type for request

bd17a27

Add validation based on proto type too


          backend: fix lint issues

f6883ca


          backend: change names in openapi spec

e0c2c47


          backend: address PR comments

520847b


          backend: change unescaping behaviour in gateway

e75fd7e


          backend: add test for escaped input

e5f72d1


          backend: add env vars to get operations

94106ed


          chore: generate protos

c042005


          backend: add env vars to response

01afc32


          chore: re-regenerate deps and protos after merge

b8cceba


          backend: restrict chars for transform names

4ef8258


          backend: generate openapi

317562e


          backend: document deploy transform endpoint

e307a3c


          backend: add 404 http test for get transforms

70d529b


          backend: add documentation for handle deploy wasm

65590a8

weeco force-pushed the add-wasm-transform-proto branch from 2b908ec to 65590a8 Compare

February 20, 2024 22:34

weeco added 2 commits

February 20, 2024 23:25


          chore: ignore whitespace in middle for git diff

efeed4f


          backend: fix http method for transforms doc

2a50527

weeco merged commit fab9e0e into master

6 checks passed

weeco deleted the add-wasm-transform-proto branch

February 21, 2024 09:56

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet