feat: ucanto invocation router with bare minimal store add invocation #4

vasco-santos · 2022-11-11T10:24:49Z

This PR adds ucanto invocation router with store/add invocation.

It integrates following AWS Resources in ApiStack:

S3 Bucket to store CAR files (carpark)
- we could have re-used the current bucket w3up-cars, but it is in wrong region, so let's just be coherent with CF naming and use carpark-prod-0 ? :)
Dynamo DB with store_table to track mapping between Users and stored CARs
- keeping same namings as current schema. We will need to perform a migration, so you might decide to rename things. Let's consider it later on new PRs

Notes:

previous services folder renamed into API, so that we can use service folder per ucanto framework current used names
uses hardcoded service DID, we need in follow up PR to wire it up with env var - also see authority.js + config.js in w3up repo
we add info to store_table like previously done in w3up repo. Probably worth considering as follow up to add a column that tracks wether the CAR already made it to the bucket. On S3 Put Event, we would update that boolean. Otherwise, we have things in the DB that we don't have immediate visibility if the given CARs ever made it

Follow up needs (will create issues):

handle account from store/add
wire up service DID with secrets

vasco-santos · 2022-11-14T13:53:48Z

api/test/functions/ucan-invocation-router.test.js

+})
+
+// TODO: Need to set ENV for dbEndpoint...
+test.skip('ucan-invocation-router', async (t) => {


@olizilla should we just drop this test to the lambda directly?

It is quite hard to inject the needed resources here... I would suggest follow up PR with more integration kind of tests, where we run the API Stack with all routes and perform tests using HTTP Calls.

package.json

vasco-santos · 2022-11-14T13:58:29Z

api/service/store/add.js

+      // Only use capability account for now to check if account is registered.
+      // This must change to access account/info!!
+      // We need to use https://github.com/web3-storage/w3protocol/blob/9d4b5bec1f0e870233b071ecb1c7a1e09189624b/packages/access/src/agent.js#L270
+      const account = capability.with


Will handle account/agent stuff in follow up PR

olizilla · 2022-11-14T14:26:02Z

stacks/api-stack.js

+      payloadCID: 'string',
+      applicationDID: 'string',
+      origin: 'string',
+      size: 'number',


is it size in bytes?

Yes, see https://github.com/web3-storage/w3protocol/pull/160/files

olizilla · 2022-11-14T14:27:14Z

stacks/api-stack.js

+   */
+   const storeTable = new Table(stack, 'store_table', {
+    fields: {
+      uploaderDID: 'string',


Is this the account DID or the agent DID?

camelCase?

olizilla · 2022-11-14T14:29:48Z

stacks/api-stack.js

+   * Force name of bucket to be "car-park-prod-0" in prod.
+   */
+  const ingestBucketCdkConfig =
+    stack.stage === 'prod' ? { cdk: { bucket: { bucketName: 'car-park-prod-0' } } } : {}


Some of our per env config is in config.js. We should try and be consistent

olizilla · 2022-11-14T14:31:15Z

stacks/api-stack.js

+  const ingestBucketCdkConfig =
+    stack.stage === 'prod' ? { cdk: { bucket: { bucketName: 'car-park-prod-0' } } } : {}
+
+  const carStore = new Bucket(stack, 'carStore', {


in this file we have

kebab-case

camelCase

snake_case

for resource ids. plz can we not.

olizilla · 2022-11-14T14:32:19Z

stacks/api-stack.js

+        permissions: [storeTable, carStore],
+        environment: {
+          STORE_TABLE_NAME: storeTable.tableName,
+          CAR_STORE_BUCKET_NAME: carStore.bucketName


Suggested change

CAR_STORE_BUCKET_NAME: carStore.bucketName

STORE_BUCKET_NAME: carStore.bucketName

olizilla · 2022-11-14T14:33:46Z

api/functions/ucan-invocation-router.js

+import * as CBOR from '@ucanto/transport/cbor'
+
+import getServiceDid from '../authority.js'
+import { create as createCarStore } from '../buckets/car-store.js'


let's make the export from car-store be createCarStore

olizilla · 2022-11-14T14:36:59Z

api/functions/ucan-invocation-router.js

+const AWS_REGION = process.env.AWS_REGION || 'us-west-2'
+
+/**
+ * AWS API Gateway handler for POST / with ucan invocation router.


Suggested change

* AWS API Gateway handler for POST / with ucan invocation router.

* AWS HTTP Gateway handler for POST / with ucan invocation router.

olizilla · 2022-11-14T14:41:22Z

api/functions/ucan-invocation-router.js

+    signingOptions: {
+      region: AWS_REGION,
+      secretAccessKey: AWS_SECRET_ACCESS_KEY,
+      accessKeyId: AWS_ACCESS_KEY_ID,
+      sessionToken: AWS_SESSION_TOKEN,
+      bucket: bucketName,
+    }


pass in a UrlSigner or similar to follow the pattern of holding the state in an object that we set in StoreTable and CarStore

olizilla · 2022-11-14T14:44:32Z

api/functions/ucan-invocation-router.js

+    }
+  })
+  const response = await server.request({
+    // @ts-ignore - type is Record<string, string|string[]|undefined>


Suggested change

// @ts-ignore - type is Record<string, string|string[]|undefined>

// @ts-expect-error - type is Record<string, string|string[]|undefined>

also i think we should review the type sig that ucanto server is using here. that seems like a reasonable thing to pass in.

olizilla · 2022-11-14T14:55:03Z

api/database/store.js

+   * Bind a link CID to an account
+   *
+   * @param {object} data
+   * @param {string} data.accountDID


is really string type?

olizilla

Good start. Lots of nitpicks that we can iterate on together. GOOOOO VASCO!

vasco-santos force-pushed the feat/post-handler-using-ucanto-server branch 14 times, most recently from e9e5124 to ac9e58c Compare November 11, 2022 17:19

vasco-santos marked this pull request as ready for review November 11, 2022 17:21

vasco-santos requested a review from olizilla November 11, 2022 17:22

vasco-santos force-pushed the feat/post-handler-using-ucanto-server branch 4 times, most recently from 7bdca7d to 09f69bd Compare November 14, 2022 13:58

vasco-santos commented Nov 14, 2022

View reviewed changes

vasco-santos force-pushed the feat/post-handler-using-ucanto-server branch from 09f69bd to 32ce987 Compare November 14, 2022 14:01

feat: ucanto invocation router with bare minimal store add invocation

85fe048

vasco-santos force-pushed the feat/post-handler-using-ucanto-server branch from 32ce987 to 85fe048 Compare November 14, 2022 14:12

olizilla reviewed Nov 14, 2022

View reviewed changes

olizilla approved these changes Nov 14, 2022

View reviewed changes

chore: address review comments

99348ef

vasco-santos merged commit 17e1fdf into main Nov 14, 2022

vasco-santos deleted the feat/post-handler-using-ucanto-server branch November 14, 2022 15:42

This was referenced Nov 14, 2022

Handle account integration with access on store/add #6

Closed

Wire up service DID with secrets #7

Closed

Appease typescript ignores #8

Closed

Full integration tests #9

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: ucanto invocation router with bare minimal store add invocation #4

feat: ucanto invocation router with bare minimal store add invocation #4

vasco-santos commented Nov 11, 2022 •

edited

Loading

vasco-santos Nov 14, 2022

olizilla Nov 14, 2022

vasco-santos Nov 14, 2022

olizilla Nov 14, 2022

vasco-santos Nov 14, 2022

olizilla Nov 14, 2022

olizilla Nov 14, 2022

olizilla Nov 14, 2022

olizilla Nov 14, 2022

olizilla Nov 14, 2022

olizilla Nov 14, 2022

olizilla Nov 14, 2022

olizilla Nov 14, 2022

olizilla Nov 14, 2022

olizilla left a comment

	CAR_STORE_BUCKET_NAME: carStore.bucketName
	STORE_BUCKET_NAME: carStore.bucketName

	* AWS API Gateway handler for POST / with ucan invocation router.
	* AWS HTTP Gateway handler for POST / with ucan invocation router.

	// @ts-ignore - type is Record<string, string\|string[]\|undefined>
	// @ts-expect-error - type is Record<string, string\|string[]\|undefined>

feat: ucanto invocation router with bare minimal store add invocation #4

feat: ucanto invocation router with bare minimal store add invocation #4

Conversation

vasco-santos commented Nov 11, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

olizilla left a comment

Choose a reason for hiding this comment

vasco-santos commented Nov 11, 2022 •

edited

Loading