feat: persist ucan invocation car to s3 and replicate to r2 #83
Conversation
View stack outputs
|
f663ccf to
428bf65
Compare
428bf65 to
f4a9b03
Compare
Stack outputs updated
|
f4a9b03 to
bbfa51e
Compare
Stack outputs updated
|
| /** | ||
| * @param {import('@serverless-stack/resources').StackContext} properties | ||
| */ | ||
| export function UcanInvocationStack({ stack, app }) { |
There was a problem hiding this comment.
minor: perhaps this is the ucan-archive-stack ?
| export function UcanInvocationStack({ stack, app }) { | |
| export function UcanArchiveStack({ stack, app }) { |
There was a problem hiding this comment.
it is not archive in reality. We do other kind of post processing on it. I like invocation because of being a spec term https://github.com/ucan-wg/spec#29-invocation that is what we are doing.
|
|
||
| const eventBusName = 'event-bus-arn' | ||
|
|
||
| test('notifies event bus when an .idx file is added to the satnav bucket', async t => { |
| // @ts-expect-error different type interface in AWS expected request | ||
| await persistUcanInvocation(request, ucanStore) | ||
|
|
||
| const requestCar = await CAR.codec.decode(request.body) |
There was a problem hiding this comment.
do we need to test we can round trip the ucan by fetching it from the bucket and parsing it? there is a base64 decode step that gives me pause
There was a problem hiding this comment.
yeah adding. That is the aws base64 thing, so adding tests as I agree we should validate things
upload-api/buckets/ucan-store.js
Outdated
| put: async (carCid, bytes) => { | ||
| const putCmd = new PutObjectCommand({ | ||
| Bucket: bucketName, | ||
| Key: carCid, |
There was a problem hiding this comment.
maybe we should match the carpark keys here and /${carCid}/${carCid}
|
#86 (review) handled in 0d55985 |
|
The latest discussion landed on "store the invocation CAR as is and store keys for the block cids that point to the car they are in like @alanshaw @vasco-santos what do you think? |
|
This also sounds like something we could add in later if storing the invocations as cars is useful anyway |
| t.is(s3Response.$metadata.httpStatusCode, 200) | ||
|
|
||
| // @ts-expect-error AWS types with readable stream | ||
| const bytes = (await s3Response.Body.toArray())[0] |
There was a problem hiding this comment.
What is this wild magic? Why would a toArray method stick all the bytes in the first element of an array
There was a problem hiding this comment.
https://nodejs.org/api/stream.html#readabletoarrayoptions
It is kind of the easier way of obtaining the contents of a stream. And for a test purpose seems good
Per last night call, persisting CAR files of UCAN invocations handled by the service into S3 with replication to R2.
Implementation details:
TODO: