-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: change access-api wrangler.toml to be no_bundle=false #737
Conversation
Motivation: * latest commit on main branch was failing release workflow [here](https://github.com/web3-storage/w3protocol/actions/runs/3621669639/jobs/6105393187#step:2:35) * I think it's because the release-please config/manifest still referred to `package/wallet`, which this PR removes * seems like logical followup to #248 Co-authored-by: Hugo Dias <hugomrdias@gmail.com>
superseedes #139 <img width="897" alt="Screenshot 2022-12-06 at 16 47 14" src="https://user-images.githubusercontent.com/314190/205972386-05fec7b8-e17b-4a3d-8f18-ede71e4b5ad1.png"> closes #139
Goals: 1. Decouple store from agent 2. Simplify agent creation 3. Agent governs data format not store 4. Initialization of agent data, not store 5. DRY initialization in agent, not repeated in each store impl ~~See: https://gist.github.com/alanshaw/ea4bd2b0ab215ade696eac1300be577d~~ outdated ```js // In regular use: const store = new StoreIndexedDB() const data = await store.load() const agent = data ? Agent.from(data, { store }) : await Agent.create({}, { store }) // Then StoreMemory can be deleted, since the AgentData already stores all data in // memory. It's equivalent to: const agent = await Agent.create() ```
🤖 I have created a release *beep* *boop* --- ## [8.0.0](access-v7.0.2...access-v8.0.0) (2022-12-06) ### ⚠ BREAKING CHANGES * access-client store decoupling ([#228](#228)) * upgrade to `@ucanto/{interface,principal}`@^4.0.0 ([#238](#238)) * follow up on the capabilities extract ([#239](#239)) ### Features * **access-client:** cli and recover ([#207](#207)) ([720dafb](720dafb)) * follow up on the capabilities extract ([#239](#239)) ([717fcaa](717fcaa)) * Revert "feat!: upgrade to `@ucanto/{interface,principal}`@^4.0.0" ([#245](#245)) ([197439e](197439e)) * upgrade to `@ucanto/{interface,principal}`@^4.0.0 ([#238](#238)) ([309aff0](309aff0)) ### Bug Fixes * connection method is not async ([#222](#222)) ([7c4e0b6](7c4e0b6)) ### Code Refactoring * access-client store decoupling ([#228](#228)) ([64c7496](64c7496)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [3.0.0](upload-client-v2.1.0...upload-client-v3.0.0) (2022-12-06) ### ⚠ BREAKING CHANGES * upload/list items have a shards array property ([#229](#229)) * upgrade to `@ucanto/{interface,principal}`@^4.0.0 ([#238](#238)) * follow up on the capabilities extract ([#239](#239)) ### Features * **access-client:** cli and recover ([#207](#207)) ([720dafb](720dafb)) * follow up on the capabilities extract ([#239](#239)) ([717fcaa](717fcaa)) * Revert "feat!: upgrade to `@ucanto/{interface,principal}`@^4.0.0" ([#245](#245)) ([197439e](197439e)) * upgrade to `@ucanto/{interface,principal}`@^4.0.0 ([#238](#238)) ([309aff0](309aff0)) * upload/list items have a shards array property ([#229](#229)) ([723b281](723b281)) ### Bug Fixes * list response results return type ([#206](#206)) ([e49c685](e49c685)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [8.0.1](access-v8.0.0...access-v8.0.1) (2022-12-07) ### Bug Fixes * conf driver can store top level undefined ([5abc522](5abc522)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
…#246) Motivation: * second attempt at this #238 * after first was reverted #245 Warning: * the versions of ucanto we're upgrading to were backward-incompatible. storacha/ucanto#149 * Old 'SignerArchive' exports will probably no longer import using the new `@ucanto/principal@^4.0.0` `SignerArchive` * in slack, @alanshaw said he thinks this is unlikely to cause problems: "There’s no code in production that uses toArchive" Blockers * [x] Wait for these two to land first (@hugomrdias says so) * [x] @hugomrdias #207 * [x] @alanshaw #228 * [x] then will need to resolve merge conflicts on this branch * [x] @gobengo resolve conflicts after #207 * [x] @gobengo resolve conflicts after #228 Co-authored-by: Irakli Gozalishvili <contact@gozala.io>
🤖 I have created a release *beep* *boop* --- ## [9.0.0](access-v8.0.1...access-v9.0.0) (2022-12-07) ### ⚠ BREAKING CHANGES * upgrade access-api @ucanto/* and @ipld/dag-ucan major versions ([#246](#246)) ### Features * upgrade access-api @ucanto/* and @ipld/dag-ucan major versions ([#246](#246)) ([65d191c](65d191c)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [4.0.0](upload-client-v3.0.0...upload-client-v4.0.0) (2022-12-07) ### ⚠ BREAKING CHANGES * upgrade access-api @ucanto/* and @ipld/dag-ucan major versions ([#246](#246)) ### Features * upgrade access-api @ucanto/* and @ipld/dag-ucan major versions ([#246](#246)) ([65d191c](65d191c)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [5.0.0](upload-client-v4.0.0...upload-client-v5.0.0) (2022-12-07) ### ⚠ BREAKING CHANGES * upgrade access-api @ucanto/* and @ipld/dag-ucan major versions ([#246](#246)) * upload/list items have a shards array property ([#229](#229)) * upgrade to `@ucanto/{interface,principal}`@^4.0.0 ([#238](#238)) * follow up on the capabilities extract ([#239](#239)) * rename callback ([#198](#198)) ### Features * [#153](#153) ([#177](#177)) ([7fbf2a1](7fbf2a1)) * a base for a new web3.storage upload-client ([#141](#141)) ([72c08d7](72c08d7)) * **access-client:** cli and recover ([#207](#207)) ([720dafb](720dafb)) * account recover with email ([#149](#149)) ([91ad47d](91ad47d)) * add static uploads API ([#148](#148)) ([8b735fa](8b735fa)) * allow custom shard size ([#185](#185)) ([2130405](2130405)) * explicit resource ([#181](#181)) ([c127431](c127431)) * follow up on the capabilities extract ([#239](#239)) ([717fcaa](717fcaa)) * Revert "feat!: upgrade to `@ucanto/{interface,principal}`@^4.0.0" ([#245](#245)) ([197439e](197439e)) * support pagination ([#204](#204)) ([363e3ac](363e3ac)), closes [#201](#201) * upgrade access-api @ucanto/* and @ipld/dag-ucan major versions ([#246](#246)) ([65d191c](65d191c)) * upgrade to `@ucanto/{interface,principal}`@^4.0.0 ([#238](#238)) ([309aff0](309aff0)) * upload/list items have a shards array property ([#229](#229)) ([723b281](723b281)) ### Bug Fixes * export types ([c44a252](c44a252)) * list response results return type ([#206](#206)) ([e49c685](e49c685)) * store export ([#197](#197)) ([d2296c5](d2296c5)) * upload client list needs empty nb ([#194](#194)) ([7894fb9](7894fb9)) * uploadFile requires BlobLike not Blob ([56ec496](56ec496)) ### Code Refactoring * rename callback ([#198](#198)) ([2349763](2349763)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [2.0.0](capabilities-v1.0.0...capabilities-v2.0.0) (2022-12-07) ### ⚠ BREAKING CHANGES * upgrade access-api @ucanto/* and @ipld/dag-ucan major versions ([#246](#246)) * upgrade to `@ucanto/{interface,principal}`@^4.0.0 ([#238](#238)) * follow up on the capabilities extract ([#239](#239)) ### Features * **access-client:** cli and recover ([#207](#207)) ([720dafb](720dafb)) * follow up on the capabilities extract ([#239](#239)) ([717fcaa](717fcaa)) * Revert "feat!: upgrade to `@ucanto/{interface,principal}`@^4.0.0" ([#245](#245)) ([197439e](197439e)) * upgrade access-api @ucanto/* and @ipld/dag-ucan major versions ([#246](#246)) ([65d191c](65d191c)) * upgrade to `@ucanto/{interface,principal}`@^4.0.0 ([#238](#238)) ([309aff0](309aff0)) ### Bug Fixes * fix Access API cannot get space/info [#243](#243) ([#255](#255)) ([1a74031](1a74031)) * generated typdefs ([#258](#258)) ([dfd2bd5](dfd2bd5)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: Benjamin Goering <171782+gobengo@users.noreply.github.com>
Cannot put `undefined` in UCANs
🤖 I have created a release *beep* *boop* --- ## [9.0.1](access-v9.0.0...access-v9.0.1) (2022-12-08) ### Bug Fixes * validate agent name ([#271](#271)) ([5a1fdb2](5a1fdb2)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
I noticed earlier today that running `pnpm run docs` fails with a bunch of typescript errors unless you run `pnpm run build` first. This just updates the `docs` npm script to `pnpm run build && typedoc --out docs`, so we'll always have build output to make typedoc happy.
Buffer has a toJSON implementation which is called before our replacer function when serialising. It's JSON form is an object like `{ type: 'Buffer, data: number[] }`. This means it doesn't get identfied as $bytes, and does not get deserialised in the reviver. I think we'd rather have `Uint8Array` everywhere so this PR fixes it so that it gets revived as that. License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>
…r id (#275) Motivation: * #237 Notes * This is intended to be safe to merge/deploy as is. The new functionality can be opted-into via env vars * The ucanto server will only be constructed with an opts.id that is a signer `withDID(...)` iff `process.env.DID` is set to a DID * If `process.env.DID` is unset, the ucanto server id signer will have a `did()` corresponding to `config.PRIVATE_KEY` (it will be a `did:key` of the corresponding public key) * Before this PR, many tests of the access-api worker were configured implicitly via dotenv with whatever the `/.env.local` file happened to contain. I added a worker test that asserts functionality that is dependent on the details of the env vars, so I made an affordance for [tests to be able to configure the access-api worker with environment variables defined in the test case itself](https://github.com/web3-storage/w3protocol/pull/275/files#diff-e20ffc550d006747790e7b67da280446c1cd1d2d4f72ccb5df04f62cbb6423daR149).
- add methods toBytes and fromBytes useful to store as bytes instead of strings - `encodeDelegations` is now `delegationsToString`` - `decodeDelegations` is now `stringToDelegations`` - all functions are sync now Writing delegations to a car uses `@ipld/car/buffer-writer` now to make it sync, reading from a CAR it's more complicated 🤫. `ipld/car` does not support sync reading so this PR includes a refactor of `CarReader` to be sync, this is a temporary solution until i can get a proper PR to `ipld/car` to add support for sync reading. It's not trivial to do so because of the way ipld/car code is structured and i dont want to just add a sync copy/paste of the decoder functions to it.
The release workflow wasnt running the docs job because of an `if` clause leftover from the manual workflow.
By removing the side effect many methods that read data in memory no longer have to be async. I think this makes the API simplier. I've added a _method_ that can be called to clean up expired delegations, which shifts the responsibility onto the user. Alternatively we could move the clean up into the methods that write delegations? Open to other suggestions. resolves #219
needed by storacha/w3cli#20 to find which shards to also remove when removing an upload License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>
🤖 I have created a release *beep* *boop* --- ## [5.1.0](upload-client-v5.0.0...upload-client-v5.1.0) (2022-12-13) ### Features * sync encode/decode delegations ([#276](#276)) ([9d48372](9d48372)) * upload/remove returns removed item ([#289](#289)) ([472d6b6](472d6b6)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
- spaces.metadata is now nullable - d1 tables are now typed - moved from `workers-qb` to [`kysely`](https://github.com/koskimas/kysely) with [d1 dialect](https://github.com/aidenwallis/kysely-d1) closes #280
🤖 I have created a release *beep* *boop* --- ## [9.1.0](access-v9.0.1...access-v9.1.0) (2022-12-13) ### Features * sync encode/decode delegations ([#276](#276)) ([9d48372](9d48372)) ### Bug Fixes * destructured save ([#272](#272)) ([b6aaf9b](b6aaf9b)) * handle Buffer serialization ([#277](#277)) ([8e8f745](8e8f745)) * make d1 spaces.metadata nullable and change to kysely ([#284](#284)) ([7f09479](7f09479)), closes [#280](#280) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
This adds an About section with some context about the access-client and how it fits into the bigger picture. It also adds some detail to the Usage section, which now has subsections for creating Agents & Spaces and registering Spaces with the access service. Closes #251
🤖 I have created a release *beep* *boop* --- ## [2.1.0](capabilities-v2.0.0...capabilities-v2.1.0) (2022-12-13) ### Features * sync encode/decode delegations ([#276](#276)) ([9d48372](9d48372)) ### Bug Fixes * make d1 spaces.metadata nullable and change to kysely ([#284](#284)) ([7f09479](7f09479)), closes [#280](#280) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
…dProgress callback (#720) Use the ipfs-utils fetch implementation to enable upload progress reporting. Users of `upload-client` can now pass an `onUploadProgress` callback to `uploadFile` and `uploadDirectory` which will be handed off to the `fetch` polyfill from `ipfs-utils`. We wrap `onUploadProgress` in order to set `url` on the progress status events we get from `fetch` so that clients can differentiate between upload progress events from different shards, which will be interleaved.
🤖 I have created a release *beep* *boop* --- ## [8.3.0](upload-client-v8.2.0...upload-client-v8.3.0) (2023-04-05) ### Features * allow clients to pass uploadFile and uploadDirectory an onUploadProgress callback ([#720](#720)) ([20bc69f](20bc69f)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Motivation: * #623 * test theory that things will work if we upgrade to toucan-js@3, which is a rewrite that sends source maps in a new way
🤖 I have created a release *beep* *boop* --- ## 1.0.0 (2023-04-06) ### ⚠ BREAKING CHANGES * add did mailto package, replacing `createDidMailtoFromEmail` ([#722](#722)) ### Features * add did mailto package, replacing `createDidMailtoFromEmail` ([#722](#722)) ([5a4123f](5a4123f)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [12.0.0](access-v11.2.0...access-v12.0.0) (2023-04-06) ### ⚠ BREAKING CHANGES * add did mailto package, replacing `createDidMailtoFromEmail` ([#722](#722)) ### Features * add did mailto package, replacing `createDidMailtoFromEmail` ([#722](#722)) ([5a4123f](5a4123f)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [5.5.1](w3up-client-v5.5.0...w3up-client-v5.5.1) (2023-04-06) ### Bug Fixes * add newline to space.js to test release-please 694 fix ([#696](#696)) ([6bc4dc3](6bc4dc3)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: Travis Vachon <travis.vachon@protocol.ai>
🤖 I have created a release *beep* *boop* --- ## [6.0.0](access-api-v5.2.1...access-api-v6.0.0) (2023-04-06) ### ⚠ BREAKING CHANGES * add did mailto package, replacing `createDidMailtoFromEmail` ([#722](#722)) ### Features * add did mailto package, replacing `createDidMailtoFromEmail` ([#722](#722)) ([5a4123f](5a4123f)) ### Bug Fixes * bug where did:mailto with pct-encoded email parts was not decoded correctly ([#719](#719)) ([c367639](c367639)) * upgrade toucan-js and @web3-storage/worker-utils ([#728](#728)) ([853559d](853559d)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: Benjamin Goering <171782+gobengo@users.noreply.github.com>
) Motivation: * #623 (comment) * this was suggested by toucan-js author as no longer being needed in 3.0.0.
Motivation: * #623 - see if errors from this show up different than errors from ucanto HandlerExceptionError
packages/access-api/wrangler.toml
Outdated
@@ -23,20 +17,22 @@ preview_id = "62e57652625c44a3b1fef2f840ffc882" | |||
[[d1_databases]] | |||
binding = "__D1_BETA__" | |||
database_name = "spaces-dev" | |||
database_id = "7c676e0c-b9e7-4711-97c8-7b1c8eb229ae" | |||
database_id = "e3331d3b-d8dd-48b0-ab80-2dba93c2c256" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
new db because the old one is quite out of date, has no important data, and got hung up on migrations. No one has been using dev
afaik so I think this should be safe.
packages/access-api/wrangler.toml
Outdated
|
||
[[r2_buckets]] | ||
binding = "DELEGATIONS_BUCKET" | ||
bucket_name = "w3up-delegations-dev-0" | ||
preview_bucket_name = "w3up-delegations-dev-0" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wrangler asked me to do this when deploy to dev env
packages/access-api/wrangler.toml
Outdated
|
||
[vars] | ||
ENV = "dev" | ||
DEBUG = "true" | ||
DID = "did:web:local.web3.storage" | ||
PRIVATE_KEY="MgCYWjE6vp0cn3amPan2xPO+f6EZ3I+KwuN1w2vx57vpJ9O0Bn4ci4jn8itwc121ujm7lDHkCW24LuKfZwIdmsifVysY=" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is from ../../.env.tpl
so already public
I can't think of any downsides. It would accomplish:
downside
Sanity check
|
1d27b25
to
effe0ff
Compare
I force pushed to this source branch and deleted it because I accidentally pushed a commit containing a sentry dsn I used to test. I disabled the DSN in sentry admin, so it's now useless. I followed these docs to delete using It should be impossible to find the accidentally pushed DSN, and if it is found, it should be useless |
Motivation:
no_bundle=true
along with d1.Experiment
release
calledbengo-dev-0
sentry errors hereGET /.debug/error
no_bundle=false
(or omittingno_bundle
) is incompatible with wrangler d1 bindingsFindings:
access/authorize
handler. That worked fine, and I got an email from dev env. The error happened when I clicked the email (invokingaccess/confirm
), when theaccess/confirm
invoacation handler tried to write to d1dev
doesn't have latest migrations applied that created that table. So I provisioned a new d1 database for this dev env, and rannpx wrangler --env=dev d1 migrations apply __D1_BETA__
. This would error every couple migrations, but if I re-ran it it would make progress and error again (Internal error [code: 7501]
), but eventually it would succeed at all migrations.access/claim
(reads from D1), and got no error all the way through invoking from second deviceNext steps:
./scripts/cli.js
e.g. to run migrations, so I probably need to modifycli.js
to still work in tests at least)1.0.0
always. I think I can come up with something better than that quickly tomorrow