Deno run with postinstall scripts

[nshiab]

Version:

• deno 2.0.0-rc.5 (release candidate, release, aarch64-apple-darwin)
• v8 12.9.202.13-rusty
• typescript 5.6.2

I would like to use the simple-data-analysis library with Deno.

The library relies on duckdb, which has postinstall scripts, from what I understand.

The documentation says this.

So I created a simple main.ts file.

import { SimpleDB } from "npm:simple-data-analysis";

const sdb = new SimpleDB();

And I ran deno run --node-modules-dir=auto main.ts.

I got this error.

Warning Trying to set up 'arrow2csv' bin for "/Users/naelshiab/Desktop/test/node_modules/.deno/apache-arrow@17.0.0/node_modules/apache-arrow", but the entry point "/Users/naelshiab/Desktop/test/node_modules/.deno/apache-arrow@17.0.0/node_modules/apache-arrow/bin/arrow2csv.cjs" doesn't exist.
Warning Following packages are deprecated:
┠─ npm:glob@7.2.3 (Glob versions prior to v9 are no longer supported)
┠─ npm:rimraf@3.0.2 (Rimraf versions prior to v4 are no longer supported)
┠─ npm:npmlog@5.0.1 (This package is no longer supported.)
┠─ npm:npmlog@6.0.2 (This package is no longer supported.)
┠─ npm:inflight@1.0.6 (This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.)
┠─ npm:are-we-there-yet@3.0.1 (This package is no longer supported.)
┠─ npm:are-we-there-yet@2.0.0 (This package is no longer supported.)
┠─ npm:gauge@3.0.2 (This package is no longer supported.)
┠─ npm:glob@8.1.0 (Glob versions prior to v9 are no longer supported)
┠─ npm:@npmcli/move-file@2.0.1 (This functionality has been moved to @npmcli/fs)
┗─ npm:gauge@4.0.4 (This package is no longer supported.)
Warning Following packages contained npm lifecycle scripts (preinstall/install/postinstall) that were not executed:
┠─ npm:duckdb@1.1.0
┃
┠─ This may cause the packages to not work correctly.
┗─ To run lifecycle scripts, use the `--allow-scripts` flag with `deno install`:
   deno install --allow-scripts=npm:duckdb@1.1.0
✅ Granted all env access.
┏ ⚠️  Deno requests read access to "/Users/naelshiab/Desktop/test/node_modules/.deno/duckdb@1.1.0/node_modules/duckdb/pack
✅ Granted all read access.
error: Uncaught (in promise) Error: Cannot find module '/Users/naelshiab/Desktop/test/node_modules/.deno/duckdb@1.1.0/node_modules/duckdb/lib/binding/duckdb.node'
Require stack:
- /Users/naelshiab/Desktop/test/node_modules/.deno/duckdb@1.1.0/node_modules/duckdb/lib/duckdb-binding.js
- /Users/naelshiab/Desktop/test/node_modules/.deno/duckdb@1.1.0/node_modules/duckdb/lib/duckdb.js
- /Users/naelshiab/Desktop/test/node_modules/.deno/duckdb@1.1.0/node_modules/duckdb/lib/duckdb.js
    at Function.Module._resolveFilename (node:module:622:15)
    at Function.Module._load (node:module:500:27)
    at Module.require (node:module:684:19)
    at require (node:module:808:16)
    at Object.<anonymous> (file:///Users/naelshiab/Desktop/test/node_modules/.deno/duckdb@1.1.0/node_modules/duckdb/lib/duckdb-binding.js:4:15)
    at Object.<anonymous> (file:///Users/naelshiab/Desktop/test/node_modules/.deno/duckdb@1.1.0/node_modules/duckdb/lib/duckdb-binding.js:7:4)
    at Module._compile (node:module:748:34)
    at Object.Module._extensions..js (node:module:767:10)
    at Module.load (node:module:665:32)
    at Function.Module._load (node:module:537:12)

There is a suggestion to run deno install --allow-scripts=npm:duckdb@1.1.0 npm:simple-data-analysis.

So I ran it, and everything looked fine.

Created deno.json configuration file.
Add npm:simple-data-analysis@3.9.0

I tried to run main.ts again with deno run main.ts, but it crashed again.

✅ Granted all env access.
┏ ⚠️  Deno requests read access to "/Users/naelshiab/Library/Caches/deno/npm/registry.npmjs.org/duckdb/1.1.0/package.json"
✅ Granted all read access.
error: Uncaught (in promise) Error: Cannot find module '/Users/naelshiab/Library/Caches/deno/npm/registry.npmjs.org/duckdb/1.1.0/lib/binding/duckdb.node'
Require stack:
- /Users/naelshiab/Library/Caches/deno/npm/registry.npmjs.org/duckdb/1.1.0/lib/duckdb-binding.js
- /Users/naelshiab/Library/Caches/deno/npm/registry.npmjs.org/duckdb/1.1.0/lib/duckdb.js
- /Users/naelshiab/Library/Caches/deno/npm/registry.npmjs.org/duckdb/1.1.0/lib/duckdb.js
    at Function.Module._resolveFilename (node:module:622:15)
    at Function.Module._load (node:module:500:27)
    at Module.require (node:module:684:19)
    at require (node:module:808:16)
    at Object.<anonymous> (file:///Users/naelshiab/Library/Caches/deno/npm/registry.npmjs.org/duckdb/1.1.0/lib/duckdb-binding.js:4:15)
    at Object.<anonymous> (file:///Users/naelshiab/Library/Caches/deno/npm/registry.npmjs.org/duckdb/1.1.0/lib/duckdb-binding.js:7:4)
    at Module._compile (node:module:748:34)
    at Object.Module._extensions..js (node:module:767:10)
    at Module.load (node:module:665:32)
    at Function.Module._load (node:module:537:12)

What am I doing wrong?

Thank you!

[bartlomieju]

DuckDB relies on native Node addons - unfortunately they are not currently supported with npm: specifiers. If you added simple-data-analysis to your deno.json and remove npm: specifier from your file everything should work correctly. See response below

Just last night we landed #25786 that prints a warning in situation like yours - if you upgrade to latest canary (deno upgrade canary) you should receive information guiding to the correct solution.

[lucacasonato]

@nshiab You can try deno run --node-modules-dir=auto --allow-scripts=npm:duckdb@1.1.0 main.ts or deno install --allow-scripts=npm:duckdb@1.1.0 npm:simple-data-analysis.

[nshiab]

DuckDB relies on native Node addons - unfortunately they are not currently supported with npm: specifiers. If you added simple-data-analysis to your deno.json and remove npm: specifier from your file everything should work correctly. See response below

Just last night we landed #25786 that prints a warning in situation like yours - if you upgrade to latest canary (deno upgrade canary) you should receive information guiding to the correct solution.

Thanks for the answer!

I ran deno upgrade canary.

I now have the version:

• deno 2.0.0-rc.5+8cdb309 (canary, release, aarch64-apple-darwin)
• v8 12.9.202.13-rusty
• typescript 5.6.2

So I followed the same steps.

It suggested again to run deno install --allow-scripts=npm:duckdb@1.0.0.

And I got the same crash when I ran deno run main.ts.

I guess I am still doing something wrong. :/

[nshiab]

@nshiab You can try deno run --node-modules-dir=auto --allow-scripts=npm:duckdb@1.1.0 main.ts or deno install --allow-scripts=npm:duckdb@1.1.0 npm:simple-data-analysis.

Thanks @lucacasonato !

When I run deno run --node-modules-dir=auto --allow-scripts=npm:duckdb@1.1.0 main.ts, I get this error:

error: unexpected argument '--allow-scripts' found

  tip: a similar argument exists: '--allow-write'
  tip: to pass '--allow-scripts' as a value, use '-- --allow-scripts'

Usage: deno run --node-modules-dir[=<MODE>] [SCRIPT_ARG]...

And when I run deno install --allow-scripts=npm:duckdb@1.1.0 npm:simple-data-analysis, everything looks fine:

Created deno.json configuration file.
Add npm:simple-data-analysis@3.9.1

But then it crashes when I try deno run main.ts or deno run --node-modules-dir=auto main.ts...

Thank you for your help.

[bartlomieju]

@nshiab please "start from scratch", ie. remove node_modules/, deno.json and deno.lock.

Then if you do:

$ deno install --node-modules-dir=auto --allow-scripts=npm:duckdb@1.0.0 npm:simple-data-analysis
$ deno run --node-modules-dir=auto -A main.ts

Everything should work fine.

[nathanwhit]

Yeah, I think the issue here is missing --node-modules-dir=auto in the deno install invocation. We should recommend that in our warning where it's necessary.

In general, though, I'd recommend putting "nodeModulesDir": "auto" into your deno.json instead of passing the flag each time - it's easy to forget to pass it.

Maybe we could add a warning if there's a node_modules dir already present when you run deno install and you're missing --node-modules-dir?

[nshiab]

@nshiab please "start from scratch", ie. remove node_modules/, deno.json and deno.lock.

Then if you do:

$ deno install --node-modules-dir=auto --allow-scripts=npm:duckdb@1.0.0 npm:simple-data-analysis
$ deno run --node-modules-dir=auto -A main.ts

Everything should work fine.

It works! Thanks 🙏 @bartlomieju !

Since you asked more questions, @nathanwhit, I'll let you decide whether to close this issue.

Thank you for all of your work. Deno 2 looks terrific. :)

[nathanwhit]

I think this PR will cover my concern well enough #25878. With that, we would warn about scripts not running on the install command missing --node-modules-dir=auto, so it would be clear something was amiss (instead of the lack of output making it seem like it was successful).