Very slow (4x slower than ts-node)

[brillout]

Bug description

Given that tsx uses esbuild I'd expect to be fast, really fast.

But, in my case, it's slow, prohibitively so.

I'm guessing that's because tsx doesn't externalize node_modules/. Am I correct with that assumption? In my experience esbuild is pretty much instantaneous most of the time. Yet, for a server that has only 50 LoC (see real-world reproduction), tsx takes more 12 seconds, whereas ts-node takes 3 seconds.

I love many things about tsx (especially its wonderful work on ESM-CJS compat) but its slowness is a blocker.

Strictly speaking, this isn't a bug, but I do consider it as a flaw big enough to justify the existence of this GitHub issue.

Thanks a lot for tsx, I just wish tsx would externalize node_modules/.

Reproduction

Real-world reproduction: https://github.com/brillout/tsx-slow.
Minimal reproduction: https://github.com/brillout/tsx-slow-minimal.

Environment

System:
    OS: Linux 4.14 Debian GNU/Linux 11 (bullseye) 11 (bullseye)
    CPU: (2) x64 Intel(R) Celeron(R) N4020 CPU @ 1.10GHz
    Memory: 311.07 MB / 3.72 GB
    Container: Yes
    Shell: 5.1.4 - /bin/bash
  Binaries:
    Node: 18.12.1 - ~/.nix-profile/bin/node
    Yarn: 1.22.19 - ~/.nix-profile/bin/yarn
    npm: 8.19.2 - ~/.nix-profile/bin/npm
    Watchman: 4.9.0 - /home/rom/.nix-profile/bin/watchman
  npmPackages:
    tsx: ^3.12.1 => 3.12.1

Can you contribute a fix?

• I’m interested in opening a pull request for this issue.

[privatenumber]

Thanks for the report.

I'm happy to investigate but your reproduction isn't minimal at all. Do you mind removing all irrelevant code?

I'm guessing the bottleneck is caused by createServer in Vite, so not even express is required for the reproduction.

[brillout]

Minimal reproduction: https://github.com/brillout/tsx-slow-minimal.

It really seems like tsx doesn't externalize node_modules. I wonder why? To improve ESM-CJS interoperability? I've some ideas about this.

[nickserv]

I feel like a lot of this processing is no longer necessary, considering all maintained LTS versions of Node have supported ESM natively for a while. Could we experiment with dropping all processing of CJS modules and node_modules?

[privatenumber]

tsx aims to support older versions of Node.js down to the latest v12, but I agree we can definitely make improvements on this.

I haven't had the time but I plan on testing esbuild's transformations to see which ones are necessary for which Node.js versions so I can start applying them more selectively.

[privatenumber]

This was actually not super slow for me to begin with, but it was 2.5x slower compared to ts-node.

In https://github.com/privatenumber/tsx/releases/tag/v4.0.0, we optimized compilation to be more selective by implementing ESM detection.

Hope this works well for you!

Benchmarks using Node.js v18.7.0

Before (tsx v3)

tsx was slower

$ hyperfine 'ts-node ./index.ts' 'tsx --no-cache ./index.ts'
Benchmark 1: ts-node ./index.ts
  Time (mean ± σ):     352.2 ms ±  34.5 ms    [User: 301.8 ms, System: 52.1 ms]
  Range (min … max):   333.7 ms … 437.8 ms    10 runs

  Warning: The first benchmarking run for this command was significantly slower than the rest (437.8 ms). This could be caused by (filesystem) caches that were not filled until after the first run. You should consider using the '--warmup' option to fill those caches before the actual benchmark. Alternatively, use the '--prepare' option to clear the caches before each timing run.

Benchmark 2: tsx --no-cache ./index.ts
  Time (mean ± σ):     871.7 ms ±  19.5 ms    [User: 826.7 ms, System: 147.5 ms]
  Range (min … max):   833.4 ms … 886.1 ms    10 runs

Summary
  ts-node ./index.ts ran
    2.47 ± 0.25 times faster than tsx --no-cache ./index.ts

After (tsx v4)

tsx is now slightly faster

$ hyperfine 'ts-node ./index.ts' 'tsx --no-cache ./index.ts'
Benchmark 1: ts-node ./index.ts
  Time (mean ± σ):     374.7 ms ±  29.9 ms    [User: 310.7 ms, System: 55.5 ms]
  Range (min … max):   342.3 ms … 431.7 ms    10 runs

Benchmark 2: tsx --no-cache ./index.ts
  Time (mean ± σ):     340.0 ms ±  26.6 ms    [User: 306.7 ms, System: 52.5 ms]
  Range (min … max):   319.4 ms … 407.7 ms    10 runs

Summary
  tsx --no-cache ./index.ts ran
    1.10 ± 0.12 times faster than ts-node ./index.ts

Compared to Node.js

Still a long way to go to get similar speeds to Node.js. Will rinvestigate this benchmark further.

$ hyperfine 'tsx ./index.ts' 'node index.js'
Benchmark 1: tsx ./index.ts
  Time (mean ± σ):     350.9 ms ±  39.8 ms    [User: 399.8 ms, System: 66.7 ms]
  Range (min … max):   313.3 ms … 446.4 ms    10 runs

Benchmark 2: node index.js
  Time (mean ± σ):      91.8 ms ±  11.8 ms    [User: 83.5 ms, System: 10.1 ms]
  Range (min … max):    83.8 ms … 134.4 ms    30 runs

Summary
  node index.js ran
    3.82 ± 0.66 times faster than tsx ./index.ts