Esbuild clear output directory

[rctneil]

How can I get the output directory to be cleared when recompiling takes place?

[dhh]

You can call the javascript:clobber task to clear the output directory. No option available to have that happen automatically on every recompile. Presumably your outputs are stable though?

[rctneil]

@dhh, Sorry, I'm not clear on what you mean by "outputs are stable"?

I thought, that each time I save a js file in this gem or a css file in the cssbundling-rails gem, that the files in the output directory were updated?

I find sometimes that i'm getting a lot of leftover old files in the build directory.

Also, even though I have the output directory ignored, my assets are failing when being deployed and i'm not seeing the latest JS and CSS. Any suggestions on that one?

[brenogazzola]

Are tou having problems with the build (app/assets/build) or the output (public/assets) directory?

Files in the build directory do not have a digest, so every change overwrites the previous version, and there should be mo left over.

In the output directory, files are only placed there when the precompile task is run. This task does not automatically removes old files. That’s intentional. To keep that folder clean there are two extra tasks. “clobber” removes all files in the output. “clean” keeps up to 3 versions besides the current one, so that if anything still needs those (cached assets, server that haven’t been rebooted, lazy imports, etc) they don’t get 404s.

So, if you want to keep the output clean, you must do “clean precompile” or “clobber precompile”

[rctneil]

@brenogazzola Well i'm not quite sure.

I've just deployed a new version of my app via Capistrano and connected via SSH and ran RAILS_ENV=production bundle exec rails assets:precompile

The output to that was:

yarn install v1.22.4
[1/4] Resolving packages...
success Already up-to-date.
Done in 0.29s.
yarn run v1.22.4
$ esbuild app/javascript/*.* --bundle --splitting --format=esm --sourcemap --outdir=app/assets/builds

  app/assets/builds/calendar-PG4BZTFI.js                                   368.4kb
  app/assets/builds/chunk-6RNAEUJW.js                                      218.2kb
  app/assets/builds/plyr.min-ITQ4QADP.js                                   149.7kb
  app/assets/builds/chunk-ZN6BNXGE.js                                      123.2kb
  app/assets/builds/chunk-6VS5KWER.js                                       85.2kb
  app/assets/builds/slimselect.min-ZDOA4PIK.js                              46.1kb
  app/assets/builds/chunk-65MAXTY3.js                                       35.1kb
  app/assets/builds/application.js                                          25.5kb
  app/assets/builds/markerclustererplus.esm-4MVQDGG3.js                     22.6kb
  app/assets/builds/calendar-FHHVBCXK.css                                   20.7kb
  app/assets/builds/application.css                                         20.6kb
  app/assets/builds/layout_gallery-GT2UINRG.js                              13.8kb
  app/assets/builds/chunk-HYEUH2XI.js                                       12.7kb
  app/assets/builds/index.esm-3LMY7TPX.js                                   12.2kb
  app/assets/builds/index.esm-XVTUGHBB.js                                    6.3kb
  app/assets/builds/filterChange-YFMMYPNZ.js                                 3.9kb
  app/assets/builds/esm-MIWVOD6C.js                                          3.8kb
  app/assets/builds/chunk-PBR5S63H.js                                        1.5kb
  app/assets/builds/chunk-RPC23XZB.js                                        1.3kb
  app/assets/builds/gallery_photo_click-7KM7RJLG.js                          1.1kb
  app/assets/builds/initialise_new_in_ridden_in_year_checkbox-RFZFVUYY.js    949b
  app/assets/builds/get_query_string_parameters-PP2NW5C4.js                  750b
  app/assets/builds/photoswipe_parse_hash-EMYQWIKD.js                        653b
  app/assets/builds/chunk-GHEFBBSR.js                                        427b
  app/assets/builds/set_target_row_height-NCFAK7HM.js                        400b
  app/assets/builds/chunk-SZKNGM2F.js                                        376b
  app/assets/builds/updateGrid-AVFB2SKN.js                                   316b
  ...and 49 more output files...

Done in 0.79s.
yarn install v1.22.4
[1/4] Resolving packages...
success Already up-to-date.
Done in 0.29s.
yarn run v1.22.4
$ sass ./app/assets/stylesheets/application.sass.scss:./app/assets/builds/application-styles.css ./app/assets/stylesheets/angry.sass.scss:./app/assets/builds/angry.css --no-source-map --load-path=node_modules
Done in 3.86s.

But i'm not seeing the latest assets on the site. If you view the site at https://neiltonge.co.uk, if you look at the very very bottom left of the homepage after scrolling all the way down, you'll see a new "Back to top" unstyled link. If the assets were correct then, this link would be styled and somewhere on the right hand side.

[brenogazzola]

Silly question: did you restart the server? Propshaft caches the manifest file so it takes a restart for it to know there are new files.

[brenogazzola]

Ah, one more thing: do NOT delete the build directory. It MUST be commited to the repository and must exist in the server before the precompile. Otherwise you get some weird behavior where files are not copied from build to output.

[dhh]

@brenogazzola We should probably have a check on whether the build directory exists and raise a proper exception if it's not.

[rctneil]

@brenogazzola I didn't restart the server. I am using Sprockets by the way. Eg, I've not manually switched to Propshaft yet.

My builds folder was being gitignored. I have readded it now and my styles are now working, HOWEVER, my Javascript has now all died. Any suggestions on how to resolve that?

I feel a bit more documentation would be very helpful too. Especially for people like me who are not 100% sure how things all integrate. I do understand docs take time after your comment about this last time @brenogazzola so i realise they will come over time.

[brenogazzola]

@rctneil Before you adopted this gem, did you precompile without restarting and it worked? I'm checking through Sprockets and in production it uses a cached environment, so it should require a restart for new assets to be picked up.

Could you give me more details about what exactly is happening to the javascript? Do you see an error? Is it not being copied to the public/assets folder? Is it not being generated in the build directory? Is the browser giving you 404 errors when trying to fetch it? Also is the problem in your local machine or production server (if production, is it Heroku or your own)?

@dhh Should we add this to jsbundling/cssbundling which are the gems that write to build or to sprockets/propshaft which are the gems that read from build?

[brenogazzola]

@rctneil As for documentation, I'm taking a look at the asset pipeline guide, and thinking of how to update it to better explain the new gems. If you have a list of questions and problems you have/had, that would help with updating it. Some stuff I wouldn't think to add because I'm too used to doing them or having the build system do automatically (like Heroku)

[rctneil]

@brenogazzola Before the gem, I honestly cannot remember.

The builds folder looks fine in production and the public assets directory also looks correct.

The JS requests are all showing in the Inspector apart from one chunk file which is 404'ing. This appears to be causing none of the JS to actually run. You know the JS is running if the logo at the top left wobbles when you hover over it.

In development on my local machine, it all runs fine. It's production where it fails.

[brenogazzola]

Ah, yes. Common problem. Your JS file is bundled and expects the chunk to be named something like trix-chunk.js. But after the asset pipeline compiles it in production, the file is actually renamed to trix-chunk-[digest].js. This causes the file to not be found.

You have to use the digest skip feature. In your bundler config use the following

Webpack (this is what I used to use):

module.exports = {
  output: {
    filename: '[name].js',
    chunkFilename: '[name]-[contenthash].digested.js',
  }
}

Esbuild (copied this from #15):

require('esbuild').build({
  chunkNames: '[name]-[hash].digested',
}).catch(() => process.exit(1))

The important part is that that the files that will be included with javascript_include_tag do not have a digest, but the chunks have a digest and the .digested keyword.

[rctneil]

I'll have a go at this shortly. Thanks for taking a look. I will report back

[rctneil]

@brenogazzola Hmmm,

This is weird,

I'm converting from the one line script to a config style, to allow me to add the chunkNames option.

My config file is:

const path = require('path');

require("esbuild").build({
    absWorkingDir: path.join(process.cwd(), "app/javascript"),
    entryPoints: ["application.js"],
    bundle: true,
    splitting: true,
    format: "esm",
    outdir: path.join(process.cwd(), "app/assets/builds"),
    chunkNames: '[name]-[hash].digested',
    plugins: [],
}).catch(() => process.exit(1));

The line in my package.json file is:

"build": "node esbuild.config.js",

When I run ./bin/dev, I just get a large error starting with:

19:14:09 js.1     | yarn run v1.22.17
19:14:09 css.1    | $ sass ./app/assets/stylesheets/application.sass.scss:./app/assets/builds/application-styles.css ./app/assets/stylesheets/angry.sass.scss:./app/assets/builds/angry.css --no-source-map --load-path=node_modules --watch
19:14:09 js.1     | $ node esbuild.config.js --watch
19:14:10 docker.1 | Container neiltongesite-search-1  Running
19:14:10 docker.1 | Container neiltongesite-db-1  Running
19:14:10 docker.1 | Attaching to neiltongesite-db-1, neiltongesite-search-1
19:14:10 js.1     | Done in 0.16s.
19:14:10 web.1    | => Booting Puma
19:14:10 web.1    | => Rails 7.0.2.2 application starting in development
19:14:10 web.1    | => Run `bin/rails server --help` for more startup options
19:14:10 css.1    | Sass is watching for changes. Press Ctrl-C to stop.
19:14:10 css.1    |
19:14:10 js.1     | exited with code 0
19:14:10 system   | sending SIGTERM to all processes
19:14:10 web.1    | Exiting
19:14:10 jobs.1   | ["DEBUGGER Exception: /Users/neiltonge/.rbenv/versions/3.0.3/lib/ruby/gems/3.0.0/gems/debug-1.4.0/lib/debug/thread_client.rb:986"

What could be going on?

[brenogazzola]

I'm not sure. I did the initial esbuild setup for our monolith, but after that we never touched it again, so I don't have much experience with it. For reference, here's our working config file (minus chunkNames, which we don't use):

#!/usr/bin/env node

const path = require('path')
const watch = process.argv.includes('--watch')
const esbuild = require('esbuild')

const config = {
  absWorkingDir: path.join(process.cwd(), 'app/javascript'),
  entryPoints: ['application.js'],
  outdir: path.join(process.cwd(), 'app/assets/builds'),

  bundle: true,
  minify: true,
  sourcemap: true,
  target: 'es6',

  watch: watch
}

esbuild.build(config).catch(() => process.exit(1))

Try using this one. If it still breaks, remove everything on your Procfile other than the line for esbuild and run ./bin/dev then post the full stack trace here.

[rctneil]

Hi @brenogazzola Apologies for the delay in getting back to you.

I have managed to get this to deploy correctly now. I used your file from above with some minor changes and it's all going swimmingly for now at least! If anything changes I will report back!