Esbuild Configuration for Javascript Debugging

[ToolKami]

Migrated from Webpacker to Esbuild using jsbundling-rails and it seems like Javascript debugging no longer works for either VSCode nor Rubymine.

This is the ESBuild settings I am using.

require("esbuild").build({
  entryPoints: ["application.js", "administrate.js"],
  bundle: true,
  outdir: path.join(process.cwd(), "app/assets/builds"),
  absWorkingDir: path.join(process.cwd(), "app/javascript"),
  watch: watch,
  plugins: [ImportGlobPlugin()],
}).catch(() => process.exit(1));

[ToolKami]

For more context.

The compiled assets can be found in app/assets/builds/*.js or public/assets/**/*.js.

The bundled JS that is actually loaded is http://localhost:3000/assets/application.debug-fc91062bf3c7540aaa82649048a601114b102e175ab6b54a445960fb29a8ebc2.js.

but I'm also seeing this warning:
DevTools failed to load source map: Could not load content for http://localhost:3000/assets/application.js-9bc83b28e900922b5437611c775ac156746a4b54b883ec28346dc6ee1be3bc3b.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE

[dhh]

Source mapping is still not fully supported. Need alterations to Sprockets.

[czj]

While sprockets is not compatible, you can setup a custom route like so:

if Rails.env.development?
  redirector = ->(params) { ApplicationController.helpers.asset_path(params[:name].split("-").first.append(".map")) }
  constraint = ->(request) { request.path.ends_with?(".map") }
  get "assets/*name", to: redirect(redirector), constraints: constraint
end

If the assets pipeline cannot find a file corresponding to the a given sourcemap file name, Rails' routing will pick up any URL starting with /assets/ and ending in .map and redirect to the URL of the corresponding .css.map or .js.map file in the asset pipeline.

Works well for us with jsbundling-rails and cssbundling-rails. Maybe it also works with propshaft.

[johnmcdowall]

I had to slightly modify the above example for my Rails 6.1.4.1 app, but it does work:

if Rails.env.development?
  redirector = lambda { |params, _req|
    ApplicationController.helpers.asset_path(params[:name].split('-').first + '.map')
  }
  constraint = ->(request) { request.path.ends_with?('.map') }
  get 'assets/*name', to: redirect(redirector), constraints: constraint
end

I now have full sourcemaps working for now.

[jhirn]

Figured I'd share my esbuild file as I've been working on it for a few weeks now and had to dig for a lot of this myself. The inline sourcemaps are not ideal for production but they work for now. Also svgr, loader, injecting shims and adding watch was a bit tricker than I hoped for but well worth it so far. Hopefully this helps someone.

#!/usr/bin/env node

const esBuild = require('esbuild')
const svgrPlugin = require('esbuild-plugin-svgr')

const watch = process.argv.includes('--watch')
const minify = process.argv.includes('--minify')

const watchOptions = {
  onRebuild: (error, result) => {
    if (error) {
      console.error('watch build failed:', error)
    } else {
      console.log(result)
      console.log('watch build succeeded. ')
    }
  }
}

const svgrOptions = {
  typescript: false
}

const loaders = {
  '.png': 'file'
}

esBuild.build({
  entryPoints: ['app/javascript/application.js'],
  logLevel: 'info',
  bundle: true,
  outdir: 'app/assets/builds',
  plugins: [
    svgrPlugin(svgrOptions)
  ],
  watch: watch && watchOptions,
  sourcemap: 'inline',
  loader: loaders,
  publicPath: '/assets',
  inject: ['.esbuild/shims/react-shim.js'],
  minify
}).then(result => {
  console.log(result)
  if (watch) {
    console.log('Build finished, watching for changes...')
  } else {
    console.log('Build finished, Congrats')
  }
}).catch(result => {
  console.log(result)
  process.exit(1)
})

[jwilsjustin]

@dhh Do you know if there is an issue tracked somewhere that lays out what needs to change with sprockets? I went looking and found nothing that matches directly.

[dhh]

Don't think we have anything fully recorded. We need to find a way to produce source maps with digested file names that can stay stable. Maybe @brenogazzola has an idea? There's a straight shot for this for webpack.

[brenogazzola]

@dhh I'll update the regex in Propshaft since it currently does not support multiple file extensions after .digested. I also have a PR for Sprockets to replicate that: rails/sprockets#718, but I still need to check what happens when you give it a file with multiple extensions 😅

I haven't switched to esbuild yet, but I've taken a look through its documentation and I think that --entry-names is what we are looking for. Since there's no option for choosing the name of the sourcemap I'll assume it will use the name of the original file and simply add .map. If someone could test it:

require('esbuild').buildSync({
  entryNames: '[name]-[hash].digested',
})

And here's the webpack equivalent:

output: {
  filename: '[name].js',
  chunkFilename: '[name]-[contenthash].digested.js',
  sourceMapFilename: '[name]-[contenthash].digested.js.map',
  path: path.resolve(__dirname, 'app/assets/builds')
},

If this configuration works for esbuild, and the generated source map ends as something like application-abcdef1234567.digested.js.map, then it's just a matter of validating the two open PRs for propshaft and sprockets.

[dhh]

Sourcemaps generated by esbuild (or any other transpiler) are now supported since sprockets-rails 3.4.0.

[TastyPi]

I switched to jsbundling-rails recently and came across this issue, despite having sprockets-rails 3.4.2. Esbuild is creating the [name].js.map files, but then the browser complains that it can't find [name].js-[contenthash].map.

I've worked around this by removing the --sourcemap flag from package.json and adding --sourcemap=inline to the command in Procfile.dev, but it would be nice if the separate file worked as intended.

[dhh]

Can you try to produce an example app that triggers this? It's working for me when I try.

[jaredcwhite]

FWIW, source maps via esbuild -> Sprockets working for me as well. 👍

[fcheung]

Just hopping in here to say that setting config.assets.debug to false (used to default to true in dev, doesn't for apps generated after rails/rails@adec7e7#diff-2d045bcceb384e942cad5480ff01eff82255819f8944bef7a03e1609cc949004 ) fixed this for me

with config.assets.debug set to true, the AddSourceMapCommentToAssetProcessor ( https://github.com/rails/sprockets/blob/24c97270fbf6de11b4ffff0311bb427b7a8a3a83/lib/sprockets/add_source_map_comment_to_asset_processor.rb) processor in sprockets appends a second, incorrect source map comment

[TastyPi]

Setting config.assets.debug = false fixed it for me too, thanks @fcheung!

[WriterZephos]

@dhh I think this is still an issue, but it's intermittent. Just randomly stopped working after I made a change to a stimulus controller. Tried the config.assets.debug = false fix but it didn't work for me.

The map files are generated each time but the browser seems to be looking for the wrong file.

UPDATE:

So it seems that rails assets:clobber fixes the issue temporarily and lets the new source map be retrieved, but then it breaks again as soon as I make any js changes. So strange that this just started happening. I wonder if I am doing something wrong.

config.assets.debug = false breaks the above workaround in development... which is weird. It's like it is precompiling when it shouldn't be.

[danieldocki]

I was having the same problem with migrating from webpacker to jsbundling-rails with esbuild, every time I changed a js file the css file lost the styles.

Then I noticed that some js files were doing:

import 'something.css'

I removed all of them from the js files and it seems it solved my problem, I don't know if it has anything to do with this situation.

[jaredcwhite]

I removed all of them from the js files and it seems it solved my problem, I don't know if it has anything to do with this situation.

@danieldocki yeah that's a known issue. see: https://github.com/rails/jsbundling-rails#why-does-esbuild-overwrite-my-applicationcss

[nachoal]

I'm currently in the middle of migrating a 7.0.2.3 rails app from webpack to jsbuild and started having the DevTools failed to load source map error.

After checking the following:

Sourcemaps generated by esbuild (or any other transpiler) are now supported since sprockets-rails 3.4.0.

My sprockets version is 3.4.2 but the issue still exists, after trying @fcheung solution
config.assets.debug = false the warning message stopped happening but that's definitely not the right approach if I want to keep logging via JS/Stimulus

If someone can explain why this is happening or guide me in the right direction to be able to have debug = true it would be awesome, ty!

[jhirn]

@nachoal, Can't guarantee help but if you could share your esbuild script/config as well as any other relevant asset configuration that would help to diagnose your problem I'll take a look.

[nachoal]

Thanks @jhirn

package.json:

{
  "name": "x",
  "private": true,
  "dependencies": {
    "@hotwired/stimulus": "^3.0.1",
    "@hotwired/turbo-rails": "^7.1.3",
    "@rails/actioncable": "^6.0.5",
    "@rails/actiontext": "^6.0.5",
    "@rails/activestorage": "^6.0.5",
    "@rails/ujs": "^6.0.5",
    "autoprefixer": "^10.4.7",
    "esbuild": "^0.14.48",
    "postcss": "^8.4.14",
    "stimulus": "^3.0.1",
    "tailwindcss": "npm:@tailwindcss/postcss7-compat",
    "trix": "^2.0.0-beta.0"
  },
  "version": "0.1.0",
  "devDependencies": {},
  "scripts": {
    "build": "esbuild app/javascript/*.* --bundle --sourcemap --outdir=app/assets/builds --public-path=assets"
  }
}

Procfile.dev

web: bin/rails server
jobs: bin/sidekiq
css: bin/rails tailwindcss:watch
js: yarn build --watch

Having config.assets.debug = true

[nachoal]

While sprockets is not compatible, you can setup a custom route like so:

if Rails.env.development?
  redirector = ->(params) { ApplicationController.helpers.asset_path(params[:name].split("-").first.append(".map")) }
  constraint = ->(request) { request.path.ends_with?(".map") }
  get "assets/*name", to: redirect(redirector), constraints: constraint
end

If the assets pipeline cannot find a file corresponding to the a given sourcemap file name, Rails' routing will pick up any URL starting with /assets/ and ending in .map and redirect to the URL of the corresponding .css.map or .js.map file in the asset pipeline.

Works well for us with jsbundling-rails and cssbundling-rails. Maybe it also works with propshaft.

Had to modify this approach a little bit, here's the one that's staying:

if Rails.env.development?
    redirector = ->(params, _) { ApplicationController.helpers.asset_path("#{params[:name].split('-').first}.map") }
    constraint = ->(request) { request.path.ends_with?(".map") }
    get "assets/*name", to: redirect(redirector), constraints: constraint
 end

[jhirn]

Ah I see. I only ever got inline to work. Will give this a shot if work resumes on that project.

Despite the bit of yak shaving, esbuild is so freaking awesome. If webpacke(r) were a thing, I'd burn it in a public event.

[ccastillop]

I've spent few hours struggling with this.

Setting config.assets.debug = false on config/environments/development.rb did not add expected //# sourceMappingURL=application.js.map url on assets/application.js

Then I set config.assets.debug = true on config/environments/development.rb and expected //# sourceMappingURL=application.js.map showed but DevTools failed to load source map

At the end I added suggested #40 (comment) to routes.rb and It worked.

[salimhb]

For some reason Sprockets 4.2.0 breaks this for me.
The workaround in routes.rb is not catching the sourcemap request anymore.

ActionController::RoutingError (No route matches [GET] "/application.js-ed5ef27ac34ec75105c2c5f54bcfb407a5ecceb43da87cfe9437f57c9cf55d45.map"):

I'm not sure which change exactly affects this.

I downgraded back to 4.1.1 to get sourcemaps working again.

[navidemad]

I'm getting this No route match error since this commit rails/sprockets@0692518

[gap777]

Sprockets 4.2.0 also broke sourcemaps for me (esbuild + Rails 7.0.4.1). config.assets.debug doesn't seem to help either way. Rolling back to 4.1.1 and setting config.assets.debug = true seems to restore them.

@dhh?

[kkurcz]

Has anyone found a fix for this?

[scratchoo]

sourcemap is working (Rails 7.0.3.1) however the console show the wrong stimulus controller name, the logged line is correct, only the file name is not

This issue seems to happen only on chrome and brave browsers, I tested in firefox and it's working fine

[gap777]

@dhh This seems to have recently broken for @salimhb , @navidemad, @kkurcz and myself (at least).
Any ideas or workarounds?

[alexandrepalma]

Here's what worked for me on Rails 7.0.4.3, since the redirection route approach didn't work.
Create the following file:

app/middleware/sourcemap_redirect_middleware.rb

class SourcemapRedirectMiddleware
  def initialize(app)
    @app = app
  end

  def call(env)
    if env['PATH_INFO'].match?(/.*\.(js|css)-.*\.map/)
      asset_name = env['PATH_INFO'].split('-').first
      new_path = ActionController::Base.helpers.asset_path("#{asset_name}.map")

      # Preventing redirection loop
      if env['PATH_INFO'] != new_path
        return [301, { 'Location' => new_path }, []]
      end
    end

    @app.call(env)
  end
end

have your development.rb configured like this:

  require_relative '../../app/middleware/sourcemap_redirect_middleware'
  config.middleware.use SourcemapRedirectMiddleware
  config.assets.debug = true

Hope it works for you guys!

[lreardon]

@alexandrepalma's solution worked for me with a small tweak. I had to replace the line

asset_name = env['PATH_INFO'].split('-').first

with

asset_name = env['PATH_INFO'].split('-')[0..-2].join('-')

to handle cases where the path has multiple hyphens.

[gap777]

Worked for us, too... but found out that we needed a backtrace silencer to preserve stack traces:

# config/initializers/backtrace_silencers.rb
Rails.backtrace_cleaner.add_silencer { |line| line.include?('app/middleware') }

[navidemad]

Hello guys, any update regarding this issue ?
Have a great day

[navidemad]

up

[gap777]

Is this fixed with Rails 7.1.2?

[kkurcz]

@gap777 if you find a definitive solution please let me know

[bsoglin]

Still an issue