How to force recompile?

[silvenon]

• babel-plugin-preval version: 1.4.1
• node version: 8.1.3
• npm (or yarn) version: npm 5.2.0

Relevant code or config

const result = preval`
  const fs = require('fs');
  const path = require('path');
  const content = fs.readFileSync(path.join(__dirname, 'file.md'));
  module.exports = content.toString();
`;

console.log(result);

What you did: I ran the code above, to export contents of file.md.

What happened: when I modify file.md the result of preval didn't change when I tried to compile it again. Maybe I'm missing something?

Reproduction repository: demo

Problem description: preval probably doesn't see the reason to recompile because its tagged template literal didn't change.

Suggested solution: I don't know. 😅

[kentcdodds]

The problem isn't with preval, but with the babel cache. You could disable that and it should work (though I don't think that it would recompile in watch mode unless you change the importing file anyway). Perhaps @hzoo has some ideas for us 😀

[kentcdodds]

Another way to get around this problem would be to use the import syntax or the // @preval comment. That should resolve most (all?) problems related to this. If that works for you, do you mind adding something as a FAQ?

[silvenon]

Thanks for your suggestion! I updated my demo with @preval (I also tried import) but it still didn't update when I tried to recompile. I will gladly add a FAQ entry when we figure this out.

[kentcdodds]

Thanks for giving that a try. Surprised that didn't work! I don't have any time to dedicate to working on this particular issue right now, so anyone's welcome to help with this! Thanks!

[mattphillips]

Hey I've just taken a look. It's definitely that babel-node is caching and the js code "doesn't" change. You could use nodemon to watch for changes and disable the babel cache.

Something like this should work:
nodemon.json:

{
  "verbose": false,
  "ignore": ["node_modules"],
  "env": {
    "NODE_ENV": "development",
    "BABEL_DISABLE_CACHE": 1
  },
  "execMap": {
    "js": "babel-node"
  },
  "ext": ".js,.md",
  "watch": "./src/"
}

Then your start script will become "start": "nodemon index.js".

[souporserious]

Just ran into this with my docs stuff 😅 I have to manually go in and save the file to recompile. I'm using the import syntax as well with no luck. Going to try and see if I can figure anything out. Some of this stuff is over my head, but I'll report any findings.

[kentcdodds]

Thanks @souporserious! I experience this with Next.js and it's pretty annoying, I wind up just adding a // 12345 etc... comment to the file using/importing preval and that works alright, but it's not a super great experience. I'd love something that would help avoid this issue for Next/Webpack/Babel/etc.

[loganfsmyth]

Babel's caching is definitely lacking. There's no core functionality for it, so everyone implements it outside core, which means there aren't any core primitives to express when caches should be invalidated. I'm still hoping I can get something like that into core for 7.x, but I can't promise it'll happen.

[kentcdodds]

How can we help you @loganfsmyth?

[loganfsmyth]

Good question. Basically trying to figure out how I can actually get paid to do something interesting at the moment, because working on Babel full-time for free for a few months has kind of eaten through my motivation :(

[kentcdodds]

Ah! You're still looking for a job? Ping me on twitter DM or something and we'll see if we can help you find something! 😄

[kenvunz]

Another workaround is clearing out the babel cache folder (default at ./node_modules/.cache/babel-loader before build, my scripts example

  "scripts": {
    "start": "yarn clear:babel-cache && next",
    "build": "yarn clear:babel-cache next build && next export",
    "clear:babel-cache": "rimraf -rf ./node_modules/.cache/babel-loader/*"
  }

[souporserious]

Is it bad to set up a watch task and have that run every time a file is changed @kenvunz? Been sort of a rough workflow having to start and stop my dev server to see any changes would love to have it work on file change.

[kentcdodds]

Bad? No. Performant? No also. Is it noticable? Depends, but probably not.

[souporserious]

Has anyone gotten this to work with webpack-dev-server? I tried disabling babel-cache by setting an environment variable with no luck :/ I still have to start and stop the server to get any changes.

[loganfsmyth]

@souporserious babel-loader's cache is enabled based on the presence of the cacheDirectory option to the loader. Do you have that flag set?

[souporserious]

Hmm interesting 🤔 I don't have that set at all. Is it possible babel cache doesn't have anything to do with not getting the changes? Could it be something else?

Here's my config:

  const { resolve } = require('path')
  const webpack = require('webpack')

  const config = {
    entry: [
      'webpack-dev-server/client?http://localhost:8080',
      'webpack/hot/only-dev-server',
      resolve(__dirname, 'example/index.js'),
    ],

    output: {
      path: resolve(__dirname, 'example'),
      filename: 'bundle.js',
    },

    devtool: 'inline-source-map',

    devServer: {
      host: '0.0.0.0',
      contentBase: resolve(__dirname, 'example'),
      hot: true,
      inline: true,
      historyApiFallback: true,
      disableHostCheck: true,
    },

    module: {
      rules: [
        {
          test: /\.js$/,
          exclude: /node_modules/,
          use: [
            {
              loader: 'babel-loader',
              options: {
                plugins: ['preval', 'import-glob'],
                presets: [['es2015', { modules: false }], 'stage-0', 'react'],
              },
            },
          ],
        },
      ],
    },

    plugins: [
      new webpack.HotModuleReplacementPlugin(),
      new webpack.NamedModulesPlugin(),
    ],
  }

  module.exports = config

[loganfsmyth]

You know I don't think I actually thought through my comment last night properly, sorry.

With your setup, it's Webpack itself that is doing the in-memory caching. If you change a referenced file in preval, Webpack has no way of knowing that the JS file needs to be reprocessed. The Babel build process would actually have to return a list of referenced files so Webpack could be told what to watch. I'm not actually 100% sure Webpack exposes an API for that, though I know it does expose an .addDependency , not sure if it's what we'd want.

[souporserious]

Ahh I see 😕 thanks for the explanation! A little over my head, but I'll see if I can get anywhere. This stuff is so powerful, I can deal with the nuances for now 😸

[kentcdodds]

I don't think there's anything we can do in this package to make this work. Would someone like to document this issue in the README (in the FAQ)? Then we can close this.

[sokra]

Workaround:

module.rules: [
  {
    // add this before the babel-loader
    test: path.resolve(__dirname, "file.js"),
    use: {
      loader: path.resolve(__dirname, "add-dependency-loader.js"),
      options: {
        file: path.resolve(__dirname, "file.md")
      }
    }
  }
]

// add-dependency-loader.js
module.exports = function(source, map) {
  this.addDependency(this.query.file);
  this.callback(null, source, map);
}

[evenchange4]

I am not sure this is the right place to follow up the babel cache problem.

As you know I am working on graphql.macro last weekend. Everything works great for one-time production build, but If we do some IO side effects in the babel compile time it will break the recompile mechanism. [graphql.macro#6] 😥

Maybe we should add a note in the babel-plugin-macros repository?

[kentcdodds]

Yes, we should probably add a "caveats" section for things that are currently problems that we're still working on. Would you like to do that @evenchange4?

[evenchange4]

Sure, I will send a PR later.

[kitze]

I had only one file called files.js that's using preval to read my entire file hierarchy, so any solution including webpack/babel is an overkill. I don't change the file regularly, but I'm changing the files that it reads all the time.

I made a simple node script called watch-files.js that does this:

const fs = require('fs');
const path = require('path');

let filesPath = path.join(__dirname, 'src', 'config', 'files.js');

setInterval(() => {
  const file = fs.readFileSync(filesPath, 'utf8');
  fs.writeFileSync(filesPath, file + ' ');
  setTimeout(() => {
    fs.writeFileSync(filesPath, file);
  }, 100);
}, 1000);

It works perfectly.

[kentcdodds]

@kitze that's a fantastic hack 💯

[kentcdodds]

Whoops, didn't mean to close! That hack is not considered an "acceptable solution" 😉

[devthejo]

I just published a lib for this purpose, it works by adding annotation to your js file:

/*!@compileDependencies([
  './my/js-dependencie1.js',
  '../dependencie2.js',
  '../a-directory/',
])*/

when a specified dependency change, the file containing annotation is automatically recompiled,
directories are watched recursively and must end with a trailing slash

babel-watch-extra is coupled to nodemon

here is the link:
https://github.com/di-ninja/babel-watch-extra

usage:
babel-watch-extra --src src --dist dist entry-point1.js entry-point2.js

It has also others advantages against official @babel/cli watcher:

• take into account deletion and others events, keeping everything synced
• fix a bug of watching that was caused by awaitWriteFinish option of chokidar used in official @babel/cli watcher (with my favorite editor, geany, when I save a file, it write a temp file first, and then move it to dest, and sometimes the recompile watching was just lost and never fired again until I restart babel watch)

see also: https://stackoverflow.com/a/53510227/5338073

[kitze]

@TakioN that's great but I still have no idea how to use your lib in combination with babel-plugin-preval?

[devthejo]

@kitze
first install it:
npm i -D babel-watch-extra
and use it instead of babel watch and nodemon
babel-watch-extra --src src --dist dist entry-point1.js
where src is your source dir and dist your output dir
and in your dependent file you include the comment annotation referencing your dependencies dirs or files, ex:

file.js

/*!@compileDependencies([
  './directory1/',
])*/

when a file is added, modified or deleted in directory1 or it's subdirectories, file.js will be recompiled by the watcher

I use it with NodeJS, I don't know how to use it with webpack because it's not a babel plugin but a replacer for officiel babel watcher and so can't be loaded by babel-loader.
If you want the same mecanism with webpack you'll have to investigate how compilation can be tweaked in webpack and write a plugin or some wrapper, but you can inspire from babel-watch-extra for the main logic.

[kentcdodds]

I just noticed that this issue doesn't link to where this would really be solved. If this gets implemented, then we could solve this issue for real: babel/babel#8497

[wrthwhl]

Another workaround is clearing out the babel cache folder (default at ./node_modules/.cache/babel-loader) before build

For me this directory did not exist; my default cache was located at ~/.babel.json (file contents are cached in the JSON). Hopefully this comment saves someone the time it took for me to track that down.

For me neither of those worked. Instead rm -rf ./node_modules/.cache/@babel/register did the trick.

[budarin]

my opinion is that not every preval file needs to be recompiled every time, so we need another magic preval comment that will signal the preval plugin to recompile the file again every time.

[haku-d]

Another workaround is clearing out the babel cache folder (default at ./node_modules/.cache/babel-loader before build, my scripts example

  "scripts": {
    "start": "yarn clear:babel-cache && next",
    "build": "yarn clear:babel-cache next build && next export",
    "clear:babel-cache": "rimraf -rf ./node_modules/.cache/babel-loader/*"
  }

It's really helpful for me. Thanks

[alexgleason]

I'm trying to display the commit hash of my application in the UI, and running into this whenever making a new commit or changing branches locally (it continues to show the old commit hash until I clear the cache). So far all the comments are about watching some other file, but does anyone have ideas about dealing with an external command?

// @preval
const pkg = require('../../../package.json');
const { execSync } = require('child_process');

const shortRepoName = url => new URL(url).pathname.substring(1);
const trimHash = hash => hash.substring(0, 7);

const version = pkg => {
  try {
    const head = String(execSync('git rev-parse HEAD'));
    const tag = String(execSync(`git rev-parse v${pkg.version}`));

    if (head !== tag) return `${pkg.version}-${trimHash(head)}`;
  } catch (e) {
    // Continue
  }

  // Fall back to version in package.json
  return pkg.version;
};

module.exports = {
  name: pkg.name,
  url: pkg.repository.url,
  repository: shortRepoName(pkg.repository.url),
  version: version(pkg),
};

My best idea is to watch for changes in the .git directory, but that's an entire folder.

[alexgleason]

I figured out I could do this to avoid cache on a single file:

const rules = [
  {
    // First is my "real" babel-loader rule
    test: /\.(js|jsx|mjs)$/,
    include: [source_path],
    exclude: /node_modules/,
    use: [
      {
        loader: 'babel-loader',
        options: {
          cacheDirectory: true, // Cache is enabled
        },
      },
    ],
  },
  {
    // Below I add babel-loader again, except I disable cache for just this file
    test: resolve(__dirname, '../../app/soapbox/utils/code.js'),
    use: [
      {
        loader: 'babel-loader',
        options: {
          cacheDirectory: false, // Cache is disabled
        },
      },
    ],
  },
]

The only problem is, it doesn't refresh automatically when the git repo changes. That could be fixed by adding a custom loader as suggested here: #19 (comment)

EDIT: Watching .git/logs/HEAD did what I want.

[MauriceArikoglu]

If I understand the problem correctly this Babel/PR-11741 solves the issue by allowing to specify files as dependencies, causing them to rebuild...
Maybe we could let the babel authors know we want to get this merged in the PR discussion?