Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there a way to import json as a static file? #501

Closed
mcfarljw opened this issue Jan 6, 2018 · 49 comments
Closed

Is there a way to import json as a static file? #501

mcfarljw opened this issue Jan 6, 2018 · 49 comments

Comments

@mcfarljw
Copy link

mcfarljw commented Jan 6, 2018

I'm using a few libraries that want to load json from a path. Is there a way to serve static files (similar to what the copy-webpack-plugin does)?

@DeMoorJasper
Copy link
Member

DeMoorJasper commented Jan 6, 2018

require, import and fs.readFileSync will all include the json file into the bundle

@mcfarljw
Copy link
Author

mcfarljw commented Jan 6, 2018

Hmm, @DeMoorJasper can you clarify a bit? I just tried using require for my json file but it still gives me an object rather than a static file path.

@DeMoorJasper
Copy link
Member

DeMoorJasper commented Jan 6, 2018

That's the point, it resolves the json file as an object, it parses it for you.

@mcfarljw
Copy link
Author

mcfarljw commented Jan 6, 2018

Sorry, I read that as use require.... import and fs.readFileSync will all include the json file into the bundle. The doesn't really help answer my question, but rather just confirms what I already know. Does that mean there is no way to import a json file as a static file rather than having it included in the bundle?

@DeMoorJasper
Copy link
Member

The only way i can think of resolving json as a raw asset is to overwrite the asset type by writing a plugin

@mcfarljw
Copy link
Author

mcfarljw commented Jan 6, 2018

Hmm, I wouldn't really want to overwrite an asset type. I guess what I'm looking for is a way to use parcel in development that works like the static-serve middleware for express.

@devongovett
Copy link
Member

Why do you need to copy the JSON file instead of bundling it into your code? Just trying to understand the usecase.

@mcfarljw
Copy link
Author

mcfarljw commented Jan 6, 2018

@devongovett I use a few 3rd party libraries that only support passing a path to load json files via xhr. The example I ran into today was with phaser 2 sprite atlases:

game.load.atlas('items', 'assets/sprites/items.png', 'assets/sprites/items.json')

It's probably not too common, but I can recall using a few other libraries that have their own preload systems that treat json assets the same as images.

@sunnylqm
Copy link

sunnylqm commented Jan 9, 2018

@devongovett For example, I want to lazy load some (or a lot) json data.

@shunia
Copy link

shunia commented Jan 10, 2018

@sunnylqm Fair point.
We run into this problem too as we have a bunch of assets(images,json files,etc.) to be loaded.

require & import does most the way we wanted, except with json files, it bundles json into javascript instead of return the path map. But what we wanted is the return the path way as other files do.

The reason is, we are building a loading sequence for all the assets other than javascript files, to show a loading bar to users, which will indicates the loading progress.

Webpack actually offers a extended require function WebpackRequire for us to make this happen, by mapping file path before bundle and after (WebpackRequire.context), we can dynamically replace path used inside javascript codes into hashed path, to make a successful load (by xhr) action.

@devongovett
Copy link
Member

See here for some potential strategies to allowing multiple import formats: facebook/create-react-app#3722

@FDiskas
Copy link

FDiskas commented Mar 25, 2018

@RELNO
Copy link

RELNO commented Apr 15, 2018

To extend @shunia & @mcfarljw point, THREE - THREE.ObjectLoader() API is XHR JSON loader which fails in parcel dev. server:
screen shot 2018-04-14 at 21 34 02

@FDiskas
Copy link

FDiskas commented Apr 15, 2018

#536

@michaeljota
Copy link

michaeljota commented May 26, 2018

Just to say this is needed if you are using something like pixi.js. I understand not a huge user base, but is something. I guess I have to go back and use Webpack, and waste 4 hours of my life.

Or do: #1080 (comment)

Thanks.

@shunia
Copy link

shunia commented May 28, 2018

@michaeljota Change file type from .json to something like .data, then let pixi.js to load them as json type. Problem solved.
But you may lose type check(json format validation) from your ide.

@michaeljota
Copy link

I just try that and I does not work.

  • If I load an image by requireing it, and use the name that require function resolves, it does work.

The problem now, is exactly about the json atlas files. But not quite because is a json file or not.

  1. pixi seems to validate the kind of file that you are trying to load, and I don't quite know how to tell pixi this is a json file with another name
  2. The atlas file has a property that should resolve to the name of the atlas file. Because Parcel will always add a hash to whatever resource I need to load, the name in atlas will never match. (Again, if this is possible, please tell me how, because I really don't know how).

Having a bundler tool with zero config is great, but I guess you need to consider what are the main use cases you want to cover, and at the end, I think is real hard to cover them all.

@shunia
Copy link

shunia commented May 28, 2018

@michaeljota It's not so straight forward to use neither parcel nor webpack, to work with pixi.js. Some extra work need to be done when using parcel, same as webpack.

If you want to get the real path after parcel builds your .json file(after changing your file type to .data for example), this would just works:

  let jsonURL = require('path/to/your/json/file');

If you want to load any file as json format file(of course they need to be parsed correctly by JSON.parse), just provide a proper loaderType/XHRType to pixi.js's loader, something like:

  PIXI.loader.add({
    url: jsonURL, 
    loader: 'json loader type defined by pixi.js', 
    xhrType: 'json xhrType defined by pixi.js'
  });

If you want to patch json data to atlas resolver, you have to understand how pixi.js's resource loader is working, and write some hack code to make it working, I can not provide example here.

@michaeljota
Copy link

If you want to get the real path after parcel builds your .json file(after changing your file type to .data for example), this would just works:

 let jsonURL = require('path/to/your/json/file');

I do it that way, and this is the way I use to load things that are not a json file. This is the main issue with this approach. The JSON file have a property that points to the path out the atlas image. As I understand there is not currently a way to tell parcel to move a file without renaming it, so there is no way for the JSON atlas describer to know where the atlas image will be after the build.

If you want to load any file as json format file(of course they need to be parsed correctly by JSON.parse), just provide a proper loaderType/XHRType to pixi.js's loader, something like:

I don't know much about pixi but I did lookup for something like this, and the loader, and xhrType options seems to be related more to something like xrh, audio, video, and image. But I did not found something like json.

@jedhastwell
Copy link

This is not strictly answering the question, but for those of you just looking to load a PIXI sprite sheet, you can bypass the PIXI loader and manually create the sprite sheet from the raw data:

  const data = require('../assets/atlas.json');
  PIXI.loader.add('atlas', require('../assets/atlas.png'), (resource) => {
    if(resource.error) {
      console.error(resource.error);
    } else {
      const texture = resource.texture.baseTexture;
      const sheet = new PIXI.Spritesheet(texture, data);
      sheet.parse((textures) => {});
    }
  });

@michaeljota
Copy link

@jedhastwell This will create an atlas key in the TextureCache with the data from the atlas descriptor or something like that? I'm sorry, I just don't understand Pixi that well.

@jedhastwell
Copy link

@michaeljota It's the same as if you'd just used the loader to load the atlas file directly from a JSON file. It will add all the textures defined in the atlas file to the TextureCache. Then you can just create sprites using the names defined in the atlas like:
let ball = Sprite.fromImage('ball.png');

@michaeljota
Copy link

michaeljota commented May 30, 2018

Oh! OK, OK... Will try that. Thanks you.

@jedhastwell Thanks man. Really helpful. Worked as intended. Thanks again.

@skozin
Copy link

skozin commented Jun 5, 2018

To extend the point for adding static files serving/copying: I'm using some proprietary third-party library that expects me to pass URL to a static directory provided with the library. This directory contains about two hundreds of files, something that looks like this:

image

These files are JS/CSS/HTML files, as well as images, cursors and fonts. I cannot require every single file, and even if I could it won't help as the library expects to get each of them using XHR under specific, compiled-in name. I can only change shared URL prefix.

As far as I can see, in order to make this setup work under Parcel, I need two things:

  1. When building production bundle: ability to copy a bunch of files and folders to the destination directory without any parsing and modification.
  2. When developing: ability to serve static contents from a local directory under specific URL prefix.

Please let me know if this can be accomplished in some other way.

@shunia
Copy link

shunia commented Jun 6, 2018

@skozin Just try const assets = require('relative/path/to/assets/folder/*'). Notice the * in the url.
The object assets should contains everything you need.

@michaeljota
Copy link

As I understand, the problem is not loading those files, but letting those files load themselves.

@skozin
Copy link

skozin commented Jun 6, 2018

@shunia, the problem is not loading those files, as @michaeljota correctly said. I don't need them; the library I'm using needs them, and it wants to load them using XHR, and expects them to have pre-defined file names which are compiled into the library code. It does this to avoid loading unnecessary stuff: the total size of all these files is something like 20Mb, and most of them are not needed in every case. I suppose that .js bundles were actually generated using webpack's code splitting mechanism and are being loaded using dynamic imports or require.ensure. Note that I have no access to the library sources, I can only use already bundled and minified code.

@shunia
Copy link

shunia commented Jun 8, 2018

Try parcel-plugin-json-url-loader

@stevage
Copy link

stevage commented Jun 18, 2018

Just wanted to add my use case: loading Geojson files (that is, spatial data containing location information). It doesn't make any sense to me to bundle that data. It would make the bundle enormous, and presumably all the loading would have to happen before the site displayed. (In my case I'm using mapbox-gl-js, which means I could pass the loaded geojson object, instead of a URL...but I'd prefer not to.)

Really surprised there isn't a simple solution like, "anything in /static doesn't get bundled" and can be loaded using XHR.

@kpollich
Copy link

+1 for the aforementioned three.js use cases here. Looking into migrating a WebGL project away from Webpack, but we need to be able to load object files and other WebGL assets via XHR.

@skozin
Copy link

skozin commented Jun 28, 2018

@shunia thanks! we'll try this :)

@mikedpad
Copy link

Just started using Parcel recently (❤️it, btw) and ran into this issue recently myself. My use case was much simpler... I just wanted to include (copy) some static HTML files over to the dist folder.

I eventually just used an anchor in my index.html entry to include the file:

<body>
  <noscript>You must enable JavaScript to use this app.</noscript>
  <div id="root"></div>
  <a href="myStaticFile.html"></a>
  <script src="./index.js"></script>
</body>

</html>

... which is incredibly hacky. I don't want to link from the page, but this is how I got them to be included without bundling them.

Like @stevage mentioned, it'd be great if there was a simple /static subfolder where assets aren't processed or bundled.

@NotIntMan
Copy link

@shunia, parcel-plugin-json-url-loader works bad when you need to refer json from html.

<link rel="manifest" href="~/../static/manifest.json">

@shunia
Copy link

shunia commented Aug 28, 2018

@NotIntMan
This plugin is not intend to do this. Because this intention has been handled by parcel.
Try use relative path to your html file rather than a absolute path to your computer!

@thejohnfreeman
Copy link

My use case is I want a separate JSON configuration file that my administrator can tweak in the deployment. (I would love to use continuous integration, but I have to fit my solution into the problem's constraints.)

@kleinfreund
Copy link

kleinfreund commented Feb 25, 2019

How do I get Parcel to resolve a path like this and add the JSON to the output?

xhr. open('GET', '../data/some.json', true);

This flat out doesn’t do anything expected. For some reason, the result is not even a 404 error, but my HTML entry point file. This is extremely confusing.


Weird workaround:

npm i parcel-plugin-json-url-loader --save-dev
xhr. open('GET', require('../data/some.json'), true);

@bsides
Copy link

bsides commented Jun 21, 2019

Just wanted to leave my use case here.

I'm developing a Chrome Extension and it is required in the final build folder to have a manifest.json file there. Right now I'm just cp manifest.json ./dist/ but I wish I could just put it as an entry file with an option to just copy it.

@connorjclark
Copy link

connorjclark commented Jul 14, 2019

A use case for bundling JSON as a string and deferring parsing to runtime: it can actually be faster. The JSON grammar is faster to parse than JS.

@puka-tchou
Copy link

For all of you who just need to copy a file from a static directory to the bundled one, I would recommend giving a shot at parcel-plugin-static-files-copy.
I don't know how it handles the paths though.

@bmmpt
Copy link

bmmpt commented Oct 26, 2019

Seems like everybody is adding their usage case here, so why not? :)

I am beginning to use the amazing react-i18next library and it's ability to asynchronously load new translations and add them to the application context. It supports plugins for xhr and fetch, but in both cases I get the HTML entry point file.

Both plugins try to fetch:
http://localhost:3005/locales/en-CA/translation.json

I don't even have to tell i18next where to get the files from as long as I put them in the default path it looks for translation files (public/locales/...). I also have no say on how it's getting the translations, neither would I want to. That's the beauty of the plugin. That being said, I can't even use the "weird workaround" with require mentioned above.

There will be dozens of use cases for stuff like this. Can Parcel be awesome at this too?

@mischnic
Copy link
Member

With Parcel 2 alpha 3 and this parcelrc

{
  "extends": "@parcel/config-default",
  "transforms": {
    "url:*": ["@parcel/transformer-raw"]
  }
}

you can import files which will then be copied to the dist folder:

import file from "url:./static.json"
fetch(file)...

@deepred5
Copy link

deepred5 commented Dec 7, 2019

use pixi.js and load a json file . Thanks @jedhastwell

import { Application,  Sprite, Loader, Spritesheet } from 'pixi.js';

import myjosn from './assets/treasureHunter.json';
import mypng from './assets/treasureHunter.png';

const loader = Loader.shared;

const app = new Application({
  width: 300,
  height: 300,
  antialias: true,
  transparent: false,
  resolution: 1,
  backgroundColor: 0x1d9ce0
});

document.body.appendChild(app.view);

loader
.add('mypng', mypng)
.load(setup)

function setup() {
 
  const texture = loader.resources["mypng"].texture.baseTexture;
  const sheet = new Spritesheet(texture, myjosn);
  sheet.parse((textures) => {
    const treasure = new Sprite(textures["treasure.png"]);
    treasure.position.set(100, 100);
    app.stage.addChild(treasure);
  });
}

@danmarshall
Copy link
Contributor

Try parcel-plugin-json-url-loader

This worked for me, thanks @shunia ! 🥇

@n8isjack
Copy link

n8isjack commented Apr 1, 2020

So I am using Parcel... love it and don't want to go back to webpack.

I have been dealing with this and my use case is for Pixi.js with spritesheets. I can't get it to work without a static file name. (I have done d3 maps before and see how it would be the same problem)

I also have cases where I want the JSON loaded as an object, so I can't just over-ride JSON imports.

In the end I was able to get PIXI to work with a data url.

I suppose if it was a huge file I'd have to learn about Parcel's Code Splitting for lazy loading.

import sprite_png from "../sprites/sprites.png";
import sprite_data from "../sprites/sprites.json";

sprite_data.meta.image = sprite_png;

let data64 = btoa(JSON.stringify(sprite_data));
let dataURL = `data:text/json;base64,${data64}`;

const theGame = new PIXI.Application({
  resizeTo: window
});

theGame.loader.add("sprites", dataURL);

@wyozi
Copy link

wyozi commented Apr 9, 2020

For playcanvas users struggling with .json model imports, you can rename model .json files to .modeljson and add a custom loader:

(app.loader.getHandler("model") as pc.ModelHandler).addParser(
  new (pc as any).JsonModelParser(app.graphicsDevice),
  function (url, data) {
    return pc.path.getExtension(url) === ".modeljson";
  }
);

@opyate
Copy link

opyate commented Oct 22, 2020

Just leaving my workaround here for Phaser 3 + Parcel for the simple case where there's one spritesheet, in case it helps anyone.

ls -1 assets
other.png
spritesheet.json
spritesheet.png
import Phaser from 'phaser'
// @ts-ignore
import images from '../assets/*.png'
import spritesheetJson from "../assets/spritesheet.json"

export default class BootScene extends Phaser.Scene {
  constructor () {
    super({ key: 'boot' })
  }

  preload () {

    console.table(images)

    this.load.image('other', images.other)

    spritesheetJson.textures[0].image = images.spritesheet.substring(1)
    this.load.multiatlas('mysprites', spritesheetJson)
  }

  update () {
    this.scene.start('next-scene')
  }
}

Elsewhere:

this.add.sprite(0, 0, 'mysprites', 'the-key')

@TimDaub
Copy link

TimDaub commented Mar 12, 2021

With Parcel 2 alpha 3 and this parcelrc

{
  "extends": "@parcel/config-default",
  "transforms": {
    "url:*": ["@parcel/transformer-raw"]
  }
}

you can import files which will then be copied to the dist folder:

import file from "url:./static.json"
fetch(file)...

It seems transforms was renamed to transformers: https://v2.parceljs.org/configuration/plugin-configuration/

@TimDaub
Copy link

TimDaub commented Mar 16, 2021

A few days ago, I tried importing a .json file in the source code that I manage using parcel. My assumption is that anything that I use in an import ... from ... from statement gets bundled into my regular bundle.
I'm aware that *.json imports are not yet stable in the current ES specification. That's OK.

I then discovered this thread and saw that different from my expectation, many people want to lazy load *.json files to optimize for their user experience. Here's a few of those:

However, the problem is that I think the import ... from ... statement was never intended to lazy load anything. That's what import(...) is for. Hence, I find it to be a regression that we now can define import paths starting with url:... for import ... from "url:..." to code split and lazy load (as I pointed that out in my last comment).

From parcel, what I'd expect when inputting import ... from "*.json" is that it bundles the *.json file in the main bundle once the ES standard on `.json imports has matured.

To import JSON with import ... from ..., I recommend using babel-plugin-inline-json-import. You can find instructions on how to integrate on the npm page.

@jaredkrinke
Copy link

jaredkrinke commented Jan 14, 2024

With Parcel 2 alpha 3 and this parcelrc

{
  "extends": "@parcel/config-default",
  "transforms": {
    "url:*": ["@parcel/transformer-raw"]
  }
}

you can import files which will then be copied to the dist folder:

import file from "url:./static.json"
fetch(file)...

Thanks for this! The only tweak I would suggest is to use URL since it's more obvious that it's producing a URL (or string). I was able to use this for lazy loading translations for react-intl:

const urlToFetchForJson = new URL("url:../content/messages-compiled/zh-CN.json", import.meta.url);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests