How do I resolve dependencies of 3rd party libs for testing?

[NullVoxPopuli]

Env Info

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 15.10
Release:    15.10
Codename:   wily

$ ng --version
Could not start watchman; falling back to NodeWatcher for file system events.
Visit http://ember-cli.com/user-guide/#watchman for more info.
angular-cli: 1.0.0-beta.5
node: 5.10.1
os: linux x64

Steps

• Create angular-cli app.
• Use JSData for a data layer, cause production apps are complex, and need a data layer.
• Try to use xhr-interceptor to mock json returned from the server because JSData doesn't use @angular/http, but uses axios for async http. An example of how this is done can be seen with ember-cli-mirage. Previously, I tried pretender, but had similar problems with nested dependencies.
• run ng test
• get errors similar to Uncaught error: Could not find module 'methods' from 'path/to/xhr-interceptor/index.js'

All I want to do is mock JSON responses from a fake server (one that isn't actually running, but is just scoped to an individual test).

Here are some relevant configuration things I've done:

// config/karma.conf.js
module.exports = function (config) {
  config.set({
    basePath: '../',
    frameworks: ['jasmine', 'commonjs'],
    plugins: [
      require('karma-jasmine'),
      require('karma-chrome-launcher'),
      require('karma-spec-reporter'),
      require('karma-coverage'),
      require('karma-commonjs'),
    ],
    customLaunchers: {
      // chrome setup for travis CI using chromium
      Chrome_travis_ci: {
        base: 'Chrome',
        flags: ['--no-sandbox']
      }
    },
    files: [
      { pattern: 'dist/vendor/es6-shim/es6-shim.js', included: true, watched: false },
      { pattern: 'dist/vendor/zone.js/dist/zone.js', included: true, watched: false },
      { pattern: 'dist/vendor/reflect-metadata/Reflect.js', included: true, watched: false },
      { pattern: 'dist/vendor/systemjs/dist/system-polyfills.js', included: true, watched: false },
      { pattern: 'dist/vendor/systemjs/dist/system.src.js', included: true, watched: false },
      { pattern: 'dist/vendor/zone.js/dist/async-test.js', included: true, watched: false },
      { pattern: 'dist/vendor/xhr-interceptor/**/*', served: true, watched: false },
      { pattern: 'dist/vendor/methods/**/*', served: true, watched: false },
      'dist/vendor/methods/index.js',
      'dist/vendor/http-browserify/index.js',
      { pattern: 'config/karma-test-shim.js', included: true, watched: true },
      // Distribution folder.
      { pattern: 'dist/**/*', included: false, watched: true }
    ],
    exclude: [
      // Vendor packages might include spec files. We don't want to use those.
      'dist/vendor/**/*.spec.js'
    ],
    preprocessors: {
      'dist/vendor/http-browserify/**/*.js': ['commonjs'],
      'dist/vendor/methods/**/*.js': ['commonjs'],
      'dist/vendor/xhr-interceptor/**/*.js': ['commonjs'],
      'dist/app/**/*.js': 'coverage',
    },
    reporters: [
      'spec',
      // 'coverage'
    ],
    coverageReporter: {
      type : 'html',
      dir : 'coverage/'
    },
    port: 9876,
    colors: true,
    logLevel: config.LOG_INFO,
    autoWatch: true,
    browsers: ['Chrome'],
    singleRun: false
  });
};

// src/system-config.ts
/***********************************************************************************************
 * User Configuration.
 **********************************************************************************************/
/** Map relative paths to URLs. */
const map: any = {
  '@portal': '../src/app',
  'js-data': 'vendor/js-data',
  'js-data-http': 'vendor/js-data-http',
  'js-data-jsonapi': 'vendor/js-data-jsonapi',
  'traceur': 'vendor/traceur',

  // for testing
  'xhrinterceptor': 'vendor/xhr-interceptor',
  'methods': 'vendor/methods',
  'http': 'vendor/http-browserify',
};

const paths: any = {

}

/** User packages configuration. */
const packages: any = {
  'js-data': {
    main: 'dist/js-data.min.js'
  },
  'js-data-http': {
    main: 'dist/js-data-http.min.js'
  },
  'js-data-jsonapi': {
    main: 'dist/js-data-jsonapi.js'
  },
  'traceur': {
    main: 'bin/traceur.js'
  },
  'xhrinterceptor': {
    format: 'cjs',
    defaultExtension: 'js',
    main: 'index.js'
  },
  'methods': {
    format: 'cjs',
    defaultExtension: 'js',
    main: 'index.js'
  },
  'http': {
    format: 'cjs',
    defaultExtension: 'js',
    main: 'index.js'
  }

};

////////////////////////////////////////////////////////////////////////////////////////////////
/***********************************************************************************************
 * Everything underneath this line is managed by the CLI.
 **********************************************************************************************/
const barrels: string[] = [
  // Angular specific barrels.
  '@angular/core',
  '@angular/common',
  '@angular/compiler',
  '@angular/http',
  '@angular/router',
  '@angular/platform-browser',
  '@angular/platform-browser-dynamic',


  // Thirdparty barrels.
  'rxjs',
  'xhr-interceptor',
  'xhrinterceptor',

  // App specific barrels.
  'app',
  'app/shared'
  /** @cli-barrel */
];

const cliSystemConfigPackages: any = {};
barrels.forEach((barrelName: string) => {
  cliSystemConfigPackages[barrelName] = { main: 'index' };
});

/** Type declaration for ambient System. */
declare var System: any;

// Apply the CLI SystemJS configuration.
System.config({
  map: {
    '@angular': 'vendor/@angular',
    'rxjs': 'vendor/rxjs',
    'main': 'main.js',
    'xhrinterceptor': 'vendor/xhr-interceptor'
  },
  packages: cliSystemConfigPackages
});

// Apply the user's configuration.
System.config({ map, packages });

// angular-cli-build.js
/* global require, module */

var Angular2App = require('angular-cli/lib/broccoli/angular2-app');
var renderTemplates = require("broccoli-render-template");

module.exports = function(defaults) {
  var app = new Angular2App(defaults, {
    // these all get copied to dist/vendor
    vendorNpmFiles: [
      'systemjs/dist/system-polyfills.+(js|js.map)',
      'systemjs/dist/system.src.js',
      'zone.js/dist/**/*.+(js|js.map)',
      'es6-shim/es6-shim.js',
      'reflect-metadata/**/*.+(js|js.map)',
      'rxjs/**/*.+(js|js.map)',
      '@angular/**/*.+(js|js.map)',
      'js-data/**/*.+(js|js.map)',
      'js-data-http/**/*.+(js|js.map)',
      'js-data-jsonapi/**/*.js',
      'traceur/**/*.js',

      // for testing...
      'xhr-interceptor/**/*.js',
      'methods/**/*.js',
      'http-browserify/**/*.js',
    ],
  });

  // // Render all of the jade we have available
  var renderedJadeTree  = renderTemplates(app, { pretty:true });
  return renderedJadeTree;
};

// src/typings.d.ts
/// <reference path="../typings/browser.d.ts" />
declare var module: { id: string };
declare function require(id: string): any;
declare module 'xhrinterceptor' {
  //var Interceptor: any;
  export class Interceptor{
  }
}

// src/app/data/store.spec.ts
import { it, describe, expect } from '@angular/core/testing';
import { store, basePath } from './store';
import { Interceptor } from 'xhrinterceptor';

describe('JSData: Adaptor Configuration', () => {
  let interceptor;

  beforeEach(() => {
    // debugger;
    interceptor = new Interceptor;
  });
  ...

screenshot of dist/

So, things are getting copied to dist/vendor, but because dist/vendor/xhr-interceptor has some requirements of things that aren't in its node_module's directory, require is barfing.

Any ideas?

[filipesilva]

Off the top of my head, to resolve dependencies of 3rd party libs you'd need to have SystemJS configured to find them and vendorNpmFiles copying them over. SystemJS should be able to find them inside dist and load them in whatever way configured.

If you provide me with a repo where this happens, I can try to debug, but from the info here I don't think I can figure it out without digging into it.

[NullVoxPopuli]

@filipesilva I have it doing that. But the import / require statements aren't finding them.

[filipesilva]

Can you give me either a repo or a set of repro steps I can follow where that happens?

[NullVoxPopuli]

Yeah, all you have to do is:

import { it, describe, expect } from '@angular/core/testing';
import { Interceptor } from 'xhrinterceptor';

describe('JSData: Adaptor Configuration', () => {
  let interceptor;

  beforeEach(() => {
    interceptor = new Interceptor;
  });

the error happens on new Interceptor

I can't give a repo cause for some reason this is a closed source project, but all the information you need is in the original post, (as well as knowing that the error is triggered by new Interceptor)

[filipesilva]

Can you make a bare bones repo with the minimum code needed to reproduce the problem?

I don't really know any of these libraries, and I can't really say I have time to read up on them to try and figure out how they are installed, configured and used. There's also a lot of configuration that you listed that seems tangential to the problem, so copy pasting those files might end up causing more issues in and of itself.

I know it's a bit of a bother but if you don't do it, then it means that I have to do it, and at that point I have to decide if this issue is really worth the time that I could be spending on other, more well defined and reproducible issues.

[NullVoxPopuli]

kk, I'll post a link here when I get a repo up

[filipesilva]

Cheers, and thanks!

[NullVoxPopuli]

I'm getting a slightly different error now, but here is the repo (and teh diff from a new ng new): NullVoxPopuli/angular-cli-3rdparty-nested-dependency-issue@0d16d31

[filipesilva]

Think I've found the issue. For karma, you were adding files in karma.conf.js - but karma is shimmed to use SystemJS for all app files. That means that the SystemJS config is what ultimately specifies where the import'd files are.

Note: you were missing http-browserify from package.json.

I think I was able to configure some of the dependencies:

• system-config.ts:

(...)
/** Map relative paths to URLs. */
const map: any = {
  // for testing
  'xhrinterceptor': 'vendor/xhr-interceptor',
  'methods': 'vendor/methods',
  'http': 'vendor/http-browserify',
  'path-to-regexp': 'vendor/path-to-regexp',
  'object-assign': 'vendor/object-assign',
  'lodash-node': 'vendor/lodash-node',
  'fake-xml-http-request': 'vendor/fake-xml-http-request',
  'isarray': 'vendor/isarray'
};

/** User packages configuration. */
const packages: any = {
  'xhrinterceptor': { format: 'cjs', defaultExtension: 'js', main: 'index.js' },
  'methods': { format: 'cjs', defaultExtension: 'js', main: 'index.js' },
  'http': { format: 'cjs', defaultExtension: 'js', main: 'index.js'},
  'path-to-regexp': { format: 'cjs', defaultExtension: 'js', main: 'index.js' },
  'object-assign': { format: 'cjs', defaultExtension: 'js', main: 'index.js'},
  'lodash-node': { format: 'cjs', defaultExtension: 'js', main: 'index.js'},
  'fake-xml-http-request': { format: 'cjs', defaultExtension: 'js', main: 'fake_xml_http_request.js'},
  'isarray': { format: 'cjs', defaultExtension: 'js', main: 'index.js'},
};
(...)

• angular-cli-build.js:

(...)
vendorNpmFiles: [
      'systemjs/dist/system-polyfills.js',
      'systemjs/dist/system.src.js',
      'zone.js/dist/**/*.+(js|js.map)',
      'es6-shim/es6-shim.js',
      'reflect-metadata/**/*.+(ts|js|js.map)',
      'rxjs/**/*.+(js|js.map)',
      '@angular/**/*.+(js|js.map)',

      // for testing...
      'xhr-interceptor/**/*.js',
      'methods/**/*.js',
      'http-browserify/**/*.js',
      'path-to-regexp/**/*.js',
      'object-assign/**/*.js',
      'lodash-node/**/*.js',
      'fake-xml-http-request/**/*.js',
      'methods/**/*.js',
      'isarray/**/*.js',
    ],
(...)

• config/karma.conf.js: remove changes introduced in NullVoxPopuli/angular-cli-3rdparty-nested-dependency-issue@0d16d31

At this point though, karma warns of other missing dependencies:

21 06 2016 04:59:57.346:WARN [web-server]: 404: /base/dist/events
21 06 2016 04:59:57.349:WARN [web-server]: 404: /base/dist/url
21 06 2016 04:59:57.372:WARN [web-server]: 404: /base/dist/stream
21 06 2016 04:59:57.373:WARN [web-server]: 404: /base/dist/Base64
21 06 2016 04:59:57.375:WARN [web-server]: 404: /base/dist/inherits
21 06 2016 04:59:57.384:WARN [web-server]: 404: /base/dist/util

So more configuration would be needed. Which is a hassle, as you said, but I kinda don't see how that can be avoided while loading Angular 2 using SystemJS and using stuff that just has a lot of dependencies. Maybe there's some advanced SystemJS-fu of the likes used in JSPM that help in this.

We are experimenting with webpack, and considering moving to it if it proves to be viable. That could help in this case, I hope.

[NullVoxPopuli]

Which is a hassle, as you said, but I kinda don't see how that can be avoided while loading Angular 2 using SystemJS and using stuff that just has a lot of dependencies.

How does ember-cli handle this problem? (with ember addons, I've never had to worry about an addon's dependencies)

We are experimenting with webpack, and considering moving to it if it proves to be viable. That could help in this case, I hope.

cool

thanks for looking in to my repo!

[filipesilva]

I think ember-cli has a mechanism for adding 3rd party libs to the broccoli build as well, and then it bundles them all up into a vendor.js for builds. I might be wrong, and it might have changed. Haven't looked at it in a while.

Is it ok for me to close this issue btw?

[NullVoxPopuli]

Yeah, this specific issue is at least mostly resolved with an absurd amount of hacks (really just the same one, until I stop getting errors).

I'd like to know your opinion on a couple things though, and if they deserve their own issues:

• Should Angular-CLI do what Ember-CLI does, and figure out the 3rd party libs? (I think it should, cause this helps with dev-experience)
• Should Angular-CLI have seperate builds setups for production and testing? (I think right now, everything I have to do to get the testing libs to work are included in the production build)

[filipesilva]

I think the CLI should figure them deps out, yes. That's the reason that led us to invest a considerable amount of time in the ongoing webpack experiment. I'm not sure how that could be done with SystemJS+broccoli, but maybe that was just a sign that they don't play that well together in some scenarios. There is discussion about it here #882.

The build setups is a more complicated question. You did have to add code that talks about these libs in the SystemJS config and the vendorNpmFiles array - but production builds will bundle everything that is used, and apply tree shaking. So none of those libs ends up in a production build, although your SystemJS config will still talk about them.

But I do believe the question of environments is a bit more complicated than that:

• You can have different build environments, which are basically sets of config that define how a build should be done (minify, bundle, sourcemaps, etc)
• You can also have different runtime environments, that are usually managed via some kind of drop in configuration that tells the app the api endpoint, service keys, etc. In Angular 2, you also tell it to run in production mode with optimizes change detection.

At the moment the CLI implements a mix of both. If you do ng buid -prod you're get a fully minimized and bundled build, and dropping in the prod.environment.ts as environment.ts. You can also do ng build --env=something and that will not affect the build, but will drop in something.environment.ts if it exists.

[NullVoxPopuli]

ok, cool. thanks for explaining the build environment stuff.

re: SystemJS + Broccoli, I know ember-cli uses broccoli and babel (just looking through their package.json ), and that works really well, though they are only transpiling ES6 to ES5, and not Typescript to ES5. so... idk.

Also, tree shaking is pretty cool.

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}