How do I resolve dependencies of 3rd party libs for testing?

[NullVoxPopuli]

Env Info

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 15.10
Release:    15.10
Codename:   wily

$ ng --version
Could not start watchman; falling back to NodeWatcher for file system events.
Visit http://ember-cli.com/user-guide/#watchman for more info.
angular-cli: 1.0.0-beta.5
node: 5.10.1
os: linux x64

Steps

• Create angular-cli app.
• Use JSData for a data layer, cause production apps are complex, and need a data layer.
• Try to use xhr-interceptor to mock json returned from the server because JSData doesn't use @angular/http, but uses axios for async http. An example of how this is done can be seen with ember-cli-mirage. Previously, I tried pretender, but had similar problems with nested dependencies.
• run ng test
• get errors similar to Uncaught error: Could not find module 'methods' from 'path/to/xhr-interceptor/index.js'

All I want to do is mock JSON responses from a fake server (one that isn't actually running, but is just scoped to an individual test).

Here are some relevant configuration things I've done:

// config/karma.conf.js
module.exports = function (config) {
  config.set({
    basePath: '../',
    frameworks: ['jasmine', 'commonjs'],
    plugins: [
      require('karma-jasmine'),
      require('karma-chrome-launcher'),
      require('karma-spec-reporter'),
      require('karma-coverage'),
      require('karma-commonjs'),
    ],
    customLaunchers: {
      // chrome setup for travis CI using chromium
      Chrome_travis_ci: {
        base: 'Chrome',
        flags: ['--no-sandbox']
      }
    },
    files: [
      { pattern: 'dist/vendor/es6-shim/es6-shim.js', included: true, watched: false },
      { pattern: 'dist/vendor/zone.js/dist/zone.js', included: true, watched: false },
      { pattern: 'dist/vendor/reflect-metadata/Reflect.js', included: true, watched: false },
      { pattern: 'dist/vendor/systemjs/dist/system-polyfills.js', included: true, watched: false },
      { pattern: 'dist/vendor/systemjs/dist/system.src.js', included: true, watched: false },
      { pattern: 'dist/vendor/zone.js/dist/async-test.js', included: true, watched: false },
      { pattern: 'dist/vendor/xhr-interceptor/**/*', served: true, watched: false },
      { pattern: 'dist/vendor/methods/**/*', served: true, watched: false },
      'dist/vendor/methods/index.js',
      'dist/vendor/http-browserify/index.js',
      { pattern: 'config/karma-test-shim.js', included: true, watched: true },
      // Distribution folder.
      { pattern: 'dist/**/*', included: false, watched: true }
    ],
    exclude: [
      // Vendor packages might include spec files. We don't want to use those.
      'dist/vendor/**/*.spec.js'
    ],
    preprocessors: {
      'dist/vendor/http-browserify/**/*.js': ['commonjs'],
      'dist/vendor/methods/**/*.js': ['commonjs'],
      'dist/vendor/xhr-interceptor/**/*.js': ['commonjs'],
      'dist/app/**/*.js': 'coverage',
    },
    reporters: [
      'spec',
      // 'coverage'
    ],
    coverageReporter: {
      type : 'html',
      dir : 'coverage/'
    },
    port: 9876,
    colors: true,
    logLevel: config.LOG_INFO,
    autoWatch: true,
    browsers: ['Chrome'],
    singleRun: false
  });
};

// src/system-config.ts
/***********************************************************************************************
 * User Configuration.
 **********************************************************************************************/
/** Map relative paths to URLs. */
const map: any = {
  '@portal': '../src/app',
  'js-data': 'vendor/js-data',
  'js-data-http': 'vendor/js-data-http',
  'js-data-jsonapi': 'vendor/js-data-jsonapi',
  'traceur': 'vendor/traceur',

  // for testing
  'xhrinterceptor': 'vendor/xhr-interceptor',
  'methods': 'vendor/methods',
  'http': 'vendor/http-browserify',
};

const paths: any = {

}

/** User packages configuration. */
const packages: any = {
  'js-data': {
    main: 'dist/js-data.min.js'
  },
  'js-data-http': {
    main: 'dist/js-data-http.min.js'
  },
  'js-data-jsonapi': {
    main: 'dist/js-data-jsonapi.js'
  },
  'traceur': {
    main: 'bin/traceur.js'
  },
  'xhrinterceptor': {
    format: 'cjs',
    defaultExtension: 'js',
    main: 'index.js'
  },
  'methods': {
    format: 'cjs',
    defaultExtension: 'js',
    main: 'index.js'
  },
  'http': {
    format: 'cjs',
    defaultExtension: 'js',
    main: 'index.js'
  }

};

////////////////////////////////////////////////////////////////////////////////////////////////
/***********************************************************************************************
 * Everything underneath this line is managed by the CLI.
 **********************************************************************************************/
const barrels: string[] = [
  // Angular specific barrels.
  '@angular/core',
  '@angular/common',
  '@angular/compiler',
  '@angular/http',
  '@angular/router',
  '@angular/platform-browser',
  '@angular/platform-browser-dynamic',


  // Thirdparty barrels.
  'rxjs',
  'xhr-interceptor',
  'xhrinterceptor',

  // App specific barrels.
  'app',
  'app/shared'
  /** @cli-barrel */
];

const cliSystemConfigPackages: any = {};
barrels.forEach((barrelName: string) => {
  cliSystemConfigPackages[barrelName] = { main: 'index' };
});

/** Type declaration for ambient System. */
declare var System: any;

// Apply the CLI SystemJS configuration.
System.config({
  map: {
    '@angular': 'vendor/@angular',
    'rxjs': 'vendor/rxjs',
    'main': 'main.js',
    'xhrinterceptor': 'vendor/xhr-interceptor'
  },
  packages: cliSystemConfigPackages
});

// Apply the user's configuration.
System.config({ map, packages });

// angular-cli-build.js
/* global require, module */

var Angular2App = require('angular-cli/lib/broccoli/angular2-app');
var renderTemplates = require("broccoli-render-template");

module.exports = function(defaults) {
  var app = new Angular2App(defaults, {
    // these all get copied to dist/vendor
    vendorNpmFiles: [
      'systemjs/dist/system-polyfills.+(js|js.map)',
      'systemjs/dist/system.src.js',
      'zone.js/dist/**/*.+(js|js.map)',
      'es6-shim/es6-shim.js',
      'reflect-metadata/**/*.+(js|js.map)',
      'rxjs/**/*.+(js|js.map)',
      '@angular/**/*.+(js|js.map)',
      'js-data/**/*.+(js|js.map)',
      'js-data-http/**/*.+(js|js.map)',
      'js-data-jsonapi/**/*.js',
      'traceur/**/*.js',

      // for testing...
      'xhr-interceptor/**/*.js',
      'methods/**/*.js',
      'http-browserify/**/*.js',
    ],
  });

  // // Render all of the jade we have available
  var renderedJadeTree  = renderTemplates(app, { pretty:true });
  return renderedJadeTree;
};

// src/typings.d.ts
/// <reference path="../typings/browser.d.ts" />
declare var module: { id: string };
declare function require(id: string): any;
declare module 'xhrinterceptor' {
  //var Interceptor: any;
  export class Interceptor{
  }
}

// src/app/data/store.spec.ts
import { it, describe, expect } from '@angular/core/testing';
import { store, basePath } from './store';
import { Interceptor } from 'xhrinterceptor';

describe('JSData: Adaptor Configuration', () => {
  let interceptor;

  beforeEach(() => {
    // debugger;
    interceptor = new Interceptor;
  });
  ...

screenshot of dist/

So, things are getting copied to dist/vendor, but because dist/vendor/xhr-interceptor has some requirements of things that aren't in its node_module's directory, require is barfing.

Any ideas?

[filipesilva]

Off the top of my head, to resolve dependencies of 3rd party libs you'd need to have SystemJS configured to find them and vendorNpmFiles copying them over. SystemJS should be able to find them inside dist and load them in whatever way configured.

If you provide me with a repo where this happens, I can try to debug, but from the info here I don't think I can figure it out without digging into it.

[NullVoxPopuli]

@filipesilva I have it doing that. But the import / require statements aren't finding them.

[filipesilva]

Can you give me either a repo or a set of repro steps I can follow where that happens?

[NullVoxPopuli]

Yeah, all you have to do is:

import { it, describe, expect } from '@angular/core/testing';
import { Interceptor } from 'xhrinterceptor';

describe('JSData: Adaptor Configuration', () => {
  let interceptor;

  beforeEach(() => {
    interceptor = new Interceptor;
  });

the error happens on new Interceptor

I can't give a repo cause for some reason this is a closed source project, but all the information you need is in the original post, (as well as knowing that the error is triggered by new Interceptor)

[filipesilva]

Can you make a bare bones repo with the minimum code needed to reproduce the problem?

I don't really know any of these libraries, and I can't really say I have time to read up on them to try and figure out how they are installed, configured and used. There's also a lot of configuration that you listed that seems tangential to the problem, so copy pasting those files might end up causing more issues in and of itself.

I know it's a bit of a bother but if you don't do it, then it means that I have to do it, and at that point I have to decide if this issue is really worth the time that I could be spending on other, more well defined and reproducible issues.

[NullVoxPopuli]

kk, I'll post a link here when I get a repo up

[filipesilva]

Cheers, and thanks!

[NullVoxPopuli]

I'm getting a slightly different error now, but here is the repo (and teh diff from a new ng new): NullVoxPopuli/angular-cli-3rdparty-nested-dependency-issue@0d16d31

[filipesilva]

Think I've found the issue. For karma, you were adding files in karma.conf.js - but karma is shimmed to use SystemJS for all app files. That means that the SystemJS config is what ultimately specifies where the import'd files are.

Note: you were missing http-browserify from package.json.

I think I was able to configure some of the dependencies:

• system-config.ts:

(...)
/** Map relative paths to URLs. */
const map: any = {
  // for testing
  'xhrinterceptor': 'vendor/xhr-interceptor',
  'methods': 'vendor/methods',
  'http': 'vendor/http-browserify',
  'path-to-regexp': 'vendor/path-to-regexp',
  'object-assign': 'vendor/object-assign',
  'lodash-node': 'vendor/lodash-node',
  'fake-xml-http-request': 'vendor/fake-xml-http-request',
  'isarray': 'vendor/isarray'
};

/** User packages configuration. */
const packages: any = {
  'xhrinterceptor': { format: 'cjs', defaultExtension: 'js', main: 'index.js' },
  'methods': { format: 'cjs', defaultExtension: 'js', main: 'index.js' },
  'http': { format: 'cjs', defaultExtension: 'js', main: 'index.js'},
  'path-to-regexp': { format: 'cjs', defaultExtension: 'js', main: 'index.js' },
  'object-assign': { format: 'cjs', defaultExtension: 'js', main: 'index.js'},
  'lodash-node': { format: 'cjs', defaultExtension: 'js', main: 'index.js'},
  'fake-xml-http-request': { format: 'cjs', defaultExtension: 'js', main: 'fake_xml_http_request.js'},
  'isarray': { format: 'cjs', defaultExtension: 'js', main: 'index.js'},
};
(...)

• angular-cli-build.js:

(...)
vendorNpmFiles: [
      'systemjs/dist/system-polyfills.js',
      'systemjs/dist/system.src.js',
      'zone.js/dist/**/*.+(js|js.map)',
      'es6-shim/es6-shim.js',
      'reflect-metadata/**/*.+(ts|js|js.map)',
      'rxjs/**/*.+(js|js.map)',
      '@angular/**/*.+(js|js.map)',

      // for testing...
      'xhr-interceptor/**/*.js',
      'methods/**/*.js',
      'http-browserify/**/*.js',
      'path-to-regexp/**/*.js',
      'object-assign/**/*.js',
      'lodash-node/**/*.js',
      'fake-xml-http-request/**/*.js',
      'methods/**/*.js',
      'isarray/**/*.js',
    ],
(...)

• config/karma.conf.js: remove changes introduced in NullVoxPopuli/angular-cli-3rdparty-nested-dependency-issue@0d16d31

At this point though, karma warns of other missing dependencies:

21 06 2016 04:59:57.346:WARN [web-server]: 404: /base/dist/events
21 06 2016 04:59:57.349:WARN [web-server]: 404: /base/dist/url
21 06 2016 04:59:57.372:WARN [web-server]: 404: /base/dist/stream
21 06 2016 04:59:57.373:WARN [web-server]: 404: /base/dist/Base64
21 06 2016 04:59:57.375:WARN [web-server]: 404: /base/dist/inherits
21 06 2016 04:59:57.384:WARN [web-server]: 404: /base/dist/util

So more configuration would be needed. Which is a hassle, as you said, but I kinda don't see how that can be avoided while loading Angular 2 using SystemJS and using stuff that just has a lot of dependencies. Maybe there's some advanced SystemJS-fu of the likes used in JSPM that help in this.

We are experimenting with webpack, and considering moving to it if it proves to be viable. That could help in this case, I hope.

[NullVoxPopuli]

Which is a hassle, as you said, but I kinda don't see how that can be avoided while loading Angular 2 using SystemJS and using stuff that just has a lot of dependencies.

How does ember-cli handle this problem? (with ember addons, I've never had to worry about an addon's dependencies)

We are experimenting with webpack, and considering moving to it if it proves to be viable. That could help in this case, I hope.

cool

thanks for looking in to my repo!

[filipesilva]

I think ember-cli has a mechanism for adding 3rd party libs to the broccoli build as well, and then it bundles them all up into a vendor.js for builds. I might be wrong, and it might have changed. Haven't looked at it in a while.

Is it ok for me to close this issue btw?

[NullVoxPopuli]

Yeah, this specific issue is at least mostly resolved with an absurd amount of hacks (really just the same one, until I stop getting errors).

I'd like to know your opinion on a couple things though, and if they deserve their own issues:

• Should Angular-CLI do what Ember-CLI does, and figure out the 3rd party libs? (I think it should, cause this helps with dev-experience)
• Should Angular-CLI have seperate builds setups for production and testing? (I think right now, everything I have to do to get the testing libs to work are included in the production build)