using dataforge in angular #3

capaneus · 2018-05-24T18:41:42Z

hi Ashley,
I'm building an angular 6 application recently and was looking for some data processing lib. So happy to find data-forge as I used pandas and it's a great lib on python.
Would you mind put some instruction on how to import data-forge into angular to be used there?

ashleydavis · 2018-05-25T00:50:39Z

Hi, I haven't used AngularJS since version 1, so please be aware that any instructions I have for you might be out of date.

For a start, Data-Forge is TypeScript so you might just be able to install it via npm then import it into your AngularJS app as you would any other TypeScript module.

I've used Data-Forge when I've made apps using AngularJS v1+ and I do have some instructions that will work for that.

Browser installation: https://github.com/data-forge/data-forge-ts/blob/master/docs/guide.md#browser-installation-and-setup

AngularJS v1+ usage: https://github.com/data-forge/data-forge-ts/blob/master/docs/guide.md#working-with-http-in-angularjs

Hope this helps. Please let me know how it goes. Maybe you could help me update the instructions noce you figure it out?

yiu31802 · 2018-05-27T08:48:31Z

This worked perfectly to me, this lib can be "the" de-facto standard of dataframe in Angular! Thank you @ashleydavis so much, I will try to spread this library within my team:-)

import { DataFrame } from 'data-forge'

@Component({
selector: 'app-top',
templateUrl: './main.html',
styleUrls: ['./main.css']
})
export class TopPageComponent implements OnInit {


public constructor(
){}

public ngOnInit(): void {
    let df = new DataFrame({
        columns: {
            'regiment': ['Nighthawks', 'Nighthawks', 'Nighthawks', 'Nighthawks', 'Dragoons', 'Dragoons', 'Dragoons', 'Dragoons', 'Scouts', 'Scouts', 'Scouts', 'Scouts'],
            'company': ['1st', '1st', '2nd', '2nd', '1st', '1st', '2nd', '2nd','1st', '1st', '2nd', '2nd'],
            'TestScore': [4, 24, 31, 2, 3, 4, 24, 31, 2, 3, 2, 3]
        }
    });
    const pivotted = df.pivot(["regiment", "company"], "TestScore", testScores => testScores.average());
    console.log(pivotted.toArray())

ashleydavis · 2018-05-27T08:53:01Z

Thanks mate! I'll add that to the docs.

Don't forget to star the repo.

You might also be interested to know that I'm writing a book on working with data in JavaScript. Please help spread the word!

capaneus · 2018-05-29T15:36:52Z

nah, I'm not super lucky :-(
@yiu31802, do you mind share a bit more how you setup it in the project? I tried different options, it gives me:

ERROR in ./node_modules/data-forge/build/index.js
Module not found: Error: Can't resolve 'fs' in 'E:\MyCode\flaskagular\project_planning\app\static\node_modules\data-forge\build'
ERROR in ./node_modules/data-forge/build/lib/dataframe.js
Module not found: Error: Can't resolve 'fs' in 'E:\MyCode\flaskagular\project_planning\app\static\node_modules\data-forge\build\lib'

and did some search seems it's normal for angular to not support fs...
angular/angular-cli#8272

how did you solve this problem?

yiu31802 · 2018-05-29T22:02:13Z

Hi, @capaneus
Weird, I have not met that error at all. My environment is Angular 5.0.5. But it makes also sense that Angular should not support fs in the browser... I will check my environment later and update you.

ashleydavis · 2018-05-30T03:43:23Z

Yeah you can't load files from the file system when running in the browser (whether you are using AngularJS, React, etc doesn't matter).

You can only use the readFile/writeFile functions under Node.js.

Don't use those functions in your code and it should work under AngularJS for you.

capaneus · 2018-06-04T14:16:11Z

hi Ashley,
thanks and back to js again :-) I actually didn't use any of the fs function in the code, what I used to test is the code from yiu31802. It comes at the compiling phase, seems it complained about something in dataframe.js which I haven't triggered at all...

ashleydavis · 2018-06-04T21:14:12Z

That's odd. Can you please give me more information about how you are compiling it?

ashleydavis · 2018-06-11T23:24:31Z

Please give me more information about how you are trying to use Data-Forge with AngularJS. Otherwise I'll close this issue.

maykon-oliveira · 2018-06-21T00:25:42Z

I have de same problem!
ERROR in ./node_modules/data-forge/build/index.js Module not found: Error: Can't resolve 'fs' in 'C:\Users\Maykon Oliveira\Documents\analise-das-escolas\node_modules\data-forge\build' ERROR in ./node_modules/data-forge/build/lib/dataframe.js Module not found: Error: Can't resolve 'fs' in 'C:\Users\Maykon Oliveira\Documents\analise-das-escolas\node_modules\data-forge\build\lib'

I don't know what I do, I need this library. Has anyone managed to solve it?

ashleydavis · 2018-06-21T00:33:00Z

Please share the broken code with me on github so I can run it and debug.

maykon-oliveira · 2018-06-21T00:37:15Z

The project repository
I'm at the beginning

ashleydavis · 2018-06-21T03:49:30Z

@maykon-oliveira I got your code, installed dependencies and then ran it using npm start.

I see a bunch of errors in the console, but I don't think any of them relate to Data-Forge.

Can you give me more information about the problem you are experiencing?

maykon-oliveira · 2018-06-21T12:12:12Z

I was trying to fix it, I changed dependency to continue with the project. This code is a new commit, the error was in the previous one. I gave up to use data-forge.

yiu31802 · 2018-06-21T23:17:19Z

Hi, @maykon-oliveira and @ashleydavis .

When I tried to add data-forge on a fresh Angular repository, indeed, the same error with fs occurred. I am trying to figure out which of many dependency package may implicitly resolve the issue in my data-forge working project. Note that our project is using DataForge in Angular even today flawlessly...

ashleydavis · 2018-06-22T01:08:16Z

Thank @yiu31802 please let me know what you come up with.

If anyone else finds a problem please submit working code for me to troubleshoot.

I used to use Angular version 1, but I've switched to React since then and I don't have time at the moment to learn the latest version of Angular. So I'll appreciate anyone who can submit code that demonstrates an issue.

yiu31802 · 2018-06-22T23:40:50Z

Hi, @ashleydavis, @capaneus @maykon-oliveira

I think I understood what happened; it is simply a version issue. With Angular 5.x, it works, but something is getting wrong with Angular 6.x versions.

Here's the sample code, which is built on top of pure Angular 5 and adding data-forge.
https://github.com/yiu31802/sample-angular5-with-data-forge

ashleydavis · 2018-06-25T06:02:22Z

Thanks @yiu31802 , hoping to look into it this week!

ashleydavis · 2018-06-26T01:08:40Z

@yiu31802 I've got your project, installed dependencies, run npm start and viewed the web page.

It says 'app works!'.

Does this have to be upgraded to Angular 6 to cause the problem? How do I do that? Is it just a matter of doing an npm install of the latest version?

ashleydavis · 2018-06-26T01:20:49Z

So I've done a migration of that project to Angular 6 using this guide.

I've reproduced the issue with Data-Forge. Thanks for submitting the code.

ashleydavis · 2018-06-26T01:30:38Z

So it seems you can't use 'fs' any more in Angular 6.

There's an open issue about it (related to Electron): angular/angular-cli#10681
There's some workarounds suggested in that issue, but I haven't tried them myself.

Other libraries are experiencing this problem, not just Data-Forge.

I'm going to sit on this for a bit. Angular might fix the problem. Otherwise I'm may have to deliver a separate version of Data-Forge without 'fs'.

yiu31802 · 2018-06-26T02:08:23Z

Got it, @ashleydavis. Thanks!

I as a user of both Angular and Data-Forge, I stay with the combination of Angular5 and Data-Forge. Data-Forge simply works perfectly on Angular5.

ashleydavis · 2018-09-10T04:44:37Z

I may have a solution to this problem for AngularJS v6!

While looking at the code and considering what I was going to do about this issue I spotted a global include of 'fs'. It's possible that this is what is causing your problem!

Please update to using Data-Forge 1.2.6 and let me know if it fixes your problem for Angularjs v6.

yiu31802 · 2018-09-10T05:54:43Z

@ashleydavis

Thanks, I just tried data-forge@1.2.6 but I am guessing your lib is trying to yet fs, according to the error message:

ERROR in ./node_modules/data-forge/build/index.js
Module not found: Error: Can't resolve 'fs' in '/home/CORPUSERS/ss/workspace/xx-individuals/dataforge-6/myapp/node_modules/data-forge/build'
ERROR in ./node_modules/data-forge/build/lib/dataframe.js
Module not found: Error: Can't resolve 'fs' in '/home/CORPUSERS/ss/workspace/xx-individuals/dataforge-6/myapp/node_modules/data-forge/build/lib'
ℹ ｢wdm｣: Failed to compile.
^C

Here's the UNIX command list:

 2060  mkdir dataforge-6
 2061  cd dataforge-6/
 2074  yarn add @angular/cli
 2077  ./node_modules/@angular/cli/bin/ng new myapp
 2079  cd myapp/
 2081  yarn install
 2082  ../node_modules/@angular/cli/bin/ng -v
 2083  ../node_modules/@angular/cli/bin/ng serve
 2084  yarn add data-forge
 2085  nano src/app/app.component.ts # Add data-forge test code by referring to the previous commit
 2086  ../node_modules/@angular/cli/bin/ng serve

The previous commit link: https://github.com/yiu31802/sample-angular5-with-data-forge/blob/bff58198d62937f63960c136a5d5c8e7229da8bb/src/app/components/one-test-view/app.component.ts

ashleydavis · 2018-09-10T21:46:12Z

Thanks for trying it out.

I'm moving forward with plans to fully extract 'fs' to a separate module.

ashleydavis · 2018-09-11T01:37:08Z

I've started the process of removing 'fs' from Data-Forge.

This will take some time to complete, but already I have published a 'beta' version of Data-Forge with 'fs' removed.

Can you please try using the beta package of Data-Forge with your project instead of the regular package:

http://npmjs.com/package/data-forge-beta

Please let me know how that goes. If that works for you then I'll know it's worth completing the change.

yiu31802 · 2018-09-12T05:28:55Z

@ashleydavis

Thanks, I tried it out. The situation was changed, but still not working correctly.

The compilation error message disappeared
A new error appeared and browser app is crashing due to that, because if I remove the DataFrame variable definition, Angular app works correctly.

TypeError: globalContext[name] is undefined[Learn More] sugar-core.js:791

I have no clue about this message at all.

ashleydavis · 2018-09-17T03:10:20Z

Thanks for trying it.

Do you have updated example code for me to look at? I'd like to get this working!

newbleach · 2018-09-25T08:48:34Z

Thanks for trying it.

Do you have updated example code for me to look at? I'd like to get this working!

I'm using data-forge-beta
with angular6 and Node 8.11.1
sugar-core.js:791 Uncaught TypeError: Cannot read property 'prototype'
I keep looking for i a good version that's work with angular6 :)

ashleydavis · 2018-09-25T09:14:50Z

The 'fs' code has been extracted from Data-Forge, so it should be close to working.

Please submit example code so that I can debug, otherwise let me know what I have to change to get it working.

I don't use Angularjs (sorry React only) so I'm not sure how to reproduce the problem.

yiu31802 · 2018-11-14T00:42:13Z

Hello, @ashleydavis . Do you have any update on this? I just re-tried it with 1.2.7 version but I still met the same symptom as I described above (#3 (comment))

ashleydavis · 2018-11-14T00:52:44Z

Have you tried the beta version with the 'fs' dependency extracted?

http://npmjs.com/package/data-forge-beta

ashleydavis · 2018-11-14T00:53:22Z

In a week or two I'll be publishing the beta version.

yiu31802 · 2018-11-14T00:59:16Z

As commented later (#3 (comment)), I already tried and failed with 1.3.0 there.

ashleydavis · 2018-11-14T01:49:35Z

Can you please submit code that demonstrates the problem. Also please submit any error messages you are receiving that might be relevant.

yiu31802 · 2018-11-14T02:10:07Z

You first build a plain Angular app without installing any in your system. It will take approx. 10 minutes to reproduce the error from scratch:

mkdir dataforge-6
cd dataforge-6/
yarn add @angular/cli
./node_modules/@angular/cli/bin/ng new myapp # Hit Enter during the dialogue

Then one more local installation of packages and serve the page

cd myapp/
yarn install
../node_modules/@angular/cli/bin/ng -v
../node_modules/@angular/cli/bin/ng serve

Make sure Angular is up at localhost:4200
Then adding dataforge, and rerun the server. In the browser, you can find error message in the console log.

yarn add data-forge-beta
nano src/app/app.component.ts # See my code below
../node_modules/@angular/cli/bin/ng serve # Page is up but no content to show because of error

Use this code in src/app/app.component.ts

import { DataFrame } from 'data-forge-beta'

import { Component } from '@angular/core';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent {
  title = 'myapp';

  public ngOnInit(): void {
    let df = new DataFrame({
        columns: {
            'regiment': ['Nighthawks', 'Nighthawks', 'Nighthawks', 'Nighthawks', 'Dragoons', 'Dragoons', 'Dragoons', 'Dragoons', 'Scouts', 'Scouts', 'Scouts', 'Scouts'],
            'company': ['1st', '1st', '2nd', '2nd', '1st', '1st', '2nd', '2nd','1st', '1st', '2nd', '2nd'],
            'TestScore': [4, 24, 31, 2, 3, 4, 24, 31, 2, 3, 2, 3]
        }
    });
    const pivotted = df.pivot(["regiment", "company"], "TestScore", testScores => testScores.average());
    console.log(pivotted.toArray())
  }
}

ashleydavis · 2018-11-14T03:12:27Z

Please put it in Gist or repo and give me instructions on how to boot it up.

I don't work with AngularJS and I don't have much time to test the problem for you.

yiu31802 · 2018-11-14T03:19:01Z

I understand that you are a busy person, but putting it in Git is not a reasonable choice as you are not using Angular. I am sure this way is much faster than (1) Installation of Angular in your system (2) Git cloning of the test code.

The instruction above is a way to bypass the system installation of Angular, and putting every files in Git is meaningless cause it mostly deals with package management.

ashleydavis · 2018-11-14T03:20:39Z

I'd just like to be able to npm install and npm start on a GitHub repo and be able to reproduce in 1-2 mins.

yiu31802 · 2018-11-14T03:25:39Z

I do not have clue now other than that unless your system has ng (Angular's) command, but let me think. But I will share the error I am seeing in data-forge.

./node_modules/sugar/string/index.js
index.js:4
__webpack_require__
bootstrap:78
./node_modules/sugar/index.js
index.js:3
__webpack_require__bootstrap:78
./node_modules/data-forge-beta/build/index.js
index.js:13:12

ashleydavis · 2018-11-14T04:03:14Z

I'm happy to look at this problem. You just need to provide code in a GitHub repo.

I should be able to clone the repo, then type npm install to install required dependencies and then npm start to start the application and have it immediately reproduce the problem - I'm happy to follow other steps as long the code is ready to go and the command line for install and run are straight forward.

Sorry I should have had a guide for people that submit issues to say the requirements and make it clear what I need from you.

yiu31802 · 2018-11-14T05:17:55Z

Please have a try with this:

git clone git@github.com:yiu31802/sample-angular-data-forge.git -b issue-3
npm install
npm start

Sorry that I was too lazy to put things in Git repository.

ashleydavis · 2018-11-14T06:45:04Z

The git clone doesn't work for me.

I'm getting a 'permission denied' error.

Do you have a link to the repo so that I can check it from the browser?

yiu31802 · 2018-11-14T06:58:59Z

Weird, it is a public repository at GitHub as you see, can you recheck the repository page?

https://github.com/yiu31802/sample-angular-data-forge

ashleydavis · 2018-11-14T07:21:00Z

Got it thanks.

ashleydavis · 2018-11-14T08:07:40Z

This seems to be a problem with using the Sugar library that Data-Forge is depending on.

Trying to track it down.

ashleydavis · 2018-11-14T08:31:14Z

I believe this is the same problem:

andrewplummer/Sugar#632

ashleydavis · 2018-11-14T08:34:34Z

Here is a workaround for you:

Add this line of code to your polyfills.ts:

window['global'] = window;
Source: andrewplummer/Sugar#632 (comment)

ashleydavis · 2018-11-14T08:35:16Z

I'll leave this issue open until the problem has been resolved with Sugar, then I'll update Sugar in Data-Forge and do a new release.

ashleydavis · 2018-11-14T08:36:25Z

@yiu31802 thanks so much for your help to reproduce this problem.

yiu31802 · 2018-11-15T02:29:08Z

Thanks @ashleydavis and I also confirmed it works with your workaround fix. I am happy that this package starts to work again in the latest versions of Angular:-)

ashleydavis · 2018-11-23T22:04:01Z

Data-Forge 1.3.0 is officially released. The file system functions have been extracted from the core API to the new file system module: https://github.com/data-forge/data-forge-fs

This should help Data-Forge be more usable in the browser and specifically with the latest AngularJS.

If you were using "data-forge-beta" please change over to the regular "data-forge".

ashleydavis · 2019-05-13T02:46:52Z

Closing this issue now. I believe the issue was addressed and has not been discussed since.

ashleydavis closed this as completed May 25, 2018

ashleydavis reopened this Jun 4, 2018

ashleydavis closed this as completed May 13, 2019

using dataforge in angular #3

using dataforge in angular #3

Comments

capaneus commented May 24, 2018

ashleydavis commented May 25, 2018

yiu31802 commented May 27, 2018

ashleydavis commented May 27, 2018

capaneus commented May 29, 2018 • edited Loading

yiu31802 commented May 29, 2018

ashleydavis commented May 30, 2018

capaneus commented Jun 4, 2018

ashleydavis commented Jun 4, 2018

ashleydavis commented Jun 11, 2018

maykon-oliveira commented Jun 21, 2018 • edited Loading

ashleydavis commented Jun 21, 2018

maykon-oliveira commented Jun 21, 2018

ashleydavis commented Jun 21, 2018

maykon-oliveira commented Jun 21, 2018

yiu31802 commented Jun 21, 2018

ashleydavis commented Jun 22, 2018

yiu31802 commented Jun 22, 2018

ashleydavis commented Jun 25, 2018

ashleydavis commented Jun 26, 2018

ashleydavis commented Jun 26, 2018

ashleydavis commented Jun 26, 2018

yiu31802 commented Jun 26, 2018

ashleydavis commented Sep 10, 2018

yiu31802 commented Sep 10, 2018

ashleydavis commented Sep 10, 2018

ashleydavis commented Sep 11, 2018

yiu31802 commented Sep 12, 2018

ashleydavis commented Sep 17, 2018

newbleach commented Sep 25, 2018

ashleydavis commented Sep 25, 2018

yiu31802 commented Nov 14, 2018

ashleydavis commented Nov 14, 2018

ashleydavis commented Nov 14, 2018

yiu31802 commented Nov 14, 2018

ashleydavis commented Nov 14, 2018

yiu31802 commented Nov 14, 2018 • edited Loading

ashleydavis commented Nov 14, 2018

yiu31802 commented Nov 14, 2018

ashleydavis commented Nov 14, 2018 • edited Loading

yiu31802 commented Nov 14, 2018 • edited Loading

ashleydavis commented Nov 14, 2018 • edited Loading

yiu31802 commented Nov 14, 2018

ashleydavis commented Nov 14, 2018

yiu31802 commented Nov 14, 2018

ashleydavis commented Nov 14, 2018

ashleydavis commented Nov 14, 2018

ashleydavis commented Nov 14, 2018

ashleydavis commented Nov 14, 2018 • edited Loading

ashleydavis commented Nov 14, 2018

ashleydavis commented Nov 14, 2018

yiu31802 commented Nov 15, 2018

ashleydavis commented Nov 23, 2018 • edited Loading

ashleydavis commented May 13, 2019

capaneus commented May 29, 2018 •

edited

Loading

maykon-oliveira commented Jun 21, 2018 •

edited

Loading

yiu31802 commented Nov 14, 2018 •

edited

Loading

ashleydavis commented Nov 14, 2018 •

edited

Loading

yiu31802 commented Nov 14, 2018 •

edited

Loading

ashleydavis commented Nov 14, 2018 •

edited

Loading

ashleydavis commented Nov 14, 2018 •

edited

Loading

ashleydavis commented Nov 23, 2018 •

edited

Loading