Huge package size investigation.

[ChALkeR]

Two things to note:

1. I am not associated with npm, nor am I using this package. This was found during mass npm packages downloading.
2. Per Flaw reports url is broken #2 (comment), you said that npm packaging issues that are not related to the code itself could be placed here:

   B) NPM related flaws are practically "packaging" related flaws, not
   flaws that are related to the "raudrohi itself", which means that
   it's not that big of a loss if their descriptions are deleted;

Hi. Your package is to the moment the largest one on npm (counting the current versions only).
Your package size is 584 MiB (packed), while the second-largest one is only 268 MiB (packed).
Unpacked size is 1459 MiB. And you still say that this is not a complete version, but a «crippled» one.

Some things to note here (everything except 1 is additive):

1. ./src/release/third_party/ and ./src/devel/third_party/ are almost duplicates. Together, they consume 1089 MiB, which is about 75% of the total unpacked size. Keeping only one of them (if possible) would save about 37% of the installed size. Differences:
   • ./src/devel/third_party/mathjax_org/v_2_2_8/docs
   • ./src/devel/third_party/mathjax_org/v_2_2_8/test
   • ./src/devel/third_party/mathjax_org/v_2_2_8/www_simile-widgets_org/2013_05
2. src/*/third_party/3D/WebGL/*/*/Samples — some WebGL sample scenes. Most probably not needed, especially in the release dir. 705 MiB total. Removing them would save about 65% of the installed size. Full list:
   • src/devel/third_party/3D/WebGL/www_babylonjs_com/2013_09_07/Samples — 351 MiB
   • src/dev_tools/third_party/3D/WebGL/www_babylonjs_com/2013_09_07/Samples — 5 MiB
   • src/release/third_party/3D/WebGL/www_babylonjs_com/2013_09_07/Samples — 351 MiB
3. MathJax png fonts are most probably not needed. Those are for ancient browsers that don't understand both web fonts and svg images. 347 MiB total. Removing them would save about 23% of the installed size. See guide on how to properly disable them. Ful list:
   • src/devel/third_party/mathjax_org/v_2_2_8/fonts/HTML-CSS/TeX/png — 116 MiB
   • src/dev_tools/third_party/mathjax_org/v_2_2_8/fonts/HTML-CSS/TeX/png — 116 MiB
   • src/release/third_party/mathjax_org/v_2_2_8/fonts/HTML-CSS/TeX/png — 116 MiB
4. Zip/tar files, most probably packed deps, not needed (also, most of them are dupes). 91 MiB, 6%. Full list:
   • ./src/dev_tools/third_party/3D/WebGL/www_babylonjs_com/2013_09_07/Exporters/FBX - OBJ/BabylonExporter.zip — 9.6 MiB
   • ./src/dev_tools/third_party/fonts/www_eurofurence_net/monofur/originals/monof_tt.zip — 132 KiB
   • ./src/dev_tools/third_party/mathjax_org/originals/2013_07/mathjax-MathJax-v2.2-8-g727332c.zip — 18 MiB
   • ./src/release/third_party/3D/WebGL/www_babylonjs_com/2013_09_07/Exporters/FBX - OBJ/BabylonExporter.zip — 9,6 MiB
   • ./src/devel/third_party/3D/WebGL/www_babylonjs_com/2013_09_07/Exporters/FBX - OBJ/BabylonExporter.zip — 9,6 MiB
   • ./src/dev_tools/third_party/fonts/www_linuxlibertine_org/2011/originals/LinLibertineTTF_5.1.3_2011_06_21.tgz — 15 MiB
   • /src/dev_tools/third_party/fonts/www_linuxlibertine_org/2011/originals/LinLibertineSRC_5.1.3_2011_06_21.tgz.tar — 29 MiB

Even without 1, with only applying 2,3,4, your installed package size reduces more than four times to 318 MiB.

There are also yui examples, xiph_org doc, and 22 MiB test bmp file for kibuvits (./src/dev_tools/lib/mmmv_devel_tools/src/bonnet/lib/kibuvits_ruby_library/src/dev_tools/selftests/data_for_selftests/kibuvits_ImageMagick_selftests/uhuu.bmp), and some other garbage. With those removed, your installed package size reduces to 259 MiB.

And I believe that a lot more of those could be removed.

[ChALkeR]

For now, I blacklisted your package in my scripts (only the current version, raudrohi@27.0.1).

[martinvahi]

Thank You for Your thorough feedback. I guess that the blacklisting is/was justified, because I mis-understood the use case for NPM. I used it like "GitHub/Sourceforge", not like "Maven for JavaScript". From that point of view the Raudrohi JavaScript Library should have not been at the NPM at all, so I "fixed" it by removing it from the NPM entirely.

It would be nice, if You removed the blacklisting, because may be I'll re-publish deployable deliveries at NPM sometime in the future, but, again, thank You for the feedback.

[ChALkeR]

Once again: I am not associated with npm, I stated that at the very top of this issue. I was only talking about my package analysis script when I mentioned blacklisting. Sorry if that caused misunderstanding, I didn't mean it that way.

The script downloads stuff from npm, unpacks it, and produces some very simple code checks, it also builds a single greppable file of all the js sources (excluding minified, third-party deps, and known libs) — for example, to search for deprecated API usage in packages. Example output: nodejs/node#2961 (comment).

That won't have any negative effects on your package despite the fact that it will be excluded from my analysis.

[martinvahi]

Thank You, but it seems to me that I understood You correctly the first time(I make a lot of mistakes, misunderstand people, etc.) and I claim that You were right regardless of Your affiliation. But, again, I'm glad for Your answer. :-D

[ChALkeR]

@martinvahi

It would be nice, if You removed the blacklisting, because may be I'll re-publish deployable deliveries at NPM sometime in the future

Removed. That wasn't technically required, because if you published a new version it wouldn't be blacklisted anyway (only one specific version 27.0.1 was, because I'm only processing latest versions of packages atm), but now my blacklist has shrinked twice, it now has one package. =)