Building v7.2.2.56 with NODE_ENV set to production lacks some files

[adriangibanelbtactic]

Building v7.2.2.56 with NODE_ENV set to production lacks some files.

I happen to set NODE_ENV to production so that development dependencies are skipped.
I guess that that means that nodejs is not brought down and build it but you use nodejs from the distro itself.
Why? Because I don't want to wait 15 hours for everything to build but 10 hours instead (not sure about the actual hours, it's a wild guess).

The build seems to end well but it does not have: out/linux_64/onlyoffice/documentserver/web-apps/apps/spreadsheeteditor/mobile/index.html file.

Your Debian/Ubuntu package for this version does indeed have this file so I guess that your automatic builds are done without setting NODE_ENV to production.

The build also spits out:

install dependencies...
Node.js version cannot be less 10.20

so that's why I suspect about NODE_ENV variable.

Steps to reproduce the problem:
In an empty Debian 11 Netinst machine as a pre-build step I do:

sudo -i
# Enter user password

sudo apt-get update
 sudo apt-get install \
    ca-certificates \
    curl \
    gnupg \
    lsb-release
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get update
apt-get install docker-ce docker-ce-cli containerd.io

systemctl status docker
# docker.io start/running, process 14394

And here there is the interesting build snippet:

sudo -i
cd /root
git clone \
    --depth=1 \
    --recursive \
    --branch v7.2.2.56 \
    https://github.com/ONLYOFFICE/build_tools.git \
    /root/build_tools
# Ignore detached head warning
cd /root/build_tools
mkdir out
docker build --tag onlyoffice-document-editors-builder .
docker run -e PRODUCT_VERSION='7.2.2' -e BUILD_NUMBER='56' -e NODE_ENV='production' -v $(pwd)/out:/build_tools/out onlyoffice-document-editors-builder /bin/bash -c 'cd tools/linux && python3 ./automate.py --branch=tags/v7.2.2.56'

and its output (well, not the complete output but some of the last lines of it):

------------------------------------------
BUILD_PLATFORM: linux_64
------------------------------------------
make file: makefiles/build.makefile_linux_64
copy warning [file not exist]: /build_tools/scripts/../../sdkjs/common/HtmlFileInternal/AllFonts.js
delete warning [folder not exist]: /build_tools/scripts/../../document-server-integration/web/documentserver-example/nodejs/node_modules
[git] update: onlyoffice.github.io
branch does not exist...
switching to master...
delete warning [file not exist]: /build_tools/scripts/../out/linux_64/onlyoffice/documentserver-snap/var/www/onlyoffice/documentserver/example/nodejs/example
install dependencies...
Node.js version cannot be less 10.20
Reinstall
install qt...
---------------------------------------------
build branch: tags/v7.2.2.56
---------------------------------------------
---------------------------------------------
build modules: desktop builder server
---------------------------------------------

build_tools base repo version: v7.2.2.56

Operating system: Debian 11 but it actually uses whatever default Docker is defining which I think it's Ubuntu 16.04

Compiler version: Whatever the Docker says.

Finally: I have taken a look at develop branch at build_tools and server repos and I haven't seen anything related to this variable so I don't think it has been fixed there either. So this might apply to master or develop branches too.

Additional details:

• My VM has 16 GB of RAM so this should not be related to a lack of ram.
• I always delete the out directory before trying to build again so that I don't run into issues.

So, yeah, I think that the code should be fixed/updated so that we can keep building this even if we have NODE_ENV set to production.

If this is not possible then documentation about NODE_ENV behaviour (if that documentation exists) should have a warning that in the current state using the default commands will end in an incomplete build. So, in that regard the documentation should say that it's better, for now, not to set NODE_ENV to production.

I have started a new build exactly the same way as described above but without defining NODE_ENV variable.
That way I will be able to prove you that NODE_ENV is the problem here.
I will update you with more details when it finishes but I have a feeling that the build will be ok.

Thank you very much for your feedback!

[adriangibanelbtactic]

My standard build:

sudo -i
cd /root
git clone \
    --depth=1 \
    --recursive \
    --branch v7.2.2.56 \
    https://github.com/ONLYOFFICE/build_tools.git \
    /root/build_tools
# Ignore detached head warning
cd /root/build_tools
mkdir out
docker build --tag onlyoffice-document-editors-builder .
docker run -e PRODUCT_VERSION='7.2.2' -e BUILD_NUMBER='56' -v $(pwd)/out:/build_tools/out onlyoffice-document-editors-builder /bin/bash -c 'cd tools/linux && python3 ./automate.py --branch=tags/v7.2.2.56'

has these few last lines of output:

switching to master...
[git] update: web-apps
[git] update: desktop-apps
[fetch & build]: boost
delete warning [file not exist]: ./boost.data
[fetch & build]: cef
delete warning [file not exist]: ./cef_binary.7z.data
[fetch & build]: icu
[fetch & build]: openssl
delete warning [file not exist]: ./openssl.data
gn gen out.gn/linux_64 --args="v8_static_library=true is_component_build=false v8_monolithic=true v8_use_external_startup_data=false use_custom_libcxx=false treat_warnings_as_errors=false target_cpu=\"x64\" v8_target_cpu=\"x64\" is_debug=false is_clang=true use_sysroot=false"
[fetch & build]: hunspell
[fetch & build]: harfbuzz
------------------------------------------
BUILD_PLATFORM: linux_64
------------------------------------------
make file: makefiles/build.makefile_linux_64
copy warning [file not exist]: /build_tools/scripts/../../sdkjs/common/HtmlFileInternal/AllFonts.js
delete warning [folder not exist]: /build_tools/scripts/../../document-server-integration/web/documentserver-example/nodejs/node_modules
[git] update: onlyoffice.github.io
branch does not exist...
switching to master...
delete warning [file not exist]: /build_tools/scripts/../out/linux_64/onlyoffice/documentserver-snap/var/www/onlyoffice/documentserver/example/nodejs/example
install dependencies...
Node.js version cannot be less 10.20
Reinstall
install qt...
---------------------------------------------
build branch: tags/v7.2.2.56
---------------------------------------------
---------------------------------------------
build modules: desktop builder server
---------------------------------------------

.

cd /root/build_tools
ls -l out/linux_64/onlyoffice/documentserver/web-apps/apps/spreadsheeteditor/mobile/index.html
ls: no se puede acceder a 'out/linux_64/onlyoffice/documentserver/web-apps/apps/spreadsheeteditor/mobile/index.html': No existe el fichero o el directorio

The same problem still happens. The file is not there and Nodej.js dependencies complain about being lower than 10.20.

Is there any other environment variable that your automated system is using that I should be aware of?
Or am I messing up with something else?

Thank you for your feedback.

[adriangibanelbtactic]

I have just realised that this bug might be a duplicate of #543 .

This bug is quite important because you are providing the Debian binary package onlyoffice-documentserver_7.2.2-56_amd64.deb but, apparently, you are not providing an equivalent source code that can be used for building it.

I wonder myself... At the time of the build Ubuntu 16.04 did not require Node.js to be 12.x and now some weeks later it requires it?
Or is it something else related to your automated build process?

Thank you again for your feedback.

[igwyd]

Hello @adriangibanelbtactic, sorry for the late reply. Nodejs in build_tools have been updated to 14 version. Unfortunately we haven't the detailed documentation about compilation process, but i created ticket #64992 for our team to add this.
Duplicate of #751

[adriangibanelbtactic]

Hello @adriangibanelbtactic, sorry for the late reply. Nodejs in build_tools have been updated to 14 version. Unfortunately we haven't the detailed documentation about compilation process, but i created ticket #64992 for our team to add this. Duplicate of #751

Thank you.

The main recurrent complaint from the community is that repos here on github are not updated to reflect such changes in a timely manner.
E.g. Nodejs in your internal build_tools was updated to 12 version but in Github it was still 10 version.