RJB freeze on Docker loading classes

[NathanReis]

I have a project where the RJB can load Java classes locally without any problem, but when I run it in a container it freezes.

My RJB config (always work locally):

Rails.application.configure do
  config.to_prepare do
    jar_paths = [
      'bin/dfe-distribuicao.jar', PoiSpreadsheet.jar_paths
    ].join(':')

    max_heap = "-Xmx#{ENV['JVM_MAX_HEAP'] || '1024M'}"
    jvm_args = [max_heap]
    Rjb::load(jar_paths, jvm_args)
  end
end

I've already tested replacing config.to_prepare with config.after_initialize, and even loading my classes just below Rjb::load, but it keeps freezing.

My current RJB config (yet doesn't work on Docker):

Rails.application.configure do
  config.after_initialize do
    jar_paths = [
      'bin/dfe-distribuicao.jar', PoiSpreadsheet.jar_paths
    ].join(':')

    max_heap = "-Xmx#{ENV['JVM_MAX_HEAP'] || '1024M'}"
    jvm_args = [max_heap, '-Xrs']
    Rjb::load(jar_paths, jvm_args)

    Rjb.classes['org.apache.poi.xssf.usermodel.XSSFWorkbook'] || Rjb.import('org.apache.poi.xssf.usermodel.XSSFWorkbook')
    Rjb.classes['org.apache.poi.xssf.streaming.SXSSFWorkbook'] || Rjb.import('org.apache.poi.xssf.streaming.SXSSFWorkbook')
    Rjb.classes['org.apache.poi.ss.usermodel.CellType'] || Rjb.import('org.apache.poi.ss.usermodel.CellType')
    Rjb.classes['java.io.FileOutputStream'] || Rjb.import('java.io.FileOutputStream')
  end
end

My Java Settings in Container:

ENV JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64

# Fixing RJB Issue: https://github.com/arton/rjb/issues/70
RUN cd ${JAVA_HOME} && mkdir -p jre/lib/amd64 && ln -s ${JAVA_HOME}/lib/server ${JAVA_HOME}/jre/lib/amd64/server

[arton]

Hi NathanReis
Thanks for your report.
I'd like to reproduce this issue, therefore could you show me the docker setting or your docker-compose.yml ?

[victorlcampos]

hi Arton,
We use rjb for years inside a ec2 (aws), it’s a amazing gem. Thanks.

but when we try to run it on ours tests on GithubCLI or on a Docker it get freeze like Nathan said.

And now, we have to move from ec2 to fargate and ours features that depends on rjb are not working.

`

Extends from the Ruby Container. This is the only

thing that you need to adjust if you change the

project itself.

FROM ruby:2.6.4
LABEL MAINTAINER Virtual360

ENV APP_ROOT /app
ENV BUNDLE_PATH /usr/local/bundle

ENV LC_ALL C.UTF-8
ENV LANG C.UTF-8
ENV LANGUAGE C.UTF-8

https://superuser.com/questions/1423486/issue-with-fetching-http-deb-debian-org-debian-dists-jessie-updates-inrelease

RUN printf "deb http://archive.debian.org/debian/ jessie main\ndeb-src http://archive.debian.org/debian/ jessie main\ndeb http://security.debian.org jessie/updates main\ndeb-src http://security.debian.org jessie/updates main" > /etc/apt/sources.list

nodesource/distributions#1266 (comment)

RUN apt-get update
RUN apt-get upgrade ca-certificates -y
RUN curl -fsSL https://deb.nodesource.com/setup_12.x | bash -
RUN apt-get install -y nodejs

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update -qq &&
apt-get install -yq software-properties-common &&
apt-get install -yq apt-utils &&
apt-get install -y build-essential libpq-dev apt-transport-https less vim &&
apt-get install -y libaio1 unzip p7zip-full &&
apt-get install -y vim nano &&
apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

Install yarn

RUN apt-get update && apt-get install -y apt-transport-https
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
RUN apt-get update && apt-get install yarn
RUN yarn install --check-files

INSTALL FREE TDS 1.1.6 FOR MS SQL Adapter

RUN apt-get update &&
apt-get install libc6-dev &&
curl -O https://www.freetds.org/files/stable/freetds-1.1.6.tar.gz &&
tar -xzf freetds-1.1.6.tar.gz &&
cd freetds-1.1.6 &&
./configure --prefix=/usr/local --with-tdsver=7.3 &&
make &&
make install

ADD ORACLE INSTANT CLIENT FOR ORACLE ADAPTER

RUN mkdir -p /opt/oracle && mkdir -p /opt/instantclient
ADD ./resources/oracle/instantclient-* /opt/instantclient/
RUN 7z x /opt/instantclient/instantclient-basic-linux.x64-12.2.0.1.0.7z.001 -tsplit &&
mv instantclient-basic-linux.x64-12.2.0.1.0.7z /opt/instantclient/ &&
7z x /opt/instantclient/instantclient-basic-linux.x64-12.2.0.1.0.7z &&
mv instantclient-basic-linux.x64-12.2.0.1.0/* /opt/oracle/ &&
unzip /opt/instantclient/instantclient-sdk-linux.x64-12.2.0.1.0.zip -d /opt/oracle &&
unzip /opt/instantclient/instantclient-sqlplus-linux.x64-12.2.0.1.0.zip -d /opt/oracle &&
mv /opt/oracle/instantclient_12_2 /opt/oracle/instantclient &&
rm -rf /opt/instantclient

RUN ln -s /opt/oracle/instantclient/libclntsh.so.12.1 /opt/oracle/instantclient/libclntsh.so
RUN ln -s /opt/oracle/instantclient/libocci.so.12.1 /opt/oracle/instantclient/libocci.so
RUN ln -s /opt/oracle/instantclient/sqlplus /usr/local/sbin/sqlplus

ENV LD_LIBRARY_PATH="/opt/oracle/instantclient"
ENV OCI_HOME="/opt/oracle/instantclient"
ENV OCI_LIB_DIR="/opt/oracle/instantclient"
ENV OCI_INCLUDE_DIR="/opt/oracle/instantclient/sdk/include"
ENV NLS_LANG="BRAZILIAN PORTUGUESE_BRAZIL.UTF8"
ENV ORACLE_HOME="/opt/oracle/instantclient"

RUN echo '/opt/oracle/instantclient/' | tee -a /etc/ld.so.conf.d/oracle_instant_client.conf && ldconfig

Install OpenJDK-11

RUN apt-get update -q &&
apt-get install -y openjdk-11-jdk &&
apt-get install -y ant &&
apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

Fix certificate issues

RUN apt-get update &&
apt-get install -y ca-certificates-java &&
apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* &&
update-ca-certificates -f

Install Google Chrome

RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - &&
echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main' | tee /etc/apt/sources.list.d/google-chrome.list &&
apt-get update -q &&
apt-get install -y google-chrome-stable

ADD . $APP_ROOT
WORKDIR $APP_ROOT

RUN mkdir -p $APP_ROOT/log
RUN touch $APP_ROOT/log/sidekiq.log

ENV JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
`

`

version: '3'

volumes:
bundle:
driver: local

services:
web:
image: v360
command: resources/start_rails.sh
build:
context: .
dockerfile: Dockerfile
volumes:
- bundle:/usr/local/bundle
ports:
# Rails Server
- "3000:3000"
env_file:
- './web.env'
`

[arton]

Hi victorlcampos

I'm not sor familiar with fargate.
I'd like to know where do you 'bundle install' ? Your /app already had bundled gems ?

[victorlcampos]

Hi arton,
I this case, I run bundle install first on github actions, bundle package and after bundle install on dockerfile

but on local machine the same problem happens inside docker with this dockerfile that I send to you

[arton]

Hi victorlcampos

In your case, I suppose that Rjb was not build correctly.
Because of an extended library. Rjb requires install time compilation with cc and JDK.
I wonder if you could show me mkmf.log on vendor/bundle/ruby/2.6.0/extensions/x86_64-linux/2.6.0-static/rjb-(version)/.

[NathanReis]

Hi arton

I found 2 files with this name in the container, they refer to RJB versions 1.6.1 and 1.6.4.

mkmf.log.zip

[arton]

Hi NathanReis

Where do they sit in ? In the same directory or separated ? Is there gem.build_complete on that directory ?
I'm afraid that your container's JDK is different with bundle installed (= Rjb compilation time) JDK.

[NathanReis]

Hi arton

These files were in the folders /usr/local/bundle/extensions/x86_64-linux/2.6.0/rjb-1.6.1 and /usr/local/bundle/extensions/x86_64-linux/2.6.0/rjb- 1.6.4, both have the gem.build_complete file.

[arton]

Hi NathanReis

It seems good, so there is another problem.
I wonder if you could more details about 'freezing'.
For example could you write log as:

    max_heap = "-Xmx#{ENV['JVM_MAX_HEAP'] || '1024M'}"
    jvm_args = [max_heap, '-Xrs']
   Rails.logger.debug "!! begin to load Rjb"
    ret = Rjb::load(jar_paths, jvm_args)
   Rails.logger.debug "!! end to load Rjb: #{ret}"

or wherever you think the freezing point and show me the output.
BTW, It's better that you may run 'bundle clean' to delete older Gems logs such as rjb-1.6.1.

Regards

[NathanReis]

Hi arton

I ran bundle clean and version 1.6.1 was removed, but still the same problem.

I tried to log as you suggested but nothing was displayed, when I put a pry I saw that the return is being nil.

[arton]

Hi NathanReis
No problem. Rjb::load returns nil if it met no errors . If it failed to load JVM, then it raises RuntimeError.
I wonder if you could verify successfuly loading as:

   RJb::load(jar_paths, jvm_args)
   jstr = Rjb.import('java.lang.String')
   abc = jstr.new('abc')
   Rails.logger.debug "call JVM: #{abc.toUpperCase}"

[NathanReis]

Hi arton

I ran your test and it passed. I got the message call JVM: ABC.

In the first message of this issue, I showed that the jars of the PoiSpreadsheet gem and the bin/dfe-distribuicao.jar are loaded.

The freeze error occurs when I try to generate a spreadsheet using PoiSpreadsheet but nothing happens and it keeps displaying timeout logs until I display an error page in the browser.

The strange thing is that if I don't call Rjb::load, it works normally. So I tried to load only bin/dfe-distribuicao.jar thinking that the PoiSpreadsheet would still work, but when I do that, I get errors saying that it didn't find some Java classes that the gem uses.

After that I tried to reproduce the error in a project from scratch to send to you, in it I used the same versions of Ruby, Rails, Java, PoiSpreadsheet and Rjb, but the error did not occur.

Could it be some kind of conflict between the RJB and another gem or something?

[arton]

Hi NathanReis

Thank you for the reproducing environment! I'll try it.
Your concern about without calling Rjb::load, I think it's not harmful because Rjb automatically loads JVM if it was invoked any methods other than load. My only concern is threading issue. it's nice to load JVM in main thread. However your local environment runs fine, it's not the problem.

Regards.

[arton]

Hi NathanReis

When I hit 'http://localhost:3000', I got words.xlsx with foo, bar, baz. Do I need call localhost:3000 many times?

Only I changed port number 3000 to 3001 (because my other developing environment already uses 3000).

arton@Xirox:~/SampleRjbError$ docker exec -t -i 2baa2739 bash
root@2baa2739df57:/app# tail -f log/development.log
Completed 200 OK in 94ms (ActiveRecord: 0.0ms | Allocations: 362)


Started GET "/" for 172.17.0.1 at 2022-03-06 01:52:40 +0000
Cannot render console from 172.17.0.1! Allowed networks: 127.0.0.0/127.255.255.255, ::1
Processing by WordsController#index as HTML
Sent file #<File:0x000055cca6b78db0> (0.3ms)
Completed 200 OK in 41ms (ActiveRecord: 0.0ms | Allocations: 365)


Started GET "/" for 172.17.0.1 at 2022-03-06 02:05:27 +0000
Cannot render console from 172.17.0.1! Allowed networks: 127.0.0.0/127.255.255.255, ::1
   (0.1ms)  SELECT sqlite_version(*)
Processing by WordsController#index as HTML
Sent file #<File:0x00007f364404a160> (0.2ms)
Completed 200 OK in 31ms (ActiveRecord: 0.0ms | Allocations: 362)


Started GET "/" for 172.17.0.1 at 2022-03-06 02:05:33 +0000
Cannot render console from 172.17.0.1! Allowed networks: 127.0.0.0/127.255.255.255, ::1
Processing by WordsController#index as HTML
Sent file #<File:0x00007f3658030008> (0.2ms)
Completed 200 OK in 28ms (ActiveRecord: 0.0ms | Allocations: 362)

BTW Docker is great!

[NathanReis]

Hi arton

Just download it once, and the error should already occur, but as seen it does not occur in this sample project.

victorlcampos and I tried to discover something else, when running rails c while our application was running, we were able to load one of the classes that PoiSpreadsheet uses, the org.apache.poi.xssf.usermodel.XSSFWorkbook , but soon after it couldn't load the org.apache.poi.xssf.streaming.SXSSFWorkbook and got frozen. After that I went to check if this happens in the application itself, but unlike rails c, it was frozen right in the first Java class.

[arton]

Hi NathanReis
Does your container has sufficient working memory for POI ?

[NathanReis]

Hi arton

I don't know how much memory is needed to use PoiSpreadsheet, but I tested it with two PCs:

• My work PC has 12 Gb of RAM and leaving only the containers running and using the browser to access the application, I have 11.2 Gb of consumption.

• My personal PC has 16 Gb of RAM and with the same scenario it has 11 Gb of consumption.

In both, the problem occurs, so it shouldn't be a lack of storage, since the personal PC has plenty of space available.

[NathanReis]

Thank you for your willingness to help me solve the problem, your gem is sensational and has already helped a lot, but we will choose to look for another alternative because we need the features that today depend on the RJB.

I am available if you would like to continue investigating this issue.

[kwerle]

Of interest, we used to have freeze issues with:

• Rjb 1.5.9
• Ruby 3.1
• While using spring
  In testing with
• Rjb 1.6.8
• Ruby 3.2
  We no longer experience the freeze issue. Sadly, we are blocked from 1.6.8 by the UTF-8 issue.

[juanb23]

Are the any updates for running in docker? I am also experiencing similar issues where the aspose cells code freezes when loading from a template and the memory it uses is never freed.

Phusion Passenger Ruby 2.6 Image
Aspose Cells
Ruby Java Bridge