Too many open file handles

[jamieb22]

OrientDB Version:

3.0.24

Java Version

1.8

OS

Linux dbtest 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Expected behavior

Do not expect huge number of open file handles

Actual behavior

File handle limit exceeded

Steps to reproduce

In this scenario, our multitenant application starts around 80 Orient DB databases. The Orient DB databases were created and are readable using Orient DB v2.2.27. Please download a full stack of all orient db file handles at the exact time open file handles error occurred: orientdb_openfilehandle_logs.zip

[valenpo]

@Laa Andrey, hello.

I'm trying copy all databases locally for debug. Right now compressing all database folder.
And find that it contains 96411. We limit for 64k opened file's. Maybe odb try open all files on one shot?

Total number of databases is 90.

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C,Utf16=off,HugeFiles=on,64 bits,16 CPUs Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (50654),ASM,AES-NI)

Scanning the drive:
2247 folders, 94164 files, 221561883935 bytes (207 GiB)                      

Creating archive: var.opt.archive.7z

Items to compress: 96411

[jamieb22]

bump

[andrii0lomakin]

Please check latest 3.0.x, it should be already fixed.

[jamieb22]

Thanks for fix, however, latest is still OrientDB 3.0.32. Must we build it ourselves?

[andrii0lomakin]

Hi, @jamieb22 let me check couple of things before that, I will let you know when you should try new version a bit later.

[andrii0lomakin]

Hi @valenpo , do you open all those databases in separate JVMs or inside of the single one?

[valenpo]

It is single one JVM Вт, 20 окт. 2020 г. в 13:41, Andrey Lomakin <notifications@github.com>:

Hi @valenpo <https://github.com/valenpo> , do you open all those databases in separate JVMs or inside of the single one? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9060 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIMLQ26WSA7T4I5UANYJNTSLVSOXANCNFSM4JLRZKRQ> .

-- C Уважением Валентин Попов.

[andrii0lomakin]

You can see the limit of open files printed during the start of the database (does not matter server or embedded). Could you send me this number and also could you send me the value of the limit of open files on your system?

[valenpo]

System has limit to 64k opened files, but we set it to unlim for test before reach limit of nodes.

On 20 Oct 2020, at 16:56, Andrey Lomakin ***@***.***> wrote: You can see the limit of open files printed during the start of the database (does not matter server or embedded). Could you send me this number and also could you send me the value of the limit of open files on your system? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9060 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIMLQ2ZH2ZIVLO4WTXOW3LSLWJJLANCNFSM4JLRZKRQ>.

Regards, Valentin Popov

[andrii0lomakin]

Limit of open database files is changed according to the OS settings. So the more limit you set the more open files you will have. Could you check the log and check the limit of open files in the console. Also, of course, check whether this problem is reproducible.