Cluster segments are missing after rebooting server

[mexeniz]

OrientDB Version: 2.2.18

Java Version: 1.8.0_144

OS: FreeBSD 11.0-RELEASE-p1

Hi, I have got a problem when I created new class in database.
I created "drives" class and then inserted approximately 100 records.
After rebooting my server, I found that cluster segments for drives class were missing.
Some queries for listing drive class do not work anymore.
It showed an error like this:

Cluster segment #142 does not exist in database 'FTO-Structure-2.0'"

Database Information

I checked the database by console.sh with info command.
Drive clusters were missing and the index of 3 of class list was also gone.
The followings are the parts of a result that are invalid:

CLUSTERS (collections)
+----+--------------+----+------------+-----------------+-----+----------+
|# |NAME | ID|CLASS |CONFLICT-STRATEGY|COUNT|SPACE-USED|
+----+--------------+----+------------+-----------------+-----+----------+
|0 |_studio | 97|_studio | | 6| 786b|
|1 |_studio_1 | 98|_studio | | 0| 0b|
|2 |_studio_2 | 99|_studio | | 0| 0b|
|3 |_studio_3 | 100|_studio | | 0| 0b|
|4 |_studio_4 | 101|_studio | | 0| 0b|
|5 |_studio_5 | 102|_studio | | 0| 0b|
|6 |_studio_6 | 103|_studio | | 0| 0b|
|7 |_studio_7 | 104|_studio | | 0| 0b|
|8 |alias | 89|alias | | 11| 810b|
|9 |alias_1 | 90|alias | | 5| 430b|
|10 |alias_2 | 91|alias | | 5| 438b|
|11 |alias_3 | 92|alias | | 5| 429b|
|12 |alias_4 | 93|alias | | 4| 366b|
|13 |alias_5 | 94|alias | | 5| 427b|
|14 |alias_6 | 95|alias | | 3| 300b|
|15 |alias_7 | 96|alias | | 3| 300b|
|16 |default | 3| |version | 0| 0b|
|17 |domains | 25|domains | | 2| 330b|
|18 |domains_1 | 26|domains | | 1| 148b|
|19 |domains_2 | 27|domains | | 0| 0b|
|20 |domains_3 | 28|domains | | 0| 0b|
|21 |domains_4 | 29|domains | | 0| 0b|
|22 |domains_5 | 30|domains | | 0| 0b|
|23 |domains_6 | 31|domains | | 0| 0b|
|24 |domains_7 | 32|domains | | 0| 0b|
|25 |e | 17|E | | 0| 0b|
|26 |e_1 | 18|E | | 0| 0b|
|27 |e_2 | 19|E | | 0| 0b|
|28 |e_3 | 20|E | | 0| 0b|
|29 |e_4 | 21|E | | 0| 0b|
|30 |e_5 | 22|E | | 0| 0b|
|31 |e_6 | 23|E | | 0| 0b|
|32 |e_7 | 24|E | | 0| 0b|

CLASSES
+----+-----+----------+-----------+--------------------------------------------------------------------------------------------------------------------+
|# |COUNT|NAME |SUPER-CLASS|CLUSTERS |
+----+-----+----------+-----------+--------------------------------------------------------------------------------------------------------------------+
|0 | 6|_studio | |_studio(97),_studio_1(98),_studio_2(99),_studio_3(100),_studio_4(101),_studio_5(102),_studio_6(103),_studio_7(104) |
|1 | 41|alias |[E] |alias(89),alias_1(90),alias_2(91),alias_3(92),alias_4(93),alias_5(94),alias_6(95),alias_7(96) |
|2 | 3|domains |[V] |domains(25),domains_1(26),domains_2(27),domains_3(28),domains_4(29),domains_5(30),domains_6(31),domains_7(32) |
|3 | | | | |
|4 | 0|E | |e(17),e_1(18),e_2(19),e_3(20),e_4(21),e_5(22),e_6(23),e_7(24) |

Database Directory

However, the database's directory still contains files for drives class.

...
-rwxr-xr-x 1 orientdb orientdb 1024 Mar 6 2017 domains_7.cpm
-rwxr-xr-x 1 orientdb orientdb 66560 Mar 6 2017 domains_7.pcl
-rwxr-xr-x 1 orientdb orientdb 66560 Feb 14 12:46 drives.cpm
-rwxr-xr-x 1 orientdb orientdb 263168 Feb 14 12:46 drives.pcl
-rwxr-xr-x 1 orientdb orientdb 66560 Feb 14 12:46 drives_1.cpm
-rwxr-xr-x 1 orientdb orientdb 263168 Feb 14 12:46 drives_1.pcl
-rwxr-xr-x 1 orientdb orientdb 66560 Feb 14 12:46 drives_2.cpm
-rwxr-xr-x 1 orientdb orientdb 263168 Feb 14 12:46 drives_2.pcl
-rwxr-xr-x 1 orientdb orientdb 66560 Feb 14 12:46 drives_3.cpm
-rwxr-xr-x 1 orientdb orientdb 263168 Feb 14 12:46 drives_3.pcl
-rwxr-xr-x 1 orientdb orientdb 66560 Feb 14 12:46 drives_4.cpm
-rwxr-xr-x 1 orientdb orientdb 263168 Feb 14 12:46 drives_4.pcl
-rwxr-xr-x 1 orientdb orientdb 66560 Feb 14 12:46 drives_5.cpm
-rwxr-xr-x 1 orientdb orientdb 263168 Feb 14 12:46 drives_5.pcl
-rwxr-xr-x 1 orientdb orientdb 66560 Feb 14 12:46 drives_6.cpm
-rwxr-xr-x 1 orientdb orientdb 263168 Feb 14 12:46 drives_6.pcl
-rwxr-xr-x 1 orientdb orientdb 66560 Feb 14 12:46 drives_7.cpm
-rwxr-xr-x 1 orientdb orientdb 263168 Feb 14 12:46 drives_7.pcl
-rwxr-xr-x 1 orientdb orientdb 1024 Mar 6 2017 e.cpm
-rwxr-xr-x 1 orientdb orientdb 66560 Mar 6 2017 e.pcl
...

orient-server.log

I have just noticed some errors in orient-server.log. The warning happened after I created new class.

2018-02-12 10:28:09:667 WARNI {db=FTO-Structure-2.0} Page in file drives.pcl with index 0 was placed in wrong free list, this error will be fixed automatically [OPaginatedCluster]
2018-02-12 10:28:13:320 WARNI {db=FTO-Structure-2.0} Page in file drives_7.pcl with index 0 was placed in wrong free list, this error will be fixed automatically [OPaginatedCluster]

I have already tried repairing by "repair database --fix-graph" and "repair database --fix-ridbags", but it did not work.
Is there any other ways to repair my database?

[hazzasym]

Sorry, any update on this case? This problem mean OrientDB may lost data in some points (in my case we lost entire cluster segment for a class). It's make our application can't going online!

[andrii0lomakin]

Hi @hazzasym you use a very old version of the database, this issue already should be fixed. The problem was that sometimes not all data were written to the file system. Do you repeat the same problem with the latest 2.2.x release?

[hazzasym]

Hi @Laa, thank you for your response.
First we try to upgrade our Orientdb latest version to both 0.2.32 and 0.2.33 release as you recommended but the latest version cause our query broke (we created new issue as #8142 ).
Now we stick with 0.2.18 release and wait for your reply to those ticket then we can confirm what we should try next and which information to provide for Orientdb team.

Best regards,

[andrii0lomakin]

Should be fixed in latest 3.0.x as we use the durable version of storage configuration.