Embedded mode is not persistent on windows

[Jurgen-Aquilina]

ArcadeDB Version: 21.10

JDK Version: Coretto 11

OS: Windows 10

Expected behavior

Stopping JVM should recover data correctly as indicated by docs.

Actual behavior

Following errors are outputted and data is lost:
Cannot find bucket type_0 for type 'Type', removing it from type configuration
Cannot find bucket type_1 for type 'Type', removing it from type configuration
Cannot find bucket type_2 for type 'Type', removing it from type configuration
Cannot find bucket type_3 for type 'Type', removing it from type configuration
Cannot find bucket type_4 for type 'Type', removing it from type configuration
Cannot find bucket type_5 for type 'Type', removing it from type configuration
Cannot find bucket type_6 for type 'Type', removing it from type configuration
Cannot find bucket type_7 for type 'Type', removing it from type configuration

<More Errors>

Error on deleting file '/<PATH_TO_DB>/txlog_0.wal'
Error on deleting file '/<PATH_TO_DB>/txlog_1.wal'
Error on deleting file '/<PATH_TO_DB>/txlog_2.wal'
Error on deleting file '/<PATH_TO_DB>/txlog_3.wal'
Error on deleting file '/<PATH_TO_DB>/txlog_4.wal'
Error on deleting file '/<PATH_TO_DB>/txlog_5.wal'

Steps to reproduce

Run the following and stop JVM

private val factory: DatabaseFactory = DatabaseFactory(<PATH_TO_DB>)
val database: Database = if (factory.exists()) {
    factory.open()
} else {
    factory.create()
}

database.transaction {
    if (!it.schema.existsType(<TYPE_NAME>)) {
        val doc = it.schema.createDocumentType(<TYPE_NAME>)
        doc.createProperty(<PROPERTY_NAME>, Type.STRING)
        doc.createProperty(<PROPERTY_NAME>, Type.STRING)
        doc.createProperty(<PROPERTY_NAME>, Type.STRING)
        doc.createProperty(<PROPERTY_NAME>, Type.INTEGER)
    }
}

[lvca]

I can't reproduce it. I wrote this test and after running it multiple times, the database is always opened correctly.

I put a System.exit(1); to skip the soft close of the database (that is database.close()):

public class TestCaseIssue100 {
  public static void main(String[] args) throws Exception {
    new TestCaseIssue100().run();
  }

  private void run() {
    //FileUtils.deleteRecursively(new File("./target/testcase-issue100"));

    DatabaseFactory factory = new DatabaseFactory("./target/testcase-issue100");
    Database database = factory.exists() ? factory.open() : factory.create();

    database.transaction(() -> {
      if (!database.getSchema().existsType("TestType")) {
        DocumentType doc = database.getSchema().createDocumentType("TestType");
        doc.createProperty("a", Type.STRING);
        doc.createProperty("b", Type.STRING);
        doc.createProperty("c", Type.STRING);
        doc.createProperty("d", Type.INTEGER);
      }
    });

    System.exit(1);
  }
}

[Jurgen-Aquilina]

@lvca did you run that on Windows?

I ran your exact code and got the following errors:

2021-10-07 12:32:07.884 WARNI [EmbeddedSchema] Cannot find bucket TestType_0 for type 'TestType', removing it from type configuration
2021-10-07 12:32:07.894 WARNI [EmbeddedSchema] Cannot find bucket TestType_1 for type 'TestType', removing it from type configuration
2021-10-07 12:32:07.895 WARNI [EmbeddedSchema] Cannot find bucket TestType_2 for type 'TestType', removing it from type configuration
2021-10-07 12:32:07.895 WARNI [EmbeddedSchema] Cannot find bucket TestType_3 for type 'TestType', removing it from type configuration
2021-10-07 12:32:07.895 WARNI [EmbeddedSchema] Cannot find bucket TestType_4 for type 'TestType', removing it from type configuration
2021-10-07 12:32:07.895 WARNI [EmbeddedSchema] Cannot find bucket TestType_5 for type 'TestType', removing it from type configuration
2021-10-07 12:32:07.895 WARNI [EmbeddedSchema] Cannot find bucket TestType_6 for type 'TestType', removing it from type configuration
2021-10-07 12:32:07.895 WARNI [EmbeddedSchema] Cannot find bucket TestType_7 for type 'TestType', removing it from type configuration
2021-10-07 12:32:07.935 WARNI [EmbeddedDatabase] Database 'testcase-issue100' was not closed properly last time
2021-10-07 12:32:07.936 WARNI [TransactionManager] Started recovery of database 'testcase-issue100'
2021-10-07 12:32:07.952 WARNI [WALFile] Error on deleting file './target/testcase-issue100/txlog_0.wal'
2021-10-07 12:32:07.952 WARNI [WALFile] Error on deleting file './target/testcase-issue100/txlog_1.wal'
2021-10-07 12:32:07.953 WARNI [WALFile] Error on deleting file './target/testcase-issue100/txlog_2.wal'
2021-10-07 12:32:07.953 WARNI [WALFile] Error on deleting file './target/testcase-issue100/txlog_3.wal'
2021-10-07 12:32:07.955 WARNI [WALFile] Error on deleting file './target/testcase-issue100/txlog_4.wal'
2021-10-07 12:32:07.955 WARNI [WALFile] Error on deleting file './target/testcase-issue100/txlog_5.wal'
2021-10-07 12:32:07.962 WARNI [TransactionManager] Recovery of database 'testcase-issue100' completed

[lvca]

It seems ArcadeDB can't write to the storage for some reason. Are you running it with docker?

[Jurgen-Aquilina]

No. it's running directly on windows and writing to a column on a separate hard disk so should have permission to read/write

[lvca]

I was able to spin a Windows machine and many tests won't pass, so there is definitely something to fix there. I'm gonna work today on it, I guess it's something about paths because all our users on Linux, Mac and other Unix based OSS never reported such problems.

[lvca]

Running the basic test suite against the engine package, this is the report:

[ERROR] Tests run: 830, Failures: 2, Errors: 507, Skipped: 3

[lvca]

Working on a separate branch (windows-issues), I already fixed most of the issues regarding the managing of paths + proper closing of file because in Windows opening a file is not "reentrant".

[lvca]

The branch windows-issues is usable under Windows. Still, some tests won't pass which we're working on them. Those remaining issues are on the tests themselves than the regular usage of the DBMS.

[lvca]

windows-issues merged into main branch. Now it's working under Windows, still, some tests won't pass. I'm pretty sure it's about how the tests are designed than the engine.