Skip to content
This repository has been archived by the owner on Oct 16, 2023. It is now read-only.

MiniAccumuloCluster fails tests on Windows #935

Closed
t92549 opened this issue Sep 20, 2021 · 3 comments · Fixed by #1026
Closed

MiniAccumuloCluster fails tests on Windows #935

t92549 opened this issue Sep 20, 2021 · 3 comments · Fixed by #1026
Assignees
@GCHQDeveloper314
Labels
bug
Milestone
v2.0.0-alpha-0.4

Comments

@t92549
Copy link
Contributor

t92549 commented Sep 20, 2021

Found: gchq/Gaffer#2509

It seems that there is an issue with file permissions when using the MiniAccumuloCluster helper package on Windows.
Full stack trace from original issue:

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] gaffer-tools                                                       [pom]
[INFO] ui                                                                 [war]
[INFO] mini-accumulo-cluster                                              [jar]
[INFO] random-element-generation                                          [jar]
[INFO] performance-testing                                                [pom]
[INFO] performance-testing-core                                           [jar]
[INFO] performance-testing-accumulo-store                                 [jar]
[INFO] performance-testing-aws                                            [jar]
[INFO] schema-builder                                                     [pom]
[INFO] schema-builder-rest                                                [war]
[INFO] slider                                                             [jar]
[INFO] analytics                                                          [pom]
[INFO] analytics-functions                                                [jar]
[INFO] analytics-operations                                               [jar]
[INFO] analytics-rest                                                     [war]
[INFO]
[INFO] ------------------< uk.gov.gchq.gaffer:gaffer-tools >-------------------
[INFO] Building gaffer-tools 1.18.1-SNAPSHOT                             [1/15]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (validate) @ gaffer-tools ---
[INFO] Starting audit...
Audit done.
[INFO]
[INFO] >>> findbugs-maven-plugin:3.0.5:check (analyze-compile) > :findbugs @ gaffer-tools >>>
[INFO]
[INFO] --- findbugs-maven-plugin:3.0.5:findbugs (findbugs) @ gaffer-tools ---
[INFO]
[INFO] <<< findbugs-maven-plugin:3.0.5:check (analyze-compile) < :findbugs @ gaffer-tools <<<
[INFO]
[INFO]
[INFO] --- findbugs-maven-plugin:3.0.5:check (analyze-compile) @ gaffer-tools ---
[INFO]
[INFO] --- maven-jar-plugin:2.4:test-jar (default) @ gaffer-tools ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO]
[INFO] --- maven-source-plugin:2.4:jar-no-fork (attach-sources) @ gaffer-tools ---
[INFO]
[INFO] --- maven-javadoc-plugin:2.10.3:jar (attach-javadocs) @ gaffer-tools ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable package
[INFO]
[INFO] -----------------------< uk.gov.gchq.gaffer:ui >------------------------
[INFO] Building ui 1.18.1-SNAPSHOT                                       [2/15]
[INFO] --------------------------------[ war ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ ui ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory C:\Users\Asus\Desktop\gaffer-tools-develop\ui\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ ui ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ ui ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ ui ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @ ui ---
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (validate) @ ui ---
[INFO] Starting audit...
Audit done.
[INFO]
[INFO] >>> findbugs-maven-plugin:3.0.5:check (analyze-compile) > :findbugs @ ui >>>
[INFO]
[INFO] --- findbugs-maven-plugin:3.0.5:findbugs (findbugs) @ ui ---
[INFO]
[INFO] <<< findbugs-maven-plugin:3.0.5:check (analyze-compile) < :findbugs @ ui <<<
[INFO]
[INFO]
[INFO] --- findbugs-maven-plugin:3.0.5:check (analyze-compile) @ ui ---
[INFO]
[INFO] --- maven-war-plugin:3.0.0:war (default-war) @ ui ---
[INFO] Packaging webapp
[INFO] Assembling webapp [ui] in [C:\Users\Asus\Desktop\gaffer-tools-develop\ui\target\ui-1.18.1-SNAPSHOT]
[INFO] Processing war project
[INFO] Copying webapp webResources [C:\Users\Asus\Desktop\gaffer-tools-develop\ui\dependencies] to [C:\Users\Asus\Desktop\gaffer-tools-develop\ui\target\ui-1.18.1-SNAPSHOT]
[INFO] Copying webapp resources [C:\Users\Asus\Desktop\gaffer-tools-develop\ui\src\main\webapp]
[INFO] Webapp assembled in [5053 msecs]
[INFO] Building war: C:\Users\Asus\Desktop\gaffer-tools-develop\ui\target\ui-1.18.1-SNAPSHOT.war
[INFO]
[INFO] --- maven-jar-plugin:2.4:test-jar (default) @ ui ---
[INFO] Building jar: C:\Users\Asus\Desktop\gaffer-tools-develop\ui\target\ui-1.18.1-SNAPSHOT-tests.jar
[INFO]
[INFO] --- maven-source-plugin:2.4:jar-no-fork (attach-sources) @ ui ---
[INFO] No sources in project. Archive not created.
[INFO]
[INFO] --- maven-javadoc-plugin:2.10.3:jar (attach-javadocs) @ ui ---
[INFO]
[INFO] --------------< uk.gov.gchq.gaffer:mini-accumulo-cluster >--------------
[INFO] Building mini-accumulo-cluster 1.18.1-SNAPSHOT                    [3/15]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ mini-accumulo-cluster ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ mini-accumulo-cluster ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ mini-accumulo-cluster ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ mini-accumulo-cluster ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @ mini-accumulo-cluster ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running uk.gov.gchq.gaffer.miniaccumulocluster.MiniAccumuloClusterControllerTest
gaffer.miniaccumulocluster.MiniAccumuloClusterController ERROR  - Failed to delete old directory
java.io.IOException: Unable to delete file: miniAccumuloCluster\zookeeper
        at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1338)
        at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:334)
        at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1121)
        at uk.gov.gchq.gaffer.miniaccumulocluster.MiniAccumuloClusterController.createDirectory(MiniAccumuloClusterController.java:269)
        at uk.gov.gchq.gaffer.miniaccumulocluster.MiniAccumuloClusterController.start(MiniAccumuloClusterController.java:121)
        at uk.gov.gchq.gaffer.miniaccumulocluster.MiniAccumuloClusterControllerTest.lambda$shouldStartAndStopCluster$0(MiniAccumuloClusterControllerTest.java:45)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.file.FileSystemException: miniAccumuloCluster\zookeeper\version-2\log.1: Impossibile accedere al file. Il file è utilizzato da un altro processo.

        at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
        at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
        at sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:108)
        at java.nio.file.Files.deleteIfExists(Files.java:1165)
        at org.apache.commons.io.file.DeletingPathVisitor.visitFile(DeletingPathVisitor.java:99)
        at org.apache.commons.io.file.DeletingPathVisitor.visitFile(DeletingPathVisitor.java:35)
        at java.nio.file.Files.walkFileTree(Files.java:2670)
        at java.nio.file.Files.walkFileTree(Files.java:2742)
        at org.apache.commons.io.file.PathUtils.visitFileTree(PathUtils.java:535)
        at org.apache.commons.io.file.PathUtils.deleteDirectory(PathUtils.java:265)
        at org.apache.commons.io.file.PathUtils.delete(PathUtils.java:254)
        at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1336)
        ... 6 more
gaffer.miniaccumulocluster.MiniAccumuloClusterController ERROR  - Failed to create temp dir
java.nio.file.FileAlreadyExistsException: miniAccumuloCluster
        at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:81)
        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
        at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:504)
        at java.nio.file.Files.createDirectory(Files.java:674)
        at uk.gov.gchq.gaffer.miniaccumulocluster.MiniAccumuloClusterController.createDirectory(MiniAccumuloClusterController.java:279)
        at uk.gov.gchq.gaffer.miniaccumulocluster.MiniAccumuloClusterController.start(MiniAccumuloClusterController.java:121)
        at uk.gov.gchq.gaffer.miniaccumulocluster.MiniAccumuloClusterControllerTest.lambda$shouldStartAndStopCluster$0(MiniAccumuloClusterControllerTest.java:45)
        at java.lang.Thread.run(Thread.java:748)
java.io.IOException: Unable to delete file: miniAccumuloCluster\zookeeper
        at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1338)
        at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:334)
        at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1121)
        at uk.gov.gchq.gaffer.miniaccumulocluster.MiniAccumuloClusterController.stop(MiniAccumuloClusterController.java:141)
        at uk.gov.gchq.gaffer.miniaccumulocluster.MiniAccumuloClusterControllerTest.shouldStartAndStopCluster(MiniAccumuloClusterControllerTest.java:66)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
        at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
        at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
        at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:43)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.Iterator.forEachRemaining(Iterator.java:116)
        at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
        at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
        at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:82)
        at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:73)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
        at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
        at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:132)
        at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:188)
        at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:154)
        at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:128)
        at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:428)
        at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
        at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:562)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:548)
Caused by: java.nio.file.FileSystemException: miniAccumuloCluster\zookeeper\version-2\log.1: Impossibile accedere al file. Il file è utilizzato da un altro processo.

        at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
        at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
        at sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:108)
        at java.nio.file.Files.deleteIfExists(Files.java:1165)
        at org.apache.commons.io.file.DeletingPathVisitor.visitFile(DeletingPathVisitor.java:99)
        at org.apache.commons.io.file.DeletingPathVisitor.visitFile(DeletingPathVisitor.java:35)
        at java.nio.file.Files.walkFileTree(Files.java:2670)
        at java.nio.file.Files.walkFileTree(Files.java:2742)
        at org.apache.commons.io.file.PathUtils.visitFileTree(PathUtils.java:535)
        at org.apache.commons.io.file.PathUtils.deleteDirectory(PathUtils.java:265)
        at org.apache.commons.io.file.PathUtils.delete(PathUtils.java:254)
        at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1336)
        ... 51 more
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 5.564 s <<< FAILURE! - in uk.gov.gchq.gaffer.miniaccumulocluster.MiniAccumuloClusterControllerTest
[ERROR] uk.gov.gchq.gaffer.miniaccumulocluster.MiniAccumuloClusterControllerTest.shouldStartAndStopCluster  Time elapsed: 5.535 s  <<< FAILURE!
java.lang.AssertionError: Unable to create directory expected null, but was:<java.lang.RuntimeException: Unable to create directory>
        at uk.gov.gchq.gaffer.miniaccumulocluster.MiniAccumuloClusterControllerTest.shouldStartAndStopCluster(MiniAccumuloClusterControllerTest.java:61)

[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   MiniAccumuloClusterControllerTest.shouldStartAndStopCluster:61 Unable to create directory expected null, but was:<java.lang.RuntimeException: Unable to create directory>
[INFO]
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for gaffer-tools 1.18.1-SNAPSHOT:
[INFO]
[INFO] gaffer-tools ....................................... SUCCESS [  4.434 s]
[INFO] ui ................................................. SUCCESS [ 10.827 s]
[INFO] mini-accumulo-cluster .............................. FAILURE [  8.730 s]
[INFO] random-element-generation .......................... SKIPPED
[INFO] performance-testing ................................ SKIPPED
[INFO] performance-testing-core ........................... SKIPPED
[INFO] performance-testing-accumulo-store ................. SKIPPED
[INFO] performance-testing-aws ............................ SKIPPED
[INFO] schema-builder ..................................... SKIPPED
[INFO] schema-builder-rest ................................ SKIPPED
[INFO] slider ............................................. SKIPPED
[INFO] analytics .......................................... SKIPPED
[INFO] analytics-functions ................................ SKIPPED
[INFO] analytics-operations ............................... SKIPPED
[INFO] analytics-rest ..................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  24.410 s
[INFO] Finished at: 2021-09-19T18:46:20+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test (default-test) on project mini-accumulo-cluster: There are test failures.
[ERROR]
[ERROR] Please refer to C:\Users\Asus\Desktop\gaffer-tools-develop\mini-accumulo-cluster\target\surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :mini-accumulo-cluster
@t92549 t92549 mentioned this issue Sep 20, 2021
Closed
@GCHQDeveloper314 GCHQDeveloper314 added this to the v2_backlog milestone Jul 11, 2022
@GCHQDeveloper314
Copy link
Member

This problem is likely similar to those in Gaffer #2717. The build output above (supplied by the original reporter) should be first be double checked against the latest version to confirm this problem is still present.

The build output shows a problem with deleting a file caused by a java.nio.file.FileSystemException of "Impossibile accedere al file. Il file è utilizzato da un altro processo". This is almost certainly the same as "The process cannot access the file because it is being used by another process" error which is seen in #2717.

@GCHQDeveloper314 GCHQDeveloper314 self-assigned this Sep 29, 2022
@github-actions github-actions bot mentioned this issue Oct 3, 2022
Merged
@GCHQDeveloper314 GCHQDeveloper314 linked a pull request Oct 3, 2022 that will close this issue
Gh-935: MiniAccumuloCluster on Windows #1026
Merged
@GCHQDeveloper314
Copy link
Member

This issue is caused by not having the Hadoop native libraries (specifically hadoop.dll) installed. With them the tests pass as expected. Fix in #1026

GCHQDeveloper314 added a commit that referenced this issue Oct 11, 2022
@GCHQDeveloper314
Gh-935: MiniAccumuloCluster on Windows (#1026)
a316e66 
Update MiniAccumuloCluster README, include more detail on building and Windows
@GCHQDeveloper314
Copy link
Member

Done (#1026)

@t92549 t92549 modified the milestones: v2_backlog, v2.0.0-alpha-0.4 Oct 11, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@GCHQDeveloper314 GCHQDeveloper314

Labels
bug
Projects
None yet
Milestone
v2.0.0-alpha-0.4
Development

Successfully merging a pull request may close this issue.

Gh-935: MiniAccumuloCluster on Windows gchq/gaffer-tools
2 participants
@t92549 @GCHQDeveloper314