Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugin hangs when too many changes in the git-repo #336

Closed
andreas-eternach opened this issue Nov 1, 2017 · 19 comments
Closed

Plugin hangs when too many changes in the git-repo #336

andreas-eternach opened this issue Nov 1, 2017 · 19 comments
Labels
Milestone

Comments

@andreas-eternach
Copy link
Contributor

andreas-eternach commented Nov 1, 2017

The plugin execution blocks when there are too many changes in the git-repo.

Probably because the stdout from git-child-process is processed after git-process terminates.

In case stdout-buffer is full the git-child-process stops working and a deadlock occurs.

Details:

  • Plugin-Version:2.2.3
  • git-command for child process "git status -s"
  • Stack traces from maven environmen

2017-11-01 08:23:18 Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.144-b01 mixed mode): "RMI TCP Connection(2)-192.168.99.1" #16 daemon prio=5 os_prio=0 tid=0x00000000223f7000 nid=0x2e08 runnable [0x000000002430e000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read(BufferedInputStream.java:265) locked <0x000000076e07b818> (a java.io.BufferedInputStream) at java.io.FilterInputStream.read(FilterInputStream.java:83) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:550) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$6/2140250730.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: <0x000000076da1a2f8> (a java.util.concurrent.ThreadPoolExecutor$Worker) "JMX server connection timeout 15" #15 daemon prio=5 os_prio=0 tid=0x000000000e672000 nid=0x2dc8 in Object.wait() [0x00000000241af000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) waiting on <0x000000076db91c30> (a [I) at com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:168) locked <0x000000076db91c30> (a [I) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: None "RMI Scheduler(0)" #14 daemon prio=5 os_prio=0 tid=0x000000000e671000 nid=0x2dc4 waiting on condition [0x0000000023f3e000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) parking to wait for <0x000000076d877090> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: None "RMI TCP Connection(1)-192.168.99.1" #13 daemon prio=5 os_prio=0 tid=0x000000000e66a000 nid=0x2dc0 runnable [0x00000000236ce000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read(BufferedInputStream.java:265) locked <0x000000076db608e0> (a java.io.BufferedInputStream) at java.io.FilterInputStream.read(FilterInputStream.java:83) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:550) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$6/2140250730.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: <0x000000076da178a8> (a java.util.concurrent.ThreadPoolExecutor$Worker) "RMI TCP Accept-0" #12 daemon prio=5 os_prio=0 tid=0x0000000020342000 nid=0x2d64 runnable [0x0000000023b3e000] java.lang.Thread.State: RUNNABLE at java.net.DualStackPlainSocketImpl.accept0(Native Method) at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199) locked <0x000000076d87d490> (a java.net.SocksSocketImpl) at java.net.ServerSocket.implAccept(ServerSocket.java:545) at java.net.ServerSocket.accept(ServerSocket.java:513) at sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept(LocalRMIServerSocketFactory.java:52) at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:400) at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:372) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: None "Service Thread" #10 daemon prio=9 os_prio=0 tid=0x000000000e669000 nid=0x2940 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE Locked ownable synchronizers: None "C1 CompilerThread3" #9 daemon prio=9 os_prio=2 tid=0x000000000e5e8000 nid=0xdd0 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE Locked ownable synchronizers: None "C2 CompilerThread2" #8 daemon prio=9 os_prio=2 tid=0x000000000e5da000 nid=0x20a0 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE Locked ownable synchronizers: None "C2 CompilerThread1" #7 daemon prio=9 os_prio=2 tid=0x000000000e5d4800 nid=0x1ebc waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE Locked ownable synchronizers: None "C2 CompilerThread0" #6 daemon prio=9 os_prio=2 tid=0x000000000e5cb800 nid=0x460 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE Locked ownable synchronizers: None "Attach Listener" #5 daemon prio=5 os_prio=2 tid=0x000000000e5c8800 nid=0x2320 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE Locked ownable synchronizers: None "Signal Dispatcher" #4 daemon prio=9 os_prio=2 tid=0x000000000e5c7000 nid=0x4bc runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE Locked ownable synchronizers: None "Finalizer" #3 daemon prio=8 os_prio=1 tid=0x000000000d406000 nid=0x1b84 in Object.wait() [0x000000001e6cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) waiting on <0x00000006c39c4f18> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143) locked <0x00000006c39c4f18> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) Locked ownable synchronizers: None "Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x000000000d3ff000 nid=0x16ec in Object.wait() [0x000000001e93f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) waiting on <0x00000006c32937c0> (a java.lang.ref.Reference$Lock) at java.lang.Object.wait(Object.java:502) at java.lang.ref.Reference.tryHandlePending(Reference.java:191) locked <0x00000006c32937c0> (a java.lang.ref.Reference$Lock) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153) Locked ownable synchronizers: None "main" #1 prio=5 os_prio=0 tid=0x00000000022da800 nid=0x16b0 runnable [0x000000000266d000] java.lang.Thread.State: RUNNABLE at java.lang.ProcessImpl.waitForInterruptibly(Native Method) at java.lang.ProcessImpl.waitFor(ProcessImpl.java:449) at pl.project13.maven.git.NativeGitProvider$JavaProcessRunner.runEmpty(NativeGitProvider.java:399) at pl.project13.maven.git.NativeGitProvider.tryCheckEmptyRunGitCommand(NativeGitProvider.java:256) at pl.project13.maven.git.NativeGitProvider.isDirty(NativeGitProvider.java:168) at pl.project13.maven.git.GitDataProvider.loadGitData(GitDataProvider.java:118) at pl.project13.maven.git.GitCommitIdMojo.loadGitDataWithNativeGit(GitCommitIdMojo.java:505) at pl.project13.maven.git.GitCommitIdMojo.loadGitData(GitCommitIdMojo.java:486) at pl.project13.maven.git.GitCommitIdMojo.execute(GitCommitIdMojo.java:372) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:309) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:194) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:107) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:993) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:345) at org.apache.maven.cli.MavenCli.main(MavenCli.java:191) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) Locked ownable synchronizers: None "VM Thread" os_prio=2 tid=0x000000000e573000 nid=0x4f4 runnable "GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00000000022ee800 nid=0x10fc runnable "GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00000000022f0000 nid=0x590 runnable "GC task thread#2 (ParallelGC)" os_prio=0 tid=0x00000000022f1800 nid=0xc60 runnable "GC task thread#3 (ParallelGC)" os_prio=0 tid=0x00000000022f3000 nid=0x26d8 runnable "GC task thread#4 (ParallelGC)" os_prio=0 tid=0x00000000022f6800 nid=0x1930 runnable "GC task thread#5 (ParallelGC)" os_prio=0 tid=0x00000000022f8800 nid=0x12b4 runnable "GC task thread#6 (ParallelGC)" os_prio=0 tid=0x00000000022fa000 nid=0xa1c runnable "GC task thread#7 (ParallelGC)" os_prio=0 tid=0x00000000022fd000 nid=0x10e8 runnable "VM Periodic Task Thread" os_prio=2 tid=0x000000000e67f000 nid=0x1e80 waiting on condition JNI global references: 247

@TheSnoozer
Copy link
Collaborator

Thanks for reporting this issue here!
I would like to try to reproduce this and thus would kindly ask for the following:

  • what do you consider too many changes in the git-repo ? touching 50k files? just want to know the proportions of what we are talking here
  • what maven version are you using?
  • what java version are you using (please include information about oracle-jdk VS open-jdk)?
  • under what OS can this be reproduced (assuming windows 10)?
  • how exactly do you execute maven (e.g. mvn -T 4 clean package)?

@TheSnoozer TheSnoozer added the bug label Nov 1, 2017
@TheSnoozer
Copy link
Collaborator

@andreas-eternach could you please provide the requested information to help trouble shoot this problem?

@TheSnoozer
Copy link
Collaborator

The stacktrace indicates the following part of the plugin:

RUNNABLE at java.lang.ProcessImpl.waitForInterruptibly(Native Method) at 
java.lang.ProcessImpl.waitFor(ProcessImpl.java:449) at 
pl.project13.maven.git.NativeGitProvider$JavaProcessRunner.runEmpty(NativeGitProvider.java:399) at 
pl.project13.maven.git.NativeGitProvider.tryCheckEmptyRunGitCommand(NativeGitProvider.java:256) at 
pl.project13.maven.git.NativeGitProvider.isDirty(NativeGitProvider.java:168) at 
pl.project13.maven.git.GitDataProvider.loadGitData(GitDataProvider.java:118) at 
pl.project13.maven.git.GitCommitIdMojo.loadGitDataWithNativeGit(GitCommitIdMojo.java:505) at 
pl.project13.maven.git.GitCommitIdMojo.loadGitData(GitCommitIdMojo.java:486) at 
pl.project13.maven.git.GitCommitIdMojo.execute(GitCommitIdMojo.java:372) at 
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)

This is pretty much this piece
https://github.com/ktoso/maven-git-commit-id-plugin/blob/master/src/main/java/pl/project13/maven/git/NativeGitProvider.java#L196
that executes git status -s

creating a repository:

#!/bin/bash
git clone https://github.com/torvalds/linux.git

for i in `seq 1 5000000`;
do
         echo $i > $i.txt
done

Disk run out of space, but end result was

$ git status -s | wc -l
1118405
mvn -T 4 clean package

with the repository worked fine (it obviously takes a bit for such repo to determine all properties)...

@TheSnoozer
Copy link
Collaborator

@andreas-eternach could you please provide the requested information to help trouble shoot this problem?

@ahoehma
Copy link

ahoehma commented Nov 14, 2017

Hello ... Here are one example of a hanging build ...

$ mvn -version
Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-03T21:39:06+02:00)
Maven home: D:\Dev\ApacheMaven\current-maven
Java version: 1.8.0_144, vendor: Oracle Corporation
Java home: D:\Dev\java\jdk8x64\jre
Default locale: de_DE, platform encoding: Cp1252
OS name: "windows 7", version: "6.1", arch: "amd64", family: "windows"

$ git status -s
RM ca01-common/src/main/java/com/siemens/spice/datamodel/application/extensions/service/pricing/CA01CommercialDataCalculationServiceImpl.java -> ca01-common/src/main/java/com/siemens/spice/datamodel/application/extensions/service/commercialdata/CommercialDataCalculationServicePmdPriceDbAdapter.java
RM ca01-common/src/test/java/com/siemens/spice/datamodel/application/extensions/service/pricing/CA01CommercialDataCalculationServiceImplTest.java -> ca01-common/src/test/java/com/siemens/spice/datamodel/application/extensions/service/commercialdata/CommercialDataCalculationServicePmdPriceDbAdapterTest.java
M configui-common/src/main/java/com/siemens/spice/configui/context/ConfigUiContext.java
M configui-common/src/main/java/com/siemens/spice/configui/header/CommercialDataArea.java
M configui-common/src/main/java/com/siemens/spice/configui/spring/ConfigUiPricingConfiguration.java
D configui-common/src/main/java/com/siemens/spice/configui/spring/ConfigUiPricingConfigurationCa01.java
M datamodel/src/main/java/com/siemens/spice/datamodel/application/extensions/service/commercialdata/CommercialDataCalculationServiceImpl.java
M datamodel/src/main/java/com/siemens/spice/datamodel/application/extensions/service/ordernumber/OrderNumberUtils.java
M datamodel/src/main/java/com/siemens/spice/datamodel/application/extensions/service/pricing/CustomerDataPricingService.java
M datamodel/src/main/java/com/siemens/spice/datamodel/application/plugins/service/commercialdata/CommercialDataCalculationService.java
M datamodel/src/test/java/com/siemens/spice/datamodel/application/extensions/service/ordernumber/OrderNumberUtilsTest.java
M environment-properties-interfaces/src/main/java/com/siemens/spice/environment/properties/hscm/HscmApplicationProperties.java
M environment-properties/src/main/java/com/siemens/spice/environment/properties/hscm/HscmCa01.java
M environment-properties/src/main/java/com/siemens/spice/environment/properties/hscm/HscmDefaults.java
M hscm-vaadin-webapp/pom.xml
D hscm-vaadin-webapp/src/main/ca01/java/com/siemens/spice/hscm/application/extensions/service/commercialdata/CommercialDataCalculationServiceImplCA01.java
D hscm-vaadin-webapp/src/main/ca01/java/com/siemens/spice/hscm/integration/CommercialDataCalculationServiceProductBundleCa01.java
D hscm-vaadin-webapp/src/main/ca01/java/com/siemens/spice/hscm/integration/configui/commercialdata/ConfigUiCommercialDataConfigurationCA01.java
D hscm-vaadin-webapp/src/main/ca01/java/com/siemens/spice/hscm/manager/initialization/HscmInitializationManagerCa01.java
D hscm-vaadin-webapp/src/main/ca01/java/com/siemens/spice/hscm/spring/plugins/PluginPricingConfigurationCA01.java
D hscm-vaadin-webapp/src/main/ca01/java/com/siemens/spice/hscm/spring/productbundle/ConfigUiCommericalDataConfigurationCA01.java
M hscm-vaadin-webapp/src/main/java/com/siemens/spice/hscm/integration/CommercialDataCalculationServiceProductBundle.java
M hscm-vaadin-webapp/src/main/java/com/siemens/spice/hscm/integration/configui/spring/ConfigUiCommercialDataConfiguration.java
M hscm-vaadin-webapp/src/main/java/com/siemens/spice/hscm/integration/configui/spring/ConfigUiSessionConfiguration.java
M hscm-vaadin-webapp/src/main/java/com/siemens/spice/hscm/manager/initialization/HscmInitializationManager.java
M hscm-vaadin-webapp/src/main/java/com/siemens/spice/hscm/spring/more/CommericalDataServiceConfiguration.java
M hscm-vaadin-webapp/src/main/java/com/siemens/spice/hscm/spring/plugins/PluginPricingConfiguration.java
M pcmplus-vaadin-webapp/src/main/ca01/java/com/siemens/spice/pcmplus/integration/goos/PcmPlusCommercialDataCalculationServiceCa01Impl.java
M pcmplus-vaadin-webapp/src/main/ca01/java/com/siemens/spice/pcmplus/spring/ca01/CommercialDataAreaConfiguration.java
M pcmplus-vaadin-webapp/src/main/ca01/java/com/siemens/spice/pcmplus/spring/ca01/PricingConfiguration.java
M pcmplus-vaadin-webapp/src/main/java/com/siemens/spice/pcmplus/integration/goos/PcmPlusCommercialDataCalculationServiceImpl.java
M pcmplus-vaadin-webapp/src/main/java/com/siemens/spice/pcmplus/spring/CommercialDataAreaConfiguration.java
?? ca01-common/src/main/java/com/siemens/spice/datamodel/application/extensions/service/commercialdata/CommercialDataCalculationServiceImplCA01.java
?? hscm-vaadin-webapp/src/main/java/com/siemens/spice/hscm/integration/CommercialDataCalculationServiceProductBundleCa01.java

So not soo many changes .... but the build hangs :)

$ /d/Dev/Java/jdk8x64/bin/jps -v
1596 Launcher -XX:+TieredCompilation -XX:TieredStopAtLevel=1 -Dmaven.artifact.threads=4 -DdependencyLocationsEnabled=false -Dclassworlds.conf=D:/Dev/ApacheMaven/current-maven/bin/m2.conf -Dmaven.home=D:/Dev/ApacheMaven/current-maven -Dmaven.multiModuleProjectDirectory=D:/Dev/git/coc/goos

$ /d/Dev/Java/jdk8x64/bin/jstack 1596
2017-11-14 10:51:06
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.144-b01 mixed mode):

"Service Thread" #10 daemon prio=9 os_prio=0 tid=0x000000000ebc6800 nid=0x31ac runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C1 CompilerThread3" #9 daemon prio=9 os_prio=2 tid=0x000000000eb5e000 nid=0xa74 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C2 CompilerThread2" #8 daemon prio=9 os_prio=2 tid=0x000000000eb38800 nid=0x164c waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" #7 daemon prio=9 os_prio=2 tid=0x000000000eb33000 nid=0x2f00 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #6 daemon prio=9 os_prio=2 tid=0x000000000eb29800 nid=0x1f50 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Attach Listener" #5 daemon prio=5 os_prio=2 tid=0x000000000eb26800 nid=0xa2c waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" #4 daemon prio=9 os_prio=2 tid=0x000000000db88800 nid=0x343c runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Finalizer" #3 daemon prio=8 os_prio=1 tid=0x000000000db84000 nid=0xbf4 in Object.wait() [0x000000001f04e000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000006c3ec7c08> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x00000006c3ec7c08> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

"Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x000000000db7d000 nid=0x1de0 in Object.wait() [0x000000001ef0f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000006c3319c40> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
- locked <0x00000006c3319c40> (a java.lang.ref.Reference$Lock)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

"main" #1 prio=5 os_prio=0 tid=0x000000000223a800 nid=0x3670 runnable [0x0000000002f3e000]
java.lang.Thread.State: RUNNABLE
at java.lang.ProcessImpl.waitForInterruptibly(Native Method)
at java.lang.ProcessImpl.waitFor(ProcessImpl.java:449)
at pl.project13.maven.git.NativeGitProvider$JavaProcessRunner.runEmpty(NativeGitProvider.java:399)
at pl.project13.maven.git.NativeGitProvider.tryCheckEmptyRunGitCommand(NativeGitProvider.java:256)
at pl.project13.maven.git.NativeGitProvider.isDirty(NativeGitProvider.java:168)
at pl.project13.maven.git.GitDataProvider.loadGitData(GitDataProvider.java:118)
at pl.project13.maven.git.GitCommitIdMojo.loadGitDataWithNativeGit(GitCommitIdMojo.java:505)
at pl.project13.maven.git.GitCommitIdMojo.loadGitData(GitCommitIdMojo.java:486)
at pl.project13.maven.git.GitCommitIdMojo.execute(GitCommitIdMojo.java:372)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:309)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:194)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:107)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:993)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:345)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:191)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

"VM Thread" os_prio=2 tid=0x000000000ead3000 nid=0xb78 runnable

"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x000000000224e800 nid=0x3644 runnable

"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x0000000002250000 nid=0x28ec runnable

"GC task thread#2 (ParallelGC)" os_prio=0 tid=0x0000000002251800 nid=0x35e4 runnable

"GC task thread#3 (ParallelGC)" os_prio=0 tid=0x0000000002253000 nid=0x34ac runnable

"GC task thread#4 (ParallelGC)" os_prio=0 tid=0x0000000002256800 nid=0x3444 runnable

"GC task thread#5 (ParallelGC)" os_prio=0 tid=0x0000000002257800 nid=0x30e8 runnable

"GC task thread#6 (ParallelGC)" os_prio=0 tid=0x000000000225a000 nid=0x15b0 runnable

"GC task thread#7 (ParallelGC)" os_prio=0 tid=0x000000000225d000 nid=0xd0c runnable

"VM Periodic Task Thread" os_prio=2 tid=0x000000000ebdd800 nid=0x375c waiting on condition

JNI global references: 271

@ahoehma
Copy link

ahoehma commented Nov 14, 2017

If I run maven with "-T 4" I got ... 4 hanging threads:

"BuilderThread 3" #15 prio=5 os_prio=0 tid=0x0000000022162800 nid=0xa0c runnable [0x000000001fc5e000]
java.lang.Thread.State: RUNNABLE
at java.lang.ProcessImpl.waitForInterruptibly(Native Method)
at java.lang.ProcessImpl.waitFor(ProcessImpl.java:449)
at pl.project13.maven.git.NativeGitProvider$JavaProcessRunner.runEmpty(NativeGitProvider.java:399)
at pl.project13.maven.git.NativeGitProvider.tryCheckEmptyRunGitCommand(NativeGitProvider.java:256)
at pl.project13.maven.git.NativeGitProvider.isDirty(NativeGitProvider.java:168)
at pl.project13.maven.git.GitDataProvider.loadGitData(GitDataProvider.java:118)
at pl.project13.maven.git.GitCommitIdMojo.loadGitDataWithNativeGit(GitCommitIdMojo.java:505)
at pl.project13.maven.git.GitCommitIdMojo.loadGitData(GitCommitIdMojo.java:486)
at pl.project13.maven.git.GitCommitIdMojo.execute(GitCommitIdMojo.java:372)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:200)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:196)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

"BuilderThread 2" #14 prio=5 os_prio=0 tid=0x0000000021d88000 nid=0x371c runnable [0x0000000022cae000]
java.lang.Thread.State: RUNNABLE
at java.lang.ProcessImpl.waitForInterruptibly(Native Method)
at java.lang.ProcessImpl.waitFor(ProcessImpl.java:449)
at pl.project13.maven.git.NativeGitProvider$JavaProcessRunner.runEmpty(NativeGitProvider.java:399)
at pl.project13.maven.git.NativeGitProvider.tryCheckEmptyRunGitCommand(NativeGitProvider.java:256)
at pl.project13.maven.git.NativeGitProvider.isDirty(NativeGitProvider.java:168)
at pl.project13.maven.git.GitDataProvider.loadGitData(GitDataProvider.java:118)
at pl.project13.maven.git.GitCommitIdMojo.loadGitDataWithNativeGit(GitCommitIdMojo.java:505)
at pl.project13.maven.git.GitCommitIdMojo.loadGitData(GitCommitIdMojo.java:486)
at pl.project13.maven.git.GitCommitIdMojo.execute(GitCommitIdMojo.java:372)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:200)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:196)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

"BuilderThread 1" #13 prio=5 os_prio=0 tid=0x0000000021d3d800 nid=0x37b8 runnable [0x0000000022f3e000]
java.lang.Thread.State: RUNNABLE
at java.lang.ProcessImpl.waitForInterruptibly(Native Method)
at java.lang.ProcessImpl.waitFor(ProcessImpl.java:449)
at pl.project13.maven.git.NativeGitProvider$JavaProcessRunner.runEmpty(NativeGitProvider.java:399)
at pl.project13.maven.git.NativeGitProvider.tryCheckEmptyRunGitCommand(NativeGitProvider.java:256)
at pl.project13.maven.git.NativeGitProvider.isDirty(NativeGitProvider.java:168)
at pl.project13.maven.git.GitDataProvider.loadGitData(GitDataProvider.java:118)
at pl.project13.maven.git.GitCommitIdMojo.loadGitDataWithNativeGit(GitCommitIdMojo.java:505)
at pl.project13.maven.git.GitCommitIdMojo.loadGitData(GitCommitIdMojo.java:486)
at pl.project13.maven.git.GitCommitIdMojo.execute(GitCommitIdMojo.java:372)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:200)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:196)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

"BuilderThread 0" #12 prio=5 os_prio=0 tid=0x0000000021cc6800 nid=0x35dc runnable [0x0000000022dee000]
java.lang.Thread.State: RUNNABLE
at java.lang.ProcessImpl.waitForInterruptibly(Native Method)
at java.lang.ProcessImpl.waitFor(ProcessImpl.java:449)
at pl.project13.maven.git.NativeGitProvider$JavaProcessRunner.runEmpty(NativeGitProvider.java:399)
at pl.project13.maven.git.NativeGitProvider.tryCheckEmptyRunGitCommand(NativeGitProvider.java:256)
at pl.project13.maven.git.NativeGitProvider.isDirty(NativeGitProvider.java:168)
at pl.project13.maven.git.GitDataProvider.loadGitData(GitDataProvider.java:118)
at pl.project13.maven.git.GitCommitIdMojo.loadGitDataWithNativeGit(GitCommitIdMojo.java:505)
at pl.project13.maven.git.GitCommitIdMojo.loadGitData(GitCommitIdMojo.java:486)
at pl.project13.maven.git.GitCommitIdMojo.execute(GitCommitIdMojo.java:372)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:200)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:196)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

@TheSnoozer
Copy link
Collaborator

Hi,
thanks for providing further details.
Based on the environment (e.g. /d/Dev/Java/jdk8x64/bin/jps) this looks like cygwin.
I try to reproduce it with something similar....

Just to get some more background about the environment (and to be on the safe side):

  • in what context maven is being executed (e.g. inside Windows Terminal, Powershell, Git Bash, /bin/bash, ...)? For this looks like cygwin
  • What version of cygwin are you using? uname -r or uname -a should tell.
  • Is this the cygwin-git or the windows git? git --version and/or git --info-path should tell.

@andreas-eternach
Copy link
Contributor Author

Hi,

thanks for the answer. More information:

  • we use git-bash in a windows-shell (cmd), opened by https://conemu.github.io/
  • "uname -a" -> "MINGW64_NT-6.1 MD1HPX8C 2.8.2(0.313/5/3) 2017-07-12 15:35 x86_64 Msys"
  • git version:
$ git --info-path
C:\Program Files\Git\mingw64/share/info

$ git --version
git version 2.14.1.windows.1 

BR, Andreas

@andreas-eternach
Copy link
Contributor Author

andreas-eternach commented Nov 29, 2017

Hi,

i have an idea, i see renaming-changes in git-output. I try to send the output as message to you.

git-status-s-output.txt
BR,

@TheSnoozer
Copy link
Collaborator

TheSnoozer commented Nov 29, 2017

Hi,
thanks for the additional information.
I fear I might not quite follow on how this would impact the status.
Regardless if you rename stuff or just modify it the status of the repository should be dirty.
Or are you trying to say whenever you rename files the process hangs, but if if you have simple changes like modifications it doesn't hang?

I can't see how this would be related to the code we have to check for the native git status -s. In general what we do here is to check if the command produced any output (see runEmpty method in https://github.com/ktoso/maven-git-commit-id-plugin/blob/master/src/main/java/pl/project13/maven/git/NativeGitProvider.java#L431). This should be independent of the actual change you had done.

I'm not sure, but looking at the release notes for git for windows there seem to happen a bug where git could run into an infinite loop trying to rename a file (see git-for-windows/git#1299) which was fixed with 2.14.2 (based on your feedback you seem to use 2.14.1.windows.1). Considering the background you might see, I would suggest to give it a shot an attempt to update the git-binary and see if the problem is still present.

Edit:
In general it seems reasonable to introduce some sort of maximum wait to gather the information requested. Unfortunately an out-of-the box solution was "only" introduced with Java 8 (https://docs.oracle.com/javase/8/docs/api/java/lang/Process.html#waitFor-long-java.util.concurrent.TimeUnit-). Since the plugin still supports Java 7 we would need to build something custom...

@TheSnoozer TheSnoozer added this to the 2.3.0 milestone Dec 5, 2017
@TheSnoozer
Copy link
Collaborator

I believe this issue is causes by a misbehaving native git binary. In particular I believe it has to do with git for windows where there seem to happen a bug where git could run into an infinite loop trying to rename a file (see above comment as well) which was fixed with 2.14.2 (based on your feedback you seem to use 2.14.1.windows.1).

Since this is nothing I can fix for the native git binary, I'll still set this issue for to be fixed for 2.3.0 milestone with the reduced scope that the native git process should use waitFor with a timeout (https://docs.oracle.com/javase/8/docs/api/java/lang/Process.html#waitFor-long-java.util.concurrent.TimeUnit-). Besides that I believe there is nothing I can do from the plugin side since in this case we rely on the native git.

@TheSnoozer TheSnoozer modified the milestones: 2.3.0, v3.0-modularized Jan 22, 2018
TheSnoozer pushed a commit to TheSnoozer/git-commit-id-maven-plugin that referenced this issue Sep 1, 2018
…s' that allows to specify a timeout for any call to the native binary in case of issues while fetching information
TheSnoozer pushed a commit to TheSnoozer/git-commit-id-maven-plugin that referenced this issue Sep 1, 2018
…s' that allows to specify a timeout for any call to the native binary in case of issues while fetching information
@Nogyara
Copy link

Nogyara commented Sep 7, 2018

Despite not being an expert in this particular area, I strongly suspect the JavaProcessRunner is to blame. Using ProcessBuilder and Process handling is quite tricky - there are multiple ways what could go wrong and as a result, the calling code may hang. The tricky part is that it doesn't happen always but only sometimes - guess it depends on timing, amount of output, etc.
Some sample issues with this desribed https://thilosdevblog.wordpress.com/2011/11/21/proper-handling-of-the-processbuilder/ (desribes hanging issue) or in
https://stackoverflow.com/questions/18423464/processbuilder-hangs-when-executing-batch-script (notices timing issue which may cause first several readLine() attempts to return null). Rather than solving this common issues, maybe a library like https://commons.apache.org/proper/commons-exec/ or more modern https://github.com/zeroturnaround/zt-exec would help here to save some headaches?

@TheSnoozer
Copy link
Collaborator

TheSnoozer commented Sep 7, 2018

Agree, any code can have issues ;-)

Unfortunately it is hard to tell what is to blame.
Is it Cygwin? is it the native git binary? Is it the plugin? Is it a combination of cygwin and some specific windows git binary?

Since this (in all fairness) seems to be an isolated problem, I would not really suspect the plugin (otherwise this project would have been flooded with issues). As of now I truthfully don't know what caused the problem, since I could not reproduce it.

To some extend I agree that using libraries might solve some problems, but in this case the plugin is performing interactions to java's processbuilder. Now assuming we use a wrapper - who tells me that i'm using the wrapper library correctly? Do we then need to use a wrapper for the wrapper to wrap a call to a processbuilder? No not really. I strongly believe in "not reinvent" the wheel, but in this specific case I'd rather fix the bug in the plugin instead of ending up to use a wrapper that might have the same issue. or in other words if I'm too stupid to use the processbuilder correctly, I'd rather fix it properly instead of relying on a wrapper.

@Nogyara
Copy link

Nogyara commented Sep 10, 2018

Unfortunately, the plugin hangs for everyone in my team but me, so today we had to remove it from our project :(. I'd love to have it back but this issues is a showstopper for us.

Please, have a look at the two links I posted. Me blaming NativeGitProvider (or JavaProcessRunner) is not just a blind guess, it's based on several hints:

  • the stacktrace - it hangs within JavaProcessRunner.runEmpty()
  • the linked articles mention two issues with current JavaProcessRunner code - one being readLine() may be called too early, the other that the output from Process must be consumed completely, otherwise the process hangs (see also Process's class JavaDoc) - which is exactly what I think happens in this case - not all the output is consumed

The reason why it happens only sometimes may be exactly the timing issue - sometimes the process returns output quickly enough so it is successfully consumed (or doesn't produce any output at ll), in other cases it blocks/hangs (within waitFor() method which probably wait for the output to be consumed).

Using wrapper libraries of not is of course your call, but in my experience it's not a bad idea if an API like this one is tricky to use.

One last thing - in our team, we all use Git on Windows, run from Windows command line, so we can rule out Cygwin, and we tried with several Git for Win versions (2.14.1 or the latest 2.18.1) - and it still happens :(

@TheSnoozer
Copy link
Collaborator

TheSnoozer commented Sep 10, 2018

Could you provide me some more information about the systems where the bug occurs?
On what specific windows instances can you reproduce the issue (e.g. windows 10, XP, ...)?
Is the repository private / public so I could reproduce it?
What version of the plugin are you using?

How many commits do you have in your repo (git rev-list --all --count)?
When the plugin is stuck in runEmpty() it tries to run git status -s. Could you manually run this command on the instances that are affected? Is it potentially a bigger commit that causes that behaviour?

What are the different status'es that are reported by git status -s?
E.g.

M = modified
A = added
D = deleted
R = renamed

What is the output of where git (under windows that should be equivalent to which).

Edit:
I would be really interested in the plugin version that is in use.
That issue might be related to 49e17a1

I'm trying to investigate this myself...

TheSnoozer pushed a commit that referenced this issue Sep 13, 2018
#346 / #336 / #350: Drop Java 1.7 support / introduce a timeout for any native git command / replace almost every guava feature with Java 8 features
andreas-eternach added a commit to andreas-eternach/maven-git-commit-id-plugin that referenced this issue Nov 2, 2018
… too much output on stdout/stderr.

The previous implementation read the output after the subprocess finished. In case the buffer is full, the subprocess waits infinitly.

Therefore the prototype implementation flushes the subprocess buffer by reading from stdout/stderr on background threads.
@andreas-eternach
Copy link
Contributor Author

i commited a prototype fix to a clone repo, @TheSnoozer pls check the commit andreas-eternach@75b1951

@TheSnoozer
Copy link
Collaborator

Thanks for (re-)raising this issue.

Have you tested if this would fix the problem?
The changes in the particular commit seem reasonable (haven't tested).
In general though your repository seem slightly out of sync with this (rebase your changes?).
Also introducing the commons-io dependency just for IOUtils.toByteArray seems a bit excessive. I agree do not reinvent the wheel, but I also would (try to) argue that dependencies for a simple call might be overkill. I would less resistant to fight dependencies like the ones who would do the magic for us (e.g. as outlined in #336 (comment)). Not sure if they would fix the problem though....

Feel free to open a MR and I would be happy to review it in more detail.

@andreas-eternach
Copy link
Contributor Author

I am still working on it, hope to come up with a final solution next week.

It took me some time to understand how the git-sandboxes are built in the integration-tests, pretty awesome stuff ;-) I will try to build a git-snadbox to reproduce the problem.

Thanks for (re-)raising this issue.

Have you tested if this would fix the problem?
The changes in the particular commit seem reasonable (haven't tested).
In general though your repository seem slightly out of sync with this (rebase your changes?).
Also introducing the commons-io dependency just for IOUtils.toByteArray seems a bit excessive. I agree do not reinvent the wheel, but I also would (try to) argue that dependencies for a simple call might be overkill. I would less resistant to fight dependencies like the ones who would do the magic for us (e.g. as outlined in #336 (comment)). Not sure if they would fix the problem though....

Feel free to open a MR and I would be happy to review it in more detail.

andreas-eternach added a commit to andreas-eternach/maven-git-commit-id-plugin that referenced this issue Nov 5, 2018
* revert introduction of new dependency for stream-handling and use jdk-stuff instead
* add a test case which reproduces the problem, the test case is added as commented because it takes 30 seconds for execution.
TheSnoozer added a commit that referenced this issue Nov 21, 2018
#336: fix hanging git subprocess in case of too much output on stdout/stderr.
TheSnoozer pushed a commit to TheSnoozer/git-commit-id-maven-plugin that referenced this issue Nov 21, 2018
@TheSnoozer
Copy link
Collaborator

Huge kudos to @andreas-eternach who reported this issue and put a lot of effort into getting it fixed.
Since I merged his changes I'm confident this can be closed for the up-comming 3.0 version.

Thank you for your contribution!

TheSnoozer added a commit that referenced this issue Nov 26, 2018
#336 / #396: apply some refactoring to simplify the entire process launching
cdc6d added a commit to cdc6d/geomesa that referenced this issue Dec 6, 2018
Upgrade git-commit-id-plugin from 2.2.1 to the latest version,
2.2.5, in the root POM.

The previous versions of the plugin were hanging for me
when trying to build.  Upgrading fixed that issue for me.
This might have to do with the following bug:

git-commit-id/git-commit-id-maven-plugin#336

Note: the version in the root POM was last updated from 2.1.13 to
2.2.1 in May 2017.

In 3 other POMs, remove git-commit-id-plugin <version> which was set
to 2.1.13 (perhaps originally copied from the root POM):

 * geomesa-accumulo-tools/pom.xml,
   geomesa-lambda-tools/pom.xml: version 2.1.13 was selected
   in the initial commits of these POMs in Sep 2016 and July 2017.
   The latter may have been copy-pasted from the former.

 * geomesa-fs-tools/pom.xml: version 2.1.13 was selected in the
   git-commit-id-plugin configuration when it was added to this
   build in Sep 2017... perhaps copy-paste from one of the above
   two POMs.
cdc6d added a commit to cdc6d/geomesa that referenced this issue Dec 6, 2018
Upgrade git-commit-id-plugin from 2.2.1 to the latest version,
2.2.5, in the root POM.

The previous versions of the plugin were hanging for me
when trying to build.  Upgrading fixed that issue for me.
This might have to do with the following bug:

git-commit-id/git-commit-id-maven-plugin#336

Note: the version in the root POM was last updated from 2.1.13 to
2.2.1 in May 2017.

In 3 other POMs, remove git-commit-id-plugin <version> which was set
to 2.1.13 (perhaps originally copied from the root POM):

 * geomesa-accumulo-tools/pom.xml,
   geomesa-lambda-tools/pom.xml: version 2.1.13 was selected
   in the initial commits of these POMs in Sep 2016 and July 2017.
   The latter may have been copy-pasted from the former.

 * geomesa-fs-tools/pom.xml: version 2.1.13 was selected in the
   git-commit-id-plugin configuration when it was added to this
   build in Sep 2017... perhaps copy-paste from one of the above
   two POMs.

Signed-off-by: Carsten Clark <github-cdc@ccri.com>
jnh5y pushed a commit to locationtech/geomesa that referenced this issue Dec 12, 2018
* GEOMESA-2478 Update git-commit-id-plugin to latest version.

Upgrade git-commit-id-plugin from 2.2.1 to the latest version,
2.2.5, in the root POM.

The previous versions of the plugin were hanging for me
when trying to build.  Upgrading fixed that issue for me.
This might have to do with the following bug:

git-commit-id/git-commit-id-maven-plugin#336

Note: the version in the root POM was last updated from 2.1.13 to
2.2.1 in May 2017.

In 3 other POMs, remove git-commit-id-plugin <version> which was set
to 2.1.13 (perhaps originally copied from the root POM):

 * geomesa-accumulo-tools/pom.xml,
   geomesa-lambda-tools/pom.xml: version 2.1.13 was selected
   in the initial commits of these POMs in Sep 2016 and July 2017.
   The latter may have been copy-pasted from the former.

 * geomesa-fs-tools/pom.xml: version 2.1.13 was selected in the
   git-commit-id-plugin configuration when it was added to this
   build in Sep 2017... perhaps copy-paste from one of the above
   two POMs.

Signed-off-by: Carsten Clark <github-cdc@ccri.com>

* GEOMESA-2478 Update all core Maven plugins to their latest versions.

Update all maven-*-plugin entries in the root POM's pluginManagement
to the latest version for each plugin.  Latest version info obtained
from https://maven.apache.org/plugins/ which turned out to match
what was available in Maven Central at
http://repo.maven.apache.org/maven2/org/apache/maven/plugins/

Add a comment to each maven-*-plugin configuration saying when
the version was released, to make it easier to spot potentially
out-of-date plugins.

Upgrading the maven-compiler-plugin required turning off its
new 'useIncrementalCompilation' option in order to keep it from
recompiling Java files that the scala-maven-plugin already compiled.

I see no significant differences in the output of "build/mvn install"
before and after this change (other than the plugin version numbers).

Signed-off-by: Carsten Clark <github-cdc@ccri.com>

* GEOMESA-2478 Add pluginManagement entries for 3 missing core Maven plugins.

Again no significant diffs in the console output of a full build.

Signed-off-by: Carsten Clark <github-cdc@ccri.com>

* GEOMESA-2478 Bump license-maven-plugin from 2.10 to 3.10 in root POM.

3.0 was released 2016-08-20 and is the latest as of 2018-11-30.

Again no significant diffs in the console output of a full build.

Signed-off-by: Carsten Clark <github-cdc@ccri.com>

* GEOMESA-2478 Upgrade scala-maven-plugin from 3.2.1 to 3.4.4 in the root POM.

This version of the plugin requires at least Maven 3.3.9.
Update README.md accordingly.  Delete the incomplete building
instructions in CONTRIBUTING.md and refer to README.md instead.

Since version 3.3.1, the scala-maven-plugin's way of invoking
Zinc changed so that Zinc may look in the wrong path for the
"javac" command.  Since version 3.3.2, this problem can be
defeated by setting the JAVA_HOME environment variable:

davidB/scala-maven-plugin#227

Add a note about this to the README because it caused me trouble.

No significant diffs in the console output of a full build.

Signed-off-by: Carsten Clark <github-cdc@ccri.com>
@JevonQ JevonQ mentioned this issue Dec 29, 2018
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants