Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resource leak ? Maven build fails with "Too many open files in system" #258

Closed
svilenvul opened this issue Feb 19, 2018 · 6 comments
Closed

Resource leak ? Maven build fails with "Too many open files in system" #258

svilenvul opened this issue Feb 19, 2018 · 6 comments
Labels
bug An unexpected problem or unintended behavior of the tools dependency caused because of a dependency enhancement
Milestone
0.6.0

Comments

@svilenvul
Copy link
Contributor

Running multiple builds in a row with the SAT tool enabled on a single node in Jenkins produced the following errors:

[INFO] --- sat-plugin:0.5.0-SNAPSHOT:checkstyle (default) @ org.eclipse.smarthome.model.lsp ---
[INFO] Adding dependency to checkstyle:0.5.0-SNAPSHOT
[INFO] Adding dependency to checkstyle:8.1
[WARNING] Require-Bundle header line number not found. Fall back to 0.
org.openhab.tools.analysis.checkstyle.api.NoResultException: `org.eclipse.smarthome.model.item.ide,org.eclipse.smarthome.model.persistence.ide,org.eclipse.smarthome.model.rule.ide,org.eclipse.smarthome.model.script.ide,org.eclipse.smarthome.model.sitemap.ide,org.eclipse.smarthome.model.thing.ide,org.eclipse.smarthome.model.item,org.eclipse.smarthome.model.persistence,org.eclipse.smarthome.model.rule,org.eclipse.smarthome.model.script,org.eclipse.smarthome.model.sitemap,org.eclipse.smarthome.model.thing` was not found in the file /var/lib/jenkins/jobs/Qivicon-Opensource-ESH/workspace/bundles/model/org.eclipse.smarthome.model.lsp/META-INF/MANIFEST.MF starting from line `0`. Check if it is split between multiple lines or it is missing
	at org.openhab.tools.analysis.checkstyle.api.AbstractStaticCheck.findLineNumber(AbstractStaticCheck.java:87)
	at org.openhab.tools.analysis.checkstyle.api.AbstractStaticCheck.findLineNumberSafe(AbstractStaticCheck.java:105)
	at org.openhab.tools.analysis.checkstyle.RequireBundleCheck.processFiltered(RequireBundleCheck.java:79)
	at com.puppycrawl.tools.checkstyle.api.AbstractFileSetCheck.process(AbstractFileSetCheck.java:79)
	at com.puppycrawl.tools.checkstyle.Checker.processFile(Checker.java:316)
	at com.puppycrawl.tools.checkstyle.Checker.processFiles(Checker.java:286)
	at com.puppycrawl.tools.checkstyle.Checker.process(Checker.java:213)
	at org.apache.maven.plugin.checkstyle.exec.DefaultCheckstyleExecutor.executeCheckstyle(DefaultCheckstyleExecutor.java:252)
	at org.apache.maven.plugin.checkstyle.AbstractCheckstyleReport.executeReport(AbstractCheckstyleReport.java:473)
	at org.apache.maven.plugin.checkstyle.CheckstyleReport.executeReport(CheckstyleReport.java:154)
	at org.apache.maven.reporting.AbstractMavenReport.generate(AbstractMavenReport.java:255)
	at org.apache.maven.reporting.AbstractMavenReport.execute(AbstractMavenReport.java:143)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	at org.twdata.maven.mojoexecutor.MojoExecutor.executeMojo(MojoExecutor.java:94)
	at org.openhab.tools.analysis.tools.AbstractChecker.executeCheck(AbstractChecker.java:106)
	at org.openhab.tools.analysis.tools.CheckstyleChecker.execute(CheckstyleChecker.java:123)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
[ERROR] Problem occurred while processing directories. The check will exit without logging any warnings!
java.nio.file.FileSystemException: /var/lib/jenkins/jobs/Qivicon-Opensource-ESH/workspace/bundles/model/org.eclipse.smarthome.model.lsp/src/main/java/org/eclipse/smarthome/model: Too many open files
	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
	at sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:427)
	at java.nio.file.Files.newDirectoryStream(Files.java:457)
	at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:300)
	at java.nio.file.FileTreeWalker.next(FileTreeWalker.java:372)
	at java.nio.file.Files.walkFileTree(Files.java:2706)
	at java.nio.file.Files.walkFileTree(Files.java:2742)
	at org.openhab.tools.analysis.checkstyle.PackageExportsNameCheck.getFilteredPackagesFromSourceDirectory(PackageExportsNameCheck.java:114)
	at org.openhab.tools.analysis.checkstyle.PackageExportsNameCheck.processFiltered(PackageExportsNameCheck.java:90)
	at com.puppycrawl.tools.checkstyle.api.AbstractFileSetCheck.process(AbstractFileSetCheck.java:79)
	at com.puppycrawl.tools.checkstyle.Checker.processFile(Checker.java:316)
	at com.puppycrawl.tools.checkstyle.Checker.processFiles(Checker.java:286)
	at com.puppycrawl.tools.checkstyle.Checker.process(Checker.java:213)
	at org.apache.maven.plugin.checkstyle.exec.DefaultCheckstyleExecutor.executeCheckstyle(DefaultCheckstyleExecutor.java:252)
	at org.apache.maven.plugin.checkstyle.AbstractCheckstyleReport.executeReport(AbstractCheckstyleReport.java:473)
	at org.apache.maven.plugin.checkstyle.CheckstyleReport.executeReport(CheckstyleReport.java:154)
	at org.apache.maven.reporting.AbstractMavenReport.generate(AbstractMavenReport.java:255)
	at org.apache.maven.reporting.AbstractMavenReport.execute(AbstractMavenReport.java:143)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	at org.twdata.maven.mojoexecutor.MojoExecutor.executeMojo(MojoExecutor.java:94)
	at org.openhab.tools.analysis.tools.AbstractChecker.executeCheck(AbstractChecker.java:106)
	at org.openhab.tools.analysis.tools.CheckstyleChecker.execute(CheckstyleChecker.java:123)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
[WARNING] KarafFeatureCheck will be skipped. Could not find file features/openhab-addons/src/main/feature/feature.xml
[INFO] 
[INFO] --- sat-plugin:0.5.0-SNAPSHOT:pmd (default) @ org.eclipse.smarthome.model.lsp ---
[INFO] Adding dependency to pmd:0.5.0-SNAPSHOT
[INFO] Adding dependency to pmd-core:5.8.1
[INFO] Adding dependency to pmd-java:5.8.1
[INFO] Adding dependency to pmd-javascript:5.8.1
[INFO] Adding dependency to pmd-jsp:5.8.1
[INFO] 
[INFO] --- sat-plugin:0.5.0-SNAPSHOT:findbugs (default) @ org.eclipse.smarthome.model.lsp ---
[INFO] Adding dependency to findbugs:0.5.0-SNAPSHOT
[INFO] Adding dependency to bug-pattern:1.2.4
[INFO] Adding dependency to spotbugs:3.1.0
[INFO] Fork Value is false
     [java] JVM args ignored when same JVM is used.
Unable to obtain resource from /var/lib/jenkins/.m2/repository/org/apache/maven/maven-model-builder/3.5.0/maven-model-builder-3.5.0.jar: java.io.FileNotFoundException: /var/lib/jenkins/.m2/repository/org/apache/maven/maven-model-builder/3.5.0/maven-model-builder-3.5.0.jar (Too many open files)
java.io.FileNotFoundException: /var/lib/jenkins/.m2/repository/org/apache/maven/maven-model-builder/3.5.0/maven-model-builder-3.5.0.jar (Too many open files)
	at java.io.FileInputStream.open0(Native Method)
	at java.io.FileInputStream.open(FileInputStream.java:195)
	at java.io.FileInputStream.<init>(FileInputStream.java:138)
	at org.apache.tools.ant.AntClassLoader.readFully(AntClassLoader.java:1589)
	at org.apache.tools.ant.AntClassLoader.isZip(AntClassLoader.java:1578)
	at org.apache.tools.ant.AntClassLoader.getResourceURL(AntClassLoader.java:1022)
	at org.apache.tools.ant.AntClassLoader$ResourceEnumeration.findNextResource(AntClassLoader.java:152)
	at org.apache.tools.ant.AntClassLoader$ResourceEnumeration.<init>(AntClassLoader.java:113)
	at org.apache.tools.ant.AntClassLoader.findResources(AntClassLoader.java:969)
	at org.apache.tools.ant.AntClassLoader.getNamedResources(AntClassLoader.java:937)
	at org.apache.tools.ant.AntClassLoader.getResources(AntClassLoader.java:1554)
	at java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:348)
	at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393)
	at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474)
	at javax.xml.parsers.FactoryFinder$1.run(FactoryFinder.java:293)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.xml.parsers.FactoryFinder.findServiceProvider(FactoryFinder.java:289)
	at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:267)
	at javax.xml.parsers.SAXParserFactory.newInstance(SAXParserFactory.java:127)
	at org.dom4j.io.JAXPHelper.createXMLReader(JAXPHelper.java:46)
	at org.dom4j.io.SAXHelper.createXMLReaderViaJAXP(SAXHelper.java:125)
	at org.dom4j.io.SAXHelper.createXMLReader(SAXHelper.java:78)
	at org.dom4j.io.SAXReader.createXMLReader(SAXReader.java:894)
	at org.dom4j.io.SAXReader.getXMLReader(SAXReader.java:715)
	at org.dom4j.io.SAXReader.read(SAXReader.java:435)
	at org.dom4j.io.SAXReader.read(SAXReader.java:365)
	at edu.umd.cs.findbugs.PluginLoader.getPluginDescriptor(PluginLoader.java:1195)
	at edu.umd.cs.findbugs.PluginLoader.init(PluginLoader.java:712)
	at edu.umd.cs.findbugs.PluginLoader.<init>(PluginLoader.java:396)
	at edu.umd.cs.findbugs.PluginLoader.loadCorePlugin(PluginLoader.java:1486)
	at edu.umd.cs.findbugs.PluginLoader.loadInitialPlugins(PluginLoader.java:1440)
	at edu.umd.cs.findbugs.PluginLoader.<clinit>(PluginLoader.java:150)
	at edu.umd.cs.findbugs.DetectorFactoryCollection.getCoreResource(DetectorFactoryCollection.java:351)
	at edu.umd.cs.findbugs.SystemProperties.loadPropertiesFromConfigFile(SystemProperties.java:77)
	at edu.umd.cs.findbugs.SystemProperties.<clinit>(SystemProperties.java:60)
	at edu.umd.cs.findbugs.FindBugs2.<clinit>(FindBugs2.java:85)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at org.apache.tools.ant.taskdefs.ExecuteJava.execute(ExecuteJava.java:137)
	at org.apache.tools.ant.taskdefs.Java.run(Java.java:834)
	at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:228)
	at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:137)
	at org.apache.tools.ant.taskdefs.Java.execute(Java.java:110)
	at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293)
	at sun.reflect.GeneratedMethodAccessor294.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
	at groovy.util.AntBuilder.performTask(AntBuilder.java:327)
	at groovy.util.AntBuilder.nodeCompleted(AntBuilder.java:272)
	at groovy.util.BuilderSupport.doInvokeMethod(BuilderSupport.java:154)
	at groovy.util.AntBuilder.doInvokeMethod(AntBuilder.java:207)
	at groovy.util.BuilderSupport.invokeMethod(BuilderSupport.java:67)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
	at org.codehaus.mojo.findbugs.FindBugsMojo.executeFindbugs(FindBugsMojo.groovy:1036)
	at sun.reflect.GeneratedMethodAccessor270.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166)
	at org.codehaus.mojo.findbugs.FindBugsMojo.canGenerateReport(FindBugsMojo.groovy:549)
	at org.codehaus.mojo.findbugs.FindBugsMojo.execute(FindBugsMojo.groovy:688)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	at org.twdata.maven.mojoexecutor.MojoExecutor.executeMojo(MojoExecutor.java:94)
	at org.openhab.tools.analysis.tools.AbstractChecker.executeCheck(AbstractChecker.java:106)
	at org.openhab.tools.analysis.tools.FindBugsChecker.execute(FindBugsChecker.java:176)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
@svilenvul svilenvul added bug An unexpected problem or unintended behavior of the tools help wanted labels Feb 19, 2018
@svilenvul
Copy link
Contributor Author

I have traced the problem to SpotBugs, file handles to additional SpotBugs plugins are note released. So at the end of the ESH build there are more than 250 unreleased handles like this:

lr-x------ 1 svilenvalkanov svilenvalkanov 64 фев 23 10:34 786 -> /media/svilenvalkanov/C08C78B68C78A894/prj/openHAB/git/smarthome/bundles/io/org.eclipse.smarthome.io.console.rfc147/target/code-analysis/findbugs-0.4.1.jar
lr-x------ 1 svilenvalkanov svilenvalkanov 64 фев 23 10:34 787 -> /media/svilenvalkanov/C08C78B68C78A894/prj/openHAB/git/smarthome/bundles/io/org.eclipse.smarthome.io.console.rfc147/target/code-analysis/bug-pattern-1.2.4.jar

I have found similar issue opened in the sonar-spotbugs repo : spotbugs/sonar-findbugs#128

@svilenvul svilenvul added dependency caused because of a dependency and removed help wanted labels Feb 23, 2018
@svilenvul
Copy link
Contributor Author

See spotbugs/spotbugs-maven-plugin#35

@svilenvul
Copy link
Contributor Author

See also spotbugs/spotbugs#591

Expected in spotbugs 3.1.3 or 3.1.4

@svilenvul
Copy link
Contributor Author

Fix should be already available in the latest version released by SpotBugs (3.1.5 is already available).

@dstivanov
Copy link
Collaborator

I marked the task and we will increase t he SpotBugs version as soon as possible in order to include that fixed in the next SCAT release. Thanks for the help, @svilenvul !

@VelinYordanov VelinYordanov mentioned this issue Jun 21, 2018
Merged
@dstivanov dstivanov added this to the 0.6.0 milestone Jun 27, 2018
@dstivanov
Copy link
Collaborator

As the spotbugs version is already increased I would consider this issue as resolved as well. If it happens again during the builds - we will reopen it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug An unexpected problem or unintended behavior of the tools dependency caused because of a dependency enhancement
Projects
None yet
Milestone
0.6.0
Development

No branches or pull requests
2 participants
@svilenvul @dstivanov