Skip to content

OpenGrok webapp fails to start after adding a project #2468

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
brightchuh opened this issue Oct 30, 2018 · 18 comments
Closed

OpenGrok webapp fails to start after adding a project #2468

brightchuh opened this issue Oct 30, 2018 · 18 comments
Labels

Comments

@brightchuh
Copy link

Dear OpenGrok experts,

At first there're serveral projects under dir SRC already and my OpenGrok works well. Then I want to add a new project named opengrok, so i refer to wikipage: https://github.com/oracle/opengrok/wiki/Per-project-management, and then did the following operations:

  1. move opengrok source code into SRC dir;
  2. run following command one by one:
zhul@ubuntu1804:~/work/tools/opengrok/bin$ ./projadm.py \
 -U http://localhost:8080/opengrok \
 -b $OPENGROK -R $OPENGROK/etc/readonly_configuration.xml \
 -a opengrok \
 --jar $OPENGROK/lib/opengrok.jar 
Lock 139740936275448 acquired on /tmp/projadm.py.lock
Adding project opengrok
Refreshing configuration (merging with read-only config)
Lock 139740936275448 released on /tmp/projadm.py.lock

zhul@ubuntu1804:~/work/tools/opengrok/bin$ ./reindex-project.py \
   -U http://localhost:8080/opengrok \
   -t $OPENGROK/doc/logging.properties -p '%PROJ%' -d $OPENGROK/log \
   -P opengrok \
   --jar $OPENGROK/lib/opengrok.jar -- \
   -H -P -S -G -v --progress -t 4 

zhul@ubuntu1804:~/work/tools/opengrok/bin$ ./projadm.py \
 -U http://localhost:8080/opengrok \
 -b $OPENGROK \
 -R $OPENGROK/etc/readonly_configuration.xml \
 -r \
 --jar $OPENGROK/lib/opengrok.jar 
Lock 139901594406584 acquired on /tmp/projadm.py.lock
Refreshing configuration (merging with read-only config)
Lock 139901594406584 released on /tmp/projadm.py.lock

finally, i refreshed my OpenGrok webpage: http://localhost:8080/opengrok/, and then it goes wrong, even if i restart tomcat server.
it says:

HTTP Status 404 – Not Found
Type Status Report
Message /opengrok/
Description The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.
Apache Tomcat/9.0.10

@vladak
Copy link
Member

vladak commented Oct 30, 2018 via email

@vladak
Copy link
Member

vladak commented Oct 30, 2018

Also, how exactly was the webapp deployed ? Did the web app work after deployment ?

@brightchuh
Copy link
Author

v1.1-rc58

$ java -jar ../lib/opengrok.jar -V
10月 31, 2018 9:08:51 上午 org.opengrok.indexer.index.Indexer parseOptions
信息: Indexer options: [-V]
OpenGrok v1.1-rc58 rev ea3276555c6b40174cb20dc41d15677997f1cc79

@brightchuh
Copy link
Author

There're several projects under SRC dir at first. Then I deploy OpenGrok webapp using command:

$ ./deploy.py \
	-c "$OPENGROK/etc/configuration.xml" \
	$OPENGROK/lib/source.war \
	$TOMCAT/webapps/opengrok.war 

After webapp is deployed, I ran initial indexing using command:

$ ./indexer.py \
    -J=-Djava.util.logging.config.file=$OPENGROK/doc/logging.properties \
    -a $OPENGROK/lib/opengrok.jar -- \
    -s $CODE -d $OPENGROK/data \
    -R $OPENGROK/etc/readonly_configuration.xml \
    -W $OPENGROK/etc/configuration.xml \
    -U http://localhost:8080/opengrok \
    -H -P -S -G -v -t 4 

After all these are done, opengrok can be accessed at "http://localhost:8080/opengrok/", it works well.

Several days later, i added a new project just as what i said in my first comment, And it goes wrong.

@brightchuh
Copy link
Author

I'm not familiar with tomcat, so i attach all logs of yesterday.
tomcat_log_10.30.zip

@vladak
Copy link
Member

vladak commented Oct 31, 2018

In catalina log I see:

30-Oct-2018 17:12:20.849 <E4><BF><A1><E6><81><AF> [http-nio-8080-exec-18] org.opengrok.web.WebappListener.contextInitialized Starting webapp with version 1.1-rc52 (5ab556b6168923c51a64883b89e8d03398efc3da)
30-Oct-2018 17:12:20.849 <E4><BF><A1><E6><81><AF> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.Configuration.read Reading configuration from /home/zhul/work/tools/opengrok/etc/configuration.xml
30-Oct-2018 17:12:20.872 <E8><AD><A6><E5><91><8A> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.Project.getProject Path of project null is not set
30-Oct-2018 17:12:20.873 <E8><AD><A6><E5><91><8A> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.Project.getProject Path of project null is not set
30-Oct-2018 17:12:20.873 <E8><AD><A6><E5><91><8A> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.Project.getProject Path of project null is not set
30-Oct-2018 17:12:20.873 <E8><AD><A6><E5><91><8A> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.Project.getProject Path of project null is not set
30-Oct-2018 17:12:20.873 <E8><AD><A6><E5><91><8A> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.Project.getProject Path of project null is not set
30-Oct-2018 17:12:20.873 <E8><AD><A6><E5><91><8A> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.Project.getProject Path of project null is not set
30-Oct-2018 17:12:20.873 <E8><AD><A6><E5><91><8A> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.Project.getProject Path of project null is not set
30-Oct-2018 17:12:20.874 <E8><AD><A6><E5><91><8A> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.Project.getProject Path of project null is not set
30-Oct-2018 17:12:20.874 <E8><AD><A6><E5><91><8A> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.Project.getProject Path of project null is not set
30-Oct-2018 17:12:20.874 <E8><AD><A6><E5><91><8A> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.Project.getProject Path of project null is not set
30-Oct-2018 17:12:20.874 <E8><AD><A6><E5><91><8A> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.Project.getProject Path of project null is not set
30-Oct-2018 17:12:20.874 <E8><AD><A6><E5><91><8A> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.Project.getProject Path of project null is not set
30-Oct-2018 17:12:20.875 <E8><AD><A6><E5><91><8A> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.Project.getProject Path of project null is not set
30-Oct-2018 17:12:20.875 <E8><AD><A6><E5><91><8A> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.Project.getProject Path of project null is not set
30-Oct-2018 17:12:20.876 <E4><B8><A5><E9><87><8D> [http-nio-8080-exec-18] org.apache.catalina.core.StandardContext.startInternal One or more listeners failed to start. Full details will be found in the appropriate container log file
30-Oct-2018 17:12:20.877 <E4><B8><A5><E9><87><8D> [http-nio-8080-exec-18] org.apache.catalina.core.StandardContext.startInternal Context [/opengrok] startup failed due to previous errors
30-Oct-2018 17:12:20.878 <E4><BF><A1><E6><81><AF> [http-nio-8080-exec-18] org.opengrok.indexer.configuration.RuntimeEnvironment.stopWatchDogService Watchdog stoped
30-Oct-2018 17:12:20.892 <E4><BF><A1><E6><81><AF> [Thread-8] org.opengrok.suggest.Suggester.init No index directories found, exiting...

What are the contents of configuration.xml and readonly_configuration.xml ?

@vladak
Copy link
Member

vladak commented Oct 31, 2018

Also, there are some curious errors like:

30-Oct-2018 00:00:03.222 <E4><B8><A5><E9><87><8D> [ForkJoinPool-7-worker-2] org.opengrok.suggest.SuggesterProjectData.initSearchCountMap Could not create ChronicleMap, most popular completion disabled, if you are using JDK9+ make sure to specify: --add-exports java.base/jdk.internal.ref=ALL-UNNAMED --add-exports java.base/jdk.internal.misc=ALL-UNNAMED --add-exports java.base/sun.nio.ch=ALL-UNNAMED
 java.lang.RuntimeException: java.lang.IllegalAccessException: class net.openhft.chronicle.hash.impl.util.CleanerUtils cannot access class jdk.internal.ref.Cleaner (in module java.base) because module java.base does not export jdk.internal.ref to unnamed module @5cdd5ff9
        at net.openhft.chronicle.hash.impl.util.CleanerUtils.createCleaner(CleanerUtils.java:34)
        at net.openhft.chronicle.hash.impl.VanillaChronicleHash.registerCleaner(VanillaChronicleHash.java:469)
        at net.openhft.chronicle.map.ChronicleMapBuilder.prepareMapPublication(ChronicleMapBuilder.java:1672)
        at net.openhft.chronicle.map.ChronicleMapBuilder.createWithFile(ChronicleMapBuilder.java:1650)
        at net.openhft.chronicle.map.ChronicleMapBuilder.recoverPersistedTo(ChronicleMapBuilder.java:1586)
        at net.openhft.chronicle.map.ChronicleMapBuilder.createOrRecoverPersistedTo(ChronicleMapBuilder.java:1569)
        at net.openhft.chronicle.map.ChronicleMapBuilder.createOrRecoverPersistedTo(ChronicleMapBuilder.java:1560)
        at net.openhft.chronicle.map.ChronicleMapBuilder.createOrRecoverPersistedTo(ChronicleMapBuilder.java:1554)
        at org.opengrok.suggest.popular.impl.chronicle.ChronicleMapAdapter.<init>(ChronicleMapAdapter.java:55)
        at org.opengrok.suggest.SuggesterProjectData.initSearchCountMap(SuggesterProjectData.java:300)
        at org.opengrok.suggest.SuggesterProjectData.rebuild(SuggesterProjectData.java:237)
        at org.opengrok.suggest.Suggester.lambda$getRebuildRunnable$1(Suggester.java:247)
        at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1603)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
Caused by: java.lang.IllegalAccessException: class net.openhft.chronicle.hash.impl.util.CleanerUtils cannot access class jdk.internal.ref.Cleaner (in module java.base) because module java.base does not export jdk.internal.ref to unnamed module @5cdd5ff9
        at java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:360)
        at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:589)
        at java.base/java.lang.reflect.Method.invoke(Method.java:556)
        at net.openhft.chronicle.hash.impl.util.CleanerUtils.createCleaner(CleanerUtils.java:30)
        ... 15 more

What is the Java version on the system ?

@vladak
Copy link
Member

vladak commented Oct 31, 2018

The localhost log contains more info related to the null project:

30-Oct-2018 16:47:38.288 <E4><B8><A5><E9><87><8D> [main] org.apache.catalina.core.StandardContext.listenerStart Exception sending context initialized event to listener instance of class [org.opengrok.web.WebappListener]
 java.lang.NullPointerException
        at org.opengrok.indexer.configuration.Project.compareTo(Project.java:407)
        at org.opengrok.indexer.configuration.Project.compareTo(Project.java:41)
        at java.base/java.util.TreeMap.compare(TreeMap.java:1291)
        at java.base/java.util.TreeMap.put(TreeMap.java:536)
        at java.base/java.util.TreeSet.add(TreeSet.java:255)
        at java.base/java.util.AbstractCollection.addAll(AbstractCollection.java:352)
        at java.base/java.util.TreeSet.addAll(TreeSet.java:312)
        at java.base/java.util.TreeSet.<init>(TreeSet.java:160)
        at org.opengrok.indexer.configuration.RuntimeEnvironment.setConfiguration(RuntimeEnvironment.java:1476)
        at org.opengrok.indexer.configuration.RuntimeEnvironment.readConfiguration(RuntimeEnvironment.java:1322)
        at org.opengrok.web.WebappListener.contextInitialized(WebappListener.java:69)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4643)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5109)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:742)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:718)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:703)
        at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:986)
        at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1858)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
        at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:118)
        at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:772)
        at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:426)
        at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1585)
        at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:308)
        at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)
        at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:424)
        at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:367)
        at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:966)
        at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:839)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1427)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1417)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
        at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
        at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:943)
        at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:258)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
        at org.apache.catalina.core.StandardService.startInternal(StandardService.java:422)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
        at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:770)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
        at org.apache.catalina.startup.Catalina.start(Catalina.java:682)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:350)
        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:492)

@brightchuh
Copy link
Author

JAVA version

$ java --version
openjdk 10.0.1 2018-04-17
OpenJDK Runtime Environment (build 10.0.1+10-Ubuntu-3ubuntu1)
OpenJDK 64-Bit Server VM (build 10.0.1+10-Ubuntu-3ubuntu1, mixed mode)

@brightchuh
Copy link
Author

configuration.xml and readonly_configuration.xml
xml.zip

@vladak
Copy link
Member

vladak commented Nov 1, 2018

I guess some of the failures can be attributed to running with Java 10 - this is not well tested.

@tulinkry
Copy link
Contributor

tulinkry commented Nov 1, 2018

I think I see the reason for

30-Oct-2018 16:47:38.288 <E4><B8><A5><E9><87><8D> [main] org.apache.catalina.core.StandardContext.listenerStart Exception sending context initialized event to listener instance of class [org.opengrok.web.WebappListener]
 java.lang.NullPointerException
        at org.opengrok.indexer.configuration.Project.compareTo(Project.java:407)
        at org.opengrok.indexer.configuration.Project.compareTo(Project.java:41)

looks like a bug.

The read-only configuration contains a sample as in https://github.com/oracle/opengrok/wiki/Per-project-configuration - project (without a name) in a project map. Usually, when you run indexer the project should be filled with name in Indexer.java#934-942. However, the way you use it, running reindex-project (where is no option for -R) does not trigger this code. And having it with projadm just merges the read-only and the configuration.xml all together with missing project name.

This in the end means, that the configuration is invalid and any operation with projects will result in NPE.

@vladak
Copy link
Member

vladak commented Nov 1, 2018

Yes, this is actually a dup of #2147 for which the workaround is to comment out the per-project settings in read-only configuration after the first global reindex and then set per project properties using RESTful API.

@vladak vladak closed this as completed Nov 1, 2018
@brightchuh
Copy link
Author

brightchuh commented Nov 30, 2018

@vladak

Does the latest opengrok version resolve this issue?

@vladak
Copy link
Member

vladak commented Nov 30, 2018

No, I don't think so.

@vladak
Copy link
Member

vladak commented Nov 30, 2018

We need to make the code more resilient against invalid project configuration first.

@brightchuh
Copy link
Author

@vladak So, you mean that if i want to add a new project, i need to run initial index once more?

@vladak
Copy link
Member

vladak commented Dec 12, 2018

The workaround is to comment out any per-project configuration in read-only configuration and re-merge. Using per-project properties only works when running the indexer for the whole source root.

E.g. in our production's read-only config we have this part:

  <!-- Linux repositories have too rich history - disable handling of renamed files 
       However, this can only be enabled during initial reindex of the whole source root.
       After that this section has to be commented out, see https://github.com/oracle/opengrok/issues/2147
  -->
  <!--
  <void property="projects">
   <void method="put">
    <string>linux-mainline</string>
    <object class="org.opengrok.indexer.configuration.Project">
     <void property="handleRenamedFiles">
      <boolean>false</boolean>
     </void>
    </object>
   </void>

   <void method="put">
    <string>linux-mainline-next</string>
    <object class="org.opengrok.indexer.configuration.Project">
     <void property="handleRenamedFiles">
      <boolean>false</boolean>
     </void>
    </object>
   </void>
-->

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants