Timeout for request POST /graphql when loading the Graphryder dashboard

[tanius]

In many cases, the current version of the Graphryder 2 dashboard will not load. The screen will then show this error message:

Error

JSON.parse: unexpected character at line 1 column 1 of the JSON data

As seen in the browser's Developer Tools console, this is caused by a timeout ("504 Gateway Time-out 60109 ms") of the following XHR request:

POST http://graphryder.edgeryders.eu/graphql

Observations:

• During the failing request POST /graphql, the Neo4J process on the server will consume one CPU core completely, and also continue to do so for some time after the POST request timed out. This is not the case when the request succeeds; then, the Neo4J process only occupies the CPU for a few seconds.

• This seems not to be a heap memory related issue. Increasing the Neo4J config file variable dbms.memory.heap.max_size from 4G to 7G did not change anything, and also, the memory usage of the whole process was below 3 GiB for the whole time during the failing POST request.

• This issue is connected to a state on the server so that it always affects a row of independent requests (resp. attempts to load the Graphryder dashboard). Either it happens not at all, or it happens for all attempts in a row to load the Graphryder dashboard, independent of which coding project you are trying to load into the Graphryder dashboard. It will then even happen for very small and even empty ethnographic corpora. (Example for an empty one that triggered the issue.)

• Restarting Neo4J with sudo systemctl restart neo4j.service will not resolve the issue. Or at least, not immediately. Probably it does resolve it after 3-5 minutes, when the Neo4J startup activities have subsided.

• Trying again after 20-30 minutes often resolves the issue.

• 💡 Restarting the whole server resolves the issue reliably.

• This issue has seemingly been introduced or triggered by the transition from tags to projects in Open Ethnographer. This required corresponding changes in graphryder-import-script as well. It also increased the size of the Neo4J database, as one code used in two projects is now represented as two separate database records; while before, one code associated with two Discourse ethno-* tags was one record in the database.

(This is an issue we discovered and discussed before. I could not find the original discussion and observations, though. Maybe @aerugo knows.)