Bug/4645 graph node containing keyword

[ibrahimWassouf]

📑 Summary

Fix issues relating to keywords being parsed in node idStrings, vertex text, and edge text

Resolves #4645

📏 Design Decisions

Previously, tokens were modularized to many parts. E.g., there was no Alphanumeric token, instead, there was an alphabet token ALPHA, a numbers token NUM, and an underscore token UNDERSCORE. The parser originally used recursion and concatenation on token types for things such as idString. However, this made it very difficult to debug issues such as the call and click keywords eliciting their respective states inside vertex and edge texts. Also, it was impossible to allow keywords in idStrings, because the parser would have to parse a keyword and then concatenate it to other tokens, resulting in eliciting errors for parsing a keyword in the wrong places.

To combat this, I added separate states for vertex and edge text. Now, we can add anything between the vertices and links without adding additional token types to the grammar (doing this is what resulted in super long definitions such as alphaNumToken). We can also have quotes inside vertices and edges without escaping them.

One problem I could not solve was allowing node idStrings to start with keywords. Previously, if the keyword occurred anywhere in the idString it would elicit an error. Now, it only elicits an error if it appears at the start of idString. E.g., graph-node and graph.node would elicit an error. However, it's important to note that an error will not be elicited if the keyword is followed by an underscore. I.e., graph_node is valid. I'm not sure why this happens, as the regex is valid. I spent a couple days on this problem alone and could not make it work.

📋 Tasks

Make sure you

• 📖 have read the contribution guidelines
• 💻 have added necessary unit/e2e tests.
• 📓 have added documentation. Make sure MERMAID_RELEASE_VERSION is used for all new features.
• 🔖 targeted develop branch

[codecov]

Codecov Report

Merging #4657 (34bf618) into develop (f8ebfee) will increase coverage by 31.68%.
The diff coverage is n/a.

Additional details and impacted files

@@             Coverage Diff              @@
##           develop    #4657       +/-   ##
============================================
+ Coverage    45.32%   77.00%   +31.68%     
============================================
  Files           52      144       +92     
  Lines         6645    14583     +7938     
  Branches        18      563      +545     
============================================
+ Hits          3012    11230     +8218     
+ Misses        3633     3243      -390     
- Partials         0      110      +110     

Flag	Coverage Δ
e2e	83.89% <ø> (?)
unit	45.32% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 138 files with indirect coverage changes

[ibrahimWassouf]

@nirname, let me know what you think when you get the chance

[nirname]

Kinda immediate answer without looking into the code: graph-node and graph.node will cause a error because recognizing graph takes precedence over that token you want to get during tokenizing.

[ibrahimWassouf]

Kinda immediate answer without looking into the code: graph-node and graph.node will cause a error because recognizing graph takes precedence over that token you want to get during tokenizing.

Shouldn't this logic also apply to when there is an underscore? I've tested it and it works

[ibrahimWassouf]

@nirname @sidharthv96, I thought of an additional edge case and was writing a unit test for it. Imagine we had a node with text such as A[This is a "()" in text]. Would we want the quotations marks to appear? In my current implementation, the quotes will appear but the parser will throw an error for including a node shape token instead of text. This is because quotation marks do not start the string state when inside a vertex or edge text.

I can do that of course, so in the end A[This is a "()" as text] will appear as a node containing This is a () as text. However, then users will not be able to easily including quotation marks. I believe they would have to resort to making the whole thing a string and using #quot; like they do now.

[sidharthv96]

Imagine we had a node with text such as A[This is a "()" in text]. Would we want the quotations marks to appear?

No. Quotes, if present, should wrap the entire string.

The following seems to be a good path for us to follow. What do you think?

Input	Expected Output	Current Behavior
A[This is a () in text]	Error	Same
A[This is a "()" in text]	Error	Same
A[This is a \"()\" in text]	Error	Same
A["This is a "()" in text"]	Error	Same
A["This is a () in text"]	This is a () in text	Same
A["This is a \"()\" in text"]	This is a "()" in text	This is a () in text

[sidharthv96]

Shouldn't this logic also apply to when there is an underscore? I've tested it and it works

node current expected
graph_node working working
graph-node error ?
graph.node error ?
graph/node error ?
class_node working working
class-node error ?
class.node error ?
class/node error ?

This is not trivial task. Perhaps we can postpone this, or may be even reject. Because it looks really strange. In case we allow dots and other special characters in a node name we can assume that someone would want to use graph_node along with graph.node. This is not critical, so may be we could keep graph_node because it works now, and prohibit other strange node names.

Supporting underscore is similar to variable names in most programming languages. We should keep things as-is here, and prohibit strange node names.

[nirname]

Insightful job! Nicely done