Parsing of nested templates is slow: template <... template<...>> #1685
Comments
|
Is this the same problem which is slowing down the following? |
|
I bumped in this issue in 1.3.2, just for information I thought would be fixed. |
|
Hello @ericlemes, We are currently working on a 2.0 release and your support would be very welcome. Regards, |
|
Other samples: using List= TypeA<A, B>; // fast: 1
using List= TypeA<A, TypeB<B, C>>; // fast: 2
using List= TypeA<A, TypeB<B, TypeC<C, D>>>; // fast: 3
using List= TypeA<A, TypeB<B, TypeC<C, TypeD<D, E>>>>; // fast: 4
using List= TypeA<A, TypeB<B, TypeC<C, TypeD<D, TypeE<E, F>>>>>; // fast: 5
using List= TypeA<A, TypeB<B, TypeC<C, TypeD<D, TypeE<E, TypeF<F, G>>>>>>; // fast: 6
using List= TypeA<A, TypeB<B, TypeC<C, TypeD<D, TypeE<E, TypeF<F, TypeG<G, H>>>>>>>; // 3s: 7
using List= TypeA<A, TypeB<B, TypeC<C, TypeD<D, TypeE<E, TypeF<F, TypeG<G, TypeH<H, I>>>>>>>>; // 127s: 8AST for <translationUnit tokenValue="using" tokenLine="1" tokenColumn="0">
<declaration tokenValue="using" tokenLine="1" tokenColumn="0">
<aliasDeclaration tokenValue="using" tokenLine="1" tokenColumn="0">
<USING tokenValue="using" tokenLine="1" tokenColumn="0"/>
<IDENTIFIER tokenValue="List" tokenLine="1" tokenColumn="6"/>
<ASSIGN tokenValue="=" tokenLine="1" tokenColumn="10"/>
<definingTypeId tokenValue="TypeA" tokenLine="1" tokenColumn="12">
<typeName tokenValue="TypeA" tokenLine="1" tokenColumn="12">
<className tokenValue="TypeA" tokenLine="1" tokenColumn="12">
<simpleTemplateId tokenValue="TypeA" tokenLine="1" tokenColumn="12">
<templateName tokenValue="TypeA" tokenLine="1" tokenColumn="12">
<IDENTIFIER tokenValue="TypeA" tokenLine="1" tokenColumn="12"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="17"/>
<innerTemplateId tokenValue="A" tokenLine="1" tokenColumn="18">
<templateArgument tokenValue="A" tokenLine="1" tokenColumn="18">
<typeName tokenValue="A" tokenLine="1" tokenColumn="18">
<className tokenValue="A" tokenLine="1" tokenColumn="18">
<IDENTIFIER tokenValue="A" tokenLine="1" tokenColumn="18"/>
</className>
</typeName>
</templateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="19"/>
<innerTypeId tokenValue="TypeB" tokenLine="1" tokenColumn="21">
<innerTrailingTypeSpecifier tokenValue="TypeB" tokenLine="1" tokenColumn="21">
<innerSimpleTemplateId tokenValue="TypeB" tokenLine="1" tokenColumn="21">
<templateName tokenValue="TypeB" tokenLine="1" tokenColumn="21">
<IDENTIFIER tokenValue="TypeB" tokenLine="1" tokenColumn="21"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="26"/>
<innerTemplateArgumentList tokenValue="B" tokenLine="1" tokenColumn="27">
<innerTemplateArgument tokenValue="B" tokenLine="1" tokenColumn="27">
<typeName tokenValue="B" tokenLine="1" tokenColumn="27">
<className tokenValue="B" tokenLine="1" tokenColumn="27">
<IDENTIFIER tokenValue="B" tokenLine="1" tokenColumn="27"/>
</className>
</typeName>
</innerTemplateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="28"/>
<innerTemplateArgument tokenValue="TypeC" tokenLine="1" tokenColumn="30">
<typeName tokenValue="TypeC" tokenLine="1" tokenColumn="30">
<className tokenValue="TypeC" tokenLine="1" tokenColumn="30">
<simpleTemplateId tokenValue="TypeC" tokenLine="1" tokenColumn="30">
<templateName tokenValue="TypeC" tokenLine="1" tokenColumn="30">
<IDENTIFIER tokenValue="TypeC" tokenLine="1" tokenColumn="30"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="35"/>
<innerTemplateId tokenValue="C" tokenLine="1" tokenColumn="36">
<templateArgument tokenValue="C" tokenLine="1" tokenColumn="36">
<typeName tokenValue="C" tokenLine="1" tokenColumn="36">
<className tokenValue="C" tokenLine="1" tokenColumn="36">
<IDENTIFIER tokenValue="C" tokenLine="1" tokenColumn="36"/>
</className>
</typeName>
</templateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="37"/>
<innerTypeId tokenValue="TypeD" tokenLine="1" tokenColumn="39">
<innerTrailingTypeSpecifier tokenValue="TypeD" tokenLine="1" tokenColumn="39">
<innerSimpleTemplateId tokenValue="TypeD" tokenLine="1" tokenColumn="39">
<templateName tokenValue="TypeD" tokenLine="1" tokenColumn="39">
<IDENTIFIER tokenValue="TypeD" tokenLine="1" tokenColumn="39"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="44"/>
<innerTemplateArgumentList tokenValue="D" tokenLine="1" tokenColumn="45">
<innerTemplateArgument tokenValue="D" tokenLine="1" tokenColumn="45">
<typeName tokenValue="D" tokenLine="1" tokenColumn="45">
<className tokenValue="D" tokenLine="1" tokenColumn="45">
<IDENTIFIER tokenValue="D" tokenLine="1" tokenColumn="45"/>
</className>
</typeName>
</innerTemplateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="46"/>
<innerTemplateArgument tokenValue="TypeE" tokenLine="1" tokenColumn="48">
<typeName tokenValue="TypeE" tokenLine="1" tokenColumn="48">
<className tokenValue="TypeE" tokenLine="1" tokenColumn="48">
<simpleTemplateId tokenValue="TypeE" tokenLine="1" tokenColumn="48">
<templateName tokenValue="TypeE" tokenLine="1" tokenColumn="48">
<IDENTIFIER tokenValue="TypeE" tokenLine="1" tokenColumn="48"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="53"/>
<innerTemplateId tokenValue="E" tokenLine="1" tokenColumn="54">
<templateArgument tokenValue="E" tokenLine="1" tokenColumn="54">
<typeName tokenValue="E" tokenLine="1" tokenColumn="54">
<className tokenValue="E" tokenLine="1" tokenColumn="54">
<IDENTIFIER tokenValue="E" tokenLine="1" tokenColumn="54"/>
</className>
</typeName>
</templateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="55"/>
<innerTypeId tokenValue="TypeF" tokenLine="1" tokenColumn="57">
<innerTrailingTypeSpecifier tokenValue="TypeF" tokenLine="1" tokenColumn="57">
<innerSimpleTemplateId tokenValue="TypeF" tokenLine="1" tokenColumn="57">
<templateName tokenValue="TypeF" tokenLine="1" tokenColumn="57">
<IDENTIFIER tokenValue="TypeF" tokenLine="1" tokenColumn="57"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="62"/>
<innerTemplateArgumentList tokenValue="F" tokenLine="1" tokenColumn="63">
<innerTemplateArgument tokenValue="F" tokenLine="1" tokenColumn="63">
<typeName tokenValue="F" tokenLine="1" tokenColumn="63">
<className tokenValue="F" tokenLine="1" tokenColumn="63">
<IDENTIFIER tokenValue="F" tokenLine="1" tokenColumn="63"/>
</className>
</typeName>
</innerTemplateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="64"/>
<innerTemplateArgument tokenValue="TypeG" tokenLine="1" tokenColumn="66">
<typeName tokenValue="TypeG" tokenLine="1" tokenColumn="66">
<className tokenValue="TypeG" tokenLine="1" tokenColumn="66">
<simpleTemplateId tokenValue="TypeG" tokenLine="1" tokenColumn="66">
<templateName tokenValue="TypeG" tokenLine="1" tokenColumn="66">
<IDENTIFIER tokenValue="TypeG" tokenLine="1" tokenColumn="66"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="71"/>
<innerTemplateId tokenValue="G" tokenLine="1" tokenColumn="72">
<templateArgument tokenValue="G" tokenLine="1" tokenColumn="72">
<typeName tokenValue="G" tokenLine="1" tokenColumn="72">
<className tokenValue="G" tokenLine="1" tokenColumn="72">
<IDENTIFIER tokenValue="G" tokenLine="1" tokenColumn="72"/>
</className>
</typeName>
</templateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="73"/>
<innerTypeId tokenValue="TypeH" tokenLine="1" tokenColumn="75">
<innerTrailingTypeSpecifier tokenValue="TypeH" tokenLine="1" tokenColumn="75">
<innerSimpleTemplateId tokenValue="TypeH" tokenLine="1" tokenColumn="75">
<templateName tokenValue="TypeH" tokenLine="1" tokenColumn="75">
<IDENTIFIER tokenValue="TypeH" tokenLine="1" tokenColumn="75"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="80"/>
<innerTemplateArgumentList tokenValue="H" tokenLine="1" tokenColumn="81">
<innerTemplateArgument tokenValue="H" tokenLine="1" tokenColumn="81">
<typeName tokenValue="H" tokenLine="1" tokenColumn="81">
<className tokenValue="H" tokenLine="1" tokenColumn="81">
<IDENTIFIER tokenValue="H" tokenLine="1" tokenColumn="81"/>
</className>
</typeName>
</innerTemplateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="82"/>
<innerTemplateArgument tokenValue="I" tokenLine="1" tokenColumn="84">
<typeName tokenValue="I" tokenLine="1" tokenColumn="84">
<className tokenValue="I" tokenLine="1" tokenColumn="84">
<IDENTIFIER tokenValue="I" tokenLine="1" tokenColumn="84"/>
</className>
</typeName>
</innerTemplateArgument>
</innerTemplateArgumentList>
</innerSimpleTemplateId>
</innerTrailingTypeSpecifier>
</innerTypeId>
</innerTemplateId>
<BW_RSHIFT tokenValue=">>" tokenLine="1" tokenColumn="85"/>
</simpleTemplateId>
</className>
</typeName>
</innerTemplateArgument>
</innerTemplateArgumentList>
</innerSimpleTemplateId>
</innerTrailingTypeSpecifier>
</innerTypeId>
</innerTemplateId>
<BW_RSHIFT tokenValue=">>" tokenLine="1" tokenColumn="87"/>
</simpleTemplateId>
</className>
</typeName>
</innerTemplateArgument>
</innerTemplateArgumentList>
</innerSimpleTemplateId>
</innerTrailingTypeSpecifier>
</innerTypeId>
</innerTemplateId>
<BW_RSHIFT tokenValue=">>" tokenLine="1" tokenColumn="89"/>
</simpleTemplateId>
</className>
</typeName>
</innerTemplateArgument>
</innerTemplateArgumentList>
</innerSimpleTemplateId>
</innerTrailingTypeSpecifier>
</innerTypeId>
</innerTemplateId>
<BW_RSHIFT tokenValue=">>" tokenLine="1" tokenColumn="91"/>
</simpleTemplateId>
</className>
</typeName>
</definingTypeId>
<SEMICOLON tokenValue=";" tokenLine="1" tokenColumn="93"/>
</aliasDeclaration>
</declaration>
<EOF tokenValue="EOF" tokenLine="1" tokenColumn="105"/>
</translationUnit>AST for <translationUnit tokenValue="using" tokenLine="1" tokenColumn="0">
<declaration tokenValue="using" tokenLine="1" tokenColumn="0">
<aliasDeclaration tokenValue="using" tokenLine="1" tokenColumn="0">
<USING tokenValue="using" tokenLine="1" tokenColumn="0"/>
<IDENTIFIER tokenValue="List" tokenLine="1" tokenColumn="6"/>
<ASSIGN tokenValue="=" tokenLine="1" tokenColumn="10"/>
<definingTypeId tokenValue="TypeA" tokenLine="1" tokenColumn="12">
<typeName tokenValue="TypeA" tokenLine="1" tokenColumn="12">
<className tokenValue="TypeA" tokenLine="1" tokenColumn="12">
<simpleTemplateId tokenValue="TypeA" tokenLine="1" tokenColumn="12">
<templateName tokenValue="TypeA" tokenLine="1" tokenColumn="12">
<IDENTIFIER tokenValue="TypeA" tokenLine="1" tokenColumn="12"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="17"/>
<templateArgumentList tokenValue="A" tokenLine="1" tokenColumn="18">
<templateArgument tokenValue="A" tokenLine="1" tokenColumn="18">
<typeName tokenValue="A" tokenLine="1" tokenColumn="18">
<className tokenValue="A" tokenLine="1" tokenColumn="18">
<IDENTIFIER tokenValue="A" tokenLine="1" tokenColumn="18"/>
</className>
</typeName>
</templateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="19"/>
<templateArgument tokenValue="TypeB" tokenLine="1" tokenColumn="21">
<typeName tokenValue="TypeB" tokenLine="1" tokenColumn="21">
<className tokenValue="TypeB" tokenLine="1" tokenColumn="21">
<simpleTemplateId tokenValue="TypeB" tokenLine="1" tokenColumn="21">
<templateName tokenValue="TypeB" tokenLine="1" tokenColumn="21">
<IDENTIFIER tokenValue="TypeB" tokenLine="1" tokenColumn="21"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="26"/>
<templateArgumentList tokenValue="B" tokenLine="1" tokenColumn="27">
<templateArgument tokenValue="B" tokenLine="1" tokenColumn="27">
<typeName tokenValue="B" tokenLine="1" tokenColumn="27">
<className tokenValue="B" tokenLine="1" tokenColumn="27">
<IDENTIFIER tokenValue="B" tokenLine="1" tokenColumn="27"/>
</className>
</typeName>
</templateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="28"/>
<templateArgument tokenValue="TypeC" tokenLine="1" tokenColumn="30">
<typeName tokenValue="TypeC" tokenLine="1" tokenColumn="30">
<className tokenValue="TypeC" tokenLine="1" tokenColumn="30">
<simpleTemplateId tokenValue="TypeC" tokenLine="1" tokenColumn="30">
<templateName tokenValue="TypeC" tokenLine="1" tokenColumn="30">
<IDENTIFIER tokenValue="TypeC" tokenLine="1" tokenColumn="30"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="35"/>
<templateArgumentList tokenValue="C" tokenLine="1" tokenColumn="36">
<templateArgument tokenValue="C" tokenLine="1" tokenColumn="36">
<typeName tokenValue="C" tokenLine="1" tokenColumn="36">
<className tokenValue="C" tokenLine="1" tokenColumn="36">
<IDENTIFIER tokenValue="C" tokenLine="1" tokenColumn="36"/>
</className>
</typeName>
</templateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="37"/>
<templateArgument tokenValue="TypeD" tokenLine="1" tokenColumn="39">
<typeName tokenValue="TypeD" tokenLine="1" tokenColumn="39">
<className tokenValue="TypeD" tokenLine="1" tokenColumn="39">
<simpleTemplateId tokenValue="TypeD" tokenLine="1" tokenColumn="39">
<templateName tokenValue="TypeD" tokenLine="1" tokenColumn="39">
<IDENTIFIER tokenValue="TypeD" tokenLine="1" tokenColumn="39"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="44"/>
<templateArgumentList tokenValue="D" tokenLine="1" tokenColumn="45">
<templateArgument tokenValue="D" tokenLine="1" tokenColumn="45">
<typeName tokenValue="D" tokenLine="1" tokenColumn="45">
<className tokenValue="D" tokenLine="1" tokenColumn="45">
<IDENTIFIER tokenValue="D" tokenLine="1" tokenColumn="45"/>
</className>
</typeName>
</templateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="46"/>
<templateArgument tokenValue="TypeE" tokenLine="1" tokenColumn="48">
<typeName tokenValue="TypeE" tokenLine="1" tokenColumn="48">
<className tokenValue="TypeE" tokenLine="1" tokenColumn="48">
<simpleTemplateId tokenValue="TypeE" tokenLine="1" tokenColumn="48">
<templateName tokenValue="TypeE" tokenLine="1" tokenColumn="48">
<IDENTIFIER tokenValue="TypeE" tokenLine="1" tokenColumn="48"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="53"/>
<templateArgumentList tokenValue="E" tokenLine="1" tokenColumn="54">
<templateArgument tokenValue="E" tokenLine="1" tokenColumn="54">
<typeName tokenValue="E" tokenLine="1" tokenColumn="54">
<className tokenValue="E" tokenLine="1" tokenColumn="54">
<IDENTIFIER tokenValue="E" tokenLine="1" tokenColumn="54"/>
</className>
</typeName>
</templateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="55"/>
<templateArgument tokenValue="TypeF" tokenLine="1" tokenColumn="57">
<typeName tokenValue="TypeF" tokenLine="1" tokenColumn="57">
<className tokenValue="TypeF" tokenLine="1" tokenColumn="57">
<simpleTemplateId tokenValue="TypeF" tokenLine="1" tokenColumn="57">
<templateName tokenValue="TypeF" tokenLine="1" tokenColumn="57">
<IDENTIFIER tokenValue="TypeF" tokenLine="1" tokenColumn="57"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="62"/>
<templateArgumentList tokenValue="F" tokenLine="1" tokenColumn="63">
<templateArgument tokenValue="F" tokenLine="1" tokenColumn="63">
<typeName tokenValue="F" tokenLine="1" tokenColumn="63">
<className tokenValue="F" tokenLine="1" tokenColumn="63">
<IDENTIFIER tokenValue="F" tokenLine="1" tokenColumn="63"/>
</className>
</typeName>
</templateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="64"/>
<templateArgument tokenValue="TypeG" tokenLine="1" tokenColumn="66">
<typeName tokenValue="TypeG" tokenLine="1" tokenColumn="66">
<className tokenValue="TypeG" tokenLine="1" tokenColumn="66">
<simpleTemplateId tokenValue="TypeG" tokenLine="1" tokenColumn="66">
<templateName tokenValue="TypeG" tokenLine="1" tokenColumn="66">
<IDENTIFIER tokenValue="TypeG" tokenLine="1" tokenColumn="66"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="71"/>
<templateArgumentList tokenValue="G" tokenLine="1" tokenColumn="72">
<templateArgument tokenValue="G" tokenLine="1" tokenColumn="72">
<typeName tokenValue="G" tokenLine="1" tokenColumn="72">
<className tokenValue="G" tokenLine="1" tokenColumn="72">
<IDENTIFIER tokenValue="G" tokenLine="1" tokenColumn="72"/>
</className>
</typeName>
</templateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="73"/>
<templateArgument tokenValue="TypeH" tokenLine="1" tokenColumn="75">
<typeName tokenValue="TypeH" tokenLine="1" tokenColumn="75">
<className tokenValue="TypeH" tokenLine="1" tokenColumn="75">
<simpleTemplateId tokenValue="TypeH" tokenLine="1" tokenColumn="75">
<templateName tokenValue="TypeH" tokenLine="1" tokenColumn="75">
<IDENTIFIER tokenValue="TypeH" tokenLine="1" tokenColumn="75"/>
</templateName>
<LT tokenValue="<" tokenLine="1" tokenColumn="80"/>
<templateArgumentList tokenValue="H" tokenLine="1" tokenColumn="81">
<templateArgument tokenValue="H" tokenLine="1" tokenColumn="81">
<typeName tokenValue="H" tokenLine="1" tokenColumn="81">
<className tokenValue="H" tokenLine="1" tokenColumn="81">
<IDENTIFIER tokenValue="H" tokenLine="1" tokenColumn="81"/>
</className>
</typeName>
</templateArgument>
<COMMA tokenValue="," tokenLine="1" tokenColumn="82"/>
<templateArgument tokenValue="I" tokenLine="1" tokenColumn="84">
<typeName tokenValue="I" tokenLine="1" tokenColumn="84">
<className tokenValue="I" tokenLine="1" tokenColumn="84">
<IDENTIFIER tokenValue="I" tokenLine="1" tokenColumn="84"/>
</className>
</typeName>
</templateArgument>
</templateArgumentList>
<GT tokenValue=">" tokenLine="1" tokenColumn="85"/>
</simpleTemplateId>
</className>
</typeName>
</templateArgument>
</templateArgumentList>
<GT tokenValue=">" tokenLine="1" tokenColumn="87"/>
</simpleTemplateId>
</className>
</typeName>
</templateArgument>
</templateArgumentList>
<GT tokenValue=">" tokenLine="1" tokenColumn="89"/>
</simpleTemplateId>
</className>
</typeName>
</templateArgument>
</templateArgumentList>
<GT tokenValue=">" tokenLine="1" tokenColumn="91"/>
</simpleTemplateId>
</className>
</typeName>
</templateArgument>
</templateArgumentList>
<GT tokenValue=">" tokenLine="1" tokenColumn="93"/>
</simpleTemplateId>
</className>
</typeName>
</templateArgument>
</templateArgumentList>
<GT tokenValue=">" tokenLine="1" tokenColumn="95"/>
</simpleTemplateId>
</className>
</typeName>
</templateArgument>
</templateArgumentList>
<GT tokenValue=">" tokenLine="1" tokenColumn="97"/>
</simpleTemplateId>
</className>
</typeName>
</templateArgument>
</templateArgumentList>
<GT tokenValue=">" tokenLine="1" tokenColumn="99"/>
</simpleTemplateId>
</className>
</typeName>
</definingTypeId>
<SEMICOLON tokenValue=";" tokenLine="1" tokenColumn="100"/>
</aliasDeclaration>
</declaration>
<EOF tokenValue="EOF" tokenLine="1" tokenColumn="112"/>
</translationUnit> |
|
@guwirth is there any bits of test written that you give a chunk of C++ code and try to get the parsing results? |
|
@ericlemes it depends what you mean? You can create and tests an AST from a string or read a .cpp file. You can use the SSLR toolkit to insert code and see the resulting AST (also as XML). But the problem with the issue above (the nested templates) is not a parsing error. The problem is that the parser is slow. Root cause seems to be the parser lookahead related with |
|
Hi @guwirth, I meant a repro of the issue. I actually did one, but unfortunately I need to read a physical file. It seems like we can't write unit tests for this, but integration tests are possible. Humm. Thanks for the insight related to the grammar. It seems that to understand the issue better I'll have to debug sslr code as at least for me it hangs on the parsing time. I'm putting some effort on this, but it seems like not a simple issue. |
@ericlemes comming back to your question. In https://github.com/SonarOpenCommunity/sonar-cxx/tree/master/cxx-squid/src/test/resources/parser is sample C++ code to verify the functionality/compatibility of the parser. The test https://github.com/SonarOpenCommunity/sonar-cxx/blob/master/cxx-squid/src/test/java/org/sonar/cxx/parser/CxxParserTest.java#L55 is parsing them. You can do it in the same way if you like to write a test with an external file. In the folder https://github.com/SonarOpenCommunity/sonar-cxx/tree/master/cxx-squid/src/test/java/org/sonar/cxx/parser you find also other parser tests, e.g. https://github.com/SonarOpenCommunity/sonar-cxx/blob/master/cxx-squid/src/test/java/org/sonar/cxx/parser/TemplatesTest.java. These are using strings to test the parser. I would prefer this way. Hope this helps. |
|
This is being one of the most interesting issues I've seen in a long time. The implementation of sslr is really fun, yet very complex. I think implementing a profiler on SSLR would be really useful. Something like adding for each AST Node the number of instructions processed to generate that node. So, you are right @guwirth. The problem is not on the templates at all. The problem is that the shift expression is really greedy. And when the parser finds a >> it takes a long time to evaluate the other rules until it matches the template ones. I don't know exactly how to fix (yet), but seems like working on the order of rules or some sort of stop condition for the greedy >> may fix the issue. It is really a pain to debug this. |
|
Hi @ericlemes, older versions of https://github.com/SonarSource/sslr had better possibilities to debug the code, but they removed it. If you are looking to the code: using List= TypeA<A, TypeB<B, TypeC<C, TypeD<D, TypeE<E, TypeF<F, TypeG<G, TypeH<H, I>>>>>>>>;The parser has to parse something like the code below first, to detect at the end the >'s: void f() {
TypeA<A, TypeB<B, TypeC<C, TypeD<D, TypeE<E, TypeF<F, TypeG<G, TypeH<H;
}Making this sample for you I found out that also this sample is horrible slow. So the issue seems to be already here? Maybe relationalExpression?
I only can recommend to play around with samples in the SSLR toolkit (like code above) to narrow down the issue. And then you have to play with the rules. I'm not sure if debugging/profiling the SSLR code makes sense? What would be helpful is a possibility to see the rules the parser consumes after reading a token (and also which rules he had to reject). That's really a hard nut... Regards, |
|
@guwirth, yep. Definitely a hard one. It is horrible because it seems to be something that is not a parser issue or a poor implementation issue, but the way the grammar is currently implemented. What I found out is that inner templates are the issue (at least with your example or another similar example I created which is just a function with 4 parameters that are all with templates inside templates). When you have templates inside templates without spaces, the parser matches a shift operator before the templates (>>). This makes the parser to go down the whole tree (which is a hell load of instructions based on the C++ grammar) until it doesn't find a match and then move to the template part of the grammar. I got to that conclusion just commenting out in our grammar the shiftexpression and parsing the same bit of code (using sslr). In your AST xml, you can find some BW_RSHIFT. A potential fix would be to make sure the parser rules out templates on the shift expression before going down the whole tree. My knowledge is pretty crap at building the grammar, so I didn't find anything good yet. I'm working on that on my "free" time, but it is being slow and painful. I'm posting my progress here in case someone with more experience with the grammar wants to step in. |
Grammar is more or less 1:1 C++, you find it here https://github.com/SonarOpenCommunity/sonar-cxx/tree/master/cxx-squid/dox. There are some compiler specific extensions and the extension to handle Do you have an idea why this is also slow? void f() {
TypeA<A, TypeB<B, TypeC<C, TypeD<D, TypeE<E, TypeF<F, TypeG<G, TypeH<H;
} |
|
@guwirth, this example for me parsed in < 1s. |
|
This one is slow on my PC: void f() {
TypeA<A, TypeB<B, TypeC<C, TypeD<D, TypeE<E, TypeF<F, TypeG<G, TypeH<H;
} |
Previously, nested templates with >> at the end were handled via a grammar extension (N1757/Approach 3). For deeper nested templates this led to many look-ahead symbols and exponentially increasing execution time. The treatment has now been moved to the Lexer: Solving the problem amounts to decreeing that under some circumstances a >> token is treated as two right angle brackets instead of a right shift operator (N1757/Approach 1). - add 'Right Angle Brackets N1757' document - close SonarOpenCommunity#1685 Some samples which are much faster now: ```C++ using List= TypeA<A, TypeB<B, TypeC<C, TypeD<D, TypeE<E, TypeF<F, TypeG<G, TypeH<H, I>>>>>>>>; UARTTask<BL6523GXController<UART<UART_NUM_2, 4800, 8, 'N', 1>>> uartTask; void f1() { TypeA<A, TypeB<B, TypeC<C, TypeD<D; } void f2() { TypeA<A, TypeB<B, TypeC<C, TypeD<D>>>> t; } ```
Parsing of nested templates is slow if no blanks are between closing >. In extreme case user are thinking plugin is hanging.
Known workaround: add blanks between closing >.
Some real world samples:
The text was updated successfully, but these errors were encountered: