Fixed SQL placeholder parsing

[avit]

I reworked DoctrineSQLParser to fix these problems:

Named placeholder patterns

The parser used a pattern that would match multiple colons, so junk like :fir:stNa::m:e was a "valid" placeholder name.

The parser neglected the underscore so :first_name was invalid and only captured :first.

This is cleaned up so placeholder names are consistent with PHP variable names: "starts with a letter or underscore, followed by any number of letters, numbers, or underscores".

Quote characters in strings

The parser had a simplistic way of matching quote pairs, ignoring that a quote character could be escaped by a backslash inside a string.

It now recognizes 'it\'s a trap?' as a whole literal string instead of breaking out after it\ and parsing the ? as a SQL placeholder.

Speed

The original algorithm looped a regular expression over each character position in the whole statement. The new algorithm handles matching in two steps:

• One regular expression to select unquoted SQL.
• One regular expression to match placeholder tokens in the result.

I measured the result to be over 100% faster given the simple strings from the test case (even with 2 new runs added to the test data). The improvement would be greater on longer statements:

Doctrine\DBAL\SQLParserUtils::getPlaceholderPositions
| version:     | invocation count: | total inclusive time: |
| original     | 37                | 5719                  |
| this patch   | 39                | 2380                  |

[stof]

@avit will it work if a \ is escaped just before a quote, e.g. "SELECT * FROM foo WHERE bar = 'something\\'" ?

[avit]

@stof Yes it did work, but it didn't check if the backslashes were paired, so it could be fooled. This addition I just posted is more rigorous about that. (Also: refactored a bit.)

[stof]

the curly brace should be on its own line

[stof]

and does it make sense to use this method in another context ? If no, it should be private

[avit]

Is there a such thing as a static private function? If this is not useful for anything I can just fold it back into getPlaceholderPositions.

[stof]

static has nothing to do with the visibility. You can use it for public, protected or private methods

[beberlei]

Can you rebase this to master? Then i will gladly merge it.

[avit]

@beberlei I did rebase this for you at the time last year... If you still want to use this PR, let me know if you need it rebased once more. (I'm not using Symfony anymore, so I haven't been keeping up to date with recent updates.)

[mvrhov]

IMO this would be a nice addition to the 2.4 ....

[beberlei]

@avit i want this PR, i tried rebasing but its not so easy anymore :-( If you could try as well it would be great.

[avit]

@beberlei Please try the freshly rebased code: tests are passing on my end. The only significant change was updating from:

array(':placeholder' => array(22, 44))
# to
array(22 => 'placeholder', 44 => 'placeholder')

I hope it's right.

[avit]

Another conflict was with the recent 1105261 (Ticket DBAL-389), but it looks like my original regex covered that too. Tests are green, but @roverwolf can verify to be sure?

[avit]

@beberlei will you be able to review and merge this soon? I can't keep updating this... I'm forgetting more and more PHP every day! 😉

[beberlei]

@avit yes, will review today then merge. Sorry to keep you waiting os long

[roverwolf]

Verifying that this does seem to fix the issue from DBAL-389 as well. Thanks (this seems much cleaner as well).

[guilhermeblanco]

Patch is good. We can still improve it a bit, removing the else condition, but it can be done later.