Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve quoting of token count query #223

Merged
merged 1 commit into from
Mar 22, 2022
Merged

Conversation

missinglink
Copy link
Member

There appears to be a bug when using the token count query with reserved tokens such as #.
This PR wraps that query in the same way we wrap the other ones to it can accept any characters as input.

npm run cli -- 'san fran #1'

> pelias-placeholder@0.0.0-development cli
> node cmd/cli.js "san fran #1"

san fran #1


/Users/peter/code/pelias/placeholder/lib/Queries.js:151
    return stmt.get(token).cnt;
                ^
SqliteError: fts5: syntax error near "#"
    at /Users/peter/code/pelias/placeholder/lib/Queries.js:151:17
    at Array.map (<anonymous>)

@orangejulius
Copy link
Member

Ah yeah, much better!

Pretty wild the hoops you have to jump through to properly quote, my initial version of this just removed the characters that the query didn't like 😆.

@missinglink
Copy link
Member Author

missinglink commented Mar 22, 2022

Yeah its weird, MATCH expects an FTS query as its right operand, and that query language has its own quoting rules.

On top of all that are the quoting rules when binding query variables, it all ends up a bit convoluted, I remember looking into it a while back and I'm pretty confident that this is correct, despite being a bit weird looking.

@missinglink missinglink merged commit 6fc8535 into master Mar 22, 2022
@missinglink missinglink deleted the token-count-quoting branch March 22, 2022 13:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants