Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG]: Semicolons break fulltext search #8787

Closed
lithdoh opened this issue Apr 4, 2023 · 22 comments
Closed

[BUG]: Semicolons break fulltext search #8787

lithdoh opened this issue Apr 4, 2023 · 22 comments
Labels
kind/bug Something is broken.

Comments

@lithdoh
Copy link

lithdoh commented Apr 4, 2023 •
edited
Loading

What version of Dgraph are you using?

dgraph/standalone:latest

Schema:

type Stems  {
	id: ID! 
	link: String! 
	image: String! 
	name: String! @search(by:[fulltext]) 
	length: Int @search 
	rise: Int @search 
	clampDiameter: Float @search 
	steererDiameter: String @search(by:[hash]) 
	color: String @search(by:[hash]) 
	material: String! @search(by:[hash]) 
	price: Float! @search 
	weight: Int @search 
	brand: String! @search(by:[hash]) 
}

Have you tried reproducing the issue with the latest release?

Yes

What is the hardware spec (RAM, CPU, OS)?

Irrelevant

What steps will reproduce the bug?

Make any query that uses fulltext search "alloftext" or "anyoftext" that contains one or more semicolons or backslashes.

Expected behavior and actual result.

When I enter any string containing a semicolon or backslash in the fulltext search, I get the error

“no query string supplied in request”

This is a sample GraphQL query that produces the error

query MyQuery {
  queryItems(filter: {name: {alloftext: "text;with;semicolons"}}) {
    id
    name
    price
  }
}

The issue does not occur when the same query is used in the Dgraph Cloud Explorer for some reason.
I’m using the Docker dgraph/standalone:latest image.

Additional information

The full error message was:

{"errors":[{"message":"no query string supplied in request"}],"extensions":{"tracing":{"version":1,"startTime":"2023-04-04T23:41:00.55157713Z","endTime":"2023-04-04T23:41:00.551966171Z","duration":389041}}}
@lithdoh lithdoh added the kind/bug Something is broken. label Apr 4, 2023
@mangalaman93
Copy link
Contributor

mangalaman93 commented Apr 7, 2023 •
edited
Loading

Hi @lithdoh, would it be possible for you to provide the GraphQL schema that you are using? That would help us easily reproduce this issue and hopefully fix it too.

@lithdoh
Copy link
Author

lithdoh commented Apr 7, 2023 via email

@mangalaman93
Copy link
Contributor

Everything seems to work for me. I tried the following:

  1. Run dgraph from the standalone image
docker run --rm -it -p 8080:8080 dgraph/standalone:latest
  1. Setup schema
curl --request POST \
  --url http://localhost:8080/admin \
  --header 'Content-Type: application/json' \
  --data '{"query":"mutation updateGQLSchema($sch: String!) {\n    updateGQLSchema(input: { set: { schema: $sch }}) {\n\t\t\tgqlSchema {\n\t\t\t\tid\n\t\t\t}\n\t\t}\n\t}","operationName":"updateGQLSchema","variables":{"sch":"type Stems  {\n id: ID! \n link: String! \n image: String! \n name: String! @search(by:[fulltext]) \n length: Int @search \n rise: Int @search \n clampDiameter: Float @search \n steererDiameter: String @search(by:[hash]) \n color: String @search(by:[hash]) \n material: String! @search(by:[hash]) \n price: Float! @search \n weight: Int @search \n brand: String! @search(by:[hash]) }"}}'
  1. Run query:
curl --request POST \
  --url http://localhost:8080/graphql \
  --header 'Content-Type: application/json' \
  --data '{"query":"query MyQuery {\n  queryStems(filter: {name: {alloftext: \"text;with;semicolons\"}}) {\n    id\n    name\n    price\n  }\n}\n","operationName":"MyQuery","variables":{}}'

I get the following response:

{
  "data": {
    "queryStems": []
  },
  "extensions": {
    "tracing": {
      "version": 1,
      "startTime": "2023-04-17T00:55:38.603403852Z",
      "endTime": "2023-04-17T00:55:38.604507486Z",
      "duration": 1103634,
      "execution": {
        "resolvers": [
          {
            "path": [
              "queryStems"
            ],
            "parentType": "Query",
            "fieldName": "queryStems",
            "returnType": "[Stems]",
            "startOffset": 107965,
            "duration": 991551,
            "dgraph": [
              {
                "label": "query",
                "startOffset": 161326,
                "duration": 936808
              }
            ]
          }
        ]
      }
    }
  }
}
``

@mangalaman93
Copy link
Contributor

Maybe, if you could share the exact steps that you are following, or try the steps that I mentioned here and see whether these steps work for you. Thank you

@lithdoh
Copy link
Author

lithdoh commented Apr 17, 2023 •
edited
Loading

@mangalaman93 Thanks for your reply.

I followed the steps you mentioned exactly and got the same result that you did:

{
    "data": {
        "queryStems": []
    },
    "extensions": {
        "tracing": {
            "version": 1,
            "startTime": "2023-04-17T02:39:26.318015422Z",
            "endTime": "2023-04-17T02:39:26.329421964Z",
            "duration": 11406542,
            "execution": {
                "resolvers": [
                    {
                        "path": [
                            "queryStems"
                        ],
                        "parentType": "Query",
                        "fieldName": "queryStems",
                        "returnType": "[Stems]",
                        "startOffset": 271250,
                        "duration": 11115042,
                        "dgraph": [
                            {
                                "label": "query",
                                "startOffset": 397875,
                                "duration": 10984501
                            }
                        ]
                    }
                ]
            }
        }
    }
}

But immediately after when I tried a similar query like this:

http://localhost:8080/graphql?query=
{ queryStems(filter: {name: {alloftext: "text;with;semicolons"}}) { id name price } }

I got the error:

{"errors":[{"message":"no query string supplied in request"}],"extensions":{"tracing":{"version":1,"startTime":"2023-04-17T02:34:22.367479503Z","endTime":"2023-04-17T02:34:22.367862962Z","duration":383584}}}

Note: Your example escapes the quotations around the search terms: {name: {alloftext: \"text;with;semicolons\"}}, whereas mine uses {name: {alloftext: "text;with;semicolons"}}. I get the same error either way.

I verified that if I try the same query with no semicolons:

http://localhost:8080/graphql?query=
{ queryStems(filter: {name: {alloftext: "textwithoutsemicolons"}}) { id name price } }

I get the correct result of an empty array:

{
    "data": {
        "queryStems": []
    },
    "extensions": {
        "tracing": {
            "version": 1,
            "startTime": "2023-04-17T02:44:44.76351425Z",
            "endTime": "2023-04-17T02:44:44.770049083Z",
            "duration": 6534834,
            "execution": {
                "resolvers": [
                    {
                        "path": [
                            "queryStems"
                        ],
                        "parentType": "Query",
                        "fieldName": "queryStems",
                        "returnType": "[Stems]",
                        "startOffset": 901459,
                        "duration": 5616250,
                        "dgraph": [
                            {
                                "label": "query",
                                "startOffset": 1353375,
                                "duration": 5161500
                            }
                        ]
                    }
                ]
            }
        }
    }
}

@mangalaman93
Copy link
Contributor

Could you give me exact curl commands that you ran?

@lithdoh
Copy link
Author

lithdoh commented Apr 17, 2023 •
edited
Loading

I just copied and pasted the ones you provided.

  1. Run dgraph from the standalone image
    docker run --rm -it -p 8080:8080 dgraph/standalone:latest

  2. Setup schema

curl --request POST \
  --url http://localhost:8080/admin \
  --header 'Content-Type: application/json' \
  --data '{"query":"mutation updateGQLSchema($sch: String!) {\n    updateGQLSchema(input: { set: { schema: $sch }}) {\n\t\t\tgqlSchema {\n\t\t\t\tid\n\t\t\t}\n\t\t}\n\t}","operationName":"updateGQLSchema","variables":{"sch":"type Stems  {\n id: ID! \n link: String! \n image: String! \n name: String! @search(by:[fulltext]) \n length: Int @search \n rise: Int @search \n clampDiameter: Float @search \n steererDiameter: String @search(by:[hash]) \n color: String @search(by:[hash]) \n material: String! @search(by:[hash]) \n price: Float! @search \n weight: Int @search \n brand: String! @search(by:[hash]) }"}}'

Run query:

curl --request POST \
  --url http://localhost:8080/graphql \
  --header 'Content-Type: application/json' \
  --data '{"query":"query MyQuery {\n  queryStems(filter: {name: {alloftext: \"text;with;semicolons\"}}) {\n    id\n    name\n    price\n  }\n}\n","operationName":"MyQuery","variables":{}}'

Is there some reason why making the query using the endpoint "http://localhost:8080/graphql?query=" would yield different results?

@mangalaman93
Copy link
Contributor

I am thinking that this could have something to do with bash. The shell usually interprets the ? or ``` chars. Could you check that or copy the exact curl command?

@mangalaman93
Copy link
Contributor

If you ran the following on bash, this clearly won't work:

http://localhost:8080/graphql?query=
`{ queryStems(filter: {name: {alloftext: "text;with;semicolons"}}) { id name price } }`

@akon-dey
Copy link

akon-dey commented Apr 18, 2023 via email

@lithdoh
Copy link
Author

lithdoh commented Apr 19, 2023 via email

@mangalaman93
Copy link
Contributor

yeah, I now realize that you are doing it from the browser. The issue is related to this golang/go#25192.

@mangalaman93
Copy link
Contributor

I am not sure how it would work in the browser because the query part of the URL (everything after the ?) is not allowed to have a ;. I am closing this issue for now, if there is anything else, please feel free to re-open it. Thanks for the issue and bringing this to our attention.

@lithdoh
Copy link
Author

lithdoh commented Apr 19, 2023 •
edited
Loading

But isn't it a common thing to use an HTTP client with a request URL programmatically and not in the browser? Like an http get request with a URL like we are talking about?

@mangalaman93
Copy link
Contributor

Could you help us understand your use case a bit more? Why do you want to use semicolons in the function arguments?

@lithdoh
Copy link
Author

lithdoh commented Apr 19, 2023 •
edited
Loading

It's because I have a search input that filters by Stem names and if a semicolon or back slash "\" is entered I get the error. I don't want to have to implement a filter on the input to remove these characters. I don't feel that it should throw an error if someone tries to filter Stems with any string containing those characters.

@mangalaman93
Copy link
Contributor

Will it be possible for you to share the javascript code that does this? I am thinking it should be possible to send the query parameter in the body of the request instead of the URL like I do in curl. That should avoid the problem that you are seeing.

@lithdoh
Copy link
Author

lithdoh commented Apr 19, 2023 •
edited
Loading

I'm using Angular, here is the code:

export class ExampleHttpDatabase {
  constructor(private http: HttpClient) {}

  baseURL = 'http://localhost:8080/graphql?query=';
  
  getStems(search: string): Observable<Stems[]> {
    const requestURL = this.baseURL + `{ 
      queryStems(filter: {name: ${search ? '{alloftext: \"' + search + '\"}' : '{}'}})
      { link clampDiameter brand color image length material name price rise steererDiameter weight } }`;
    return this.http.get<Stems[]>(requestURL);
  }
}

@mangalaman93
Copy link
Contributor

We found more documentation from the Go stdlib docs https://go.dev/doc/go1.17#semicolons

@mangalaman93
Copy link
Contributor

One thing that could help here is to URL encode the parameters. Though, I think what you are doing could lead to GraphQL injection and instead you should send the query and the variable in the body. I am not familiar with angular but if you can't figure it out, I can look into it. There should be a way to send the query and the variable in the request body. You can look at the reference here https://dgraph.io/docs/graphql/graphql-clients/endpoint/

@mangalaman93
Copy link
Contributor

The URL after encoding parameters look like this http://localhost:8080/graphql?query={%20queryStems(filter:%20{name:{alloftext:%20%22text%3Bwith%3Bsemicolons%22}})%20{%20id%20name%20price%20}%20} and it works for me.

Though, we discourage the use of the GET request because a URL cannot be longer than 2048 chars as well this could lead to GraphQL injection. Instead, one should use the POST request to hit the /graphql endpoint as explained here https://dgraph.io/docs/graphql/graphql-clients/endpoint/graphql-request/

@lithdoh
Copy link
Author

lithdoh commented Apr 20, 2023

Wow thanks! That is very helpful. I will research that now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug Something is broken.
Milestone
No milestone
Development

No branches or pull requests
3 participants
@akon-dey @mangalaman93 @lithdoh