Ex. 1: "#1 and #3" as query?

[Sejmou]

Hi,
sorry for asking so many questions, but I just stumbled over a weird query - it was just "#1 and #3". Is this an error in the dataset? What would be a "meaningful" categorization for the result of such a query? I just picked "topic" because the result mentioned 1-minute and 3-minute toothbrushing, so I guess it kinda fits?

Best,
Samo

[sebastian-hofstaetter]

Hi,

Yes, that is a weird query! The question whether it is an error or not is almost philosophical :) Because that is a real user query from the TripClick dataset, and it even is a query that has been asked many times. Maybe it is a quirk of the user interaction: a user wants to search for the first and third term of their previous query? Maybe ... But we don't have any session information, so we don't know what the users asked before. Or maybe it is the name of a method, like the toothbrushing (probably not, but we can't be sure). If we provide a search engine we always have to deal with weird and ambiguous queries and so does the evaluation. In this case, topic or wrong is probably the way to go, and we just have to accept that we will have some noise in the resulting query set - we try to mitigate this by having lots of queries, so that the effect of a few outliers is negligible in the end.

Best,
Sebastian