Query group by parsing regression #3939

tokovach · 2022-05-27T10:59:39Z

Current Behavior

Since RDF4J 4.0, the Query Parser is unable to handle implicit aggregate operations and other variables in the projection. Implicit Group by operations throw the following parser exception:

org.eclipse.rdf4j.query.MalformedQueryException: non-aggregate expression 'MathExpr (/)
   Var (name=cnt)
   Var (name=cnt)
' not allowed in projection when using GROUP BY.

	at org.eclipse.rdf4j.query.parser.sparql.SPARQLParser.buildQueryModel(SPARQLParser.java:220)
	at org.eclipse.rdf4j.query.parser.sparql.SPARQLParser.parseQuery(SPARQLParser.java:178)
	at org.eclipse.rdf4j.query.parser.QueryParserUtil.parseQuery(QueryParserUtil.java:93)
	at org.eclipse.rdf4j.query.parser.QueryParserUtil.parseTupleQuery(QueryParserUtil.java:108)
	at org.eclipse.rdf4j.repository.sail.SailRepositoryConnection.prepareTupleQuery(SailRepositoryConnection.java:288)
	at org.eclipse.rdf4j.repository.sail.SailRepositoryConnection.prepareTupleQuery(SailRepositoryConnection.java:59)
	at org.eclipse.rdf4j.repository.base.AbstractRepositoryConnection.prepareTupleQuery(AbstractRepositoryConnection.java:143)
	at org.eclipse.rdf4j.repository.RepositoryConnection.prepareTupleQuery(RepositoryConnection.java:212)
	at com.ontotext.trree.TestGDBGroupByCorrectness.testImplicit(TestGDBGroupByCorrectness.java:91)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
Caused by: org.eclipse.rdf4j.query.parser.sparql.ast.VisitorException: non-aggregate expression 'MathExpr (/)
   Var (name=cnt)
   Var (name=cnt)
' not allowed in projection when using GROUP BY.
	at org.eclipse.rdf4j.query.parser.sparql.TupleExprBuilder.visit(TupleExprBuilder.java:644)
	at org.eclipse.rdf4j.query.parser.sparql.TupleExprBuilder.visit(TupleExprBuilder.java:234)
	at org.eclipse.rdf4j.query.parser.sparql.ast.ASTSelect.jjtAccept(ASTSelect.java:32)
	at org.eclipse.rdf4j.query.parser.sparql.TupleExprBuilder.visit(TupleExprBuilder.java:397)
	at org.eclipse.rdf4j.query.parser.sparql.TupleExprBuilder.visit(TupleExprBuilder.java:234)
	at org.eclipse.rdf4j.query.parser.sparql.ast.ASTSelectQuery.jjtAccept(ASTSelectQuery.java:24)
	at org.eclipse.rdf4j.query.parser.sparql.TupleExprBuilder.visit(TupleExprBuilder.java:346)
	at org.eclipse.rdf4j.query.parser.sparql.TupleExprBuilder.visit(TupleExprBuilder.java:234)
	at org.eclipse.rdf4j.query.parser.sparql.ast.ASTQueryContainer.jjtAccept(ASTQueryContainer.java:26)
	at org.eclipse.rdf4j.query.parser.sparql.SPARQLParser.buildQueryModel(SPARQLParser.java:218)
	... 37 more

Expected Behavior

The following query should not throw a parsing exception and should be handled by the query engine:

Query -

select ?s (count (?o) as ?cnt) {
?s <u:p> ?o .
} group by ?s

When you parse the query in the following validatоr you can see that it is valid and produces a query algebra model of the type:

  1 (base <http://example/base/>
  2   (project (?s ?cnt)
  3     (extend ((?cnt ?.0))
  4       (group (?s) ((?.0 (count ?o)))
  5         (quadpattern (quad <urn:x-arq:DefaultGraphNode> ?s <u:p> ?o))))))

Steps To Reproduce

Just parse the following tuple query:

select ?s (count (?o) as ?cnt) {
?s <u:p> ?o .
} group by ?s

Version

4.0.1

Are you interested in contributing a solution yourself?

No response

Anything else?

No response

The text was updated successfully, but these errors were encountered:

abrokenjester · 2022-05-29T23:18:23Z

Possible regression caused by fix for GH-3782

hmottestad · 2022-06-02T21:13:18Z

I've tried testing this with the MemoryStore without any success.

I've tested against the main branch and the 4.0.1 release tag and also the 4.0.0 release tag and a bunch of other commits too.


import org.eclipse.rdf4j.model.util.Values;
import org.eclipse.rdf4j.model.vocabulary.RDF;
import org.eclipse.rdf4j.model.vocabulary.RDFS;
import org.eclipse.rdf4j.query.BindingSet;
import org.eclipse.rdf4j.query.TupleQuery;
import org.eclipse.rdf4j.query.TupleQueryResult;
import org.eclipse.rdf4j.repository.sail.SailRepository;
import org.eclipse.rdf4j.repository.sail.SailRepositoryConnection;

public class Temp {

	public static void main(String[] args) {

		SailRepository sailRepository = new SailRepository(new MemoryStore());
		try (SailRepositoryConnection connection = sailRepository.getConnection()) {
			connection.begin();
			connection.add(Values.bnode(), RDF.TYPE, RDFS.RESOURCE);
			connection.add(Values.bnode(), Values.iri("u:p"), RDFS.RESOURCE);
			connection.add(Values.iri("u:p2"), Values.iri("u:p"), RDFS.RESOURCE);
			connection.add(Values.iri("u:p2"), Values.iri("u:p"), Values.bnode());
			connection.commit();
			connection.begin();
			TupleQuery tupleQuery = connection.prepareTupleQuery("select ?s (count (?o) as ?cnt) {\n" +
				"?s <u:p> ?o .\n" +
				"} group by ?s\n");
			try (TupleQueryResult evaluate = tupleQuery.evaluate()) {
				for (BindingSet bindings : evaluate) {
					System.out.println(bindings);
				}
			}
			connection.commit();
		}


	}

}

hmottestad · 2022-06-02T21:19:55Z

I am however able to reproduce it with the following query:

select ?s  (?cnt / ?cnt as ?cnt) {
   ?s <u:p> ?o .
} group by ?s

Exception in thread "main" org.eclipse.rdf4j.query.MalformedQueryException: non-aggregate expression 'MathExpr (/)
   Var (name=cnt)
   Var (name=cnt)
' not allowed in projection when using GROUP BY.
	at org.eclipse.rdf4j.query.parser.sparql.SPARQLParser.buildQueryModel(SPARQLParser.java:220)
	at org.eclipse.rdf4j.query.parser.sparql.SPARQLParser.parseQuery(SPARQLParser.java:178)
	at org.eclipse.rdf4j.query.parser.QueryParserUtil.parseQuery(QueryParserUtil.java:93)
	at org.eclipse.rdf4j.query.parser.QueryParserUtil.parseTupleQuery(QueryParserUtil.java:108)
	at org.eclipse.rdf4j.repository.sail.SailRepositoryConnection.prepareTupleQuery(SailRepositoryConnection.java:288)
	at org.eclipse.rdf4j.repository.sail.SailRepositoryConnection.prepareTupleQuery(SailRepositoryConnection.java:59)
	at org.eclipse.rdf4j.repository.base.AbstractRepositoryConnection.prepareTupleQuery(AbstractRepositoryConnection.java:143)
	at org.eclipse.rdf4j.repository.RepositoryConnection.prepareTupleQuery(RepositoryConnection.java:212)
	at org.eclipse.rdf4j.sail.memory.Temp.main(Temp.java:33)
Caused by: org.eclipse.rdf4j.query.parser.sparql.ast.VisitorException: non-aggregate expression 'MathExpr (/)
   Var (name=cnt)
   Var (name=cnt)
' not allowed in projection when using GROUP BY.
	at org.eclipse.rdf4j.query.parser.sparql.TupleExprBuilder.visit(TupleExprBuilder.java:644)
	at org.eclipse.rdf4j.query.parser.sparql.TupleExprBuilder.visit(TupleExprBuilder.java:234)
	at org.eclipse.rdf4j.query.parser.sparql.ast.ASTSelect.jjtAccept(ASTSelect.java:32)
	at org.eclipse.rdf4j.query.parser.sparql.TupleExprBuilder.visit(TupleExprBuilder.java:397)
	at org.eclipse.rdf4j.query.parser.sparql.TupleExprBuilder.visit(TupleExprBuilder.java:234)
	at org.eclipse.rdf4j.query.parser.sparql.ast.ASTSelectQuery.jjtAccept(ASTSelectQuery.java:24)
	at org.eclipse.rdf4j.query.parser.sparql.TupleExprBuilder.visit(TupleExprBuilder.java:346)
	at org.eclipse.rdf4j.query.parser.sparql.TupleExprBuilder.visit(TupleExprBuilder.java:234)
	at org.eclipse.rdf4j.query.parser.sparql.ast.ASTQueryContainer.jjtAccept(ASTQueryContainer.java:26)
	at org.eclipse.rdf4j.query.parser.sparql.SPARQLParser.buildQueryModel(SPARQLParser.java:218)
	... 8 more

hmottestad · 2022-06-03T08:01:35Z

Wrong query. Found the actual failing query and opened a new issue: #3954

tokovach added the 🐞 bug issue is a bug label May 27, 2022

hmottestad added this to the 4.0.2 milestone Jun 2, 2022

hmottestad self-assigned this Jun 2, 2022

hmottestad added cannot reproduce issue can not be reproduced (close reason) and removed 🐞 bug issue is a bug labels Jun 3, 2022

hmottestad removed this from the 4.0.2 milestone Jun 3, 2022

hmottestad closed this as completed Jun 3, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Query group by parsing regression #3939

Query group by parsing regression #3939

tokovach commented May 27, 2022 •

edited

Loading

abrokenjester commented May 29, 2022

hmottestad commented Jun 2, 2022

hmottestad commented Jun 2, 2022

hmottestad commented Jun 3, 2022

Query group by parsing regression #3939

Query group by parsing regression #3939

Comments

tokovach commented May 27, 2022 • edited Loading

Current Behavior

Expected Behavior

Steps To Reproduce

Version

Are you interested in contributing a solution yourself?

Anything else?

abrokenjester commented May 29, 2022

hmottestad commented Jun 2, 2022

hmottestad commented Jun 2, 2022

hmottestad commented Jun 3, 2022

tokovach commented May 27, 2022 •

edited

Loading