group_by() function

[springcomp]

group_by function

JEP	18
Author	Maxime Labelle
Created	05-August-2022
SemVer	MINOR
Status	Draft
[Discussion]

Abstract

This JEP introduces a new group_by() function.

Specification

group_by

object group_by(array[object] $elements, expression->expression->(boolean|number|string) $expr)

Given an array of objects $elements, group_by() returns an object whose keys are the unique set of values that result from the evaluation of $expr against each element, converted to a string ; and whose respective values are the objects from the original array matching the group criteria.

Objects that do not match the group criteria are grouped under a key named "null".

Examples

Given the following input JSON document:

{
  "array": [
    { "a": 1, "b": true },
    { "a": 1, "b": false },
    { "a": 2 },
    { "a": null, "b": false }
  ]
}

Here are several examples:

Expression	Result
group_by(array, &a)	{ "1": [{"a": 1, "b": true}, {"a": 1, "b": false}], "2": [{"a": 2}], "null": [{"a": null, "b": false}] }
group_by(array, &b)	{ "true": [{"a": 1, "b": true}], "false": [{"a": 1, "b": false}, {"a": null, "b": false}], "null": [{"a": 2}] }
group_by(array, &!!b)	{ "true": [{"a": 1, "b": true}], "false": [{"a": 1, "b": false}, {"a": null, "b": false}, {"a": 2}] }

Alternatives considered

Concerns are raised against introducing too many functions into JMESPath where clever syntax and existing functions would be possible instead.

History

[springcomp]

@glenveegee I took the liberty to publish an initial draft of a JEP based on your suggestion.

Please feel free to provide feedback.

The main difference based on your suggestion is that the result should be a valid JSON object, whose keys are string types.
I also could not make the difference between examples you shared with and without an expression-type argument.

[springcomp]

I’m not sure we should keep the non matching items and consider whether to drop them outright.
Grouping them under a "null" key feels like a kludge. What if the group_by() criteria must be based on a "null" property?

The result of the grouping in the first motivating example would then be:

{
  "1": [{"a": 1, "b": true}, {"a": 1, "b": false}],
  "2": [{"a": 2}]
}

Whereas the result from the second example would be:

{
  "true": [{"a": 1, "b": true}],
  "false": [{"a": 1, "b": false}, {"a": null, "b": false}]
}

[springcomp]

I definitely think that grouping objects that do not match the group criteria should not be done. Those should be discarded.

Otherwise, we need to specify what does not macthing a group criteria mean. Does it mean evaluates to null?
In that case, we may have a hard time distinguishing objects that do not contain a property to others that actually do.

For instance, consider the following input JSON document:

{
  "array": [
    { "a": 1, "b": true },
    { "a": 1, "b": false },
    { "a": 2 },
    { "a": 2, "b": null }
  ]
}

Let’s say we wanted to group on the value of b:

• group_by(@, &b)

How should group_by() process the last two entries of the array, as &b evaluates to null in both cases.
Should it rely on the property actually being present on the object.

Ultimately, I think this is the job of the expression author to convey their own semantics and should be left outside the purview of the group_by() function.

It is always possible to craft an expression to evaluate to the grouping criteria – that determines the group key in the target object.
For instance, the following expression will distinguish between those cases:

• search( group_by(array, &to_string(b)), … ) -> …

This returns:

{
  "true": [{"a": 1, "b": true}],
  "false": [{"a": 1, "b": false}],
  "null": [{"a": 2}, {"a": 2, "b": null}]
}

• search( group_by(array, &(contains(keys(@), 'b') && to_string(b)) || 'not_specified' ), … ) -> …

This returns:

{
  "true": [{"a": 1, "b": true}],
  "false": [{"a": 1, "b": false}],
  "not_specified": [{"a": 2}],
  "null": [{"a": 2, "b": null}]
}

[springcomp]

I think we should only restrict the group_by() function to grouping on string types.
That is, the expression-type should evaluate to a string.

The signature becomes:

object group_by(array[object] $elements, expression->expression->string $expr)

The first and second motivating examples would become (although they would not be relevant anymore in the spec):

• group_by(array, &to_string(a))
• group_by(array, &to_string(b))

[springcomp]

In that case, the JEP would indicate that a group_by() expression always returns a JSON object. If not items matched the group criteria, the result is an empty {} JSON object.

[springcomp]

This alternate proposal is currently being demonstrated here.