Auto-generate aggregation classes #1918

miguelgrinberg · 2024-09-13T18:40:44Z

This change adds docstrings and types to all aggregation classes.

elasticsearch_dsl/query.py

elasticsearch_dsl/search_base.py

elasticsearch_dsl/utils.py

examples/async/composite_agg.py

miguelgrinberg · 2024-09-27T10:15:58Z

tests/test_aggs.py

@@ -220,7 +220,13 @@ def test_filters_correctly_identifies_the_hash() -> None:


 def test_bucket_sort_agg() -> None:
-    bucket_sort_agg = aggs.BucketSort(sort=[{"total_sales": {"order": "desc"}}], size=3)
+    bucket_sort_agg = aggs.BucketSort(sort=[{"total_sales": {"order": "desc"}}], size=3)  # type: ignore


For several examples in this file I kept the original dict based solution with ignored typing errors, and right below I've added a correctly typed version, so that we make sure we are backwards compatible.

This deserves to be an in-file comment IMO, because it's tempting to remove in a refactoring.

utils/generator.py

miguelgrinberg · 2024-09-27T10:44:55Z

elasticsearch_dsl/aggs.py

+        return FieldBucketData(self, search, data)
+
+
+class RandomSampler(Bucket[_R]):


I'm manually adding the missing RandomSampler for now. I think once it is added to the schema this is going to cause a failure in the type checking, so I'll know I can remove it then.

RandomSampler is in the specification now. I've just backported to 8.15 in case it helps here.

pquentin

Thanks! LGTM.

pquentin · 2024-10-02T12:21:01Z

tests/test_aggs.py

@@ -220,7 +220,13 @@ def test_filters_correctly_identifies_the_hash() -> None:


 def test_bucket_sort_agg() -> None:
-    bucket_sort_agg = aggs.BucketSort(sort=[{"total_sales": {"order": "desc"}}], size=3)
+    bucket_sort_agg = aggs.BucketSort(sort=[{"total_sales": {"order": "desc"}}], size=3)  # type: ignore


This deserves to be an in-file comment IMO, because it's tempting to remove in a refactoring.

pquentin · 2024-10-02T13:02:23Z

elasticsearch_dsl/types.py

-        query: Union[str, float, bool, DefaultType] = DEFAULT,
-        analyzer: Union[str, DefaultType] = DEFAULT,
-        auto_generate_synonyms_phrase_query: Union[bool, DefaultType] = DEFAULT,
-        cutoff_frequency: Union[float, DefaultType] = DEFAULT,
-        fuzziness: Union[str, int, DefaultType] = DEFAULT,
-        fuzzy_rewrite: Union[str, DefaultType] = DEFAULT,
-        fuzzy_transpositions: Union[bool, DefaultType] = DEFAULT,
-        lenient: Union[bool, DefaultType] = DEFAULT,
-        max_expansions: Union[int, DefaultType] = DEFAULT,
-        minimum_should_match: Union[int, str, DefaultType] = DEFAULT,
-        operator: Union[Literal["and", "or"], DefaultType] = DEFAULT,
-        prefix_length: Union[int, DefaultType] = DEFAULT,
-        zero_terms_query: Union[Literal["all", "none"], DefaultType] = DEFAULT,
-        boost: Union[float, DefaultType] = DEFAULT,
-        _name: Union[str, DefaultType] = DEFAULT,
+        query: Union[str, float, bool, DefaultType] = DEFAULT,
+        analyzer: Union[str, DefaultType] = DEFAULT,
+        auto_generate_synonyms_phrase_query: Union[bool, DefaultType] = DEFAULT,
+        cutoff_frequency: Union[float, DefaultType] = DEFAULT,
+        fuzziness: Union[str, int, DefaultType] = DEFAULT,
+        fuzzy_rewrite: Union[str, DefaultType] = DEFAULT,
+        fuzzy_transpositions: Union[bool, DefaultType] = DEFAULT,
+        lenient: Union[bool, DefaultType] = DEFAULT,
+        max_expansions: Union[int, DefaultType] = DEFAULT,
+        minimum_should_match: Union[int, str, DefaultType] = DEFAULT,
+        operator: Union[Literal["and", "or"], DefaultType] = DEFAULT,
+        prefix_length: Union[int, DefaultType] = DEFAULT,
+        zero_terms_query: Union[Literal["all", "none"], DefaultType] = DEFAULT,
+        boost: Union[float, DefaultType] = DEFAULT,
+        _name: Union[str, DefaultType] = DEFAULT,


Go home GitHub, you're drunk?

pquentin · 2024-10-02T13:07:58Z

elasticsearch_dsl/aggs.py

+    """
+    A single bucket aggregation that narrows the set of documents to those
+    that match a query.
+
+    :arg filter: A single bucket aggregation that narrows the set of
+        documents to those that match a query.
+    """


I love how we're keeping essentially the same code but adding documentation!

pquentin · 2024-10-02T13:25:04Z

elasticsearch_dsl/aggs.py

+        return FieldBucketData(self, search, data)
+
+
+class RandomSampler(Bucket[_R]):


RandomSampler is in the specification now. I've just backported to 8.15 in case it helps here.

github-actions · 2024-10-04T15:25:17Z

The backport to 8.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-8.x 8.x
# Navigate to the new working tree
cd .worktrees/backport-8.x
# Create a new branch
git switch --create backport-1918-to-8.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 ec8da5577f5c127a0c8fba9b0bfcfabb3e5eb171
# Push it to GitHub
git push --set-upstream origin backport-1918-to-8.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-8.x

Then, create a pull request where the base branch is 8.x and the compare/head branch is backport-1918-to-8.x.

* auto-generate aggregation classes * feedback (cherry picked from commit ec8da55)

* auto-generate aggregation classes * feedback (cherry picked from commit ec8da55) Co-authored-by: Miguel Grinberg <miguel.grinberg@gmail.com>

* auto-generate aggregation classes * feedback

miguelgrinberg force-pushed the generate-aggregations branch from 2f33a36 to 588f58e Compare September 13, 2024 18:41

miguelgrinberg force-pushed the generate-aggregations branch 5 times, most recently from 0d6d89d to 4212e69 Compare September 27, 2024 09:48

auto-generate aggregation classes

2378640

miguelgrinberg force-pushed the generate-aggregations branch from 4212e69 to 2378640 Compare September 27, 2024 10:01

miguelgrinberg marked this pull request as ready for review September 27, 2024 10:08