Support array aggregate sum function #7242

jayzhan211 · 2023-08-09T09:13:49Z

Which issue does this PR close?

Ref #7213 .
Ref #7214 .

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Note

array_aggregate not support somethings like SELECT list_aggregate([2, 4, 8, 42], 'string_agg', '|'), so I don't close Implement array_aggregate function #7213 .
array_sum is not included in this PR

jayzhan211 · 2023-08-10T01:40:30Z

datafusion/core/tests/sqllogictests/test_files/array.slt

+
+# TODO: Support nulls in array.
+# query error DataFusion error: This feature is not implemented: Arrays with different types are not supported: \{Int64, Null\}
+# select array_sum([1, null, 3, null]);


I would like to fix this in the next PR.

Might be related to #7142

Yes, I agree -- I think if we add a coercion pass that tries to coerce array elements into the same type this will magically start working

jayzhan211 · 2023-08-10T03:34:50Z

@izveigor @alamb Ready for review, thanks 👍

alamb

Thank you @jayzhan211 -- I didn't even know this (array aggregate functions) was a thing!

I think we should consider using the existing aggregate / accumulators rather than reimplementing aggregates for arrays and left some detailed suggestions. Let me know what you think

alamb · 2023-08-10T16:48:12Z

docs/source/user-guide/sql/scalar_functions.md

@@ -1495,6 +1496,26 @@ from_unixtime(expression)
 - [make_list](#make_list)
 - [trim_array](#trim_array)

+### `array_aggregate`
+
+Allows the execution of arbitrary existing aggregate functions on the elements of a list.


Suggested change

Allows the execution of arbitrary existing aggregate functions on the elements of a list.

Allows the execution of arbitrary existing aggregate function `name` on the elements of a list.

alamb · 2023-08-10T16:48:28Z

datafusion/core/tests/sqllogictests/test_files/array.slt

+# array aggregate function
+## array aggregate
+query I
+select array_aggregate([1, 3, 5, 7], 'sum');


wow -- this is pretty neat

alamb · 2023-08-10T16:49:54Z

datafusion/core/tests/sqllogictests/test_files/array.slt

+
+# TODO: Support nulls in array.
+# query error DataFusion error: This feature is not implemented: Arrays with different types are not supported: \{Int64, Null\}
+# select array_sum([1, null, 3, null]);


Yes, I agree -- I think if we add a coercion pass that tries to coerce array elements into the same type this will magically start working

alamb · 2023-08-10T16:58:36Z

datafusion/physical-expr/src/array_expressions.rs

+    let func_name = args[1].as_string::<i32>().value(0);
+    let args = &args[0..1];
+    match func_name {
+        "sum" => array_sum(args),


If you implement this in terms of an AggregateExpr and then Accumulator you could use the existing aggregate implementations. This would have several benefits:

We would avoid code duplication and get all existing aggregates "for free"

As new aggregates were added they would also be usable as an array aggregate

They would work for non primitive types (e.g. DecimalArray, strings, etc)

The basic idea would be to look up the aggregate expr during the analysis / parsing phase, create a physical version, and then use it here to instantiate the accumulator

Maybe you could prototype this approach by simply hard coding the mapping sum --> [AggregateFunction::sum] here and using that to create an Accumulator 🤔

Agree with @alamb. Only I want to say that array_aggregate function must only accept aggregate functions with one argument.

jayzhan211 · 2023-08-11T13:36:31Z

Thank you @jayzhan211 -- I didn't even know this (array aggregate functions) was a thing!

I think we should consider using the existing aggregate / accumulators rather than reimplementing aggregates for arrays and left some detailed suggestions. Let me know what you think

I also prefer to reuse existing functions, this might not be trivial but I will try to figure it out.

alamb · 2023-08-12T11:52:53Z

I also prefer to reuse existing functions, this might not be trivial but I will try to figure it out.

I can probably find time to help figure out how this might be able to work next week if that would help

jayzhan211 · 2023-08-12T23:34:13Z

One of the problems I met is I would get NullArray from batch (RecordBatch) if I did not set the target partition to 1.
https://github.com/apache/arrow-datafusion/blob/00627785718d9d98998021bf44585f32c33af3ea/datafusion/core/src/physical_plan/aggregates/no_grouping.rs#L112-L114

CoalescePartitionsExec is the one that sent the batch with NullArray. The default partition seems to be 4. I would need to set set datafusion.execution.target_partitions = 1; in array.slt. I'm not yet fully understand why it that, not confident on whether this is the correct fix or workaround.

Exact batch I get with default partition count

loop_batch: RecordBatch { schema: Schema { fields: [Field { name: "column1", data_type: List(Field { name: "item", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }], metadata: {} }, columns: [ListArray
[
PrimitiveArray
[
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
],
PrimitiveArray
[
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
],
PrimitiveArray
[
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
],
PrimitiveArray
[
31,
32,
33,
34,
35,
26,
37,
38,
39,
40,
],
]], row_count: 4 }
loop_batch: RecordBatch { schema: Schema { fields: [Field { name: "SUM(arrays_values_without_nulls.column1)[sum]", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }], metadata: {} }, columns: [PrimitiveArray
[
null,
]], row_count: 1 }
loop_batch: RecordBatch { schema: Schema { fields: [Field { name: "SUM(arrays_values_without_nulls.column1)[sum]", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }], metadata: {} }, columns: [PrimitiveArray
[
null,
]], row_count: 1 }
loop_batch: RecordBatch { schema: Schema { fields: [Field { name: "SUM(arrays_values_without_nulls.column1)[sum]", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }], metadata: {} }, columns: [PrimitiveArray
[
null,
]], row_count: 1 }

Only the first one is expected, the other 3 are null.

The query I was testing with select array_aggregate(column1, 'sum') from arrays_values_without_nulls;

jayzhan211 · 2023-08-13T06:14:59Z

Another big issue, we might need Array for aggregate function Sum so we can process array and return ArrayRef.
I try to process and return ScalarValue, but ScalarValue::List is not a good replacement for ArrayRef. We might need ColumnarValue that can accept either ScalarValue or ArrayRef for fn sum_batch and SumAccumulator.

https://github.com/apache/arrow-datafusion/blob/00627785718d9d98998021bf44585f32c33af3ea/datafusion/physical-expr/src/aggregate/sum.rs#L269-L300

Unfortunately, we might need to change the signature in Accumulator from ScalarValue to ColumnarValue. Do you think this signature change is a good decision for accepting the Aggregate function to process Array?

https://github.com/apache/arrow-datafusion/blob/00627785718d9d98998021bf44585f32c33af3ea/datafusion/expr/src/accumulator.rs#L49-L63

jayzhan211 · 2023-08-13T07:54:28Z

It seems that ColumnarValue for Accumulator works! Just need to confirm this is a reasonable change.

An approach without set datafusion.execution.target_partitions = 1; has not been solved.

alamb · 2023-08-13T11:50:50Z

Unfortunately, we might need to change the signature in Accumulator from ScalarValue to ColumnarValue. Do you think this signature change is a good decision for accepting the Aggregate function to process Array?

I wonder if you can call Accumulator::update_batch instead? It is already possible to go from ColumnarValue --> Array via https://docs.rs/datafusion/latest/datafusion/logical_expr/enum.ColumnarValue.html#method.into_array

alamb · 2023-08-13T11:53:03Z

datafusion/core/src/physical_plan/aggregates/mod.rs

+                .map(|accumulator| 
+                {
+                    println!("mode: {:?}", mode);
+                    let res = accumulator.evaluate_v2();


I wonder if you could do

Suggested change

let res = accumulator.evaluate_v2();

let res = accumulator.evaluate()?;

res.to_array()

The problem is that I need to find a way to represent Array (Column support) with ScalarValue. So that I can get the ArrayRef from ScalarValue to_array. ScalarValue::List does not fit at all.

jayzhan211 · 2023-08-13T15:03:07Z

I miss the latest comment, let me try out that.

jayzhan211 · 2023-08-14T00:26:09Z

datafusion/physical-expr/src/aggregate/sum.rs

+                    }
+                }
+            }
+            let arr = ScalarValue::iter_to_array(scalars.into_iter())?;


This is where I construct Array.

so in this case, I was thinking sum would be called on the values of the list, not the list itself 🤔 -- could we call evaluate on each scalar 🤔

jayzhan211 · 2023-08-14T00:32:53Z

datafusion/core/src/physical_plan/aggregates/mod.rs

-                .map(|accumulator| accumulator.evaluate().map(|v| v.to_array()))
-                .collect::<Result<Vec<ArrayRef>>>()
+                .map(|accumulator| {
+                    let column_value = accumulator.evaluate_v2();


@alamb
If we want accumulator.evaluate() here, we might need to introduce something like ScalarValue::Array(Option<Vec<ScalarValue>>), the Vec here is to represent values of rows.

i.e. ScalarValue::Array(vec![1, 2, 3, 4])
-> Int64Array(1,2,3,4), which is a four-row value with (1,2,3,4).

With ScalarValue::List(vec![1,2,3,4])
I would get ListArray(Int64Array(1,2,3,4)), which is theone-row value with list[1,2,3,4].

It seems not a good idea to introduce ScalarValue:Array
https://github.com/apache/arrow-datafusion/blob/2ec0bc106540e4cbebd8b7b7269cbf3fa8ae34e3/datafusion/common/src/scalar.rs#L60-L61

jayzhan211 · 2023-08-15T01:05:21Z

datafusion/physical-expr/src/aggregate/sum.rs

+                std::mem::size_of_val(self) - std::mem::size_of_val(sv) + sv.size()
+            }
+            // TODO: Return Correct value
+            ColumnarValue::Array(ref array) => 0,


Not sure how to get the size of array: ArrayRef

alamb · 2023-08-16T19:05:28Z

I am sorry @jayzhan211 -- I don't think I will have time to help work on this PR for the next few days. I will try to find time this wekeend or next week but I am too busy now with TopK and copy related work to help here too. Sorry I am spread too thin 😢

jayzhan211 · 2023-10-18T11:44:02Z

It is significantly simpler after #7352 :)

alamb · 2023-10-18T20:57:38Z

I have this on my review list, and I hope to start working that list down tomorrow

alamb

Thank you for the contribution @jayzhan211 and I apologize for the time it has taken to do a review.

I am concerned about the changes to sum in this PR -- I can see why you did so (to avoid code duplication) but in this case the performance is so critical I think it would be better to have a special implementation

alamb · 2023-08-16T18:57:13Z

datafusion/physical-expr/src/aggregate/sum.rs

+                    }
+                }
+            }
+            let arr = ScalarValue::iter_to_array(scalars.into_iter())?;


so in this case, I was thinking sum would be called on the values of the list, not the list itself 🤔 -- could we call evaluate on each scalar 🤔

datafusion/common/src/scalar.rs

alamb · 2023-10-23T20:52:50Z

datafusion/physical-expr/src/aggregate/sum.rs

-            let v = self.sum.get_or_insert(T::Native::usize_as(0));
-            *v = v.add_wrapping(x);
+        // Wrap single-row input into multiple-rows input and use the same logic as multiple-rows input
+        let list_values = match as_list_array(&values[0]) {


Can we please not ignore the error here? Each error requires a string allocation and this is the performance critical inner loop

I don't think I ignored the "error" here. I use the "error" of as_list_array to know whether it is List or non-List.

Btw, Each error requires a string allocation, can you elaborate more on why there is string allocation?

alamb · 2023-10-23T20:53:54Z

datafusion/sql/src/expr/function.rs

@@ -59,6 +60,26 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
        if let Ok(fun) = BuiltinScalarFunction::from_str(&name) {
            let args =
                self.function_args_to_expr(function.args, schema, planner_context)?;
+
+            // Translate array_aggregate to aggregate function with array argument.
+            if fun == BuiltinScalarFunction::ArrayAggregate {


If we do the rewrite here, it will only apply to SQL (so array_aggregate will not work if it is constructed via the dataframe API or an an Expr directly)

If I understand correctly, DataFrame use
https://github.com/apache/arrow-datafusion/blob/eee790f695a58a99e880957d50a33c1f075c8edc/datafusion/expr/src/expr_fn.rs#L153-L161

to call aggregate function. If we need to introduce AggregateFunction::ArraySum we just need to use another function that utilizes ArraySum, it will have a different path than SQL, right?

Let's raise another PR to have a common rewriting rule for SQL, Dataframe, and Expr. I would need more time to deep dive into them.

datafusion/physical-expr/src/aggregate/sum.rs

alamb · 2023-10-23T20:57:00Z

datafusion/physical-expr/src/aggregate/sum.rs

@@ -167,7 +172,7 @@ impl PartialEq<dyn Any> for Sum {

 /// This accumulator computes SUM incrementally


I am very worried about changing how the Sum accumulator works, as the Sum accumulator is one of the performance critical aggregators.

While it would result in duplicated code, I think in this case it would make sense to have a separate accumulator implementation for array_sum given this concern

I'm thinking about introducing AggregateFunction::ArraySum for array_aggreate('sum') and creating an accumulator SumAccumulator that is specialized for array cases, instead of introducing ArraySumAccumulator under the existing AggregateFunction::Sum.

We also need to do the same to other functions, e.g. avg, min, max

An alternative approach is to go with SumAccumulator or ArraySumAccumulator based on the data_type aka ListArray.

datafusion/common/src/scalar.rs

alamb · 2023-10-27T20:49:48Z

@jayzhan211 I wonder if you could start looking at the LIst / Array code and find ways to make it simpler before we begin implementing new features --

I feel like the code is at a place where doing anything is challenging to get a timely review (because the code is so complicated). Also, when making changes like #7629 results in several regressions, that is a sign to me that the existing code needs to be simplified / improved before we can add significant new features

jayzhan211 · 2023-10-27T23:31:40Z

@jayzhan211 I wonder if you could start looking at the LIst / Array code and find ways to make it simpler before we begin implementing new features --

I feel like the code is at a place where doing anything is challenging to get a timely review (because the code is so complicated). Also, when making changes like #7629 results in several regressions, that is a sign to me that the existing code needs to be simplified / improved before we can add significant new features

Ok

edmondop · 2023-11-13T02:32:33Z

Following up my discussion on #7214 what's the status of this? Are there blockers that need to be removed / other PR that need to be worked on before this one?

jayzhan211 · 2023-11-13T11:02:34Z

Following up my discussion on #7214 what's the status of this? Are there blockers that need to be removed / other PR that need to be worked on before this one?

I plan to merge #8141 first, then maybe review #7242 (comment).

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>

jayzhan211 · 2023-11-19T08:53:33Z

@jayzhan211 I wonder if you could start looking at the LIst / Array code and find ways to make it simpler before we begin implementing new features --

I feel like the code is at a place where doing anything is challenging to get a timely review (because the code is so complicated). Also, when making changes like #7629 results in several regressions, that is a sign to me that the existing code needs to be simplified / improved before we can add significant new features

@alamb I think ArrayAggregate is so different that cleanup existing array function does not help so much. Maybe we can move on this again? Let me know if there is any issue we should focus on before moving on ArrayAggregate.

alamb · 2023-11-20T14:28:08Z

@jayzhan211 I wonder if you could start looking at the LIst / Array code and find ways to make it simpler before we begin implementing new features --
I feel like the code is at a place where doing anything is challenging to get a timely review (because the code is so complicated). Also, when making changes like #7629 results in several regressions, that is a sign to me that the existing code needs to be simplified / improved before we can add significant new features

@alamb I think ArrayAggregate is so different that cleanup existing array function does not help so much. Maybe we can move on this again? Let me know if there is any issue we should focus on before moving on ArrayAggregate.

My concern is that aggregate functions are some of the most important features in DataFusion as they are widely used and their performance is very important perspective. I am very concerned that any change in how the aggregates operate will cause issues downstream (either functionality or performance) as well as make it harder to maintain.

For this case especially I am worried about adding features for their own sake without anyone who is waiting for it

So one question I have is if anyone is waiting on this feature, and if so perhaps they can help implement / test it.

jayzhan211 · 2023-11-20T15:24:24Z

datafusion/physical-expr/src/aggregate/sum.rs

@@ -101,6 +106,43 @@ impl AggregateExpr for Sum {
    }

    fn create_accumulator(&self) -> Result<Box<dyn Accumulator>> {
+        if let DataType::List(field) = &self.data_type {


@alamb I agree that we should not introduce any downgrade especially for this widely used function.

I think we can either run the comparison that ensure there is no performance downgrade for sum or we just consider the initial approach that we build array_aggreate like other array function.

In current approach, ArraySum is actually very different from Sum. We have another Accumulator for Array version. I think this is the only place that might effect the performance of current sum aggregate, where we need to differentiate Array and non-Array cases.

Actually, the initial goal that we try to done this in Accumulator "To reduce code duplication" is actually no longer true. Unless we have other reason that we need ArraySum Accumulator. Maybe we should move this back to array_expression.rs?

Is compare with the performance of Sum enough to ensure there is no downgrade? Not sure what is functionality effect like.

Should we have ArraySum Accumulator for other reason or feature?

If we implement array aggregate function in array_expression.rs is there any concern for the overall design?

If we implement array aggregate function in array_expression.rs is there any concern for the overall design?

Maybe we could put it in array_aggregates.rs (array_expressions.rs is already quite substantial)

Upon further reflection I agree that it makes sense to keep ArraySum separate from the normal Sum aggregator

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>

alamb · 2024-02-02T21:49:56Z

I am trying to go through old PRs and make sure we don't lose any -- this one has not had much activity and has accumulated conflicts. Marking as draft so it isn't on the review queue. Please feel free to reopen / mark as ready for review if it is

github-actions · 2024-05-04T01:45:32Z

Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.

github-actions bot added logical-expr Logical plan and expressions physical-expr Physical Expressions core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) labels Aug 9, 2023

jayzhan211 force-pushed the array_agg_sum branch from ac0f054 to d91397b Compare August 10, 2023 01:39

jayzhan211 commented Aug 10, 2023

View reviewed changes

jayzhan211 marked this pull request as ready for review August 10, 2023 01:48

alamb reviewed Aug 10, 2023

View reviewed changes

jayzhan211 marked this pull request as draft August 11, 2023 13:35

github-actions bot added the sql SQL Planner label Aug 13, 2023

alamb reviewed Aug 13, 2023

View reviewed changes

jayzhan211 force-pushed the array_agg_sum branch from d1e3c8f to a1bea52 Compare August 13, 2023 15:01

jayzhan211 commented Aug 14, 2023

View reviewed changes

jayzhan211 commented Aug 15, 2023

View reviewed changes

jayzhan211 requested a review from alamb August 15, 2023 01:33

jayzhan211 mentioned this pull request Aug 27, 2023

Make ScalarValue an ArrayRef Wrapper #7353

Open

jayzhan211 force-pushed the array_agg_sum branch from 5da37e4 to 349639b Compare October 15, 2023 07:21

github-actions bot added the optimizer Optimizer rules label Oct 15, 2023

jayzhan211 changed the title ~~Support array aggregate function~~ Support array aggregate sum function Oct 15, 2023

jayzhan211 force-pushed the array_agg_sum branch from 349639b to 86ac3f7 Compare October 17, 2023 00:01

jayzhan211 marked this pull request as ready for review October 18, 2023 11:35

alamb reviewed Oct 23, 2023

View reviewed changes

jayzhan211 force-pushed the array_agg_sum branch from c2c7058 to a874af0 Compare October 25, 2023 00:15

jayzhan211 marked this pull request as draft October 25, 2023 13:23

jayzhan211 marked this pull request as ready for review October 25, 2023 14:45

jayzhan211 requested a review from alamb October 25, 2023 14:45

jayzhan211 marked this pull request as draft October 28, 2023 05:54

jayzhan211 mentioned this pull request Nov 12, 2023

Implement array aggregate functions #7214

Open

20 tasks

squash for rebase

18b657b

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>

jayzhan211 force-pushed the array_agg_sum branch from af7a8c6 to 18b657b Compare November 18, 2023 13:07

jayzhan211 added 2 commits November 18, 2023 21:22

cleanup

d3bd12f

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>

cleanup

24daa14

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>

jayzhan211 marked this pull request as ready for review November 19, 2023 08:49

jayzhan211 commented Nov 20, 2023

View reviewed changes

bench

a7eeee5

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>

github-actions bot added the core Core DataFusion crate label Nov 26, 2023

jayzhan211 mentioned this pull request Nov 27, 2023

ArraySum #8325

Closed

alamb marked this pull request as draft February 2, 2024 21:50

github-actions bot added the Stale PR has not had any activity for some time label May 4, 2024

github-actions bot closed this May 11, 2024

	Allows the execution of arbitrary existing aggregate functions on the elements of a list.
	Allows the execution of arbitrary existing aggregate function `name` on the elements of a list.

-                    let res = accumulator.evaluate_v2();
+                    let res = accumulator.evaluate()?;
+                    res.to_array()

		@@ -167,7 +172,7 @@ impl PartialEq<dyn Any> for Sum {

		/// This accumulator computes SUM incrementally

Support array aggregate sum function #7242

Support array aggregate sum function #7242

Conversation

jayzhan211 commented Aug 9, 2023 • edited Loading

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Note

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jayzhan211 commented Aug 10, 2023

alamb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jayzhan211 commented Aug 11, 2023

alamb commented Aug 12, 2023

jayzhan211 commented Aug 12, 2023 • edited Loading

jayzhan211 commented Aug 13, 2023 • edited Loading

jayzhan211 commented Aug 13, 2023

alamb commented Aug 13, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jayzhan211 commented Aug 13, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jayzhan211 Aug 14, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alamb commented Aug 16, 2023

jayzhan211 commented Oct 18, 2023

alamb commented Oct 18, 2023

alamb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jayzhan211 Oct 24, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jayzhan211 Oct 24, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alamb commented Oct 27, 2023

jayzhan211 commented Oct 27, 2023 • edited Loading

edmondop commented Nov 13, 2023

jayzhan211 commented Nov 13, 2023

jayzhan211 commented Nov 19, 2023 • edited Loading

alamb commented Nov 20, 2023 • edited Loading

jayzhan211 Nov 20, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alamb commented Feb 2, 2024

github-actions bot commented May 4, 2024

jayzhan211 commented Aug 9, 2023 •

edited

Loading

jayzhan211 commented Aug 12, 2023 •

edited

Loading

jayzhan211 commented Aug 13, 2023 •

edited

Loading

jayzhan211 Aug 14, 2023 •

edited

Loading

jayzhan211 Oct 24, 2023 •

edited

Loading

jayzhan211 Oct 24, 2023 •

edited

Loading

jayzhan211 commented Oct 27, 2023 •

edited

Loading

jayzhan211 commented Nov 19, 2023 •

edited

Loading

alamb commented Nov 20, 2023 •

edited

Loading

jayzhan211 Nov 20, 2023 •

edited

Loading