[R] dplyr::compute should convert from grouped arrow_dplyr_query to arrow Table

[asfimport]

It is expected that dplyr::compute() will perform the calculation on the arrow dplyr query and convert it to a Table, but it does not seem to work correctly for grouped arrow dplyr queries and does not result in a Table.

mtcars |> arrow::arrow_table() |> dplyr::group_by(cyl) |> dplyr::compute() |> class()
#> [1] "arrow_dplyr_query"
mtcars |> arrow::arrow_table() |> dplyr::group_by(cyl) |> dplyr::ungroup() |> dplyr::compute() |> class()
#> [1] "Table"        "ArrowTabular" "ArrowObject"  "R6"

as_arrow_table() works fine.

mtcars |> arrow::arrow_table() |> dplyr::group_by(cyl) |> class()
#> [1] "arrow_dplyr_query"
mtcars |> arrow::arrow_table() |> dplyr::group_by(cyl) |> dplyr::compute() |> class()
#> [1] "arrow_dplyr_query"
mtcars |> arrow::arrow_table() |> dplyr::group_by(cyl) |> dplyr::collect(FALSE) |> class()
#> [1] "arrow_dplyr_query"
mtcars |> arrow::arrow_table() |> dplyr::group_by(cyl) |> arrow::as_arrow_table() |> class()
#> [1] "Table"        "ArrowTabular" "ArrowObject"  "R6"

It seems to revert to arrow dplyr query in the following line.

arrow/r/R/dplyr-collect.R

Lines 73 to 75 in 7cfdfbb

	df <- as_adq(df)
	df$group_by_vars <- query$group_by_vars
	df$drop_empty_groups <- query$drop_empty_groups

 

Reporter: SHIMA Tatsuya / @eitsupi
Assignee: SHIMA Tatsuya / @eitsupi

Related issues:

• [R] Groups before conversion to a Table must not be restored after collect() (is related to)

PRs and other links:

• GitHub Pull Request #14160

_{Note: This issue was originally created as ARROW-17738. Please see the migration documentation for further details.}

[asfimport]

Neal Richardson / @nealrichardson:
Could you clarify what is not working?

[asfimport]

SHIMA Tatsuya / @eitsupi:
I have updated the description.
Grouped arrow dplyr queries are not converted to tables by dplyr::compute.

[asfimport]

SHIMA Tatsuya / @eitsupi:
Ah, is this the intended behavior?
I didn't understand why this behavior was intended, I think compute should return a Table here, just as dbplyr and dtplyr do.

[asfimport]

Neal Richardson / @nealrichardson:
They are evaluated and converted to Tables, but then if there are groups, group_by is called on the Table, which results in an arrow_dplyr_query object containing the Table. So, yes, this was intentional. Do you have a use case where this is a problem?

[asfimport]

SHIMA Tatsuya / @eitsupi:
I think it is confusing to users when compute does not result in a Table as intended when the group is left after summarise, etc. is executed.

mtcars |> arrow::arrow_table() |> dplyr::group_by(vs, am) |> dplyr::summarise(wt = mean(wt)) |> dplyr::compute()
#> Table (query)
#> vs: double
#> am: double
#> wt: double
#>
#> * Grouped by vs
#> See $.data for the source Arrow object

[asfimport]

Dewey Dunnington / @paleolimbot:
Issue resolved by pull request 14160
#14160