Skip to content

Commit

Permalink
Add section on pyarrow support in Batched DoFns
Browse files Browse the repository at this point in the history
  • Loading branch information
TheNeuralBit committed Nov 3, 2022
1 parent 05c89ed commit 01524c1
Showing 1 changed file with 10 additions and 0 deletions.
10 changes: 10 additions & 0 deletions website/www/site/content/en/documentation/programming-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -7968,6 +7968,16 @@ batch typehint – but Beam supports typehints from other libraries as well.
| `Any` | |
| [Beam Schema Types](#schemas) | `pd.DataFrame` |

#### [pyarrow](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/typehints/arrow_type_compatibility.py)

| Element Typehint | Batch Typehint |
| ---------------- | -------------- |
| Numeric types (`int`, `np.int32`, `bool`, ...) | `pd.Series` |
| `Any` | |
| `List` | |
| `Mapping` | |
| [Beam Schema Types](#schemas) | `pa.Table` |

#### Other types?
If there are other batch types you would like to use with Batched DoFns, please
[file an issue](https://github.com/apache/beam/issues/new/choose).
Expand Down

0 comments on commit 01524c1

Please sign in to comment.