Skip to content

GH-32609: [Python] Incorporate zen-xu/pyarrow-stubs for typing #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
paddyroddy wants to merge 222 commits into from
Closed

GH-32609: [Python] Incorporate zen-xu/pyarrow-stubs for typing #1

paddyroddy wants to merge 222 commits into from

Conversation

@paddyroddy
Copy link

@paddyroddy paddyroddy commented Jul 19, 2025

Rationale for this change

Currently, PyArrow is not typed and users have to use the stubs provided by https://github.com/zen-xu/pyarrow-stubs. This is not ideal and requires extra effort by zen-xu who has already expressed desire in donating his library (see zen-xu/pyarrow-stubs#186). Will fix apache#32609.

What changes are included in this PR?

This PR includes:

Are these changes tested?

They will be in a similar mechanism to the workflows NumPy has (.github/workflows/mypy.yml, .github/workflows/mypy_primer.yml, .github/workflows/mypy_primer_comment.yml).

Are there any user-facing changes?

The users will now have types!

zen-xu and others added 30 commits December 21, 2022 14:17
@zen-xu
Initial commit
1059230
@zen-xu
init project
516b35f
@zen-xu
complete most of the annotations
5738e6e
@zen-xu
fix FixedSizeBufferWriter init annotation
49f64d3
@zen-xu
bump 10.0.1.2
e432726
@zen-xu
complete parquet core annotations
1b330f0
@zen-xu
bump 10.0.1.3
eca9a4b
@zen-xu
re-export modules
f5d26a4
@zen-xu
fix: add return type for foreign_buffer
697b875
@zen-xu
fix output_stream and read_message annotations
bf0edc0
@zen-xu
ci: add release job
ba659b3
@zen-xu
pre-commit specify flake8 version to 5.0.4
f6f53f6
@zen-xu
flake8 ignore F821 for private files
fecc6ef
@zen-xu
optimize annotations
3215f5b
@zen-xu
bump 10.0.1.4
73235b2
@zen-xu
if param supports IOBase, it should also support NativeFile
3d5857d
@zen-xu
bump 10.0.1.5
108bbd1
@zen-xu
pre-commit adds mypy lint
4c9e6c9
@zen-xu
bump 10.0.1.6
a182e01
@zen-xu
fix ci name
45909ac
@zen-xu
Remove version restrictions for Python.
d2dce33
@zen-xu
release 10.0.1.7
736a6ea
@zen-xu
update poetry ci
ecfdb8c
@TallJimbo
Fix stubs for Table factory methods
c630066 
The main problem was that these were annotated as instance methods rather than static/class methods, but I've added some detail, too.
@zen-xu
Merge pull request #3 from TallJimbo/patch-1
607d7cf 
Fix stubs for Table factory methods
@zen-xu
update pre-commit
8d5a1e0
@zen-xu
update
13b887d
@zen-xu
Merge pull request #9 from zen-xu/update-pre-commit
6c44fde 
chore: update pre-commit
@zen-xu
fix: make fs.FileSystem.from_uri and hdfs.HadoopFileSystem.from_uri a…
e6c7193 
…s classmethod
@zen-xu
Merge pull request #10 from zen-xu/fix-from-uri
4264ab4 
fix: make fs.FileSystem.from_uri and hdfs.HadoopFileSystem.from_uri as classmethod
zen-xu and others added 17 commits May 14, 2025 03:47
@zen-xu
release 19.4 (apache#235)
2f5b7ac
@zen-xu
lint(pyright): disable reportUnknownMemberType (apache#239)
9f7a6e6
@pre-commit-ci
[pre-commit.ci] pre-commit autoupdate (apache#236)
816d663 
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.11.9 → v0.11.13](astral-sh/ruff-pre-commit@v0.11.9...v0.11.13)
- [github.com/RobertCraigie/pyright-python: v1.1.400 → v1.1.401](RobertCraigie/pyright-python@v1.1.400...v1.1.401)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@zen-xu
feat: support pyarrow 20.0 (apache#240)
b2bd7bd
@pre-commit-ci
[pre-commit.ci] pre-commit autoupdate (apache#241)
8aed7c8 
updates:
- [github.com/RobertCraigie/pyright-python: v1.1.401 → v1.1.402](RobertCraigie/pyright-python@v1.1.401...v1.1.402)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@zen-xu
support docstring (apache#242)
b95cfaf 
* doc: complete tensor doc

* doc: complete table doc

* doc: complete scalar doc

* doc: complete orc doc

* doc: complete memory doc

* doc: complete lib doc

* doc: complete json doc

* doc: complete hdfs doc

* doc: complete gcsfs doc

* doc: complete fs doc

* doc: complete flight doc

* doc: complete dataset doc

* doc: complete dataset parquet doc

* doc: complete dataset parquet encryption doc

* doc: complete cuda doc

* doc: complete csv doc

* doc: complete azurefs doc

* doc: complete core doc

* doc: complete interchange doc

* doc: complete array doc

* doc: complete builder doc

* doc: complete device doc

* doc: complete io doc

* doc: complete ipc doc

* doc: complete types doc

* mark deprecated apis

* doc: complete _compute doc

* doc: complete compute doc

* doc: update compute doc

* lint code
@zen-xu
release 20.0.0.20250618 (apache#243)
42090e0
@tmct
fix: make ParquetFileFormat constructor args optional (apache#244)
d7041d3
@zen-xu
fix: Field.remove_metadata should return Self (apache#246)
8d0ebd8
@pre-commit-ci
[pre-commit.ci] pre-commit autoupdate (apache#245)
dd933eb 
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.11.13 → v0.12.0](astral-sh/ruff-pre-commit@v0.11.13...v0.12.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@zen-xu
release 20.0.0.20250627 (apache#247)
d4fbd25
@zen-xu
fix: chunked_array with type should be specified (apache#250)
b8ecaea
@pre-commit-ci
[pre-commit.ci] pre-commit autoupdate (apache#248)
4f61d86 
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.12.0 → v0.12.3](astral-sh/ruff-pre-commit@v0.12.0...v0.12.3)
- [github.com/RobertCraigie/pyright-python: v1.1.402 → v1.1.403](RobertCraigie/pyright-python@v1.1.402...v1.1.403)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@zen-xu
release 20.0.0.20250715 (apache#251)
6e23b7e
@zen-xu
fix: The type parameter of array should be covariant (apache#253)
1830b4a
@zen-xu
release 20.0.0.20250716 (apache#254)
6bedee7
@paddyroddy
Add py.typed file to signify that the library is typed
3f68c55 
See the relevant PEP https://peps.python.org/pep-0561
@github-actions
Copy link

github-actions bot commented Jul 19, 2025

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

See also:

@paddyroddy paddyroddy changed the title Incorporate zen-xu/pyarrow-stubs as a first effort in typing GH-32609: [Python] Incorporate zen-xu/pyarrow-stubs for typing Jul 19, 2025
@github-actions
Copy link

github-actions bot commented Jul 19, 2025

❌ GitHub issue apache#32609 could not be retrieved.

@paddyroddy paddyroddy mentioned this pull request Jul 19, 2025
Merged
@paddyroddy
Copy link
Author

paddyroddy commented Jul 19, 2025

Closing in favour of #3 as history is polluted

@paddyroddy paddyroddy closed this Jul 19, 2025
@paddyroddy paddyroddy self-assigned this Jul 19, 2025
@paddyroddy paddyroddy deleted the incorporate-arrow-stubs branch July 19, 2025 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rok rok Awaiting requested review from rok rok will be requested when the pull request is marked ready for review rok is a code owner

Assignees

@paddyroddy paddyroddy

Labels

awaiting review Component: Python

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Python] Type checking support [Python] pyarrow missing py.typed marker file