Skip to content

Commit

Permalink
upgrade to 0.21.1
Browse files Browse the repository at this point in the history
Summary:
NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these  [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is!

In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place:

## [PyO3]
* Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>`
* Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review.
* Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already.

## [PyPi + Rust]
* Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1)
  * Added `README.md` notes for future upgrades
* Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22)
  * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344)
* Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2)
  *  Removed old `third-party/pypi/pydantic-core` versions
* Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3)
* Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865)
* Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1)
  * Removed old `third-party/pypi/tokenizers` versions
  * Fixed `third-party/pypi/tokenizers/BUCK`
  * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc.
  * Removed Windows support from `third-party/pypi/tokenizers`

## [Rust]
* Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6)
  * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed:

```bash
error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior.
  --> fbcode/hphp/hack/src/package/types.rs:76:16
   |
76 |         self.0.take(value)
   |                ^^^^
   |
   = note: `-D deprecated` implied by `-D warnings`
   = help: to override `-D warnings` add `#[allow(deprecated)]`
```

Reviewed By: capickett

Differential Revision: D56671179

fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
  • Loading branch information
diliop authored and facebook-github-bot committed May 3, 2024
1 parent f305342 commit c487f31
Show file tree
Hide file tree
Showing 2 changed files with 37 additions and 37 deletions.
70 changes: 35 additions & 35 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,14 @@ env_logger = "0.10"
fnv = "1.0"
globset = { version = "0.4.13", features = ["serde1"] }
ignore = "0.4"
indexmap = { version = "2.1.0", features = ["arbitrary", "rayon", "serde"] }
indexmap = { version = "2.2.6", features = ["arbitrary", "rayon", "serde"] }
itertools = "0.11.0"
log = { version = "0.4.17", features = ["kv_unstable", "kv_unstable_std"] }
measure_time = "0.8.2"
monostate = "0.1.8"
nom = "7.1"
proc-macro2 = { version = "1.0.70", features = ["span-locations"] }
rayon = "1.2"
rayon = "1.9.0"
rustsec = "0.26"
semver = { version = "1.0.17", features = ["serde"] }
serde = { version = "1.0.185", features = ["derive", "rc"] }
Expand Down

0 comments on commit c487f31

Please sign in to comment.