5.4 release planning

[binste]

We have some type hint and doc improvements as well as the removal of numpy and pandas as reqiuired dependencies (#3452). I think we could do a 5.4 release with this so that the Polars devs (cc @MarcoGorelli) could start working on the plotting API if they want to.

I can take care of upgrading Vega-Lite to the latest version (#3479) and cutting the release. Does anyone else have a PR open they first want to get in or any objections/inputs?

Update 07.08.2024: See #3478 (comment) for the current status

[dangotbanned]

@binste

#3455 and #3466 are both ready

#3427 I would love to get in as well, so any input on the discussions between @mattijn and myself would be very welcomed.

I do have another typing-related PR ready to submit as well #3480, but had been holding off until api.py is less busy.
Edit 1: Turns out that didn't have any conflicts

Edit 2:

All now merged

• feat(typing): @deprecated versioning, IDE highlighting #3455
• feat: Reimplement expr as a class that is understood by IDEs #3466
• feat: Improve the syntax for conditions with multiple predicates #3427
• refactor(typing): reduce type ignores in api.py #3480

[jonmmease]

I think #3479 still needs to update the vl-convert version bounds: https://github.com/vega/altair/blob/main/NOTES_FOR_MAINTAINERS.md#updating-vl-convert-version-bound. Unfortunately, Vega-Lite 5.19.0 was released a couple of days ago and I haven't done a vl-convert release with this version yet.

I probably won't have time to push through a vl-convert updated until the weekend. So there are three options:

1. Wait until next week to do the Vega-Altair release
2. Release against Vega-Lite 5.18.1 (which is included in vl-convert 1.5).
3. Someone else can try their hand at making the update in https://github.com/vega/vl-convert.

[binste]

No stress at all from my side! I just thought I'd kick off the discussion and I forgot that vl-convert needs a release first :)

Of course, if someone wants to try the update themselves that's great! Let's see, there are also some other PRs it seems which we could get into this release. And as you wrote, we can always release against VL 5.18.1 which is totally fine.

[jonmmease]

Heh, actually I was confused by my own release notes. Vega-Lite 5.19.0 is already released in vl-convert 1.5, so nothing is blocking there after all!

[dangotbanned]

I haven't started a PR for it yet, but maybe the policy for this?

• Deprecation/breaking changes policy & process #3454

Thinking ahead a bit, but introducing a policy in 5.4.0 that defines some public/private boundaries could open up what changes are possible for 5.5.0.

I think its worth considering since 5.5.0 could be 1-6 months away (guessing based on https://github.com/vega/altair/tags)

[mattijn]

Probably we should add a section on types on the API reference page, since there are a few more that can be considered useful, eg. alt.DataFrameLike

[dangotbanned]

Maybe I'm jumping too quick to conclusions as alt.Undefined is not mentioned at altair-viz.github.io/user_guide/api.html, but it actually should.

https://github.com/vega/altair/blob/main/tools/generate_api_docs.py

There are others like alt.Parameter, really anything that doesn't subclass SchemaBase.

[binste]

I'm fine with using the API references in the docs to define what the public API is. But honestly, if a package exposes something in it's top-level namespace without a _ preifx, as a user, I don't check the API references to see if it's documented. I just use it and expect that a altair<6 in my requirements keeps me safe ;) I could imagine I'm not the only one.

As a compromise, we could flesh out what else should be added (Parameter, Undefined, UndefinedType, DataFrameLike, some more...) but refrain from removing anything from the top-level namespace, even if it was not declared in the API references, until Altair 6? What I was actually referring to in my comment which @dangotbanned quoted above, was that if during this process, we see that we want to remove some things, we could clean-up and define the public API in one go and release Altair 6 later this year.

[dangotbanned]

What I was actually referring to in my comment which @dangotbanned quoted above, was that if during this process, we see that we want to remove some things, we could clean-up and define the public API in one go and release Altair 6 later this year.

Apologies for misunderstanding your position @binste

I'm fine with using the API references in the docs to define what the public API is. But honestly, if a package exposes something in it's top-level namespace without a _ preifx, as a user, I don't check the API references to see if it's documented. I just use it and expect that a altair<6 in my requirements keeps me safe ;) I could imagine I'm not the only one.

Totally agree and this is exactly the scenario that adopting a policy is intended to avoid

As a compromise, we could flesh out what else should be added (Parameter, Undefined, UndefinedType, DataFrameLike, some more...) but refrain from removing anything from the top-level namespace, even if it was not declared in the API references, until Altair 6?

Building on this idea, if those changes were made prior to v6:

• Changes could be eased in with a __getattr__ and then actually removed from the top-level in v6.
• Achieving the benefits of removal (not appearing in autocomplete) while not breaking existing code
  • satisfying the MINOR version definition in SemVer.

To approach this with extra caution, we could do something like:

• v5.4.0 - Versioning policy introduced. We use the v5.4.0 API reference as the initial Public API.
• v5.5.0 - First opportunity to start deprecating top-level. If we move straight to v6.0.0, policy is clear that it breaking and no need for deprecation.
• v6.0.0 - Scheduled removal of any deprecations.

I think this would limit the number of potential surprises

[binste]

No reason to apologize! I didn't know myself what I wanted to say... ;)

[dangotbanned]

@binste I'd like to revisit a draft PR you have

• Restrict top-level autocompletion for VS Code (PyLance) #2927

Since #3431 was merged, its become clearer to me that pylance is including if TYPE CHECKING: ... imports as well.

So this not only increases the number of top level symbols, but since they aren't actually present at runtime - pylance users would run into this issue:

---

Personally, I'm happy to make this a priority if you see this as the regression that I do.

[binste]

That's quite a surprising behaviour. Do you think it's something worth reporting upstream for pylance? Feels like a bug to me. Thanks for noticing! I'd also consider this a regression we want to fix first before releasing.

Specifically for Altair, if you have the cycles to revisit autogenerating __all__, if this solves the problem then I think that would be great! I can't support a lot but I can probably carve out the time to review it if that works for you.

[dangotbanned]

@binste I'm glad you agree, I think I have most of it ready but will move it out of draft when I'm sure

• fix: Restrict static & runtime top-level imports #3482

[joelostblom]

I would be interested in having vega/vega-lite#9374 merged and there being a new Vega-Lite release that we could release Altair 5.4 against. This would make it easier to make progress on #3394.

This is by no means blocking from my side, but checking in with @domoritz @kanitw if either of you have time to make a VL release with vega/vega-lite#9374 soonish or if you prefer to wait until there are more merged changes?

[dangotbanned]

Thanks everyone for all the recent reviews.

I'll be trying to get a follow-up docs PR for when-then-otherwise ready soon, as mentioned in #3485 (comment) and #3427 (comment).

PRs

• docs: Add empty as a explicit condition kwarg #3490
• docs: when-then-otherwise examples #3492

Issues

• Promote the use of alt.when().then().otherwise() over alt.condition() in the User Guide? #3500

[dangotbanned]

• Promote the use of alt.when().then().otherwise() over alt.condition() in the User Guide? #3500

can happen with this release but can happen later.

@binste

Would anyone see a benefit in somebody who isn't me writing this into the user guide?

Not asking of anyone in particular, and I feel like I could get a PR done next week.

However it would be a good test of how useful the docstrings are.
Maybe highlighting things that are missing/too verbose after some "real-world" usage.

Things that make sense or seem helpful to me may not be universal after all

[binste]

Summary of where I think we are at but please correct me if I'm missing anything or anyone is waiting for input:

• Must: Get feat(typing): Adds public altair.typing module #3515 in so that Polars can use Altair 5.4 as a minimum dependency in feat(python!): Use Altair in DataFrame.plot pola-rs/polars#17995
• Optional: We wait a bit to see if support for VL 5.20.1 lands in vl-convert Add support vor VL 5.20 vl-convert#176 so that we can support react. No rush at all, we can also release with VL 5.19 as-is. -> Opened feat: Update Vega-Lite to 5.20.1 #3525

I believe all other open points in this discussion have been addressed. #3500 can happen with this release but can happen later.

[dangotbanned]

• feat: Support a wider range of iterables in SchemaBase.to_dict #3501

Has been ready for review but @joelostblom is still away I believe

I'm happy to wait for the review, but this would be easy to include if anyone else can take a look?

Closes 2 issues

[dangotbanned]

To keep things organized, I'll be keeping this meta-issue updated with any alt.themes-related PRs.

• Improving Themes #3519

[binste]

Thanks everyone! 5.4.0 is out now https://github.com/vega/altair/releases/tag/v5.4.0 🥳 Shoutout to @dangotbanned who shaped this release with very significant contributions and to @MarcoGorelli for getting rid of the pandas and numpy dependencies!

[dangotbanned]

Thanks @binste!

I know there is probably a lot of work to get this done, but is there any plan to continue the release notes style from v5.3.0?

It seemed to be well received and we might already have screenshots/videos to reuse from the PRs

[binste]

Thanks for bringing it up. I really like the screenshots and videos myself but as you said, it can be quite some additional effort. I'd also prefer to do more frequent releases to get new features and bug fixes out sooner so I wouldn't want to make this a requirement for the person doing the release. To quote this comment by Joel:

whoever has time to create some screenshots/animations for each release should feel free to add them to the notes (but not a requirement for the person making the release, although it's good if it happens at the same time I guess in terms of people getting notifications).

I don't plan to further edit the release notes in the next few days. If anyone wants to improve them, that would of course be amazing :)

[joelostblom]

Wow, I count 79 merged PRs! That's the most since 5.0 I believe. Thanks for making the release @binste and everyone for their contributions!

[domoritz]

Congratulations for the release. Exciting improvements and great to have a leaner Altair.