Simplify CI, fix mybinder.org build, bump python (and other) versions

[yuvipanda]

I started looking into this more as it was reported broken on mybinder.org: https://jupyter.zulipchat.com/#narrow/channel/469744-jupyterhub/topic/try.20mybinder.20jupyterleab.20demo.20broken

Debugging the failure on mybinder.org via #147, I discovered:

1. xeus-cling+friends was causing mamba to fail to solve
2. mybinder.org's build memory limits were causing mamba to fail
3. CI was failing here for unrelated reasons

(2) is fixed with jupyterhub/mybinder.org-deploy#3211, and this 'fixes' (1) by removing xeus-cling. I also unpinned the semi-pinned versions in environment.yml. If we want version pinning, we should use a lock file. This bumps up versions of various packages, including python (to 3.12 from 3.9)

This works on mybinder now! https://mybinder.org/v2/gh/yuvipanda/jupyterlab-demo/remove-xtensor?urlpath=lab

Merging this will unbreak this repo on mybinder.

I then looked through CI, and simplified it by:

1. Not taking on the responsibility of setting up environments in our build script. This fundamentally was the major reason for complication, as first CI setup an environment and then the environment tried to do more complex things with environments. Now environment is only set up on CI, and on local machines, it's the user's responsibility to setup environment.
2. Getting rid of pyinvoke and turning things into a simple build.py file. invoke is great for things that are more complex with multiple interdependencies. By removing environment management, what needs to happen got far more simpler - clone some repos, and copy some files. This can be done with straightforward python more easily, which I have done here.
3. Only port the demofiles and talks invoke targets, and ignore everything else. This means the one for updating environments is also removed - as that should instead be done via lockfiles.
4. Add irkernel back to the environment.yml file, as that was required for CI to pass. This does bring back rstudio into the mybinder launch, but that's fine.

Originally reported in https://jupyter.zulipchat.com/#narrow/channel/469744-jupyterhub/topic/try.20mybinder.20jupyterleab.20demo.20broken

[github-actions]

👈 Launch a Binder on branch yuvipanda/jupyterlab-demo/remove-xtensor

[yuvipanda]

I'm going to try to fix CI by vastly simplifying it. It currently does a large number of things (including trying to manage environments, even though it itself needs an environment before it can run).

[krassowski]

Do we want to keep this TODO in the readme or open an issue instead?

[krassowski]

Do we want to pin it to >=4.3.5? Currently it resolves to

If we are giving a demo of jupyterlab, it feels like this one package should probably be pinned as we would not want to receive reports for things already fixed in newer versions?

[yuvipanda]

@krassowski I dug into why it was resolving to this, and it's because that's the base version in repo2docker - last updated in jupyterhub/repo2docker#1369. I agree that we should try to use latest jupyterlab here if possible. If we pin to >=4.3.5, it means that when repo2docker goes to a version above 4.3.5, it'll simply just get pinned to that rather than latest. Do you know if there's a way to specify 'latest' with a conda environment.yml?

[yuvipanda]

@krassowski ok, so I've made jupyterhub/repo2docker#1407 which bumps up all versions of things, including lab to 4.3.5. For this PR, my suggestion is that we either find a way to specify 'latest' or let it be unpinned, as once that repo2docker PR lands >=4.3.5 is a no-op

[krassowski]

I've checked with a colleague working on conda and it seems there is no way to specify latest but if we had an option to customize the CLI invocation we could use --prune to ignore constraints from currently installed versions.

[krassowski]

Thank you @yuvipanda, I think it is fine to merge as-is given the constraints!