improve docs

[tylerflex]

Some things we could do to make this better:

Landing page

• Toggle between command line and python instructions for installation and configure.
• (possibly?) Toggle between linux / mac and windows instructions
• Add link to colab notebook with the minimal example and API key loaded as environment variable.
• Add nice banners

Aesthetics

• Improve overall theme.
• Make the font better
• Improve general page styling so it's more readable
• Add emojis, generally make the docs look nicer.
• wider columns for titles and code.
• make table of contents display more logical
• Find a dark theme logo that works - I think important because I and I assume others, exclusively work in dark themes. Currently chatting with Qing. (I can modify the existing one easily if approved)
• Cooler intro page and pictures
• Make toctrees nicer

API Documentation Improvements

From @e-g-melo As an user of documentation, I think that it would help a lot if

• The user can find more context when reading though the class definitions
• Including equations when needed
• Schematics to show geometric parameters
• An example on how to use that object in a simulation.
• Link to examples and tutorials
• We need to sort out the way the attributes are propagated.
• Cleaner API viewer
• Solve the hierarchical class method search issue Selective documentation of inherited attributes #1197
• Hide irrelevant private methods on API
• Sort out the material library toctree situation TODO autosummary generate restructure in this case re fundamental hierarchy
• Improve the API tab search and hierarchy
• Make nice illustration of medium property types, ie anisotropic isotropic for the uninitated
• Make nice illustrations of all the sources and geometrically how they are represented
• Standalone examples for most common class methods etc.
• Solve the class page index issue on the right, especially for Simulation that it became quite big.
• Any other way to solve the inheritance method propagation issue apart from the manual autosummary? Maybe there's no alternative and this is it. Completeness by documenting all base classes on absract/ done
• Fix the m2r2 situation. This may require a patch of that repository.
• Backup a main and develop branch of the tidy3d-docs just in case which might get overwritten
• Potentially work on the adjoint docstrings depending on what Tyler wants re his restructure
• Fix all the broken links from the 2.5 structure

Functionality

• how to build locally.
• remove all warnings.
• Build pdf automatically receipe
• Docstring updating script so it's easier to change APIs throughout the multiple versions

New Structure

• Write down poetry instructions
• Document development tidy3d scripts
• Create the ported tidy3d-notebooks. Update already one exists, bring it back to life?

Pre-Integration Checklist

• Make sure all the github action workflows pass after the upgrade
• Verify tests
• Think about how to port the latest docs repo changes to be integrated? Mantain history or how do we do this?
• How do we deal with versioning here?
• Sort out the final home of the notebooks repo, the mirror repo, and the main source code repo
• Add docstring image example Emerson suggested and sent

Implementation Tasks

• Upgrade the sphinx version. I believe a lot of what we want out of documentation can be solved in a first pass through a theme dependency upgrade (because I’ve used the same theme before, haha - so I know!) There's some packages conflicts I'm currently dealing with.
• Guarantee important equivalence between the new and old documentation conf.py
• Demonstrate reproducible working environment using poetry
• Remove old unnecessary dependencies I feel this should be done by someone deeply involved in the project.
• Sort out this table of contents situation
• Write documentation verification environment script on tidy3d cli
• Write documentation build script on tidy3d cli
• Write automatic configuration of notebook submodule on tidy3d cli
• Do we like this live code binder integration of our current theme. I'm kind of keen
• Setup the colab notebooks and live notebook links on the selected notebooks we want.
• Install a few pre-commits to guarantee docs + code quality.
• Commit between the two repositories
• Restructure the docs project within the project
• Remove bugs from the installation scripts
• Verify reproducibility of the envrionment installation scripts
• Thoroughly verify that the documentation contains all the same information as before.
• Update README.md for notebooks
• Update README.md for source
• Reorganise the image img paths from the restructure
• Sort out the sitemap building issue. Why is it failing? Last equivalence point I think.
• Do we need to set up a version branch deployment action for RTD?
• Have I screwed up the draw_package and generate_doc in docs?
• FIX all the API links in the notebooks
• Improve the way that images and equations get rendered into our API theme.
• Fix dark pygments viewing on dark mode in web outputs.

Notebook repo:

• Precommit sorting out .ipynb notebooks conflicts automatically

Implementation Plan

• Merge all the latest changes from tidy3d-docs into tidy3d-notebooks.
• Update the new structure tidy3d submodule links to this develop branch.
• Set up live demo project before actual merge that demonstrates live development workflow.
• Fix all the broken API links in the notebooks
• Update the readthedocs deployment branches to develop, and verify the github actions should work for those branches too
• Update the latest develop branch of tidy3d-notebooks accordingly the day we decide we won't use tidy3d-docs anymore.
• DANGER: Change the docs mirror repository name on the sync-readthedocs-repo action on the tidy3d develop branch. This will overwrite the develop branch so that it is mirroring the new tidy3d structure.
• Verify that the new notebooks repository branch has all the github actions that still work for web deployment.
• Merge the repo_merge_no_history branch of the tidy3d repo into the develop branch.
• Bring back python 3.8 support

Post First Implementation Plan

• Upgrade the jax requirements to caret and test across all platforms
• Test tidy3d develop improvements to windows installation
• Explore pdf building
• Restructure back to python3.8
• Upgrade the github actions accordingly.
• Whether the backend still works with this commit. Can confirm with momchil, daniil, or weiliang.
• Whether the SEO works as expected with the newly hosted docs page. maybe you can ping yongwei in the em_all channel.
• Finish migrating all the pre/2.5 docs
• Finish migrating all the pre/2.5 notebooks
• Sort out the Changelog.md situation
• CLI test single notebook via arguments
• Update the development docs to the latest flow.
• Fix pandas table visualisation
• See if we can remove the extra API classes nder the table
• Include the relevant markdown files inot their corresponding API sections.
• Integrate the FAQ from the new repo

Final deployment tasks:

• Replace all hardcoded links with relative links in order to avoid version mismatches in docs
• Replace all hardcoded links with absolute links in order to avoid version mismatches in notebooks
• Update 404 links to warn of mismatched paths
• Write tests to check for latest submodule
• FAQ submodule

[momchil-flex]

Just wanted to +1 improving the table of contents organization as maybe the most important thing in my mind. As far as I remember we tried to play around with this a bit but the sphinx template was hard to understand (we could consider a different template..) But basically there is no organization in the left panel, which makes it almost useless for our example library and our api reference

I sometimes use the contents on the right, but it's not as intuitive, and if you're looking on mobile you don't even see that.

I'm bringing this up not only because it has been bugging me personally, but I had the same complaint from a user yesterday - that our examples are not organized. I showed him the main page and he was like oh ok, but he had been looking at the left panel up to then.

[daquintero]

Hi guys, all sounds good and I also agree with those points - cheers for the clear plan ahead too! I'm already working on this on these branches and will PR when several points are solved.

https://github.com/daquintero/tidy3d/tree/docs_improvement_suggestions
https://github.com/daquintero/tidy3d-docs/tree/docs_improvements_suggestions

Shall I make a flex username too or wait till the instructions of the first day?

[tylerflex]

Thanks @daquintero ! There are also some related issues tagged "daq" in the front end that I was planning to consolidate into here at some point. So if you are already working on it, might want to check those out too.

I think we can wait for your first day (Wednesday?) as then you get onboarding and will get a flexcompute email address to use.

[daquintero]

Yeah, I was checking them out to be more familiar when starting! Sounds good, we'll chat then

[daquintero]

So I've been thoroughly looking at a way that solves many of the issues in a single shot, and a strategy to propose. Mainly to get up to speed to contribute well when I start.

In summary, based on the analysis of this "brainstorming notes" discussion #1220 (please you don't need to read it all, it's more like a scratch paper to me):

I think a good approach is this: we have a pyproject.toml that controls all the requirements for the main project, and documentation + tests. This means that we have full control over how to create a virtual environment that will be functionally correct and that is shared and verified between the two source and documentation projects. We can build a mambaconfiguration installs script eg. for automatic environment configuration eg. with binaries such as pandoc and for reproducible packaging recipe for multiple platforms based on this. Also the mamba pandoc mamba recipe already includes the actual binary which apparently was an issue when reproducing the local install in the documentation repository.

This might be an integrated solution for improving an easier local build in #1148 , having mantainable and distributable multi-platform environment and package distributed via mamba, and performs an upgrade of the current setup.py scripts too re #1068 .

Maybe we'll chat more when I start, for now, I am just getting to speed so won't implement this until we chat on Wednesday.

[daquintero]

Just saw your message on the discussions and thanks for catching me up to speed, much appreciated! Really interesting chat and thinking about it

[tylerflex]

No problem, great to be having these discussions. I think it would be useful for you to continue taking a fresh perspective on this to come up with an approach to organizing things that makes sense and simplifies our development process. The only real "constraints" are:

1. we don't want huge notebook diffs to clutter our git history.
2. the docs must be hosted by flexcompute-readthedocs organization (not flexcompute), unless we can find a way around this to restrict readthedocs from seeing our private repos.
3. I feel somewhat strongly that the tidy3d repo should still be installed simply as pip install tidy3d and shouldn't need to involve anything else, like conda.

and in general I think the main "nice to have"s from my perspective (in addition to the minor things scattered around in the issues):
A. simplify the workflow of adding a new feature, potentially requiring (a) backend changes (b) frontend changes, and (c) a new notebook or some additions to the docs. Ideally, we'd like (b) and (c) to be combined and tracked in the same PR.
B. A single pyproject.toml containing all of the package requirements with both flexibility for dependency grouping (docs, features, etc) but also requirement version dependencies being defined in an organized way that avoids issues (eg. using poetry).

[Frank-DingYC]

@tylerflex These days, I have read the code on sources. I found there were no documentation about the source_time. I make some try on these part. Here are the markdown file and source code on these part.
Hopefully, these can help reader to understand the source waveform well~
Source_time.zip

[daquinteroflex]

Just writing down the changes in progress:

• Upgrade the sphinx version. I believe a lot of what we want out of documentation can be solved in a first pass through a theme dependency upgrade (because I’ve used the same theme before, haha - so I know!) There's some packages conflicts I'm currently dealing with.

The good news is that I've been testing the new pyproject.toml environment and am pretty happy using it.

[tylerflex]

@Frank-DingYC

Thanks for contributing the explanation on the various types of source time.

A few notes:

I found there were no documentation about the source_time.

The GaussianPulse is present in the documentation.

We just recently added the other two officially to the API in this commit, but it hasn't been released yet, so those will come out next release.

Note we also have an issue open #1243 for adding the equations into the documentation, which will help out.

I think your explanation and the plots are useful, I'm just not really sure how to include them right now. Maybe we could put a section in the FAQ explaining what kind of source times are available and use your plots? I don't think the python notebook is extensive enough to warrant its own separate example yet. @tomflexcompute do you have any thoughts on this?

[tylerflex]

@daquintero

• Upgrade the sphinx version. I believe a lot of what we want out of documentation can be solved in a first pass through a theme dependency upgrade (because I’ve used the same theme before, haha - so I know!) There's some packages conflicts I'm currently dealing with.

The good news is that I've been testing the new pyproject.toml environment and am pretty happy using it.

That's great. Yea we've been stuck with an older versions because nobody could figure out a way out of the dependency hell. Glad to hear that the new setup could be helping a lot in this regard!

[tomflexcompute]

@Frank-DingYC

Thanks for contributing the explanation on the various types of source time.

A few notes:

I found there were no documentation about the source_time.

The GaussianPulse is present in the documentation.

We just recently added the other two officially to the API in this commit, but it hasn't been released yet, so those will come out next release.

Note we also have an issue open #1243 for adding the equations into the documentation, which will help out.

I think your explanation and the plots are useful, I'm just not really sure how to include them right now. Maybe we could put a section in the FAQ explaining what kind of source times are available and use your plots? I don't think the python notebook is extensive enough to warrant its own separate example yet. @tomflexcompute do you have any thoughts on this?

Thanks. A separate example would probably be too basic. FAQ entry sounds good. I think eventually it would be nice to have capabilities in the API references to show schematic pictures. Currently we show equations in a few places but they can be added to more.

[daquinteroflex]

I must say I've only just come to appreciate how much of a dependency hell it actually is!

Cracking on with a good strategy I think - a lot of the issues I think are related to reproducible environment toolset upgrades

[tylerflex]

@Frank-DingYC

Maybe you could try making a pull request into tidy3d-docs adding a short discussion on the types of source time?

Basically this will involve an edit of this file https://github.com/flexcompute-readthedocs/tidy3d-docs/blob/develop/docs/source/faq.rst

You can see in the file some examples of how we do links and images in rst and it could be a good, simple exercise to learn how to contribute in the future.

[Frank-DingYC]

OK~ I wil make some try on this part~

[daquinteroflex]

Unsure if I am running into a new issue here, but I am finding that if I have more than a few notebooks open my Pycharm IDE really struggles with the cache and crashes (and I would say it's a top end computer). Maybe users are running into this and we wouldn't know, but maybe my case is an edge case.

This relates to what @lucas-flexcompute mentioned in this dicusssion:

was also looking into jupytext when trying to improve our workflow, but I could not find ans easy way to have the run results also cached in the repo. The discussion I had with Tyler and Momchil at the time was that having the cell outputs in the repository is very desirable, because PR reviews are much easier when we can compare plots side-by-side, for example, straight in github. It's also convenient to have the notebook results ready when we're making any changes without having to run them before we start (some notebooks can be very long).

My ideal solution would be to have the jupytext version of each notebook in the repository history and the ipynb versions only for the branch tip, but I don't know if that's viable. If jupytext and ipynb are kept in sync, we might be able to set up a git hook that removes old versions of the ipynb when we commit, but both working with synced files and using commit hooks are prone to error.

I am debating how to approach this personally. It might be valuable to implement something on the lines of this idea, as some of these files are large. But maybe there's no workaround at wanting to save run history and they have to be large. Am just thinking how it fits in the current flow, and in any case we can do so later on after the first merge.

UPDATE: I think this issue is primarily for VM based IDEs. vscode was fine when using it. Worth seeing how it reproduces on a standard jupyter environment

[daquinteroflex]

Important Final Decisions before First Merge
These are the breaking changes that could occur in the new docs implementation:

The Github actions as they were configured before will not work exactly due to the new files and repository structure. This could affect the syncing to MagicCloud temporarily whilst we upgrade them. It should not erase what existed there before, and any other branches. This means we need to make a decision how to accomodate for this, and update the scripts accordingly (which won't work first time)

Summary of changes:

• tidy3d consolidates docs + source for development.
• tidy3d-notebooks is the new sole directory of notebooks development. (This could replace tidy3d-example-notebooks if we wanted)
• We need to decide the future of tidy3d-docs below:

Currently the demonstrated demo implemented flow goes as follows:

• Development occurs in tidy3d/repo_merge_no_history. The sync-to-readthedocs-repo action synchronises the main repository with flexcompute-readthedocs/tidy3d-docs-demo which can be built and hosted by readthedocs without access to the flexcompute organisation. This demo is set up with the corresponding flexcompute/tidy3d-notebooks/develop, so that the docs build and pull this github submodule. This is what I aim to replicate with the main repositories.

My suggested approach is this which means that we can keep things broadly the same with a few caveats: We decide tidy3d-docs should just become the mirror repo of tidy3d, just like currently tidy3d-docs-demo is implemented currently. This means that readthedocs gets deployed as before. Whenever we want to make a new version, we just have to branch on tidy3d-docs repository just like before, but the develop branch gets updated automatically. To implement this we need to move the github sync actions to run from both repositories, tidy3d and the mirrored tidy3d-docs repository. Because MC only have already configured their GH_PAT token on the tidy3d-docs repository, this allows them to sync to their web repositories. tidy3d and tidy3d-notebooks currently don't have a token that allows them to do this, which means we depend on that token to prevent breaking things. The actions should be completed, but will run in tidy3d and fail, whilst they will succeed in tidy3d-docs. This is specifically concerning the sync-to-proxy-repo.yaml and sync.yaml actions currently in tidy3d-docs.

Another future easier approach to configure is this, but requires more MC involvement: If we decide tidy3d-docs should just become the mirror repo of tidy3d, just like currently tidy3d-docs-demo is implemented currently, then we need to move all the MagicCloud sync actions into the main repositor. The actions will run on the tidy3d development repo, until someone from MC puts in a GH_PAT token to upload to their repositories directly from there.

Reason why this is an issue:

• The notebooks repository Github action sync.yaml and sync-to-proxy-repo.yaml have changed the source file directories, as it has to do with the restructure. They should get triggered approximately the same as before, and everything else should be normal like before, but they need to be updated and run from the right place.
• Development from the tidy3d-docs repo will have to move into the new flow at a speed we choose, but notebooks branches might have to move to the new structure.

Worse case scenario:

• Syncing to magic cloud specific repos will be temporarily down. But they should retain everything they had before until the action triggers and it gets updated. If it doesn't get updated right we might have to retrigger the action.

Accomodations we could consider if we wanted:

• After the review and merge of the PRs, we could think of this: everyone else works in a migration branch as normal on the current important repositories, whilst I upgrade the development branch, then we merge the two when the kinks are ironed so it doesn't affect everyone else's work?

[daquinteroflex]

So there's a minor issue I've been trying to solve, and just putting feelers out there because the solution might involve code restructuring, or just leaving it as is:

See the new Simulation class and scroll to the bottom. You might notice that the properties, attributes and methods for the generated classes are a bit all over the place as you scroll down. This is because sphinx generates the documentation in the order in which the code is written PEP 520 .

The solution might involve restructuring the classes so that things appear in order in the documentation, which is something maybe there is could be some hesitancy in introducing unknown bugs. Other people have evaluated this problem and it's mention it's very difficult to change from a pure sphinx side. Info in related issues: software-mansion/starknet.py#716 and sphinx-doc/sphinx#5270

[tylerflex]

I dont foresee see too much of a problem with this, I'd say it's ok to reshuffle the methods. The only thing we need to be aware of is that I think the order is important for pydantic validators. For example, if validator A modifies the value somehow, then it goes to validator B. If we reverse the order, there could be effects there. But I think the methods and properties can be moved around or regrouped as needed.

[daquinteroflex]

Just updated base into 2.5 and writing up here a list of changes:

• MANIFEST.in becomes a part of include and exclude poetry statements. File gets removed
• create_config_folder in setup.py gets moved to the __init__ declaration as pyproject.tomls are declarative, not functional. I can hack it but does this move make sense anyway? Unsure about it since it includes an os import on init.py, maybe we need to discuss more. setup.py gets replaced by pyproject.toml UPDATE: web/cli/app already creates TIDY3D_DIR, so maybe I just get rid of it in __init__.py?
• Important to note I had replaced tox.ini with a custom script that uses poetry on github actions when I was first debugging poetry - which I think is equivalent. It has been updated with vtk and I have updated it with the vtk commands now. Do we want to matain local testing with tox? I can see a workaround to test it with poetry too both on GhActions and locally if we want. @momchil-flex
• Verified all old and new requirements are ported into the new pyproject.toml, updated the poetry lock file. Deleted all the requirements directories
• Fixed docstring conflicts from updated classes/parents
• Moved relevant attribute docstrings from Simulation to AbstractSimulation.
• Going to make a tidy3d command line for make_package.py? I'm unconvinced it should be standalone here in the root of the package and the tests would remain the same.
• Because of the integration of the notebooks and package, there's now a few more cli tidy3d develop functions to choose what type of testing we want to do.
• The MC web restructure has meant so many docs links have been broken 😦
• Warning: Two version docstrings have to be updated: one in pyproject.toml and the other in version.py. We could use something like bump2version if we wanted too to manage this like gdsfactory.
• There's a CHANGELOG.md to rst situation with m2r2 compatibility. I might have to PR or patch the package.

Updates:

• All tests are passing in my machine with the poetry install setup and all my changes. Need to see what's happening on the cloud.
• Docs are building again but links and a few things will be broken

[daquinteroflex]

Leaving a trace on the poetry jax situation:

• Poetry is struggling to find jax --find-links equivalent python-poetry/poetry#1391 it can only find PEP503 compatible projects.
• We need to configure a repository to search for sources https://whls.blob.core.windows.net/unstable/index.html
• PEP 503 builds are not supported by the CUDA jax repo re Please provide PEP 503 compliant indices for CUDA versions of packages jax-ml/jax#5410 however remember we want CPU PEP503 compatible versions
• Added this potential source solution Please provide PEP 503 compliant indices for CUDA versions of packages jax-ml/jax#5410 (comment)

Unclear to me when I tested pyproject.toml upgrade before all tests passed but now running into this.

Update: I've done some digging when I tested the pyproject.toml before. Turns out the github action was saving the venv in the cache and reusing the old one pre pyproject.toml. Didn't realise this before, so kind of makes sense why these issues are arising now.

Potential solution for the PEP503 packaging situation here. There is a repository for all PEP503 compatible builds. Bad news is that we need to increase requirements for poetry to recognise them.

I guess one of the things that poetry throws are the caveats that you don't find via pip. For example in the source APIs which has the 503 compliant builds, there is no linux cpu for for our specified 0.3.14 minimum version which supports py311, and I am wondering why it is resolving for the lowest version only (maybe because the requirements are restrictive and "verified").

[nocuda/jaxlib-0.3.10-cp39-none-manylinux2014_x86_64.whl](https://storage.googleapis.com/jax-releases/nocuda/jaxlib-0.3.10-cp39-none-manylinux2014_x86_64.whl)
[nocuda/jaxlib-0.3.14-cp310-none-manylinux2014_x86_64.whl](https://storage.googleapis.com/jax-releases/nocuda/jaxlib-0.3.14-cp310-none-manylinux2014_x86_64.whl)
[nocuda/jaxlib-0.3.14-cp37-none-manylinux2014_x86_64.whl](https://storage.googleapis.com/jax-releases/nocuda/jaxlib-0.3.14-cp37-none-manylinux2014_x86_64.whl)
[nocuda/jaxlib-0.3.14-cp38-none-manylinux2014_x86_64.whl](https://storage.googleapis.com/jax-releases/nocuda/jaxlib-0.3.14-cp38-none-manylinux2014_x86_64.whl)
[nocuda/jaxlib-0.3.14-cp39-none-manylinux2014_x86_64.whl](https://storage.googleapis.com/jax-releases/nocuda/jaxlib-0.3.14-cp39-none-manylinux2014_x86_64.whl)
[nocuda/jaxlib-0.3.15-cp310-none-manylinux2014_x86_64.whl](https://storage.googleapis.com/jax-releases/nocuda/jaxlib-0.3.15-cp310-none-manylinux2014_x86_64.whl)
[nocuda/jaxlib-0.3.15-cp37-none-manylinux2014_x86_64.whl](https://storage.googleapis.com/jax-releases/nocuda/jaxlib-0.3.15-cp37-none-manylinux2014_x86_64.whl)
[nocuda/jaxlib-0.3.15-cp38-none-manylinux2014_x86_64.whl](https://storage.googleapis.com/jax-releases/nocuda/jaxlib-0.3.15-cp38-none-manylinux2014_x86_64.whl)

Python 11 support for CPU installation began in 0.4.1 so going to move onto that as minimum requirement?

Also there's the windows situation that PEP503 compatible builds are only found after 0.4.13 and must be for Python 3.9 onwards @tylerflex Maybe there is a requirement to increase the minimum python version here unfortunately? What do you think?

[mac/jaxlib-0.4.9-cp39-cp39-macosx_11_0_arm64.whl](https://storage.googleapis.com/jax-releases/mac/jaxlib-0.4.9-cp39-cp39-macosx_11_0_arm64.whl)
[windows/jaxlib-0.4.13-cp310-cp310-win_amd64.whl](https://storage.googleapis.com/jax-releases/windows/jaxlib-0.4.13-cp310-cp310-win_amd64.whl)
[windows/jaxlib-0.4.13-cp311-cp311-win_amd64.whl](https://storage.googleapis.com/jax-releases/windows/jaxlib-0.4.13-cp311-cp311-win_amd64.whl)
[windows/jaxlib-0.4.13-cp39-cp39-win_amd64.whl](https://storage.googleapis.com/jax-releases/windows/jaxlib-0.4.13-cp39-cp39-win_amd64.whl)
[windows/jaxlib-0.4.14-cp310-cp310-win_amd64.whl](https://storage.googleapis.com/jax-releases/windows/jaxlib-0.4.14-cp310-cp310-win_amd64.whl)
[windows/jaxlib-0.4.14-cp311-cp311-win_amd64.whl](https://storage.googleapis.com/jax-releases/windows/jaxlib-0.4.14-cp311-cp311-win_amd64.whl)
[windows/jaxlib-0.4.14-cp39-cp39-win_amd64.whl](https://storage.googleapis.com/jax-releases/windows/jaxlib-0.4.14-cp39-cp39-win_amd64.whl)
[windows/jaxlib-0.4.16-cp310-cp310-win_amd64.whl](https://storage.googleapis.com/jax-releases/windows/jaxlib-0.4.16-cp310-cp310-win_amd64.whl)
[windows/jaxlib-0.4.16-cp311-cp311-win_amd64.whl](https://storage.googleapis.com/jax-releases/windows/jaxlib-0.4.16-cp311-cp311-win_amd64.whl)
[windows/jaxlib-0.4.16-cp39-cp39-win_amd64.whl](https://storage.googleapis.com/jax-releases/windows/jaxlib-0.4.16-cp39-cp39-win_amd64.whl)

I'm afraid we need to make a decision, but at least this implementation seems to work and is reproducible.

[daquinteroflex]

So we just need to take the decision above pending rebase.There are many things to fix still but generally can be rebased imo so we can begin merging and updating the simulation flow including the docs live update.

[tylerflex]

I'll look in more detail soon but regarding 3.8 support: Is there a way perhaps to support >= 3.8 generally but only >= 3.9 for the adjoint plugin (jax dependency group) and windows users? Eg if a windows user tries to pip install tidy3d[jax] with python < 3.9 we provide a useful error message?

[daquinteroflex]

I think we can do that, the problem will be that the jax tests won't pass for that version. We can also try to create test groups which we maybe have to do anyway for the notebooks as it's all consolidated. This just means it's an upgrade on the tests pipeline accordingly.

Are you also able to help me when you can with configuring the rebase properly? I think I accidentally merged due to the automatic git config, rather than actually rebase. Going to backup in a new branch but unsure how to best approach this, or start from a new branch instead and merge again all the changes? Not inclined to keep adding new features until I'm properly set up to the flow.

[tylerflex]

Here's one thing I noticed, the extra elements in the TOC at the bottom are kind of distracting.

should they removed or put under their own header?

[tylerflex]

Another issue: pandas DataFrames dont render very well in this dark mode. Is there a way around this?:

[tylerflex]

what's the status with this issue? still in progress?

[daquinteroflex]

Yeah so I'm editing the tasks at the top to consolidate the missing things

[daquinteroflex]

Closing this in favour of the post migration #1568