FXC-3769 inverse design seminar notebooks

[yaugenst-flex]

No description provided.

[github-actions]

Spell Check Report

2025-10-09-invdes-seminar/00_setup_guide.ipynb:

Cell 1, Line 3: 'dual-layer'
  > > In this notebook, we will set up a baseline simulation for a dual-layer grating coupler. This device is designed to efficiently couple light from an optical fiber to a photonic integrated circuit. We will define the geometry, materials, and all the necessary components for a Tidy3D simulation. This initial setup will serve as the starting point for our optimization in the subsequent notebooks.

---

2025-10-09-invdes-seminar/01_bayes.ipynb:

Cell 1, Line 3: 'black-box'
  > > With our simulation setup in place, we now turn to optimization. Our goal is to find a set of grating parameters that maximizes the coupling efficiency. Since each simulation is computationally expensive, we will use Bayesian optimization. This technique is ideal for optimizing "black-box" functions that are costly to evaluate.
Cell 1, Line 7: 'higher-dimensional'
  > Exhaustive searches would require thousands of simulations. Bayesian optimization instead builds a probabilistic surrogate of the objective, balancing exploration of uncertain regions with exploitation of promising designs to converge in far fewer solver calls. It intelligently explores the parameter space to find the optimal design with a minimal number of simulations. Bayesian optimization works best when the design space has only a handful of effective degrees of freedom; beyond roughly five independent variables the surrogate becomes harder to learn, so we reserve higher-dimensional searches for gradient-based methods discussed later in the series.
Cell 5, Line 4: 'pbounds'
  > - `parameter_bounds` (the `pbounds` argument) defines the design window we explore.
Cell 5, Line 8: '-Parameter'
  > ## Framing the Problem: A 5-Parameter Global Search
Cell 5, Line 10: 'five-dimensional', 'inter-layer'
  > Rather than tune every tooth individually (30 variables per layer), we search a five-dimensional space of uniform widths, gaps, and inter-layer offset. This captures the dominant physics, keeps simulations fast, and yields a design that later gradient-based passes can refine.
Cell 10, Line 3: 'high-efficiency'
  > We extract the optimizer history, track the best observed loss, and visualize how the search converges toward high-efficiency gratings.
Cell 14, Line 3: 'optimizer's'
  > We reconstruct the best-performing structure, inspect its geometry, and analyze the spectral response to confirm the optimizer's progress.
Cell 23, Line 17: 'utf-'
  > with export_path.open("w", encoding="utf-8") as f:

---

2025-10-09-invdes-seminar/02_adjoint.ipynb:

Cell 1, Line 1: 'High-Dimensional'
  > # Adjoint Optimization: High-Dimensional Gradient-Based Refinement
Cell 1, Line 3: 'low-dimensional'
  > > In the previous notebook, we used Bayesian Optimization to find a good starting design. The strength of that global optimization approach was its ability to efficiently search a low-dimensional parameter space. However, it was limited: we assumed the grating was uniform, with every tooth and gap being identical.
Cell 1, Line 5: '-element', 'apodize', 'dual-layer'
  > > To push the performance further, we need to apodize the grating, which means varying the dimensions of each tooth individually to better match the profile of the incoming Gaussian beam. This drastically increases the number of design parameters. For our 15-element dual-layer grating, the design space just expanded from 5 global parameters to over 60 individual feature dimensions!
Cell 1, Line 7: 'high-dimensional'
  > > For such a high-dimensional problem, a global search is no longer efficient. In this notebook, we switch to a powerful local, gradient-based optimization technique, enabled by the adjoint method, to refine our design.
Cell 2, Line 5: 'higher-performance'
  > This is where the adjoint method comes in. Tidy3D's automatic differentiation capability uses this method under the hood. It allows us to compute the gradient of our objective function (the coupling efficiency) with respect to all design parameters simultaneously in just two simulations per iteration, regardless of how many parameters there are. This efficiency is what makes it possible to locally optimize structures with thousands of free parameters. We start from the global design found earlier and use these gradients to walk toward a nearby, higher-performance solution.
Cell 5, Line 1: 'High-Dimensional'
  > ## High-Dimensional Parameterization
Cell 5, Line 3: 'per-tooth'
  > We load the best uniform design from the Bayesian search and expand those scalars into per-tooth arrays. Each layer now has individual widths and gaps, and `first_gap_si` remains a crucial phase-matching variable.
Cell 6, Line 2: 'utf-'
  > with Path("./results/gc_bayes_opt_best.json").open("r", encoding="utf-8") as f:
Cell 11, Line 3: 'adjoint-driven'
  > The steadily rising power confirms the adjoint-driven search is homing in on a better design.
Cell 16, Line 3: 'mode-match'
  > Visual inspection highlights the non-uniform duty cycle discovered by the optimizer to better mode-match the incident beam.
Cell 19, Line 2: 'JSON-serializable'
  > """Detach autograd containers into JSON-serializable Python objects."""
Cell 19, Line 19: 'utf-'
  > with export_path.open("w", encoding="utf-8") as f:
Cell 20, Line 3: 'high-dimensional'
  > Switching to a gradient-based approach unlocked high-dimensional refinements and reduced the coupling loss by more than a decibel. The resulting design is finely tuned for nominal fabrication, so the next notebook introduces robust optimization to preserve performance under realistic manufacturing variations.

---

2025-10-09-invdes-seminar/03_sensitivity.ipynb:

Cell 1, Line 3: 'adjoint-optimized'
  > > The adjoint-optimized grating from the previous notebook delivers excellent nominal performance. In practice, however, fabrication variability means the manufactured device rarely matches the design exactly. Here we quantify how the current design responds to some assumed process deviations to see whether it is robust or brittle.
Cell 1, Line 5: 'follow-up', 'well-controlled'
  > > In the adjoint notebook we purposefully focused on maximizing performance at the nominal geometry. The natural follow-up question is: *how does that optimized design behave once it leaves the computer?* Photonic fabrication processes inevitably introduce small deviations in etched dimensions. Even a well-controlled foundry run can exhibit ±20 nm variations in tooth widths and gaps due to lithography or etch bias. A design that is overly sensitive to these changes might look great in simulation yet fail to meet targets on wafer, so our immediate goal is to measure that sensitivity before pursuing robustness improvements.
Cell 2, Line 3: 'over-etched', 'under-etched'
  > We begin by reloading the best adjoint design and defining a simple bias model. A ±20 nm shift in feature dimensions is a realistic foundry tolerance, so we will simulate three cases: the nominal geometry, an over-etched device (features narrower than intended), and an under-etched device (features wider than intended). This gives an intuitive first look at the design's sensitivity before launching a full Monte Carlo analysis.
Cell 4, Line 2: 'numpy-friendly'
  > """Load a design JSON (Bayes or adjoint) into numpy-friendly fields."""
Cell 4, Line 3: 'utf-'
  > data = json.loads(Path(path).read_text(encoding="utf-8"))
Cell 6, Line 11: 'over-etched'
  > # Create simulations for each fabrication scenario: over-etched, nominal,
Cell 6, Line 12: 'under-etched'
  > # and under-etched. Positive bias widens features, while a negative bias
Cell 6, Line 13: 'over-etching'
  > # corresponds to over-etching that narrows them.
Cell 6, Line 15: 'Over-etched'
  > "Over-etched (-20 nm)": builder(etch_bias=-bias),
Cell 6, Line 17: 'Under-etched'
  > "Under-etched (+20 nm)": builder(etch_bias=bias),
Cell 8, Line 3: 'high-efficiency'
  > The curves below compare the nominal spectrum to ±20 nm biased geometries. The separation between them conveys how quickly our high-efficiency design degrades under realistic fabrication shifts in tooth width and gap. Watch for both a drop in peak efficiency and a shift of the optimal wavelength.
Cell 9, Line 3: 'Over-etched'
  > "Over-etched (-20 nm)": "tab:orange",
Cell 9, Line 5: 'Under-etched'
  > "Under-etched (+20 nm)": "tab:blue",
Cell 11, Line 3: 'foundry-provided'
  > After inspecting the deterministic bias sweep, we broaden the analysis with a Monte Carlo study. We randomly sample overlay, spacer, and width variations according to foundry-provided sigma values to estimate the distribution of coupling efficiency across a wafer.
Cell 14, Line 1: 'silicon-width'
  > We draw overlay, spacer, and silicon-width perturbations from independent Gaussian models whose sigmas come straight from the (hypothetical) foundry tolerance table. Each row in the `samples` array represents one die that we will feed into the simulation pipeline.
Cell 21, Line 10: 'lightgray'
  > color="lightgray",
Cell 23, Line 1: 'center-wavelength', 'single-number'
  > The helper converts the center-wavelength transmission into dB loss and aggregates mean, standard deviation, and percentile values. These single-number metrics offer a quick dashboard before moving on to more detailed adjoint sensitivities.
Cell 26, Line 3: 'silicon-width'
  > Before launching a full robust optimization we want directional information: which fabrication knobs most strongly impact coupling efficiency near the nominal point? The objective below evaluates a single perturbed simulation and, through `value_and_grad`, returns both the power and its gradient with respect to the overlay, spacer, and silicon-width errors.
Cell 32, Line 3: 'gradient-scaled'
  > Normalizing the gradient-scaled sigmas reveals how much each parameter contributes to the linearized variance. Plotting the breakdown highlights the dominant sensitivities we should target when we redesign for robustness.
Cell 37, Line 5: 'lightgray'
  > color="lightgray",

---

2025-10-09-invdes-seminar/04_adjoint_robust.ipynb:

Cell 3, Line 3: 'determins', 'over-etched', 'under-etched'
  > We evaluate the design under three fabrication scenarios: nominal, over-etched (−20 nm), and under-etched (+20 nm). We then maximize the mean transmission and simultaneously minimize the standard deviation in performance between these different scenarios, which should lead to a more robust design overall. The amount of weight we place on the standard deviation minimization determins the tradeoff between nominal performance and robustness.
Cell 5, Line 3: 'fabrication-sensitive'
  > We seed the optimizer with the fabrication-sensitive adjoint design and enforce the same foundry limits as before so the updates remain manufacturable.
Cell 6, Line 1: 'utf-'
  > data = json.loads(Path("./results/gc_adjoint_best.json").read_text(encoding="utf-8"))
Cell 7, Line 3: 'adjoint-optimized'
  > Starting from the adjoint-optimized design found earlier, we use Adam to minimize the robust objective.
Cell 11, Line 1: 'Post-Optimization'
  > ### Pre- and Post-Optimization Bias Sweeps
Cell 12, Line 16: 'Over-etched'
  > ("Over-etched (-20 nm)", apply_bias(param_dict, -bias)),
Cell 12, Line 18: 'Under-etched'
  > ("Under-etched (+20 nm)", apply_bias(param_dict, bias)),
Cell 14, Line 1: 'Over-etched'
  > labels = ["Over-etched (-20 nm)", "Nominal", "Under-etched (+20 nm)"]
Cell 14, Line 1: 'Under-etched'
  > labels = ["Over-etched (-20 nm)", "Nominal", "Under-etched (+20 nm)"]
Cell 14, Line 3: 'Over-etched'
  > "Over-etched (-20 nm)": "tab:orange",
Cell 14, Line 5: 'Under-etched'
  > "Under-etched (+20 nm)": "tab:blue",
Cell 15, Line 3: 're-running'
  > Finally we save the fabrication-aware geometry so downstream notebooks - or a GDS handoff - can reuse it without re-running the optimization loop.
Cell 16, Line 13: 'utf-'
  > export_path.write_text(json.dumps(export_payload, indent=2), encoding="utf-8")

---

2025-10-09-invdes-seminar/05_robust_comparison.ipynb:

Cell 1, Line 5: 'robustness-optimized'
  > > This notebook compares the nominal adjoint design against the robustness-optimized variant using a matched Monte Carlo experiment, highlighting the yield benefits of carrying fabrication awareness into the optimization loop.
Cell 4, Line 2: 'numpy-friendly'
  > """Load a design JSON (Bayes or adjoint) into numpy-friendly fields."""
Cell 4, Line 3: 'utf-'
  > data = json.loads(Path(path).read_text(encoding="utf-8"))
Cell 11, Line 1: 'Center-Wavelength'
  > ## Distribution of Center-Wavelength Loss
Cell 13, Line 10: '-percentile', 'worst-case'
  > 90th-percentile loss improves (2.80 → 2.75 dB, better worst-case).
Cell 13, Line 11: '-percentile', 'best-case'
  > 10th-percentile loss rises slightly (2.19 → 2.22 dB, marginally less best-case).
Cell 14, Line 2: 'wafer-level'
  > Even a few hundredths of a decibel can translate to higher wafer-level yield when scaled to thousands of devices.

---

2025-10-09-invdes-seminar/06_measurement_calibration.ipynb:

Cell 1, Line 3: 'as-built'
  > > Our robust adjoint design is ready for fabrication, but once real devices come back from the foundry, their spectral responses rarely match the nominal simulation exactly. In this notebook we demonstrate a way to calibrate the simulation model to match measured data using adjoint optimization, recovering the as-built geometry so subsequent optimization or analysis stays grounded in reality.
Cell 6, Line 5: 'utf-'
  > robust_data = json.loads(robust_path.read_text(encoding="utf-8"))
Cell 11, Line 3: 'mean-squared'
  > We adjust the SiN tooth widths so the simulated spectrum matches the measured one. The loss is the mean-squared error between spectra sampled at the monitor frequencies, optimized with Adam while respecting fabrication bounds.
Cell 18, Line 3: 'higher-yield'
  > By calibrating the simulation to match measurement we keep the model and fabricated hardware in sync. Combined with robust optimization this closes the loop between design, fabrication, and test, enabling faster debug and higher-yield deployment of inverse-designed photonics.

---

Checked 7 notebook(s). Found spelling errors in 7 file(s).
Generated by GitHub Action run: https://github.com/flexcompute/tidy3d-notebooks/actions/runs/18682606416

[tylerflex]

These are a great walkthrough of the process for doing robust gradient-based design. It's very clear. I did a read through and had a few high level comments:

Do want to add these to the TOC / rst files so they show up in the docs? and do they deserve their own section maybe?

The sensitivity values probably should have units? or is it ok? maybe some mention because people might not immediately get that these are literally just units of [objective fn return units] / [parameter unit]

The robust Monte Carlo adjoint actually didnt seem to work that well from the plots?

I wonder if this is something we can improve with a different example? or trying different parameters? or is it just not expected to work that well?

When you quantify the robust vs nominal designs in section 5, do you sample the same fab variations used in the adjoint optimization? or is it a new sample? is there a risk of overfitting to the specific sample?

It seems the measurement calibration didn’t recover the actual structure? can this be explained with the same issue we saw in the example I did regarding the lack of invertibility? Should we comment on it?

[yaugenst-flex]

Do want to add these to the TOC / rst files so they show up in the docs? and do they deserve their own section maybe?

Yeah I didn't do it mainly because I didn't know how this ends up interacting with the web examples 😄 The notebooks do need to be in their own section I think, especially since they all share the same setup code. @daquinteroflex can I just add this to the TOC or do we have to take care of anything else?

The sensitivity values probably should have units? or is it ok? maybe some mention because people might not immediately get that these are literally just units of [objective fn return units] / [parameter unit]

Yeah good idea. In this case the parameters are all in um so that should be quite intuitive.

The robust Monte Carlo adjoint actually didnt seem to work that well from the plots?
I wonder if this is something we can improve with a different example? or trying different parameters? or is it just not expected to work that well?

Yes from just the spectra the difference is not obvious. One of them is that this is all single-frequency, so you really have to look at the difference only at the central wavelength, everything else kind of doesn't matter. There are a couple of other factors that also play into this, which I discuss at the end of 05_robust_comparison.ipynb - do you think that covers these concerns? In my opinion effect is large enough for an educational demo. You can get a stronger effect depending on the starting design, which depends on the seed in the Bayesian optimization. I could play around with that and try to get a bit luckier but I'm not super convinced of that approach 😄

When you quantify the robust vs nominal designs in section 5, do you sample the same fab variations used in the adjoint optimization? or is it a new sample? is there a risk of overfitting to the specific sample?

The robust adjoint here is actually not stochastic, we sample 3 fixed scenarios:

scenarios = {
    "nominal": params,
    "over": apply_bias(params, -ETCH_BIAS),
    "under": apply_bias(params, ETCH_BIAS),
}

So I don't think overfitting to the distribution is a concern. I did this to keep things simple and deterministic (and not to run too many simulations). I was planning on expanding on these considerations a bit if we publish this as some sort of tutorial/blog series.

It seems the measurement calibration didn’t recover the actual structure? can this be explained with the same issue we saw in the example I did regarding the lack of invertibility? Should we comment on it?

I think it works quite well? Maybe it's an issue with the presentation? The green line starts out at the nominal design and converges toward the orange line, which it is trying to fit. The remaining difference between green and orange here is on the order of the noise injected into the spectrum, so I think that's expected. I chose only a single degree of freedom here to illustrate the concept and avoid problems with nonuniqueness. Granted it's a bit contrived but my goal was just to demonstrate the concept.

[tylerflex]

You can get a stronger effect depending on the starting design, which depends on the seed in the Bayesian optimization. I could play around with that and try to get a bit luckier but I'm not super convinced of that approach 😄

I'm mostly wondering if tweaking the meta parameters of the robust optimization can improve things. eg more simulations, more variation in the sims.

Also I'm not suggesting removing this, because it's useful for educational aid but one question is whether the bayesian optimization part is useful here. vs just doing adjoint from the start? It uses a lot of sims.

A related question is I wonder if some of this (bayes, also Monte Carlo) would be good to do using the design plugin, just for more examples of that.

I think it works quite well? Maybe it's an issue with the presentation? The green line starts out at the nominal design and converges toward the orange line, which it is trying to fit.

It converges towards but it doesn't ever reach it. I noticed the same thing in my own study and ultimately ended up fixing it through changing parameters. I just anticipate this is a question people might have and it would probably be nicer if we could get it to find the true values?

The remaining difference between green and orange here is on the order of the noise injected into the spectrum, so I think that's expected.

Hm, but the noise injected is not systematic? to me the read of the plot is that it systematically underestimates the tooth sizes? I'm not sure I understand why but unless I'm missing something I'd expect the noise would just mean some green points would be above or below the orange, not all the same amount below them.

I chose only a single degree of freedom here to illustrate the concept and avoid problems with nonuniqueness. Granted it's a bit contrived but my goal was just to demonstrate the concept.

Yea this is fair, I just think it's possible we can augment these examples a bit to make them even stronger, but they demonstrate the concept very well.

[daquinteroflex]

Why don't you convert this to a rst file, add the toctree in here for each notebook and then it's relatively self contained to its own section in the docs. Then you just need to add this file to a notebook section toctree at whichever section hierarchy you want?

[daquinteroflex]

Basically you can move this file to docs/ but change the toctree to the notebook folder path in this directory

[daquinteroflex]

Actually, the bigger problem will be the way that this will get represented in the commercial website notebooks. That's why they're all at the same top level of this repo.