Initial test on an instrument (CRISP)

[Tom-Willemsen]

• As a scientist I'd like to see and provide feedback on an initial version of the library
• As a developer I'd like to know how our bluesky scans behave on a real instrument

Acceptance criteria

• Run some relatively simple test scans on a real instrument
  • CRISP is a likely first candidate as not fully scheduled, but representative. Talk to CK.
  • The library will eventually be used across different science groups, but for this ticket, testing on a single real instrument is sufficient
• Test should cover:
  • Using a block (e.g. scanning a motor would cover this)
    • Ideally try using multiple types of blocks, e.g. BlockRw and BlockMot.
  • Using the DAE, including some kind of normalization to e.g. monitors or good frames
    • Ideally test both "one run per scan point" and "one period per scan point" modes, even if the instrument in question would only actually use one of those modes
  • Plotting the results of the scan
  • Fitting the results of the scan (assuming Callbacks: fitting #19 is completed)
  • Writing an output file (assuming Callbacks: file writing (human-readable files) #7 is completed)
• Feedback from test collected:
  • What went well
  • What didn't go well (ensure tickets exist for these things)
  • Any missing functionality not covered by existing tickets / not noted explicitly in the requirements document
  • Share feedback / notes / screenshots etc at the next scans library regular meeting after this test takes place

Notes

Ensure we set expectations appropriately with scientists: this is an initial, exploratory test of an early version of the library. Problems, missing functionality, etc are expected, that is the point of this testing.

[Tom-Willemsen]

Provisional dates: 21st/22nd November on CRISP

[Tom-Willemsen]

Some documentation from scientists about what/how we'll be testing on CRISP stored in

\\isis\shares\ISIS_Experiment_Controls\data for tickets\ibex_bluesky_core_32

[jackbdoughty]

Configuration script we used for testing on CRISP:
crisp_bluesky.zip

[Tom-Willemsen]

Test is substantively complete, was run on CRISP with CK and @jackbdoughty

Will keep in progress pending a review on Monday with more of the refl scientists.

---

Test scans on CRISP have been demo'd directly to CK (Refl), MC (Refl), JL (Muons).

• Overall feedback was very positive
  • CK comment is that this is already broadly comparable to, and in a few cases (e.g. adaptive scanning) better than, the existing scans library. Effectively giving sign-off that this library is at or near the point where it could replace the existing scans code for refl use-cases.
  • CK commented that performance of scans is much faster in bluesky than in the old scans code (likely due to e.g. reading detectors in parallel via asyncio - I also saw very substantial speedup from this on my dev machine)
• We were able to align a sample on CRISP, using bluesky.
  • This is a core-use case for reflectometers.
• All of the major bits of functionality we've implemented worked
  • Tested a variety of scans on different reflectometry parameters & blocks
  • Tested the main DAE run modes: period per point & run per point, saving files or not
  • Plotting and fitting generally worked well, some usability feedback noted in issues below
  • File formats are generally fine at the moment (both the overall scans record and also the fit records)
• As all of the "simple" step scans worked rather easily, we also tried adaptive scans; these also worked well with minimal setup
• Scientists so far have been generally ok with ctrl-c behaviour and typing RE() around plans and other command-line syntax.
• Reflectometry instrument scripts install a global python logging handler which causes a lot of messages to be printed to screen (only after using their instrument scripts)
  • Solution to this is to fix reflectometry scripts to not install a global handler, and to instead only install a handler on it's own messages.

---

Below is one normal step-scan, and one adaptive scan, of the same axis using the DAE in period mode on CRISP (notice the tight resampling over the gaussian region by the adaptive scan, giving good peak statistics for a reasonably low point cost):

Example command was:

x = 0.3
RE(adaptive_scan(b.SAMPPHI, x-0.25, x+0.25, min_step=0.005, max_step=0.03, target_delta=0.002, frames=50, mon=1, det=3, model=fit.Gaussian()))

---

Issues have been generated as a result of this testing - mostly style/usability things rather than functional ones:

• init guess label not appearing in legend of LiveFitPlot #69
• LiveFitPlot - set title to parameters of fit #70
• Slit scan guess #72
• LivePlot include units on axes #73
• Scan IDs, filenames, saved files, rb numbers, archiving, etc #24 (comment added)

[Tom-Willemsen]

Since CK commented on scan performance I thought I would benchmark a representative scan on CRISP while I had access... (ignore the difference between PHI and SAMPPHI, it's scanning the same variable, that's an artefact of CRISP's refl setup...). Starting both from PHI = x = 0.3:

Old scans code

start=time.time();crisp_scan(b.phi, x-0.25, x+0.25, 51, frames=10, fit=Gaussian);end=time.time()
213.7 seconds

Bluesky

start=time.time();RE(scan(b.SAMPPHI, x-0.25, x+0.25, 51, frames=10, model=fit.Gaussian()));end=time.time()
131.9 seconds

Theoretical

(assuming perfectly-moving motors, infinitely fast DAE, etc...)

• Counting: 51 * 10 frames / 40 fps = 12.7 seconds
• Moving: ~4 seconds of movement time (accounting for repeated acceleration/deceleration of axis)

~16.7 secs total for a theoretically perfect version of this step scan

---

i.e. bluesky is a bit faster than old library (mostly due to more efficient CA usage), but still an order of magnitude off the physical limits