access APS proposal and ESAF information

[prjemian]

There is a wealth of information available from the APS proposal system, the experiment safety form system, and the beam time scheduling system. Provide access to this information so it can be included with the run metadata.

To be useful at many beam lines, provide:

• a local EPICS IOC to host a copy of the selected proposal and ESAF for the current beam time
• GUI screens for the IOC (MEDM or caQtDM)
• GUI button to load proposal/ESAF
• python program to configure default parameters in the local IOC
• python program to load a specific proposal and/or ESAF into the IOC
• python program to report available proposals and ESAFs
• python program to report current EPICS IOC values
• bash shell script to manage the EPICS IOC
• unit tests for Python code
• documentation
• packaging requirements

This follows from https://jira.aps.anl.gov/browse/BCDA_GROUP-589 (restricted to APS onsite-only access).

[prjemian]

Can the information in the beam time scheduling system be accessed to identify the current proposal and ESAF? At present, the proposal number AND the ESAF number must be entered by hand.

[prjemian]

The python program(s) can use ophyd to manage the structured information.

[prjemian]

Use the package already available (dm: APS data management package) to interface with the proposal system and the ESAF system.

[prjemian]

Either

conda install -c aps-anl-tag aps-dm-api

or

conda install -c sveseli aps-dm-api

but note the most recent version might not be available on both. Check.

[prjemian]

Also, dm is Mac or linux, no Windows version provided.

[prjemian]

Run the EPICS IOC with softIoc -d beamtime_info.db where the database has been designed based on the information available through the dm package.

[prjemian]

The base name of bss_info has been used in development where bss is the APS Beam time Scheduling System. Expect this to be sufficiently unique and self-explanatory.

[prjemian]

For the python programs, combine them into one program with subcommands:

(bluesky_2020_5) jemian@wow ~/.../apstools/beamtime $ ./bss_info.py  -h
usage: bss_info.py [-h]
                   {beamlines,current,cycles,esaf,proposal,clear,setup,update}
                   ...

Retrieve specific records from the APS Proposal and ESAF databases

optional arguments:
  -h, --help            show this help message and exit

subcommand:
  {beamlines,current,cycles,esaf,proposal,clear,setup,update}
    beamlines           print list of beamlines
    current             print current ESAF(s) and proposal(s)
    cycles              print APS cycle names
    esaf                print specific ESAF
    proposal            print specific proposal
    clear               EPICS PVs: clear
    setup               EPICS PVs: setup
    update              EPICS PVs: update from BSS

[prjemian]

Build unit tests for this suite, knowing that certain key tests can only work on APS workstations.

[prjemian]

This adds dm as a required package to run this code. Only works on APS workstations.

[prjemian]

python program to respond to EPICS IOC requests to load proposal/ESAF

Build this into bss_info.py as a new subcommand.

[prjemian]

python program to report current EPICS IOC values

Build this into bss_info.py as a new subcommand.

[prjemian]

Change the name from bss_info to apsbss.

[prjemian]

shell script for IOC management:

(bluesky_2020_5) mintadmin@mint-vm:~/.../apstools/beamtime$ ./manage_ioc.sh 
Usage: manage_ioc.sh {start|stop|restart|status|console|run} [NAME [PREFIX]]

    COMMANDS
        console   attach to IOC console if IOC is running in screen
        restart   restart IOC
        run       run IOC in console (not screen)
        start     start IOC
        status    report if IOC is running
        stop      stop IOC

    OPTIONAL TERMS
        NAME      name of IOC session (default: apsbss)
        PREFIX    IOC prefix (default: ioc:bss:)

[prjemian]

From https://jira.aps.anl.gov/browse/BCDA_GROUP-589:

Provide information from the APS Oracle user database as PVs at each beam line so that this information may be added to new data files

GOAL: create an EPICS user_metadata.db to define the PVs to be provided at each instrument.

• proposal number
• proposal type
• link to proposal (URL)
• ESAF number
• title
• principal investigator
• contact email
• ISO8601 time string (since synApps 5.8 as $(P):iso8601)

PVs only update when calendar system changes (which is when scheduling system sees a change).

[prjemian]

Most of this is already done here. No link to proposal (URL), not sure that proposal type is available. Not sure how to connect this with the beam time scheduling system for semi-automatic updates.

For now, instrument team must update on-demand, providing APS cycle, ESAF, and proposal numbers. Each instance (of the EPICS database) will be pre-configured for IOC prefix and beamline name (as represented by dm system).

[prjemian]

This note from 2017:

The data management system does not use the SOAP/SUDS interface from the scheduling system. It has direct access to views into the IS group’s databases via a database service account.

The data management tools have a Python API and command line tools to get current users, proposal names, etc. One could potentially use these. Another way is to directly access the user databases. This may be better so as to minimize dependencies.

[prjemian]

Another note from 2018:

from dm.aps_bss.impl.bssClient import BssClient

This looks to be the core of what I'll need to for the service I have in mind. The service is to run an EPICS IOC for each instrument that has the current information from the scheduling service. The various PVs can be used as metadata in data files, display content for GUI screens, or other.

[prjemian]

That import is not available today:

In [1]: from dm.aps_bss.impl.bssClient import BssClient                                                                                          
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-0bc219528942> in <module>
----> 1 from dm.aps_bss.impl.bssClient import BssClient

ModuleNotFoundError: No module named 'dm.aps_bss'

[prjemian]

The import shown above has been refactored and is now used in the code here:

apstools/apstools/beamtime/apsbss.py

Line 31 in fc33050

api_bss = dm.BssApsDbApi(DM_APS_DB_WEB_SERVICE_URL)

[prjemian]

Note also this repository in the APS (only) GitLab server: https://git.aps.anl.gov/jemian/aps_bss_user_info and this version of dm has access to the schedule system. That interface is not available in the current dm version, waiting on the IS group to provide access to all beam lines (currently limited to only one specific).

[prjemian]

This can be a remote shell execution command from the MEDM/caQtDM GUI. Does not need more code. The command would call apsbss update PREFIX