The instructions below simulate the DESI survey as a sequence of tile exposures and record metadata for each exposure and accumulate some summary statistics. Simulations are stochastic since the scheduling algorithms respond to randomly generated weather conditions.
These instructions do not perform fiber assignment or simulate any spectra, but instead provide the necessary scheduling inputs for these tasks.
Please create an issue with any corrections or suggestions for improvement to this tutorial.
Login to cori, then:
source /project/projectdirs/desi/software/desi_environment.sh 18.12 mkdir -p $SCRATCH/desi/output export DESISURVEY_OUTPUT=$SCRATCH/desi/output surveyinit surveysim
The results are then saved as stats_surveysim.fits
and exposures_surveysim.fits in $DESISURVEY_OUTPUT.
For a tutorial on interpreting
these outputs start here.
For more details and variations on these steps, read on.
If this is your first exposure to DESI software, start here. We use git for source control and you will need to install the base DESI packages on your laptop or else work at NERSC.
The following DESI packages must be installed to run this tutorial:
In addition, the following non-DESI packages must be installed via pip since they are not included with the anaconda distribution:
- fitsio
- speclite
- ephem
- healpy
Note that these packages are already included in the custom DESI anaconda distribution installed at NERSC, so only need to be installed when running on your laptop or if you need to use non-default versions.
Setup the standard DESI conda environment using, for example:
source /project/projectdirs/desi/software/desi_environment.sh 18.12
Replace 18.12 with master for the latest and greatest (which might not work),
or leave it out for the current default.
If you want to use new features of desisurvey or surveysim that are
not yet included in a numbered DESI conda environment, swap them in using:
module swap desisurvey/master module swap surveysim/master
For even more bleeding-edge features that are only available on a development branch, use, for example:
module unload desisurvey pip install --user git+https://github.com/desihub/desisurvey@refactor export PATH=$HOME/.local/bin:$PATH
where refactor is the branch name in this example.
The following instructions assume that you have installed the anaconda scientific python distribution and will create a new python environment for your DESI work. Start from the directory you wish to install software into, then:
conda create --name desi pip ipython jupyter numpy scipy astropy pyyaml requests h5py scikit-learn matplotlib basemap
source activate desi
pip install fitsio speclite ephem healpy
for package in specsim desiutil desimodel desisurvey surveysim; do
pip install git+https://github.com/desihub/$package
done
export DESIMODEL=$PWD/desimodel
install_desimodel_data -d $DESIMODEL
Notes for experts:
- The instructions above assume that you are using the bash shell, and need to be modified slightly for (t)csh.
- The matplotlib and basemap packages are not required to follow the instructions below but are useful for plotting the outputs.
Create an output directory to hold all survey planning and simulation outputs and create an environment variable pointing to it.
Ensure that your $DESIMODEL environment variable points to a valid data directory:
ls $DESIMODEL/data/weather
Also check that the relevant command-line scripts are in your path:
surveyinit --help surveysim --help
Note that all output from these commands goes into $DESISURVEY_OUTPUT so they
can be run from any directory and will not write anything to your current working
directory.
Save the output to the $SCRATCH volume, for example:
mkdir -p $SCRATCH/desi/output export DESISURVEY_OUTPUT=$SCRATCH/desi/output
Note that we use $SCRATCH for faster I/O but files are periodically
removed. See NERSC best practices
for details.
Enter the parent directory where you will save outputs, then:
mkdir output export DESISURVEY_OUTPUT=$PWD/output
If you followed the installation recipe above then make sure you have activated your
desi environment with:
conda activate desi
(Older versions of conda might require source activate desi instead.)
Parameters for planning and scheduling the DESI survey are stored in a
configuration file
which is well commented and provides a good overview of the assumptions being made.
You do not normally need to change these parameters, but are welcome to experiment by copying and editing
this file then passing your custom version to the surveyinit and surveysim scripts described below
using their config-file option.
Before starting the survey, we precompute some tabulated planning data using:
surveyinit --verbose
This step takes about 50 minutes (on cori) and writes the following files into $DESISURVEY_OUTPUT:
ephem_2019-01-01_2025-12-31.fits: tabulated ephemerides during 2019-25.surveyinit.fits: estimated average weather and optimized initial hour angle (HA) assignments for each tile.
These files take some time to generate, but are cached and not regenerated
after the first time you run this command. If you want to force these files
to be recalculated, add the --recalc option. To avoid generating these
files yourself, you can also copy them into your $DESISURVEY_OUTPUT from
this NERSC directory:
$DESI_ROOT/datachallenge/surveysim2018/shared/
To ensure they have been copied correctly, you should still run surveyinit --verbose,
which should now exit immediately.
To simulate the nomimal 5-year survey, use:
surveysim
This should complete in about 2 minutes (on cori) and writes two FITS files to
$DESISURVEY_OUTPUT:
stats.fits: tables of per-tile and per-night summary statistics.exposures.fits: table listing all simulated exposures in time order.
For a tutorial on interpreting these outputs start here.
By default, simulations are run entirely in memory for speed. However, during
operations the internal states of the afternoon planner and tile scheduler are
written to disk daily and then restored the next day. Use the --save-restore
option to surveysim to run in this mode, and write daily files:
planner_YYYY-MM-DD.fits: internal state of the planner after afternoon planning for YYYY-MM-DD.scheduler_YYYY-MM-DD.fits: internal state of the tile scheduler after observing on the night of YYYY-MM-DD.
This mode gives identical results but is slower (about 3 minutes) and writes many files (about 3.6K files totalling amost 1Gb), so is mainly intended as a technical check of this mode and for developing tools that read these intermediate files.
There are many options you can experiment with to simulate a different survey weather, strategy, or schedule, for example. For a full list, refer to:
surveyinit --help surveysim --help
You can also vary parameters in the survey configuration file.
In order to keep the outputs from different runs separate, use a separate output
directory each time a change to the surveyinit outputs is required.
For example, when changing the tiles file or nominal survey start/stop dates.
To run with a different output directory you can either update
$DESISURVEY_OUTPUT or else use the --output-path option
of surveyinit and surveysim.
For different runs with the same surveyinit outputs, use the --name
and --comment options to surveysim to distinguish each run.
For example:
surveysim --name twilight --comment 'Include twilight in schedule' --twilight
Will run with twilight included in the schedule and save
stats_twilight.fits and exposures_twilight.fits to $DESISURVEY_OUTPUT.
To study how survey progress depends on the random weather realization (including seeing and transparency), change the default seed (1), for example:
surveysim --name weather1 --comment 'Random weather realization #1' --seed 1 surveysim --name weather2 --comment 'Random weather realization #2' --seed 2 surveysim --name weather3 --comment 'Random weather realization #3' --seed 3
To simulate with an estimate of the worst-case weather, replay the historical dome-open fractions from 2015 during each year of the simulation with:
surveysim --name worstcase --comment 'Worst-case dome-open fractions' --replay Y2015
Instead of running surveysim, you can incorporate and customize the following
top-level simulation driver directly into your own script or jupyter notebook:
import datetime
import desisurvey.config
import desisurvey.rules
import desisurvey.plan
import desisurvey.scheduler
import surveysim.weather
import surveysim.stats
import surveysim.exposures
import surveysim.nightops
def simulate_survey(rules, weather, use_twilight=False):
# Simulate the nominal survey dates.
config = desisurvey.config.Configuration()
start, stop = config.first_day(), config.last_day()
num_nights = (stop - start).days
# Initialize simulation progress tracking.
stats = surveysim.stats.SurveyStatistics()
explist = surveysim.exposures.ExposureList()
# Initialize afternoon planning.
planner = desisurvey.plan.Planner(rules, simulate=True)
# Initialize next tile selection.
scheduler = desisurvey.scheduler.Scheduler()
# Loop over nights.
num_simulated = 0
for num_simulated in range(num_nights):
night = start + datetime.timedelta(num_simulated)
# Perform afternoon planning.
explist.update_tiles(night, *planner.afternoon_plan(night))
if not desisurvey.utils.is_monsoon(night) and not scheduler.ephem.is_full_moon(night):
# Simulate one night of observing.
surveysim.nightops.simulate_night(
night, scheduler, stats, explist, weather=weather, use_twilight=use_twilight)
planner.set_donefrac(scheduler.tiles.tileID, scheduler.snr2frac)
if scheduler.plan.survey_completed():
break
return stats, explist
To run a simulation, define the survey strategy rules, e.g.:
rules = desisurvey.rules.Rules('rules-depth.yaml')
and the random weather realization to use, e.g.:
weather = surveysim.weather.Weather(seed=1, replay='random')
then call the function defined above:
stats, exposures = simulate_survey(rules, weather)
The surveymovie script reads simulation outputs and generates a movie with one
frame per exposure to visualize the scheduler algorithm and survey progress:
surveymovie --verbose
An example is available. A key describing the information displayed in each frame is here. To generate a PNG of a single frame, use:
surveymovie --expid 123 --save exposure123
to create exposure123.png.
To generate a smaller summary movie with one frame per night, use the --nightly option, e.g.:
surveymovie --nightly --save summary
The surveymovie script uses the external ffmpeg program to generate movies, so
this must be installed. At NERSC, use:
module add ffmpeg