Skeleton Twinkles 1 Requirements Document

[connolly]

Define and document SSim, CS and DM requirements to generate and process Twinkles data

[drphilmarshall]

@wmwv and I decided to do this in latex. I will start a skeleton, using the SRM style, and import the run 1 plan content from #11 before closing that out. Renaming this issue and deadlining for Thursday's reading group ( #54 ).

[drphilmarshall]

Hi @DarkEnergyScienceCollaboration/twinkles - I'll be discussing the Twinkles 1 RQ skeleton I made yesterday with @cwwalter and others at the SSim meeting this morning, 1000 EDT. Here's the meeting page on confluence if you want to join. Don't feel you have to though - you can also reply with comments and I'll take them to the meeting with me.

[drphilmarshall]

Oops, that should have said 1000 PDT for the SSim meeting. Starting on BlueJeans in 8 mins! Details on the confluence meeting page.

[SimonKrughoff]

If it is helpful, here is a link to the documentation on git-submodule: https://git-scm.com/docs/git-submodule

[rbiswas4]

I would also second submodule. When things are ok, at least on smaller repos I have used it n, it is quite smooth and you don't notice at all. I have been told that it creates problems for large repos (which is apparently the reason lsst does not use it?), and I have wondered if git-python scripts might be an option in that regime..

[drphilmarshall]

Some notes from SSim meeting:

• Validation checks need to be laid out in RQ docs. In sections, or at end? Try in sections and see.
• If we want to make a requirements table, this could mean writing a "requirement" macro. See how we go.
• Where should Twinkles 1 RQs live? In the Twinkles repo or a more general DC repo? SSim group need to collate documents to manage all of DC1 efficiently: it might make more sense to keep material there (where more people can follow it), and just link to it from the Twinkles repo. Alternatively, a DC repo could contain submodules, and pull in latex files from them via \input{}. Action: Phil to investigate and experiment
• Twinkles RQ/design doc is different than SSim DC1 overview doc. We could enable both to be made from the same input files, if the section files were written in the right way. For example: twinkles1/catsim.tex would just contain a terse list of requirements, and be \input{} from both twinkles1/simulations.tex and also dc1-catsim.tex.

[drphilmarshall]

Thanks @SimonKrughoff and @rbiswas4 . I'll try setting something up and we'll see how @cwwalter likes it. This does seem to fit with the feeling we had towards the end of the meeting that the SSim and Twinkles docs might be different enough that continuing to keep the Twinkles doc in the Twinkles repo could still make sense, provided it was easy for the SSim doc to pull in Twinkles content.

[sethdigel]

Here are a few moderately high level comments and questions on the skeleton.

It might be worth thinking of the document as defining specifications rather than requirements, but that could be a distinction without a difference. At the project level, requirements documents are used more formally than I think is the intent here.

In the skeleton document the analysis issues defined in Sec. 2.1 and 2.2 Supernovae and Strong Lensing look very similar, not surprisingly. Do either of them have unique analysis issues? I think it is safe to assume that, e.g., photometric calibration and photometric error accuracy will be in common across the science areas.

Also, I'd guess that the point of mentioning the analysis issues/challenges in the requirements/specification document would be to inform how the simulations are generated, but for some of them it is hard to see the connection. For example, it would be possible to design the simulations to be useful to study host galaxy light contamination effects, but would anything special need to be done on the simulation side for studying the impacts of inaccuracy in the photometric calibration?

My vague concept of Twinkles 1 was that it would be a 10-year DDF-cadence simulation of some small patch (100 sq arcmin) of the sky. For Strong Lensing the skeleton says that the sampling should be WFD, with several pointings. And no region size is specified for either. So I'm not so sure that I understand the scope of Twinkles 1.

Sec. 5 (Analysis Requirements) lists some 'Level 4' algorithms, and no Level 3. I had not seen Level 4 mentioned before - I had though Level 3 was the highest.

[drphilmarshall]

My vague concept of Twinkles 1 was that it would be a 10-year DDF-cadence simulation of some small patch (100 sq arcmin) of the sky. For Strong Lensing the skeleton says that the sampling should be WFD, with several pointings. And no region size is specified for either. So I'm not so sure that I understand the scope of Twinkles 1.

I'd say the scope of Twinkles 1 has only been approximately specified: writing this document should finish the job. The problem is Twinkles 1's single filter nature: in SL we are only really interested in the WFD observations, because the DD fields are so small. If we are going to test time delay estimation in a meaningful way, the night to night cadence needs to be 1) a few days and 2) similar to what the multi-filter WFD will achieve. If a single filter DDF cadence achieves that then we're good: but it seemed to me that we might need to interleave several WFD time samplings (not pointings) to get the desired cadence. This might be more trouble that its worth though: happy to change the text back to the SRM plan.

Sec. 5 (Analysis Requirements) lists some 'Level 4' algorithms, and no Level 3. I had not seen Level 4 mentioned before - I had though Level 3 was the highest.

It is: Level 4 is new terminology. Do you like it?

[sethdigel]

It is: Level 4 is new terminology. Do you like it?

No, although it reminds me of the amplifiers in This is Spinal Tap. Also, the algorithms look like what I thought was Level 3.

[drphilmarshall]

My understanding is that Level 3 code is to be written in the DM framework, so that it could (one day) be absorbed and run during Level 2 processing. I expect we will have code that is not written in the DM framework and/or would not be wanted to be run as part of the Level 2 processing - that's what I'd been thinking of as Level 4. We can't just call it "DESC Analysis Software" because some DESC analysis software will be Level 3.

[drphilmarshall]

I have had a go at implementing the vision (still blurry, but hey) that we came up with this morning, over in a new experimental SSim DataChallenges repository. I've left the Twinkles requirement document in here though for now, while we see whether these submodules are going to work out. I also renamed our Twinkles document to TwinklesRQ.tex and entitled it "Twinkles: Design and Specifications" to distinguish it from the terser, differently focused, all-of-DC1 thing that the SSim group has tasked itself with writing. I'll add people to the SSim Team by hand so you can see (and write to) that repo - let me know if you can't see it and want to.

[rbiswas4]

On the requirements document:
Quick comment on Section 4.2:
CATSIM /SN : SNCosmo can also be used to simulate core collapse as well based on either the Nugent templates, or Nugent templates modified by observed light curves. There are simply a lot more unknowns for which one has to make arbitrary choices.

[drphilmarshall]

Thanks @rbiswas4 ! Great that you are considering a pull request :-)

[drphilmarshall]

Good - skeleton seems acceptable, or at least better addressed with new issues. Closing this one so we can focus on the RQ doc content. Thanks all!

[cwwalter]

For followup discussion on how the structure relates to the SSim RQ Doc please see:

DarkEnergyScienceCollaboration/DataChallenges/#1

Thanks!

-Chris