Review and test template

[z-y-huang]

Hi all,

The goal of this issue is for all assignees to:

1. Read through the RA manual
2. Check out a clean copy of template
3. Following the RA manual and setup instructions in README.md, build the repository from start to finish
4. Test template and attempt to break it; report any bugs or errors
5. Provide feedback and identify room for improvement

Note that the current example scripts are pretty lacking and one of the items on our todos is to flesh them out to be more comprehensive. Let me know if you have any ideas here!

[z-y-huang]

Notes:

• Per discussions with @DavidRitzwoller, think through correct merge strategy for .log files in .gitattributes. Currently, merge strategy is set to binary (i.e., keep exact version in work tree, but leave file as conflicted). Alternatively, can set merge strategy to ours (i.e. resolve all conflicts by keeping exact version in work tree), but will require all clean checkouts to type git config merge.ours.driver true.

[DavidRitzwoller]

Notes:

• @z-y-huang. There's a file that throws a git-lfs error when we clone in to the repo.
  
• Update the comment in the config_user_template to name the file config_user.yaml.
• Initial set-up errors
  • 1. After executing python check_setup.py
       
  • 2. After adding future to requirements.txt and executing python check_setup.py
       
  • 3. We set my local version of tornado==4.5 to be compatible with my local version of python (2.7.3) and got
       

[z-y-huang]

@DavidRitzwoller:

Addressed all the comments here except for (3)(c), where I'm not sure if there's a good non-local solution.

[DavidRitzwoller]

Small note: Let's add Rplots.pdf to the .gitignore.

[DavidRitzwoller]

Summary of conversation with @z-y-huang:

We wanted to clarify the protocol for the use of output_local, and how this differs from the build directory in the GSlab template.

Suppose we have a project with two active issues and two modules, data and analysis. I'm assigned to one issue and Zong is assigned to the other. In my issue, I change something about how we build our derived data. Zong is working on a set of plots that take the derived data as an input. The derived data are large enough that we do not want to commit. So, our build script should output to output_local, which should then be uploaded to external storage, and our analysis scripts should call the externally stored data.

But I don't want my work editing how we derive our data to interfere with Zong's plots, so until my work has been peer-reviewed, I shouldn't upload my output_local to the external storage. In a sense, we want the derived data stored externally to be a "master" branch. This suggests the following addendum to our Issue/ Pull Request work flow.

1. Open issue branch
2. Execute the requirements for a given issue
3. Send to PR
4. When PR is cleared, upload and relevant output_local to the external stroage
5. Merge to master

Should an issue require work that spans both the analysis and data modules, the input path for the derived data in analysis should be changed to the output_local directory in data during development. Prior to merging to master, the derived data in the output_local directory in data should be uploaded to external storage, the input path should be reverted back to point to the external storage, and the analysis module should be rerun with the updated path.

[gentzkow]

Thanks @DavidRitzwoller. I like this workflow.

What I had been imagining was a different solution: everything on the external storage is manually versioned. So if there is an external directory called /reponame/analysis/ we don't store files directly there but rather store them in /reponame/analysis/1/, /reponame/analysis/2/, etc. Downstream modules call the versioned directories so if I add /reponame/analysis/3/ this does not break any existing code people might be working on. This has the additional advantage of making it more likely results are replicable in the future; it has the cost of chewing up more storage space.

In any case, I think it's a good idea to use something like your protocol to make sure data only gets "released" to the external drive once an issue is complete. We might allow some flexibility for cases where we want to release a version mid-issue, noting that this should only be when there is a good reason to do so.

[mengsongouyang]

Hi @z-y-huang,

Here are my notes.

• In practice, most projects are very big. I would imagine users want to have subdirectories under each module. For example, have /descriptive/ and /estimation/ under /analysis/. This is not difficult to implement. Users only need to create the subdirectories with similar structure and correctly specify the paths. Not sure if it would be better to add some sentences about this in the RA Manual.

• In the Workflow page, I would prefer to put Merge after Peer Review, because merge is the step after peer review.

• In the Repository Structure page, I would prefer to put Modules in front of Standard Modules, because Standard Modules are the higher-level structure. Also, I think it would be better to rename the titles of these two sections so that people can distinguish between them.

• For the order of files/dirs in the Standard Module Structure, I would prefer to put externals.txt, inputs.txt, make.py to the front because inputs.txt and make.py are files that need to edit for every module.

• I think it would be better to be consistent in using plural or singular form for input, external, etc. Now, we use /input/ and /external/ for folders, inputs.txt and externals.txt for .txt files.

• A typo in line 15 here:

[DavidRitzwoller]

no progress in past two weeks

[z-y-huang]

@mengsongouyang:

I addressed all of your comments. Thanks!

[DavidRitzwoller]

@z-y-huang. It seems like this thread is going quiet. Do you have any advice for how @mengsongouyang and I should proceed prior to wrapping this up? Would developing some example scripts be a useful contribution?

[z-y-huang]

@DavidRitzwoller, @mengsongouyang:

It would be great if you two could try to break the template as much as possible to see if there are any bugs/errors I need to fix.

[z-y-huang]

Changes implemented:

• Added run_all.py so repository can be built with a single script
• Migrated code in make.py scripts to gslab_make library to reduce clutter
• Cleaned up traceback for Python 3
• Added more description to error messages

[z-y-huang]

Changes implemented:

• Added run_module function to clean up run_all.py script
• Added copy_file to facilitate exporting output locals

[DavidRitzwoller]

no progress in past 2 weeks

[DavidRitzwoller]

no progress in past 2 weeks

[DavidRitzwoller]

no progress in past two weeks

[DavidRitzwoller]

no progress in past two weeks

[DavidRitzwoller]

no progress in past two weeks

[gentzkow]

Another infinite loop. You should just close this one if you don't have time to do the testing.

…

[DavidRitzwoller]

Thanks @gentzkow. @mengsongouyang and I completed initial rounds of testing this summer. @z-y-huang is there anything in particular you would like us to look at?

[z-y-huang]

@DavidRitzwoller:

I'll close this for now--currently implementing some changes, but may recruit you all again once those are finished.

Summary:
First round of review by GSLab RAs. Incorporated feedback regarding repository structure and workflow.