Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding examples runs to the travis scripts #602

Closed
sbenthall opened this issue Mar 30, 2020 · 9 comments
Closed

Adding examples runs to the travis scripts #602

sbenthall opened this issue Mar 30, 2020 · 9 comments

Comments

@sbenthall
Copy link
Contributor

It looks like several of the examples currently run with errors on master.

These need to be fixed.
Once they are fixed, it would make sense to add the line to run then all (maybe just the notebooks) as part of travis.

@sbenthall
Copy link
Contributor Author

The currently pytest report I get on the HARK/examples/ notebooks

https://gist.github.com/sbenthall/cf7e68ccd19b3989efa38a07f00cddf7

@MridulS
Copy link
Member

MridulS commented Mar 30, 2020

Once they are fixed, it would make sense to add the line to run then all (maybe just the notebooks) as part of travis.

How much time did it take the run the suite?

@sbenthall
Copy link
Contributor Author

Quite some time. I didn't time it.

The BayerLuetticke/notebooks/DCT-Copula-Illustration.ipynb fails because of a use of input() which works fine in a notebook; just breaks when operated via pytest.

So we'll need to move it to REMARK to clear it out if we want to go down this road. That's ticketed at #531

@sbenthall
Copy link
Contributor Author

Regarding how long it takes to run the notebook, my view is:

  • Time spent running notebooks on Travis is cheaper than time spent running notebooks on a developer's laptop
  • We can adjust the notebooks to reduce their runtime if needed.
    • By either setting different parameters
    • Or improving HARK's performance in other ways

@sbenthall
Copy link
Contributor Author

The notebook Gentle-Intro/Gentle-Intro-To-HARK.ipynb fails when executed because of one cell where it is intended that the user manually changes the notebook to assign a value to mPlotBottom before a cell is executed.

The presumption that these notebooks are interactive rubs up against the idea that these notebooks will be turned into static documentation rendered here:
https://hark.readthedocs.io/en/latest/example_notebooks/Gentle-Intro-To-HARK.html

Indeed the later cells in this notebook are not being executed and rendered at the moment.

This makes these notebooks less effective as documentation.

I'll remove the presumed interactive and fully render the notebook in a PR.

@llorracc
Copy link
Collaborator

@sbenthall

For my teaching purposes, I found it useful to split the "Gentle Intro" into two parts, one about the Perfect Foresight model and a second about the model with transitory and permanent income shocks. (The two notebooks are still in DemARK).

  1. The Perfect Foresight model is a nice starting point to teach students about HARK because it's a clean model with an analytical solution, so basically students are confident that they understand the math and therefore can concentrate on how that math is instantiated in HARK.
    • Since it is mostly about HARK and how to use it, this notebook would be appropriate for the examples directory
  2. In my sequence of teaching this stuff, I have several more lectures before I get to the point where I need to introduce the ConsIndShock type which is the first model they have to solve completely numerically. So that notebook aims to teach about both the economics and about HARK.
    • It could also go in examples, and maybe I could aspire to make it more oriented to teaching HARK and assume that the user knows the economics, which could be taught in some other, perhaps not so gentle, notebook.

@mnwhite was the primary author of the Gentle Introduction and he might want to leave it as-is for his teaching purposes (when we taught a class together, he presented the whole thing as a standalone set-piece). But as-is it can't go in examples for the reason you note: It doesn't run unattended. Maybe we could move it to the TITLARK (Teaching and Instructional Tools for Learning the ARK) -- though my suspicion is that Matt hates the TITLARK and therefore would not like his Gentle Intro to live there.

@sbenthall
Copy link
Contributor Author

I don't presume anything regarding the pedagogical materials (DemARKs, QuARKs, TITLARKs, etc.) ...

except that since they seem to be mainly about curriculum design, which is a complex and multifaceted problem with a lot of complexity, that they be decoupled from the development of HARK core so that the latter can adhere more closely to best practices in software engineering.

@llorracc
Copy link
Collaborator

except that since they seem to be mainly about curriculum design, which is a complex and multifaceted problem with a lot of complexity, that they be decoupled from the development of HARK core so that the latter can adhere more closely to best practices in software engineering.

That's a good dividing line. If making something comply is too much work, then the content should not be part of Econ/ARK, but can be private and USE HARK (like anyone else).

@sbenthall
Copy link
Contributor Author

Somebody (@MridulS ?) has implemented this as GitHub Actions. Closing ticket.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants