Time path varying parameters

[jdebacker]

This PR updates how several policy parameters and TFP are handled to allow them to vary over the time path.

[jdebacker]

@hdoupe I'm having trouble getting tests to work out and it is related to me missing something about how the parameters class is working - maybe you can help.

What I'm trying to do with this PR is to allow some parameters to be specified such that they vary over the time path. These parameters are:

tp_param_list = ['alpha_G', 'alpha_T', 'Z', 'world_int_rate',
                         'delta_tau_annual', 'tau_b', 'tau_bq',
                         'tau_payroll', 'h_wealth', 'm_wealth',
                         'p_wealth', 'retirement_age']

What I've done is to specify these parameters as single element lists in default_parameters.json and then, in lines 119-217 of parameters.py these lists are extrapolated over the time path (so for p.T + p.S number of periods) by taking the list of values for each parameters and then repeating the last value specified for the remainder of the time path that is not specified in the list. So, for example, if a user updates the parameter alpha_G = [0.5, 0.6, 0.65] then the value of alpha_G in the first two periods is 0.5 and 0.6 respectively, and the value is 0.65 for all periods after the second. If alpha_G = [0.5, 0.6, 0.65, 0.7], then alpha_G is 0.5, 0.6, 0.65 in the first , second, and third periods respectively, and the value is 0.7 for all periods after the third. In this way the user only has to enter the parameter values for the periods from the first up until the parameter stops changing (e.g., at the end of the budget window).

I can get the model to solve with this, but when I try to run some unit tests, I'm finding a couple issues that I'm unsure how to handle:

1. When I change T or S from the default values, I need to update all the parameters that vary over the time path (even though the defaults are single element lists so could be consistent with any feasible T and S). While I can get tests to run by specifying these parameters again before calling parameters.update_specifications(), it's obnoxious to have to enter default values again and could lead to potential errors if they are misspecified. You can see how this is done on lines 15-27 of test_tax.py.
2. There appears to be some issue when trying to update the retirement_age parameter, even though I don't see how it is any different than the other parameters that vary over the time path. This is raising an exception and I don't know how to get it to pass.

You can how the second issue is arising if you work from this branch and then run py.test test_tax.py from \OG-USA\ogusa\tests. The difference in the shape of the retirement_age parameter between the default and updated parameters is shown by a print command that displays:

Parameter = retirement_age shape = (1, 1) ()

I've been unable to trace why the retirement_age parameter is acting any different than others, which have the same dimensions both in default_parameters.json and in the updates specified in lines 15-27 of test_tax.py.

Any help with issues (1) and (2) -- or other thoughts on how best to specify time-path-varying parameters, would be very much appreciated!

[jdebacker]

I figured out the issue with the retirement_age parameter (issue 2 in the above comment). It had to do with the ranges depending upon other parameter values that were integers (not lists):

"range": {"min": "starting_age", "max": "ending_age"},

If I set this to:

"range": {"min": 20, "max": 100},

Then things work as expected when updating the retirement_age parameter. Any way to keep the dependence of the range on other parameter values without significantly modifying the functions?

[jdebacker]

Found a quick fix to (1) above by slicing the "old" arrays of the time path parameters if there is new T and S that sum to a number less than the default values. This is not ideal, but will be serviceable for now.

[jdebacker]

Model with time path varying parameters can reproduce old results. Need to add a few new tests for cases where parameters vary over the time path.

[hdoupe]

Hey Jason, I'm trying to get up to speed on the problems that you ran into in PR #433. I think I figured out what's going on with the "starting_age" and "ending_age" problem. It looks like the validation value isn't set to the right shape. When the min and max values are set to hard-coded values instead of the value of another variable (i.e. "min": 20 vs "min": "starting_age"), the validation value is set to the proper numpy arrary shape with this line of code:

validation_value = np.full(param_value.shape,
                            validation_value)

However, if the min value points to another variable, then the simple_eval method is called and that line of code isn't hit:

lines 574-578 of parameters.py:

if isinstance(validation_value, six.string_types):
    validation_value = self.simple_eval(validation_value)
else:
    validation_value = np.full(param_value.shape,
                                validation_value)

I removed the else clause so that those lines are:

if isinstance(validation_value, six.string_types):
    validation_value = self.simple_eval(validation_value)

validation_value = np.full(param_value.shape, validation_value)

I reverted retirement_age validation back to using the beginning_age and ending_age values and tested the fix with this script:

from ogusa.parameters import Specifications
  
p = Specifications()

p.update_specifications({"retirement_age": [70] * p.retirement_age.shape[0]})


p = Specifications()

p.update_specifications({"retirement_age": [int(p.starting_age - 1)] * p.retirement_age.shape[0]})

and got this output:

(ogusa-dev) HDoupe-MacBook-Pro:OG-USA henrydoupe$ python test.py 
retirement_age  has been updated to:  [70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70]
retirement_age  has been updated to:  [19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19]
Traceback (most recent call last):
  File "test.py", line 10, in <module>
    p.update_specifications({"retirement_age": [int(p.starting_age - 1)] * p.retirement_age.shape[0]})
  File "/Users/henrydoupe/Documents/OG-USA/ogusa/parameters.py", line 439, in update_specifications
    raise ValueError('\n' + self.parameter_errors)
ValueError: 
ERROR: retirement_age value 19 < min value 20
ERROR: retirement_age value 19 < min value 20
ERROR: retirement_age value 19 < min value 20
ERROR: retirement_age value 19 < min value 20
ERROR: retirement_age value 19 < min value 20
ERROR: retirement_age value 19 < min value 20
ERROR: retirement_age value 19 < min value 20
ERROR: retirement_age value 19 < min value 20
ERROR: retirement_age value 19 < min value 20
ERROR: retirement_age value 19 < min value 20
ERROR: retirement_age value 19 < min value 20

# goes on for a bit more with the same message

Does this fix resolve the issue that you were having around the min/max validation for the retirement age variable?

[jdebacker]

@hdoupe -- thanks for this. Works great and I'd have not seen this as the fix very easily.

[jdebacker]

@rickecon This is passing all tests. Let me know if the PR looks good to you.