Fix the Bisection Issue in five_preferences.md

[HumphreyYang]

This PR resolves #147.

[HumphreyYang]

Hi @mmcky,

This PR should fix the issue. It was quite hard to find : )

[mmcky]

Nice investigative work @HumphreyYang

So is the new SciPy more sensitive to initial values?

Before I merge I would just like to understand what the issue was. Thanks @HumphreyYang

[HumphreyYang]

Hi @mmcky,

Thanks for asking. I am very happy to go though my long journey last night.

Essentially, I looked over all the changes in scipy from 1.10.1 (under Anaconda 2023.03) to 1.11.1 (under Anaconda 2023.07).

The root cause of the error lies in the convergence of the line

res = optimize.root_scalar(problem, args=(c_1, u_bar), bracket=bracket, method=method)

in function solve_root_problem. The function is called in:

c_2_grid_cons = solve_root_problem(constraint_pref_root_problem, cons_crit_bar, c_1_grid, method='bisect', bracket=[0.3, 4.4], verbose=True) 

The differences in the result of c_2_grid_cons causes the non-convergence of

fp_cons = optimize.fixed_point(func_approx, x0, args=([c_2_grid_cons]))

under scipy==1.11.1, c_2_grid_cons is

[       nan        nan        nan        nan        nan        nan
        nan        nan        nan 4.27158534 3.9723196  3.70486456
 3.46477285 3.24835709 3.0525381  2.87472741 2.71273568 2.56470044
 2.4290288  2.30435159 2.18948637 2.08340748 1.98522151 1.89414719
 1.8094988  1.73067237 1.65713414 1.58841091 1.52408195        nan
 1.46562282 1.44679935 1.42862095 1.41105227 1.39406062 1.37761566
 1.36168924 1.34625517 1.33128907 1.31676821 1.30267134 1.28897861
 1.27567144 1.2627324  1.25014515 1.23789433 1.22596549 1.21434504
 1.20302016 1.19197878 1.18120948 1.17070147 1.16044457 1.15042911
 1.14064595 1.13108641 1.12174227 1.1126057  1.10366929 1.09492597
 1.08636901 1.07799201 1.06978888 1.0617538  1.0538812  1.04616581
        nan        nan        nan        nan        nan        nan
        nan        nan        nan        nan        nan        nan
        nan        nan        nan        nan        nan        nan
        nan        nan        nan        nan        nan        nan
        nan        nan        nan        nan        nan        nan
        nan        nan        nan        nan        nan]

but under scipy==1.10.1, c_2_grid_cons is

[       nan        nan        nan        nan        nan        nan
        nan        nan        nan 4.27158534 3.9723196  3.70486456
 3.46477285 3.24835709 3.0525381  2.87472741 2.71273568 2.56470044
 2.4290288  2.30435159 2.18948637 2.08340748 1.98522151 1.89414719
 1.8094988  1.73067237 1.65713414 1.58841091 1.52408195 1.48512961
 1.46562282 1.44679935 1.42862095 1.41105227 1.39406062 1.37761566
 1.36168924 1.34625517 1.33128907 1.31676821 1.30267134 1.28897861
 1.27567144 1.2627324  1.25014515 1.23789433 1.22596549 1.21434504
 1.20302016 1.19197878 1.18120948 1.17070147 1.16044457 1.15042911
 1.14064595 1.13108641 1.12174227 1.1126057  1.10366929 1.09492597
 1.08636901 1.07799201 1.06978888 1.0617538  1.0538812  1.04616581
 1.03860254 1.03118657 1.02391326 1.01677819 1.00977711 1.00290597
 0.99616087 0.98953807 0.98303401 0.97664523 0.97036844 0.96420046
 0.95813826 0.95217889 0.94631955 0.94055751 0.93489016 0.929315
 0.92382958 0.91843159 0.91311876 0.90788892 0.90273998 0.89766992
 0.89267677 0.88775864 0.88291373 0.87814026 0.87343652 0.86880087
 0.86423172 0.85972751 0.85528676 0.85090802 0.84658989]

Therefore, I suspect that there is a change in the way scipy handles the input.

Indeed, in scipy==1.11.0, the PR, MAINT: optimize.root_scalar: raise when NaN is encountered starts to raise the issue when the input of a bracket causes a function to generate NaN. There are some related fixes around this later, but they jointly result in the nan values shown above. Therefore, the fix is to try different bracket values so that this does not happen (which is the fix proposed in this PR)

Below is the convergence at different points under different scipy versions reinforcing my guess:

Under scipy==1.10.1 the convergence is not interrupted

      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 4.271585336476663
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 3.97231959568694
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 3.7048645614540416
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 3.464772847601898
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 3.2483570907065316
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 3.05253809507044
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 2.8747274126790674
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 2.712735681848017
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 2.5647004392249526
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 2.4290287988159895
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 2.3043515857465087
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 2.189486371380235
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 2.0834074812582464
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.9852215062947387
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.8941471881883907
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.809498804831901
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.7306723738255188
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.6571341385831604
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.5884109136896314
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.524081952715869
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.4851296088117585
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.4656228204996753
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.446799349661933
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.4286209476259046
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.4110522737945304
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.3940606186466082
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.3776156582428032
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.3616892360423207
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.3462551685996464
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.3312890720575068
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.3167682068743032
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.3026713385747373
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.288978612636811
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.275671441773784
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.2627324042761305
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.2501451521046425
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.2378943276719387
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.225965488331667
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.2143450377997853
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.2030201636987388
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.1919787806614752
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.1812094783837443
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.1707014740951764
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.1604445690986271
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.1504291088780978
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.1406459464955785
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.1310864089598451
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.121742266235333
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.112605702712108
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.103669290883363
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.0949259670141602
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.08636900866718
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.0779920138729273
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.0697888818474446
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.061753795097184
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.053881202806633
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.0461658053930933
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.038602540142847
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.0311865678243068
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.0239132602333938
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.016778188555554
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.0097771124996597
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 1.0029059701627834
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.9961608685251576
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.9895380745939748
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.9830340070543117
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.976645228494317
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.970368438074138
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.9642004646758779
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.9581382604339069
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.9521788946902691
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.9463195482670472
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.9405575081190819
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.9348901622700851
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.9293149950358837
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.9238295825310615
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.9184315884366243
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.9131187599876741
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.9078889241810886
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.9027399842069371
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.897669916058885
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.8926767653347724
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.887758644201267
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.882913728515132
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.8781402551122938
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.8734365192050518
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.8688008719582739
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.8642317181289854
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.8597275138924032
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.8552867647089897
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.8509080233518491
      converged: True
           flag: 'converged'
 function_calls: 43
     iterations: 41
           root: 0.8465898879900577

Under scipy==1.11.1 the convergence is interrupted as it encounters nan values (as the iteration and function_calls are all 1:

      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 4.271585336476663
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 3.97231959568694
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 3.7048645614540416
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 3.464772847601898
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 3.2483570907065316
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 3.05253809507044
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 2.8747274126790674
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 2.712735681848017
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 2.5647004392249526
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 2.4290287988159895
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 2.3043515857465087
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 2.189486371380235
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 2.0834074812582464
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.9852215062947387
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.8941471881883907
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.809498804831901
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.7306723738255188
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.6571341385831604
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.5884109136896314
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.524081952715869
      converged: False
           flag: The function value at x=1.5171875000000001 is NaN; solver cannot continue.
 function_calls: 8
     iterations: nan
           root: 1.5171875000000001
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.4656228204996753
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.446799349661933
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.4286209476259046
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.4110522737945304
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.3940606186466082
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.3776156582428032
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.3616892360423207
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.3462551685996464
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.3312890720575068
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.3167682068743032
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.3026713385747373
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.288978612636811
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.275671441773784
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.2627324042761305
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.2501451521046425
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.2378943276719387
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.225965488331667
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.2143450377997853
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.2030201636987388
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.1919787806614752
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.1812094783837443
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.1707014740951764
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.1604445690986271
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.1504291088780978
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.1406459464955785
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.1310864089598451
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.121742266235333
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.112605702712108
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.103669290883363
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.0949259670141602
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.08636900866718
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.0779920138729273
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.0697888818474446
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.061753795097184
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.053881202806633
      converged: True
           flag: converged
 function_calls: 43
     iterations: 41
           root: 1.0461658053930933
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3
      converged: False
           flag: The function value at x=0.3 is NaN; solver cannot continue.
 function_calls: 1
     iterations: nan
           root: 0.3

Therefore, this is a very subtle issue : )

[mmcky]

thanks @HumphreyYang that's a great history.

I guess I was wondering if we should try and fix the function solve_root_problem or just update the boundaries.

[HumphreyYang]

thanks @HumphreyYang that's a great history.

I guess I was wondering if we should try and fix the function solve_root_problem or just update the boundaries.

Hi @mmcky,

I think the function is all good. It is just how scipy handles invalid values in the bracket has changed. It returns NaN directly now.

[mmcky]

@HumphreyYang

It is just how scipy handles invalid values in the bracket has changed. It returns NaN directly now.

I guess where I am going with the function is should we add code to check for np.nan and take corrective action (i.e. return the previous iteration and quit, etc. etc.)?

[HumphreyYang]

Hi @mmcky,

I guess where I am going with the function is should we add code to check for np.nan and take corrective action (i.e. return the previous iteration and quit, etc. etc.)?

We can definitely do that. I think you are suggesting to "restore" what the scipy used to do. I can wrap the function and force it to react to the invalid bracket, but this would make more changes than what is included in this PR. Would you like me to have a go at this?

Many thanks in advance.

[mmcky]

@HumphreyYang let's meet up briefly to discuss. As I understand it -- scipy is now returning np.nan under certain conditions which are fixed by adjusting the bracket values. What I don't understand is if someone learning the lecture comes along and plays around around with bracket values -- how are they to understand what is going on in SciPy.

What I am thinking is if we can add in some code to make the function more robust if a condition is met the SciPy solver is not well behaved or cannot solve. Are there some tests that can be performed (boundary checks)? -- there may not and that is fine (if that is the case).

[mmcky]

@HumphreyYang I will merge this for now -- but let's chat. This is less of an issue then if this was in QuantEcon.py for example.