2024-02-13 - grid order

[sbenthall]

seeds = 6

AMMPS config parameters:

--mm_lucas_factor [0.0, 0.1, 1.0]
--dividend_std [0.0055, 0.011] [This should be synced across SHARKFin and AMMPS!]

SHARKFin parameters:

--dphm [1, 148, 1097]

--attention [0.001, 0.1]
--zeta [0.0, 1.0]

--dividend_growth_rate 1.0002
--dividend_std [0.0055, 0.011] [This should be synced across SHARKFin and AMMPS!]

--pop_CRRA 5
--pop_DiscFac 0.974

size: 432

[sbenthall]

@wjt5121 Would you please execute this grid? Please note that the dividend_std parameter needs to be set in both the AMMPS configuration and SHARKFin command line argument, and these should be the same value.

This should be a rather short run -- 432 small -- so it's just to make sure things are all in order before doing a bigger one to get across the finish line.

[sbenthall]

If keeping the two --dividend_std parameters synced is too difficult to implement in a short time, we can I suppose run both as a 'grid' and then discard the data where the two don't match.

That is computationally inefficient of course but would do in a pinch.

[wjt5121]

Ok I have that grid running now. But I don't think I have the --dividend_std parameters synced.
Here is a sample of the commands I am using to start the sims. @mesalas should I be sendig --dividend_stds on the amps config command?

AMMPS Config Generator:
/usr/bin/python3 /usr/simulation/ammps_config_generator/acg/simulations/make_lucas_shark_config.py --seed 1327 --name test_conf1.xlsx --days 240 --mm_lucas_factor 0.0 --out-dir /usr/simulation/

AMMPS:
dotnet /usr/simulation/ammps_bin/amm.engine.dll RunConfFromFile /usr/simulation/test_conf1.xlsx /shared/home/ammpssharkfin/output/jebo5run1out --number 1 --compression true --rabbitMQ-host 172.210.0.136 --rabbitMQ-queue jebo5run1 --prefix lshark

SHARKfin:
/usr/bin/python3 /usr/simulation/SHARKFin/simulate/run_any_simulation.py /shared/home/ammpssharkfin/output/jebo5run1 --simulation Attention --expectations InferentialExpectations --market ClientRPCMarket --zeta 0.0 --attention 0.001 --dphm 1 --queue jebo5run1 --rhost 172.210.0.136 --tag r3 --seed 1327 --quarters 4 --pop_CRRA 5 --pop_DiscFac 0.974 --pop_aNrmInitMean 4.015 --dividend_growth_rate 1.0002 --dividend_std 0.011

JEBO5__parametergrid.csv

[sbenthall]

@wjt5121 Since @mesalas might be asleep now -- yes, dividend_std should be passed to the amps config generator, please

[wjt5121]

ok great. Run6 is going now.
Here is the run5 which is with out passing 'dividend_std'
jebo5_completed_parametergrid.csv

[wjt5121]

Unfortunately I had a bug in the most recent deployment and last nights run failed to properly start.
I've restarted it and confirmed the sims are running. The are being executed by seed so all the sims in the first seed should finish before the second seeds sims start.
Here is the empty parameter grid.
JEBO_smoketest2_parametergrid.csv

You can pull the data directly from the storage table using the function below. You will need the storage connection string which I've sent via email.

from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
from azure.data.tables import TableServiceClient
from azure.data.tables import TableClient
import pandas as pd

class simulationGrid():
    def __init__(self, gridTableName,CONN_STRING):
        self.table_name = gridTableName
        self.conn_str = CONN_STRING
        self.table_service = TableServiceClient.from_connection_string(self.conn_str)
        self.table_client = self.table_service.get_table_client(self.table_name)



def get_dataframe(experimentName):
    simgrid = simulationGrid(experimentName,CONN_STRING)
    entities = simgrid.table_client.list_entities()
    tableEntities = list(entities)
    print(f"Located {len(tableEntities)} entities.")
    edf = pd.DataFrame(tableEntities)
    return edf

#example usage
CONN_STRING = <enter provided connection string>
experimentName = 'jebosmoketest2'
df = get_dataframe(experimentName)

[sbenthall]

Thanks @wjt5121 . Hopefully this works, and then I'll put in a much larger grid order later today.

[wjt5121]

@sbenthall
Second run of the smoke test complete successfully. I'll keep an I out for the next grid and get it started right away.

Total simulations in grid is 432.
Simulation completed: 431 / 99.76 percent of total.
Each simulation contains 4 quarters of 60 days for a total of 103680.
431 simulations exited successfully with status 0 and completed all 240 days successfully for a total of 103440
0 simulations exited with status code 1. Simulations existing with code 1 completed 0 days
1 simulations exited with status code 3. Simulations existing with code 3 completed 88 days
The total number of simulated days is 103528 which is 99.85 percent of the total.

jebosmoketest2_results.csv

[sbenthall]

@wjt5121 Thanks again for this.

But there's some issues with this smoke test data.

• I'm not seeing the dividend_std column in the output. This is now a design parameter, so it should be a column in the parameter grid table. We also need to see how the output varies with respect to it.
  • This is complicated by the fact that we have both the AMMPS config and SHARKFin versions of this parameter. We will need both in the output data, so that we can filter for equality.
• There are some additional sim_stats that I've included as of a couple of days ago (this commit). Is it possible that your SHARKFin is not running up to date with master?

I'm sorry to have to lay this on you at the last minute. How much is it achievable to address this? An even smaller smoke test, say with seeds = 2, would be just fine for testing this.

[wjt5121]

@sbenthall no problem at all, I have a 2 seed grid running now with the above changes and it should be done in about 40 minutes.

The way it is currently building the configs the dividend_std variable is set only one and is used while build both the SHARKfin and AMMPS configs so for these run you can be sure dividend_std will be the same.
I've also added it to the grid table so you'll see it in the dataframe now.

As for the latest commit, yep I was running an older version, I did a pull and saw 3 files changed so we should be good to go now.

[sbenthall]

Oh, fantastic, thanks @wjt5121 ! I'll check out the new data when it's ready.
I'm also working on the next grid design, assuming it works.

[wjt5121]

Run has completed.
jebosmoketest3_results.csv

Total simulations in grid is 144.
Simulation completed: 144 / 100.0 percent of total.
Each simulation contains 4 quarters of 60 days for a total of 34560.
144 simulations exited successfully with status 0 and completed all 240 days successfully for a total of 34560
0 simulations exited with status code 1. Simulations existing with code 1 completed 0 days
0 simulations exited with status code 3. Simulations existing with code 3 completed 0 days
The total number of simulated days is 34560 which is 100.0 percent of the total.

[sbenthall]

It looks great, @wjt5121 . Thank you so much