Dir fixes (#19)

Fixes in dir structure
capillariesio · Mar 12, 2023 · 8aba1c1 · 8aba1c1
1 parent 615914c
commit 8aba1c1
Show file tree

Hide file tree

Showing 29 changed files with 11 additions and 6 deletions.
diff --git a/doc/started.md b/doc/started.md
@@ -10,7 +10,7 @@ If you are running Windows, you will be using WSL for development, so make sure
 
 ## Prepare data directories
 
-In production environments, Capillaries server components ([Daemon](glossary.md#daemon), [Toolbelt](glossary.md#toolbelt), [Webapi](glossary.md#webapi)) need access to configuration files, source data files and target directories. In dev environments, we want Capillaries components to access those files and directories in the uniform way: for dockerized component and for the scenarios when [Daemon](glossary.md#daemon), [Toolbelt](glossary.md#toolbelt), and [Webapi](glossary.md#webapi) are run by developers. We use `/tmp/capitest_*` directories that can be accessed using the same path - from the host machine and from containers (see [docker-compose.yml](../docker-compose.yml) for volume definitions). 
+In production environments, Capillaries server components ([Daemon](glossary.md#daemon), [Toolbelt](glossary.md#toolbelt), [Webapi](glossary.md#webapi)) need access to configuration files, source data files and target directories. In dev environments, we want Capillaries components to access those files and directories in the uniform way: for dockerized component and for the scenarios when [Daemon](glossary.md#daemon), [Toolbelt](glossary.md#toolbelt), and [Webapi](glossary.md#webapi) are run by developers. We use `/tmp/capi_*` directories that can be accessed using the same path - from the host machine and from containers (see [docker-compose.yml](../docker-compose.yml) for volume definitions). 
 
 Run these commands from the root project directory, they will create those data directories and populate them with sample configurations and sample source data:
 
@@ -103,7 +103,7 @@ If you are running Windows, you will be using WSL for development, so make sure
 
 Is there a way to develop and debug Capillaries server components in a dev environment like VSCode without running it from WSL? Yes, but you will have to solve two problems.
 
-1. Data directories `/tmp/capitest_*` will not be available from Windows, so you will have to tweak all configuration files and shell scripts so they reference Windows paths. This is doable, but it's a tedious job.
+1. Data directories `/tmp/capi_*` will not be available from Windows, so you will have to tweak all configuration files and shell scripts so they reference Windows paths. This is doable, but it's a tedious job.
 
 2. When [test_tag_and_denormalize](../test/code/tag_and_denormalize/README.md) integration test runs in WSL and uses Webapi executed from Windows dev environment, `curl` command will not be able to connect to Webapi's `http://localhost:6543` because of the known WSL limitation descussed at https://github.com/microsoft/WSL/issues/5211 and at https://superuser.com/questions/1679757/how-to-access-windows-localhost-from-wsl2 . You will need to use host IP address or use `$(localhost).local` instead of `localhost` in the shell script.
 

diff --git a/test/data/cfg/tag_and_denormalize_quicktest/script_params_one_run_input_https.json b/test/data/cfg/tag_and_denormalize_quicktest/script_params_one_run_input_https.json
@@ -2,6 +2,6 @@
     "totals_start_policy": "auto",
     "uri_suffix": "?raw=1",
     "dir_cfg": "https://github.com/capillariesio/capillaries/blob/main/test/data/cfg/tag_and_denormalize_quicktest",
-    "dir_in": "https://github.com/capillariesio/capillaries/blob/main/test/data/in/tag_and_denormalize",
+    "dir_in": "https://github.com/capillariesio/capillaries/blob/main/test/data/in/tag_and_denormalize_quicktest",
     "dir_out": "/tmp/capi_out/tag_and_denormalize_quicktest"
 }
diff --git a/...a/in/lookup/olist_order_items_dataset.csv → ...p_quicktest/olist_order_items_dataset.csv b/...a/in/lookup/olist_order_items_dataset.csv → ...p_quicktest/olist_order_items_dataset.csv
diff --git a/test/data/in/lookup/olist_orders_dataset.csv → ...lookup_quicktest/olist_orders_dataset.csv b/test/data/in/lookup/olist_orders_dataset.csv → ...lookup_quicktest/olist_orders_dataset.csv
diff --git a/test/data/in/portfolio/accounts.csv → .../data/in/portfolio_quicktest/accounts.csv b/test/data/in/portfolio/accounts.csv → .../data/in/portfolio_quicktest/accounts.csv
diff --git a/test/data/in/portfolio/holdings.csv → .../data/in/portfolio_quicktest/holdings.csv b/test/data/in/portfolio/holdings.csv → .../data/in/portfolio_quicktest/holdings.csv
diff --git a/test/data/in/portfolio/txns.csv → test/data/in/portfolio_quicktest/txns.csv b/test/data/in/portfolio/txns.csv → test/data/in/portfolio_quicktest/txns.csv
diff --git a/...n/py_calc/olist_order_items_dataset00.csv → ...quicktest/olist_order_items_dataset00.csv b/...n/py_calc/olist_order_items_dataset00.csv → ...quicktest/olist_order_items_dataset00.csv
diff --git a/...n/py_calc/olist_order_items_dataset01.csv → ...quicktest/olist_order_items_dataset01.csv b/...n/py_calc/olist_order_items_dataset01.csv → ...quicktest/olist_order_items_dataset01.csv
diff --git a/...n/py_calc/olist_order_items_dataset02.csv → ...quicktest/olist_order_items_dataset02.csv b/...n/py_calc/olist_order_items_dataset02.csv → ...quicktest/olist_order_items_dataset02.csv
diff --git a/...n/py_calc/olist_order_items_dataset03.csv → ...quicktest/olist_order_items_dataset03.csv b/...n/py_calc/olist_order_items_dataset03.csv → ...quicktest/olist_order_items_dataset03.csv
diff --git a/...n/py_calc/olist_order_items_dataset04.csv → ...quicktest/olist_order_items_dataset04.csv b/...n/py_calc/olist_order_items_dataset04.csv → ...quicktest/olist_order_items_dataset04.csv
diff --git a/...tag_and_denormalize/flipcart_products.tsv → ...normalize_quicktest/flipcart_products.tsv b/...tag_and_denormalize/flipcart_products.tsv → ...normalize_quicktest/flipcart_products.tsv
diff --git a/test/data/in/tag_and_denormalize/tags.csv → ...in/tag_and_denormalize_quicktest/tags.csv b/test/data/in/tag_and_denormalize/tags.csv → ...in/tag_and_denormalize_quicktest/tags.csv
diff --git a/...der_date_value_grouped_inner_baseline.csv → ...der_date_value_grouped_inner_baseline.csv b/...der_date_value_grouped_inner_baseline.csv → ...der_date_value_grouped_inner_baseline.csv
diff --git a/...ate_value_grouped_left_outer_baseline.csv → ...ate_value_grouped_left_outer_baseline.csv b/...ate_value_grouped_left_outer_baseline.csv → ...ate_value_grouped_left_outer_baseline.csv
diff --git a/...lookup/order_item_date_inner_baseline.csv → ...cktest/order_item_date_inner_baseline.csv b/...lookup/order_item_date_inner_baseline.csv → ...cktest/order_item_date_inner_baseline.csv
diff --git a/...p/order_item_date_left_outer_baseline.csv → ...t/order_item_date_left_outer_baseline.csv b/...p/order_item_date_left_outer_baseline.csv → ...t/order_item_date_left_outer_baseline.csv
diff --git a/test/data/out/lookup_quicktest/readme.txt b/test/data/out/lookup_quicktest/readme.txt
@@ -0,0 +1 @@
+Placeholder for out files
diff --git a/...o/account_period_sector_perf_baseline.csv → ...t/account_period_sector_perf_baseline.csv b/...o/account_period_sector_perf_baseline.csv → ...t/account_period_sector_perf_baseline.csv
diff --git a/.../portfolio/account_year_perf_baseline.csv → ..._quicktest/account_year_perf_baseline.csv b/.../portfolio/account_year_perf_baseline.csv → ..._quicktest/account_year_perf_baseline.csv
diff --git a/.../portfolio/account_year_perf_official.csv → ..._quicktest/account_year_perf_official.csv b/.../portfolio/account_year_perf_official.csv → ..._quicktest/account_year_perf_official.csv
diff --git a/test/data/out/py_calc_quicktest/readme.txt b/test/data/out/py_calc_quicktest/readme.txt
@@ -0,0 +1 @@
+Placeholder for out files
diff --git a/...py_calc/taxed_order_items_go_baseline.csv → ...icktest/taxed_order_items_go_baseline.csv b/...py_calc/taxed_order_items_go_baseline.csv → ...icktest/taxed_order_items_go_baseline.csv
diff --git a/...py_calc/taxed_order_items_py_baseline.csv → ...icktest/taxed_order_items_py_baseline.csv b/...py_calc/taxed_order_items_py_baseline.csv → ...icktest/taxed_order_items_py_baseline.csv
diff --git a/test/data/out/tag_and_denormalize_quicktest/readme.txt b/test/data/out/tag_and_denormalize_quicktest/readme.txt
@@ -0,0 +1 @@
+Placeholder for out files
diff --git a/...g_and_denormalize/tag_totals_baseline.tsv → ...rmalize_quicktest/tag_totals_baseline.tsv b/...g_and_denormalize/tag_totals_baseline.tsv → ...rmalize_quicktest/tag_totals_baseline.tsv
diff --git a/...products_for_operator_review_baseline.csv → ...products_for_operator_review_baseline.csv b/...products_for_operator_review_baseline.csv → ...products_for_operator_review_baseline.csv
diff --git a/test/deploy/README.md b/test/deploy/README.md
@@ -44,8 +44,8 @@ Host <genesis_bastion_ip>
 Just for convenience, let's store deploy tool arguments and other configuration settings in shell variables, for example:
 ```
 export capideploy=../../build/capideploy.exe
-export DEPLOY_ARGS="-prj capideploy_project_dreamhost.json -prj_params $HOME/capideploy_project_params_dreamhost.json"
-export DEPLOY_ROOT_KEY=$HOME/.ssh/sampledeployment001_rsa
+export DEPLOY_ARGS="-prj capideploy_project_genesis.json -prj_params $HOME/capideploy_project_params_genesis.json"
+export DEPLOY_ROOT_KEY=$HOME/.ssh/sampledeployment002_rsa
 export BASTION_IP=<dreamhost_bastion_ip>
 ```
 
@@ -97,7 +97,9 @@ This command will populate /tmp/capi_in, /tmp/capi_cfg, /tmp/capi_out
 cd ./test/code/lookup
 ./1_create_quicktest_data.sh
 ./create_bigtest_data.sh
-cd test/code/tag_and_denormalize
+cd ../py_calc
+./1_create_quicktest_data.sh
+cd ../tag_and_denormalize
 ./1_create_quicktest_data.sh
 ```