From dae06a88a217350e034e020cab10685815a09818 Mon Sep 17 00:00:00 2001 From: kelle Date: Wed, 14 May 2025 15:53:20 -0400 Subject: [PATCH 01/11] more info about the felis scripts --- docs/pages/make_new_db/getting_started_new_database.rst | 9 +++++++-- docs/pages/make_new_db/modifying_schema.rst | 2 +- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/docs/pages/make_new_db/getting_started_new_database.rst b/docs/pages/make_new_db/getting_started_new_database.rst index 44d4c06..76a0f68 100644 --- a/docs/pages/make_new_db/getting_started_new_database.rst +++ b/docs/pages/make_new_db/getting_started_new_database.rst @@ -11,5 +11,10 @@ Overview -------- 1. Make a new Github repository using the `astrodb-template-db `_ template repository. -2. :doc:`Modify the schema ` to suit your use case. -3. Ingest data using ``astrodb_utils`` functions. \ No newline at end of file +2. Update the README.md file with your database name and description. + - Please retain the credit line to the template and the AstroDB Toolkit. +3. :doc:`Modify the schema ` in ``schema/schema.yaml`` to suit your use case. +4. Generate a new entity relationship diagram (ERD) and documentation pages for your schema. + - To make a new ERD, run ``scripts/build_schema_docs.py``. This generates a .PNG file in the ``docs/figures`` directory. + - To make new documentation pages, run ``scripts/build_schema_docs.py``. This generates a new set of MarkDown files in the ``docs/schema`` directory. +5. Ingest data by modifing the JSON files by hand (in the ``data/`` directory) or by using ``astrodb_utils`` functions. \ No newline at end of file diff --git a/docs/pages/make_new_db/modifying_schema.rst b/docs/pages/make_new_db/modifying_schema.rst index 3e04dd6..f722850 100644 --- a/docs/pages/make_new_db/modifying_schema.rst +++ b/docs/pages/make_new_db/modifying_schema.rst @@ -30,7 +30,7 @@ These are included in the template database and can be used as models for other Philosophy and guidelines ---------------------- +-------------------------- We encourage users to follow the detailed best practices for astronomical databases outlined in `Chen et al. 2022 `_. From 3e07bd67a33a7b6283540e69d6bdece2abe3ebc7 Mon Sep 17 00:00:00 2001 From: kelle Date: Wed, 14 May 2025 16:12:50 -0400 Subject: [PATCH 02/11] fix pointer to template docs --- .../getting_started_new_database.rst | 39 +++++++++++++------ .../pages/make_new_db/the_template_schema.rst | 2 - .../pages/template_schema/template_schema.rst | 6 +-- 3 files changed, 31 insertions(+), 16 deletions(-) delete mode 100644 docs/pages/make_new_db/the_template_schema.rst diff --git a/docs/pages/make_new_db/getting_started_new_database.rst b/docs/pages/make_new_db/getting_started_new_database.rst index 76a0f68..a3813a3 100644 --- a/docs/pages/make_new_db/getting_started_new_database.rst +++ b/docs/pages/make_new_db/getting_started_new_database.rst @@ -1,20 +1,37 @@ Making a New Database -====================================== +===================== .. toctree:: - the_template_schema - modifying_schema + :titlesonly: + :maxdepth: 2 + ../template_schema/template_schema + modifying_schema Overview -------- -1. Make a new Github repository using the `astrodb-template-db `_ template repository. -2. Update the README.md file with your database name and description. - - Please retain the credit line to the template and the AstroDB Toolkit. -3. :doc:`Modify the schema ` in ``schema/schema.yaml`` to suit your use case. -4. Generate a new entity relationship diagram (ERD) and documentation pages for your schema. - - To make a new ERD, run ``scripts/build_schema_docs.py``. This generates a .PNG file in the ``docs/figures`` directory. - - To make new documentation pages, run ``scripts/build_schema_docs.py``. This generates a new set of MarkDown files in the ``docs/schema`` directory. -5. Ingest data by modifing the JSON files by hand (in the ``data/`` directory) or by using ``astrodb_utils`` functions. \ No newline at end of file +#. Make a new Github repository using the `astrodb-template-db `_ template repository. + +#. Update the README.md file with your database name and description. + + * Please retain the credit line to the template and the AstroDB Toolkit. + +#. :doc:`Modify the schema ` in ``schema/schema.yaml`` to suit your use case. + + * We highly recommend using a AI coding assistant (like GitHub CoPilot) when modifing this file. + +#. Generate a new entity relationship diagram (ERD) and documentation pages for your schema. + + * To make a new ERD, run ``scripts/build_schema_docs.py``. This generates a .PNG file in the ``docs/figures`` directory. + + * To make new documentation pages, run ``scripts/build_schema_docs.py``. This generates a new set of MarkDown files in the ``docs/schema`` directory. + +#. Ingest data by modifing the JSON files by hand (in the ``data/`` directory) or by using ``astrodb_utils`` functions. + + + + + + diff --git a/docs/pages/make_new_db/the_template_schema.rst b/docs/pages/make_new_db/the_template_schema.rst deleted file mode 100644 index a3da8ef..0000000 --- a/docs/pages/make_new_db/the_template_schema.rst +++ /dev/null @@ -1,2 +0,0 @@ -The template schema -===================== \ No newline at end of file diff --git a/docs/pages/template_schema/template_schema.rst b/docs/pages/template_schema/template_schema.rst index 65f5ec7..50d69c4 100644 --- a/docs/pages/template_schema/template_schema.rst +++ b/docs/pages/template_schema/template_schema.rst @@ -1,6 +1,6 @@ -################ -Template schema -################ +############################ +The AstroDB Template schema +############################ .. toctree:: :glob: From 0a54085e6695f9b4bc28c2f7eee6776359dfcf04 Mon Sep 17 00:00:00 2001 From: kelle Date: Wed, 14 May 2025 16:18:25 -0400 Subject: [PATCH 03/11] lint from copilot --- docs/pages/make_new_db/getting_started_new_database.rst | 4 ++-- docs/pages/make_new_db/modifying_schema.rst | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/pages/make_new_db/getting_started_new_database.rst b/docs/pages/make_new_db/getting_started_new_database.rst index a3813a3..d223b92 100644 --- a/docs/pages/make_new_db/getting_started_new_database.rst +++ b/docs/pages/make_new_db/getting_started_new_database.rst @@ -20,7 +20,7 @@ Overview #. :doc:`Modify the schema ` in ``schema/schema.yaml`` to suit your use case. - * We highly recommend using a AI coding assistant (like GitHub CoPilot) when modifing this file. + * We highly recommend using a AI coding assistant (like GitHub CoPilot) when modifying this file. #. Generate a new entity relationship diagram (ERD) and documentation pages for your schema. @@ -28,7 +28,7 @@ Overview * To make new documentation pages, run ``scripts/build_schema_docs.py``. This generates a new set of MarkDown files in the ``docs/schema`` directory. -#. Ingest data by modifing the JSON files by hand (in the ``data/`` directory) or by using ``astrodb_utils`` functions. +#. Ingest data by modifying the JSON files by hand (in the ``data/`` directory) or by using ``astrodb_utils`` functions. diff --git a/docs/pages/make_new_db/modifying_schema.rst b/docs/pages/make_new_db/modifying_schema.rst index f722850..6974566 100644 --- a/docs/pages/make_new_db/modifying_schema.rst +++ b/docs/pages/make_new_db/modifying_schema.rst @@ -30,7 +30,7 @@ These are included in the template database and can be used as models for other Philosophy and guidelines --------------------------- +------------------------- We encourage users to follow the detailed best practices for astronomical databases outlined in `Chen et al. 2022 `_. @@ -42,7 +42,7 @@ TODO: Need an example Column names -~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~ * **Use lowercase column names.** This is a convention. * **Include units in the column name.** Since we do not have a way of storing Quantities in the database, we recommend including the units in the column name. @@ -52,7 +52,7 @@ Column names Units ---------------------------- +----- Per `Chen et al. 2022 `_, we explicitly define the units for each table in their name (e.g., in the `Sources` table, the column with Right Ascension values is named `ra_deg`). Doing so removes unit ambiguity when querying and modifying the database. From e7bd0e39fb11b32b44d8456be9dc5aecabc03028 Mon Sep 17 00:00:00 2001 From: Kelle Cruz Date: Wed, 14 May 2025 16:26:07 -0400 Subject: [PATCH 04/11] Update docs/pages/make_new_db/getting_started_new_database.rst Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/pages/make_new_db/getting_started_new_database.rst | 4 ---- 1 file changed, 4 deletions(-) diff --git a/docs/pages/make_new_db/getting_started_new_database.rst b/docs/pages/make_new_db/getting_started_new_database.rst index d223b92..1d5c3f7 100644 --- a/docs/pages/make_new_db/getting_started_new_database.rst +++ b/docs/pages/make_new_db/getting_started_new_database.rst @@ -31,7 +31,3 @@ Overview #. Ingest data by modifying the JSON files by hand (in the ``data/`` directory) or by using ``astrodb_utils`` functions. - - - - From 7252a2f38e352b286576806ce777822671b8063e Mon Sep 17 00:00:00 2001 From: Kelle Cruz Date: Wed, 14 May 2025 16:26:12 -0400 Subject: [PATCH 05/11] Update docs/pages/make_new_db/getting_started_new_database.rst Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/pages/make_new_db/getting_started_new_database.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/pages/make_new_db/getting_started_new_database.rst b/docs/pages/make_new_db/getting_started_new_database.rst index 1d5c3f7..51f5508 100644 --- a/docs/pages/make_new_db/getting_started_new_database.rst +++ b/docs/pages/make_new_db/getting_started_new_database.rst @@ -26,7 +26,7 @@ Overview * To make a new ERD, run ``scripts/build_schema_docs.py``. This generates a .PNG file in the ``docs/figures`` directory. - * To make new documentation pages, run ``scripts/build_schema_docs.py``. This generates a new set of MarkDown files in the ``docs/schema`` directory. + * To make new documentation pages, run ``scripts/build_schema_docs.py``. This generates a new set of Markdown files in the ``docs/schema`` directory. #. Ingest data by modifying the JSON files by hand (in the ``data/`` directory) or by using ``astrodb_utils`` functions. From fa2b3ab061ab09f0d37380899e32390819ee2168 Mon Sep 17 00:00:00 2001 From: Kelle Cruz Date: Wed, 14 May 2025 16:26:19 -0400 Subject: [PATCH 06/11] Update docs/pages/make_new_db/getting_started_new_database.rst Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/pages/make_new_db/getting_started_new_database.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/pages/make_new_db/getting_started_new_database.rst b/docs/pages/make_new_db/getting_started_new_database.rst index 51f5508..b8a2bbf 100644 --- a/docs/pages/make_new_db/getting_started_new_database.rst +++ b/docs/pages/make_new_db/getting_started_new_database.rst @@ -20,7 +20,7 @@ Overview #. :doc:`Modify the schema ` in ``schema/schema.yaml`` to suit your use case. - * We highly recommend using a AI coding assistant (like GitHub CoPilot) when modifying this file. + * We highly recommend using an AI coding assistant (like GitHub Copilot) when modifying this file. #. Generate a new entity relationship diagram (ERD) and documentation pages for your schema. From 70ee14c9e17d5acbf41512ca6de86c701dc9ccbe Mon Sep 17 00:00:00 2001 From: Kelle Cruz Date: Wed, 14 May 2025 16:26:25 -0400 Subject: [PATCH 07/11] Update docs/pages/make_new_db/getting_started_new_database.rst Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/pages/make_new_db/getting_started_new_database.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/pages/make_new_db/getting_started_new_database.rst b/docs/pages/make_new_db/getting_started_new_database.rst index b8a2bbf..bbac451 100644 --- a/docs/pages/make_new_db/getting_started_new_database.rst +++ b/docs/pages/make_new_db/getting_started_new_database.rst @@ -12,7 +12,7 @@ Making a New Database Overview -------- -#. Make a new Github repository using the `astrodb-template-db `_ template repository. +#. Make a new GitHub repository using the `astrodb-template-db `_ template repository. #. Update the README.md file with your database name and description. From af780d9ed88b05b57dc4891b7608315073e118bf Mon Sep 17 00:00:00 2001 From: kelle Date: Wed, 14 May 2025 16:41:46 -0400 Subject: [PATCH 08/11] fix TOC --- .../ingesting/getting_started_ingesting.rst | 16 +++++++++++++--- docs/pages/ingesting/spectra/index.rst | 5 ++++- .../make_new_db/getting_started_new_database.rst | 8 ++++---- 3 files changed, 21 insertions(+), 8 deletions(-) diff --git a/docs/pages/ingesting/getting_started_ingesting.rst b/docs/pages/ingesting/getting_started_ingesting.rst index f80be02..e19f1ce 100644 --- a/docs/pages/ingesting/getting_started_ingesting.rst +++ b/docs/pages/ingesting/getting_started_ingesting.rst @@ -1,19 +1,29 @@ Ingesting and Modifying Data -==================================== +============================ .. toctree:: :glob: + :maxdepth: 2 + :titlesonly: + ingesting_publications + spectra/* ingest_scripts - spectra + .. note:: If you want to add or modify the tables or columns (rather than just add/modify database contents), see :doc:`/pages/make_new_db/modifying_schema`. -API documentation +API Documentation ----------------- + :py:mod:`astrodb_utils.sources.ingest_source` + Function to ingest source data + :py:mod:`astrodb_utils.sources.ingest_names` + Function to ingest alternative name data + :py:mod:`astrodb_utils.utils.ingest_instrument` + Function to ingest instrument data \ No newline at end of file diff --git a/docs/pages/ingesting/spectra/index.rst b/docs/pages/ingesting/spectra/index.rst index 664bec8..83def96 100644 --- a/docs/pages/ingesting/spectra/index.rst +++ b/docs/pages/ingesting/spectra/index.rst @@ -3,7 +3,10 @@ Spectra .. toctree:: :glob: - spectra/* + :maxdepth: 2 + :titlesonly: + + converting_spectra/* Check if spectra are plottable diff --git a/docs/pages/make_new_db/getting_started_new_database.rst b/docs/pages/make_new_db/getting_started_new_database.rst index d223b92..d21a9f9 100644 --- a/docs/pages/make_new_db/getting_started_new_database.rst +++ b/docs/pages/make_new_db/getting_started_new_database.rst @@ -12,21 +12,21 @@ Making a New Database Overview -------- -#. Make a new Github repository using the `astrodb-template-db `_ template repository. +#. Make a new GitHub repository using the `astrodb-template-db `_ template repository. #. Update the README.md file with your database name and description. - * Please retain the credit line to the template and the AstroDB Toolkit. + * Please retain the credit line to the AstroDB Toolkit. #. :doc:`Modify the schema ` in ``schema/schema.yaml`` to suit your use case. - * We highly recommend using a AI coding assistant (like GitHub CoPilot) when modifying this file. + * We highly recommend using an AI coding assistant (like GitHub Copilot) when modifying this file. #. Generate a new entity relationship diagram (ERD) and documentation pages for your schema. * To make a new ERD, run ``scripts/build_schema_docs.py``. This generates a .PNG file in the ``docs/figures`` directory. - * To make new documentation pages, run ``scripts/build_schema_docs.py``. This generates a new set of MarkDown files in the ``docs/schema`` directory. + * To make new documentation pages, run ``scripts/build_schema_docs.py``. This generates a new set of Markdown files in the ``docs/schema`` directory. #. Ingest data by modifying the JSON files by hand (in the ``data/`` directory) or by using ``astrodb_utils`` functions. From 95408f3f3dbec674abfefb8e1fa273dbbcb5d74e Mon Sep 17 00:00:00 2001 From: kelle Date: Wed, 14 May 2025 18:54:14 -0400 Subject: [PATCH 09/11] nitpicks --- docs/pages/make_new_db/modifying_schema.rst | 8 ++++---- docs/pages/template_schema/template_schema.rst | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/pages/make_new_db/modifying_schema.rst b/docs/pages/make_new_db/modifying_schema.rst index 6974566..2c1e203 100644 --- a/docs/pages/make_new_db/modifying_schema.rst +++ b/docs/pages/make_new_db/modifying_schema.rst @@ -1,5 +1,5 @@ Modifying the schema -====================== +==================== The template database comes with an existing schema, consisting of a set of tables and columns. It is expected that every usecase will modify this schema to suit their needs. @@ -12,7 +12,7 @@ The `Felis Data Model documentation `_ provides a detailed Required tables ----------------- +--------------- There are several tables which are expected by ``astrodb_utils`` and should be modified with great care: * Sources @@ -23,7 +23,7 @@ There are several tables which are expected by ``astrodb_utils`` and should be m * Instruments Optional tables ----------------- +--------------- Optional tables are things like Spectra, Photometry, Radial Velocities, etc. These are included in the template database and can be used as models for other data tables and can be removed/modified if not needed. @@ -36,7 +36,7 @@ We encourage users to follow the detailed best practices for astronomical databa Long vs Wide tables -~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~ Think carefully about the structure of your tables. TODO: Need an example diff --git a/docs/pages/template_schema/template_schema.rst b/docs/pages/template_schema/template_schema.rst index 50d69c4..931f465 100644 --- a/docs/pages/template_schema/template_schema.rst +++ b/docs/pages/template_schema/template_schema.rst @@ -1,6 +1,6 @@ -############################ +########################### The AstroDB Template schema -############################ +########################### .. toctree:: :glob: From 688549eeef1db7e00b85c403095ea04bc3c03ffb Mon Sep 17 00:00:00 2001 From: kelle Date: Wed, 14 May 2025 18:54:23 -0400 Subject: [PATCH 10/11] add bit about LICENSE file --- docs/pages/make_new_db/getting_started_new_database.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/pages/make_new_db/getting_started_new_database.rst b/docs/pages/make_new_db/getting_started_new_database.rst index 436f4c8..2abeb26 100644 --- a/docs/pages/make_new_db/getting_started_new_database.rst +++ b/docs/pages/make_new_db/getting_started_new_database.rst @@ -14,19 +14,21 @@ Overview #. Make a new GitHub repository using the `astrodb-template-db `_ template repository. -#. Update the README.md file with your database name and description. +#. Update the ``README.md`` file with your database name and description. * Please retain the credit line to the AstroDB Toolkit. +#. Update the ``LICENSE`` file with your license of choice. + #. :doc:`Modify the schema ` in ``schema/schema.yaml`` to suit your use case. * We highly recommend using an AI coding assistant (like GitHub Copilot) when modifying this file. #. Generate a new entity relationship diagram (ERD) and documentation pages for your schema. - * To make a new ERD, run ``scripts/build_schema_docs.py``. This generates a .PNG file in the ``docs/figures`` directory. + * To make a new ERD, run ``scripts/build_schema_docs.py``. This generates a PNG file in the ``docs/figures/`` directory. - * To make new documentation pages, run ``scripts/build_schema_docs.py``. This generates a new set of Markdown files in the ``docs/schema`` directory. + * To make new documentation pages, run ``scripts/build_schema_docs.py``. This generates a new set of Markdown files in the ``docs/schema/`` directory. #. Ingest data by modifying the JSON files by hand (in the ``data/`` directory) or by using ``astrodb_utils`` functions. From c81bfc7ee9e6700fba7010b185924328a95757b9 Mon Sep 17 00:00:00 2001 From: kelle Date: Wed, 14 May 2025 19:07:28 -0400 Subject: [PATCH 11/11] fix link to converting spectra --- docs/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/index.rst b/docs/index.rst index 3fb3440..67ea332 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -20,7 +20,7 @@ User Guide pages/make_new_db/getting_started_new_database pages/template_schema/template_schema pages/making_private_version - pages/converting_spectra/converting_spectra + pages/ingesting/spectra/converting_spectra/converting_spectra pages/getting_help/index Developer documentation API documentation