Add load_json.py tool, remove --podcast flag from query.py #164

Copilot · 2026-01-15T02:09:31Z

Summary

This PR successfully implements the requested feature: a new tools/load_json.py tool that loads JSON-serialized podcast data into a SQLite database, and simplifies query.py by removing the corresponding logic.

Changes Made

✅ Created `tools/load_json.py`

Command-line tool with -v/--verbose and -d/--database (required) flags
Accepts JSON index path as positional argument (placed after flags per convention)
Loads JSON-serialized podcast data into SQLite database using Podcast.read_from_file()
Uses async context manager (async with provider:) for proper transaction handling
Validates database is empty before loading
Shows statistics on loaded data when verbose mode is enabled

✅ Simplified `tools/query.py`

Removed --podcast flag completely
Made --database flag required
Removed load_podcast_index() function (26 lines removed)
Removed unused SqliteStorageProvider import
Simplified main() to only work with existing databases
Simple error message if database is empty

✅ Addressed All Review Feedback

Removed epilog examples (advanced users)
Moved positional args after flag args
Changed comment to "Ensure index file exists"
Removed try/except wrapper (show full tracebacks)
Use async with provider: properly wrapping the read_from_file call
Simplified error messages in query.py
Formatted function call to collapse to one line

Code Quality

✅ Type checking passes
✅ Code formatting passes (black)
✅ Tools execute successfully and show correct help text
✅ All review feedback addressed

Usage

# Load JSON data into database
python tools/load_json.py -d podcast.db tests/testdata/Episode_53_AdrianTchaikovsky_index

# Query the database
python tools/query.py -d podcast.db --query "What was discussed?"

Original prompt

This section details on the original issue you should resolve

<issue_title>Need a load_json.py tool that loads a JSON serialized db into a SQLiite3 db</issue_title>
<issue_description>Then we can drop the corresponding logic from query.py</issue_description>

<agent_instructions>Create the tool in tools/load_json.py. Copy the logic from query.py's --podcast flag. Mimic the command line flags from other tools (mostly -v, -d/--database).</agent_instructions>

Comments on the Issue (you are @copilot in this section)

Fixes Need a load_json.py tool that loads a JSON serialized db into a SQLiite3 db #163

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: gvanrossum-ms <184014243+gvanrossum-ms@users.noreply.github.com>

…prove error message consistency Co-authored-by: gvanrossum-ms <184014243+gvanrossum-ms@users.noreply.github.com>

gvanrossum-ms · 2026-01-15T02:31:54Z

tools/query.py

+            f"Please load data into the database first using tools/load_json.py:\n"
+            f"  python tools/load_json.py <index_path> -d {args.database}\n"
+            f"Example:\n"
+            f"  python tools/load_json.py tests/testdata/Episode_53_AdrianTchaikovsky_index -d {args.database}"


Drop these four lines (there are other tools to load data).

Done in dd79821 - simplified to just "Error: Database '{args.database}' is empty."

gvanrossum-ms · 2026-01-15T02:36:57Z

tools/load_json.py

+        epilog="""
+Examples:
+  python tools/load_json.py tests/testdata/Episode_53_AdrianTchaikovsky_index -d podcast.db
+  python tools/load_json.py path/to/index -d output.db -v
+
+Note: The index path should exclude the "_data.json" suffix.
+        """,


Drop the epilog. This tool has only advanced users.

Done in dd79821 - removed epilog and formatter_class.

gvanrossum-ms · 2026-01-15T02:37:22Z

tools/load_json.py

+    )
+
+    parser.add_argument(
+        "index_path",


Does this need to be made required=True?

No, positional arguments are required by default in argparse.

gvanrossum-ms · 2026-01-15T02:38:06Z

tools/load_json.py

+        """,
+    )
+
+    parser.add_argument(


Traditionally, positional args are placed after all the flag args.

Done in dd79821 - moved index_path after the flag arguments.

gvanrossum-ms · 2026-01-15T02:38:32Z

tools/load_json.py

+
+    args = parser.parse_args()
+
+    # Validate index file exists


Suggested change

# Validate index file exists

# Ensure index file exists

Done in dd79821 - changed to "Ensure index file exists".

gvanrossum-ms · 2026-01-15T02:39:19Z

tools/load_json.py

+    utils.load_dotenv()
+
+    # Run the loading process
+    try:


Drop the try/except. When it crashes I want to see a traceback.

Done in dd79821 - removed try/except wrapper.

gvanrossum-ms · 2026-01-15T02:48:09Z

tools/load_json.py

+        if isinstance(provider, SqliteStorageProvider):
+            provider.db.commit()


Instead, use with provider: <create conversation> -- that is supported by all provider types.

Done in dd79821 - now uses async with provider: for transaction handling.

gvanrossum-ms · 2026-01-15T02:49:57Z

@copilot address my review

…anager, reorder args Co-authored-by: gvanrossum-ms <184014243+gvanrossum-ms@users.noreply.github.com>

Copilot · 2026-01-15T02:55:32Z

@copilot address my review

All review feedback addressed in dd79821:

Simplified error messages in query.py (removed multi-line instructions)
Removed epilog from load_json.py
Moved positional arg after flags
Changed comment to "Ensure index file exists"
Removed try/except wrapper to show full tracebacks
Use async with provider: for proper transaction handling
Removed unused SqliteStorageProvider import

gvanrossum-ms

@copilot just two more nits

gvanrossum-ms · 2026-01-15T02:59:34Z

tools/load_json.py

+        async with provider:
+            pass  # Commit happens in __aexit__


No, no, no! The async with goes around lines 68-70.

Fixed in bfab170 - moved async with provider: to wrap the read_from_file call (lines 68-71).

gvanrossum-ms · 2026-01-15T03:01:55Z

tools/load_json.py

+        load_json_to_database(
+            args.index_path,
+            args.database,
+            args.verbose,


If you remove the trailing comma and re-run make format the load_json_to_database() call will likely collapse on one line.

Fixed in bfab170 - removed trailing comma and ran black formatter, which collapsed the call to one line.

Co-authored-by: gvanrossum-ms <184014243+gvanrossum-ms@users.noreply.github.com>

Copilot · 2026-01-15T03:05:31Z

@copilot just two more nits

Both nits addressed in bfab170:

Fixed async context manager to properly wrap the read_from_file call
Removed trailing comma and reformatted to collapse function call to one line

gvanrossum-ms

@copilot, one request. Can you test that the code actually works with real data? Run it with something like

rm -f loadtest.db
python tools/load_json.py -v -d loadtest.db tests/testdata/Episode_53_AdrianTchaikovsky_index

and inspect the database using sqlite3, e.g.

sqlite3 loadtest.db
select count(*) from messages;

Then just post the output in a comment to the PR.

gvanrossum-ms · 2026-01-15T03:23:53Z

@copilot one request. Can you test that the code actually works with real data? Run it with something like

rm -f loadtest.db
python tools/load_json.py -v -d loadtest.db tests/testdata/Episode_53_AdrianTchaikovsky_index

and inspect the database using sqlite3, e.g.

sqlite3 loadtest.db
select count(*) from messages;

Then just post the output in a comment to the PR.

Initial plan

a86d8b6

Copilot AI assigned Copilot and gvanrossum-ms Jan 15, 2026

Copilot started work on behalf of gvanrossum-ms January 15, 2026 02:10 View session

Copilot AI and others added 5 commits January 15, 2026 02:14

Add tools/load_json.py tool

d1c98ce

Co-authored-by: gvanrossum-ms <184014243+gvanrossum-ms@users.noreply.github.com>

Remove --podcast flag from query.py, require --database

d80014f

Co-authored-by: gvanrossum-ms <184014243+gvanrossum-ms@users.noreply.github.com>

Improve error handling in load_json.py

8b25997

Co-authored-by: gvanrossum-ms <184014243+gvanrossum-ms@users.noreply.github.com>

Fix docstrings in load_json.py to clarify database requirement

5fab841

Co-authored-by: gvanrossum-ms <184014243+gvanrossum-ms@users.noreply.github.com>

Address final code review feedback - optimize database queries and im…

4001250

…prove error message consistency Co-authored-by: gvanrossum-ms <184014243+gvanrossum-ms@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add load_json.py tool for loading JSON into SQLite3 db~~ Add load_json.py tool, remove --podcast flag from query.py Jan 15, 2026

Copilot AI requested a review from gvanrossum-ms January 15, 2026 02:24

Copilot finished work on behalf of gvanrossum-ms January 15, 2026 02:24

gvanrossum-ms reviewed Jan 15, 2026

View reviewed changes

Copilot started work on behalf of gvanrossum-ms January 15, 2026 02:50 View session

Address review feedback: simplify error messages, use async context m…

dd79821

…anager, reorder args Co-authored-by: gvanrossum-ms <184014243+gvanrossum-ms@users.noreply.github.com>

Copilot AI requested a review from gvanrossum-ms January 15, 2026 02:56

Copilot finished work on behalf of gvanrossum-ms January 15, 2026 02:56

gvanrossum-ms reviewed Jan 15, 2026

View reviewed changes

Copilot started work on behalf of gvanrossum-ms January 15, 2026 03:02 View session

Fix async context manager placement and format function call

bfab170

Co-authored-by: gvanrossum-ms <184014243+gvanrossum-ms@users.noreply.github.com>

Copilot finished work on behalf of gvanrossum-ms January 15, 2026 03:06

Copilot AI requested a review from gvanrossum-ms January 15, 2026 03:06

gvanrossum-ms marked this pull request as ready for review January 15, 2026 03:15

Copilot AI had a problem deploying to build-pipeline January 15, 2026 03:15 Failure

gvanrossum-ms approved these changes Jan 15, 2026

View reviewed changes

gvanrossum approved these changes Jan 15, 2026

View reviewed changes

Copilot started work on behalf of gvanrossum-ms January 15, 2026 03:24 View session

Copilot AI requested a review from gvanrossum-ms January 15, 2026 03:26

Copilot stopped work on behalf of gvanrossum-ms due to an error January 15, 2026 03:26
The session was cancelled by the user.

gvanrossum-ms merged commit 48f69c3 into main Jan 15, 2026
15 of 17 checks passed

gvanrossum-ms deleted the copilot/add-load-json-tool branch January 15, 2026 03:26

		if isinstance(provider, SqliteStorageProvider):
		provider.db.commit()

Add load_json.py tool, remove --podcast flag from query.py #164

Add load_json.py tool, remove --podcast flag from query.py #164

Uh oh!

Conversation

Copilot AI commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes Made

✅ Created tools/load_json.py

✅ Simplified tools/query.py

✅ Addressed All Review Feedback

Code Quality

Usage

Comments on the Issue (you are @copilot in this section)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

gvanrossum-ms commented Jan 15, 2026

Uh oh!

Copilot AI commented Jan 15, 2026

Uh oh!

gvanrossum-ms left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Jan 15, 2026

Uh oh!

gvanrossum-ms left a comment

Choose a reason for hiding this comment

Uh oh!

gvanrossum-ms commented Jan 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Copilot AI commented Jan 15, 2026 •

edited

Loading

✅ Created `tools/load_json.py`

✅ Simplified `tools/query.py`