No way to create layers by modifying the working copy

[olsen232]

If you delete a layer in the working copy, then you should be able to commit this to delete the layer in your repo.

If you add a layer to a working copy, then you should be able to commit this to add the layer to your repo.
This is a bit trickier: the layer will need to be modified to have the triggers added etc.

[olsen232]

Deleting works, but you still can't add a layer just by manually creating it in your working copy.

[Zumitify]

Hello @olsen232, Sumit here,
I am new to the Kart community and I would like to work on this issue if no one else is already looking into it.

I have a few questions regarding the issue.
So I tried the following:

1. Initialized Kart repo and opened the default layer using QGIS
2. Tried editing attributes and they showed up in kart diff correctly. ( This was followed by committing the changes)
3. Created 2 more layers (Shapefile and SpatiaLite layers) in the same QGIS project and edited those two layers.
   After editing and saving these changes, nothing was reflected in kart diff.
   Note: These 2 layers are visible in the layers panel of QGIS and not under the Kart plugin-> repo ->Datasets.

If you add a layer to a working copy.......
This is a bit trickier: the layer will need to be modified to have the triggers added etc.

• Is there any other procedure to add layers to the working copy? Because in my case, there was no diff, and hence no chance of committing even after modifying the changes for the newly added layers.

Also, the expectation of the issue is that: Once a layer is added into the working copy, that itself should set the diff trigger (that something has changed in the repo and therefore allow a commit) without having to explicitly make changes in the newly added layer.

• Is my understanding of the issue correct?

Thanks

[craigds]

Currently the only eay to add a dataset is by using the kart import command. Creating a dataset directly in the working copy doesn't work - kart doesn't start tracking changes and producing diffs and allow commits for tables it doesn't know about, hence this issue

[Zumitify]

Hey @craigds, thanks for clarifying my query. I misunderstood QGIS layers for datasets/tables.
So the high-level tasks to achieve the solution would be:

1. To have some kind of a listener on the working copy that detects a new dataset(layer) is being added.
2. Then for that dataset, do what the kart import command internally does in order to add it to the working copy.

Am I thinking in the right direction for this?

[craigds]

I think perhaps this ticket description could be clearer about what is required here.

I think there are two ways we could solve this issue:

1. Make diff command show the entire contents of every untracked table, and make commit commit those tables automatically. When a new table is committed, add the triggers to the working copy so we will notice changes to the table for future diffs/commits.
2. Don't automatically include new tables in diffs/commits, but have an explicit add-dataset command that marks the table for inclusion in working copy diffs and in the next commit. (very similar to the add command in vanilla git)

(1) is the 'obvious' way to solve the problem, but I'm not sure it's the best way since tables are probably created for all sorts of reasons. For example, GPKG and the geopackage extensions specify a number of extra tables with various names - kart's working copy geopackages have rtree spatial-indexes for their spatial datasets - and those tables should not be committed to the repository.

While we could maintain a blocklist of table names and name patterns that shouldn't be committed or included in diffs, it's unlikely it'd be exhaustive. So I think a better solution might be an explicit add-dataset command.

It would be useful to see @olsen232 's thoughts on this as he's probably thought about it more than I have.

[olsen232]

I don't think my thinking has gone any further than that. I agree that the add-dataset approach is less likely to run into obstacles - the other approach, where untracked tables are added/committed by default, could indeed cause problems if users are purposely storing untracked tables inside their working copy - users might already be doing so, in which case a new Kart version which commits them will be disruptive.

You are right this is a bit larger than most of our "good first issues" - I've been struggling to find issues that are small and self-contained.
Potentially as a separate issue - we could do something to help user's find untracked tables. Eg kart add-dataset --list-untracked

[Zumitify]

Thanks for your insights and comments @craigds and @olsen232.
The new issue - "listing untrackable tables using kart add-dataset --list-untracked command seems less complex than this issue, so should I create a new enhancement request issue and take that up? @olsen232

In that case, as a starting point, I can start by exploring how git status shows untracked files before we add them with git add.
Thanks.

[craigds]

The new issue - "listing untrackable tables using kart add-dataset --list-untracked command seems less complex than this issue, so should I create a new enhancement request issue and take that up? @olsen232

Sounds good to make a separate issue and tackle that piece first. I would probably prefer kart status --list-untracked; although the internals of what's required will be different than what's currently in the status/diff commands, it seems a more sensible place to put it from a user perspective.

In that case, as a starting point, I can start by exploring how git status shows untracked files before we add them with git add.

You may find this less relevant than you might think. Vanilla git uses a staging area ("the index") to track not-yet-committed files, and Kart disables that concept for simplicity. Our add-dataset command won't use a git index, so to list untracked files you might just want to add something to the various working copy classes to directly list tables from the database, and then filter out ones that are already datasets.

[aadityasinha-dotcom]

Is this issue available?