sql lite schema migration

[khoroshevskyi]

After the new release (0.5.0), I noticed that you updated the SQL table schema. This introduces a problem: the previous SQL files with cached data no longer work, causing all packages dependent on pyBiocFileCache to crash in case if they are using already existed cache file.

Do you have any SQL schema migration scripts or an easy way to handle this without losing data?

[jkanche]

Hi @khoroshevskyi, sorry i overlooked at the schema migrations!

Published a new version, to test install:

pip install pybiocfilecache==0.5.2-dev1

Can you try this and let me know if you run into any issues? Thanks.

[khoroshevskyi]

Thank you @jkanche

Unfortunately, I faced with another issue:

Migration failed: Unsupported migration path: 0.99.4 -> 0.5.0
Failed to run migrations: Unsupported migration path: 0.99.4 -> 0.5.0
Traceback (most recent call last):
  File "/home/bnt4me/virginia/venv/geniml_dep/bin/geniml", line 8, in <module>
    sys.exit(main())
  File "/home/bnt4me/virginia/venv/geniml_dep/lib/python3.10/site-packages/geniml/cli.py", line 128, in main
    bbc = BBClient(cache_folder=args.cache_folder)
  File "/home/bnt4me/virginia/venv/geniml_dep/lib/python3.10/site-packages/geniml/bbclient/bbclient.py", line 58, in __init__
    self._bedfile_cache = BiocFileCache(os.path.join(cache_folder, DEFAULT_BEDFILE_SUBFOLDER))
  File "/home/bnt4me/virginia/venv/geniml_dep/lib/python3.10/site-packages/pybiocfilecache/cache.py", line 78, in __init__
    self.migrator.migrate()
  File "/home/bnt4me/virginia/venv/geniml_dep/lib/python3.10/site-packages/pybiocfilecache/migrations/migration.py", line 112, in migrate
    raise ValueError(f"Unsupported migration path: {current} -> {target_version}")
ValueError: Unsupported migration path: 0.99.4 -> 0.5.0

And because of this, I have few questions related to versioning, that I would like just clarify for myself.

Background:

In previous releases, the database version was 0.99.4, which is the most recent version supported by the current R BiocFileCache package. You can find the supported versions listed in the biocFlieCahe utils

The same version was in 0.4.1

In version 0.5.1, there was introduced tags column, and unfortunately, I can't find this column in R package.

Questions:

1. Should we maintain the same database versioning as the R package?
2. Are tags column included in the R version of the database as well?
3. I am a little bit confused with versioning, could you please clarify it?

In our package, we are aiming to make caching compatible between R and Python. Since I am not deeply familiar with the R BiocFileCache, I wanted to clarify these points.

I am happy to help if needed.

[jkanche]

actually never mind, i'm going to drop some of the newer fields to maintain consistency with the R cache and spin off a separate package for the usecases in the recent changes.