Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate alembic and add hg38 DB schema as initial revision #81

Merged
merged 8 commits into from
Apr 21, 2020

Conversation

oltionchampari
Copy link
Contributor

@oltionchampari oltionchampari commented Mar 3, 2020

This PR depeneds on datavisyn/tdp_core#292

Closes #64

Summary

  • Integrated alembic according to the instructions in https://wiki.datavisyn.io/tdp/extension-points/db-migration.
  • Exported the hg_38 schema from the postgres 12 database and imported it in the upgrade function to be executed as the initial revision.
  • Tested running db-migration:upgrade:head:tdp_publicdb both in a postgres 10-alpine container and and postgres 12.1 container. In both cases the schema gets imported and the tdp_publicdb_alembic_version table with the current revision ID gets created in the public schema.

The following script to create the tdp_publicdb_alembic_version table needs to be executed in the hg_38 postgres 12.1 database so the alembic upgrade command does not try to upgrade it.

CREATE TABLE public.tdp_publicdb_alembic_version
(
    version_num character varying(32) COLLATE pg_catalog."default" NOT NULL,
    CONSTRAINT tdp_publicdb_alembic_version_pkc PRIMARY KEY (version_num)
)
WITH (
    OIDS = FALSE
)
TABLESPACE pg_default;

ALTER TABLE public.tdp_publicdb_alembic_version
    OWNER to publicdb;

INSERT INTO public.tdp_publicdb_alembic_version(version_num)
	VALUES('b477004c3673');
  • Already tested the above script in a 12 and 10 postgres container.

To test

  1. Switch tdp_core branch to the mp/291_db_migration (necessary since the alembic integration is not yet in the develop branch). --> merged into develop
  2. Create a postgres12.1 container and a matching pgadmin container in your docker-compose.yml file.
  3. Open pgadmin on the browser and add ordino and postgres users/roles.
  4. Execute alembic command db-migration:upgrade:head:tdp_publicdb.
  5. Check if the schema looks okay and also that the table tdp_publicdb_alembic_version has been created.

@oltionchampari oltionchampari added type: feature New feature or request release: minor PR merge results in a new minor version labels Mar 3, 2020
@oltionchampari oltionchampari requested a review from thinkh March 3, 2020 08:49
Copy link
Member

@thinkh thinkh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a minor request to add a comment to the commented line.

@thinkh thinkh changed the title Integrate alembic and add hg_38 schema as the initial revision. Integrate alembic and add hg_38 schema as the initial revision Mar 3, 2020
@thinkh thinkh changed the title Integrate alembic and add hg_38 schema as the initial revision Integrate alembic and add hg38 DB schema as initial revision Mar 3, 2020
@oltionchampari
Copy link
Contributor Author

oltionchampari commented Apr 20, 2020

Running the first alembic revision in an empty database produces the below error when querying the data.

materialized_view

This error can be fixed by refreshing the view right after creating it:

REFRESH MATERIALIZED VIEW cellline.mutationalburden;

Source: scenic-views/scenic#184

Copy link
Member

@thinkh thinkh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the integration and testing. I ran a test now and it works well. 👍

@thinkh thinkh merged commit 6e09e23 into develop Apr 21, 2020
@thinkh thinkh deleted the ochampari/64_database_migrations branch April 21, 2020 15:22
@dvvanessastoiber dvvanessastoiber mentioned this pull request Aug 13, 2020
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release: minor PR merge results in a new minor version type: feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants