Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

all: add cockroachdb support #1348

Merged
merged 1 commit into from
May 10, 2019
Merged

all: add cockroachdb support #1348

merged 1 commit into from
May 10, 2019

Conversation

lopezator
Copy link
Contributor

add cockroachdb support

Signed-off-by: David López not4rent@gmail.com

Related PR

#1326

@lopezator lopezator force-pushed the feature/crdb branch 10 times, most recently from 02e5bc9 to 681b114 Compare April 11, 2019 14:51
@lopezator
Copy link
Contributor Author

@aeneasr I cannot figure out where is the problem here with the tests, can you give me a hand with this?

@lopezator lopezator force-pushed the feature/crdb branch 2 times, most recently from 4b67d66 to b620489 Compare April 11, 2019 15:07
@lopezator
Copy link
Contributor Author

Related commit in x: lopezator/x@44fa603

@lopezator lopezator force-pushed the feature/crdb branch 4 times, most recently from c545e64 to bd57a03 Compare April 11, 2019 17:14
@aeneasr
Copy link
Member

aeneasr commented Apr 12, 2019

Thank you for all your hard work! The most recent error seems to be stemming from broken SSL?

--- FAIL: TestXXMigrations (302.66s)
    helper.go:58: Could not connect to database: Unable to connect to postgres (cockroach://root@localhost:26257/defaultdb?sslmode=disable): pq: SSL is not enabled on the server

@lopezator
Copy link
Contributor Author

lopezator commented Apr 12, 2019

I've been doing some advances on this and been able to pass almost all tests, but still some failing. I've been working on my fork to avoid spamming you. It's a draft implementation though:

If you could take a look, it would be great, to collect some feedback.

lopezator@d7ce922

I'll keep you posted if I make some significative advance.

@aeneasr
Copy link
Member

aeneasr commented Apr 15, 2019

I think it's only a couple of copy&paste errors in the circle config. Sometimes you're using postgres instead of cockroach which probably causes weirdness when connecting. Left some comments on your commit.

@lopezator lopezator force-pushed the feature/crdb branch 2 times, most recently from 0ca3eb1 to 5862093 Compare April 15, 2019 10:17
lopezator referenced this pull request in lopezator/hydra Apr 15, 2019
add cockroachdb support

Signed-off-by: David López <not4rent@gmail.com>
@aeneasr
Copy link
Member

aeneasr commented Apr 15, 2019

So this is basically ready for a final review and then merge (after resolving conflicts)?

@lopezator
Copy link
Contributor Author

lopezator commented Apr 15, 2019

Not yet.

I wanted to clean-up some code (ifs) I added for the cockroachdb/postgres dsn overrides etc...

Anyway, correct me If I am wrong but maybe I should first send a PR to ory/x, and then you review/merge it before this, as it doesn't break anything (just adds cockroach compat to x), in order to remove the replace statement from the go.mod file and point to a proper version of ory/x here:

https://github.com/ory/hydra/pull/1348/files#diff-37aff102a57d3d7b797f152915a6dc16R88

I'm working now on passing tests and cleaning up things on x here:

lopezator/x@1b7f14d

I'll send you a PR there when I'm done.

I also opened this issue in sqlx we'll see what happens, because it would help to have a cleaner code removing the need of wrapping some sqlx functionality:

jmoiron/sqlx#511

@lopezator lopezator force-pushed the feature/crdb branch 2 times, most recently from f49f921 to 41d99cf Compare April 17, 2019 14:52
@aeneasr
Copy link
Member

aeneasr commented May 6, 2019

#1423

Copy link
Member

@aeneasr aeneasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few more changes then it should be good to go!

client/migrations/sql/tests/cockroach/1_test.sql Outdated Show resolved Hide resolved
consent/migrations/sql/cockroach/1.sql Outdated Show resolved Hide resolved
consent/migrations/sql/tests/cockroach/1_test.sql Outdated Show resolved Hide resolved
jwk/migrations/sql/tests/cockroach/1_test.sql Outdated Show resolved Hide resolved
oauth2/migrations/sql/tests/cockroach/1_test.sql Outdated Show resolved Hide resolved
@lopezator
Copy link
Contributor Author

PTAL @aeneasr

I changed all the cockroach sql file numbers to match the postgres and the mysql migrations number, and since the tests are just inserts and now share the same index, we can safely delete the cockroach test migration sql files/folders.

I originally reordered some columns on cockroach create table migrations for readability, e.g. (pk as first column, created_at and updated_at as latest columns...). I changed that. Now the columns follow the same exact order as mysql and postgres "finished" databases, so the comparison against them is easier.

@lopezator lopezator force-pushed the feature/crdb branch 3 times, most recently from 9015db2 to 35d57a4 Compare May 8, 2019 08:01
@aeneasr
Copy link
Member

aeneasr commented May 8, 2019

Awesome! I think that's it - once tests pass this will be merged. Thank you for this immense effort!

@aeneasr
Copy link
Member

aeneasr commented May 8, 2019

I think tests fail because we're using the same shared test sqls!

@lopezator
Copy link
Contributor Author

Just for the record, we freeze this until the underlying required change on ory/x got merged to correlate migration sql file ids, and test sql file ids.

Then, just pointing to new ory/x should fix the failing tests.

Just FYI, we packed lopezator/hydra in a container and deployed it on our cloud (to test out the cockroach support), and AFAIK it's working great.

@aeneasr
Copy link
Member

aeneasr commented May 8, 2019

Just FYI, we packed lopezator/hydra in a container and deployed it on our cloud (to test out the cockroach support), and AFAIK it's working great.

That's great news!!!

@lopezator lopezator force-pushed the feature/crdb branch 7 times, most recently from 244459d to 50ea156 Compare May 9, 2019 16:42
@lopezator
Copy link
Contributor Author

PTAL @aeneasr

Pending:

  1. Remove the go.mod replace after merging migratest: add FindMatchingTestMigrations func x#52
  2. Squash the second commit: 50ea156 into the first one, I've left it like this for easier review as it's a big PR to focus only on what changed.

add cockroachdb support

Signed-off-by: David López <not4rent@gmail.com>

Signed-off-by: David López <not4rent@gmail.com>
@lopezator
Copy link
Contributor Author

go.mod updated, rebase&squash done. Almost there! 😅

@aeneasr
Copy link
Member

aeneasr commented May 10, 2019

Screenshot 2019-05-10 at 13 17 32

😎

@aeneasr aeneasr merged commit f8f2363 into ory:master May 10, 2019
@aeneasr
Copy link
Member

aeneasr commented May 10, 2019

Thank you!

@aeneasr aeneasr mentioned this pull request May 10, 2019
2 tasks
@lopezator
Copy link
Contributor Author

Thank you for merging and supporting me during the whole proccess! 🎉 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat New feature or request.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants