[jdbc] Fix tableCaseSensitiveItemNames for PostgreSQL/TimescaleDB

[JonathanvdGHU]

Case-sensitive tables working for TimescaleDB database via JDBC persistence service

Description

The following issue is resolved. My implementation is maybe not the best one because I didn't take the time to understand everything. So I don't mind if someone makes some small adjustments and makes a new pull request :)

[jlaur]

Thanks for the fix. I'll review this in the weekend.

[JonathanvdGHU]

Thanks for the fix. I'll review this in the weekend.

Good to hear. Is it a big problem I made a mess with the commits? I am trying to do a rebase but it's not really working.

[jlaur]

Good to hear. Is it a big problem I made a mess with the commits?

It seems some commits are incorrectly signed. If there is at least one correctly signed commit, I can use that when squash-merging. Otherwise you will need to rebase to fix it.

[JonathanvdGHU]

Good to hear. Is it a big problem I made a mess with the commits?

It seems some commits are incorrectly signed. If there is at least one correctly signed commit, I can use that when squash-merging. Otherwise you will need to rebase to fix it.

I fixed it. This was my first rebase so I am also a little proud about myself xD

[jlaur]

The fix looks valid, but I have provided some feedback on the approach. I think it would be preferable to keep the DTO's unchanged and instead directly fix the queries which only concern PostgreSQL/TimescaleDB.

[JonathanvdGHU]

The fix looks valid, but I have provided some feedback on the approach. I think it would be preferable to keep the DTO's unchanged and instead directly fix the queries which only concern PostgreSQL/TimescaleDB.

I implemented your requested changes :)

[jlaur]

I implemented your requested changes :)

Thanks! I'll take one more look. In the meantime, I see there are formatting changes for comments, why is that? I just want to avoid having this changed back and forth depending on some individual IDE settings, if that's the case. It also disturbs diff'ing. 😉

[jlaur]

Last batch of comments, I hope. 🙂

[jlaur]

I think it would be preferable to keep the DTO's unchanged and instead directly fix the queries which only concern PostgreSQL/TimescaleDB.

I can see now after the latest changes the dilemma here when overridden queries differ only in table name being quoted. With the approach I suggested, those queries are now overridden, which feels like duplication. For example:

sqlDropTable = "DROP TABLE \"#tableName#\"";

overriding:

    protected String sqlDropTable = "DROP TABLE #tableName#";

With your original approach, instead we would have had to overload the methods only to call a different DTO method, which would have been even worse.

So it seems we would need to do something different to fully avoid duplication. What would you think about introducing a protected method getTableName in JdbcBaseDAO, something like this:

    protected String getTableName(ItemsVO vo) {
        return vo.getTableName();
    }

which could then be used from all query methods, i.e. getTableName(vo) rather than vo.getTableName()?

This would allow for overriding this method for PostgreSQL to return the quoted table name, but without changing any query methods or queries when they are not otherwise needed to be adapted for PostgreSQL?

This is just from the top of my head and all optional, and if you prefer to stick with your current implementation, that's fine as well.

[JonathanvdGHU]

I implemented your requested changes :)

Thanks! I'll take one more look. In the meantime, I see there are formatting changes for comments, why is that? I just want to avoid having this changed back and forth depending on some individual IDE settings, if that's the case. It also disturbs diff'ing. 😉

It was the IDE 😅.

[JonathanvdGHU]

I think it would be preferable to keep the DTO's unchanged and instead directly fix the queries which only concern PostgreSQL/TimescaleDB.

I can see now after the latest changes the dilemma here when overridden queries differ only in table name being quoted. With the approach I suggested, those queries are now overridden, which feels like duplication. For example:

sqlDropTable = "DROP TABLE \"#tableName#\"";

overriding:

    protected String sqlDropTable = "DROP TABLE #tableName#";

With your original approach, instead we would have had to overload the methods only to call a different DTO method, which would have been even worse.

So it seems we would need to do something different to fully avoid duplication. What would you think about introducing a protected method getTableName in JdbcBaseDAO, something like this:

    protected String getTableName(ItemsVO vo) {
        return vo.getTableName();
    }

which could then be used from all query methods, i.e. getTableName(vo) rather than vo.getTableName()?

This would allow for overriding this method for PostgreSQL to return the quoted table name, but without changing any query methods or queries when they are not otherwise needed to be adapted for PostgreSQL?

This is just from the top of my head and all optional, and if you prefer to stick with your current implementation, that's fine as well.

I also thought it was duplication so that's why I added the function getQuotedTableName to the DTO's. I think it's a good idea but to be honest, I don't have much time to do it. So maybe someone can change it later to improve the code.

[JonathanvdGHU]

Last batch of comments, I hope. 🙂

I think it's now done.

[jlaur]

LGTM - thanks! I'll wait for confirmation that everything still works correctly after the latest changes. 🙂

[jlaur]

I also thought it was duplication so that's why I added the function getQuotedTableName to the DTO's.

The problem with that is that you would need to overload all methods in order to call the alternative DTO method, so that would generate even more duplication - it would just be duplicated methods rather than duplicated query strings.

I think it's a good idea but to be honest, I don't have much time to do it. So maybe someone can change it later to improve the code.

That's fine, I'll see if I can find some time to refactor this later.

[JonathanvdGHU]

LGTM - thanks! I'll wait for confirmation that everything still works correctly after the latest changes. 🙂

Thanks for the patience for all this 😌

[jlaur]

Thanks for the patience for all this 😌

Likewise. 😉 Before merging, besides test confirmation, there is one more thing needed: When checking the failed DCO, I noticed that your real (full) name is missing in your signoffs. You can fix both issues by rebasing your commits. If you click "Details" right to "DCO", you can see how to do that. If you are not comfortable with that, I can fix the signoff when squash-merging the PR. In that case I'd just need your full name. Thanks!

[JonathanvdGHU]

Thanks for the patience for all this 😌

Likewise. 😉 Before merging, besides test confirmation, there is one more thing needed: When checking the failed DCO, I noticed that your real (full) name is missing in your signoffs. You can fix both issues by rebasing your commits. If you click "Details" right to "DCO", you can see how to do that. If you are not comfortable with that, I can fix the signoff when squash-merging the PR. In that case I'd just need your full name. Thanks!

My full name is included in my email, that you can see in the signoff. I'll update it if I return for any future bug fixes! 😅. So you can use that name for squash-merging. I tested on another branch how to rebase successfully (there are a lot of merge conflicts) but I think it's the easiest way if you do it.

[JonathanvdGHU]

Thanks for the patience for all this 😌

Likewise. 😉 Before merging, besides test confirmation, there is one more thing needed: When checking the failed DCO, I noticed that your real (full) name is missing in your signoffs. You can fix both issues by rebasing your commits. If you click "Details" right to "DCO", you can see how to do that. If you are not comfortable with that, I can fix the signoff when squash-merging the PR. In that case I'd just need your full name. Thanks!

What is your estimate when the pull request will be merged?

[jlaur]

What is your estimate when the pull request will be merged?

I"ll merge when you have confirmed that everything is still working after the latest changes.

[JonathanvdGHU]

What is your estimate when the pull request will be merged?

I"ll merge when you have confirmed that everything is still working after the latest changes.

Oh a small misunderstanding I guess. Everything related to the bug is working now, so it's fixed.

[JonathanvdGHU]

@jlaur I double checked and found out I missed one table in the PostgresqlDOA😶. Without the fix you can't see the graphs in OpenHAB, beside that everything works fine. The fix is a oneliner that I commited on the same branch. Would you suggest to make a new pull request or are you gonna take it with your refactor (in case you are going to do it)?

[jlaur]

@jlaur I double checked and found out I missed one table in the PostgresqlDOA😶. Without the fix you can't see the graphs in OpenHAB, beside that everything works fine. The fix is a oneliner that I commited on the same branch. Would you suggest to make a new pull request or are you gonna take it with your refactor (in case you are going to do it)?

It would be highly appreciated if you could create a new PR with the fix. Using the same branch is a bit messy, because all the previous commits will appear again. Since it's a one-liner and since we use squash-merge, it will be fine this time. For your next contributions (hoping there will be some 😉) it is recommended to create a new branch from your local main branch after pulling latest changes.

[jlaur]

@JonathanvdGHU - I think the quotes here were a mistake, since the table names are not used for referencing the tables directly, but rather being used to query the information schema. Do you agree? See also #17597.

[JonathanvdGHU]

I understand what you mean, but to be honest I am not sure how postgreSQL reacts when you remove them.

[jlaur]

It depends how it internally stores the table names. Would it possibly for you to run a query so we can see?

[JonathanvdGHU]

I will try it when I have the time.

[JonathanvdGHU]

It's the same case here. (it's PostgreSQL btw, I know a little bit confusing)

[jlaur]

Can you try:

SELECT table_name
FROM information_schema.tables

I would expect it to return e.g. TotalUsage rather than "TotalUsage".

Or to be specific as in your query (single quotes were missing):

SELECT *
FROM information_schema.tables
WHERE table_name = 'TotalUsage'

[JonathanvdGHU]

2. (still postgreSQL)

[jlaur]

Excellent, thanks. So indeed the double-quotes should not be included in those queries, thus already fixed as part of #17597.