InfluxDB Persistence not working anymore in 3.1.0-M3 build

[aMU5Ed]

I'm using OH 3.1.0 M3 build on Ubuntu 20.04 LTS 64bit on Raspberry Pi 4 (8GB RAM) and Azul Embedded Java 64Bit. With M2 sumSince calculations ware working well like "SM_PV_EigenverbrauchGesamt.postUpdate(SM_PV_EigenverbrauchTag.sumSince(parse("2021-01-01T00:00+01:00[Europe/Vienna]")) as Number". Since OH3 the item name is written as Tag in the Influx DB (version 1.8). Caluclations (with values that have a tag or not) are working in M2. With M3 the calculations only sum up the values that have the Item name stored as tag in InfluxDB. I do not have any tags configured in my OH Persitence config. Also charts do not show the values (which are not used in the sumSince calculation).
Log says: 2021-04-04 15:26:18.493 [INFO ] [b.internal.InfluxDBStateConvertUtils] - Could not find item 'null' in registry

[ole-elo]

I have the same issue here. It is possible to solve it by manually add the "item" tags to influxdb. In my case this is a lot of manual work, because I have a lot in influx.
Is there any "repair" or "migrate" function available? Would it be possible to use the _measurement value if no "item" tag is available as fallback in the influx addon?

[aMU5Ed]

@ole-elo Unfortunatly I do not have any work around. I just downgraded to M2 and it started to work as expected. Hopefully there will be a fix for M4 or release.
@cweitkamp you removed the issue from openhab-core. Which component would be the correct one?

[cweitkamp]

The add-on org.openhab.persistence.influxdb is located in this repository: org.openhab.persistence.influxdb. Thus the issue belongs here.

The cause for your issue may be related to the latest changes: #9943

[aMU5Ed]

@cweitkamp thanks for the hint. Seems that this change could be the root cause. Hoepfully this gets fixed in M4!

[J-N-K]

Is this still true after a new value has been stored? I'm usinf .M3 and don't see these issues

[aMU5Ed]

@J-N-K 7one exampe: values stored in the influx measurement: 1,2,3 without item tag 4,5,6 with item tag. M2: will sum up 1+2+3+4+5+6 M3 will sum up only 4+5+6

[J-N-K]

I can reproduce that now. But the issue must be much older than M3. I need to use data that is older than 150 days to get your issues.

FTR:

logInfo("Test 30", WEZ_hum.sumSince(now.minusDays(30), "influxdb").toString)
logInfo("Test 60", WEZ_hum.sumSince(now.minusDays(60), "influxdb").toString)
logInfo("Test 90", WEZ_hum.sumSince(now.minusDays(90), "influxdb").toString)
logInfo("Test 120", WEZ_hum.sumSince(now.minusDays(120), "influxdb").toString)
logInfo("Test 150", WEZ_hum.sumSince(now.minusDays(150), "influxdb").toString)
logInfo("Test 180", WEZ_hum.sumSince(now.minusDays(180), "influxdb").toString)

Only the last line produces the fault, that's why I didn't see it before (I only tried last 60 days, which is much more than M3).

[microneer]

I've upgraded directly from 2.5.12 to 3.1.0M3 and OH charts don't show any data from before the upgrade.

Am I right that sometime between 2.5.12 and 3.1.0M3, OH started adding data to influxdb with the Item name as a tag? This is an example of what I see:

> show series from Bathroom_Humidity
key
---
Bathroom_Humidity
Bathroom_Humidity,item=Bathroom_Humidity

So I think the problem is more general than rule calculations: 3.1.0M3 just isn't accessing any legacy series without the item= part. In my case, that's everything up till the day before yesterday.

[microneer]

@cweitkamp, I think you're right about it being introduced in #9943. @DerOetzi, I believe the issue lies with lines 60 and 66 of Influx1FilterCriteriaQueryCreatorImpl.java in cdd99c9.

        Select select = select().column("\"" + COLUMN_VALUE_NAME_V1 + "\"::field")
                .column("\"" + TAG_ITEM_NAME + "\"::tag")
                .fromRaw(null, fullQualifiedTableName(retentionPolicy, tableName, hasCriteriaName));

        Where where = select.where();

        if (itemName != null && !tableName.equals(itemName)) {
            where = where.and(BuiltQuery.QueryBuilder.eq(TAG_ITEM_NAME, itemName));
        }

Until cdd99c9 there was no mention of tags in the query. Adding the tag to the query means that historic data is being filtered out?

I'm at the limit of my abilities here, not really able to actually fix it, sorry.

[brevilo]

Should this be added to the 3.1 issue list now that the feature itself (#9943) is going to be released with 3.1?

[J-N-K]

It would probably be better to fix it similar to this

https://github.com/smarthomej/addons/blob/d35d14ee4c28dab496fec2d9806979f500b8575e/bundles/org.smarthomej.persistence.influxdb/src/main/java/org/smarthomej/persistence/influxdb/internal/influx1/InfluxDB1RepositoryImpl.java#L197-L199

[aMU5Ed]

Should this be added to the 3.1 issue list now that the feature itself (#9943) is going to be released with 3.1?

That would be nice. Actually I am in a dead end with M2 until a fix is ready on that issue

[Intenos]

I have the same issue and would appreciate a fix as well.

[wborn]

Can you have a look at this @DerOetzi or @lujop?

[lujop]

I will try to take a look.
@microneer Can you enable trace log level for the org.openhab.persistence.influxdb and append the query that is send to InfluxDB

[J-N-K]

Please see my post above. The problem is not the query but how the response is handled.

[lujop]

Please see my post above. The problem is not the query but how the response is handled.

@J-N-K I'm not sure without reproducing it. Because althought your suggested changes are cleaner I think that they do exactly the same as the current code.
Because in current code when columnName is not found it uses the itemName variable that was previously set.

openhab-addons/bundles/org.openhab.persistence.influxdb/src/main/java/org/openhab/persistence/influxdb/internal/influx1/InfluxDB1RepositoryImpl.java

Line 174 in 183fb0e

String itemName = series.getName();

openhab-addons/bundles/org.openhab.persistence.influxdb/src/main/java/org/openhab/persistence/influxdb/internal/influx1/InfluxDB1RepositoryImpl.java

Lines 202 to 204 in 183fb0e

	if (itemNameColumn != null) {
	itemName = (String) valuess.get(i).get(itemNameColumn);
	}

[lujop]

Excuse me @J-N-K I've just seen it just after writing the comment. I didn't realize the requireNonNullElse

[J-N-K]

No, it does not.

[lujop]

I will make a PR with J-N-K suggested change, it looks good to me but I will do a small check.
The problem is with data that was written with InfluxDB addon before the 3.0.0 version that didn't set the item tag name.

[lujop]

If someone is able to test I attached a binary in that PR #10680
I tried to simulate in an InfluxDB 1.0 but better if someone with the problem can confirm.

[Intenos]

Hi lujop,
Very great, that did it for me. It works. Thanks a lot for the fast fix.