OPC-UA input plugin roadmap

[sjwang90]

Consolidation of OPC-UA plugin issues/feature requests. A lot of bugs and feature requests have arisen since inputs.opcua was released. In this issue would like to consolidate what are library issues vs Telegraf bugs vs plugin feature requests

Background:

• A good number of issues seem to be caused by the gopcua library. However this seems to be the only Go OPC UA library out there. Will need to determine the most efficient way on moving forward with these issues.
• the number of OPC-UA devices out there are endless. The plugin was built and tested using the Prosys Simulator and that is what Telegraf maintainers use to troubleshoot as well. We need to create a group of community users that are willing to troubleshoot and test OPC-UA plugin issues.

Bugs / issues

• Support opcua auto-reconnect and don't use registered reads #11559

gopcua Library issues

• [inputs.opcua] Error in plugin: RegisterNodes Read failed: The operation timed out. StatusBadTimeout (0x800A0000) #8890
  • Status: More info in comment [inputs.opcua] Error in plugin: RegisterNodes Read failed: The operation timed out. StatusBadTimeout (0x800A0000) #8890 (comment)
• [inputs.opcua] StatusBadResponseTooLarge during connection #9163
  • Status: opened StatusBadResponseTooLarge during connection gopcua/opcua#435
• Telegraf OPCUA runtime error: invalid memory address or nil pointer dereference #9164
  • Status: Waiting on testing feedback - May have been addressed from library upgrades
• [inputs.opcua] StatusBadSessionIDInvalid  #8583 - Receives StatusBadSessionIDInvalid error message even though everything is working fine.
  • Status: Waiting on user response - but issue may possibly due to gopcua, getData function within the opcua client on line 455
• Read failed: opcua: invalid extension object with id ns=4;i=5086 #9900
  • Status: opened Potential Bug: StatusBadTimeout Error when attempting connection using numeric id gopcua/opcua#478

Feature Requests

• Title: Workarounds settings subtable
  Description: Add an [inputs.opcua.workarounds] subtable similar to the Modbus input plugin workarounds to address device settings that may differ from the OPC foundation specifications. This will help us address settings that may differ for non-tested devices.
  Type: Feature Request, Device
  Status:
  Priority: High
  related issues: OPC UA input plugin erroneous bad data (code 0xC0) #8826 (comment)
  related PRs: fix: Accept non-standard OPC UA OK status by implementing a configurable workaround #10384

• Title: Quality should be a tag not field
  Description: Quality is a commonly filtered metric and should be stored as a tag.
  Type: Data type
  Status: Workaround - use converter processor
  Priority: Low
  related issues: [inputs.opcua] Quality should be a tag and not a field #9405
  related PRs: feat: Quality as a tag workaround #10458

• Title: Events / subscription based plugin
  Description: OPC UA is subscription based, this behavior will be expected by a lot of plugin users
  Type: Feature Request
  Status: Ideally would like to implement in existing plugin
  Priority: High but large work
  related issues: OPC UA Events based plugin #8083
  related PRs:

• Title: Collect additional attributes such as datatype
  Description: Plugin currently only collects main value, but would be useful to also read in other attributes such as datatype.
  Type: Metric collection
  Status:
  Priority: Low
  related issues: inputs.opcua: Add metric for datatype #8669
  related PRs:

• Title: Support for ExtensionObjects
  Description:
  Type: Feature Request
  Status:
  Priority: Medium / High
  related issues: OPC-UA Client: Support for ExtensionObjects #9911
  related PRs:

• Title: Add Hierarchical configuration
  Description: Be able to set node configuration as tag_name=foo_* to read a stream of available tags from OPC UA server.
  Type: Metric collection
  Status:
  Priority: Medium
  related issues: Add Hierarchical configuration to OPC UA plugin #8275
  related PRs:

Device types

Fanuc - @JoSchott on community.influxdata.com
KUKA DeviceConnector 2 - @mirkocomparetti-synthesis in #9163
OPC Red Lion DA 10 servers - @jnangle server issue addressed in #9807

[magiconair]

Hi, I am one of the main authors and maintainers of the https://github.com/gopcua/opcua library and I am really happy that this library is being used in more areas. I was caught a bit by surprise by the inclusion to influx.

So far we have been using and testing this mostly in our factories at Northvolt with Siemens and Beckhoff PLCs and other maintainers were able to test with a different setup. But we for sure didn't cover the entire OPC/UA landscape with all their edge cases.

Yesterday, we have released the v0.2.0 version of the library which includes auto-reconnect and a cleanup of the subscription code. This was a year in the making since it was a big refactor but we've finally released this and I would like to go back to a faster cadence of smaller releases and bug fixes.

It would be great if we can address the issues you are finding and make the library more stable and useful for a greater community but we need your help since we usually don't have access to the equipment or the software.

[magiconair]

What usually helps to troubleshoot the issues are tcpdumps.

[jnangle]

Hello @magiconair, thank you for finding the Influx group and for providing the updates to the gopcua library. I opened a ticket in that github group for errors involving connections to the Red Lion OPC UA server using numeric identifiers. You had asked for tcpdumps for a working connection, but I haven't had the time to get back my system and run tcpdump. I understand that that is a valuable tool for you, I'll do my best to get you some data today.

[magiconair]

@jnangle Thank you!

[sjwang90]

@magiconair Thanks for reaching out! Yes this OPC UA plugin is becoming pretty popular with influx users. Here are some things I wanted to run by you.

I'll add the following items to the plugin readme:

• Devices that have been tested
  • List out devices that the gopcua library is consistently tested with (the list you have included above). Influx currently does not do exhaustive testing with the plugin but will update this list when we do.
  • This will help users understand what devices are fully tested. If they are encountering issues on other devices outside of those listed, open an issue.
• Troubleshooting steps

Library improvements:
Any plugin issues that could be a problem with the gopcua library we'll make sure there's an issue opened in your repo as well.

[magiconair]

At Northvolt we run the code in production in all our factories against Siemens S7-1500 PLCs, Beckhoff PLCs and some Kepware gateways. We are in the process of migrating from v0.1.13 to v0.2.x.

[magiconair]

I'll update the gopcua readme as well. I think this is a good idea.

[magiconair]

https://github.com/gopcua/opcua#tested-platforms

[R290]

@sjwang90 Thanks for creating this overview! Are you able to find some time to update it?

[sjwang90]

@R290 Just updated. Let me know if there's anything that's missing. That's for all your PRs for the OPC-UA work.

[BSommen]

@magiconair, any expected timing on adding the "Events / subscription based plugin" feature to the plugin?
In our factory we would like to monitor mutiple (200-500) tags and write them to Influx.
At the moment we do this with NodeRed, but it fails on the amount of tags he has to write to the DB.
I'm hoping that Telegraf and the integration with OPC UA, has a higher performance on this?

While waiting on this new feature, any suggestion to do this?

[Jayclifford345]

I am just linking two community requests for adding subscriptions. I believe this is quickly elevating to the top of the feature list:

1. https://community.influxdata.com/t/telegraf-missing-data/25703/3
2. https://community.influxdata.com/t/telegraf-opc-ua-ingest-and-visualisation-slow/25698

[kung-foo]

@magiconair, any expected timing on adding the "Events / subscription based plugin" feature to the plugin? In our factory we would like to monitor mutiple (200-500) tags and write them to Influx. At the moment we do this with NodeRed, but it fails on the amount of tags he has to write to the DB. I'm hoping that Telegraf and the integration with OPC UA, has a higher performance on this?

While waiting on this new feature, any suggestion to do this?

@BSommen I think you should direct the question to the Telegraf maintainers. @magiconair (and myself) maintain the underlying OPC-UA library (separate from Telegraf), which already supports "subscription" based interfaces (at Intelecy we use it to stream process data with millisecond latency into ClickHouse for up to 40k tags in a single process).

[powersj]

@LarsStegman @R290

Looking for some wisdom and guidance on the remaining four remaining OPC-UA issues listed in the original issue report:

• [inputs.opcua] StatusBadSessionIDInvalid  #8583
  • There was an upstream issue filed around this that is now closed.
  • Seemed to come from a user using the wrong identifier type, but also a change to allow auto-reconnect. Telegraf has that ability now via Support opcua auto-reconnect and don't use registered reads #11559.
  • Should this issue be marked closed/resolved?
• OPC-UA Client: Support for ExtensionObjects #9911
  • Based on one of the responses from the library maintainer, these objects need to be declared and registered at compile time. As a result, I am not sure how telegraf would support this without first adding the dynamic definition in the underlying library. How useful/important is this support?
• Read failed: opcua: invalid extension object with id ns=4;i=5086 #9900
  • Is this one referenceing support for extensionObject as well?
• Add Hierarchical configuration to OPC UA plugin #8275
  • This final issue revolves around setting and using regex values to avoid more complex configurations. Is there a way to do discovery of what fields are there to support this type of syntax?

Thanks!

[LarsStegman]

• [inputs.opcua] StatusBadSessionIDInvalid  #8583 is probably solved, considering the solution was created.
• OPC-UA Client: Support for ExtensionObjects #9911 shouldn't be closed I think. The way I understand it, it is possible to register a type with the library while the program is running. The more difficult part is in how to create the go types to register. Maybe we could create some kind of build script that can create the types from a configuration file or something. Another solution would be to create types in run, but I am not sure if that is possible in golang.
• Read failed: opcua: invalid extension object with id ns=4;i=5086 #9900 yes
• Add Hierarchical configuration to OPC UA plugin #8275 yes, this is definitely possible. You can browse the address space (recursively) to discover all the available nodes.

[magiconair]

Re #9911: it is possible to create a new type at runtime (https://pkg.go.dev/reflect#StructOf). Whether it's useful is a different question. It might actually be more useful in the Telegraf context than in our physical factory context.

[LarsStegman]

We might not even need to build the types actually. We might be able to look at the object properties and just log those. Not sure if that will work with the library though.

[magiconair]

If you want to log field values of structured types (extension objects) you need to read them in full first and they need to match the type on the OPC/UA server exactly. The only way to do this with the library is to have a corresponding Go type.