Conntrack XML parsing is consuming a lot of CPU

[2opremio]

See #1985 (comment)

[2opremio]

@alban Suggests replacing the conntrack command (and in turn parsing its XML output) by talking to the kernel directly through a Netlink socket.

[2opremio]

Found https://github.com/typetypetype/conntrack but the status is not really reassuring ...

[2opremio]

Using netlink is going to be a bit more work than I originally expected due to the incomplete Golang support of NETLINK_NETFILTER and my almost total ignorace about netlink until yesterday.

• https://github.com/vishvananda/netlink doens't support Conntrack directly (I created Add Conntrack support (NETLINK_NETFILTER protocol) vishvananda/netlink#171)
• https://github.com/typetypetype/conntrack/ is in dubious state (I tested it and seems to work though) and doesn't fully support flows either. There's Follow() which reads flows into Conn. However Conn lacks the Orig side of the flow (which we need) and there is no simple or public way to send commands to netfilter (which we may need to implement conntrack --any-nat, currently used by Scope).

My plan is to fork https://github.com/typetypetype/conntrack/ to reuse all the parsing but it may be more sensible to bite the bullet and implement vishvananda/netlink#171 (which I think would take a much longer time). @awh Thoughts?

[2opremio]

After an offline discussion with @awh I've confirmed that supporting a new netlink protocol is a non-trivial task and that we should instead try improving parsing first. I will proceed in the following order, reevaluating performance after each step:

• Try to find a better performing XML library
• Try parsing the textual output of conntrack (i.e. without -o xml)
• Use netlink

[2opremio]

Try to find a better performing XML library

I have failed to find an alternative XML library at all. Moving on to parse the conntrack text manually.

[tomwilkie]

Huh, I thought codec did it, but it doesn't.

https://godoc.org/github.com/ugorji/go/codec

On Tue, 15 Nov 2016 at 12:53, Alfonso Acosta notifications@github.com
wrote:

Try to find a better performing XML library

I have failed to find an alternative XML library at all. Moving on to
parse the conntrack text manually.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#1991 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAbGhQ1NpJUVmd3I1U4XyyBHu7u5WPDsks5q-aspgaJpZM4KrLIo
.