Fix DataOutputAgent so that it can output items with multiple categories

[knu]

The to_xml method encodes { "category": ["a", "b"] } as follows:

<item>
 <category>
  <category>a</category>
  <category>b</category>
 </category>
</item>

Instead of this:

<item>
 <category>a</category>
 <category>b</category>
</item>

Even if category is a singular noun. This feature prevents DataOutputAgent from emitting multiple <category> elements (or <enclosure>, etc.) properly, so I've added a tweak to fix the resulted XML document.

I know the code in the current form is far from optimal, so I think we'll have to revisit here soon or later...

[dsander]

That makes total sense, I think we have a few related issues/feature requests in the past.

Turns out one can parse XML with regular expressions 😉 The worst that could happen is that the regex is not matching and the XML is not transformed as expected, right? If you see any potential that the gsub chain could raise an exception it might be worth rescuing it and returning the XML generated by to_xml.

[knu]

Thanks for the review. These gsub calls should be safe and would never raise exceptions.

[knu]

I think we should implement a new XML generator that generates a DOM tree you can manipulate before serializing to the final string form. The to_xml method is too wild to handle.

[dsander]

That sounds like a good idea but also some work 😄 This could also allow us to add special tags to embed HTML/XML in CTAGS

[mightymt]

I have a question about the current implementation here. As far as I can see, arrays are only handled as described above if they are defined in the agent configuration at design time.
However when an array is passed from an incoming event it is always implicitly joined within a single "category" element instead of there being multiple "category" elements.

Is this by design or am I doing something wrong?

[dsander]

@mightymt Are you accessing the array with the as_object filter like in the spec here? If you are using {{ some_key }} liquid will join the array into a string.

[mightymt]

@dsander Thanks for your reply, that's what I was looking for! Didn't know about this filter.