Update configuration_metadata.go

[AamnaZahid]

Support added for the configuration-properties.names property.

Summary

Use Cases

Checklist

• I have viewed, signed, and submitted the Contributor License Agreement.
• I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)
• I have added an integration test, if necessary.
• I have reviewed the styleguide for guidance on my code quality.
• I'm happy with the commit history on this PR (I have rebased/squashed as needed).

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: AamnaZahid / name: Aamna Zahid (00998bd, eb32c22)
• ✅ login: anthonydahanne / name: Anthony Dahanne (d52c2e3)

[anthonydahanne]

Hello @AamnaZahid !
Thanks for your contribution!
Can you please:

• provide context / explain what those changes will enable?
• add unit tests
• sign the CLA

So that we'll be able to review.
Thank you!

[AamnaZahid]

Context/Explanation of Changes

Before:
The function only processed the configuration-properties.classes field by splitting its values and appending them to the classes list. It then matched these classes against SourceType values in metadata groups and properties.

After:
In addition to classes, code now handle configuration-properties.names similarly, where the names values are also processed and stored in a names list.
Both classes and names lists are merged into a combined list, which is used to match against the SourceType or Name fields in metadata Groups and Properties.
This ensures that both properties (classes and names) are additive and provide more flexibility for users to define mappings based on either class or name.

What those changes will enable?

1. Support for configuration-properties.names: Now, users can specify both classes and names in the dataflow-configuration-metadata.properties file. This makes the configuration process more intuitive and documented.
2. Expanded Mapping Capabilities: The new logic allows for mapping based on both SourceType and Name fields, rather than being restricted to just SourceType. This provides more flexibility in defining and processing configuration metadata.

[AamnaZahid]

Hi @anthonydahanne
Hope you are good
I added some test cases.
Plz review it now. Thanks

[dmikusa]

Support for configuration-properties.names: Now, users can specify both classes and names in the dataflow-configuration-metadata.properties file. This makes the configuration process more intuitive and documented.

OK, interesting. This is what I was trying to figure out as well. Where do these files come from, what apps are depending on them.

Can you provide a doc link that explains these files a little more? I want to make sure that there is some context here so we, as maintainers, know how this feature is being used. That'll prevent regressions in the future. Thanks

[anthonydahanne]

is that enough @dmikusa ?
https://dataflow.spring.io/docs/applications/application-metadata/

[AamnaZahid]

Yes

…

On Wed, 23 Oct 2024, 07:22 Anthony Dahanne, ***@***.***> wrote: is that enough @dmikusa <https://github.com/dmikusa> ? https://dataflow.spring.io/docs/applications/application-metadata/ — Reply to this email directly, view it on GitHub <#513 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A324ALWFDQAOGPUCBZ4CVQDZ44B7FAVCNFSM6AAAAABPHAMSK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZQGY4DQMRRGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[anthonydahanne]

@AamnaZahid I fixed a test, and added another one in a separate commit.

[AamnaZahid]

Awesome, thanks for the update!

…

On Thu, 24 Oct 2024, 04:40 Anthony Dahanne, ***@***.***> wrote: @AamnaZahid <https://github.com/AamnaZahid> I fixed a test, and added another one in a separate commit. — Reply to this email directly, view it on GitHub <#513 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A324ALRRZ7EIIP5LRQ2G3KTZ5AXWVAVCNFSM6AAAAABPHAMSK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZTG44DOMBRGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[onobc]

To elaborate a bit more on the link provided by @anthonydahanne .

Here is a little background on the Dataflow dedicated metadata. It is used to drive the UI/API during stream/task definition.

This describes why we have the dedicated metadata:
https://dataflow.spring.io/docs/applications/application-metadata/#dedicated-metadata-artifacts

This describes the metadata file:
https://dataflow.spring.io/docs/applications/application-metadata/#data-flow-configuration-metadata

This describes the metadata label:
https://dataflow.spring.io/docs/applications/application-metadata/#metadata-container-image-label

Here is a concrete example that uses 3 of the pre-built stream-applications (e.g. splitter used below that include said metadata.

The stream is defined as time | splitter | log and you can see in the UI the user is presented w/ only the properties that are in this resulting metadata (otherwise we would have to show them all N of the available Spring Boot config props).

I hope this helps w/ some more context.

Thanks