External Mapping Source Processor

External Mapping Source Processor [externalMapping]

This processor is similar to Translate as it's used to map one field's value to one or multiple values of the target field. Unlike Translate, it uses an external mapping (dictionary), accessible by a URL. The external mapping is loaded using an HTTP GET request to a configured URL and is cached internally. The internal copy is refreshed once in a defined period (60 sec by default).

• sourceField

• targetField

• mappingSourceUrl

• mappingRefreshPeriodInMillis (default: 60000)

• externalMappingConnectTimeout (default 5000)

• externalMappingReadTimeout (default: 10000)

It's possible to disable periodic mapping refresh by setting mappingRefreshPeriodInMillis to -1

Example

{
    "steps": [
        {
            "externalMapping": {
                "config": {
                    "sourceField": "author",
                    "targetField": "books",
                    "mappingSourceUrl": "http://localhost:8080/books"
                }
            }
        }
    ]
}

Mapping File Example

To use this processor, an external mapping should be in .properties format (key-value pairs delimited by = sign)

Charles Dickens = Oliver Twist, A Christmas Carol, The Chimes
Jack London = White Fang, Martin Eden
Ernest Hemingway = For Whom the Bell Tolls, A Farewell to Arms, The Old Man and the Sea
Jack London = The Sea Wolf

Upload Mapping File to Logz.io

To use this processor in Logz.io, upload a mapping file to Logz.io to use in your pipeline.