Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse also the port from log sources #9460

Closed
wants to merge 1 commit into from

Conversation

jsoriano
Copy link
Member

@jsoriano jsoriano commented Dec 10, 2018

If source is a host, and it contains ip and port, it fails
to index after #8902 with an error like:

{"type":"mapper_parsing_exception","reason":"failed to parse field [log.source.ip] of type [ip]","caused_by":{"type":"illegal_argument_exception","reason":"'127.0.0.1:59835' is not an IP string literal."}

It happens at least with the syslog input.

This change parses the source in network inputs so if it
contains a port it is also added to the event in a separate
field.

@ruflin
Copy link
Member

ruflin commented Dec 10, 2018

I'm wondering if we should call the filed address instead as it could also break if it's hostname like localhost I assume?

This also has an affect on #9435

@jsoriano
Copy link
Member Author

jsoriano commented Dec 10, 2018

I'm wondering if we should call the filed address instead as it could also break if it's hostname like localhost I assume?

Not sure, I guess this is like the usual discussion about having a field that accepts hostnames or ip. In this case I think this is always an IP, but not sure if under some circunstances this can make inverse lookups.

In any case we can think what to do with the host/ip field in another PR and keep this one only for the port.

@ruflin
Copy link
Member

ruflin commented Dec 11, 2018

If we introduce address I wonder if we even have to create ip / port on our end or just can leave it as address? @webmat Perhaps you can chime in here?

@jsoriano
Copy link
Member Author

Oh, you mean using log.source.address instead of log.source.ip to contain host:port and then not adding log.source.port?
address would need to be added to ECS too, right? In ECS I think this can also be confusing because actually and ip is also an address.

@ruflin
Copy link
Member

ruflin commented Dec 11, 2018

For the discussion around address in ECS see: elastic/ecs#247 TL;DR; It's mainly a field for ingestion and should be split up in the processor or later.

For the port: yes, if we use address, I would not add it as a separate field. I see this more as meta information then information that is often queried on.

@jsoriano
Copy link
Member Author

Ok, I'm fine with using address then, shall I wait to the ECS issue to be resolved?

@ruflin
Copy link
Member

ruflin commented Dec 11, 2018

@jsoriano Yes, should go in latest tonight.

@jsoriano
Copy link
Member Author

Created #9487 to go for log.source.address, closing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants