issue with importing beats dashboard

[zaqweb]

I post the questions here link

beats & ES & kibana are 5.4.0 with x-pack trial license

problem is we cannot use any customized index-pattern name in .kibana. no matter use CLI

./import_dashborad -i new-index-pattern

or use XXXbeats.yml configure

dashboards.index: new-index-pattern

are take no effect. the default index-pattern (like metricbeat-*) imported.

And I found the only way to change the pattern name is modify "title" in metricbeats.json in beats-dashboards-5.4.0.zip. but this cause another problem when I open dashboard

Fielddata is disabled on text fields by default. Set fielddata=true on [your_field_name] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory.

[ewgRa]

just my 2 cents, I try to solve this problem and made research...

The problem here, that for example when zip file have several beats, like "heartbeat-", "winlogbeat-" and so on, when you call import_dashboards with "-i" option, for which one beat this option is?

Current logic is change all titles, index and so on, but for "index-pattern" we will have several records (not renamed), numbers of them equals to number of imported beats. Each index pattern records have "fieldFormatMap". I think this current logic is wrong. Each beat have its own index. No?

If we join all beats under one, overwritten index-pattern - we can't handle this "fieldFormatMap".

I suppose that "-i" option can't be used when dashboard have several beats, or there is must be more precise renaming, like - "this beat - this name" and so on.

Correct me if I'm wrong.

ps. During research I come to this fix https://gist.github.com/ewgRa/747344aa7a18e48e2e32eb64b891f2a4 . But I think it is not right solution.

Right solution will be add option rename index for specific beat. What do you think?

[ruflin]

A general problem is on what happens, in case someone is not using the default index pattern. I think when importing dashboards, it should take the index pattern from the config file even if it was changed. But this would also mean modifying parts of the kibana files I think which becomes trickier.

[ewgRa]

@ruflin There is already modifying of Kibana files, for example https://github.com/elastic/beats/blob/master/libbeat/dashboards/importer.go#L263

So, the questions can we modify all beats to one index, or each beat must have its own index.

[ewgRa]

@ruflin Since import_dashboard command removed now in master (during #4409) is it make sense to work on this issue now?

[ruflin]

@ewgRa Good question.

@monicasarbu Perhaps you can comment on this one?

[tsg]

While the import_dashboards command is being removed, the logic is moved under the setup command, so I think the issue still exists. IMO we should support putting all Beats data under the same index pattern, but I'm not sure if a clean solution can be found with the current implementation.

[andyliddle]

This is happening with metricbeat and elasticsearch 6.2.0

my setting in in metricbeat,yml
setup.template.name: "dev-metricbeat-%{[beat.version]}"
setup.template.pattern: "dev-"
setup.dashboards.index: "dev-eth-"

yet the index pattern on the system overview visualisations was still metricbeat-*.

I was able to work around this by manually setting the index_pattern in the Metricbeat-system-overview.json back to "index_pattern":"*" (this is what it was in version 6.1

[urso]

This issue is for an old version of beats, I suggest to try out the latest version of beats. If this is still a problem in the newly released version feel free to reopen this issue. Thanks.