Adding Datawarehouse management

[moolitayer]

Adding Datawarehouse management.
Commits:

• Datawarehouse manager with an empty refresh
• Ems Datawarehouse add/delete/display

Based on bits from the current hawkular manager.

Intent: A new provider in the system. The end goal is using it to attach events, metrics and inventory to other providers in the system. One Datawarehouse manager can serve many target providers. Currently there is one hawkular implementation. elastic search is planned next.

To view the new provider screens in the UI, set in settings.yml (or settings.local.yml)

:product:
  :datawarehouse_manager: true

add

list

view

[Fryguy]

@blomquisg @Ladas @durandom Please also review.

[moolitayer]

cc @simon3z @cben

[durandom]

@martinpovolny is this a problem, that route key and singular_route_key are the same?

I see that MiddlewareManager has that too.

[moolitayer]

@martinpovolny @himdel It seems to only works this way, I'm not what are the implications...

[himdel]

Honestly me neither, but both CloudManager and MiddlewareManager do a similar thing, so.. :)

[durandom]

so... leave until it breaks :)
no, ok, fine then

[Ladas]

the definitions look correct 👍

@moolitayer you have several valid CI failures there, let us know if you need any help with fixing them :-)

[moolitayer]

Thanks @Ladas. I started fixing those. I'm not sure why I'm getting this:

  1) ExtManagementSystem validates across sub-classes, from vmware to manageiq/providers/hawkular/datawarehouse_manager duplicate hostname
     Failure/Error: manager = FactoryGirl.build(t, :hostname => @ems_vmware.hostname)

     ArgumentError:
       Factory not registered: manageiq/providers/hawkular/datawarehouse_manager
     # /home/mtayer/.rvm/gems/ruby-2.3.0/gems/factory_girl-4.5.0/lib/factory_girl/registry.rb:24:in `find'
     # /home/mtayer/.rvm/gems/ruby-2.3.0/gems/factory_girl-4.5.0/lib/factory_girl/decorator.rb:10:in `method_missing'
     # /home/mtayer/.rvm/gems/ruby-2.3.0/gems/factory_girl-4.5.0/lib/factory_girl.rb:76:in `factory_by_name'
     # /home/mtayer/.rvm/gems/ruby-2.3.0/gems/factory_girl-4.5.0/lib/factory_girl/factory_runner.rb:12:in `run'
     # /home/mtayer/.rvm/gems/ruby-2.3.0/gems/factory_girl-4.5.0/lib/factory_girl/strategy_syntax_method_registrar.rb:20:in `block in define_singular_strategy_method'
     # ./spec/models/ext_management_system_spec.rb:414:in `block (6 levels) in <top (required)>'

[moolitayer]

cc @lucasponce @pilhuhn

[Ladas]

@moolitayer here it goes through all subclasses of ExtManagementSystem https://github.com/Ladas/manageiq/blob/7fb40688b4401f772490ec18f1e011660d51cd0b/spec/models/ext_management_system_spec.rb#L395-L395

So it takes automatically also your new Ems class. You can add the factory and make it pass, or just list it in the exceptions few lines below (with todo to add the specs later). Try if it's enough to add the factory, but I think you will need few actions implemented to pass some of those specs.

[moolitayer]

@miq-bot add_label wip
Moving to work on event collection for a poc, will catch up on this

[miq-bot]

<pr_mergeability_checker />This pull request is not mergeable. Please rebase and repush.

[lucasponce]

Hi @moolitayer I have asked @pilhuhn to review more the architectural aspects and the relation between the datawarehouse and middleware provider.
From my side I am going to review more details at implementation level.

[pilhuhn]

Why 0 ?

[moolitayer]

Back from reserves duty and continuing with this effort

[moolitayer]

@durandom @Fryguy @lucasponce @blomquisg I got comments from several people on the name (datawarehouse) not being adequate. One comment was that datawarehouses is a specific term for things that persist history for very long time. I would like to go for something more generic like datastore.

[durandom]

Datastore could be misleading as Vmware also has that name in it's terminology. But I get the point of DWH being to specific

[moolitayer]

@blomquisg I'm hoping this can be merged soon given that it is skeletal and hidden.
note this no longer has an sql migration(I can't seem to remove it manually).

[moolitayer]

@blomquisg can we merge this skeletal pr or is there anything you want changed?
(the provider screens are hidden)

[dclarizio]

@martinpovolny @h-kataria please suggest some specs that could be added to cover the UI areas added here. Thx, Dan

[miq-bot]

Checked commits moolitayer/manageiq@717d498~...04e5ef6 with ruby 2.2.5, rubocop 0.37.2, and haml-lint 0.16.1
32 files checked, 19 offenses detected

app/helpers/application_helper/toolbar/ems_datawarehouse_center.rb

• ❗ - Line 31, Col 3 - Style/IndentArray - Indent the right bracket the same as the first position after the preceding left parenthesis.
• ❗ - Line 33, Col 5 - Style/IndentArray - Use 2 spaces for indentation in an array, relative to the first position after the preceding left parenthesis.
• ❗ - Line 3, Col 5 - Style/IndentArray - Use 2 spaces for indentation in an array, relative to the first position after the preceding left parenthesis.
• ❗ - Line 47, Col 3 - Style/IndentArray - Indent the right bracket the same as the first position after the preceding left parenthesis.
• ❗ - Line 49, Col 5 - Style/IndentArray - Use 2 spaces for indentation in an array, relative to the first position after the preceding left parenthesis.
• ❗ - Line 62, Col 3 - Style/IndentArray - Indent the right bracket the same as the first position after the preceding left parenthesis.
• ❗ - Line 64, Col 5 - Style/IndentArray - Use 2 spaces for indentation in an array, relative to the first position after the preceding left parenthesis.
• ❗ - Line 75, Col 11 - Style/AlignHash - Align the elements of a hash literal if they span more than one line.
• ❗ - Line 79, Col 3 - Style/IndentArray - Indent the right bracket the same as the first position after the preceding left parenthesis.

app/helpers/application_helper/toolbar/ems_datawarehouses_center.rb

• ❗ - Line 3, Col 5 - Style/IndentArray - Use 2 spaces for indentation in an array, relative to the first position after the preceding left parenthesis.
• ❗ - Line 45, Col 3 - Style/IndentArray - Indent the right bracket the same as the first position after the preceding left parenthesis.
• ❗ - Line 47, Col 5 - Style/IndentArray - Use 2 spaces for indentation in an array, relative to the first position after the preceding left parenthesis.
• ❗ - Line 65, Col 3 - Style/IndentArray - Indent the right bracket the same as the first position after the preceding left parenthesis.
• ❗ - Line 67, Col 5 - Style/IndentArray - Use 2 spaces for indentation in an array, relative to the first position after the preceding left parenthesis.
• ❗ - Line 85, Col 3 - Style/IndentArray - Indent the right bracket the same as the first position after the preceding left parenthesis.

app/helpers/application_helper/toolbar_chooser.rb

• ❕ - Line 477, Col 121 - Metrics/LineLength - Line is too long. [153/120]

app/presenters/menu/default_menu.rb

• ❗ - Line 117, Col 13 - Style/IndentArray - Use 2 spaces for indentation in an array, relative to the first position after the preceding left parenthesis.
• ❗ - Line 119, Col 13 - Style/IndentArray - Indent the right bracket the same as the first position after the preceding left parenthesis.

spec/controllers/ems_datawarehouse_controller_spec.rb

• ❗ - Line 13, Col 12 - Style/NestedParenthesizedCalls - Add parentheses to nested method call allow(controller).to receive(:edit).

[moolitayer]

@dclarizio I've added

• ems_datawarehouse_controller_spec.rb
• ems_datawarehouse_routing_spec.rb

There are changes to ext_management_system_spec.rb as well

Does this cover specs?

[blomquisg]

@Loicavenel @moolitayer @Fryguy @durandom

I'm fine with us starting this way (having a Data Warehouse Manager). But, I'm still not convinced this is the ultimate end goal. I'm thinking we will eventually likely re-brand this thing as a way to share data collection resources across managers. I'm still seeing a distinction between the things that we manage (Storage, Network, Containers, Compute) and the types of data we collect from those managed systems (inventory, events, and metrics). I see the Data Warehouse manager falling into the second group.

But, I don't really know what that looks like yet and I have no ideas how to even describe it in a meaningful way.

So, I'll merge this, we can move forward with this concept until we have a better notion of separating out these two ideas (if we ever figure it out 😄).