Inventory improvements for nornir 3.0

[dbarrosop]

For details go to https://github.com/nornir-automation/nornir/blob/register_inventory_plugins/docs/upgrading/2_to_3.rst

Not yet completed but figured I'd share progress

The main objective of this is:

1. To remove unnecessary complexity
2. Bring correctness (the inventory creation needs to be more explicit)
3. Prepare a generic placeholder to register and store plugins
4. Removes the need for pydantic (similar work needed for the config object)
5. Port inventory plugins to the new system
6. Adapt InitNornir

As an example of an inventory plugin written in this new way:

https://github.com/nornir-automation/nornir/blob/register_inventory_plugins/tests/conftest.py#L22

It could be made shorter and more concise using things like **h but this is more explicit, easier to read and to troubleshoot. Also, note that inventory plugins no longer deal with dictionaries, they need to create the correct types and return an inventory.

[carlmontanari]

Possibly dumb question(s)/statements for confirmation to make sure I'm understanding so I can update ansible bits accordingly:

1. no more pydantic serialize/deserialize so the inventory needs to return basically a "full" InventoryElement in all cases (i.e have all the port/user/pass/connection options/stuff). the base class has the dict method to basically do the "deserialization" piece, and because we start with a "full" inventory we dont have to worry about serializing it and ensuring all elements are there. am I mostly tracking this?
2. is the inventory plugin registration pattern done/doneish? spent some time trying to understand it but dont want to try to make any updates to try to hit a moving target if at all avoidable!
3. did the state of simple inventory get decided? sounds like it will remain the default setting but a super quick test of some tests on unrelated nornir things w/ the inventory_registration branch seems to indicate no, as there is no "" inventory plugin registered meaning to me that there is no default -- and the test referenced here is passing in that inventory_from_yaml in the conftest file
4. ive seen "entrypoint" mentioned a few times, but it looks like the inventory registration PluginRegister class isn't instantiated/told to fire off the auto_register method right now. is the plan to use "real" entry points (as in entry points in setup.py or whatever poetry equivalent I assume) that are discovered by this method, or just manually register plugins on import (what I have in current ansible inventory branch). If the former, I was just messing around and this seems cool:

    entry_points="""
        [nornir.plugin]
        nornir_ansible=nornir_ansible.inventory.AnsibleInventory
    """

but would need the auto_register method to be more like:

 for entry_point in pkg_resources.iter_entry_points(group="nornir.plugin")

then anything with an entrypoint "group" of nornir.plugin would get picked up by auto register. I've not done anything plugin related though so disregard if thats silly :)

think thats all for now... im sure ill have more questions when I get back into playing around with it!

[dbarrosop]

Sorry, got caught up last couple of weeks and I haven't found the time to finish this PR and write about this. Let me show you an example:

1. Registration: https://github.com/nornir-automation/nornir_utils/blob/first/pyproject.toml#L13-L14
2. Plugin: https://github.com/nornir-automation/nornir_utils/blob/first/nornir_utils/plugins/inventory/simple.py

So to answer specifically your questions:

1. Correct, no deserializer so you need to create the objects yourself.
2. Yes, it is, and it should work. At least it worked in my tests...
3. This is something I haven't fully decided. I think I don't want any inventory in the nornir repo but this is something I am still debating with myself (one of the ideas I am thinking of is having nornir-core with the framework only and then nornir as a sort of distro that installs the core and some common plugin packages like netmiko, napalm, jinja2, etc...)
4. It should... otherwise I need to fix it.

[carlmontanari]

Ah perfect, cool will try to get ansible updated shortly then! Thanks for the clarifications/examples, super helpful -- I'm sure I was just being dense with the registration thing, that nornir_utils repo looks like enough to make it all click for me and get me rolling, thanks!

[dbarrosop]

merged this one into 3.0.0 branch to avoid issues with future PRs but if someone has comments feel free to add them in this PR or open an issue and I will address them later