Add the "Views" feature

[peteeckel]

fixes #15
fixes #171
fixes #173

This PR adds the long awaited feature of "Views" to NetBox DNS.

A view essentially is only an optional name that can be added to any zone. There are two consequences if a zone is associated with a view:

• As long as the views of two zones differ or one of them has a view associated with it and the other hasn't, the zone name no longer needs to be unique. There can be two zones with the name "zone1.example.com", provided they are scoped differently by the view feature.
• When PTR records are manipulated for A and AAAA records, the reverse zone the PTR record is created, deleted or updated in needs to be in the same view as the zone of the A/AAAA record.

There is some rather involved logic in place that kicks in when zones are moved from one view to the other or when zones are added or removed from views. Generally the code tries to do "the right thing", i.e. doesn't create duplicate zones in the same or without a view and move the PTR records between reverse zones when the zone containing the address records is moved.

[peteeckel]

I rolled this version out in my own production environment today and was able to correctly import all my DNS data as they currently are in my DNS infrastructure, views and all. After some minor issues it now works smoothly for me, so I guess this PR is ready for review.

[jcollie]

I'm working with this in my home setup and am having some problems with the REST API. The normal web interface works OK. So far the problems I'm seeing are:

1. Querying zones by view name or id doesn't work. I have to list all of the zones by name and iterate over them looking for the one with the proper view.
2. Creating a record gives me an integrity error.

2022-04-30T16:20:03  [139640227260224] DEBUG urllib3.connectionpool https://netbox.ocjtech.us:443 "POST /api/plugins/netbox-dns/records/ HTTP/1.1" 500 213
Traceback (most recent call last):
  File "/home/jeff/dev/octodns-netbox-dns/.venv/bin/octodns-sync", line 8, in <module>
    sys.exit(main())
  File "/home/jeff/dev/octodns-netbox-dns/.venv/lib/python3.10/site-packages/octodns/cmds/sync.py", line 38, in main
    manager.sync(eligible_zones=args.zone, eligible_sources=args.source,
  File "/home/jeff/dev/octodns-netbox-dns/.venv/lib/python3.10/site-packages/octodns/manager.py", line 523, in sync
    total_changes += target.apply(plan)
  File "/home/jeff/dev/octodns-netbox-dns/.venv/lib/python3.10/site-packages/octodns/provider/base.py", line 226, in apply
    self._apply(plan)
  File "/home/jeff/dev/octodns-netbox-dns/octodns_netbox_dns/__init__.py", line 168, in _apply
    self._client.plugins.netbox_dns.records.create(
  File "/home/jeff/dev/octodns-netbox-dns/.venv/lib/python3.10/site-packages/pynetbox/core/endpoint.py", line 318, in create
    ).post(args[0] if args else kwargs)
  File "/home/jeff/dev/octodns-netbox-dns/.venv/lib/python3.10/site-packages/pynetbox/core/query.py", line 399, in post
    return self._make_call(verb="post", data=data)
  File "/home/jeff/dev/octodns-netbox-dns/.venv/lib/python3.10/site-packages/pynetbox/core/query.py", line 287, in _make_call
    raise RequestError(req)
pynetbox.core.query.RequestError: The request failed with code 500 Internal Server Error: {'error': 'duplicate key value violates unique constraint "netbox_dns_record_pkey"\nDETAIL:  Key (id)=(16) already exists.\n', 'exception': 'IntegrityError', 'netbox_version': '3.2.2', 'python_version': '3.9.5'}

[peteeckel]

Hi @jcollie, thanks! Would you mind opening a separate issue for each of the problems? I'll look into them as soon as possible.

If possible, please also include the request that led to the issue. That makes reproducing and testing easier.

[jcollie]

I figured out the record creation issue, I had to specify disable_ptr=True when creating the record. After supper I'll create a short reproducer for the query problem and open a new issue.

[jcollie]

Opened issue #171

[jcollie]

Querying by view name still doesn't work. Querying by view ID works now that I tried setting up a small reproducer.

[peteeckel]

The latest version should take care of the issues. Thanks again @jcollie for reporting!

[jcollie]

Things are looking good with the latest patches. I've even ginned up a octodns-netbox-dns module that hooks octoDNS into NetBox DNS. Note that there's a bug in pynetbox that will cause problems. Fortunately there's a pull request to fix it, unfortunately it's been sitting there for a while without any action.

[peteeckel]

Just rebased to the current main. Currently I don't have anything on my list that needs attention, so unless someone else finds something missing/going wrong I'm not planning any further changes to this PR.