Merge pull request #124 from nautobot/u/whitej6-napps-19

initial pass on refactor
nautobot · Nov 21, 2023 · 81c5ae0 · 81c5ae0
2 parents 63343a1 + 3640d5d
commit 81c5ae0
Show file tree

Hide file tree

Showing 69 changed files with 1,178 additions and 2,337 deletions.
diff --git a/.cookiecutter.json b/.cookiecutter.json
@@ -10,7 +10,7 @@
         "project_slug": "nautobot-plugin-device-onboarding",
         "repo_url": "https://github.com/nautobot/nautobot-plugin-device-onboarding",
         "base_url": "nautobot-device-onboarding",
-        "min_nautobot_version": "2.0.0",
+        "min_nautobot_version": "2.0.3",
         "max_nautobot_version": "2.9999",
         "camel_name": "NautobotDeviceOnboarding",
         "project_short_description": "Device Onboarding",
@@ -32,4 +32,4 @@
             "baked_commit_ref": "094719b3173bd24329c250c54c586b699be5f0c1"
         }
     }
-}
+}
diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -5,7 +5,7 @@ about: Report a reproducible bug in the current release of nautobot-device-onboa
 
 ### Environment
 * Python version:  <!-- Example: 3.11.4 -->
-* Nautobot version:  <!-- Example: 2.0.0 -->
+* Nautobot version:  <!-- Example: 2.0.3 -->
 * nautobot-device-onboarding version:  <!-- Example: 1.0.0 -->
 
 <!-- What did you expect to happen? -->

diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md
@@ -5,7 +5,7 @@ about: Propose a new feature or enhancement
 ---
 
 ### Environment
-* Nautobot version:  <!-- Example: 2.0.0 -->
+* Nautobot version:  <!-- Example: 2.0.3 -->
 * nautobot-device-onboarding version:  <!-- Example: 1.0.0 -->
 
 <!--

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -95,7 +95,7 @@ jobs:
       fail-fast: true
       matrix:
         python-version: ["3.11"]
-        nautobot-version: ["2.0.0"]
+        nautobot-version: ["2.0.3"]
     env:
       INVOKE_NAUTOBOT_DEVICE_ONBOARDING_PYTHON_VER: "${{ matrix.python-version }}"
       INVOKE_NAUTOBOT_DEVICE_ONBOARDING_NAUTOBOT_VER: "${{ matrix.nautobot-version }}"
@@ -138,7 +138,7 @@ jobs:
       fail-fast: true
       matrix:
         python-version: ["3.11"]
-        nautobot-version: ["2.0.0"]
+        nautobot-version: ["2.0.3"]
     env:
       INVOKE_NAUTOBOT_DEVICE_ONBOARDING_PYTHON_VER: "${{ matrix.python-version }}"
       INVOKE_NAUTOBOT_DEVICE_ONBOARDING_NAUTOBOT_VER: "${{ matrix.nautobot-version }}"
@@ -181,7 +181,7 @@ jobs:
         include:
           - python-version: "3.11"
             db-backend: "postgresql"
-            nautobot-version: "2.0.0"
+            nautobot-version: "2.0.3"
           # - python-version: "3.11"
           #   db-backend: "mysql"
           #   nautobot-version: "stable"

diff --git a/README.md b/README.md
@@ -13,14 +13,15 @@
 
 ## Overview
 
-The `nautobot-device-onboarding` plugin is using the [netmiko](https://github.com/ktbyers/netmiko) and [NAPALM](https://napalm.readthedocs.io/en/latest/) libraries to simplify the onboarding process of a new device into Nautobot down to, in many cases, an *IP Address* and a *Site*. In some cases, the user may also have to specify a specific *Device Platform* and *Device Port*.
+The `nautobot-device-onboarding` plugin is using the [netmiko](https://github.com/ktbyers/netmiko) and [NAPALM](https://napalm.readthedocs.io/en/latest/) libraries to simplify the onboarding process of a new device into Nautobot down to, in many cases, an *IP Address* and a *Location*. In some cases, the user may also have to specify a specific *Device Platform* and *Device Port*.
 
-Regardless, the Onboarding Plugin greatly simplifies the onboarding process
-by allowing the user to specify a small amount of info and having the plugin populate a much larger amount of device data in Nautobot.
+Regardless, the Onboarding Plugin greatly simplifies the onboarding process by allowing the user to specify a small amount of info and having the app populate a much larger amount of device data in Nautobot.
 
 ### Screenshots
 
-![Overview](https://raw.githubusercontent.com/nautobot/nautobot-plugin-device-onboarding/develop/docs/images/onboarding_tasks_view.png)
+Device Onboarding is a Job that allows you to provide a few required pieces of information and onboard the device.
+
+![job input](https://raw.githubusercontent.com/nautobot/nautobot-plugin-device-onboarding/develop/docs/images/do_job_inputs.png)
 
 ## Try it out!
 

diff --git a/development/Dockerfile b/development/Dockerfile
@@ -6,8 +6,8 @@
 # -------------------------------------------------------------------------------------
 # !!! USE CAUTION WHEN MODIFYING LINES BELOW
 
-# Accepts a desired Nautobot version as build argument, default to 2.0.0
-ARG NAUTOBOT_VER="2.0.0"
+# Accepts a desired Nautobot version as build argument, default to 2.0.3
+ARG NAUTOBOT_VER="2.0.3"
 
 # Accepts a desired Python version as build argument, default to 3.11
 ARG PYTHON_VER="3.11"

diff --git a/docs/admin/compatibility_matrix.md b/docs/admin/compatibility_matrix.md
@@ -6,3 +6,4 @@
 | 1.1.2                     | 1.2.0                          | 1.99                          |
 | 1.2.0                     | 1.4.0                          | 1.99                          |
 | 2.0.0                     | 2.0.0                          | 2.99                          |
+| 3.0.0                     | 2.0.3                          | 2.99                          |
diff --git a/docs/admin/install.md b/docs/admin/install.md
@@ -4,7 +4,7 @@ Here you will find detailed instructions on how to **install** and **configure**
 
 ## Prerequisites
 
-- The plugin is compatible with Nautobot 2.0.0 and higher.
+- The app is compatible with Nautobot 2.0.3 and higher.
 - Databases supported: PostgreSQL, MySQL
 
 !!! note
@@ -14,22 +14,23 @@ Here you will find detailed instructions on how to **install** and **configure**
 
 #### NAPALM Credentials
 
-The Onboarding Plugin uses NAPALM. You can configure a default NAPALM username and password in `nautobot_config.py`.
+The Onboarding App uses NAPALM. You can configure a default NAPALM username and password in `nautobot_config.py`.
 
-When `NAPALM_USERNAME` and `NAPALM_PASSWORD` are configured in `nautobot_config.py`, the user does not have to specify the `Username` and `Password` fields in the Device Onboarding Task, unless they wish to override the values in `nautobot_config.py`:
+When `NAPALM_USERNAME`, `NAPALM_PASSWORD` and `NAPALM_ARGS` are configured in `nautobot_config.py`, the user does not have to use the Credentials/SecretGroup fields in the Device Onboarding job, unless they wish to override the values in `nautobot_config.py`:
 
 ```python
 # Credentials that Nautobot will use to authenticate to devices when connecting via NAPALM.
 NAPALM_USERNAME = "<napalm username>"
 NAPALM_PASSWORD = "<napalm pwd>"
+NAPALM_ARGS = {"secret": "<enable secret pwd>"}
 ```
 
 ## Install Guide
 
 !!! note
-    Plugins can be installed manually or using Python's `pip`. See the [nautobot documentation](https://nautobot.readthedocs.io/en/latest/plugins/#install-the-package) for more details. The pip package name for this plugin is [`nautobot-device-onboarding`](https://pypi.org/project/nautobot-device-onboarding/).
+    App can be installed manually or using Python's `pip`. See the [nautobot documentation](https://nautobot.readthedocs.io/en/latest/plugins/#install-the-package) for more details. The pip package name for this plugin is [`nautobot-device-onboarding`](https://pypi.org/project/nautobot-device-onboarding/).
 
-The plugin is available as a Python package via PyPI and can be installed with `pip`:
+The app is available as a Python package via PyPI and can be installed with `pip`:
 
 ```shell
 pip install nautobot-device-onboarding
@@ -41,7 +42,7 @@ To ensure Device Onboarding is automatically re-installed during future upgrades
 echo nautobot-device-onboarding >> local_requirements.txt
 ```
 
-Once installed, the plugin needs to be enabled in your Nautobot configuration. The following block of code below shows the additional configuration required to be added to your `nautobot_config.py` file:
+Once installed, the app needs to be enabled in your Nautobot configuration. The following block of code below shows the additional configuration required to be added to your `nautobot_config.py` file:
 
 - Append `"nautobot_device_onboarding"` to the `PLUGINS` list.
 - Append the `"nautobot_device_onboarding"` dictionary to the `PLUGINS_CONFIG` dictionary and override any defaults.

diff --git a/docs/admin/release_notes/version_1.2.md b/docs/admin/release_notes/version_1.2.md
@@ -10,9 +10,10 @@ This document describes all new features and changes in `nautobot-device-onboard
 ### Added
 
 ### Changed
+
 - [#53](https://github.com/nautobot/nautobot-plugin-device-onboarding/pull/53) - Documentation updates
 - [#87](https://github.com/nautobot/nautobot-plugin-device-onboarding/pull/87) - Add OnboardingTask status and failure reason metrics
-- 
 
 ### Fixed
+
 - [#75](https://github.com/nautobot/nautobot-plugin-device-onboarding/pull/75) - Fix reverse migration for migrations 0002, 0003
diff --git a/docs/admin/release_notes/version_3.0.md b/docs/admin/release_notes/version_3.0.md
@@ -0,0 +1,22 @@
+# v3.0 Release Notes
+
+!!! warning
+    Nautobot Device Onboarding v2.0.0-2.0.2 contains a vulnerability where the credentials used to log into a device may be visible in clear text on the Job Results page under the Additional Data tab. It is recommended to review all OnbaordingTasks from the affected releases, delete any affeccted JobResults, and upgrade to v3.0.0. For more information please see the full write up on the issue which is available as [a security advisory](https://github.com/nautobot/nautobot-plugin-device-onboarding/security/advisories/GHSA-qf3c-rw9f-jh7v) on the repo. Nautobot Device Onboarding app versions v2.0.0-2.0.2 have been removed for PyPI to ensure all gaps are closed. v2.0.3 is published with disabled functionality and banner message encouraging to upgrade to v3.0.0. [CVE-2023-48700](https://www.cve.org/CVERecord?id=CVE-2023-48700) has been issued for this vulnerability.
+
+## Release Overview
+
+## v3.0.0 2023-11-21
+
+### Changed
+
+- [#124](https://github.com/nautobot/nautobot-plugin-device-onboarding/pull/124) - Device onboarding is now provided via a Nautobot Job.
+- [#124](https://github.com/nautobot/nautobot-plugin-device-onboarding/pull/124) - CSV import has changed to a comma separated list of IPs/FQDNs as a job input
+
+### Fixed
+
+- [#124](https://github.com/nautobot/nautobot-plugin-device-onboarding/pull/124) - Leaking of device credentials if username & password were provided on creation of an instance of the `OnboardingTask` object.
+
+### Removed
+
+- [#124](https://github.com/nautobot/nautobot-plugin-device-onboarding/pull/124) - Removed all models, UI Views, and API Views from app
+- [#124](https://github.com/nautobot/nautobot-plugin-device-onboarding/pull/124) - All data for instances of `OnboardingTask` & `OnboardingDevice` will be removed on upgrade, affected `JobResults` from tasks created while on affected versions should be reviewed & deleted.
diff --git a/docs/dev/code_reference/api.md b/docs/dev/code_reference/api.md
diff --git a/docs/dev/code_reference/models.md b/docs/dev/code_reference/models.md
diff --git a/docs/dev/dev_environment.md b/docs/dev/dev_environment.md
@@ -13,7 +13,7 @@ This is a quick reference guide if you're already familiar with the development
 
 The [Invoke](http://www.pyinvoke.org/) library is used to provide some helper commands based on the environment. There are a few configuration parameters which can be passed to Invoke to override the default configuration:
 
-- `nautobot_ver`: the version of Nautobot to use as a base for any built docker containers (default: 2.0.0)
+- `nautobot_ver`: the version of Nautobot to use as a base for any built docker containers (default: 2.0.3)
 - `project_name`: the default docker compose project name (default: `nautobot-device-onboarding`)
 - `python_ver`: the version of Python to use as a base for any built docker containers (default: 3.11)
 - `local`: a boolean flag indicating if invoke tasks should be run on the host or inside the docker containers (default: False, commands will be run in docker containers)
@@ -180,7 +180,7 @@ The first thing you need to do is build the necessary Docker image for Nautobot
 #14 exporting layers
 #14 exporting layers 1.2s done
 #14 writing image sha256:2d524bc1665327faa0d34001b0a9d2ccf450612bf8feeb969312e96a2d3e3503 done
-#14 naming to docker.io/nautobot-device-onboarding/nautobot:2.0.0-py3.11 done
+#14 naming to docker.io/nautobot-device-onboarding/nautobot:2.0.3-py3.11 done
 ```
 
 ### Invoke - Starting the Development Environment
@@ -211,9 +211,9 @@ This will start all of the Docker containers used for hosting Nautobot. You shou
 ```bash
 ➜ docker ps
 ****CONTAINER ID   IMAGE                            COMMAND                  CREATED          STATUS          PORTS                                       NAMES
-ee90fbfabd77   nautobot-device-onboarding/nautobot:2.0.0-py3.11  "nautobot-server rqw…"   16 seconds ago   Up 13 seconds                                               nautobot_device_onboarding_worker_1
-b8adb781d013   nautobot-device-onboarding/nautobot:2.0.0-py3.11  "/docker-entrypoint.…"   20 seconds ago   Up 15 seconds   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   nautobot_device_onboarding_nautobot_1
-d64ebd60675d   nautobot-device-onboarding/nautobot:2.0.0-py3.11  "mkdocs serve -v -a …"   25 seconds ago   Up 18 seconds   0.0.0.0:8001->8080/tcp, :::8001->8080/tcp   nautobot_device_onboarding_docs_1
+ee90fbfabd77   nautobot-device-onboarding/nautobot:2.0.3-py3.11  "nautobot-server rqw…"   16 seconds ago   Up 13 seconds                                               nautobot_device_onboarding_worker_1
+b8adb781d013   nautobot-device-onboarding/nautobot:2.0.3-py3.11  "/docker-entrypoint.…"   20 seconds ago   Up 15 seconds   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   nautobot_device_onboarding_nautobot_1
+d64ebd60675d   nautobot-device-onboarding/nautobot:2.0.3-py3.11  "mkdocs serve -v -a …"   25 seconds ago   Up 18 seconds   0.0.0.0:8001->8080/tcp, :::8001->8080/tcp   nautobot_device_onboarding_docs_1
 e72d63129b36   postgres:13-alpine               "docker-entrypoint.s…"   25 seconds ago   Up 19 seconds   0.0.0.0:5432->5432/tcp, :::5432->5432/tcp   nautobot_device_onboarding_postgres_1
 96c6ff66997c   redis:6-alpine                   "docker-entrypoint.s…"   25 seconds ago   Up 21 seconds   0.0.0.0:6379->6379/tcp, :::6379->6379/tcp   nautobot_device_onboarding_redis_1
 ```
@@ -414,7 +414,7 @@ namespace.configure(
     {
         "nautobot_device_onboarding": {
             ...
-            "nautobot_ver": "2.0.0",
+            "nautobot_ver": "2.0.3",
 	    ...
         }
     }

diff --git a/docs/images/csv_import_view.png b/docs/images/csv_import_view.png
diff --git a/docs/images/do_job_inputs.png b/docs/images/do_job_inputs.png
diff --git a/docs/images/menu.png b/docs/images/menu.png
diff --git a/docs/images/onboarding_form_filled.png b/docs/images/onboarding_form_filled.png
diff --git a/docs/images/onboarding_tasks_full_view.png b/docs/images/onboarding_tasks_full_view.png
diff --git a/docs/images/onboarding_tasks_view.png b/docs/images/onboarding_tasks_view.png
diff --git a/docs/images/platform_cisco_ios.png b/docs/images/platform_cisco_ios.png
diff --git a/docs/images/platform_juniper_junos.png b/docs/images/platform_juniper_junos.png
diff --git a/docs/images/single_device_form.png b/docs/images/single_device_form.png
diff --git a/docs/user/app_getting_started.md b/docs/user/app_getting_started.md
@@ -15,32 +15,20 @@ You will need:
 - a device reachable from the Nautobot instance
     - this can an IP or DNS name
 - the device's credentials
-- to create a site in Nautobot
+- to create a location in Nautobot
 
 The device must be reachable from the Nautobot and Nautobot worker instances (usually if one can reach it, the other can as well). You can test reachability directly with ssh. Since the plugin uses Napalm and Netmiko, they could also be used for a more accurate test.
 
-Sites are the only other Nautobot prerequisite for the plugin to onboard a device.
+Locations are the only other Nautobot prerequisite for the plugin to onboard a device.
 
 ### Onboarding a Device
 
-Navigate to the Device Onboarding plugin: Plugins > Onboarding Tasks. Clicking the plus button takes you directly to the Onboarding form.
+Navigate to the Device Onboarding Job: Jobs > Perform Device Onboarding.
 
-![Device Onboarding Navigation Menu](../images/menu.png)
-
-From the Onboarding Tasks view, click the **add** button in the top right.
-
-![In the Onboarding Tasks view, click the add button](../images/onboarding_tasks_full_view.png)
-
-This will bring you to the *Add a new onboarding task* form.
-
-![Onboarding task form](../images/single_device_form.png)
-
-Fill in the form with your device's data, then click the create button to start onboarding the device.
-
-![Onboarding form filled in](../images/onboarding_form_filled.png)
+## What are the next steps?
 
-The Nautobot worker will initiate an onboarding task and will reach out to the device and attempt to onboard it.
+You can check out the [Use Cases](app_use_cases.md) section for more examples or try out the job inputs with at least the required fields.
 
-## What are the next steps?
+![job input](../images/do_job_inputs.png)
 
-You can check out the [Use Cases](app_use_cases.md) section for more examples.
+The Nautobot job will pass the job execution to the worker which will initiate an onboarding and will reach out to the device and attempt to onboard it.
diff --git a/docs/user/app_overview.md b/docs/user/app_overview.md
@@ -9,38 +9,38 @@ This [Nautobot](https://github.com/nautobot/nautobot) App allows to easily onboa
 
 ## Description/Overview
 
-The `nautobot-device-onboarding` plugin uses the [netmiko](https://github.com/ktbyers/netmiko) and [NAPALM](https://napalm.readthedocs.io/en/latest/) libraries to simplify the onboarding process of a new device into Nautobot down to, in many cases, an *IP Address* and a *Site*. In some cases, the user may also have to specify a specific *Device Platform* and *Device Port*.
+The `nautobot-device-onboarding` app uses the [netmiko](https://github.com/ktbyers/netmiko) and [NAPALM](https://napalm.readthedocs.io/en/latest/) libraries to simplify the onboarding process of a new device into Nautobot down to, in many cases, an *IP Address* and a *Location*. In some cases, the user may also have to specify a specific *Device Platform* and *Device Port*.
 
-Regardless, the Onboarding Plugin greatly simplifies the onboarding process by allowing the user to specify a small amount of info and having the plugin populate a much larger amount of device data in Nautobot.
+Regardless, the Onboarding App greatly simplifies the onboarding process by allowing the user to specify a small amount of info and having the app populate a much larger amount of device data in Nautobot.
 
 In most cases, the user would specify:
 
 - Device Name
-- Site
+- Location
 - Platform *
 - Transport Port *
 
 !!! note
     * `Platform` and `Transport Port` are necessary for onboarding NXOS API, Arista EOS, or any other platform not using SSH as a transport.
 
-And the Onboarding Plugin would populate the following:
+And the Onboarding App would populate the following:
 
-- Device Type (Model) - Creates if it does not exist
-- Device Role - defaults to `network`
-- Platform - Creates Cisco IOS, Cisco NXOS (ssh), and Junos Platforms if they do not exist
-- Manufacturer - Creates Cisco/Juniper/Arista if it does not exist
-- Management Interface
-- Management Interface IP
-- Serial Number (when available)
+- Device Type (Model) - Creates it if it does not exist.
+- Role - via a Role of content-type "dcim:device". Defaults to `network`.
+- Platform - Creates Cisco IOS, Cisco NXOS (ssh), and Junos Platforms if they do not exist.
+- Manufacturer - Creates Cisco/Juniper/Arista if it does not exist.
+- Management Interface.
+- Management Interface IP.
+- Serial Number (when available).
 
-The goal of this plugin is not to import everything about a device into Nautobot. Rather, the goal is to quickly build an inventory of basic device data in Nautobot that provides basic info on how to access the devices.
+The goal of this app is not to import everything about a device into Nautobot. Rather, the goal is to quickly build an inventory of basic device data in Nautobot that provides basic info on how to access the devices.
 For example, getting the Management IP and Platform data into Nautobot allows a follow-on tool that uses the basic info to access each device, retrieve data, and then populate Nautobot with that data.
 
 One example of a solution that can retrieve that additional device data and import it into Nautobot is the [Network Importer](https://github.com/networktocode/network-importer). Other options would include an Ansible playbook or a Python script.
 
 ## Audience (User Personas) - Who should use this App?
 
-The Onboarding Plugin is meant for new Nautobot users who want to start importing their devices directly rather than from another, existing, source. Even with other sources for device information, they may not include everything that is necessary.
+The Onboarding App is meant for new Nautobot users who want to start importing their devices directly rather than from another, existing, source. Even with other sources for device information, they may not include everything that is necessary.
 
 Existing Nautobot users may want to incorporate the Onboarding Plugin as part of onboarding new devices to the platform.
 
@@ -59,7 +59,5 @@ Existing Nautobot users may want to incorporate the Onboarding Plugin as part of
 
 ## Nautobot Features Used/Employed
 
-- Data Models
-- NAV Menu Items
-- REST API Endpoints
-- Views
+- Secrets & SecretsGroup
+- Jobs