Merge branch 'master' into patch-1

CHANGELOG.md

@@ -8,21 +8,32 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ### Added
 
+- Extend http source configurations to include read_timeout value
 - enterasys800 model for enterasys 800-series fe/ge switches (@javichumellamo)
 - add ES3526XA-V2 support in EdgeCOS model (@moisseev)
 - model for eltex mes-series switches (@glaubway)
 - model for zte c300 and c320 olt (@glaubway)
 - model for LANCOM (@systeembeheerder)
 - model for Aruba CX switches (@jmurphy5)
+- model for NEC IX devices (@mikenowak)
+- Added docs for Dell/EMC Networking OS10 devices (@davromaniak)
 
 ### Changed
 
+- Add show-sensitive flag on export command on Mikrotik RouterOS when remove_secret is off (@kedare)
 - rubocop dependency now ~> 0.81.0, the last one with ruby 2.3 support
 - change pfSense secret scrubbing to handle new format in 2.4.5+
 - Dockerfile rebased to phusion/baseimage-docker bionic-1.0.0
 - scrub PoE related messages from routeros config output (@pioto)
 - support for d-link dgs-1100 series switches in dlink model (@glaubway)
 - enterasys model now works with both ro and rw access (@sargon)
+- restore including last configuration change IOS if the change is done by real user. Resolves #1921 (@raunz)
+- Additional scrubbing for SonicOS v7 devices (@gerard780)
+- improved Telnet support for enterasys (@jplitza)
+- Include "show version" output for enterasys (@jplitza)
+- xmppdiff now also shows diffs with only removed or only added lines (@jplitza)
+- xmppdiff now persists its connection to the XMPP server and MUC (@jplitza)
+- routeros no longer backups infos on available updates (@jplitza)
 
 ### Fixed
 
@@ -32,6 +43,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 - fixed snmp secret handling in netgear model (@CirnoT)
 - filter next periodic save schedule time in xos model output (@sargon)
 -  Fix when auto-saved is configured on xos switches  (@trappiz)
+- fixed ArubaOS-CX enviroment/system inconsistent values #2297 (@raunz)
+- Update AirFiber prompt regex (@murrant)
+- System time and running time are now stripped from tplink model output (@spike77453)
+
 
 ## [0.28.0 - 2020-05-18]
 

README.md

@@ -385,7 +385,7 @@ If you would like to be a maintainer for Oxidized then please read through the b
 
 ## YES, I WANT TO HELP
 
-Awesome! Simply send an email to Saku Ytti <saku@ytti.fi>.
+Awesome! Simply send an e-mail to Saku Ytti at <saku@ytti.fi>.
 
 ## Further reading
 

docs/Configuration.md

@@ -124,7 +124,7 @@ Finally, multiple private keys can be specified as an array of file paths, such
 
 ## SSH Proxy Command
 
-Oxidized can `ssh` through a proxy as well. To do so we just need to set `ssh_proxy` variable with the proxy host information and optionally set the `ssh_proxy_port` with the SSH port if it is not listening no port 22.
+Oxidized can `ssh` through a proxy as well. To do so we just need to set `ssh_proxy` variable with the proxy host information and optionally set the `ssh_proxy_port` with the SSH port if it is not listening on port 22.
 
 This can be provided on a per-node basis by mapping the proper fields from your source.
 

docs/Creating-Models.md

@@ -10,11 +10,11 @@ This methodology allows local site changes to be preserved during Oxidized versi
 
 An Oxidized model, at minimum, requires just three elements:
 
-* A model file, this file should be placed in the ~/.config/oxidized directory and named after the target OS type.
+* A model file, this file should be placed in the ~/.config/oxidized/model directory and named after the target OS type.
 * A class defined within this file with the same name as the file itself that inherits from `Oxidized::Model`, the base model class.
 * At least one command that will be executed and the output of which will be collected by Oxidized.
 
-A bare-bone example for a fictional model running the OS type `rootware` could be introduced by creating the file `~/.config/oxidized/rootware.rb`, with the following content:
+A bare-bone example for a fictional model running the OS type `rootware` could be introduced by creating the file `~/.config/oxidized/model/rootware.rb`, with the following content:
 
 ```ruby
 class RootWare < Oxidized::Model

docs/Model-Notes/OS10.md

@@ -0,0 +1,33 @@
+# OS10 Configuration
+
+Disable banner/motd
+
+```text
+banner login disable
+banner motd disable
+```
+
+Add allowed commands to privilege level 4
+
+```text
+privilege exec priv-lvl 4 "show inventory"
+privilege exec priv-lvl 4 "show inventory media"
+privilege exec priv-lvl 4 "show running-configuration"
+```
+
+Create the user will the role sysadmin (it will see the full config, including auth info and users) and the privilege level 4
+
+```text
+username oxidized password verysecurepassword role sysadmin priv-lvl 4
+```
+
+The commands Oxidized executes are:
+
+1. terminal length 0
+2. show inventory
+3. show inventory media
+4. show running-configuration
+
+Oxidized can now retrieve your configuration!
+
+Back to [Model-Notes](README.md)

docs/Model-Notes/README.md

@@ -17,6 +17,7 @@ Cisco IOS|[IOS](IOS.md)|29 Mar 2019
 Juniper|[MX/QFX/EX/SRX/J Series](JunOS.md)|18 Jan 2018
 Netgear|[Netgear](Netgear.md)|11 Apr 2018
 Nokia|[Nokia ISAM](Nokia.md)|22 Aug 2018
+Dell/EMC|[Dell EMC Networking OS10](OS10.md)|07 Dec 2021
 Viptela|[Viptela](Viptela.md)|1 Jul 2018
 Zyxel|[XGS4600 Series](XGS4600-Zyxel.md)|1 Feb 2018
 Linux|[LinuxGeneric](LinuxGeneric.md)|10 Jun 2019

docs/Sources.md

@@ -148,6 +148,7 @@ source:
     delimiter: !ruby/regexp /:/
     user: username
     pass: password
+    read_timeout: 120
     map:
       name: hostname
       model: os

docs/Supported-OS-Types.md

@@ -163,6 +163,8 @@
 * MRV
   * [MasterOS](/lib/oxidized/model/masteros.rb)
   * [FiberDriver](/lib/oxidized/model/fiberdriver.rb)
+* NEC
+  * [NEC IX](/lib/oxidized/model/necix.rb)
 * Netgear
   * [Netgear switches](/lib/oxidized/model/netgear.rb)
 * Netonix

lib/oxidized/hook/xmppdiff.rb

@@ -2,6 +2,44 @@
 require 'xmpp4r/muc/helper/simplemucclient'
 
 class XMPPDiff < Oxidized::Hook
+  def connect
+    @client = Jabber::Client.new(Jabber::JID.new(cfg.jid))
+
+    log "Connecting to XMPP"
+    begin
+      Timeout.timeout(15) do
+        begin
+          @client.connect
+        rescue StandardError => e
+          log "Failed to connect to XMPP: #{e}"
+        end
+        sleep 1
+
+        log "Authenticating to XMPP"
+        @client.auth(cfg.password)
+        sleep 1
+
+        log "Connected to XMPP"
+
+        @muc = Jabber::MUC::SimpleMUCClient.new(@client)
+        @muc.join(cfg.channel + "/" + cfg.nick)
+
+        log "Joined #{cfg.channel}"
+      end
+    rescue Timeout::Error
+      log "timed out"
+      @client = nil
+      @muc = nil
+    end
+
+    @client.on_exception do
+      log "XMPP connection aborted, reconnecting"
+      @client = nil
+      @muc = nil
+      connect
+    end
+  end
+
   def validate_cfg!
     raise KeyError, 'hook.jid is required' unless cfg.has_key?('jid')
     raise KeyError, 'hook.password is required' unless cfg.has_key?('password')
@@ -21,35 +59,17 @@ def run_hook(ctx)
         interesting = diff[:patch].lines.to_a[4..-1].any? do |line|
           ["+", "-"].include?(line[0]) && (not ["#", "!"].include?(line[1]))
         end
-        interesting &&= diff[:patch].lines.to_a[5..-1].any? { |line| line[0] == '-' }
-        interesting &&= diff[:patch].lines.to_a[5..-1].any? { |line| line[0] == '+' }
 
         if interesting
-          log "Connecting to XMPP"
-          client = Jabber::Client.new(Jabber::JID.new(cfg.jid))
-          client.connect
-          sleep 1
-          client.auth(cfg.password)
-          sleep 1
-
-          log "Connected"
-
-          m = Jabber::MUC::SimpleMUCClient.new(client)
-          m.join(cfg.channel + "/" + cfg.nick)
-
-          log "Joined"
-
-          title = "#{ctx.node.name} #{ctx.node.group} #{ctx.node.model.class.name.to_s.downcase}"
-          log "Posting diff as snippet to #{cfg.channel}"
-
-          m.say(title + "\n\n" + diff[:patch].lines.to_a[4..-1].join)
-
-          sleep 1
-
-          client.close
+          connect if @muc.nil?
 
-          log "Finished"
+          # Maybe connecting failed, so only proceed if we actually joined the MUC
+          unless @muc.nil?
+            title = "#{ctx.node.name} #{ctx.node.group} #{ctx.node.model.class.name.to_s.downcase}"
+            log "Posting diff as snippet to #{cfg.channel}"
 
+            @muc.say(title + "\n\n" + diff[:patch].lines.to_a[4..-1].join)
+          end
         end
       end
     rescue Timeout::Error

lib/oxidized/model/airfiber.rb

@@ -1,7 +1,7 @@
 class Airfiber < Oxidized::Model
   # Ubiquiti Airfiber (tested with Airfiber 11FX)
 
-  prompt /^AF[\w\.]+#/
+  prompt /^AF[\w\.-]+#/
 
   cmd :all do |cfg|
     cfg.cut_both

lib/oxidized/model/aoscx.rb

@@ -51,7 +51,9 @@ class Aoscx < Oxidized::Model
     comment cfg
   end
 
-  cmd ' show environment' do |cfg|
+  cmd 'show environment' do |cfg|
+    cfg.gsub! /^(LC.*\s+)\d+\s+$/, '\\1<hidden>'
+    cfg.gsub! /^(\d+\/\S+\s+\S+\s+)\d+\.\d+\s+C(.*)/, '\\1<hidden>\\2'
     comment cfg
   end
 
@@ -63,7 +65,7 @@ class Aoscx < Oxidized::Model
     comment cfg
   end
 
-  cmd 'show system' do |cfg|
+  cmd 'show system | exclude "Up Time" | exclude "CPU" | exclude "Memory"' do |cfg|
     comment cfg
   end
 

lib/oxidized/model/aosw.rb

@@ -40,6 +40,7 @@ class AOSW < Oxidized::Model
 
   cmd 'show version' do |cfg|
     cfg = cfg.each_line.reject { |line| line.match /(Switch|AP) uptime/i }
+    cfg = cfg.each_line.reject { |line| line.match /Reboot Time and Cause/i }
     rstrip_cfg comment cfg.join
   end
 

lib/oxidized/model/ciscosmb.rb

@@ -26,6 +26,8 @@ class CiscoSMB < Oxidized::Model
   end
 
   cmd 'show version' do |cfg|
+    cfg.gsub! /.*Uptime for this control.*/, ''
+    cfg.gsub! /.*System restarted.*/, ''
     cfg.gsub! /uptime is\ .+/, '<uptime removed>'
     comment cfg
   end

lib/oxidized/model/comware.rb

@@ -41,10 +41,15 @@ class Comware < Oxidized::Model
         cmd 'y', /(#{@node.prompt}|input password)/
         cmd vars(:comware_cmdline)
 
-        # HP V1950
+        # HP V1950 r2432P06
         cmd 'xtd-cli-mode on', /(#{@node.prompt}|Continue)/
         cmd 'y', /(#{@node.prompt}|input password)/
         cmd vars(:comware_cmdline)
+
+        # HP V1950 OS r3208 (v7.1)
+        cmd 'xtd-cli-mode', /(#{@node.prompt}|Continue)/
+        cmd 'y', /(#{@node.prompt}|input password)/
+        cmd vars(:comware_cmdline)
       end
     end
 

lib/oxidized/model/enterasys.rb

@@ -13,6 +13,10 @@ class Enterasys < Oxidized::Model
     comment cfg
   end
 
+  cmd 'show version' do |cfg|
+    comment cfg
+  end
+
   cmd 'show config' do |cfg|
     cfg.gsub! /^This command shows non-default configurations only./, ''
     cfg.gsub! /^Use 'show config all' to show both default and non-default configurations./, ''
@@ -22,7 +26,12 @@ class Enterasys < Oxidized::Model
     cfg
   end
 
-  cfg :ssh do
+  cfg :telnet do
+    username /^Username:/i
+    password /^Password:/i
+  end
+
+  cfg :telnet, :ssh do
     pre_logout 'exit'
   end
 end

lib/oxidized/model/fabricos.rb

@@ -6,7 +6,7 @@ class FabricOS < Oxidized::Model
   comment '# '
 
   cmd 'chassisShow' do |cfg|
-    comment cfg.each_line.reject { |line| line.match(/Time Awake:/) || line.match(/Power Usage \(Watts\):/) || line.match(/Time Alive:/) || line.match(/Update:/) }.join
+    comment cfg.each_line.reject { |line| line.match(/Time Awake:/) || line.match(/Power Usage \(Watts\):/) || line.match(/Power Usage:/) || line.match(/Time Alive:/) || line.match(/Update:/) }.join
   end
 
   cmd 'configShow -all' do |cfg|

lib/oxidized/model/ios.rb

@@ -108,9 +108,8 @@ class IOS < Oxidized::Model
   cmd 'show running-config' do |cfg|
     cfg = cfg.each_line.to_a[3..-1]
     cfg = cfg.reject { |line| line.match /^ntp clock-period / }.join
+    cfg = cfg.each_line.reject { |line| line.match /^! (Last|No) configuration change (at|since).*/ unless line =~ /\d+\sby\s\S+$/ }.join
     cfg.gsub! /^Current configuration : [^\n]*\n/, ''
-    cfg.gsub! /^! (Last|No) configuration change (at|since).*\n/, ''
-    cfg.gsub! /^! NVRAM config last updated.*\n/, ''
     cfg.gsub! /^ tunnel mpls traffic-eng bandwidth[^\n]*\n*(
                   (?: [^\n]*\n*)*
                   tunnel mpls traffic-eng auto-bw)/mx, '\1'

lib/oxidized/model/necix.rb

@@ -0,0 +1,30 @@
+class NecIX < Oxidized::Model
+  prompt /^(\([\w.-]*\)\s[#$]|^\S+[$#]\s?)$/
+  comment '! '
+  expect /^--More--$/ do |data, re|
+    send ' '
+    data.sub re, ''
+  end
+
+  cmd 'show running-config' do |cfg|
+    cfg = cfg.each_line.to_a[3..-2].join
+    cfg.gsub! /^.*Current time.*$/, ''
+    cfg
+  end
+
+  cfg :telnet do
+    username /^Username:/
+    password /^Password:/
+  end
+
+  cfg :telnet, :ssh do
+    post_login do
+      send "configure\n"
+    end
+
+    pre_logout do
+      send "\cZ"
+      send "exit\n"
+    end
+  end
+end