2. Writing test and config file

Writing config file

Config file supports following parameters:

1. hosts: here you have to specify login credential of device to be connected.
          If you want to connect to multiple devices then write all login credentials in one yaml file and import it here.
2. tests: specify test files that you want to run
3. sqlite(optional): Use this key if you want to compare/store snapshots in database.
4. mail(optional): Use this key if you want to get notification of test results via mail.
5. local(optional): Use this key if you want to run snapcheck on stored snapshots. Will work with --snapcheck command only.

Examples

• Sample config file for single device

hosts:
  - device: 10.20.1.24
    username : foo
    passwd: bar
tests:
 - test_no_diff.yml
 - test_bgp_neighbor.yml
# (optional) use only when you want to store and compare snapshots from database  
sqlite:
  - store_in_sqlite: True
    check_from_sqlite: True 
    database_name: jbb.db
    compare: 1,0
# (optional) use when you want to send mail about test results 
mail: send_mail.yml
# (optional) use when you want to run snapcheck on stored snapshots. 
# Will work with --snapcheck command only.
# Specify the list of stored snapshot names on which you want to run snapcheck 
local:
  - STORED
  - STORED_42
  - STORED_314

• Sample config file for connecting to multiple devices

# for multiple devices with databse
hosts:
  - include: devices.yml
    group: EX
tests:
  - test_is_equal.yml
  - test_is_in.yml
# (optional) use only when you want to store and compare snapshots from database
sqlite:
  - store_in_sqlite: yes
    check_from_sqlite: yes
    database_name: jbb.db
# (optional) use when you want to send mail about test results
mail: send_mail.yml

• devices.yml

MX:                      
  - 10.20.1.20:
      username: root
      passwd: root123
  - 10.21.13.14:
      username: root
      passwd: root123
  - 10.20.6.26:
      username: jsnapy
      passwd: jsnapy123
EX: 
  - 10.2.15.210:                    
      username: root
      passwd: root123
  - 10.9.16.22:
      username: abc
      passwd: pqr
QFX:                     
  - 10.29.1.24:
      username: abc
      passwd: pqr
  - 10.29.6.1:
      username: abc
      passwd: pqr123

Writing test file

Purpose of writing test file is to specify command/rpc whose snapshot is 
to be taken and what all nodes user wants to test and how to test them.

1. tests_include(optional): use this tag if you want to include only some test cases.
                            If you do not include this tag, then jsnapy will run all test cases by default.
2. command/rpc : can give either command or rpc to perform testing
   - format: can specify output format
        [text,xml]
         for comparing text output, only --diff option is supported
3. args(optional): (used only with rpc)
      - filter_xml: (can provide filtered output)
      - other arguments
   example:
      1.   - rpc: get-config
           - args:
               filter_xml: configuration/system/login
      2.   - rpc: get-interface-information
             format: text
           - args:
                interface-name: em0
                media: True
                detail: True
4. item/iterate: (can have multiple iterate/item under one command/rpc)
       item: if you want to use only first node in xpath
       iterate: if want to keep iterating for all nodes in xpath
5. xpath: path from where u want to test output
6. tests: (this section specify test-cases, can have multiple test cases inside 
          one iterate/item)
      test-operator <condition>
      info <mssg>
      err  <mssg>
   Eg:
   tests:
     - is-equal: //minimum-time, 60
       info: "Test Succeeded!!, minimum-time now is equal to <{{post['//minimum-time']}}>"
       err: "Test Failed!!!, minimum-time is not equal to 60, it is <{{post['//minimum-time']}}>"

Examples

1] Sample test file1 (test_is_equal.yml)

test_interfaces_terse:
  - command: show interfaces terse lo* 
  - item:
      id: ./name
      xpath: //physical-interface[normalize-space(name) = "lo0"]
      tests:
        - is-equal: admin-status, down
          info: "Test Succeeded !! admin-status is equal to post: <{{post['admin-status']}}>  pre: <{{pre['admin-status']}}> with oper-status pre:<{{pre['oper-status']}}> post:<{{post['oper-status']}}>"
          err: "Test Failed !! admin-status is not equal to down, it is post: <{{post['admin-status']}}>  pre:<{{pre['admin-status']}}> with oper-status <{{pre['oper-status']}}>"

2] Sample test file2 (multiple iterators and tests for single command)

tests_include:
  - test_multiple_tests
  - test_multiple_iter

test_multiple_tests:
  - command: show interfaces terse lo* 
  - item:
      id: ./name
      xpath: //physical-interface[normalize-space(name) = "lo0"]
      tests:
        - is-equal: admin-status, down
          info: "Test Succeeded !! admin-status is equal to <{{post['admin-status']}}>  <{{pre['admin-status']}}> with oper-status <{{pre['oper-status']}}>"
          err: "Test Failed !! admin-status is not equal to down, it is <{{post['admin-status']}}> <{{pre['admin-status']}}> with oper-status <{{pre['oper-status']}}>"
        - is-in: oper-status, downoo, up
          info: "Test Succeeded!! Physical operational status is-in downoo-up, it is: <{{post['oper-status']}}> with admin status <{{post['admin-status']}}>"
          err: "Test Failed!!! Physical operational status is not in downoo-up, it is: <{{post['oper-status']}}> with admin status <{{post['admin-status']}}> "

test_multiple_iter:
  - command: show interfaces terse lo* 
  - item:
      id: ./name
      xpath: //physical-interface[normalize-space(name) = "lo0"]
      tests:
        - is-equal: admin-status, down
          info: "Test Succeeded !! admin-status is equal to <{{post['admin-status']}}>  <{{pre['admin-status']}}> with oper-status <{{pre['oper-status']}}>"
          err: "Test Failed !! admin-status is not equal to down, it is <{{post['admin-status']}}> <{{pre['admin-status']}}> with oper-status <{{pre['oper-status']}}>"

  - iterate:
      xpath: //physical-interface[normalize-space(name) = "lo0"]
      tests:
        - is-in: oper-status, down, up
          info: "Test Succeeded!! Physical operational status is-in downoo-up, it is: <{{post['oper-status']}}> with admin status <{{post['admin-status']}}>"
          err: "Test Failed!!! Physical operational status is not in downoo-up, it is: <{{post['oper-status']}}> with admin status <{{post['admin-status']}}> "

3] Sample test file3 (test_bgp_neighbor.yml)

tests_include:
  - test_command_bgp
  - test_rpc_bgp

test_command_bgp:
  - command: show bgp neighbor
  - iterate:
      xpath: '/bgp-information/bgp-peer'
      tests:
        - is-equal: peer-address, 10.209.19.203       # element in which test is performed
          err: "Test Failed!! peer-address got changed, it is now <{{post['peer-address']}}>"
          info: "Test succeeded!! peer-address is equal to <{{post['peer-address']}}>"

        - in-range: peer-as, 100,900        # element in which test is performed
          err: "Test Failed!! peer-as is not in range of 100-200, it is: <{{post['peer-as']}}>"
          info: "Test succeeded!! peer-as is in range of 100-200, it is now <{{post['peer-as']}}>"

  - iterate:
      xpath: '//bgp-information/bgp-peer/bgp-option-information'
      tests:
        - is-gt: holdtime, 10       # element in which test is performed
          err: "Test Failed!! holdtime is not greater than 10, it is: <{{post['holdtime']}}>"
          info: "Test succeeded!! holdtime is greater than 10, it is: <{{post['holdtime']}}>"

        - is-lt: preference, 200      # element in which test is performed
          err: "Test Failed!! preference is not less than 10, <{{post['preference']}}>"
          info: "Test succeeded!! preference is less than 10, <{{post['preference']}}>"

test_rpc_bgp:
  - rpc: get-bgp-neighbor-information
  - iterate:
      xpath: '//bgp-information/bgp-peer'
      tests:
        - not-equal: last-state,Idle       # element in which test is performed
          err: "Test Failed!! last state is <{{post['last-state']}}>"
          info: "Test succeeded!! last state is not equal to idle, it is: <{{post['last-state']}}>"

        - all-same: flap-count
          err: "Test Failed!!! flap count are not all same!!, it is <{{post['flap-count']}}> "
          info: "Test Succeeded!! flap count are all same, it is now <{{post['flap-count']}}>!!!"

        - is-equal: flap-count, 0
          err: "Test Failed!!! flap count is not equal to 0, it is: <{{post['flap-count']}}> "
          info: "Test Succeeded!! flap count is equal to <{{post['flap-count']}}> !!"

4] Sample test file containing multiple IDs
For defining multiple ids, there are two ways

a) make a list of ids
     id: [name, ../peer-address]
b) mention all ids in one line, as comma separated string
     id: name, ../peer-address

Example: test_bgp_summary.yml

bgp-summary:
- command: show bgp summary
- iterate:
    id: name, ../peer-address 
    tests:
    - delta: active-prefix-count, 20%
      err: ' ERROR: The number of active prefix of the BGP Table have changed more than 20%. name: {{id_0}} and peer-address: {{id_1}} [Before = {{pre["active-prefix-count"]}} / After = {{post["active-prefix-count"]}}]'
      info: 'Checking BGP peer active prefix count (tolerance 20%) name is <{{id_0}}> and peer-address: {{id_1}} pre:  {{pre["active-prefix-count"]}} post:{{post["active-prefix-count"]}}'
    xpath: '/bgp-information/bgp-peer/bgp-rib'