Skip to content

Commit

Permalink
Merge pull request sonic-net#156 from a-barboza/master
Browse files Browse the repository at this point in the history 
NTP Restart Avoidance 0.5
  • Loading branch information
@dks19
dks19 authored Apr 20, 2021
2 parents c20c201 + 8a97b06 commit 9eed90e
Showing 1 changed file with 349 additions and 4 deletions.
353 changes: 349 additions & 4 deletions manageability/ntp/ntp_restart_avoid.md
View file
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
# The NTP Restart Avoidance Design
# High Level Design Document
### Rev 0.1
### Rev 0.5

# Table of Contents

# Revision
| Rev | Date | Author | Change Description |
|:---:|:-----------:|:------------------:|-----------------------------------|
|:---:|:-----------:|:------------------:|:---------------------------------:|
| 0.1 | 11/22/2020 | Arun Barboza | Initial draft |
| 0.2 | 11/25/2020 | Arun Barboza | Internal Review(trusted key) |
| 0.3 | 12/02/2020 | Arun Barboza | Appendix ntpconfd design |
| 0.4 | 12/14/2020 | Arun Barboza | Mgmt VRF handling |
| 0.5 | 01/19/2021 | Arun Barboza | Updates for the keys handling |

# About this Manual
This document provides general information about the design of NTP Restart avoidance in SONiC.
Expand Down Expand Up @@ -170,10 +173,12 @@ The ntpd daemon does not require controlkey to be used for status queries. It re

## Config File Diff

The NBIs and DHCP directly generate the ntpd configuration files (ntp.conf, ntp.keys). These files will be cached, and diffed to generate the runtime reconfiguration commands that are to sent to ntpd. A new ntp.conf file is generated for ntpd, in case it is restarted (or restarts).
The NBIs and DHCP directly generate the ntpd configuration files (ntp.conf, ntp.keys, /run/ntp.conf.dhcp). These files will be cached, and diffed to generate the runtime reconfiguration commands that are to sent to ntpd. A new ntp.conf file is generated for ntpd, in case it is restarted (or restarts).

ntp.keys file is privileged information, and should be handled with similar diligence as the ntpd daemon.

Per existing SONiC/Linux behavior, the DHCP ntp configuration file (/run/ntp.conf.dhcp) file is given higher precedence than the /etc/ntp.conf file. The DHCP ntp configuration file is generated from /etc/ntp.conf with the NTP servers configuration lines replaced with the ones received from DHCP.

## Avoid Restart

The following runtime reconfigurations, when generated by a qualified program, will be handled without a restart of ntpd. Qualified programs are hostcfgd, and dhclient.
Expand All @@ -186,7 +191,7 @@ The following runtime reconfigurations, when generated by a qualified program, w

### keys

Re-reads the keys file.
Re-reads the keys file. (However restart may be required because of authentication change, see below under Require Restart)

### trustedkey

Expand All @@ -202,6 +207,10 @@ Following reconfigurations will require a ntpd restart.
ntpd does not support an inverse (untrustkey) for the trustedkey command.
However, if the only trustedkey changes are additive, i.e. additional keys are being trusted, then restart may not be required.

### /etc/ntp.keys

If there are any modifications to the keys file (including deleting the keys file line from the ntp.conf file to disable reading of ntp.keys), restart may be required. If keys are deleted from the ntp.keys file, restart will be required to purge ntpd of the removed keys.

### vrf

Unrestricted changing of the vrf is not supported by the kernel.
Expand Down Expand Up @@ -245,3 +254,339 @@ ntpd. It may be necessary to restart ntpd if there are an excessive number of re
- non-qualified (Eg: manually editing the files) reconfigurations cause ntpd restarts.


# Appendix ntpconfd Design

## Constants

```
ntpconfd_logdir = '/var/log/ntpconfd/'
saveconfig_logdir = ntpconfd_logdir + 'saveconfig/'
ntpd_confdir = '/var/run/ntpconfd/etc'
ntpd_conffile = ntpd_confdir + 'ntp.conf'
ntpd_keyfile = ntpd_confdir + 'ntp.keys'
ntpd_mgmtvrffile = ntpd_confdir + 'mgmt.vrf'
```

## Service File

## Libraries

### Difflib

Given 2 files, obtain a list of subtractions, and additions.

Use an existing library?

### Call ntpq binary

Send some commands to ntpq, and get a reply after each.
If any command returns error (other than "Config Succeeded", then stop)
can subprocess be used?

```
admin@S:~$ ntpq
ntpq> keytype MD5
ntpq> keyid 1
ntpq> passwd brcm
ntpq> association
No association ID's returned
ntpq>
ntpq> config-from-file /tmp/sample.conf
Sending configuration file, one line at a time.
Line No: 1 Config Succeeded: interface listen Ethernet0
Line No: 2 Config Succeeded: interface listen Ethernet1
Done sending file
ntpq>
```

```
ntpq> :config keys /etc/ntp.keys
Config Succeeded
ntpq>
```

```
ntpq> saveconfig n2.conf
Configuration saved to 'n2.conf'
ntpq>
```

** Do not use ntpq -c option because it may cause eavesdroppers to view the controlkey passwd that is passed on the command line **

```
admin@S:~$ ntpq --help
ntpq - standard NTP query program - Ver. 4.2.8p10
Usage: ntpq [ -<flag> [<val>] | --<name>[{=| }<val>] ]... [ host ...]
Flg Arg Option-Name Description
-4 no ipv4 Force IPv4 DNS name resolution
- prohibits the option 'ipv6'
-6 no ipv6 Force IPv6 DNS name resolution
- prohibits the option 'ipv4'
-c Str command run a command and exit
- may appear multiple times
...
```


### Random number

What is available to use as a key? Python version. (secrets, random)

Generation of controlkey

### Logging

logging library


## Command chart handling for diffs of ntp.conf

The commands under Add column are sent via ntpq when the line is added.
The commands under Subtracted column are sent via ntpq when the line is deleted.


| Token | Add | Subtract | Change Description |
|:-------------:|:-------------:|:-------------:|:---------------------------:|
| | | | |
| | | | |
| server host \ | server host \ | unpeer host | |
| ... | ... | | |
| | | | |
| | | | |
| interface \ | interface \ | interface \ | |
| listen iface | listen iface | ignore iface | |
| | | | |
| | | | |
| enable auth | enable auth | disable auth | |
| | | | |
| | | | |
| keys file | See (b) | See (b) | file perm (ntp:ntp 0400) |
| | | | |
| | | | |
| trustedkey \ | See (c) | See (c) | |
| k1 [k2 ... ] | | | |
| | | | |
| | | | |


Notes:

(a) vrf does not occur in the ntp.conf file, but is handled with a restart of ntp-daemon service

(b) Restart ntp-daemon if keys file is subtracted from the configuration. If any keys are removed from /etc/ntp.keys file, the ntp-daemon needs to be restarted.

(c) trustedkey is handled only for the incremental case. If there are added trusted keyids, we can send the trustedkey command via ntpq; But, if any trusted keyids are deleted then we need to restart the ntp-daemon. controlkey keyid needs to be added to list of trustedkeys.


## Signal Handler

### SIGHUP

### SIGTERM

### SIGUSR2

## Initialization

### Create ntpd Configuration File

```
if '/run/ntp.conf.dhcp' exists
read the 'server' lines from '/run/ntp.conf.dhcp' as dhcp_ntp_servers
read the '/etc/ntp.conf' file with (server|peer|pool) replaced \
with 'server' lines replaced with dhcp_ntp_servers, as prev_conffile
else
read the '/etc/ntp.conf' file as prev_conffile
read the '/etc/ntp.keys' file as prev_keyfile
read the ConfigDB MGMT_VRF_CONFIG["vrf_global"]["mgmtVrfEnabled"] as prev_vrf_enabled
read the ConfigDB NTP["global"]["vrf"] as prev_vrf_configured
controlkey_inuse = false
keyidfull = false
if keyid space is full
keyidfull = true
if (not controlkey_inuse) and (not keyid_full)
assign controlkey_keyid (begin search from 65535)
generate controlkey_key
write prev_keyfile with generated controlkey as ntpd_keyfile
write prev_conffile with generated controlkey as ntpd_conffile
if prev_vrf_enabled == "true" and prev_vrf_configured != "default"
touch ntpd_mgmtvrffile
else
rm ntpd_mgmtvrffile
```

### Start ntp-daemon service

```
service ntp-daemon start
```


## Main Loop

### Check And Handle SIGTERM

```
Stop ntp-daemon service
Exit
```

### Check And Handle SIGHUP

```
read the ConfigDB MGMT_VRF_CONFIG["vrf_global"]["mgmtVrfEnabled"] as vrf_enabled
read the ConfigDB NTP["global"]["vrf"] as vrf_configured
if '/run/ntp.conf.dhcp' exists
read the 'server' lines from '/run/ntp.conf.dhcp' as dhcp_ntp_servers
read the '/etc/ntp.conf' file with (server|peer|pool) replaced \
with 'server' lines replaced with dhcp_ntp_servers, as new_conffile
else
read the '/etc/ntp.conf' file as new_conffile
read the '/etc/ntp.keys' file as new_keyfile
if (vrf_enabled != prev_vrf_enabled) or (vrf_configured != prev_vrf_configured)
write new_keyfile as ntpd_keyfile
write new_conffile as ntpd_conffile
if prev_vrf_enabled == "true" and prev_vrf_configured != "default"
touch ntpd_mgmtvrffile
else
rm ntpd_mgmtvrffile
prev_* = new_*
ReStart ntp-daemon service (Stop, Sleep, Start)
continue
if controlkey_inuse or keyidfull
write new_keyfile as ntpd_keyfile
write new_conffile as ntpd_conffile
if prev_vrf_enabled == "true" and prev_vrf_configured != "default"
touch ntpd_mgmtvrffile
else
rm ntpd_mgmtvrffile
prev_* = new_*
ReStart ntp-daemon service (Stop, Sleep, Start)
continue
if controlkey is being used
controlkey_inuse = true
write new_keyfile as ntpd_keyfile
write new_conffile as ntpd_conffile
if prev_vrf_enabled == "true" and prev_vrf_configured != "default"
touch ntpd_mgmtvrffile
else
rm ntpd_mgmtvrffile
prev_* = new_*
ReStart ntp-daemon service (Stop, Sleep, Start)
continue
if keyid space is full
keyidfull = true
write new_keyfile as ntpd_keyfile
write new_conffile as ntpd_conffile
if prev_vrf_enabled == "true" and prev_vrf_configured != "default"
touch ntpd_mgmtvrffile
else
rm ntpd_mgmtvrffile
prev_* = new_*
ReStart ntp-daemon service (Stop, Sleep, Start)
continue;
assign new_controlkey_keyid
generate new_controlkey_key
write new_keyfile with generated controlkey as ntpd_keyfile
write new_conffile with generated controlkey as ntpd_conffile
if prev_vrf_enabled == "true" and prev_vrf_configured != "default"
touch ntpd_mgmtvrffile
else
rm ntpd_mgmtvrffile
Diff the Configuration (prev_ and new_ of keyfile, and conffile)
prev_* = new_*
if the Diff cannot be reconfigured (Eg: restart limit hit, keys subtracted, complex changes)
ReStart ntp-daemon service (Stop, Sleep, Start)
continue
Send the Diff via ntpq
if Failure to Send
ReStart ntp-daemon service (Stop, Sleep, Start)
continue
```

### Check And Handle SIGUSR2

This is optional.

```
Dump some statistics/debug to the debug log files
```

### Sleep Forever

```
if sleep was interrupted due to SIGHUP
Throttle it if necessary
```


# Appendix Scripts Design

## ntp-daemon

service

script


## ntp

service

script

## syslog

debug log

log rotation

## techsupport dump file

debug logs

ntp status


0 comments on commit 9eed90e

Please sign in to comment.