Skip to content

Commit 7ec2732

Browse files
PrabhuSreenivasanjghalamdutta-parthasrideepDellnirenjan
authored and
abdosi
committed
SONiC Management Framework Release 1.0 (sonic-net#3488)
* Added sonic-mgmt-framework as submodule / docker * fix build issues * update sonic-mgmt-framework submodule branch to master * Merged changes 70007e6d2ba3a4c0b371cd693ccc63e0a8906e77..00d4fcfed6a759e40d7b92120ea0ee1f08300fc6 00d4fcfed6a759e40d7b92120ea0ee1f08300fc6 Modified environemnt variables * Changes to build sonic-mgmt-framework docker * bumped up sonic-mgmt-framework commit-id * version bump for sonic-mgmt-framework commit-it * bumped up sonic-mgmt-framework commit-id * Add python packages to docker * Build fix for docker with python packages * added libyang as dependent package * Allow building images on NFS-mounted clones Prior to this change, `build_debian.sh` would generate a Debian filesystem in `./fsroot`. This needs root permissions, and one of the tests that is performed is whether the user can create a character special file in the filesystem (using mknod). On most NFS deployments, `root` is the least privileged user, and cannot run mknod. Also, attempting to run commands like rm or mv as root would fail due to permission errors, since the root user gets mapped to an unprivileged user like `nobody`. This commit changes the location of the Debian filesystem to `/fsroot`, which is a tmpfs mount within the slave Docker. The default squashfs, docker tarball and zip files are also created within /tmp, before being copied back to /sonic as the regular user. The side effect of this change is that the contents of `/fsroot` are no longer available once the slave container exits, however they are available within the squashfs image. Signed-off-by: Nirenjan Krishnan <Nirenjan.Krishnan@dell.com> * bumped up sonc-mgmt-framework commit to include PR #18 * REST Server startup script is enahnced to read the settings from ConfigDB. Below table provides mapping of db field to command line argument name. ============================================================ ConfigDB entry key Field name REST Server argument ============================================================ REST_SERVER|default port -port REST_SERVER|default client_auth -client_auth REST_SERVER|default log_level -v DEVICE_METADATA|x509 server_crt -cert DEVICE_METADATA|x509 server_key -key DEVICE_METADATA|x509 ca_crt -cacert ============================================================ * Replace src/telemetry as submodule to sonic-telemetry * Update telemetry commit HEAD * Update sonic-telemetry commit HEAD * libyang env path update * Add libyang dependency to telemetry * Add scripts to create JSON files for CLI backend Scripts to create /var/platform/syseeprom and /var/platform/system, which are back-end files for CLI, for system EEPROM and system information. Signed-off-by: Howard Persh <Howard_Persh@dell.com> * In startup script, create directory where CLI back-end files live Signed-off-by: Howard Persh <Howard_Persh@dell.com> * build dependency pkgs added to docker for build failure fix * Changes to fix build issue for mgmt framework * Fix exec path issue with telemetry * s5232[device] PSU detecttion and default led state support * Processing of first boot in rc.local should not have premature exit Signed-off-by: Howard Persh <Howard_Persh@dell.com> * docker mount options added for platform, system features * bumped up sonic-mgmt-framework commit id to pick 23rd July 2019 changes * Added mount options for telemetry docker to get access for system and platform info. * Update commit for sonic-utilities * [dell]: Corrected dport map and renamed config files for S5232F * Fix telemetry submodule commit * added support for sonic-cli console * [Dell S5232F, Z9264F] Harden FPGA driver kernel module For Dell S5232F and Z9264F platforms, be more strict when checking state in ISR of FPGA driver, to harden against spurious interrupts. Signed-off-by: Howard Persh <Howard_Persh@dell.com> * update mgmt-framework submodule to 27th Aug commit. * remove changes not related to mgmt-framework and sonic-telemetry * Revert "Replace src/telemetry as submodule to sonic-telemetry" This reverts commit 11c3192. * Revert "Replace src/telemetry as submodule to sonic-telemetry" This reverts commit 11c3192. * make submodule changes and remove a change not related to PR * more changes * Update .gitmodules * Update Dockerfile.j2 * Update .gitmodules * Update .gitmodules * Update .gitmodules reverting experimental change * Removed syspoll for release_1.0 Signed-off-by: Jeff Yin <29264773+jeff-yin@users.noreply.github.com> * Update docker-sonic-mgmt-framework.mk * Update sonic-mgmt-framework.mk * Update sonic-mgmt-framework.mk * Update docker-sonic-mgmt-framework.mk * Update docker-sonic-mgmt-framework.mk * Revert "Processing of first boot in rc.local should not have premature exit" This reverts commit e99a91f. * Remove old telemetry directory * Update docker-sonic-mgmt-framework.mk * Resolving merge conflict with Azure * Reverting the wrong merge * Use CVL_SCHEMA_PATH instead of changing directory for telemetry startup * Add missing export * Add python mmh3 to slave dockerfile * Remove sonic-mgmt-framework build dep for telemetry, fix dialout startup issues * Provided flag to disable compiling mgmt-framework * Update sonic-utilites point latest commit id * Point sonic-utilities to Azure accepted SHA * Updating mgmt framework to right sha * Add sonic-telemetry submodule * Update the mgmt-framework commit id Co-authored-by: jghalam <joe.ghalam@gmail.com> Co-authored-by: Partha Dutta <51353699+dutta-partha@users.noreply.github.com> Co-authored-by: srideepDell <srideep_devireddy@dell.com> Co-authored-by: nirenjan <nirenjan@users.noreply.github.com> Co-authored-by: Sachin Holla <51310506+sachinholla@users.noreply.github.com> Co-authored-by: Eric Seifert <seiferteric@gmail.com> Co-authored-by: Howard Persh <hpersh@yahoo.com> Co-authored-by: Jeff Yin <29264773+jeff-yin@users.noreply.github.com> Co-authored-by: Arunsundar Kannan <31632515+arunsundark@users.noreply.github.com> Co-authored-by: rvasanthm <51932293+rvasanthm@users.noreply.github.com> Co-authored-by: Ashok Daparthi-Dell <Ashok_Daparthi@Dell.com> Co-authored-by: anand-kumar-subramanian <51383315+anand-kumar-subramanian@users.noreply.github.com>
1 parent 6045e34 commit 7ec2732

25 files changed

+225
-64
lines changed

‎.gitmodules

+6
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,12 @@
6666
[submodule "platform/mellanox/mlnx-sai/SAI-Implementation"]
6767
path = platform/mellanox/mlnx-sai/SAI-Implementation
6868
url = https://github.com/Mellanox/SAI-Implementation
69+
[submodule "src/sonic-mgmt-framework"]
70+
path = src/sonic-mgmt-framework
71+
url = https://github.com/Azure/sonic-mgmt-framework
72+
[submodule "src/sonic-telemetry"]
73+
path = src/sonic-telemetry
74+
url = https://github.com/Azure/sonic-telemetry
6975
[submodule "Switch-SDK-drivers"]
7076
path = platform/mellanox/sdk-src/sx-kernel/Switch-SDK-drivers
7177
url = https://github.com/Mellanox/Switch-SDK-drivers

‎dockers/docker-sonic-mgmt-framework/Dockerfile.j2

+42
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
FROM docker-config-engine-stretch
2+
3+
ARG docker_container_name
4+
RUN [ -f /etc/rsyslog.conf ] && sed -ri "s/%syslogtag%/$docker_container_name#%syslogtag%/;" /etc/rsyslog.conf
5+
6+
## Make apt-get non-interactive
7+
ENV DEBIAN_FRONTEND=noninteractive
8+
9+
RUN apt-get update
10+
RUN pip install connexion==1.1.15 \
11+
setuptools==21.0.0 \
12+
grpcio-tools==1.20.0 \
13+
pyangbind==0.6.0 \
14+
certifi==2017.4.17 \
15+
python-dateutil==2.6.0 \
16+
six==1.11.0 \
17+
urllib3==1.21.1
18+
19+
20+
21+
## Install redis-tools dependencies
22+
## TODO: implicitly install dependencies
23+
RUN apt-get -y install libjemalloc1 libatomic1 liblua5.1-0 lua-bitop lua-cjson
24+
25+
COPY \
26+
{% for deb in docker_sonic_mgmt_framework_debs.split(' ') -%}
27+
debs/{{ deb }}{{' '}}
28+
{%- endfor -%}
29+
debs/
30+
31+
RUN dpkg -i \
32+
{% for deb in docker_sonic_mgmt_framework_debs.split(' ') -%}
33+
debs/{{ deb }}{{' '}}
34+
{%- endfor %}
35+
36+
COPY ["start.sh", "rest-server.sh", "/usr/bin/"]
37+
COPY ["supervisord.conf", "/etc/supervisor/conf.d/"]
38+
39+
RUN apt-get clean -y; apt-get autoclean -y; apt-get autoremove -y
40+
RUN rm -rf /debs
41+
42+
ENTRYPOINT ["/usr/bin/supervisord"]

‎dockers/docker-sonic-mgmt-framework/base_image_files/sonic-cli

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
#!/bin/bash
2+
3+
docker exec -it mgmt-framework /usr/sbin/cli/clish_start "$@"
4+

‎dockers/docker-sonic-mgmt-framework/rest-server.sh

+52
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
#!/usr/bin/env bash
2+
3+
# Startup script for SONiC Management REST Server
4+
5+
SERVER_PORT=
6+
LOG_LEVEL=
7+
CLIENT_AUTH=
8+
SERVER_CRT=
9+
SERVER_KEY=
10+
CA_CERT=
11+
12+
# Read basic server settings from REST_SERVER|default entry
13+
HAS_REST_CONFIG=$(sonic-cfggen -d -v "1 if REST_SERVER and REST_SERVER['default']")
14+
if [ "$HAS_REST_CONFIG" == "1" ]; then
15+
SERVER_PORT=$(sonic-cfggen -d -v "REST_SERVER['default']['port']")
16+
CLIENT_AUTH=$(sonic-cfggen -d -v "REST_SERVER['default']['client_auth']")
17+
LOG_LEVEL=$(sonic-cfggen -d -v "REST_SERVER['default']['log_level']")
18+
fi
19+
20+
# Read certificate file paths from DEVICE_METADATA|x509 entry.
21+
HAS_X509_CONFIG=$(sonic-cfggen -d -v "1 if DEVICE_METADATA and DEVICE_METADATA['x509']")
22+
if [ "$HAS_X509_CONFIG" == "1" ]; then
23+
SERVER_CRT=$(sonic-cfggen -d -v "DEVICE_METADATA['x509']['server_crt']")
24+
SERVER_KEY=$(sonic-cfggen -d -v "DEVICE_METADATA['x509']['server_key']")
25+
CA_CRT=$(sonic-cfggen -d -v "DEVICE_METADATA['x509']['ca_crt']")
26+
fi
27+
28+
# Create temporary server certificate if they not configured in ConfigDB
29+
if [ -z $SERVER_CRT ] && [ -z $SERVER_KEY ]; then
30+
echo "Generating temporary TLS server certificate ..."
31+
(cd /tmp && /usr/sbin/generate_cert --host="localhost,127.0.0.1")
32+
SERVER_CRT=/tmp/cert.pem
33+
SERVER_KEY=/tmp/key.pem
34+
fi
35+
36+
37+
REST_SERVER_ARGS="-ui /rest_ui -logtostderr"
38+
[ ! -z $SERVER_PORT ] && REST_SERVER_ARGS+=" -port $SERVER_PORT"
39+
[ ! -z $LOG_LEVEL ] && REST_SERVER_ARGS+=" -v $LOG_LEVEL"
40+
[ ! -z $CLIENT_AUTH ] && REST_SERVER_ARGS+=" -client_auth $CLIENT_AUTH"
41+
[ ! -z $SERVER_CRT ] && REST_SERVER_ARGS+=" -cert $SERVER_CRT"
42+
[ ! -z $SERVER_KEY ] && REST_SERVER_ARGS+=" -key $SERVER_KEY"
43+
[ ! -z $CA_CRT ] && REST_SERVER_ARGS+=" -cacert $CA_CRT"
44+
45+
echo "REST_SERVER_ARGS = $REST_SERVER_ARGS"
46+
47+
48+
export CVL_SCHEMA_PATH=/usr/sbin/schema
49+
export LIBYANG_EXTENSIONS_PLUGINS_DIR=/usr/lib/x86_64-linux-gnu/libyang/extensions
50+
export LIBYANG_USER_TYPES_PLUGINS_DIR=/usr/lib/x86_64-linux-gnu/libyang/user_types
51+
52+
exec /usr/sbin/rest_server ${REST_SERVER_ARGS}

‎dockers/docker-sonic-mgmt-framework/start.sh

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
#!/usr/bin/env bash
2+
3+
mkdir -p /var/sonic
4+
echo "# Config files managed by sonic-config-engine" > /var/sonic/config_status
5+
6+
rm -f /var/run/rsyslogd.pid
7+
8+
supervisorctl start rsyslogd
9+
10+
supervisorctl start rest-server

‎dockers/docker-sonic-mgmt-framework/supervisord.conf

+28
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
[supervisord]
2+
logfile_maxbytes=1MB
3+
logfile_backups=2
4+
nodaemon=true
5+
6+
[program:start.sh]
7+
command=/usr/bin/start.sh
8+
priority=1
9+
autostart=true
10+
autorestart=false
11+
stdout_logfile=syslog
12+
stderr_logfile=syslog
13+
14+
[program:rsyslogd]
15+
command=/usr/sbin/rsyslogd -n
16+
priority=2
17+
autostart=false
18+
autorestart=false
19+
stdout_logfile=syslog
20+
stderr_logfile=syslog
21+
22+
[program:rest-server]
23+
command=/usr/bin/rest-server.sh
24+
priority=3
25+
autostart=false
26+
autorestart=true
27+
stdout_logfile=syslog
28+
stderr_logfile=syslog

‎dockers/docker-sonic-telemetry/dialout.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#!/usr/bin/env bash
22

33
# Start with default config
4-
4+
export CVL_SCHEMA_PATH=/usr/sbin/schema
55
exec /usr/sbin/dialout_client_cli -insecure -logtostderr -v 2
66

‎dockers/docker-sonic-telemetry/supervisord.conf

+2-2
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,8 @@ nodaemon=true
66
[eventlistener:supervisor-proc-exit-listener]
77
command=/usr/bin/supervisor-proc-exit-listener
88
events=PROCESS_STATE_EXITED
9-
autostart=always
10-
autorestart=unexpected
9+
autostart=true
10+
autorestart=false
1111

1212
[program:start.sh]
1313
command=/usr/bin/start.sh

‎dockers/docker-sonic-telemetry/telemetry.sh

+1
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ X509=`sonic-cfggen -d -v "DEVICE_METADATA['x509']"`
66
TELEMETRY=`sonic-cfggen -d -v 'TELEMETRY.keys() | join(" ") if TELEMETRY'`
77

88
TELEMETRY_ARGS=" -logtostderr"
9+
export CVL_SCHEMA_PATH=/usr/sbin/schema
910

1011
if [ -n "$X509" ]; then
1112
SERVER_CRT=`sonic-cfggen -d -v "DEVICE_METADATA['x509']['server_crt']"`

‎files/build_templates/mgmt-framework.service.j2

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
[Unit]
2+
Description=Management Framework container
3+
Requires=swss.service
4+
After=swss.service
5+
Before=ntp-config.service
6+
7+
[Service]
8+
User={{ sonicadmin_user }}
9+
ExecStartPre=/usr/bin/{{docker_container_name}}.sh start
10+
ExecStart=/usr/bin/{{docker_container_name}}.sh wait
11+
ExecStop=/usr/bin/{{docker_container_name}}.sh stop
12+
13+
[Install]
14+
WantedBy=multi-user.target

‎files/image_config/platform/rc.local

+3
Original file line numberDiff line numberDiff line change
@@ -331,4 +331,7 @@ if [ -f $FIRST_BOOT_FILE ]; then
331331
firsttime_exit
332332
fi
333333

334+
# Create dir where following scripts put their output files
335+
mkdir -p /var/platform
336+
334337
exit 0

‎rules/config

+3
Original file line numberDiff line numberDiff line change
@@ -98,3 +98,6 @@ DEFAULT_VS_PREPARE_MEM = yes
9898

9999
# ENABLE_SYSTEM_SFLOW - build docker-sonic-sflow for sFlow support
100100
ENABLE_SFLOW = y
101+
102+
# ENABLE_MGMT_FRAMEWORK - build docker-sonic-mgt-framework for CLI and REST server support
103+
ENABLE_MGMT_FRAMEWORK = y

‎rules/docker-sonic-mgmt-framework.mk

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# docker image for mgmt-framework
2+
3+
DOCKER_MGMT_FRAMEWORK_STEM = docker-sonic-mgmt-framework
4+
DOCKER_MGMT_FRAMEWORK = $(DOCKER_MGMT_FRAMEWORK_STEM).gz
5+
DOCKER_MGMT_FRAMEWORK_DBG = $(DOCKER_MGMT_FRAMEWORK_STEM)-$(DBG_IMAGE_MARK).gz
6+
7+
$(DOCKER_MGMT_FRAMEWORK)_PATH = $(DOCKERS_PATH)/$(DOCKER_MGMT_FRAMEWORK_STEM)
8+
9+
$(DOCKER_MGMT_FRAMEWORK)_DEPENDS += $(REDIS_TOOLS) $(SONIC_MGMT_FRAMEWORK)
10+
$(DOCKER_MGMT_FRAMEWORK)_DBG_DEPENDS = $($(DOCKER_CONFIG_ENGINE_STRETCH)_DBG_DEPENDS)
11+
$(DOCKER_MGMT_FRAMEWORK)_DBG_DEPENDS += $(REDIS_TOOLS) $(SONIC_MGMT_FRAMEWORK_DBG)
12+
13+
SONIC_DOCKER_IMAGES += $(DOCKER_MGMT_FRAMEWORK)
14+
$(DOCKER_MGMT_FRAMEWORK)_LOAD_DOCKERS += $(DOCKER_CONFIG_ENGINE_STRETCH)
15+
$(DOCKER_MGMT_FRAMEWORK)_DBG_IMAGE_PACKAGES = $($(DOCKER_CONFIG_ENGINE_STRETCH)_DBG_IMAGE_PACKAGES)
16+
17+
ifeq ($(ENABLE_MGMT_FRAMEWORK), y)
18+
SONIC_INSTALL_DOCKER_IMAGES += $(DOCKER_MGMT_FRAMEWORK)
19+
SONIC_STRETCH_DOCKERS += $(DOCKER_MGMT_FRAMEWORK)
20+
endif
21+
22+
SONIC_DOCKER_DBG_IMAGES += $(DOCKER_MGMT_FRAMEWORK_DBG)
23+
ifeq ($(ENABLE_MGMT_FRAMEWORK), y)
24+
SONIC_INSTALL_DOCKER_DBG_IMAGES += $(DOCKER_MGMT_FRAMEWORK_DBG)
25+
SONIC_STRETCH_DBG_DOCKERS += $(DOCKER_MGMT_FRAMEWORK_DBG)
26+
endif
27+
28+
$(DOCKER_MGMT_FRAMEWORK)_CONTAINER_NAME = mgmt-framework
29+
$(DOCKER_MGMT_FRAMEWORK)_RUN_OPT += --net=host --privileged -t
30+
$(DOCKER_MGMT_FRAMEWORK)_RUN_OPT += -v /etc/sonic:/etc/sonic:ro
31+
$(DOCKER_MGMT_FRAMEWORK)_RUN_OPT += -v /etc:/host_etc:ro
32+
$(DOCKER_MGMT_FRAMEWORK)_RUN_OPT += --mount type=bind,source="/var/platform/",target="/mnt/platform/"
33+
34+
$(DOCKER_MGMT_FRAMEWORK)_BASE_IMAGE_FILES += sonic-cli:/usr/bin/sonic-cli

‎rules/docker-telemetry.mk

+1
Original file line numberDiff line numberDiff line change
@@ -27,5 +27,6 @@ endif
2727
$(DOCKER_TELEMETRY)_CONTAINER_NAME = telemetry
2828
$(DOCKER_TELEMETRY)_RUN_OPT += --net=host --privileged -t
2929
$(DOCKER_TELEMETRY)_RUN_OPT += -v /etc/sonic:/etc/sonic:ro
30+
$(DOCKER_TELEMETRY)_RUN_OPT += --mount type=bind,source="/var/platform/",target="/mnt/platform/"
3031

3132
$(DOCKER_TELEMETRY)_FILES += $(SUPERVISOR_PROC_EXIT_LISTENER_SCRIPT)

‎rules/sonic-mgmt-framework.mk

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# SONiC mgmt-framework package
2+
3+
ifeq ($(ENABLE_MGMT_FRAMEWORK), y)
4+
5+
SONIC_MGMT_FRAMEWORK = sonic-mgmt-framework_1.0-01_amd64.deb
6+
$(SONIC_MGMT_FRAMEWORK)_SRC_PATH = $(SRC_PATH)/sonic-mgmt-framework
7+
$(SONIC_MGMT_FRAMEWORK)_DEPENDS = $(LIBYANG_DEV) $(LIBYANG)
8+
$(SONIC_MGMT_FRAMEWORK)_RDEPENDS = $(LIBYANG)
9+
SONIC_DPKG_DEBS += $(SONIC_MGMT_FRAMEWORK)
10+
11+
SONIC_MGMT_FRAMEWORK_DBG = sonic-mgmt-framework-dbg_1.0-01_amd64.deb
12+
$(SONIC_MGMT_FRAMEWORK_DBG)_DEPENDS += $(SONIC_MGMT_FRAMEWORK)
13+
$(SONIC_MGMT_FRAMEWORK_DBG)_RDEPENDS += $(SONIC_MGMT_FRAMEWORK)
14+
$(eval $(call add_derived_package,$(SONIC_MGMT_FRAMEWORK),$(SONIC_MGMT_FRAMEWORK_DBG)))
15+
16+
endif

‎rules/telemetry.mk

+3-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# SONiC telemetry package
22

33
SONIC_TELEMETRY = sonic-telemetry_0.1_$(CONFIGURED_ARCH).deb
4-
$(SONIC_TELEMETRY)_SRC_PATH = $(SRC_PATH)/telemetry
4+
$(SONIC_TELEMETRY)_SRC_PATH = $(SRC_PATH)/sonic-telemetry
5+
$(SONIC_TELEMETRY)_DEPENDS = $(LIBYANG_DEV) $(LIBYANG)
6+
$(SONIC_TELEMETRY)_RDEPENDS = $(LIBYANG)
57
SONIC_DPKG_DEBS += $(SONIC_TELEMETRY)

‎sonic-slave-stretch/Dockerfile.j2

+3
Original file line numberDiff line numberDiff line change
@@ -354,6 +354,9 @@ RUN pip install mockredispy==2.9.3
354354
RUN pip install pytest-runner==4.4
355355
RUN pip install setuptools==40.8.0
356356

357+
# For mgmt-framework build
358+
RUN pip install mmh3
359+
357360
# Install dependencies for isc-dhcp-relay build
358361
RUN apt-get -y build-dep isc-dhcp
359362

‎src/sonic-mgmt-framework

Submodule sonic-mgmt-framework added at 8b199a9

‎src/sonic-telemetry

Submodule sonic-telemetry added at aaa9188

‎src/telemetry/Makefile

-20
This file was deleted.

‎src/telemetry/debian/changelog

-5
This file was deleted.

‎src/telemetry/debian/compat

-1
This file was deleted.

‎src/telemetry/debian/control

-17
This file was deleted.

‎src/telemetry/debian/rules

-3
This file was deleted.

‎src/telemetry/debian/telemetry.init.d

-14
This file was deleted.

0 commit comments

Comments
 (0)