Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Topotato docs getting started #139

Open
wants to merge 3 commits into
base: topotato-base
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ["sphinx.ext.todo", "sphinx.ext.graphviz", "sphinx.ext.autodoc"]
extensions = ["sphinx.ext.todo", "sphinx.ext.graphviz", "sphinx.ext.autodoc", "sphinx_rtd_theme"]

# Add any paths that contain templates here, relative to this directory.
templates_path = ["_templates"]
Expand Down
114 changes: 114 additions & 0 deletions doc/getting-started/first-test.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
:description: Writing your first topotato test

=======================
Writing your first test
=======================


Example test
------------

1. Create a new file called `test_sample.py` in topotato folder, containing a function, and a test:

.. code-block:: python

# content of test_sample.py

#!/usr/bin/env python3
# SPDX-License-Identifier: GPL-2.0-or-later
# Copyright (C) 2018-2023 YOUR NAME HERE for NetDEF, Inc.
"""
Simple demo test for topotato.
"""

from topotato.v1 import *

@topology_fixture()
def topology(topo):
"""
[ r1 ]---[ noprot ]
[ ]
[ ]---[ rip ]
[ ]
[ ]---[ ripng ]
[ ]
[ ]---[ ospfv2 ]
[ ]
[ ]---[ ospfv3 ]
[ ]
[ ]---[ isisv4 ]
[ ]
[ ]---[ isisv6 ]
"""
topo.router("r1").iface_to("ripng").ip6.append("fc00:0:0:1::1/64")


class Configs(FRRConfigs):
routers = ["r1"]

zebra = """
#% extends "boilerplate.conf"
#% block main
#% for iface in router.ifaces
interface {{ iface.ifname }}
description {{ iface.other.endpoint.name }}
no link-detect
!
#% endfor
!
ip forwarding
ipv6 forwarding
!
#% endblock
"""

ripd = """
#% extends "boilerplate.conf"
#% block main
debug rip events
debug rip zebra
!
router rip
version 2
network {{ router.iface_to('rip').ip4[0].network }}
#% endblock
"""

ripngd = """
#% extends "boilerplate.conf"
#% block main
debug ripng events
debug ripng zebra
!
router ripng
network {{ router.iface_to('ripng').ip6[0].network }}
#% endblock
"""


class AllStartupTest(TestBase, AutoFixture, topo=topology, configs=Configs):
"""
docstring here
"""

@topotatofunc
def test_running(self, topo, r1):
"""
just check that all daemons are running
"""
for daemon in Configs.daemons:
if not hasattr(Configs, daemon):
continue
yield from AssertVtysh.make(r1, daemon, command="show version")


2. Run the following command in your command line to run the test:

.. code-block:: bash

./run_userns.sh --frr-builddir=$PATH_TO_FRR_BUILD \
--log-cli-level=DEBUG \
-v -v -x \
sameple_test.py

3. If you should see all your test in green it means every works fine.
48 changes: 48 additions & 0 deletions doc/getting-started/installation.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
:description: Topotato installation

=========================
Installing topotato
=========================


Package installation
--------------------

Topotato lives inside FRR project so there is no installation for topotato itself.
What is needed is to install its dependencies.

.. note::
``topotato`` requires Python >= 3.8

Run theses commands below:


.. code-block:: bash

sysctl -w kernel.unprivileged_userns_clone=1


.. code-block::
:caption: Required packages

apt-get satisfy \
'graphviz' 'tshark (>=4.0)' 'wireshark-common (>=4.0)' 'tini' \
'python3 (>=3.8)' \
'python3-pytest (>=6.1)' 'python3-pytest-xdist' \
'python3-typing-extensions' 'python3-docutils' 'python3-pyinotify' \
'python3-scapy (>=2.4.5)' 'python3-exabgp (>=4.2)' \
'python3-jinja2 (>=3.1)' 'python3-lxml' 'python3-markupsafe' \
'wireshark-common' 'tini'


.. note::
if you are using ``non-debian`` based distribution, you can install theses packages above and below manually

- unshare - run program with some namespaces unshared from parent
- nsenter - run program with namespaces of other processes
- tini_ - Tini is a tiny init
- dumpcap - Dump network traffic
- ip - show / manipulate routing, network devices, interfaces and tunnels


.. _tini: https://github.com/krallin/tini
32 changes: 32 additions & 0 deletions doc/getting-started/introduction.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
:description: Learn how to get started with Topotato for FRRouting

=======================
Welcome to Topotato 🔝🥔
=======================

Topotato is a work-in-progress test framework designed for conducting system-level conformance tests for FRRouting.
Its primary purpose is to execute FRR through various scenarios and validate its behavior.

Goals
-----

The creation of Topotato was motivated by addressing the limitations of the existing FRR test framework, topotests.

Topotato aims to achieve the following objectives:

- Enhance the readability and comprehensibility of tests, particularly when failures occur.
- Improve the reliability of tests, ensuring consistent outcomes irrespective of factors like system load, parallelization, execution order, operating system, or architecture.
- Simplify the test-writing process.
- Minimize the variability in expressing a specific test, ideally reducing it to a single way. This streamlines the process of identifying the correct approach to articulate a test condition, minimizing the potential for creating unstable tests (i.e., flaky tests). A standardized approach to expressing tests also reduces the learning curve and facilitates troubleshooting when a test fails.
- Enhance the utility of test reports, primarily for failure cases but also for successful ones. Test behavior and reasons for failure should be readily understandable without the need for extensive investigation, debugging statements, or repeated test runs.
- Enable easy execution of the test suite on developers' local development systems, ensuring accessibility and speed.

Secondary Goals
---------------

In addition to the primary objectives, Topotato also aims to achieve the following secondary goals, which are influenced by the aforementioned aims:

- Encapsulate tests within a single file to eliminate the need to navigate through multiple files.
- Replace hardcoded IP addresses with semantic expressions to improve readability. For instance, while ``192.168.13.57`` is an opaque IPv4 address, the expression ``r1.iface_to('r2').ip4[0]`` could represent the same address while being more comprehensible and easier to maintain.
- Enable the test suite to run without necessitating "root" access to the system or the installation of FRR. This approach ensures ease of execution and guarantees that the test suite cannot disrupt the developer's system due to kernel-level protection. Additionally, it mitigates issues stemming from broken or mismatched installations.
- Support FreeBSD.
30 changes: 22 additions & 8 deletions doc/index.rst
Original file line number Diff line number Diff line change
@@ -1,13 +1,27 @@
topotato manual
===============
Topotato Manual
===============

Getting Started (Guide)
-----------------------

.. toctree::
:maxdepth: 2

getting-started/introduction
getting-started/installation
getting-started/first-test

Topotato Internals
------------------

.. toctree::
:maxdepth: 2

introduction
parse
assertions
machinery
frrglue
pytest_lowlevel
utils
internals/introduction
internals/parse
internals/assertions
internals/machinery
internals/frrglue
internals/pytest_lowlevel
internals/utils
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
4 changes: 4 additions & 0 deletions vm/ubuntu/topotato-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,7 @@ sudo DEBIAN_FRONTEND=noninteractive apt install graphviz tshark \
python3-venv wireshark-common python3-pip tini -y

pip install -r requirements.txt

# sphinx documentation

pip install sphinx_rtd_theme