v0.11.0 release

README.rst

@@ -54,12 +54,12 @@ The documentation is available at: https://networkattacksimulator.readthedocs.io
 
 
 
-Using with OpenAI gym
+Using with gymnasium
 ---------------------
 
-NASim implements the `Open AI Gym <https://github.com/openai/gym>`_ environment interface and so can be used with any algorithm that is developed for that interface.
+NASim implements the `Gymnasium <https://github.com/Farama-Foundation/Gymnasium/tree/main>`_ environment interface and so can be used with any algorithm that is developed for that interface.
 
-See `Starting NASim using OpenAI gym <https://networkattacksimulator.readthedocs.io/en/latest/tutorials/gym_load.html>`_.
+See `Starting NASim using gymnasium <https://networkattacksimulator.readthedocs.io/en/latest/tutorials/gym_load.html>`_.
 
 
 Authors
@@ -79,6 +79,12 @@ License
 What's new
 ----------
 
+- 2023-03-13 (v 0.11.0) (MINOR release)
+
+  + Migrated to `gymnasium (formerly Open AI gym) <https://github.com/Farama-Foundation/Gymnasium/>`_ fromOpen AI gym (thanks @rzvnbr for the suggestion).
+  + Fixed bug with action string representation (thanks @rzvnbr for the bug report)
+  + Added "sim to real considerations" explanation document to the docs (thanks @Tudyx for the suggestion)
+
 - 2023-02-27 (v 0.10.1) (MICRO release)
 
   + Fixed bug for host based actions (thanks @nguyen-thanh20 for the bug report)

docs/requirements.txt

@@ -1,60 +1,4 @@
-alabaster==0.7.12
-attrs==22.1.0
-Babel==2.10.3
-bleach==5.0.1
-cffi==1.15.1
-charset-normalizer==2.1.0
-cloudpickle==2.1.0
-commonmark==0.9.1
-cycler==0.11.0
-docutils==0.17.1
-fonttools==4.34.4
-gym==0.25.1
-gym-notices==0.0.7
-idna==3.3
-imagesize==1.4.1
-importlib-metadata==4.12.0
-iniconfig==1.1.1
-jeepney==0.8.0
-Jinja2==3.1.2
-keyring==23.7.0
-kiwisolver==1.4.4
-MarkupSafe==2.1.1
-matplotlib==3.5.2
-networkx==2.8.5
-numpy==1.23.1
-packaging==21.3
-pkginfo==1.8.3
-pluggy==1.0.0
-prettytable==3.3.0
-py==1.11.0
-pycparser==2.21
-Pygments==2.12.0
-pyparsing==3.0.9
-pytest==7.1.2
-python-dateutil==2.8.2
-pytz==2022.1
-PyYAML==6.0
-readme-renderer==35.0
-requests==2.28.1
-requests-toolbelt==0.9.1
-rfc3986==2.0.0
-rich==12.5.1
-SecretStorage==3.3.2
-six==1.16.0
-snowballstemmer==2.2.0
-Sphinx==5.1.1
-sphinx-rtd-theme==1.0.0
-sphinxcontrib-applehelp==1.0.2
-sphinxcontrib-devhelp==1.0.2
-sphinxcontrib-htmlhelp==2.0.0
-sphinxcontrib-jsmath==1.0.1
-sphinxcontrib-qthelp==1.0.3
-sphinxcontrib-serializinghtml==1.1.5
-tomli==2.0.1
-twine==4.0.1
-typing_extensions==4.3.0
-urllib3==1.26.11
-wcwidth==0.2.5
-webencodings==0.5.1
-zipp==3.8.1
+nasim
+sphinx
+sphinx-autobuild
+sphinx-rtd-theme

docs/source/conf.py

@@ -12,6 +12,7 @@
 #
 import os
 import sys
+import nasim
 sys.path.insert(0, os.path.abspath(os.path.join('..', '..')))
 
 
@@ -22,7 +23,7 @@
 author = 'Jonathon Schwartz'
 
 # The full version, including alpha/beta/rc tags
-release = '0.10.0'
+release = nasim.__version__
 
 
 # -- General configuration ---------------------------------------------------

docs/source/explanations/index.rst

@@ -9,3 +9,4 @@ More technical explanations related to NASim.
     :maxdepth: 1
 
     scenario_generation
+    sim_to_real

docs/source/explanations/sim_to_real.rst

@@ -0,0 +1,23 @@
+.. _sim_to_real_explanation:
+
+Sim-to-Real Gap Considerations
+==============================
+
+NASim is a fairly simplified simulator of network penetration testing. It's main goal is to capture some of the key features of network pentesting in a easy-to-use and fast simulator so that it can be used for rapid testing and prototyping of algorithms before these algorithms are tested on more realistic environments. That is to say there is a bit of gap between the scenarios in NASim and the real world.
+
+In this document we wanted to lay down some considerations to think about when trying to extend your algorithm beyond NASim. This is by no means an exhaustive list, but will hopefully give you something to think about for the next steps, and also give an explanation of some of the design decisions made in NASim.
+
+.. note:: This document is a work in progress so if you have any thoughts, useful references, etc on the topic of applying autonomous penetration testing in the real-world please reach out via email or open an issue on github.
+
+Handling Partial Observability
+------------------------------
+
+One of the big assumptions made by NASim is that the pentester agent has access to the network addresses of every host in the network, even in partially observable mode. This information is given to the agent in it's list of actions. In practice in the real-world, depending on the scenario, this assumption may be invalid, and part of the challenge for the pentester is to be able to discover new hosts as they navigate through the network.
+
+The main reason NASim is implemented with the network addresses being known is so that the action space size could be fixed, making it simpler to use with typical Deep Reinforcement Learning algorithms (i.e. with neural nets with fixed size input and output layers).
+
+One of the research challenges is to develop algorithms that can handle action spaces that change as the pentester discovers more network addresses, or perhaps more realistic would be that the pentester's action space is mult-dimensional and includes choosing an address and exploit/scan/etc separately. There actually is some support for this built into NASim with the nasim.envs.action.ParameterisedActionSpace action space (see :ref:`actions`), but even using that action space some information about the size of the network is given to the pentester.
+
+At this stage there is no plans to update NASim to support the no-information action space. This is partially due to time, but also to keep NASim simple and stable and because there are a lot of even better and more realistic environments being developed now (e.g. `CybORG <https://github.com/cage-challenge/CybORG>`_.)
+
+One avenue for handling changing action space is to use auto-regressive actions as was done by `AlphaStar <https://www.deepmind.com/blog/alphastar-mastering-the-real-time-strategy-game-starcraft-ii>`_.

docs/source/index.rst

@@ -3,12 +3,19 @@ Welcome to Network Attack Simulator's documentation!
 
 Network Attack Simulator (NASim) is a lightweight, high-level network attack simulator written in python. It is designed to be used for rapid testing of autonomous pen-testing agents using reinforcement learning and planning. It is a simulator by definition so does not replicate all details of attacking a real system but it instead aims to capture some of the more salient features of network pen-testing such as the large and changing sizes of the state and action spaces, partial observability and varied network topology.
 
-The environment is modelled after the `OpenAI gym <https://github.com/openai/gym>`_ interface.
+The environment is modelled after the `gymnasium (formerly Open AI gym) <https://github.com/Farama-Foundation/Gymnasium/>`_ interface.
 
 
 What's new
 ----------
 
+Version 0.11.0
+**************
+
++ Migrated to `gymnasium (formerly Open AI gym) <https://github.com/Farama-Foundation/Gymnasium/>`_ fromOpen AI gym (thanks @rzvnbr for the suggestion).
++ Fixed bug with action string representation (thanks @rzvnbr for the bug report)
++ Added "sim to real considerations" explanation document to the docs (thanks @Tudyx for the suggestion)
+
 
 Version 0.10.1
 **************

docs/source/tutorials/environment.rst

@@ -3,7 +3,7 @@
 Interacting with NASim Environment
 ==================================
 
-Assuming you are comfortable loading an environment from a scenario (see :ref:`loading_tute` or :ref:`gym_load_tute`), then interacting with a NASim Environment is very easy and follows the same interface as `OpenAI gym <https://github.com/openai/gym>`_.
+Assuming you are comfortable loading an environment from a scenario (see :ref:`loading_tute` or :ref:`gym_load_tute`), then interacting with a NASim Environment is very easy and follows the same interface as `gymnasium <https://github.com/Farama-Foundation/Gymnasium/>`_.
 
 
 Starting the environment
@@ -16,8 +16,8 @@ First thing is simply loading the environment::
   env = nasim.make_benchmark("tiny")
 
   # or using gym
-  import gym
-  env. = gym.make("nasim:Tiny-PO-v0")
+  import gymnasium as gym
+  env = gym.make("nasim:Tiny-PO-v0")
 
 
 Here we are using the default environment parameters: ``fully_obs=False``, ``flat_actions=True``, and ``flat_obs=True``.
@@ -42,7 +42,10 @@ Getting the initial observation and resetting the environment
 
 To reset the environment and get the initial observation, use the ``reset()`` function::
 
-  o = env.reset()
+  o, info = env.reset()
+
+
+The ``info`` return value contains optional auxiliary information.
 
 
 Performing a single step
@@ -57,6 +60,25 @@ A step in the environment can be taken using the ``step(action)`` function. Here
 if ``done=True`` then the goal has been reached, and the episode is over. Alternatively, if the current scenario has a step limit and ``step_limit_reached=True`` then, well, the step limit has been reached. Following both cases, it is then recommended to stop or reset the environment, otherwise theres no gaurantee of what will happen (especially the first case).
 
 
+Visualizing the environment
+---------------------------
+
+You can use the ``render()`` function to get a human readable visualization of the state of the environment. To use render correctly make sure to pass ``render_mode="human"`` to the environment initialization function::
+
+  import nasim
+  # load my environment in the desired way (make_benchmark, load, generate)
+  env = nasim.make_benchmark("tiny", render_mode="human")
+
+  # or using gym
+  import gymnasium as gym
+  env = gym.make("nasim:Tiny-PO-v0", render_mode="human")
+
+  env.reset()
+  # render the environment
+  # (if render_mode="human" is not passed during initialization this will do nothing)
+  env.render()
+
+
 An example agent
 ----------------
 
@@ -68,7 +90,7 @@ Some example agents are provided in the ``nasim/agents`` directory. Here is a qu
 
   agent = AnAgent(...)
 
-  o = env.reset()
+  o, info = env.reset()
   total_reward = 0
   done = False
   step_limit_reached = False

docs/source/tutorials/gym_load.rst

@@ -3,19 +3,19 @@
 Starting NASim using OpenAI gym
 ===============================
 
-On startup NASim also registers each benchmark scenario as an `Open AI Gym <https://github.com/openai/gym>`_ , allowing NASim benchmark environments to be loaded using ``gym.make()``.
+On startup NASim also registers each benchmark scenario as an `Gymnasium <https://github.com/Farama-Foundation/Gymnasium/>`_  environment, allowing NASim benchmark environments to be loaded using ``gymnasium.make()``.
 
-:ref:`all_benchmark_scenarios` can be loaded using ``gym.make()``.
+:ref:`all_benchmark_scenarios` can be loaded using ``gymnasium.make()``.
 
 .. note:: Custom scenarios must be loaded using the nasim library directly, see :ref:`loading_tute`.
 
 
 Environment Naming
 ------------------
 
-Unlike when starting an environment using the ``nasim`` library directly, where environment modes are specified as arguments to the ``nasim.make_benchmark()`` function, when using ``gym.make()`` the scenario and mode are specified in a single name.
+Unlike when starting an environment using the ``nasim`` library directly, where environment modes are specified as arguments to the ``nasim.make_benchmark()`` function, when using ``gymnasium.make()`` the scenario and mode are specified in a single name.
 
-When using ``gym.make()`` each environment has the following mode and naming convention:
+When using ``gymnasium.make()`` each environment has the following mode and naming convention:
 
   ``ScenarioName[PO][2D][VA]-vX``
 
@@ -53,8 +53,8 @@ For example to create a new ``TinyPO-v0`` environment:
 
 .. code:: python
 
-   import gym
+   import gymnasium as gym
    env = gym.make("nasim:TinyPO-v0")
 
-
-.. note:: With the latest updates in ``v0.10.0`` and new Open AI gym API, you may need to specify ``new_step_api=True`` when calling the ``gym.make`` function if your implementation expects the new API. This is due to how the default wrappers that are created when calling ``gym.make`` handle things.
+   # to specify render mode
+   env = gym.make("nasim:TinyPO-v0", render_mode="human")

docs/source/tutorials/loading.rst

@@ -5,7 +5,7 @@ Starting a NASim Environment
 
 Interaction with NASim is done primarily via the :class:`~nasim.envs.environment.NASimEnv` class, which handles a simulated network environment as defined by the chosen scenario.
 
-There are two ways to start a new environment: (i) via the nasim library directly, or (ii) using the `gym.make()` function of the Open AI gym library.
+There are two ways to start a new environment: (i) via the nasim library directly, or (ii) using the `gym.make()` function of the gymnasium library.
 
 In this tutorial we will be covering the first method. For the second method check out :ref:`gym_load_tute`.
 

nasim/__init__.py

@@ -1,5 +1,5 @@
-import gym
-from gym.envs.registration import register
+import gymnasium as gym
+from gymnasium.envs.registration import register
 
 from nasim.envs import NASimEnv
 from nasim.scenarios.benchmark import AVAIL_BENCHMARKS
@@ -14,7 +14,8 @@ def make_benchmark(scenario_name,
                    seed=None,
                    fully_obs=False,
                    flat_actions=True,
-                   flat_obs=True):
+                   flat_obs=True,
+                   render_mode=None):
     """Make a new benchmark NASim environment.
 
     Parameters
@@ -32,6 +33,8 @@ def make_benchmark(scenario_name,
     flat_obs : bool, optional
         if true then uses a 1D observation space. If False
         will use a 2D observation space (default=True)
+    render_mode : str, optional
+            The render mode to use for the environment.
 
     Returns
     -------
@@ -45,7 +48,8 @@ def make_benchmark(scenario_name,
     """
     env_kwargs = {"fully_obs": fully_obs,
                   "flat_actions": flat_actions,
-                  "flat_obs": flat_obs}
+                  "flat_obs": flat_obs,
+                  "render_mode": render_mode}
     scenario = make_benchmark_scenario(scenario_name, seed)
     return NASimEnv(scenario, **env_kwargs)
 
@@ -54,7 +58,8 @@ def load(path,
          fully_obs=False,
          flat_actions=True,
          flat_obs=True,
-         name=None):
+         name=None,
+         render_mode=None):
     """Load NASim Environment from a .yaml scenario file.
 
     Parameters
@@ -73,6 +78,8 @@ def load(path,
     name : str, optional
         the scenarios name, if None name will be generated from path
         (default=None)
+    render_mode : str, optional
+            The render mode to use for the environment.
 
     Returns
     -------
@@ -81,7 +88,8 @@ def load(path,
     """
     env_kwargs = {"fully_obs": fully_obs,
                   "flat_actions": flat_actions,
-                  "flat_obs": flat_obs}
+                  "flat_obs": flat_obs,
+                  "render_mode": render_mode}
     scenario = load_scenario(path, name=name)
     return NASimEnv(scenario, **env_kwargs)
 
@@ -91,6 +99,7 @@ def generate(num_hosts,
              fully_obs=False,
              flat_actions=True,
              flat_obs=True,
+             render_mode=None,
              **params):
     """Construct Environment from an auto generated network.
 
@@ -109,6 +118,8 @@ def generate(num_hosts,
     flat_obs : bool, optional
         if true then uses a 1D observation space. If False
         will use a 2D observation space (default=True)
+    render_mode : str, optional
+            The render mode to use for the environment.
     params : dict, optional
         generator params (see :class:`ScenarioGenertor` for full list)
 
@@ -119,14 +130,14 @@ def generate(num_hosts,
     """
     env_kwargs = {"fully_obs": fully_obs,
                   "flat_actions": flat_actions,
-                  "flat_obs": flat_obs}
+                  "flat_obs": flat_obs,
+                  "render_mode": render_mode}
     scenario = generate_scenario(num_hosts, num_services, **params)
     return NASimEnv(scenario, **env_kwargs)
 
 
-# Register NASimEnv with OpenAI gym
 def _register(id, entry_point, kwargs, nondeterministic, force=True):
-    """Registers NASim Open AI Gym Environment.
+    """Registers NASim as a Gymnasium Environment.
 
     Handles issues with re-registering gym environments.
     """
@@ -199,3 +210,5 @@ def _register(id, entry_point, kwargs, nondeterministic, force=True):
             },
             nondeterministic=True
         )
+
+__version__ = "0.11.0"