reorg docs, fixup links, etc.

Follow ups to microsoft#101 (comment)

documentation/RepoOrganization.md

@@ -16,6 +16,7 @@ Some notes on the directory layout organization in this repo.
 
  - [Mlos Settings System Code Generation System](../source/Mlos.SettingsSystem.CodeGen/)
  - [Mlos.Core Shared Memory Communication Channel](../source/Mlos.Core/doc/)
+ - [Mlos.Agent.Server](../source/Mlos.Agent.Server/)
 
  - [`source/Examples/`](../source/Examples/) contains sample target codes to optimize with the other MLOS components and help describe the integration methods
 

source/Examples/SmartCache/README.md

@@ -6,108 +6,147 @@ It implements a simple cache with different replacement policies and cache size
 
 It can be used to demonstrate a full end-to-end MLOS integrated microbenchmark for a "smart" component (in this case a cache).
 
-## Overview
+## Contents
+
+- [SmartCache Example](#smartcache-example)
+ - [Contents](#contents)
+ - [Building](#building)
+ - [Linux](#linux)
+ - [Windows](#windows)
+ - [Executing](#executing)
+ - [Without an optimizer](#without-an-optimizer)
+ - [Linux](#linux-1)
+ - [Windows](#windows-1)
+ - [Example output](#example-output)
+ - [With an optimizer](#with-an-optimizer)
+ - [Example output](#example-output-1)
+ - [Explanation](#explanation)
+ - [See Also](#see-also)
 
-To do that, we run the (C++) `SmartCache` executable to communicate with the (C#) [`Mlos.Agent.Server`](../../Mlos.Agent.Server/#mlos-github-tree-view) over a shared memory channel provided by the [`Mlos.Core`](../../Mlos.Core/#mlos-github-tree-view) library.
-
-The `Mlos.Agent.Server` is essentially a small wrapper around several other communication channels to allow different components to connect for component experimentation convenience.
-
-It provides
+## Building
 
-1. Shared memory communication channels via the [`Mlos.Agent`](../../Mlos.Agent/#mlos-github-tree-view) and [`Mlos.NetCore`](../../Mlos.NetCore/#mlos-github-tree-view) libraries.
+See Also
 
-2. A [`Mlos.Agent.GrpcServer`](../../Mlos.Agent.GrpcClient/#mlos-github-tree-view) GRPC channel to allow driving the experimentation process from a Jupyter notebook.
+- General [build](../../../documentation/02-Build.md) instructions
 
-3. A GRPC client to connect to the (Python) [`mlos.Grpc.OptimizerMicroserviceServer`](../../Mlos.Python/mlos/Grpc/OptimizerMicroserviceServer.py#mlos-github-tree-view) to store and track those experiments.
+> Note: these commands are given relative to the root of the MLOS repo.
 
-TODO: Diagrams
+### Linux
 
-## Building
+```sh
+make -C source/Mlos.Agent.Server
+make -C source/Examples/SmartCache all install
+```
 
-> Note: these commands are given relative to the root of the MLOS repo.
->
-> To move there, you can execute the following within the repository:
->
-> `cd $(git rev-parse --show-toplevel)`
+> The `install` target places the output in the more convenient `target/bin/...` path.
 
-To build and run the necessary components for this example
+### Windows
 
-1. [Build the Docker image](../../../documentation/01-Prerequisites.md#build-the-docker-image) using the [`Dockerfile`](../../../Dockerfile#mlos-github-tree-view) at the root of the repository.
+```cmd
+msbuild /m /r source/Mlos.Agent.Server/Mlos.Agent.Server.csproj
+msbuild /m /r source/Examples/SmartCache/SmartCache.vcxproj
+```
 
- ```shell
- docker build --build-arg=UbuntuVersion=20.04 -t mlos/build:ubuntu-20.04 .
- ```
+## Executing
 
-2. [Run the Docker image](../../../documentation/02-Build.md#create-a-new-container-instance) you just built.
+### Without an optimizer
 
- ```shell
- docker run -it -v $PWD:/src/MLOS --name mlos-build mlos/build:ubuntu-20.04
- ```
+#### Linux
 
-3. Inside the container, [build the compiled software](../../../documentation/02-Build.md#cli-make) with `make`:
+```sh
+dotnet target/bin/Release/Mlos.Agent.Server.dll \
+ --executable target/bin/Release/SmartCache
+```
 
- ```sh
- make all install
- ```
+#### Windows
 
- > This will build everything using a default `CONFIGURATION=Release`.
- >
- > To just build `SmartCache` and `Mlos.Agent.Server`, execute the following:
+```sh
+dotnet target/bin/Release/Mlos.Agent.Server.dll \
+ --executable target/bin/Release/SmartCache.exe
+```
 
- ```sh
- # Alternatively:
- make -C source/Examples/SmartCache all install && make -C source/Mlos.Agent.Server
- ```
+#### Example output
 
-4. For a `Release` build (the default), the relevant output will be installed at:
+```txt
+Mlos.Agent.Server
+Starting target/bin/Release/SmartCache
+observations: 0
+info: Microsoft.Hosting.Lifetime[0]
+ Now listening on: http://localhost:5000
+info: Microsoft.Hosting.Lifetime[0]
+ Application started. Press Ctrl+C to shut down.
+info: Microsoft.Hosting.Lifetime[0]
+ Hosting environment: Production
+info: Microsoft.Hosting.Lifetime[0]
+ Content root path: /src/MLOS
+Starting Mlos.Agent
+Found settings registry assembly at target/bin/Release/SmartCache.SettingsRegistry.dll
+observations: 1
+observations: 2
+observations: 3
+...
+```
 
- - Mlos.Agent.Server:
+### With an optimizer
 
- `target/bin/Release/Mlos.Agent.Server.dll`
+The examples above merely communicate the SmartCache's progress with the external agent using shared memory.
 
- - SmartCache:
+To also have the SmartCache tune itself by connecting to an optimizer we need to
 
- `target/bin/Release/SmartCache`
+1. Start an optimizer:
 
- - SmartCache.SettingsRegistry:
+ ```shell
+ start_optimizer_microservice launch --port 50051
+ ```
 
- `target/bin/Release/SmartCache.SettingsRegistry.dll`
+ > This assumes that the `mlos` module has already been installed.
+ >
+ > See the [Python Quickstart documentation](../../../documentation/01-Prerequisites.md#python-quickstart) for details.
 
-## Executing
+2. Add `--optimizer-uri http://localhost:50051` to the set of arguments to the command above.
 
-The following commands will start the `Mlos.Server.Agent` and cause it to start the `SmartCache` component microbenchmark:
+#### Example output
 
 ```sh
-export MLOS_SETTINGS_REGISTRY_PATH="$PWD/target/bin/Release"
-
-tools/bin/dotnet target/bin/Release/Mlos.Agent.Server.dll \
- --executable target/bin/Release/SmartCache \
+dotnet target/bin/Release/Mlos.Agent.Server.dll \
+ --executable target/bin/Release/SmartCache
  --optimizer-uri http://localhost:50051
 ```
 
 ```txt
 Mlos.Agent.Server
+Connecting to the Mlos.Optimizer
 Starting target/bin/Release/SmartCache
 observations: 0
-warn: Microsoft.AspNetCore.Server.Kestrel[0]
- Unable to bind to http://localhost:5000 on the IPv6 loopback interface: 'Cannot assign requested address'.
 info: Microsoft.Hosting.Lifetime[0]
- Now listening on: http://localhost:5000
+ Now listening on: http://[::]:5000
 info: Microsoft.Hosting.Lifetime[0]
  Application started. Press Ctrl+C to shut down.
 info: Microsoft.Hosting.Lifetime[0]
  Hosting environment: Production
 info: Microsoft.Hosting.Lifetime[0]
  Content root path: /src/MLOS
 Starting Mlos.Agent
+Waiting for Mlos.Agent to exit
 Found settings registry assembly at target/bin/Release/SmartCache.SettingsRegistry.dll
+Waiting for agent to respond with a new configuration.
+Register {
+ "cache_implementation": "LeastRecentlyUsed",
+ "lru_cache_config.cache_size": 100
+} HitRate = 0
+Suggest False {"cache_implementation": "LeastRecentlyUsed", "lru_cache_config.cache_size": 747}
 observations: 1
+Waiting for agent to respond with a new configuration.
+Register {
+ "cache_implementation": "LeastRecentlyUsed",
+ "lru_cache_config.cache_size": 747
+} HitRate = 0
+Suggest False {"cache_implementation": "MostRecentlyUsed", "mru_cache_config.cache_size": 2554}
 observations: 2
-observations: 3
 ...
 ```
 
-### Explanation
+## Explanation
 
 The `Mlos.Agent` that gets started by the `Mlos.Agent.Server` waits for a signal that the component (`SmartCache`) has connected to the shared memory region before starting to poll the component for messages to process.
 This is important in case the component is started independently.
@@ -118,35 +157,18 @@ Once started, `SmartCache` will attempt to register its component specific set o
 That includes the name of the `SettingsRegistry` assembly (`.dll`) corresponding to that component's settings/messages.
 
 The `Mlos.Agent.Server` needs to be told where it can find those assemblies in order to load them so that it can process the messages sent by the component.
-To do that, before we started the `Mlos.Agent.Server`, we first populated the `MLOS_SETTINGS_REGISTRY_PATH` environment variable with the directory path to the `SmartCache.SettingsRegistry.dll`.
-
-We also tell the `Mlos.Agent.Server` how to connect to the (Python) MLOS Optimizer Service over GRPC so that the application message handlers setup by the `SmartCache.SettingsRegistry` for the agent can request new configuration recommendations on behave of the application.
-
-For additional details please see the comments in the following code files:
+To do that, before we started the `Mlos.Agent.Server`, we can first populate the `MLOS_SETTINGS_REGISTRY_PATH` environment variable with the directory path to the `SmartCache.SettingsRegistry.dll`.
+In this particular case, everything lives in the same directory path and they're found automatically.
 
-- [Mlos.Agent.Server/MlosAgentServer.cs](../../Mlos.Agent.Server/MlosAgentServer.cs#mlos-github-tree-view)
-- [SmartCache/Main.cpp](./Main.cpp#mlos-github-tree-view)
-- [SmartCache.SettingsRegistry/AssemblyInitializer.cs](./SmartCache.SettingsRegistry/AssemblyInitializer.cs#mlos-github-tree-view)
-- [SmartCache.SettingsRegistry/Codegen/SmartCache.cs](./SmartCache.SettingsRegistry/Codegen/SmartCache.cs#mlos-github-tree-view)
+In the second example we also tell the `Mlos.Agent.Server` how to connect to the (Python) MLOS Optimizer Service over GRPC so that the application message handlers setup by the `SmartCache.SettingsRegistry` for the agent can request new configuration recommendations on behave of the application
 
-## Caveats
+For additional details, see:
 
-- The system currently only supports one shared memory region and doesn't cleanup the shared memory after itself.
+- [Mlos.Agent.Server/MlosAgentServer.cs](https://github.com/Microsoft/MLOS/tree/main/source/Mlos.Agent.Server/MlosAgentServer.cs#mlos-github-tree-view)
+- [SmartCache/Main.cpp](https://github.com/Microsoft/MLOS/tree/main/source/Examples/SmartCache/Main.cpp#mlos-github-tree-view)
+- [SmartCache.SettingsRegistry/AssemblyInitializer.cs](https://github.com/Microsoft/MLOS/tree/main/source/Examples/SmartCache/SmartCache.SettingsRegistry/AssemblyInitializer.cs#mlos-github-tree-view)
+- [SmartCache.SettingsRegistry/Codegen/SmartCache.cs](https://github.com/Microsoft/MLOS/tree/main/source/Examples/SmartCache/SmartCache.SettingsRegistry/Codegen/SmartCache.cs#mlos-github-tree-view)
 
- As such, you may see hung processes when restarting after a failed experiment.
-
- To help with this, we currently provide a helper script to remove previous incarnations of the shared memory regions:
-
- ```sh
- build/CMakeHelpers/RemoveMlosSharedMemories.sh
- ```
-
- You may also need to make sure that the processes using them are killed off.
- For instance:
-
- ```sh
- pkill SmartCache
- pkill -f dotnet.*Mlos.Agent.Server.dll
- ```
+## See Also
 
- > Note: each of these commands should be executed inside the `Mlos.Agent.Server` execution environment (e.g. inside the docker container).
+For additional implementation details and demonstration code, please see the [SmartCache CPP Notebook](https://microsoft.github.io/MLOS/notebooks/SmartCacheCPP)

source/Mlos.Agent.Server/README.md

@@ -0,0 +1,81 @@
+# Mlos.Agent.Server
+
+## Contents
+
+- [Mlos.Agent.Server](#mlosagentserver)
+ - [Contents](#contents)
+ - [Overview](#overview)
+ - [Building](#building)
+ - [Linux](#linux)
+ - [Windows](#windows)
+ - [Executing](#executing)
+ - [Caveats](#caveats)
+
+## Overview
+
+The [`Mlos.Agent.Server`](./#mlos-github-tree-view) is essentially a small and generic C# wrapper around several other communication channels to allow different components to connect for component experimentation convenience.
+
+It provides
+
+1. Shared memory communication channels via the [`Mlos.Agent`](../Mlos.Agent/#mlos-github-tree-view) and [`Mlos.NetCore`](../Mlos.NetCore/#mlos-github-tree-view) libraries.
+
+2. A [`Mlos.Agent.GrpcServer`](../Mlos.Agent.GrpcClient/#mlos-github-tree-view) GRPC channel to allow driving the experimentation process from a Jupyter notebook (work in progress).
+
+3. A GRPC client to allow connecting to the (Python) [`mlos.Grpc.OptimizerMicroserviceServer`](../Mlos.Python/mlos/Grpc/OptimizerMicroserviceServer.py#mlos-github-tree-view) to store and track those experiments.
+
+Since it is meant as a reusable agent for different components, it contains no specific message processing logic itself.
+
+Rather, it starts an [`Mlos.Agent`](../Mlos.Agent/#mlos-github-tree-view) message processing loop which loads each component's SettingsRegistry assembly dlls upon registration request (via the `RegisterComponentConfig` and `RegisterAssemblyRequestMessage` from [`Mlos.Core`](../Mlos.Core/#mlos-github-tree-view) and [`Mlos.NetCore`](../Mlos.NetCore/#mlos-github-tree-view)) and runs their `AssemblyInitializer` constructor to setup the component specific message handler callbacks.
+
+See the [`SmartCache`](../Examples/SmartCache/#mlos-github-tree-view) code, especially it's SettingsRegistry's [`AssemblyInitializer.cs`](../../Example/SmartCache/SmartCache.SettingsRegistry/AssemblyInitializer.cs#mlos-github-tree-view) for a more detailed example.
+
+## Building
+
+See Also
+
+- General [build](../../documentation/02-Build.md) instructions
+
+> Note: these commands are given relative to the root of the MLOS repo.
+
+### Linux
+
+```sh
+make -C source/Mlos.Agent.Server
+```
+
+### Windows
+
+```cmd
+msbuild /m /r source/Mlos.Agent.Server/Mlos.Agent.Server.csproj
+```
+
+## Executing
+
+The `Mlos.Agent.Server` has several different operating modes.
+Please see the `--help` usage output for additional detail.s
+
+```shell
+dotnet target/bin/Release/Mlos.Agent.Server.dll --help
+```
+
+## Caveats
+
+- The system currently only supports one shared memory region and doesn't always cleanup the shared memory after itself if something goes wrong.
+
+ As such, you may see hung processes when restarting after a failed experiment.
+
+ To help with this, we currently provide a helper script to remove previous incarnations of the shared memory regions:
+
+ ```sh
+ build/CMakeHelpers/RemoveMlosSharedMemories.sh
+ ```
+
+ You may also need to make sure that the processes using them are killed off.
+ For instance:
+
+ ```sh
+ pkill $YourSmartComponentProcess # e.g. SmartCache
+ pkill -f dotnet.*Mlos.Agent.Server.dll
+ ```
+
+ > Note: each of these commands should be executed inside the `Mlos.Agent.Server` execution environment (e.g. inside the docker container).

source/Mlos.Notebooks/SmartCacheCPP.ipynb

@@ -11,7 +11,7 @@
  "In this example, the MLOS Agent controls the execution of the workloads on the application, and we will later connect to the agent to optimize the configuration of our application.\n",
  "\n",
  "The application is a \"SmartCache\" similar to the one in the SmartCacheOptimization notebook, though with some more parameters to tune.\n",
- "The source for this example is in the `source/Examples/SmartCache` folder."
+ "The source for this example is in the [`source/Examples/SmartCache`](https://github.com/Microsoft/MLOS/tree/main/source/Examples/SmartCache) folder."
  ]
  },
  {
@@ -56,20 +56,20 @@
  "- Mlos.Agent.Server:\n",
  "\n",
  " This file corresponds to the main entry point for MLOS, written in C#. You can find the source in\n",
- " `source/Mlos.Agent.Server/MlosAgentServer.cs` and the binary at\n",
+ " [`source/Mlos.Agent.Server/MlosAgentServer.cs`](https://github.com/Microsoft/MLOS/tree/main/source/Mlos.Agent.Server/MlosAgentServer.cs) and the binary at\n",
  " `target/bin/Release/Mlos.Agent.Server.dll`\n",
  "\n",
  "- SmartCache:\n",
  "\n",
  " This is the C++ executable that implements the SmartCache and executes some workloads.\n",
- " You can find the source in `source/Examples/SmartCache/Main.cpp` and the binary at\n",
+ " You can find the source in [`source/Examples/SmartCache/Main.cpp`](https://github.com/Microsoft/MLOS/tree/main/source/Examples/SmartCache/Main.cpp) and the binary at\n",
  " `target/bin/Release/SmartCache`\n",
  "\n",
  "- SmartCache.SettingsRegistry:\n",
  "\n",
  " This is the C# code that declares the configuration options for the SmartCache component, and defines the communication\n",
  " between the the MLOS Agent and the SmartCache component. You can find the source in\n",
- " `source/Examples/SmartCache/SmartCache.SettingsRegistry/AssemblyInitializer.cs` and the binary at\n",
+ " [`source/Examples/SmartCache/SmartCache.SettingsRegistry/AssemblyInitializer.cs`](https://github.com/Microsoft/MLOS/tree/main/source/Examples/SmartCache/SmartCache.SettingsRegistry/AssemblyInitializer.cs) and the binary at\n",
  " `target/bin/Release/SmartCache.SettingsRegistry.dll`\n",
  " "
  ]
@@ -80,14 +80,16 @@
  "source": [
  "## Starting the MLOS Agent and executing the workloads:\n",
  "\n",
- "Within the docker container, we can now tell the agent where the configuration options are stored, by setting the `MLOS_Settings_REGISTRY_PATH`.\n",
+ "Within the docker container, we can now tell the agent where the configuration options are stored, by setting the `MLOS_SETTINGS_REGISTRY_PATH`.\n",
  "Then, we can run the MLOS Agent, which will in turn run the SmartCache executable.\n",
  "```sh\n",
  "export MLOS_SETTINGS_REGISTRY_PATH=\"target/bin/Release\"\n",
  "\n",
  "tools/bin/dotnet target/bin/Release/Mlos.Agent.Server.dll \\\n",
  " --executable target/bin/Release/SmartCache\n",
- "```"
+ "```\n",
+ "\n",
+ "For a more detailed explanation of what's going on, please see the [SmartCache README](https://microsoft.github.io/MLOS/source/Examples/SmartCache#explanation)"
  ]
  },
  {
@@ -487,4 +489,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 4
-}
+}