Skip to content

Commit

Permalink
Merge pull request #2 from jotak/main
Browse files Browse the repository at this point in the history
Switch to Quarkus Roq
  • Loading branch information
jotak authored Nov 28, 2024
2 parents 5f87558 + 52635df commit f4a74d5
Show file tree
Hide file tree
Showing 208 changed files with 4,015 additions and 24 deletions.
65 changes: 65 additions & 0 deletions .github/workflows/static.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
## Deploy to GH-Pages for your Quarkus Roq blog.
name: Quarkus Roq Deploy CI

on:
push:
branches: [ main ]
workflow_dispatch:

env:
QUARKUS_ROQ_GENERATOR_BATCH: true

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up JDK 21
uses: actions/setup-java@v4
with:
java-version: 21
distribution: 'temurin'
cache: 'maven'
- name: Get GitHub Pages URL
id: get_url
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
url=$(gh api "repos/$GITHUB_REPOSITORY/pages" --jq '.html_url');echo "ghp-url=$url";
path=$(echo "$url" | sed -E 's|https?://[^/]+(/.*)?|\1|')
url=$(echo "$url" | sed -E 's|(https?://[^/]+).*|\1|')
if [ -z "$path" ]; then
path="/"
fi
echo "SITE_URL=$url"; echo "SITE_URL=$url" >> $GITHUB_ENV;
echo "SITE_PATH=$path"; echo "SITE_PATH=$path" >> $GITHUB_ENV;
- name: Build & Generate Blog
run: mvn -B package quarkus:run -DskipTests -Dquarkus.http.root-path=$SITE_PATH -Dsite.url=$SITE_URL
- name: Upload site as artifact
uses: actions/upload-artifact@v4
with:
name: site
path: target/roq
retention-days: 1
deploy:
runs-on: ubuntu-latest
needs: build
permissions:
pages: write # to deploy to Pages
id-token: write # to verify the deployment originates from an appropriate source
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Download Built site
uses: actions/download-artifact@v4
with:
name: site
path: _site
- name: Setup Pages
uses: actions/configure-pages@v5
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
data/*.json
.quarkus
target/
3 changes: 0 additions & 3 deletions .gitmodules

This file was deleted.

13 changes: 0 additions & 13 deletions 404.md

This file was deleted.

3 changes: 2 additions & 1 deletion CNAME
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
netobserv.io
netobserv.io

48 changes: 42 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,44 @@
# Welcome to [netobserv.io](https://netobserv.io)
# netobserv.io

This site is under construction...
This repository contains the code for the [netobserv.io](https://netobserv.io) web site / blog.

All sources are available on [github.com/netobserv](https://github.com/netobserv)
- Network Observability Operator [getting started guide](https://github.com/netobserv/network-observability-operator/blob/main/README.md)
- Available [blog articles](./documents/blogs/index.md)
- Extra [documents and hacks](./documents/README.md)
## How it works

It uses the static site generator [Quarkus Roq](https://pages.quarkiverse.io/quarkus-roq/) to generate a static HTML site based on markdown pages.

Most of the time, writing content simply requires you to drop a new markdown file in [content/posts](./content/posts/), and assets (such as images) in [static/assets](./static/assets/). A few meta-data is also required at the top of the markdown file, such as:

```
---
layout: :theme/post
title: "Your post title"
description: "The post description"
tags: Insert,Some,Tags,Here
authors: [your-name]
---
```

Then, open a pull-request and the post will be reviewed by maintainers.

First-time contributors should also provide some info about themselves in [data/authors.yml](./data/authors.yml).

## Testing locally

To test locally, the simplest way is to use the [quarkus CLI](https://quarkus.io/guides/cli-tooling).

After you cloned this repository, from its root, run:

```bash
quarkus dev
```

And then open http://localhost:8080/

## Deploying on netobserv.io

Deploying on netobserv.io is fully automated after pull-requests are merged. This is all handled via [a github action](./.github/workflows/static.yml) that runs quarkus to generate the static HTML artifact, which is deployed through GitHub pages.

## Contributions

Like all the other NetObserv components, this blog is open-source and we are welcoming external contributions. Feel free to open pull-requests!
5 changes: 5 additions & 0 deletions content/404.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
layout: :theme/404
link: 404.html
image: iamroq.png
---
12 changes: 12 additions & 0 deletions content/about.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
title: About
description: |
NetObserv aka Network Observability is a set of components used to observe network traffic by generating NetFlows from eBPF agents, enrich those flows using a configurable pipeline that is Kubernetes-aware, export them in various ways (logs, metrics, Kafka, IPFIX...), and finally provide a comprehensive visualization tool for making sense of that data, and a CLI. Those components can be used as standalones or deployed in Kubernetes / OpenShift via an integrated Operator.
layout: :theme/page
---

# About NetObserv

NetObserv is a set of components used to observe network traffic by generating NetFlows from [eBPF agents](https://github.com/netobserv/netobserv-ebpf-agent), enriching those flows using [a configurable pipeline](https://github.com/netobserv/flowlogs-pipeline/) which is Kubernetes-aware, exporting them in various ways (logs, metrics, Kafka, IPFIX...), and finally providing a comprehensive [visualization tool](https://github.com/netobserv/network-observability-console-plugin/) for making sense of that data, and [a CLI](https://github.com/netobserv/network-observability-cli). Those components can be used as standalones or deployed in Kubernetes / OpenShift via an [integrated Operator](https://github.com/netobserv/network-observability-operator/).

It is known and distributed in Red Hat OpenShift as the [Network Observability operator](https://docs.openshift.com/container-platform/latest/observability/network_observability/network-observability-operator-release-notes.html).
21 changes: 21 additions & 0 deletions content/authors.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
title: Authors
description: |
Authors on this blog.
layout: :theme/page
---

# Authors on this blog

<div class="authors">
<!-- authors.yml is in the data/ -->
{#for id in lists:shuffle(cdi:authors.fields)}
{#let author=cdi:authors.get(id)}
<!-- the author-card tag is defined in the default Roq theme -->
{#author-card name=author.name avatar=author.avatar nickname=author.nickname profile=author.profile}
{#if author.bio}
{author.bio}
{/if}
{/author-card}
{/for}
</div>
18 changes: 18 additions & 0 deletions content/events.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
title: Timeline
description: NetObserv timeline
layout: :theme/page
---

<h1>NetObserv timeline</h1>

{#for event in cdi:events.list}
<article class="event">
<div class="event-content">
<span class="post-date"><i class="fa fa-timeline"></i>{event.parsedDate.format('yyyy, MMM dd')}</span>
<h2 class="event-title">{event.title}</h2>
<p>{event.description}</p>
{#if event.link}<a href="{site.url.resolve(event.link)}">More info...</a>{/if}
</div>
</article>
{/for}
17 changes: 17 additions & 0 deletions content/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
title: "NetObserv: network observability on Kubernetes / OpenShift"
description: Components for Network Observability, unleashing eBPF super-power.
name: NetObserv
simple-name: NetObserv
image: netobserv.svg
social-github: netobserv/network-observability-operator/discussions
social-mastodon: https://hachyderm.io/@netobserv
social-bluesky: netobserv.hachyderm.io.ap.brid.gy
social-twitter:
social-facebook:
social-linkedin:
layout: :theme/index
paginate:
collection: posts
size: 15
---
130 changes: 130 additions & 0 deletions content/posts/2022-12-12-3d_topology.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
---
layout: :theme/post
title: "Ways to display your Kubernetes / OpenShift network flows"
description: "With an innovative 3D topology inside"
tags: Topology,Visualization,3D
authors: [jpinsonneau]
---

<div class="heading-info">
Check out the revised version of this post on the Red Hat blog: <a href='https://www.redhat.com/en/blog/how-to-display-your-kubernetes-/-openshift-network-flows'>How to Display Your Kubernetes / OpenShift Network Flows</a>.
</div>

Red Hat OpenShift Container Platform (OCP) has had monitoring capabilities from the start and [Network Observability as Developer Preview](https://cloud.redhat.com/blog/a-guide-to-providing-insight-with-network-observability) since OCP 4.10 release.

Network Observability brings user interfaces in OpenShift web console administration to filter and visualize your cluster network flows as:

- Graph (top 5 flow rates stacked with total example)
![Graph example]({page.image('3d-topology/graph.png')})

- Table (showing sources and destinations sorted by time)
![Table example]({page.image('3d-topology/table.png')})

- Topology (using directed graph layout)
![Topology example]({page.image('3d-topology/topology.png')})

If you are interested in Network Observability for your cluster, check [official documentation for installation](https://docs.openshift.com/container-platform/4.12/networking/network_observability/installing-operators.html).

## Network representations

Topology view is a great way to understand how the different components of your cluster interact internally and identify which ones are communicating with the external world. It can also help you define your network policies or highlight potential threats.

To avoid huge and unreadable network graphs, we introduced in Network Observability multiple options to focus on the content you are looking for.

- Time range
Showing by default the last 5 minutes, it allows you to focus on a particular time window

- Filtering
To select a particular set of records based on Source and / or Destination criterias such as `IPs`, `Ports`, `Protocol`, `Kind`, `Name` etc

- Level of details (scope display option)
Allowing you to select from `Node` / `Namespace` / `Owner` / `Resource` aggregating metrics into the specified layer

- Groups
To arrange components by their ownership

## Example of usage

What happens in your network when you deploy a [httpd sample application](https://github.com/sclorg/httpd-ex) ?

![Sample app deployment]({page.image('3d-topology/sample-app-deployment.png')})

You can see that the final deployed pod in blue involved another pod called `httpd-sample-1-build` that pulled the image from `openshift-image-registry` (1), did some DNS resolution for provided image URL (2), pulled it from the resolved external IP (3) before creating our pod through kubernetes services (4).

Finally, some flows are showing between our pod and `openshift-ingress` after opening the hosted page (5).

After moving my time window or waiting a bit, the `httpd-sample-1-build` pod disappears as it's status is now `Completed`.

![Sample app deployed]({page.image('3d-topology/sample-app-deployed.png')})

## Complex representations

Sometimes we need to see more than a single application to troubleshoot network issues or to highlight bandwidth usage at cluster level.

2D network topology may start to show its limits as you need to switch between multiple options and never get an overview into a single render.

![Complex 2D topology]({page.image('3d-topology/huge-topology.png')})

We can do the following observation from this:
- it is almost impossible to have a good overview of your entire network
- drawing each connection on a flat representation is hard to read
- it is difficult to understand ownership and relations between objects without a concrete representation

## Another dimension

Here is where [3D Topology](https://github.com/jpinsonneau/react-three-topology) makes the scene !

3D representations are pretty old and you can find a lot of them online such as [networkmaps](https://github.com/pablomarle/networkmaps), [vagrant-mesh-net](https://github.com/IMPIMBA/vagrant-mesh-net) or even more generic ones such as [splunk-3D-graph-network-topology-viz](https://github.com/splunk/splunk-3D-graph-network-topology-viz). Each of these has its own rendering philosophy to resolve specific use cases.

This is a way to render your network using a representation everybody knows: "buildings".
```
To enable this feature in Network Observability, you will need to use Network Observability v1.0 and add "&3d=preview" in the URL.
Then go to Topology tab -> Show advanced options -> Display options
From there set Scope option as "Resource" and Layout as "3D".
```

![3D topology building]({page.image('3d-topology/3d-building.png')})

- **Building** drawn by dotted lines represent your `Cluster`
Every internal communication is inside these lines
Each floor is represented by a line
- **Bricks surrounding the building** in a circle shape account for `external ips`
- **Ground floor** with purple bricks represents your `Nodes`
You can have multiple nodes all located at ground level
- **Upper floors** in green shows your `Namespaces`
Namespaces are aligned between nodes so each floor represent a single namespace
If a namespace is not carried by a node, you will see an empty space
- **Rooms** stand for owners such as `Deployments` in dark blue or `StatefulSets` in light blue
You can have multiple owners per namespaces
They will repeat on each node + namespace combination if the load is distributed
- Finally, **rooms contents** depict `Pods`
These are represented in the proper node + namespace + owner combination according to its kindred

![3D topology ownership]({page.image('3d-topology/3d-ownership.png')})
This representation emphasizes the entire ownership chain from `Node` to `Pod` passing by `Namespace`.
It also pin up how your load is balanced between nodes.

![3D topology connections]({page.image('3d-topology/3d-connection.png')})

The important part of your network traffic is highlighted since lines are less likely to cross than in the 2D view and their size and color differ according to bytes rate.
A thin black line will represent a smaller amount than a heavy red line.

## Conclusion

We need to identify the various use cases to elaborate proper representations. This is a daily step by step work between internal teams on both engineering and UI / UX and customers. We are likely to implement more views in the future to highlight network issues, health, threats and usage.
One of the next implementations is going to be [sankey chart](https://observablehq.com/@d3/sankey) that will be useful for connection tracking.

## We need you !

Feel free to contribute by commenting this post, opening issues in [netobserv console plugin](https://github.com/netobserv/network-observability-console-plugin/issues) or opening pull request in any [netobserv component](https://github.com/netobserv)

Tell us more about your expectations, the way you currently solve issues and what could help your daily experience.

![Gallery 1]({page.image('3d-topology/gallery-1.png')})
![Gallery 2]({page.image('3d-topology/gallery-2.png')})
![Gallery 3]({page.image('3d-topology/gallery-3.png')})
![Gallery 4]({page.image('3d-topology/gallery-4.png')})
![Gallery 5]({page.image('3d-topology/gallery-5.png')})
![Gallery 6]({page.image('3d-topology/gallery-6.png')})
![Gallery 7]({page.image('3d-topology/gallery-7.png')})
![Gallery 8]({page.image('3d-topology/gallery-8.png')})
Loading

0 comments on commit f4a74d5

Please sign in to comment.