Skip to content

Commit

Permalink
Update documentation with the new hybrid mode.
Browse files Browse the repository at this point in the history
- Update targets with new 'read' and 'write' options
- Add new docker-compose yaml that exemplifies how to use new 'read' and
  'write' targets
  • Loading branch information
DylanGuedes committed Oct 27, 2021
1 parent 6ff9deb commit d2ef84a
Show file tree
Hide file tree
Showing 4 changed files with 25 additions and 17 deletions.
8 changes: 5 additions & 3 deletions docs/sources/configuration/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,11 @@ Pass the `-config.expand-env` flag at the command line to enable this way of set
### Supported contents and default values of `loki.yaml`

```yaml
# A comma-separated list of components to run. The default value "all" runs
# Loki in single binary mode.
# Supported values: all, compactor, distributor, ingester, querier, query-frontend, table-manager.
# A comma-separated list of components to run.
# The default value "all" runs Loki in single binary mode.
# The value "read" is an alias to run only read-path related components (ex: querier, query-frontend, etc) but all in the same process.
# The value "write" is an alias to run only write-path related components (ex: distributor, compactor) but all in the same process.
# Supported values: all, compactor, distributor, ingester, querier, query-frontend, table-manager, read, write.
[target: <string> | default = "all"]

# Enables authentication through the X-Scope-OrgID header, which must be present
Expand Down
31 changes: 18 additions & 13 deletions docs/sources/fundamentals/architecture/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,28 +16,30 @@ index and in stored chunks.

## Modes of operation

![modes_diagram](modes_of_operation.png)
![modes_diagram](modes.png)

Loki has a set of [components](#components), which
are internally referred to as modules. Each component spawns a gRPC server for
internal traffic and an HTTP/1 server for external API requests. All components
come with an HTTP/1 server, but most only expose readiness, health, and metrics
endpoints.

Loki can run as a single binary; all components are part of the same process.
Or, Loki can run components as microservices.
As microservices, the cluster is horizontally scalable.
Loki can run as:

When invoked, the `-target` flag on the
command line or the `target: <string>` configuration determines
the components' mode: single binary or microservices.
A `target` value of `all` runs Loki in single binary mode.
A `target` value of one of the components invokes that component
as its own microservice.
- A single binary, where all components are part of the same process
- Hybrid, where read-path components run separately from write-path components.
- As microservices, where every component run and scale separately

When invoked, the `-target` flag on the command line or the `target: <string>` configuration determines
the components' mode: monolythic, hybrid, or microservices.
A `target` value of `all` runs Loki in single binary/monolythic mode.
A `target` value of `read` runs all read-path related components (hybrid mode).
A `target` value of `write` runs all write-path related components (hybrid mode).
A `target` value of one of the components (ex: `compactor`) invokes that component as its own microservice (microservices mode).

Each component of Loki, such as the ingesters and distributors, communicate with
one another over gRPC using the gRPC listen port defined in the Loki configuration.
When running components as a single binary, this is still true: each component,
When running components with target `all`, `read` or `write`, this is still true: different components,
although running in the same process, will connect to each other over the local
network for inter-component communication.

Expand All @@ -49,8 +51,11 @@ processes with the following limitations:
with more than one replica, as each replica must be able to
access the same storage backend, and local storage is not safe for concurrent
access.
1. Individual components cannot be scaled independently, so it is not possible
to have more read components than write components.
2. Individual components cannot be scaled independently

Another option is to run in hybrid mode, where Loki is composed of multiple `read` and `write` nodes.
By using this mode, one can scale the read path separately from the write path and vice-versa, but without
the overhead of managing all the different components separately.

## Components

Expand Down
Binary file modified docs/sources/fundamentals/architecture/modes.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 2 additions & 1 deletion pkg/loki/loki.go
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,8 @@ func (c *Config) RegisterFlags(f *flag.FlagSet) {
// Set the default module list to 'all'
c.Target = []string{All}
f.Var(&c.Target, "target", "Comma-separated list of Loki modules to load. "+
"The alias 'all' can be used in the list to load a number of core modules and will enable single-binary mode. ")
"The alias 'all' can be used in the list to load a number of core modules and will enable single-binary mode. "+
"The aliases 'read' and 'write' can be used to only run components related to the read path or write path, respectively.")
f.BoolVar(&c.AuthEnabled, "auth.enabled", true, "Set to false to disable auth.")

c.registerServerFlagsWithChangedDefaultValues(f)
Expand Down

0 comments on commit d2ef84a

Please sign in to comment.