Skip to content

Commit

Permalink
Apply suggestions from code review
Browse files Browse the repository at this point in the history
  • Loading branch information
maltesander authored Sep 30, 2024
1 parent 464eb59 commit 8f0d48e
Show file tree
Hide file tree
Showing 4 changed files with 4 additions and 4 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ image::getting_started/spark_running.png[Spark job]

* `pyspark-pi-xxxx`: this is the initializing job that creates the spark-submit command (named as `metadata.name` with a unique suffix)
* `pyspark-pi-xxxxxxx-driver`: the driver pod that drives the execution
* `pythonpi-xxxxxxxxx-exec-x`: the set of executors started by the driver (in our example `spec.executor.instances` was set to 3 which is why 3 executors are running)
* `pythonpi-xxxxxxxxx-exec-x`: the set of executors started by the driver (in the example `spec.executor.instances` was set to 3 which is why 3 executors are running)

Job progress can be followed by issuing this command:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ More information about the different ways to define Spark jobs and their depende
== Stackable Operators

There are multiple ways to install the Stackable Operator for Apache Spark.
xref:management:stackablectl:index.adoc[] is the preferred way but Helm is also supported.
xref:management:stackablectl:index.adoc[] is the preferred way, but Helm is also supported.
OpenShift users may prefer installing the operator from the RedHat Certified Operator catalog using the OpenShift web console.

[tabs]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The Spark operator deploys SparkApplications, and does not offer a UI or other API, so no services are exposed.
However, the operator can also deploy HistoryServers, which do offer a UI and API.
The operator deploys a service called `<name>-historyserver` (where `<name>` is the name of the HistoryServer) through which HistoryServer can be reached.
The operator deploys a service called `<name>-historyserver` (where `<name>` is the name of the spark application) through which the HistoryServer can be reached.

This service can have three different types: `cluster-internal`, `external-unstable` and `external-stable`.
Read more about the types in the xref:concepts:service-exposition.adoc[service exposition] documentation at platform level.
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/spark-k8s/pages/usage-guide/logging.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The Spark operator installs a https://vector.dev/docs/setup/deployment/roles/#agent[vector agent] as a side-car container in every application Pod except the `job` Pod that runs `spark-submit`.
It also configures the logging framework to output logs in XML format.
This is the same https://logging.apache.org/log4j/2.x/manual/layouts.html#XMLLayout[format] used across all Stackable products and it enables the https://vector.dev/docs/setup/deployment/roles/#aggregator[vector aggregator] to collect logs across the entire platform.
This is the same https://logging.apache.org/log4j/2.x/manual/layouts.html#XMLLayout[format] used across all Stackable products, and it enables the https://vector.dev/docs/setup/deployment/roles/#aggregator[vector aggregator] to collect logs across the entire platform.

It is the user's responsibility to install and configure the vector aggregator, but the agents can discover the aggregator automatically using a discovery ConfigMap as described in the xref:concepts:logging.adoc[logging concepts].

Expand Down

0 comments on commit 8f0d48e

Please sign in to comment.