Skip to content

Commit

Permalink
Update examples and include readme
Browse files Browse the repository at this point in the history
  • Loading branch information
lfrancke committed Apr 8, 2020
1 parent 08d7c62 commit 71a90c5
Show file tree
Hide file tree
Showing 3 changed files with 27 additions and 31 deletions.
7 changes: 7 additions & 0 deletions README.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
= Orchestration System

This repository contains a few examples on how to declaratively define services in an Orchestration System. The syntax should be compatible to the Kubernetes syntax.

* link:hdfs-example.yaml[hdfs-example.yaml]
* link:hdfs-crd.yaml[hdfs-crd.yaml]
* link:kafka-example.yaml[kafka-example.yaml]
12 changes: 1 addition & 11 deletions common-example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ spec:

---
apiVersion: stable.opencore.com/v1
kind: ExternalLogStorage # Evtl. wird es ein spezialisierter ElasticsearchLogStorage
kind: ExternalLogStorage # TODO: Do we need an "ElasticsearchExternalLogStorage"? Check with K8S best-practices
metadata:
name: prod-elk
spec:
Expand All @@ -57,7 +57,6 @@ spec:
- prod-eng-policies
- prod-hr-policies


---
apiVersion: stable.opencore.com/v1
kind: Node
Expand All @@ -67,17 +66,8 @@ metadata:
hostName: server-01.opencore.io
network: 10GBit


---
apiVersion: stable.opencore.com/v1
kind: ExternalKafkaCluster
...
name: edge-kafka-unmanaged







POST /api/v1/stable.opencore.com/v1/HDFS/rebalanceHDFS
39 changes: 19 additions & 20 deletions hdfs-example.yml → hdfs-example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,15 @@ metadata:
paidForBy: HR
spec:
config:
# Globale Konfigurationen hier
# Global configuration for all HDFS things go here
configMapRef:
# Globale ConfigMaps hier
# Global ConfigMaps go here

# TODO: Weitere Ranger Properties angeben, gibt es HDFS Properties, die wir bei Ranger injecten müssen?
# TODO: Further Ranger properties. Are there any properties that we need to "inject" into Ranger as well? How'd that work?
rangerServiceRef: ...
rangerPolicySetRef: prod-policies # Möglicherweise mehrere erlauben
rangerPolicySetRef: prod-policies # Potentially allow more than one

# Nur bei HA benötigt
# Only needed when HA is enabled
zooKeeperRef: ...

roleConfig:
Expand All @@ -32,7 +32,7 @@ spec:
configMapRef:


nameNode: # Failover Controller muss hier auch laufen falls HA
nameNode: # This also needs a Failover controller when HA is enabled
minNodeCount: 1
maxNodeCount: 3
highAvailable: true
Expand All @@ -45,9 +45,9 @@ spec:
hostName: server-0[1-3].opencore.io
jvmOptions: ...
config:
dfs.namenode.disks: /foo/01, /foo/02 # TODO: Können kommaseparierte Werte irgendwie strukturierter abgelegt werden?
dfs.namenode.disks: /foo/01, /foo/02 # TODO: Can commaseparated values be provided in a more structured way?
dfs.namenode.http.principal: HTTP
configMapRef: # TODO: Brauchen wir hier mehrere? Wenn ja, wie mergen die? Vorschlag: Priorität vorgeben
configMapRef: # TODO: Do we need more than one? How do we deal with priorities/conflicts?
name: special-config
priority: 1
roleConfig:
Expand All @@ -59,23 +59,23 @@ spec:
- selector:
hostName: server-03.opencore.io
config:
dfs.datanode.disks:/foo/01,/foo/03 # /foo/02 ist kaputt
dfs.datanode.disks:/foo/01,/foo/03 # /foo/02 is broken
...
configMapRef:


secondaryNameNode: # Optional falls HA an ist nicht benötigt
secondaryNameNode: # Optional if HA is not needed
...

journalNode: # TODO: Optional falls nameNodeCount = 1
journalNode: # TODO: Optional if HA is disabled
minNodeCount: 2
maxNodeCount: 3
nodeSelector:
env: production
type: master
config:
dfs.namenode.disks: /foo/01, /foo/02 # TODO: Können kommaseparierte Werte irgendwie strukturierter abgelegt werden?
configMapRef: # TODO: Brauchen wir hier mehrere? Wenn ja, wie mergen die? Vorschlag: Priorität vorgeben
dfs.namenode.disks: /foo/01, /foo/02
configMapRef:
name: special-config
priority: 1

Expand All @@ -86,18 +86,18 @@ spec:
env: production
type: master
config:
dfs.namenode.disks: /foo/01, /foo/02 # TODO: Können kommaseparierte Werte irgendwie strukturierter abgelegt werden?
configMapRef: # TODO: Brauchen wir hier mehrere? Wenn ja, wie mergen die? Vorschlag: Priorität vorgeben
dfs.namenode.disks: /foo/01, /foo/02
configMapRef:
name: special-config
priority: 1

kerberos:
enabled: true
customKeytabScript: /opt/distro/keytabs.sh
nodeSelector: # Selektiert den Rechner auf dem es läuft
nodeSelector: # Selects the machine where we can find the Keytab Script
...

# TODO: In secrets auslagern
# TODO: Move to Secrets or separate Object?
adRef: centalAd
adHost:
adPassword:
Expand All @@ -111,12 +111,11 @@ spec:
certExtendedKeyUsage: clientAuth, serverAuth
certProviderRef: foobar

# Überschreibbar pro Logfiletyp
logging:
retention: ...
rollingInterval: ...
defaultExternalLogStorageRef: prod-elk
# TODO: Evtl. auch diese Sachen in eigene Objekte auslagern damit sie wiederverwendbar sind
# TODO: Potentially introduce a new object for this as well to make it reusable
nameNodeAuditLog:
collect: true
externalLogStorageRef: prod-audit-elk
Expand All @@ -129,4 +128,4 @@ spec:
whitelist: ...


# TODO: HttpFS, NFS Gateway als eigener Service oder hier rein?
# TODO: Should HttpFS, NFS Gateway be their own services or part of this one?

0 comments on commit 71a90c5

Please sign in to comment.