Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

devops: split nodes by responsibilities (infrastructure VS runtime) #268

Closed
diegodelemos opened this issue Feb 17, 2020 · 0 comments · Fixed by reanahub/reana-commons#192, #337, reanahub/reana-job-controller#255, reanahub/reana-workflow-controller#315 or reanahub/reana-db#72
Closed

devops: split nodes by responsibilities (infrastructure VS runtime) #268

diegodelemos opened this issue Feb 17, 2020 · 0 comments · Fixed by reanahub/reana-commons#192, #337, reanahub/reana-job-controller#255, reanahub/reana-workflow-controller#315 or reanahub/reana-db#72
Assignees
@diegodelemos
Labels
deployment/cern deployment/helm system/scalability

Comments

@diegodelemos
Copy link
Member

We shall investigate the option to use certain nodes in the production environments to run the infrastructure pods (R-S, R-W-C, R-Message-Broker, R-Cache etc..) and the rest, the runtime pods, RWE_|RJC and the user jobs in different nodes.

This will help to:

  • Have a predictable response time for the application: if the cluster is very intensively used, there will be fewer resources for the REANA system to automatically scale, decreasing its responsiveness
  • Avoid that potential crashes produced by user jobs affect the rest of the application, and therefore the rest of the users
  • Clearly differentiate the resources used by the REANA application from those used by the user jobs
  • This can pave the way to use external Kubernetes clusters to run REANA workloads (a first step towards federation)
  • ...
@diegodelemos diegodelemos mentioned this issue Feb 28, 2020
Closed
diegodelemos pushed a commit to diegodelemos/reana that referenced this issue Jun 11, 2020
templates: add optional runtime namespace
824b347 
* Allows having two different namespaces for infrastructure and runtime
  pods (closes reanahub#268).
@diegodelemos diegodelemos mentioned this issue Jun 11, 2020
Merged
diegodelemos pushed a commit to diegodelemos/reana-workflow-controller that referenced this issue Jun 11, 2020
k8s: infrastructure and runtime namespace aware
5dee03a 
* Creates REANA runtime objects in the configured runtime namespace.
  Note that if no runtime namespace is configured by default it will
  be the same as the infrastructure pods namespace
  (closes reanahub/reana#268).
diegodelemos pushed a commit to diegodelemos/reana-commons that referenced this issue Jun 11, 2020
config: separete infra and runtime namespaces
37e7c59 
* Creates a new configuration variable to hold the namespace used
  to run runtime pods in REANA (closes reanahub/reana#268).

* Creates a central configuration variable for fully qualified
  infrastructure component names so any component can contact them
  from a different namespace.
diegodelemos pushed a commit to diegodelemos/reana-job-controller that referenced this issue Jun 11, 2020
k8s: run jobs in configured runtime namespace
2732fbf 
* Closes reanahub/reana#268.
@diegodelemos diegodelemos self-assigned this Jun 11, 2020
diegodelemos pushed a commit to diegodelemos/reana-db that referenced this issue Jun 11, 2020
config: uses central FQDN component name
616d0cd 
* Uses the centrall host name for database which will include the
  fully qualified domain name inside the cluster, including the
  namespace enabling RJC to communicate to it from different
  namespaces (closes reanahub/reana#268).
diegodelemos pushed a commit to diegodelemos/reana-db that referenced this issue Jun 11, 2020
config: uses central FQDN component name
607e62b 
* Uses the centrall host name for database which will include the
  fully qualified domain name inside the cluster, including the
  namespace enabling RJC to communicate to it from different
  namespaces (closes reanahub/reana#268).
diegodelemos pushed a commit to diegodelemos/reana-job-controller that referenced this issue Jun 11, 2020
config: use centrally configured k8s namespace
e5987bc 
* In order to be able to deploy REANA in different Kubernetes
  namespaces we use the centrally configured variable from
  REANA-Commons which represents it. This configuration happens
  at deployment time and can be different than the namespace
  in which the REANA infrastructure pods run
  (closes reanahub/reana#274 reanahub/reana#268).
diegodelemos pushed a commit to diegodelemos/reana-commons that referenced this issue Jun 11, 2020
config: separete infra and runtime namespaces
b0dfda6 
* Creates a new configuration variable to hold the namespace used
  to run runtime pods in REANA (closes reanahub/reana#268).

* Creates a central configuration variable for fully qualified
  infrastructure component names so any component can contact them
  from a different namespace.
diegodelemos pushed a commit to diegodelemos/reana-commons that referenced this issue Jun 11, 2020
config: separate infra and runtime namespaces
8c2347e 
* Creates a new configuration variable to hold the namespace used
  to run runtime pods in REANA (closes reanahub/reana#268).

* Creates a central configuration variable for fully qualified
  infrastructure component names so any component can contact them
  from a different namespace.
diegodelemos pushed a commit to diegodelemos/reana-workflow-controller that referenced this issue Jun 11, 2020
config: use centrally configured k8s namespace
e9acdf1 
* In order to be able to deploy REANA in different Kubernetes
  namespaces we use the centrally configured variable from
  REANA-Commons which represents it. This configuration happens
  at deployment time (closes reanahub/reana#274).

* Creates REANA runtime objects in the configured runtime namespace.
  Note that if no runtime namespace is configured by default it will
  be the same as the infrastructure pods namespace
  (closes reanahub/reana#268).
diegodelemos pushed a commit to diegodelemos/reana-job-controller that referenced this issue Jun 11, 2020
config: use centrally configured k8s namespace
77d555d 
* In order to be able to deploy REANA in different Kubernetes
  namespaces we use the centrally configured variable from
  REANA-Commons which represents it. This configuration happens
  at deployment time and can be different than the namespace
  in which the REANA infrastructure pods run
  (closes reanahub/reana#274, closes reanahub/reana#268).
diegodelemos pushed a commit to diegodelemos/reana that referenced this issue Jun 11, 2020
templates: add optional runtime namespace
18e79e3 
* Allows having two different namespaces for infrastructure and runtime
  pods (closes reanahub#268).
diegodelemos pushed a commit to diegodelemos/reana-commons that referenced this issue Jun 11, 2020
config: separate infra and runtime namespaces
d358ecc 
* Creates a new configuration variable to hold the namespace used
  to run runtime pods in REANA (closes reanahub/reana#268).

* Creates a central configuration variable for fully qualified
  infrastructure component names so any component can contact them
  from a different namespace.
This was referenced Jun 11, 2020
Merged
Merged
Merged
Merged
Merged
diegodelemos pushed a commit to diegodelemos/reana that referenced this issue Jun 19, 2020
templates: add optional runtime namespace
8e46eeb 
* Allows having two different namespaces for infrastructure and runtime
  pods (closes reanahub#268).
diegodelemos pushed a commit to diegodelemos/reana-job-controller that referenced this issue Jun 19, 2020
config: use centrally configured k8s namespace
6221f60 
* In order to be able to deploy REANA in different Kubernetes
  namespaces we use the centrally configured variable from
  REANA-Commons which represents it. This configuration happens
  at deployment time and can be different than the namespace
  in which the REANA infrastructure pods run
  (closes reanahub/reana#274, closes reanahub/reana#268).
diegodelemos pushed a commit to diegodelemos/reana-db that referenced this issue Jun 19, 2020
config: uses central FQDN component name
f0d3d0b 
* Uses the central host name for database which will include the
  fully qualified domain name inside the cluster, including the
  namespace enabling RJC to communicate to it from different
  namespaces (closes reanahub/reana#268).
mdonadoni added a commit to mdonadoni/reana that referenced this issue Mar 5, 2024
@mdonadoni
chore: aggregate changelog
a493fb8 
chore(reana-ui/master): release 0.9.4

build(reana-ui/package): update yarn.lock (reanahub#399)

build(reana-ui/package): require jsroot<7.6.0 (reanahub#399)

ci(reana-ui/commitlint): allow release commit style (reanahub#400)

docs(reana-ui/authors): complete list of contributors (reanahub#396)

ci(reana-ui/shellcheck): exclude node_modules from the analyzed paths (reanahub#387)

fix(reana-ui/progress): update failed workflows duration using finish time (reanahub#387)

feat(reana-ui/footer): link privacy notice to configured URL (reanahub#393)

refactor(reana-ui/docs): move from reST to Markdown (reanahub#391)

ci(reana-ui/commitlint): check for the presence of concrete PR number (reanahub#390)

ci(reana-ui/shellcheck): fix exit code propagation (reanahub#390)

fix(reana-ui/launcher): remove dollar sign in generated Markdown (reanahub#389)

ci(reana-ui/release-please): update version in package.json and Dockerfile (reanahub#385)

ci(reana-ui/release-please): switch to `simple` release strategy (reanahub#383)

fix(reana-ui/router): show 404 page for invalid URLs (reanahub#382)

ci(reana-ui/release-please): initial configuration (reanahub#380)

ci(reana-ui/commitlint): addition of commit message linter (reanahub#380)

chore(reana-message-broker/master): release 0.9.3

ci(reana-message-broker/commitlint): allow release commit style (reanahub#67)

docs(reana-message-broker/authors): complete list of contributors (reanahub#66)

refactor(reana-message-broker/docs): move from reST to Markdown (reanahub#65)

ci(reana-message-broker/commitlint): check for the presence of concrete PR number (reanahub#64)

ci(reana-message-broker/shellcheck): fix exit code propagation (reanahub#64)

ci(reana-message-broker/release-please): update version in Dockerfile (reanahub#63)

fix(reana-message-broker/startup): handle signals for graceful shutdown (reanahub#59)

ci(reana-message-broker/release-please): initial configuration (reanahub#60)

ci(reana-message-broker/commitlint): addition of commit message linter (reanahub#60)

chore(reana-server/master): release 0.9.3

build(reana-server/python): bump all required packages as of 2024-03-04 (reanahub#674)

build(reana-server/python): bump shared REANA packages as of 2024-03-04 (reanahub#674)

build(reana-server/python): bump shared modules (reanahub#676)

ci(reana-server/commitlint): allow release commit style (reanahub#675)

docs(reana-server/authors): complete list of contributors (reanahub#673)

ci(reana-server/pytest): move to PostgreSQL 14.10 (reanahub#672)

refactor(reana-server/docs): move from reST to Markdown (reanahub#671)

style(reana-server/black): format with black v24 (reanahub#670)

ci(reana-server/commitlint): check for the presence of concrete PR number (reanahub#669)

ci(reana-server/shellcheck): fix exit code propagation (reanahub#669)

ci(reana-server/release-please): update version in Dockerfile/OpenAPI specs (reanahub#668)

build(reana-server/docker): non-editable submodules in "latest" mode (reanahub#656)

build(reana-server/deps): pin invenio-userprofiles to 1.2.4 (reanahub#665)

ci(reana-server/release-please): initial configuration (reanahub#665)

ci(reana-server/commitlint): addition of commit message linter (reanahub#665)

chore(reana-workflow-controller/master): release 0.9.3

build(reana-workflow-controller/python): bump all required packages as of 2024-03-04 (reanahub#574)

build(reana-workflow-controller/python): bump shared REANA packages as of 2024-03-04 (reanahub#574)

feat(reana-workflow-controller/manager): increase termination period of run-batch pods (reanahub#572)

ci(reana-workflow-controller/commitlint): allow release commit style (reanahub#575)

feat(reana-workflow-controller/manager): pass custom env variables to job controller (reanahub#571)

feat(reana-workflow-controller/manager): pass custom env variables to workflow engines (reanahub#571)

docs(reana-workflow-controller/authors): complete list of contributors (reanahub#570)

ci(reana-workflow-controller/pytest): move to PostgreSQL 14.10 (reanahub#568)

fix(reana-workflow-controller/manager): use valid group name when calling `groupadd` (reanahub#566)

refactor(reana-workflow-controller/docs): move from reST to Markdown (reanahub#567)

fix(reana-workflow-controller/stop): store engine logs of stopped workflow (reanahub#563)

fix(reana-workflow-controller/manager): graceful shutdown of job-controller (reanahub#559)

feat(reana-workflow-controller/manager): call shutdown endpoint before workflow stop (reanahub#559)

refactor(reana-workflow-controller/consumer): do not update status of jobs (reanahub#559)

style(reana-workflow-controller/black): format with black v24 (reanahub#564)

ci(reana-workflow-controller/commitlint): check for the presence of concrete PR number (reanahub#562)

ci(reana-workflow-controller/shellcheck): fix exit code propagation (reanahub#562)

ci(reana-workflow-controller/release-please): update version in Dockerfile/OpenAPI specs (reanahub#558)

build(reana-workflow-controller/docker): non-editable submodules in "latest" mode (reanahub#551)

ci(reana-workflow-controller/release-please): initial configuration (reanahub#555)

ci(reana-workflow-controller/commitlint): addition of commit message linter (reanahub#555)

chore(reana-job-controller/master): release 0.9.3

build(reana-job-controller/python): bump all required packages as of 2024-03-04 (reanahub#442)

build(reana-job-controller/python): bump shared REANA packages as of 2024-03-04 (reanahub#442)

ci(reana-job-controller/commitlint): allow release commit style (reanahub#443)

build(reana-job-controller/certificates): update expired CERN Grid CA certificate (reanahub#440)

fix(reana-job-controller/database): limit the number of open database connections (reanahub#437)

docs(reana-job-controller/authors): complete list of contributors (reanahub#434)

perf(reana-job-controller/cache): avoid caching jobs when the cache is disabled (reanahub#435)

ci(reana-job-controller/pytest): move to PostgreSQL 14.10 (reanahub#429)

refactor(reana-job-controller/docs): move from reST to Markdown (reanahub#428)

ci(reana-job-controller/commitlint): check for the presence of concrete PR number (reanahub#425)

ci(reana-job-controller/shellcheck): fix exit code propagation (reanahub#425)

feat(reana-job-controller/shutdown): stop all running jobs before stopping workflow (reanahub#423)

refactor(reana-job-controller/monitor): move fetching of logs to job-manager (reanahub#423)

refactor(reana-job-controller/db): set job status also in the main database (reanahub#423)

refactor(reana-job-controller/monitor): centralise logs and status updates (reanahub#423)

style(reana-job-controller/black): format with black v24 (reanahub#426)

ci(reana-job-controller/release-please): update version in Dockerfile/OpenAPI specs (reanahub#421)

build(reana-job-controller/docker): non-editable submodules in "latest" mode (reanahub#416)

ci(reana-job-controller/release-please): initial configuration (reanahub#417)

ci(reana-job-controller/commitlint): addition of commit message linter (reanahub#417)

chore(reana-workflow-engine-cwl/master): release 0.9.3

build(reana-workflow-engine-cwl/python): bump all required packages as of 2024-03-04 (reanahub#267)

build(reana-workflow-engine-cwl/python): bump shared REANA packages as of 2024-03-04 (reanahub#267)

docs(reana-workflow-engine-cwl/conformance-tests): update CWL conformance test badges (reanahub#264)

ci(reana-workflow-engine-cwl/commitlint): allow release commit style (reanahub#268)

docs(reana-workflow-engine-cwl/authors): complete list of contributors (reanahub#266)

refactor(reana-workflow-engine-cwl/docs): move from reST to Markdown (reanahub#263)

fix(reana-workflow-engine-cwl/progress): handle stopped jobs (reanahub#260)

ci(reana-workflow-engine-cwl/commitlint): check for the presence of concrete PR number (reanahub#262)

ci(reana-workflow-engine-cwl/shellcheck): fix exit code propagation (reanahub#262)

build(reana-workflow-engine-cwl/docker): install correct extras of reana-commons submodule (reanahub#261)

ci(reana-workflow-engine-cwl/release-please): update version in Dockerfile (reanahub#259)

build(reana-workflow-engine-cwl/docker): non-editable submodules in "latest" mode (reanahub#255)

ci(reana-workflow-engine-cwl/release-please): initial configuration (reanahub#256)

ci(reana-workflow-engine-cwl/commitlint): addition of commit message linter (reanahub#256)

chore(reana-workflow-engine-serial/master): release 0.9.3

build(reana-workflow-engine-serial/python): bump all required packages as of 2024-03-04 (reanahub#200)

build(reana-workflow-engine-serial/python): bump shared REANA packages as of 2024-03-04 (reanahub#200)

ci(reana-workflow-engine-serial/commitlint): allow release commit style (reanahub#201)

docs(reana-workflow-engine-serial/authors): complete list of contributors (reanahub#199)

refactor(reana-workflow-engine-serial/docs): move from reST to Markdown (reanahub#198)

fix(reana-workflow-engine-serial/progress): handle stopped jobs (reanahub#195)

ci(reana-workflow-engine-serial/commitlint): check for the presence of concrete PR number (reanahub#197)

ci(reana-workflow-engine-serial/shellcheck): fix exit code propagation (reanahub#197)

build(reana-workflow-engine-serial/docker): install correct extras of reana-commons submodule (reanahub#196)

ci(reana-workflow-engine-serial/release-please): update version in Dockerfile (reanahub#194)

build(reana-workflow-engine-serial/docker): non-editable submodules in "latest" mode (reanahub#190)

ci(reana-workflow-engine-serial/release-please): initial configuration (reanahub#191)

ci(reana-workflow-engine-serial/commitlint): addition of commit message linter (reanahub#191)

chore(reana-workflow-engine-yadage/master): release 0.9.4

build(reana-workflow-engine-yadage/python): bump all required packages as of 2024-03-04 (reanahub#261)

build(reana-workflow-engine-yadage/python): bump shared REANA packages as of 2024-03-04 (reanahub#261)

ci(reana-workflow-engine-yadage/commitlint): allow release commit style (reanahub#262)

docs(reana-workflow-engine-yadage/authors): complete list of contributors (reanahub#260)

refactor(reana-workflow-engine-yadage/docs): move from reST to Markdown (reanahub#259)

fix(reana-workflow-engine-yadage/progress): correctly handle running and stopped jobs (reanahub#258)

ci(reana-workflow-engine-yadage/commitlint): check for the presence of concrete PR number (reanahub#257)

ci(reana-workflow-engine-yadage/shellcheck): fix exit code propagation (reanahub#257)

build(reana-workflow-engine-yadage/docker): install correct extras of reana-commons submodule (reanahub#256)

ci(reana-workflow-engine-yadage/release-please): update version in Dockerfile (reanahub#254)

build(reana-workflow-engine-yadage/docker): non-editable submodules in "latest" mode (reanahub#249)

ci(reana-workflow-engine-yadage/release-please): initial configuration (reanahub#251)

ci(reana-workflow-engine-yadage/commitlint): addition of commit message linter (reanahub#251)

chore(reana-workflow-engine-snakemake/master): release 0.9.3

build(reana-workflow-engine-snakemake/python): bump all required packages as of 2024-03-04 (reanahub#85)

build(reana-workflow-engine-snakemake/python): bump shared REANA packages as of 2024-03-04 (reanahub#85)

ci(reana-workflow-engine-snakemake/commitlint): allow release commit style (reanahub#86)

feat(reana-workflow-engine-snakemake/config): get max number of parallel jobs from env vars (reanahub#84)

feat(reana-workflow-engine-snakemake/executor): upgrade to Snakemake v7.32.4 (reanahub#81)

docs(reana-workflow-engine-snakemake/authors): complete list of contributors (reanahub#83)

refactor(reana-workflow-engine-snakemake/docs): move from reST to Markdown (reanahub#82)

fix(reana-workflow-engine-snakemake/progress): handle stopped jobs (reanahub#78)

ci(reana-workflow-engine-snakemake/commitlint): check for the presence of concrete PR number (reanahub#80)

ci(reana-workflow-engine-snakemake/shellcheck): fix exit code propagation (reanahub#80)

build(reana-workflow-engine-snakemake/docker): install correct extras of reana-commons submodule (reanahub#79)

ci(reana-workflow-engine-snakemake/release-please): update version in Dockerfile (reanahub#77)

build(reana-workflow-engine-snakemake/docker): non-editable submodules in "latest" mode (reanahub#73)

ci(reana-workflow-engine-snakemake/release-please): initial configuration (reanahub#74)

ci(reana-workflow-engine-snakemake/commitlint): addition of commit message linter (reanahub#74)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@diegodelemos diegodelemos

Labels
deployment/cern deployment/helm system/scalability
Projects
None yet
Milestone
No milestone
1 participant
@diegodelemos