Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not enough space to cache rdd in memory warning #135

Closed
porscheme opened this issue Mar 31, 2023 · 2 comments
Closed

Not enough space to cache rdd in memory warning #135

porscheme opened this issue Mar 31, 2023 · 2 comments

Comments

@porscheme
Copy link

Please check the FAQ documentation before raising an issue

Describe the bug (required)
I am running a nebula-exchange with spark-operator, and I got Not enough space to cache '' in memory warning.

Your Environments (required)

  • nebula cluster info
    v3.4.0
    graphd: 3 (128 Gb, 2 TB SSD Drive)
    metad: 3 (128 Gb, 2 TB SSD Drive)
    storaged: 5 (128 Gb, 2 TB SSD Drive)

  • spark-operator
    driver: 1 (16 GB)
    Executor: 20 (140 GB)

  • OS: uname -a
  • Commit id (e.g. a3ffc7d8)

How To Reproduce(required)

Steps to reproduce the behavior:

  1. Step 1
  2. Step 2
  3. Step 3

Expected behavior

Additional context

{"level":"WARN","timestamp":"2023-03-30 23:38:41,633","thread":"Executor task launch worker for task 5592.0 in stage 1.0 (TID 5593)","message":"Not enough space to cache rdd_6_5592 in memory! (computed 24.6 MiB so far)"}
{"level":"WARN","timestamp":"2023-03-30 23:38:41,633","thread":"Executor task launch worker for task 5592.0 in stage 1.0 (TID 5593)","message":"Persisting block rdd_6_5592 to disk instead."}
{"level":"WARN","timestamp":"2023-03-30 23:38:42,289","thread":"Executor task launch worker for task 5580.0 in stage 1.0 (TID 5581)","message":"Not enough space to cache rdd_6_5580 in memory! (computed 94.3 MiB so far)"}
{"level":"WARN","timestamp":"2023-03-30 23:38:42,289","thread":"Executor task launch worker for task 5580.0 in stage 1.0 (TID 5581)","message":"Persisting block rdd_6_5580 to disk instead."}
{"level":"WARN","timestamp":"2023-03-30 23:38:43,286","thread":"Executor task launch worker for task 5580.0 in stage 1.0 (TID 5581)","message":"Not enough space to cache rdd_6_5580 in memory! (computed 94.3 MiB so far)"}
{"level":"WARN","timestamp":"2023-03-30 23:38:44,604","thread":"Executor task launch worker for task 5592.0 in stage 1.0 (TID 5593)","message":"Not enough space to cache rdd_6_5592 in memory! (computed 94.2 MiB so far)"}
@Nicole00
Copy link
Contributor

you can set less executor-cores for each executor and less executor number, set more executor-memory for each executor

@QingZ11
Copy link

QingZ11 commented May 5, 2023

I have noticed that the issue you created hasn’t been updated for nearly a month, so I have to close it for now. If you have any new updates, you are welcome to reopen this issue anytime.

Thanks a lot for your contribution anyway 😊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants