Skip to content

Conversation

@haitaoyao
Copy link
Contributor

  • replace java.util.Timer with ScheduledExectorService
  • avoid reference to the whole message, use message id only.

…tor to avoid long timeout cause reference message body to OOME
@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@haitaoyao haitaoyao changed the title connection ack timeout improvement, replace Timer with ScheudledExecutor... [SPARK-4342] [Core]connection ack timeout improvement, replace Timer with ScheudledExecutor... Nov 11, 2014
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reason for the magic number 2?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to avoid some task block the whole scheduler. ScheduledExecutorService executes the task in its own thread pool, instead of fork a new thread for the task execution.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just checked the codes:

    public ScheduledThreadPoolExecutor(int corePoolSize,
                                       ThreadFactory threadFactory) {
        super(corePoolSize, Integer.MAX_VALUE, 0, TimeUnit.NANOSECONDS,
              new DelayedWorkQueue(), threadFactory);
    }

2 is the min thread number, not the max thread number.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

corePoolSize the number of threads to keep in the pool, even if they are idle

Here I think 1 is enough.

@JoshRosen
Copy link
Contributor

This looks like a duplicate of another JIRA, which was resolved by #3259, so do you mind closing this? Thanks!

(Sorry that I missed this PR; I guess it got buried in a backlog of other stuff)

@asfgit asfgit closed this in 047ff57 Nov 29, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants