Skip to content

Commit

Permalink
[SPARK-47958][TESTS] Change LocalSchedulerBackend to notify scheduler…
Browse files Browse the repository at this point in the history
… of executor on start

### What changes were proposed in this pull request?

Changing to call `reviveOffers` on start (after the local executor is set up) so that the task scheduler knows about it. This matches behavior in CoarseGrainedSchedulerBackend, which will call an equivalent method on executor registration.

### Why are the changes needed?

When using LocalSchedulerBackend, the task scheduler will not know about the executor until a task is run, which can lead to unexpected behavior in tests.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
Running existing tests.

### Was this patch authored or co-authored using generative AI tooling?
No.

Closes apache#46187 from davintjong-db/local-executor-fix.

Authored-by: Davin Tjong <davin.tjong@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
  • Loading branch information
davintjong-db authored and cloud-fan committed Apr 24, 2024
1 parent 0042b67 commit 09ed09c
Showing 1 changed file with 1 addition and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,7 @@ private[spark] class LocalSchedulerBackend(
Map.empty)))
launcherBackend.setAppId(appId)
launcherBackend.setState(SparkAppHandle.State.RUNNING)
reviveOffers()
}

override def stop(): Unit = {
Expand Down

0 comments on commit 09ed09c

Please sign in to comment.