Skip to content

Asynchronous mode #60

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 58 commits into from
Dec 28, 2020
Merged

Asynchronous mode #60

merged 58 commits into from
Dec 28, 2020

Conversation

dgant
Copy link
Contributor

@dgant dgant commented Jul 28, 2020

This pull request adds an optional asynchronous operation mode to JBWAPI, similar to the non-blocking mode of TorchCraft.

The goal is to allow bot authors to more reliably ensure real-time performance for their bots. Async mode runs in two threads: One for communication with BWAPI, and one for bot operation. This allows bot code to run independently of the BWAPI control flow, and for JBWAPI to release its lock-step control of StarCraft in a timely fashion even if bot code is still running.

When a new frame from BWAPI is ready, JBWAPI copies it into a queue of frames for the bot to process. JBWAPI then waits 40ms (configurable) for the bot to finish processing all the frames in the queue. When the bot has done so or time has elapsed, JBWAPI returns control to BWAPI.

dgant and others added 29 commits July 17, 2020 12:53
…ntirely for reinsertion via BotWrapper, so this revision is in a non-working state.
…mmands to live shared memory instead of uselessly writing them to the frame buffer.
… direct ByteBuffers on successive games, due either to fragmentation or leaking (or both). Now reusing the FrameBuffer across games so we only have to allocate once, for the first game. Replaced some unnecessary references to ClientData with GameData. Cleaned up BWClient game loop.
… removed 'idleness' concept from BotWrapper.
…ontrol immediately always). Improved some metrics
…gured tests to run with sufficient memory to allocate framebuffers.
@dgant dgant changed the title Work in progress: Asynchronous mode Asynchronous mode Aug 16, 2020
@dgant
Copy link
Contributor Author

dgant commented Oct 5, 2020

New in the pull request:

  • Addressed the comments left on the original pull request
  • Introduced unsafe asynchronous mode, for eking out extra performance. Unsafe mode runs the bot against live game data while copying the live game data to the frame buffer in the background. It's "unsafe" because there is no active mechanism preventing modification of which buffer the bot is reading from simultaneous with a bot's attempt to read from it. In practice I have never observed this failing -- PurpleWave has been happily running with this mode on the AIIDE ladder and BASIL.
  • More granular performance measurements. Includes metrics which I found useful for diagnosing performance on the AIIDE ladder.
  • Fixes synchronous latency compensation mode. Since we observed that async mode and latcom are incompatible, I added checks to prevent inadvertently enabling both. These modes could be reconciled in the future.

I believe all loose ends are tied up and this PR is ready for review in its current state.

@dgant
Copy link
Contributor Author

dgant commented Dec 28, 2020

We're on passing tests now and merge compatibility with develop. @JasperGeurtz what do you think?

@JavaBWAPI JavaBWAPI deleted a comment from lgtm-com bot Dec 28, 2020
@JavaBWAPI JavaBWAPI deleted a comment from lgtm-com bot Dec 28, 2020
@JavaBWAPI JavaBWAPI deleted a comment from lgtm-com bot Dec 28, 2020
@JavaBWAPI JavaBWAPI deleted a comment from lgtm-com bot Dec 28, 2020
@JavaBWAPI JavaBWAPI deleted a comment from lgtm-com bot Dec 28, 2020
@JavaBWAPI JavaBWAPI deleted a comment from lgtm-com bot Dec 28, 2020
@JavaBWAPI JavaBWAPI deleted a comment from lgtm-com bot Dec 28, 2020
@JavaBWAPI JavaBWAPI deleted a comment from lgtm-com bot Dec 28, 2020
@JavaBWAPI JavaBWAPI deleted a comment from lgtm-com bot Dec 28, 2020
@JasperGeurtz
Copy link
Collaborator

JasperGeurtz commented Dec 28, 2020

LGTM, thank you for finishing this up!

three questions:

  • is the <rerunFailingTestsCount>10</rerunFailingTestsCount> argument needed in the pom.xml, seems to run fine on my PC without? timed tests are always a bit flaky
  • are the frames in the framebuffer being cleared up properly? (I remember a while ago this to be a potential problem, but don't know what the end result was) Only mattered for tests as we reallocate every test
  • I also want to add the "linux openbw" branch by @Bytekeeper for a "2.0" release. How (in)compatible are the two branches? They both modify how BWClient etc. work, so there will be some conflicts at least. ANy thoughts @Bytekeeper @dgant Will be merged after, so not an issue

I don't mind version 2.0 being a bit experimental for now, and just keeping 1.5.1 as the "stable" branch that anyone can use

@JasperGeurtz JasperGeurtz merged commit d21d42e into JavaBWAPI:develop Dec 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants