[llvm_instcount] Leaderboard Submission: Random Search #117
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
ChrisCummins
force-pushed
the
leaderboard-random
branch
from
94b3a0c to
9643c0e
Compare
ChrisCummins
changed the title
ChrisCummins
force-pushed
the
leaderboard-random
branch
2 times, most recently
from
790a973 to
92e9868
Compare
ChrisCummins
requested review from
hughleat,
JD-ETH,
jia-git,
soumith,
yuandong-tian and
benoitsteiner
hughleat
approved these changes
Mar 22, 2021
| python -m compiler_gym.bin.validate --env=llvm-ic-v0 \ | ||
| --reward_aggregation=geomean < "$LOGFILE" | tail -n4 | ||
| done | ||
| ``` |
There was a problem hiding this comment.
IMO, patience ratio is odd. Just straightforward patience makes more sense to me.
I can see that if you don't know the env, then you might need something generic, but not all envs will have fixed action spaces, so this won't help.
There was a problem hiding this comment.
Added extra description
ChrisCummins
force-pushed
the
leaderboard-random
branch
2 times, most recently
from
b84fa03 to
6b58e9e
Compare
JD-ETH
reviewed
Mar 22, 2021
| Pseudo-code for this search is: | ||
|
|
||
| ```c++ | ||
| float search_with_patience(CompilerEnv env, int patience, int search_time) { |
There was a problem hiding this comment.
Why is the pseudo-code in C++? Is there a C++ interface too?
There was a problem hiding this comment.
No particular reason, it just feels more pseudo-y to me with types :)
We don't advertise a C++ interface, but it is entirely usable as a C++ library. Here's is an example random search. It looks pretty much the same as in python:
ChrisCummins
force-pushed
the
leaderboard-random
branch
from
6b58e9e to
1ac1a69
Compare
ChrisCummins
changed the title
ChrisCummins
changed the title
ChrisCummins
force-pushed
the
leaderboard-random
branch
3 times, most recently
from
fc25871 to
c02dc3a
Compare
This adds a pure random policy that selects actions randomly until a fixed number of steps (the "patience" of the search) have been evaluated without an improvement to reward. The search stops after a predetermined amount of search time has elapsed. The patience value was selected by running a random 200 searches on programs from the `blas-v0` dataset and selecting the value that produced the best average reward.
ChrisCummins
force-pushed
the
leaderboard-random
branch
from
c02dc3a to
593e949
Compare
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds a pure random policy that selects actions randomly until a
fixed number of steps (the "patience" of the search) have been
evaluated without an improvement to reward. The search stops after a
predetermined amount of search time has elapsed.
The patience value was selected by running a random 200 searches on
programs from the
blas-v0dataset and selecting the value thatproduced the best average reward.
What is in this PR?