Make TrainingAnalyticsSideChannel internal #4999
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
delete-merged-branch
bot
deleted the
r13-training-sidechannel-internal
branch
philippds
added a commit
to philippds-forks/ml-agents
that referenced
this pull request
Mar 5, 2021
* [MLA-1768] retrain Match3 scene (Unity-Technologies#4943) * improved settings and move to default_settings * update models * Release 13 versions. (Unity-Technologies#4946) - updated release tag validation script to automate the updating of files with release tags that need to be changed as part of the pre-commit operation. * Update readme for release_13. (Unity-Technologies#4951) * Update docs to pass doc validation. (Unity-Technologies#4953) * update defines, compile out Initialize body on non-desktop (Unity-Technologies#4957) * Masking Discrete Actions typos (Unity-Technologies#4961) (Unity-Technologies#4964) Co-authored-by: Philipp Siedler <p.d.siedler@gmail.com> * Adding references to the Extensions package to help promote it. (Unity-Technologies#4967) (Unity-Technologies#4968) Co-authored-by: Marwan Mattar <marwan@unity3d.com> Co-authored-by: Chris Elion <chris.elion@unity3d.com> * Fix release link validations. (Unity-Technologies#4970) * Adding the Variable length observation to the readme and to the overview of ML-Agents * - * forgot a dot * InputActuatorComponent to allow the generation of an action space from an InputActionAsset (Unity-Technologies#4881) (Unity-Technologies#4974) * handle no plugins found (Unity-Technologies#4969) (Unity-Technologies#4973) * Tick extension version. (Unity-Technologies#4975) * adding a comic and readding removed feaures docs * Update 2018 project version to fix burst errors. (Unity-Technologies#4977) (Unity-Technologies#4978) * Add an example project for the InputSystemActuator. (Unity-Technologies#4976) (Unity-Technologies#4980) * Update barracuda, swtich Agents in Sorter use Burst. (Unity-Technologies#4979) (Unity-Technologies#4981) * Set ignore done=False in GAIL (Unity-Technologies#4971) * MultiAgentGroup Interface (Unity-Technologies#4923) * add SimpleMultiAgentGroup * add group reward field to agent and proto * Fix InputActuatorComponent tests asmdef. (Unity-Technologies#4994) (Unity-Technologies#4995) * Fix asmdef? (Unity-Technologies#4994) (Unity-Technologies#4996) * Make TrainingAnalyticsSideChannel internal (Unity-Technologies#4999) * [MLA-1783] built-in actuator type (Unity-Technologies#4950) * Add component menues for some sensors and actuators. (Unity-Technologies#5001) * Add component menues for some sensors and actuators. (Unity-Technologies#5001) (Unity-Technologies#5002) * Fixing the number of layers in the config of PyramidsRND * Merge master -> release_13_branch-to-master * Fix RpcCommunicator merge. * Move the Critic into the Optimizer (Unity-Technologies#4939) Co-authored-by: Ervin Teng <ervin@unity3d.com> * master -> main. (Unity-Technologies#5010) * Adding links to CV/Robotics/GameSim (Unity-Technologies#5012) * Make GridSensor a non allocating object after initialization. (Unity-Technologies#5014) Co-authored-by: Chris Elion <chris.elion@unity3d.com> * Modified the model_serialization to have correct inputs and outputs * switching from CamelCase to snake_case * Fix gpu pytests (Unity-Technologies#5019) * Move tensors to cpu before converting it to numpy * Adding a name field to BufferSensorComponent * Adding a note to the CHANGELOG about var len obs * Adding a helper method for creating observation placeholder names and removed the _h and _c placeholders * Adding a custom editor for BufferSensorComponent * Changing Sorter to use the new Editor and serialization * adding inheritdoc * Update cattrs dependencies to support python3.9 (Unity-Technologies#4821) * Python Dataflow for Group Manager (Unity-Technologies#4926) * Make buffer type-agnostic * Edit types of Apped method * Change comment * Collaborative walljump * Make collab env harder * Add group ID * Add collab obs to trajectory * Fix bug; add critic_obs to buffer * Set group ids for some envs * Pretty broken * Less broken PPO * Update SAC, fix PPO batching * Fix SAC interrupted condition and typing * Fix SAC interrupted again * Remove erroneous file * Fix multiple obs * Update curiosity reward provider * Update GAIL and BC * Multi-input network * Some minor tweaks but still broken * Get next critic observations into value estimate * Temporarily disable exporting * Use Vince's ONNX export code * Cleanup * Add walljump collab YAML * Lower max height * Update prefab * Update prefab * Collaborative Hallway * Set num teammates to 2 * Add config and group ids to HallwayCollab * Fix bug with hallway collab * Edits to HallwayCollab * Update onnx file meta * Make the env easier * Remove prints * Make Collab env harder * Fix group ID * Add cc to ghost trainer * Add comment to ghost trainer * Revert "Add comment to ghost trainer" This reverts commit 292b6ce. * Actually add comment to ghosttrainer * Scale size of CC network * Scale value network based on num agents * Add 3rd symbol to hallway collab * Make comms one-hot * Fix S tag * Additional changes * Some more fixes * Self-attention Centralized Critic * separate entity encoder and RSA * clean up args in mha * more cleanups * fixed tests * entity embeddings work with no max Integrate into CC * remove group id * very rough sketch for TeamManager interface * One layer for entity embed * Use 4 heads * add defaults to linear encoder, initialize ent encoders * add team manager id to proto * team manager for hallway * add manager to hallway * send and process team manager id * remove print * small cleanup * default behavior for baseTeamManager * add back statsrecorder * update * Team manager prototype (Unity-Technologies#4850) * remove group id * very rough sketch for TeamManager interface * add team manager id to proto * team manager for hallway * add manager to hallway * send and process team manager id * remove print * small cleanup Co-authored-by: Chris Elion <chris.elion@unity3d.com> * Remove statsrecorder * Fix AgentProcessor for TeamManager Should work for variable decision frequencies (untested) * team manager * New buffer layout, TeamObsUtil, pad dead agents * Use NaNs to get masks for attention * Add team reward to buffer * Try subtract marginalized value * Add Q function with attention * Some more progress - still broken * use singular entity embedding (Unity-Technologies#4873) * I think it's running * Actions added but untested * Fix issue with team_actions * Add next action and next team obs * separate forward into q_net and baseline * might be right * forcing this to work * buffer error * COMAA runs * add lambda return and target network * no target net * remove normalize advantages * add target network back * value estimator * update coma config * add target net * no target, increase lambda * remove prints * cloud config * use v return * use target net * adding zombie to coma2 brnch * add callbacks * cloud run with coma2 of held out zombie test env * target of baseline is returns_v * remove target update * Add team dones * ntegrate teammate dones * add value clipping * try again on cloud * clipping values and updated zombie * update configs * remove value head clipping * update zombie config * Add trust region to COMA updates * Remove Q-net for perf * Weight decay, regularizaton loss * Use same network * add base team manager * Remove reg loss, still stable * Black format * add team reward field to agent and proto * set team reward * add maxstep to teammanager and hook to academy * check agent by agent.enabled * remove manager from academy when dispose * move manager * put team reward in decision steps * use 0 as default manager id * fix setTeamReward Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com> * change method name to GetRegisteredAgents * address comments * Revert C# env changes * Remove a bunch of stuff from envs * Remove a bunch of extra files * Remove changes from base-teammanager * Remove remaining files * Remove some unneeded changes * Make buffer typing neater * AgentProcessor fixes * Back out trainer changes * use delegate to avoid agent-manager cyclic reference * put team reward in decision steps * fix unregister agents * add teamreward to decision step * typo * unregister on disabled * remove OnTeamEpisodeBegin * change name TeamManager to MultiAgentGroup * more team -> group * fix tests * fix tests * Use attention tests from master * Revert "Use attention tests from master" This reverts commit 78e052b. * Use attention from master * Renaming fest * Use NamedTuples instead of attrs classes * Bug fixes * remove GroupMaxStep * add some doc * Fix mock brain * np float32 fixes * more renaming * Test for team obs in agentprocessor * Test for group and add team reward * doc improve Co-authored-by: Ervin T. <ervin@unity3d.com> * store registered agents in set * remove unused step counts * Global group ids * Fix Trajectory test * Remove duplicated files * Add team methods to AgentAction * Buffer fixes (cherry picked from commit 2c03d2b) * Add test for GroupObs * Change AgentAction back to 0 pad and add tests * Addressed some comments * Address some comments * Add more comments * Rename internal function * Move padding method to AgentBufferField * Fix slicing typing and string printing in AgentBufferField * Fix to-flat and add tests * Rename GroupmateStatus to AgentStatus * Update comments * Added GroupId, GlobalGroupId, GlobalAgentId types * Update comment * Make some agent processor properties internal * Rename add_group_status * Rename store_group_status, fix test * Rename clear_group_obs Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com> Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com> Co-authored-by: Chris Elion <chris.elion@unity3d.com> Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com> Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com> * Removing some scenes (Unity-Technologies#4997) * Removing some scenes, All the Static and all the non variable speed environments. Also removed Bouncer, PushBlock, WallJump and reacher. Removed a bunch of visual environements as well. Removed 3DBallHard and FoodCollector (kept Visual and Grid FoodCollector) * readding 3DBallHard * readding pushblock and walljump * Removing tennis * removing mentions of removed environments * removing unused images * Renaming Crawler demos * renaming some demo files * removing and modifying some config files * new examples image? * removing Bouncer from build list * replacing the Bouncer environment with Match3 for llapi tests * Typo in yamato test * Fix issue with queuing input events that stomp on others. (Unity-Technologies#5034) Co-authored-by: Chris Elion <chris.elion@unity3d.com> Co-authored-by: Chris Goy <christopherg@unity3d.com> Co-authored-by: Marwan Mattar <marwan@unity3d.com> Co-authored-by: vincentpierre <vincentpierre@unity3d.com> Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com> Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com> Co-authored-by: Ervin Teng <ervin@unity3d.com> Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Noticed TrainingAnalyticsSideChannel was missing docs on master (@surfnerd already fixed on the release branch), and realized that the class doesn't need to be public