-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Restore use of v1 release branch for markdown-lint-check action. #14
Commits on Apr 6, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 7c67fc5 - Browse repository at this point
Copy the full SHA 7c67fc5View commit details -
Configuration menu - View commit details
-
Copy full SHA for f8e064f - Browse repository at this point
Copy the full SHA f8e064fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4d34c5c - Browse repository at this point
Copy the full SHA 4d34c5cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8db9777 - Browse repository at this point
Copy the full SHA 8db9777View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6ee7b56 - Browse repository at this point
Copy the full SHA 6ee7b56View commit details
Commits on Aug 18, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 44c65e7 - Browse repository at this point
Copy the full SHA 44c65e7View commit details
Commits on Aug 19, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 1f40fca - Browse repository at this point
Copy the full SHA 1f40fcaView commit details
Commits on Aug 21, 2020
-
Create initial github actions pipeline for building on windows (#11)
* Create initial github actions pipeline for building on windows * add the build script referenced in the github actions ci pipeline * comments * commit something that should fail the uncrustify check just to make sure that the pipelines fail * Revert "commit something that should fail the uncrustify check just to make sure that the pipelines fail" This reverts commit 3fee079. Manually verified that this does indeed fail the build pipeline. * tell editors to use 2 spaces instead of 4 by default for yaml files
Configuration menu - View commit details
-
Copy full SHA for 964c302 - Browse repository at this point
Copy the full SHA 964c302View commit details
Commits on Aug 24, 2020
-
Add CI Action Pipelines for Python (#12)
* cleanup spacing/comments from previous copy/paste/edit a tad * initial attempt at enabling various python checks in the ci pipelines note: here we run them on linux instead of windows for easier scripting, though (hopefully) the results are more or less that same in either spot (excepting for lf/crlf line ending differences which we should eventually fix separately) * various python action pipeline changes - try to run some of the checks on both windows and linux - try to support a couple of different versions of python? - run the lint and unit tests from the same workspace (saves on some runner startup cost) - try to use the "approved" python setup task from github (else the PATH env doesn't seem to work out right) * test failing the python licenseheaders check * attempting to fix running python checks on linux * Revert "test failing the python licenseheaders check" This reverts commit 9ca6b96. Manually confirmed that this does indeed fail the build as expected. * separate and document windows/linux commands for running python checks * add a bash equivalent for the grpc generation cmd while we're at it * disable python 3.8 checks for now It's failing pylint checks with some "useless statement" warnings
Configuration menu - View commit details
-
Copy full SHA for dbf2cf1 - Browse repository at this point
Copy the full SHA dbf2cf1View commit details
Commits on Aug 25, 2020
-
Configuration menu - View commit details
-
Copy full SHA for f1443b8 - Browse repository at this point
Copy the full SHA f1443b8View commit details
Commits on Aug 28, 2020
-
Merged PR 416953: Fixing the Input Space Partition for Ensemble Model…
…s (After Rebase) There was a bug in partitioning the input space in the homogeneous random forest. As a result the individual decision trees would return a Prediction object even if the features were outside of their input space.
Configuration menu - View commit details
-
Copy full SHA for 691569c - Browse repository at this point
Copy the full SHA 691569cView commit details -
Merged PR 417038: Fixing the dimension ordering issue
In preparing the search space for the Gaussian process we erroneously ordered the dimensions in the order they were specified in the OptimizationProblem, but the bayes_opt reorders them alphabetically.
Configuration menu - View commit details
-
Copy full SHA for 0c41bd2 - Browse repository at this point
Copy the full SHA 0c41bd2View commit details -
Merged PR 415851: Add some fixups for building C++ on Linux
Some selections taken from !410291 ```sh git checkout -b bpkroth/some-linux-cpp-fixups master git format-patch --stdout master..bpkroth/oss-linux-build-work -- *.cpp *.h *.inl *.vcxproj | git am ``` - un-inline some MlosPlatform.Std.inl functions - comments referencing MlosPlatform.Std.ink - tweaks to allow SmartCache to build on Linux - partial (incomplete) work on enabling additional C++ projects on Linux - comment fixup - fixups to allow some more compilation on Linux - use the macro we defined - file can't include itself? - more build fixups for clang - more fixups to allow Mlos.Unit to link - precompiled header fixup for building on Windows Related work items: #806033
Configuration menu - View commit details
-
Copy full SHA for d0ce8aa - Browse repository at this point
Copy the full SHA d0ce8aaView commit details
Commits on Aug 31, 2020
-
Merged PR 417218: Prediction class v2 (rebased)
Version2 of PredictionClass. _From Adam's email:_ **Problems with current implementation** First, returning a Prediction object for each invocation of bayesian_optimizer.predict() means that to draw one heatmap we need to return 10,000 objects (100x100 pixels). This eliminates hope for memory locality, puts a strain on allocators, requires us to reassemble a numpy array for the benefit of matplotlib, etc. Returning a single data frame would be much preferred. (Pandas data frame also opens door for using Arrow down the road.) Secondly, most of the time we don’t care about most of the values contained in the Prediction object. Each utility function cares only about one or two, the tomograph cares only about the mean and nothing else right now. Computing all the extra values in the Prediction class is a waste of CPU and memory. This PR is a restart of the abandoned: https://msdata.visualstudio.com/Database%20Systems/_git/MLOS/pullrequest/413447 Related work items: #743670, #863921
Configuration menu - View commit details
-
Copy full SHA for 9c67778 - Browse repository at this point
Copy the full SHA 9c67778View commit details -
Merged PR 418867: Smart cache active learning notebook
* Fixed Tomograph to work again with the new Prediction class * added a notebook to demonstrate active learning on smart cache in python * cleaned up MLOS SDK and MLOS Infra for Python * Fixed a bunch of bugs * Made sure that BayesianOptimizerProxy once again conforms to OptimizerInterface
Configuration menu - View commit details
-
Copy full SHA for 4a7ae58 - Browse repository at this point
Copy the full SHA 4a7ae58View commit details -
Merged PR 420231: Making ConfidenceBoundUtilityFunction more intuitive
Andreas pointed out that to the user of the BayesianOptimizer the interaction between these parameters is confusing and can lead to subtle bugs: {maximize, minimize} x {upper_confidence_bound, lower_confidence_bound}. As one would often use upper_confidence_bound for maximization and lower_confidence_bound for minimization . However, these parameters are set in different places (OptimizationProblem vs. OptimizerConfig) and entirely non-intuitive. So this PR renames them to upper_confidence_bound_on_improvement and lower_confidence_bound_on_improvement and changes the math accordingly. This way, when the user switches from maximizing, to minimizing, she/he/they doesn't/don't have to change the optimizer config. This rename is in-line with the names of other utility functions: Probability of Improvement, Expected Improvement.
Configuration menu - View commit details
-
Copy full SHA for df84a46 - Browse repository at this point
Copy the full SHA df84a46View commit details -
Merged PR 420287: Fixing pandas warnings
Pandas spits out tens of thousands lines of warnings during Unit Tests runs. This PR cleans it up.
Configuration menu - View commit details
-
Copy full SHA for a77443c - Browse repository at this point
Copy the full SHA a77443cView commit details -
Merged PR 421047: Various fixes to enable Linux build
Various fixes in C++ projects to enable Linux build. Do not use string_view in Mlos.UnitTest project. Change include order (include template specialization before using them).
Configuration menu - View commit details
-
Copy full SHA for 909a81c - Browse repository at this point
Copy the full SHA 909a81cView commit details -
Merged PR 421281: Switching from sample variance to mean variance
Using sample variance instead of mean variance to compute the confidence intervals makes them not shrink as we add more observations. While this measure can still be useful down the road to detect noisy-TV type of problems, using mean variance makes more sense for the problems we are solving now.
Configuration menu - View commit details
-
Copy full SHA for ba40a81 - Browse repository at this point
Copy the full SHA ba40a81View commit details -
Merged PR 421755: Ensure default parameters are copied
This PR ensures default parameters are copied and not accidentally modified by the user. I'm also introducing a long-haul test for BO, which was the original motivation. Happy to split that off into a different PR. I wasn't sure where to put the DefaultConfigMeta helper, lmk if there's a better place. There's two reasons this is tricky: DEFAULT is used on the class, but properties are instance-level, so we need a meta-class. Also, because of the way the dictionaries are instantiated, we can't make copies of nested points easily, so we need to use deepcopy. This solution still has the issue that we're using mutable objects as default parameters. If a user would modify ``_DEFAULT`` for some reason, the default instantiations would change.
Configuration menu - View commit details
-
Copy full SHA for 25d3e24 - Browse repository at this point
Copy the full SHA 25d3e24View commit details -
Configuration menu - View commit details
-
Copy full SHA for 03d49b7 - Browse repository at this point
Copy the full SHA 03d49b7View commit details
Commits on Sep 1, 2020
-
Hugo website first draft (#14)
* init website * remove other themes * add basic config * remove non-existant submodules * fix site building script * add empty index, update base url * add generated website to gitignore * fix booksection config * remove unused folder * add link to python api to hugo page * fix link to python api * add docs on python only installation * add generated website to gitignore * remove submodule with book theme * remove generated assets * add code to download book theme to build_site.sh * add if for book checkout * fixup readme a bit * update links in readme, clean up some empty files * remove empty gitmodules * remove generated files * add manual menu, don't lowercase files * try to make links work in hugo and on github * fix up links for hugo * easier way to get python docs into menu * don't have license in hugo menu * fix regex * fix python api doc link again * check out specific tag of book theme (v8) * clarify formulation. * add comment to explain sed commands * update menu, fix link to bayesian optimization * add sed line to change paths for BayesianOptimization notebook
Configuration menu - View commit details
-
Copy full SHA for f4c01b7 - Browse repository at this point
Copy the full SHA f4c01b7View commit details -
* add sphinx build to gitignore * add apidoc templates * mostly works * slightly cleaner * clean up sphinx apidoc generation * update index * fix source links in python api docs * restructure docs with one more menu level * change link name to main website * add python api index document
Configuration menu - View commit details
-
Copy full SHA for 63cb58b - Browse repository at this point
Copy the full SHA 63cb58bView commit details -
* add template for notebook conversion, add links to notebook download * don't keep before file in sed * make download more prominent in template * fix string replace comment
Configuration menu - View commit details
-
Copy full SHA for 11cbc25 - Browse repository at this point
Copy the full SHA 11cbc25View commit details
Commits on Sep 2, 2020
-
Merged PR 422753: python documentation fixups
python documentation fixups from microsoft#12 and !422716
Configuration menu - View commit details
-
Copy full SHA for 7876605 - Browse repository at this point
Copy the full SHA 7876605View commit details -
Merged PR 422029: Exposing Optimizer State
Adding goodness of fit measurements for the training (and some validation) datasets, along with a way to query these statistics via gRPC.
Configuration menu - View commit details
-
Copy full SHA for 2476a61 - Browse repository at this point
Copy the full SHA 2476a61View commit details -
Merged PR 422857: Implement missing Mlos.Core functionality for Linux
Implement support for Posix Named Shared Memory and Named Semaphores. SmartSharedChannel builds on Ubuntu16.04
Configuration menu - View commit details
-
Copy full SHA for 928736a - Browse repository at this point
Copy the full SHA 928736aView commit details -
Merged PR 423072: Added r_squared
Added r_squared
Configuration menu - View commit details
-
Copy full SHA for df73422 - Browse repository at this point
Copy the full SHA df73422View commit details -
Merged PR 423092: Lots of little improvements
In preparation for the demo I'm running the code over and over and I've accumulated a lot of small improvements. The fixes include: * Point.to_pandas() -> Point.to_dataframe() rename * New: Point.from_dataframe() * New: RegressionModelFitState.to_dataframe() * SmartCacheWorkloadLauncher takes duration_s as parameter * RandomSearchOptimzier utilizes Hypergrid.random_dataframe() rather than rolling its own * Tracer collects data about dataframes passed as function arguments (hence all of the args->kwargs changes) * Unskipped a couple of long haul tests * Added some convergence state tests to bayesian optimzier tests * Added: assert suggested_config in self.optimization_problem.parameter_space to BayesianOptimizer * Setting attributes on Point works as expected now. * Cleaned up a broken Unit Test
Configuration menu - View commit details
-
Copy full SHA for 5d234d9 - Browse repository at this point
Copy the full SHA 5d234d9View commit details -
Merged PR 423779: Simplifying Hypergrids
# What's changing? Second step in simplifying the Hypergrids. In this PR I eliminated the CompositeHypergrid entirely, and all of its functionality is now present in the SimpleHypergrid. All unit tests are passing and I tested all the notebooks too. The main change occurs in Hypergrids.py - most other changes are related to testing code and suitable renames. The one exception is the HypergridJsonEncoderDecoder.py where there is a functional change. This change is backwards compatible, so old code (in C#) will continue working. # Effects on client code None - CompositeHypergrid was almost entirely internal and never instantiated directly by the user. # Reasons for the change 1. It's much simpler to write the recursive functions required for the Hypergrids when there is only one class, rather than two classes. 2. Greg is implementing Hypergrids in C# and it makes no sense to duplicate something complex, if we can duplicate something simple. # Next Steps There will be some renames: GuestSubgrid -> JoinedSubgrid, etc. I will coordinate them tightly to make sure people's branches continue to work well.
Configuration menu - View commit details
-
Copy full SHA for df3fbad - Browse repository at this point
Copy the full SHA df3fbadView commit details -
Merged PR 424268: make start_optimizer_microservice a script
This allows users to run start_optimizer_microservice as a script once the package is installed
Configuration menu - View commit details
-
Copy full SHA for e50f1da - Browse repository at this point
Copy the full SHA e50f1daView commit details -
Merged PR 424204: minor polish to BO notebook
Some suggestions from Yiwen and some warm-up tasks for the students
Configuration menu - View commit details
-
Copy full SHA for 4f10b62 - Browse repository at this point
Copy the full SHA 4f10b62View commit details -
Merged PR 410291: Linux build/test/dev environment improvements
Lots of work-in-progress to try and make the Linux build/test/dev environment more natural. - Provide `Makefile` wrappers around `cmake` and `dotnet` so that developers can "just" run `make` - Consolidate some `cmake` directives into files in `build/` so that they can be reused for additional projects. - Also apply a handful of fixups to allow building/linking additional C++ projects. - Provide `cmake` functions to allow invoking `dotnet build` to generate `SettingsProvider` files for C++ files. - This allows running `make` within a given source directory to get just what you need updated and avoid race conditions by having multiple `dotnet build` processes running in parallel for different C++ projects. - Allow invoking tests via `ctest` (along with wrappers for `dotnet test` as well) - Provide a `ctags` db generation rule (to allow easier source code navigation in cli editors like `emacs` and `vim`) - Improve the `dotnet` and `cmake` fetch utilities. - Add scripts to allow setting up the shell environment to reference them - But also don't require that - Add support for both `Release` and `Debug` builds - Requires different `cmake` output directories - Disable Windows support (for now) - Some documentation (probably needs improvement) - Also added a `Dockerfile` and some docs to allow more easily testing this across different versions of Ubuntu since a number of the issues we found were due to features in Ubuntu 20.04 not being available in 16.04 (which needs to be supported for SqlServer). - Various Open Issues - `make dotnet-test` fails on the `Mlos.ModelServies.UnitTests` project at the moment due to wrong Python version being referenced - `MlosTestRun_MlosAgentServer_MlosUnitTest` is currently commented out in `Mlos.UnitTest/CMakeLists.txt` since it launches but just hangs which causes the build to hang. It also doesn't fail the test when you `pkill dotnet` (which it should). Related work items: #806033, #815235, #815236, #815237, #836531
Configuration menu - View commit details
-
Copy full SHA for ad964ad - Browse repository at this point
Copy the full SHA ad964adView commit details -
Configuration menu - View commit details
-
Copy full SHA for 80bfcef - Browse repository at this point
Copy the full SHA 80bfcefView commit details -
Merged PR 424438: Polish smart cache notebook a bit
Preparing the smart cache example to be more self-contained and ready for class.
Configuration menu - View commit details
-
Copy full SHA for f4b6456 - Browse repository at this point
Copy the full SHA f4b6456View commit details -
Configuration menu - View commit details
-
Copy full SHA for 663d3f5 - Browse repository at this point
Copy the full SHA 663d3f5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 68ed1e8 - Browse repository at this point
Copy the full SHA 68ed1e8View commit details -
Configuration menu - View commit details
-
Copy full SHA for cbe78af - Browse repository at this point
Copy the full SHA cbe78afView commit details -
Configuration menu - View commit details
-
Copy full SHA for 49b8d48 - Browse repository at this point
Copy the full SHA 49b8d48View commit details
Commits on Sep 3, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 71ec8b2 - Browse repository at this point
Copy the full SHA 71ec8b2View commit details -
Configuration menu - View commit details
-
Copy full SHA for ed40b4e - Browse repository at this point
Copy the full SHA ed40b4eView commit details -
Configuration menu - View commit details
-
Copy full SHA for b3bdafe - Browse repository at this point
Copy the full SHA b3bdafeView commit details -
Configuration menu - View commit details
-
Copy full SHA for 62ea64f - Browse repository at this point
Copy the full SHA 62ea64fView commit details -
Add Environment yml, improve conda docs (#47)
* add environment.yml * fix typo * update readme instructions * update enironment.yml * formatting fixes
Configuration menu - View commit details
-
Copy full SHA for df6216a - Browse repository at this point
Copy the full SHA df6216aView commit details -
clean up notebooks a bit (#48)
* clean up notebooks a bit * remove reference to StartHere notebook
Configuration menu - View commit details
-
Copy full SHA for 9b7b1ba - Browse repository at this point
Copy the full SHA 9b7b1baView commit details -
Add contributing instructions (#49)
* start by reorganizing the files also use relative links for the other files * add some text abount contributing to the top level file remove the nested file
Configuration menu - View commit details
-
Copy full SHA for 7a314e1 - Browse repository at this point
Copy the full SHA 7a314e1View commit details -
* add website build to CI * try to build website on gh actions * make a change * fix actions script * fix slashes * install nbconvert * install nbconvert correctly * figure out paths * fix path issues in website scripts * try to start nbconvert lol * path issues once more * install sphinx_rtd_theme * add numpydoc to requirements * try to push to github pages * only deploy on main * fix typo * use apt-get * two heads are better than one * another typo
Configuration menu - View commit details
-
Copy full SHA for e0cfcb6 - Browse repository at this point
Copy the full SHA e0cfcb6View commit details
Commits on Sep 4, 2020
-
* decrease max degree of polynomial to reduce time spent in unit tests * fixups for python scripts to explicitly look for python3.7 for now fixes some issues when runing the unit tests in an environment that has both 3.7 and 3.8 available with 3.8 as the default * add explicit timeouts for our CI steps * tweak the timeouts to give just a bit more splay * be consistent on use of apt vs apt-get
Configuration menu - View commit details
-
Copy full SHA for 9560997 - Browse repository at this point
Copy the full SHA 9560997View commit details -
Reorganize doc flow by OS rather than by Task (#41)
* Adding jupyter launch instructions to README * Regrouping Prerequisite docs by OS instead of by step * wip * wip * some more initial work on fixingup the docs * more work on reorging the prereqs document to be more OS oriented * slight tweaks on the build instructions * fix bad path Co-authored-by: Carlo Curino <carlocurino@ChristyiMac.lan> Co-authored-by: Brian Kroth <bpkroth@microsoft.com> Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for a9937cc - Browse repository at this point
Copy the full SHA a9937ccView commit details
Commits on Sep 8, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 6560640 - Browse repository at this point
Copy the full SHA 6560640View commit details -
Configuration menu - View commit details
-
Copy full SHA for b23f4f6 - Browse repository at this point
Copy the full SHA b23f4f6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 51bf767 - Browse repository at this point
Copy the full SHA 51bf767View commit details
Commits on Sep 9, 2020
-
Clarified options for the build tools installations for Linux (#53)
* Clarified options for the build tools installations for Linux * Update documentation/01-Prerequisites.md Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> * Update documentation/01-Prerequisites.md Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> * Update documentation/01-Prerequisites.md Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> * Updated toc * Reduced redundancy with publicly available documentation * Simplified (?) Windows instructions * Update documentation/01-Prerequisites.md Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> * Update documentation/01-Prerequisites.md Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> * Fixed the block quotes Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 4d881be - Browse repository at this point
Copy the full SHA 4d881beView commit details -
Redrawn MLOS Architecture (#56)
* Redrawn MLOS Architecture Adding some details on User Input and Code Gen * Updating per feedback * Iterating on image per feedback * Removing old images * Updating link in docs and minor line fix Co-authored-by: Carlo Curino
Configuration menu - View commit details
-
Copy full SHA for 291a1c4 - Browse repository at this point
Copy the full SHA 291a1c4View commit details
Commits on Sep 10, 2020
-
Fixup documentation links (#61)
A bunch of work for fixing up some broken documentation links and improving the pipelines to try and prevent that in the future. Uses a fake anchor hack discussed previously to identify which links, when rendered in github pages, should be sent back to the main github repo page, and which should remain relative, though suitably fixed up. Adds a Makefile for building the site both locally (for testing) as well as in the pipelines. This also fixes a an issue with the python_api code being copied to the wrong location so it wasn't being updated. Adds a workflow to test for broken links in the plain markdown files as well. None of them check for faulty anchors. Partial work on providing the ability to execute the notebooks prior to markdown rendering as well (there's a bug that seems to be related to the recent shell=True fixup from #46 I haven't addressed there yet though). For that we also needed the ability to install all of the necessary pip modules. I tried to add a script to avoid use of requirements.txt and instead use the conda environment.yml without actually making use of conda (kind of a hack for the moment). Eventually I intend to replace that with proper docker container build image reuse.
Configuration menu - View commit details
-
Copy full SHA for 390c206 - Browse repository at this point
Copy the full SHA 390c206View commit details -
Increase python unit test CI timeout (microsoft#66)
* Increase python unit test CI timeout * crank up the python unit test timeouts a ton to workaround some random slowness issues with the github runners
Configuration menu - View commit details
-
Copy full SHA for a3c5a78 - Browse repository at this point
Copy the full SHA a3c5a78View commit details -
Merged PR 423168: Adding live feed of optimizer convergence status
Checking in all of the demo code, which includes the Tensorboard writer.
Configuration menu - View commit details
-
Copy full SHA for 1a77c88 - Browse repository at this point
Copy the full SHA 1a77c88View commit details -
Configuration menu - View commit details
-
Copy full SHA for 784f187 - Browse repository at this point
Copy the full SHA 784f187View commit details -
Merged PR 425264: Making OptimizerMicroservice respect the config pas…
…sed in the request Fixed the bug
Configuration menu - View commit details
-
Copy full SHA for 814375a - Browse repository at this point
Copy the full SHA 814375aView commit details -
Merged PR 425369: All observations data is now exposed via gRPC
Pumped all observations data through gRPC and added tests
Configuration menu - View commit details
-
Copy full SHA for fdf5701 - Browse repository at this point
Copy the full SHA fdf5701View commit details -
Merged PR 425209: Added ability for Regression-Enhanced Random Forest…
… regression model to deal... Pull Request 425209: Added ability for Regression-Enhanced Random Forest regression model to deal with categorical dimensions The categorical dimensions are enabled in the RERF regression model via the use of dummy variables derived from a OneHotEncoder, opting to drop the first level so as to avoid the DummyVariableTrap. See references below for details - best to start with the CMU statistics class materials. Other code touched: 1. Moved the Prediction enum used by RERF (and it's unit tests) to PREDICTION_VALUE, PREDICTION_VALUE_VARIANCE, and PREDICTED_VALUE_DEGREES_OF_FREEDOM as these were previously set at the corresponding SAMPLE based fields. 2. Added RERF unit test fitting four distinctly different quadratic surfaces associated with categorical dimensions. **References** 1. Very thorough and readable notes documenting linear regression using basis functions, categorical, and ordinal features: a) http://www.stat.cmu.edu/~cshalizi/mreg/15/lectures/lecture-16.pdf - intro to encoding categorical & ordinal dimensions, and b) http://www.stat.cmu.edu/~cshalizi/mreg/15/lectures/19/lecture-19.pdf - most closely describes what's been added to the RERF regression model. 1. https://en.wikipedia.org/wiki/Categorical_variable#Dummy_coding 1. https://en.wikipedia.org/wiki/Dummy_variable_(statistics) 1. https://www.algosome.com/articles/dummy-variable-trap-regression.html 1. https://en.wikipedia.org/wiki/Design_matrix since the code references a design matrix in some comments. Related work items: #889101
Configuration menu - View commit details
-
Copy full SHA for 4dc0c4f - Browse repository at this point
Copy the full SHA 4dc0c4fView commit details -
Merged PR 427208: Corrected Numerical Instability In Variance Computa…
…tion This run revealed an issue in variance computation: https://msdata.visualstudio.com/Database%20Systems/_build/results?buildId=21291699&view=results Specifically, a variance value of -7.450581e-09 made the sqrt throw. This PR corrects it by adding a small constant to the variance in the Random Forest. <s>At the moment this PR also includes a beefed up test that revealed the problem. Once that test passes we will have good evidence that the problem is resolved. I will restore the test to its original state and check in.</s> The test ran all night redoing the computation on every iteration. It did not throw so I conjecture that the bug is fixed.
Configuration menu - View commit details
-
Copy full SHA for e8b76d8 - Browse repository at this point
Copy the full SHA e8b76d8View commit details -
Merged PR 424491: Move to new optimizer
Move to new optimizer, remove support for old simple Bayesian optimizer, remove SQLRPC, implement smart cache example (Cpp implementation of Python SmartCache demo), implement Hypergrid deserialization
Configuration menu - View commit details
-
Copy full SHA for 63aa161 - Browse repository at this point
Copy the full SHA 63aa161View commit details -
Merged PR 427596: Include smart cache in unit tests
Include smart cache in the unit tests.
Configuration menu - View commit details
-
Copy full SHA for 99b23d1 - Browse repository at this point
Copy the full SHA 99b23d1View commit details
Commits on Sep 11, 2020
-
Configuration menu - View commit details
-
Copy full SHA for f36b92f - Browse repository at this point
Copy the full SHA f36b92fView commit details -
add usage of optimum method to bayesian optimization notebook (micros…
…oft#63) * add usage of optimum method * Apply suggestions from code review line breaks for github review Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 9150f10 - Browse repository at this point
Copy the full SHA 9150f10View commit details
Commits on Sep 12, 2020
-
Fixup documentation build scripts (microsoft#70)
* add website related targets to top level Makefile for convenient full project clean * restore a file revision controlled file after we clean the rest of the sphinx output FIXME: however, running the sphinx build still alters that file
Configuration menu - View commit details
-
Copy full SHA for fc25569 - Browse repository at this point
Copy the full SHA fc25569View commit details
Commits on Sep 14, 2020
-
various python related tweaks (microsoft#73)
* make sure we generate the grpc code with the right version of python * let us pass extra params to the python unittest module For instance "-k some_test_name" * add some more rules to cleanup after the python code * add a fixup to http_proxy environment variable interaction issues with grpc * raise a warning instead of messing with the user environment * actually, just print the warning - don't interrupt * also document the issue logged for the 3.8 bug
Configuration menu - View commit details
-
Copy full SHA for 15e7774 - Browse repository at this point
Copy the full SHA 15e7774View commit details -
Configuration menu - View commit details
-
Copy full SHA for 26bfe0f - Browse repository at this point
Copy the full SHA 26bfe0fView commit details
Commits on Sep 15, 2020
-
Merged PR 427161: Glow Worm Utility Function Optimizer
The main portion of this PR is the implementation of the Glow Worm Swarm Optimization algorithm as an alternative to the RandomSearchOptimizer. You will find its implementation in: ```\mlos\Optimizers\ExperimentDesigner\UtilityFunctionOptimizers\GlowWormSwarmOptimizer.py``` The first part of this post has a good description of the algorithm: https://www.hindawi.com/journals/mpe/2016/5481602/ Note that our implementation is aware of hierarchical hypergrids and essentially will split up the glow worms so that each group operates on a single, consistent space (i.e. glow worms will not in general move from LRU to MRU implementation except for random restarts). There is still plenty of low-hanging fruit (like watermelon size) in the implementation. We should get a big improvement by remembering the best parameters from previous calls to GlowWormSwarmOptimizer.suggest() and reusing them as the starting positions for subsequent calls to suggest (with some randomness of course). Other possible improvements include: varying the step size, varying the number of iterations, implementing better stopping criteria. But this PR is already big, so we can do that later. In order to plug in this optimizer I needed to standardize the UtilityFunctionOptimizer interface, hence the base classes: UtilityFunctionOptimizer and UtilityFunction were born. I also needed an adapter to translate points into a unit continuous search space. As I was going along, I also fixed a few small bugs. Lastly, the new adapters and the existence of the glow worm optimizer make it pretty trivial to implement a random-near-incumbent strategy that Ed says is hard to beat. That would be really something - having 3 utility function optimizers, 2 surrogate models, 2 utility functions would allow us to instantiate 12 distinctly different optimizers and this number will grow super-linearly with every new implementation.
Configuration menu - View commit details
-
Copy full SHA for b5b3fab - Browse repository at this point
Copy the full SHA b5b3fabView commit details -
Merged PR 428076: Refactoring Hypergrids, adding __getattr__
Following Andreas' feature request I added SimpleHypergrid.__getitem__() API. While I was at it I also cleaned up a bit as outlined below. I still need to run the linter and carefully peruse the code since the last merge was quite big and I had a lot of conflicts. However, I wanted you to be able to start checking out the work if you'd like. * Removed the DecisionTreeRegressionModelConfig instances for consistency - more such changes to come. * Made Point.__getitem__() throw a KeyError instead of returning None to make it consistent with the dictionary behavior. * Added a Point.get() method to still allow getting a default value if the dimension is not in Point * Renamed guest_subgrids to joined_subgrids * Separated Hypergrid and SimpleHypergrid into their respective files * Made dimension an Abstract Base Class (ABC) and enforced it * Made SimpleHypergrid pretty print like so: ![image (2).png](https://msdata.visualstudio.com/5603b9e3-ece0-4518-b9c8-1a36054d9970/_apis/git/repositories/e9fd4787-d7af-4833-beb4-34d130373fe8/pullRequests/428076/attachments/image%20(2).png)
Configuration menu - View commit details
-
Copy full SHA for ffd9125 - Browse repository at this point
Copy the full SHA ffd9125View commit details -
Merged PR 429225: Renamed translate to project in HypergridAdapters
Since the HypergridAdapters actually project points from one space to another, I thought a name change would be good. Especially that translation means "moving a point by some vector". ALSO: I added a little default behavior of being able to automatically project points if the authors of a HypergridAdapter derived class decides it's not worth their while to implement it by hand. The usage of HypergridAdapters as a wonderful data pre-processing tool has not caught up yet, so this change affects only my code and I have tested it :) But I encourage you to start using the adapters, because they are a reusable, clean and effective way of translating configurations from what the user wants to see, to what the models can compute :) Here are some ideas for useful features: * a PCA adapter - to preprocess all of those context values we are about to start pumping into our models * a one-hot-encoding adapter - as a reusable way to deal with categorical dimensions * a bootstrapping adapter * a column-dropping adapter * a logit adapter - to preprocess the objectives that have finite ranges
Configuration menu - View commit details
-
Copy full SHA for e27b7b2 - Browse repository at this point
Copy the full SHA e27b7b2View commit details -
Merged PR 428922: Fixed RERF instantiation and prediction interval de…
…fects Fixed RERF instantiation and prediction interval defects. Related work items: #894559, #894585
Configuration menu - View commit details
-
Copy full SHA for 9bcd430 - Browse repository at this point
Copy the full SHA 9bcd430View commit details -
make unit test more resilent to running from a different working dire…
…ctory Note: there are other instances of this, but we'll resolve those separately
Configuration menu - View commit details
-
Copy full SHA for b47737a - Browse repository at this point
Copy the full SHA b47737aView commit details
Commits on Sep 16, 2020
-
Fix and check for bad anchor links in docs as well (microsoft#82)
* some relative link sed fixups * document an issue I ran into * perform local anchor link checking as well * address comments about extra seds to remove '/./' patterns * consolidate some pip commands
Configuration menu - View commit details
-
Copy full SHA for 7ef8527 - Browse repository at this point
Copy the full SHA 7ef8527View commit details -
reorganize and update conda/python install docs (microsoft#79)
* reorganize and update conda/python install docs * remove instructions to get different miniconda and force python=3.7 in environment * Update documentation/01-Prerequisites.md Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> * Update README.md Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> * remove nb_conda_kernels for now. weird? * skip conda packages in pip conversion * don't try to install python with pip * consolidate special cases * typo lol Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for cf78c49 - Browse repository at this point
Copy the full SHA cf78c49View commit details -
add base implementation of optimum in OptimizerInterface (microsoft#76)
* add base implementation of optimum in OptimizerInterface * rename OptimizerInterface to OptimizerBase * simplify optimum, return points * rename file OptimizerInterface * fix tests * test optimum in BayesianOptimizerProxy * add pylint disable unused argumetn * compute optimum after registering * Update source/Mlos.Python/mlos/Optimizers/OptimizerBase.py Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> * Update source/Mlos.Python/mlos/Optimizers/OptimizerBase.py Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> * fix extra whitespace * fix OptimizerBase docstring * throw nicer error when calling optimum before register * update type hint for optimum * remove unused dict from typing * add some more tests * more verbose logging * typos, more prints * fix fstrings * more typo * typo * indentation merge issue * undo merge mistake Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 59c516a - Browse repository at this point
Copy the full SHA 59c516aView commit details -
Provide some links to the notebooks in the docs pages (microsoft#84)
* remove some debug output * include a menu on the notebooks index page if you land it from the main README.md
Configuration menu - View commit details
-
Copy full SHA for c65a526 - Browse repository at this point
Copy the full SHA c65a526View commit details
Commits on Sep 17, 2020
-
Work to get C++ unit tests working in Linux CI (microsoft#85)
* include cmake test target in cake build for linux Note: cake doesn't currently provide a ctest target, so we just call the virtual "test" target that cmake generates to do the same. * stub in a cmake test definition for running SmartCache with Mlos.Agent.Server FIXME: as with Mlos.UnitTest - it currently hangs * no need to hook that particular dotnet test into cmake it's pure c# and called from the "dotnet test" target. * add infrastructure for skipping certain dotnet tests by default temporarily disabling since they're broken * tweaks to vscode settings for easier initial debugging experience * make sure that when we debug build, we define DEBUG for the compiler Typically, clang/gcc only define NDEBUG for Release builds, but don't define DEBUG for other (e.g. Debug) builds. This is to replicate MSVC behavior which defines DEBUG for Debug builds. Without this, some UnitTests which were only supposed to run for Retail builds due to failing asserts otherwise were getting build unintentionally. * quick hack to also treat files without any extension as executables for Linux * just assume any file that doesn't end in a .json is an executable The previous method didn't work because for a binary named "Mlos.UnitTest" standard path parsing tools think the extension is ".UnitTest" * enabling the C++ unit tests via the agent server via ctest * some tweaks to try and get the output from the target process quicker * initial work to add some linux build/test work to the pipelines (just dotnet for now) * reorg and include cmake-build as well * make sure some apt dependencies are installed * also run the cake build script for testing * more vscode debugger improvements * improvements to allow searching for the settings registry assembly in Linux environments plus some other comments about eventual other fixups * initial work on letting cmake run the c++ tests via ctest * comments * work around test fixtures only executing once This is a hack around the fact that we currently need to cleanup all of the shared memory regions after ourselves before we can do another (or different) run. * add cmake test to the CI runs * address some comments
Configuration menu - View commit details
-
Copy full SHA for 56e9517 - Browse repository at this point
Copy the full SHA 56e9517View commit details
Commits on Sep 21, 2020
-
SmartCache example docs (microsoft#90)
* minor fixups to existing docs * start some instructions and explanations on running the SmartCache end to end example * add the docker build/run commands in explicitly * Update source/Examples/SmartCache/README.md Co-authored-by: Andreas Mueller <t3kcit@gmail.com> * Update source/Examples/SmartCache/README.md Co-authored-by: Andreas Mueller <t3kcit@gmail.com> * Update source/Examples/SmartCache/README.md Co-authored-by: Andreas Mueller <t3kcit@gmail.com> * link tweaks * reorg to address some comments * allow browsing some details about the examples on the github io pages as well Co-authored-by: Andreas Mueller <t3kcit@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for f837570 - Browse repository at this point
Copy the full SHA f837570View commit details -
Configuration menu - View commit details
-
Copy full SHA for 74cba5c - Browse repository at this point
Copy the full SHA 74cba5cView commit details
Commits on Sep 22, 2020
-
Merged PR 429491: Wrote some docs
Leaving the output in so that it's easier to review. I recommend having one window with a raw output and one window with a preview open simultanously.
Configuration menu - View commit details
-
Copy full SHA for af5690d - Browse repository at this point
Copy the full SHA af5690dView commit details -
Merged PR 431690: Started Working on Optimizer Evaluation Toolkit
This is the first of the PRs introducing the optimizer evaluation toolkit. Much of the toolkit's description is in the OptimizerEvaluationTools notebook so please refer to it for details. Broadly speaking, I put together a notebook that demonstrates how the optimizer evaluation tools can be used. So far we only keep track of goodness of fit, but I plan to add optima, and utility function optimziers to the toolkit too. I mostly try to replicate what Ed and Andreas do to understand the optimizers' performance and automate it as much as possible. I even used a bunch of Ed's visualizations :) **About the framework**: most of the synthetic functions we have used so far, now derive from SyntheticFunctionBase - basically exposing an interface to evaluate_point(), evaluate_dataframe(), and expose: parameter_space, output_space, and context. These functions are produced by the SyntheticFunctionFactory. Ideally, we will build up a portfolio of interesting functions, and @<Andreas Mueller> said he could point us to already existing libraries. Additionally, this PR introduces a ComponentConfigStore class, which we discussed in the Mlos channel. It is meant as a replacement for all of the XYZConfig classes. So far I have only instantiated two stores: ObjectiveFunctionConfigStore, and DecisionTreeConfigStore and once we are happy with the interface, and functionality, I'll replace all occurences of XYZConfig.
Configuration menu - View commit details
-
Copy full SHA for b90070d - Browse repository at this point
Copy the full SHA b90070dView commit details -
Merged PR 433551: Deleted SimpleBayesianOptimizer
It has served its purpose and will live on in the git history.
Configuration menu - View commit details
-
Copy full SHA for d077f3c - Browse repository at this point
Copy the full SHA d077f3cView commit details -
Modified: More Python docstrings and minor usability improvements for…
… Jupyter (#59) * minor typos on documentation * use __repr__ instead of __str__ everywhere * change test output This version just resolves some merge conflicts in ADO and then brings it back to Github.
Configuration menu - View commit details
-
Copy full SHA for 96576db - Browse repository at this point
Copy the full SHA 96576dbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7dfd6bc - Browse repository at this point
Copy the full SHA 7dfd6bcView commit details -
Merged PR 431954: Restart shared channel after the failure
Restart shared channel after the failure. Fix a bug when receiving a frame with incorrect type index. Missing: Restart when there is a sequence correct frame, partial, correct. Detect inactive channel. Add unit-test.
Configuration menu - View commit details
-
Copy full SHA for dd382f9 - Browse repository at this point
Copy the full SHA dd382f9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3f6374b - Browse repository at this point
Copy the full SHA 3f6374bView commit details -
Add end to end example docs to the github pages menu (microsoft#93)
* Add end to end example docs to the github pages menu * addressing some comments Remove the less polished SmartSharedChannel example from the website view
Configuration menu - View commit details
-
Copy full SHA for 9bdea45 - Browse repository at this point
Copy the full SHA 9bdea45View commit details
Commits on Sep 23, 2020
-
2020/september/optimizing predict performance (microsoft#92)
This change improves runtime of the DecisionTreeRegressionModel.predict() function. The main idea is to use column-wise operators so that pandas and numpy can vectorize them. Tracing reveals 2x-200x improvement in filtering out invalid rows depending on the complexity of the hypergrid and the number of rows in the dataframe.
Configuration menu - View commit details
-
Copy full SHA for 4f9480f - Browse repository at this point
Copy the full SHA 4f9480fView commit details -
initial work to binplace binaries on linux (microsoft#88)
* initial work to binplace binaries on linux This provides "make install" style targets to binplace binaries to target/bin/{Release,Debug}/ for easier execution. Note that SettingsRegistry assemblies were already being placed there. With the current Mlos.Agent lookup behavior, this means that we can now run ```sh cd target/bin/Release ./Mlos.Agent.Server ./Mlos.UnitTest ``` Instead of having to specify 1. the path to the dotnet tool 2. the paths to the Mlos.Agent.Server assembly 3. the path to the target executable 4. the MLOS_SETTINGS_REGISTRY_PATH environment variable To do this, we provide an Mlos.NetCore.Publishing.targets file that provides a new target "PublishProjectBinary" that in turn runs an appropriate "dotnet publish" command for the given platform. This method requires the fewest changes to the existing build system since we can specify the RuntimeIdentifier in the command line instead of in all of the build files. To allow recursive operation for projects that don't include that target, we let that target be a no-op most of the time. Next, we hook that into the existing Makefile and cmake wrappers as well. TODO: - tests - documentation * back out 'dotnet publish' commands * add the rules necessary to binplace dotnet dlls This is so we can use 'dotnet target/bin/Release/foo.dll' to run either the codegen tool (for external projects) or the Mlos.Agent.Server The main changes: - add binplace rules for all the dependencies - especially grpc - also add the *.runtimeconfig.json file for our two executables * include debug symbols for release builds Per discussion with grlap * addressing comments - consolidate GrpcVersion references into a property - make Mlos.NetCore projects binplace by default * with netcore projects binplacing by default this isn't necessary * Change binplacing logic for settings registry code gen to accommodate external projects
Configuration menu - View commit details
-
Copy full SHA for 4c2f79f - Browse repository at this point
Copy the full SHA 4c2f79fView commit details
Commits on Sep 24, 2020
-
Extending Optimum Functionality (microsoft#89)
Added a class enumerating all supported definitions of "optimum" and implemented three of them: * PREDICTED_VALUE_FOR_OBSERVED_CONFIG * LOWER_CONFIDENCE_BOUND_FOR_OBSERVED_CONFIG * UPPER_CONFIDENCE_BOUND_FOR_OBSERVED_CONFIG Users can select which of the definitions to use based on their risk profiles. For the confidence bound based optima, the users can specify the probability of type 1 error (alpha) to indicate their required level of certainty.
Configuration menu - View commit details
-
Copy full SHA for 24138c3 - Browse repository at this point
Copy the full SHA 24138c3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 53b9290 - Browse repository at this point
Copy the full SHA 53b9290View commit details -
Document SmartCache C++ example code (microsoft#99)
It is currently quite difficult to follow what's going on across three different languages. This PR attempts to heavily comment the SmartCache C++ example to help inform new users how the flow works across each component. It also links some of the other component markdown documentation into the documentation linking structure for easier browsing on the web page. * wip: documenting smart cache example * some initial work on a troubleshooting faq for starters, we just document a workaround to using the dotnet vscode plugins when in vscode (fixes microsoft#98) * unnecessary - it's included in stdafx.h * make vscode consistent with our Makefile methods for generating cmake output * add some comments * fixup a lack of null check and include remarks along with summary comments * don't line wrap * add a link to the troubleshooting documentation so it shows up in the webpages view * add more docs * mark some fields as private * more comments * comment about and disable the agent termination messages * Revert "comment about and disable the agent termination messages" This reverts commit e806def. * some docs for the Mlos.Agent.Server * add some more links in the overview readme * reduce some duplicate text * add some comments about the optimizer property for the C# agent MlosContext * update the overview comments in that file * simplify the execution commands * Revert "mark some fields as private" This reverts commit 55e85ad. This change needs to be done internally on ADO to handle some additional classes first. * Update source/Examples/SmartCache/Main.cpp Co-authored-by: Andreas Mueller <t3kcit@gmail.com> * Update source/Examples/SmartCache/Main.cpp Co-authored-by: Andreas Mueller <t3kcit@gmail.com> * Update source/Examples/SmartCache/SmartCache.SettingsRegistry/Codegen/SmartCache.cs Co-authored-by: Andreas Mueller <t3kcit@gmail.com> * Update source/Mlos.Agent.Server/MlosAgentServer.cs Co-authored-by: Andreas Mueller <t3kcit@gmail.com> * Update source/Mlos.Agent.Server/MlosAgentServer.cs Co-authored-by: Andreas Mueller <t3kcit@gmail.com> * Update source/Mlos.Agent.Server/MlosAgentServer.cs Co-authored-by: Andreas Mueller <t3kcit@gmail.com> * addressing feedback, trying to improve some comments * remove a 'using' statement that somehow got added * fixup a botched merge that broke the stylecop checks * cleanup and link in various other in-tree markdown documentation - add a reference the the more complete settings system codegen markdown documentation in the SmartCache.cs comments - cleanup that markdown file (both linting and some language) - rename it so that we can include it in the webpage - do the same for several other files - link them from the repo organization page so we can browse to them from the generated web pages * fixups for website link checking * address comments * simplify some commands, fix some links * comments to help readers see where the external agent message handlers are * addressing comments Co-authored-by: Andreas Mueller <t3kcit@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 8e838b0 - Browse repository at this point
Copy the full SHA 8e838b0View commit details
Commits on Sep 25, 2020
-
Adding Config Stores (microsoft#102)
This replaces the XYZConfig classes with ComponentConfigStore objects.
Configuration menu - View commit details
-
Copy full SHA for a3d8348 - Browse repository at this point
Copy the full SHA a3d8348View commit details -
Merged PR 434705: Fix Mlos.Agent race condition on shutdown
Fixing the crash found in the CDPx pipeline. Agent disposed the shared memory while there was a still active reader thread. Link to the failure: https://msdata.visualstudio.com/Database%20Systems/_build/results?buildId=22386882&view=logs&j=fd490c07-0b22-5182-fac9-6d67fe1e939b&t=dcf93f35-c1ba-5dce-0078-8d628250da3a
Configuration menu - View commit details
-
Copy full SHA for cac7e59 - Browse repository at this point
Copy the full SHA cac7e59View commit details -
Merged PR 438048: Fix Mlos.Agent active learning hang
In active learning mode Mlos.Agent hangs, when there is an unhandled exception. Agent waits for the target process termination and target process waits for the message from the agent.
Configuration menu - View commit details
-
Copy full SHA for 3a1eda4 - Browse repository at this point
Copy the full SHA 3a1eda4View commit details -
Merged PR 438053: Optimizer returns an integer when expected.
Optimizer returns an integer when expected. This simplfies parsing parameters (JsonElement to int).
Configuration menu - View commit details
-
Copy full SHA for 8f304c2 - Browse repository at this point
Copy the full SHA 8f304c2View commit details -
Make Mlos.Server.Agent use CommandLineParser for configurable endpoin…
…ts (microsoft#106) * setup a commandline parser for Mlos.Agent.Server for configuring the endpoints * listen on any ip address for incoming grpc connections * check for the executable * update msbuild invocation of mlos.agent.server for new cli syntax as well
Configuration menu - View commit details
-
Copy full SHA for 88c2b4c - Browse repository at this point
Copy the full SHA 88c2b4cView commit details -
added feature imputation to RERF predict() (microsoft#100)
* added feature imputation to RERF predict() * ensured unit test exercised single row x prediction; retained more info from fit to use for prediction Co-authored-by: Ed Thayer <edthaye@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for d1824eb - Browse repository at this point
Copy the full SHA d1824ebView commit details
Commits on Sep 27, 2020
-
2020/september/extending grpc tests (microsoft#107)
All tests in TestBayesianOptimizer now test both a local optimizer and an optimizer created via gRPC. These tests ended up being pretty slow since we didn't have an API to register observations in bulk over gRPC. So I added that, along with a bunch of tracing to diagnose the problem and validate the improvement.
Configuration menu - View commit details
-
Copy full SHA for 7f3b381 - Browse repository at this point
Copy the full SHA 7f3b381View commit details
Commits on Sep 28, 2020
-
Grpc linux fixups (microsoft#108)
* improve and add debugging targets * add a 'dotnet-build-quick' target for easier access to faster rebuilds * spelling * remove an environment variable that was breaking unencrypted grpc connections This overrode and made the following line useless: AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true); The resulting error was: Unhandled exception. Grpc.Core.RpcException: Status(StatusCode=Internal, Detail="Bad gRPC response. Response protocol downgraded to HTTP/1.1.") Further details in: #4 grpc/grpc-dotnet#654 dotnet/runtime#988 * make one of the unit tests also invoke the optimizer * build the mlos.agent.server when building projects that need it for unit tests This makes it easier to run "make all test" in one project directory. * need to make sure that the mlos modules is already installed and available
Configuration menu - View commit details
-
Copy full SHA for 0186f9e - Browse repository at this point
Copy the full SHA 0186f9eView commit details -
rework grpc dll binplace operations to a single project ...
... so there are no duplicate binplace issues for certain build systems
Configuration menu - View commit details
-
Copy full SHA for e91f632 - Browse repository at this point
Copy the full SHA e91f632View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0b1bfe1 - Browse repository at this point
Copy the full SHA 0b1bfe1View commit details -
Merged PR 437736: Implement managed MlosContext, Mlos.Agent.Sevice cl…
…eanup shared memory in Active Learning mode Implement managed MlosContext, NetCore shared channel can be instantiated with different channel policies. Mlos.Agent.Sevice unlinks shared memory when running in active learning mode. GlobalMemoryRegion has a counter with number of active processes, the last process will cleanup OS resources (unlink shared memory on Linux). We are not detecting the crashes, when the terminated process did not decrease usage counter.
Configuration menu - View commit details
-
Copy full SHA for f8ce37e - Browse repository at this point
Copy the full SHA f8ce37eView commit details -
Cpp notebook with docker (microsoft#101)
* start on CPP example notebook for smart cache
Configuration menu - View commit details
-
Copy full SHA for 4febf6f - Browse repository at this point
Copy the full SHA 4febf6fView commit details
Commits on Sep 29, 2020
-
add instructions for lab2 (microsoft#118)
* add instructions for lab2.5 * address Brian's comments * Update source/Mlos.Notebooks/SmartCacheCPP.ipynb Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> * fix docker invocation, clean run * unrelated typo from walkthrough Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 41eb797 - Browse repository at this point
Copy the full SHA 41eb797View commit details -
Cli parser fixups (microsoft#116)
* tweaks to default debugger setup - don't require optimizer * tweak a notebook link to use the github pages rendered link * Revert "tweak a notebook link to use the github pages rendered link" This reverts commit c289504. * look for extra args and output more useful help text * a few more usage help tweaks * reformat the strings in the code, and make it work for non "Errors" as well like having ExtraArgs * addressing comments - move cli parsing to a separate file - change error message for unknown extra arguments * add a simple test target to check for proper parsing of bad arguments note that parsing of the right arguments already happens in other unit tests This invokation method appears to work on both windows and linux * Revert "add a simple test target to check for proper parsing of bad arguments" This reverts commit 2ec178a.
Configuration menu - View commit details
-
Copy full SHA for 5936ceb - Browse repository at this point
Copy the full SHA 5936cebView commit details
Commits on Oct 1, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 231c8eb - Browse repository at this point
Copy the full SHA 231c8ebView commit details
Commits on Oct 6, 2020
-
Fixup comment word wrap settings for vscode vim (microsoft#122)
* tweaks to vscode plugins to reflow comments at a width that matches the rest of the code style * addressing comments
Configuration menu - View commit details
-
Copy full SHA for 489846f - Browse repository at this point
Copy the full SHA 489846fView commit details
Commits on Oct 12, 2020
-
Fixups for libstdc++-10 support (microsoft#133)
* make sure to include the wide character string functions * Replace older __in SAL notation with current _In_ to avoid libstc++-10 issues GNU libstdc++ 10 uses the token `__in` in some generated code, so we need to avoid it.
Configuration menu - View commit details
-
Copy full SHA for 9450467 - Browse repository at this point
Copy the full SHA 9450467View commit details -
Merged PR 439762: Binplace Net.Core applications
Binplace Net.Core applications. To binplace the application, include following line in the csproj: <Import Project="$(BaseDir)\build\Mlos.NetCore.Binplace.targets" /> VS builds fine.
Configuration menu - View commit details
-
Copy full SHA for a678358 - Browse repository at this point
Copy the full SHA a678358View commit details -
Configuration menu - View commit details
-
Copy full SHA for 30b3649 - Browse repository at this point
Copy the full SHA 30b3649View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8aae0e1 - Browse repository at this point
Copy the full SHA 8aae0e1View commit details -
provide a script for installing python 3.7
related fixup for clang script as well
Configuration menu - View commit details
-
Copy full SHA for 1f2cf3e - Browse repository at this point
Copy the full SHA 1f2cf3eView commit details -
Merged PR 443258: Enable rule SA1642 Constructor documentation summar…
…y should include standard... Enable rule SA1642 Constructor documentation summary should include standard text Initializes a new instance of the <see cref="... "/> struct (or class). Enforcing rule, text is autogenerated. Check-in first as an effort to minimize amount of changes in PR "Only the target process creates the config shared memory".
Configuration menu - View commit details
-
Copy full SHA for f23f29b - Browse repository at this point
Copy the full SHA f23f29bView commit details -
Merged PR 443290: Rename shared memory methods
Rename shared memory methods Create to CreateNew, Open to OpenExisting
Configuration menu - View commit details
-
Copy full SHA for 7cf243a - Browse repository at this point
Copy the full SHA 7cf243aView commit details -
Merged PR 441469: The target process creates the config shared memory.
Only the target process creates the config shared memory. Global shared memory now contains the information about registered memory regions and registered settings assembly. Now the target process creates the config shared memory, where it create it with the desired size and name. (Configuration is not exposed yet, but changing it does not require changes in the agent and the target process). We do support only one shared config memory region, but now this can be extended. Fix a bug when shared maps are not cleanup in case of initialization failure. Implementation of dictionary lookups is moved to SharedConfigDictionaryLookup struct. Next PR: Remove static fields from MlosContext.
Configuration menu - View commit details
-
Copy full SHA for ead861a - Browse repository at this point
Copy the full SHA ead861aView commit details
Commits on Oct 13, 2020
-
Docker image publishing work (microsoft#128)
Lots of build/pipeline cleanup with the goal of making and publishing standard Docker images the for Linux developers and the CI pipelines (microsoft#36). - Most implementation notes are in `.github/workflows/main.yml`, but here's a brief summary: - Nightly we build fresh docker images for Ubuntu {16.04, 18.04, 20.04}. They are only published if all of the usual build/test jobs pass (using the same logic as for PRs). That also helps prime the local caches for the following: - On each PR, we 1. (re)build the image from a local buildx cache that's keyed on the sha of the PR, and optionally will also pull from the latest "main" build (since it presumably worked). That is initially only stored in a local cache. Part of the goal here is to avoid the need to push unvetted docker images (e.g. tagged by sha) to the public registry and then have to clean them up later (which is not always even possible). 2. For each build/test job, we again "rebuild" the image, using that cache from step 1, so that it goes quickly. Then we load the image into the local runner's docker daemon and start an instance to run our tests. This reload is necessary because individual jobs are assigned to new runners so we can't reuse state between jobs. Jobs are organized into dependency graphs to account for this cache reuse. At the moment, all of the tests run on all versions of Ubuntu, which makes the CI pipeline a bit slow. - DONE: We could consider only running 16.04 and 20.04 in the CI pipelines and then running 18.04 in the nightly scheduled run or else just drop support for it entirely. 3. Once all of the tests pass, and the PR is merged, we execute a final job to update the published version of the job by retagging it and pushing it to various registries. The current registries are all Github services (for convenience to get started): - docker.pkg.github.com: this is nice because it supports the built in `GITHUB_TOKEN` secret so doesn't require any additional work for a forked repo to use for testing (they push to their own path though). However, it doesn't support anonymous pulls and is being deprecated by ... - ghcr.io: this the newly announced Github Container Registry service. It requires a special set of secrets to be assigned to the repo, so needs some additional work to make use of. Also, as of now, the Microsoft org doesn't seem to have it enabled, so for now I've set it to push to the Microsoft-CISL org. Additional work: - Reorg the Dockerfile to use multi-stage builds to allow skipping copying the source code into it so that we only distribute the build tools. We can also optionally build slimmer "python-only" images this way. In a future PR we'll take on the work to switch from straight pip to conda (microsoft#83) - TODO: Could optionally prime the `/src/MLOS` folder in the image so that one could `git pull` immediately to get the latest code if they wanted to without a local copy as a bind mount. - Added a `make docker-image` target to simplify repeated local docker image build invocations. - Switch to using tools (cmake, dotnet) installed in the system rather than fetched during build (microsoft#109) - These scripts are also nice reusable elements for users to install in their own environments if they wish. - DONE: Update the documentation to reference them instead of relying on the old "fetch" method. - DONE: There is work in ADO that provides a similar `install.python.sh` script to setup the `python3.7` install. Will port that over later as well. - Cleanup _most_ of the build system to match (I have not yet removed the old setup scripts, but they are no longer automatically called). - Added vscode integration support for developing inside the container (microsoft#105, microsoft#2) - Added support for running as a non-root user so that bind mounts of the repo source tree work without re-chowning files on the host system as root. This required a fair bit of fixups, but I think should cut down on some issues with students/others and makes the vscode integration easier as well. - Add nightly run scheduling of Python long haul tests (microsoft#113) - TODO: Fixup docker related documentation (microsoft#87)
Configuration menu - View commit details
-
Copy full SHA for 307bd12 - Browse repository at this point
Copy the full SHA 307bd12View commit details
Commits on Oct 14, 2020
-
MLOS lab 3 Leveldb tuning (microsoft#129)
* Added leveDB tuning notebook * Added brief explanatino * added reference * added leveldb installation instructions and run_workload function * checkpoint * added run optimizer functionality to leveldb_tuning.ipynb * PR changes
Configuration menu - View commit details
-
Copy full SHA for 00efbf5 - Browse repository at this point
Copy the full SHA 00efbf5View commit details
Commits on Oct 15, 2020
-
Merged PR 444061: Binplacing, always include Platform in the binplace…
… folder name Add Mlos.Cpp.Binplace.targets for to binplace native projects. Resolve property BinplaceRootDir after including Microsoft.Cpp or Microsoft.NET.Sdk, so the Plaftorm propery is always set. Add option SkipBinplaceSettingsRegistry (do not binplace Mlos.NetCore). CakeBuild, when creating nuget spec, verify if there are files in the provided path. Include missing .inl file in vcxproj.
Configuration menu - View commit details
-
Copy full SHA for 7e2c0c3 - Browse repository at this point
Copy the full SHA 7e2c0c3View commit details -
Merged PR 445008: Fix Linux warnings (except unknown pragmas)
Fix Linux warnings (except unknown pragmas)
Configuration menu - View commit details
-
Copy full SHA for eb06b90 - Browse repository at this point
Copy the full SHA eb06b90View commit details -
Merged PR 446343: MlosContext singleton
Remove static fields from MlosContext class. Create a static Mlos Instance field (workaround until we have Experiment class).
Configuration menu - View commit details
-
Copy full SHA for b63f66a - Browse repository at this point
Copy the full SHA b63f66aView commit details -
Merged PR 447592: Integration work with SqlServer
Remove usage of auto from c++ files, remove Mlos.Agent.Server from nuget lib folder (it is present in bin folder)
Configuration menu - View commit details
-
Copy full SHA for 4249272 - Browse repository at this point
Copy the full SHA 4249272View commit details -
Merged PR 448296: Set correctly SettingsAssemblyBinplaceDir in Mlos.T…
…est for CoreXT builds Set correctly SettingsAssemblyBinplaceDir in Mlos.Test for CoreXT builds
Configuration menu - View commit details
-
Copy full SHA for e0d1c3f - Browse repository at this point
Copy the full SHA e0d1c3fView commit details -
Merged PR 448359: Register Settings Assembly, remove ApplicationFilePath
Remove ApplicationFilePath from SettingsAssembly config. Mlos.Agent.Server is using env variable to locate settings assembly registry.
Configuration menu - View commit details
-
Copy full SHA for eeebe72 - Browse repository at this point
Copy the full SHA eeebe72View commit details -
Merged PR 449274: Remove unused file
Remove unused file
Configuration menu - View commit details
-
Copy full SHA for 422a77a - Browse repository at this point
Copy the full SHA 422a77aView commit details -
Merged PR 448661: Additional NetCore build/doc cleanup
Some followup work to PR !444061 and additional minor fixups. - Update the docs to reference the new paths - Make `cmake` aware of the `target/bin/` placed items - Fixup some `cmake` dotnet parallel build issues by changing which target references the dependencies (`dotnet build` has issues when invoked in parallel) - Start removing references to the `MLOS_SETTINGS_REGISTRY_PATH` environment variable - Make sure the `Mlos.Agent.Server` exits non-zero if one of the components or `Mlos.Agent` fails
Configuration menu - View commit details
-
Copy full SHA for 665fe8d - Browse repository at this point
Copy the full SHA 665fe8dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 18bb119 - Browse repository at this point
Copy the full SHA 18bb119View commit details -
Merged PR 449101: Adding standard comments in Net.Core projects
Adding standard comments in Net.Core projects Removing the following exceptions from the analyzer's rules. SA1623 The property's documentation should begin with: Gets (or Sets) SA1633 The file header is missing or not located at the top of the file CA1801 Parameter of method is never used
Configuration menu - View commit details
-
Copy full SHA for 04607f3 - Browse repository at this point
Copy the full SHA 04607f3View commit details -
Merged PR 449435: Strict compilation on Linux
Continuation of Brian's work bpkroth/more-cpp-compiler-warning-fixups. Enable clang compilation flags: -Wall -Wextra -Wpedantic -Werror. Remove macros from Mlos.Core. Rename variable name from elementLength to elementDataSize.
Configuration menu - View commit details
-
Copy full SHA for e5ee53c - Browse repository at this point
Copy the full SHA e5ee53cView commit details -
quick tweak to support updating dotnet on a runner
Works around some CI issues with "dotnet tool" commands for Cake build with older dotnet versions on older Ubuntu 16.04 ADO runners and similar issues for cmake. TODO: This will need additional fixes in the future. We still have the setup-cmake.sh, setup-dotnet.sh type of scripts which can install local copies, but we've started the process of stripping use of those out of the build system. One benefit of those is that they also allow us to fetch (pin) an explicit version (though that was probably a security issue) and use that version even if the default system repo lists don't have a newer version for us. A more complete fix will be to include the logic to check for an appropriate version in the existing system repos (not just existence of the package), and if it doesn't exist add the necessary package repos as well.
Configuration menu - View commit details
-
Copy full SHA for fc795cb - Browse repository at this point
Copy the full SHA fc795cbView commit details -
Some LevelDB Notebook cleanup (microsoft#135)
* work to add external project notebooks to the website menus * tweak some of the formatting a tad * copy the leveldb notebook images into place * convert them to markdown style links
Configuration menu - View commit details
-
Copy full SHA for 38987e5 - Browse repository at this point
Copy the full SHA 38987e5View commit details -
some build pipeline tweaks (microsoft#138)
- run build.cake for the right config - run cmake in parallel inside cake - run/test the cmake-install target
Configuration menu - View commit details
-
Copy full SHA for 25f536b - Browse repository at this point
Copy the full SHA 25f536bView commit details
Commits on Oct 17, 2020
-
Configuration menu - View commit details
-
Copy full SHA for b92af2f - Browse repository at this point
Copy the full SHA b92af2fView commit details
Commits on Oct 19, 2020
-
Update readme to mention class and fix some links (microsoft#140)
* fixup some markdown errors * add a reference to the class in the README * fixup notebook links following microsoft#135
Configuration menu - View commit details
-
Copy full SHA for 42d4634 - Browse repository at this point
Copy the full SHA 42d4634View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6e551ca - Browse repository at this point
Copy the full SHA 6e551caView commit details
Commits on Oct 20, 2020
-
increase longhaul job step timeout beyond default (microsoft#142)
* increase longhaul job step timeout beyond default previously we only set the overall job timeout beyond the default of 360 minutes (6 hours) with this change we also explicitly set the timeout for the longhaul execution step in the job (in case that was also defaulting to 360) still not sure if this will fix things, but worth a try
Configuration menu - View commit details
-
Copy full SHA for 732b9d7 - Browse repository at this point
Copy the full SHA 732b9d7View commit details
Commits on Oct 22, 2020
-
Adding OneHotEncoder HypergridAdapter (microsoft#130)
* initial commit for OneHotEncoder HypergridAdapter * Cleaned up project/unproject; dropped point project/unproject methods; corrected some defects in Point, HypergridAdapter; Added more unit tests * Addressing PR feedback * Addressing pylint * Addressing PR feedback Co-authored-by: Ed Thayer <edthaye@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 7c0e1fc - Browse repository at this point
Copy the full SHA 7c0e1fcView commit details -
Added pickling and pickling tests (microsoft#144)
Co-authored-by: Adam Śmiechowski <adsmiech@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 1803308 - Browse repository at this point
Copy the full SHA 1803308View commit details -
2020/october/bug fixes (microsoft#143)
* A bunch of bug fixes * More bug fixes Co-authored-by: Adam Śmiechowski <adsmiech@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 0637f16 - Browse repository at this point
Copy the full SHA 0637f16View commit details
Commits on Oct 23, 2020
-
Removed RegressionModelConfig class (microsoft#146)
Co-authored-by: Adam Śmiechowski <adsmiech@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 05fbde3 - Browse repository at this point
Copy the full SHA 05fbde3View commit details -
Fixed how we compute the number of samples per tree (microsoft#147)
* Fixed how we compute the number of samples per tree * Fixed bug in TestBayesianOptimizer Co-authored-by: Adam Śmiechowski <adsmiech@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 1682175 - Browse repository at this point
Copy the full SHA 1682175View commit details
Commits on Oct 26, 2020
-
Adjusted parameter ranges for many of the Optimizer's components. (mi…
…crosoft#149) Co-authored-by: Adam Smiechowski <adsmiech@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 2f8586c - Browse repository at this point
Copy the full SHA 2f8586cView commit details -
Added a little functionality to Tracer and to Logger. (microsoft#151)
Co-authored-by: Adam Smiechowski <adsmiech@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 0207287 - Browse repository at this point
Copy the full SHA 0207287View commit details
Commits on Oct 27, 2020
-
Configuration menu - View commit details
-
Copy full SHA for fe43ab1 - Browse repository at this point
Copy the full SHA fe43ab1View commit details -
reorg smartcache docs, fixup links, etc. (microsoft#123)
* reorg docs, fixup links, etc. Follow ups to microsoft#101 (comment) * link fixup * Update source/Mlos.Agent.Server/README.md Co-authored-by: Andreas Mueller <t3kcit@gmail.com> * Update source/Mlos.Agent.Server/README.md Co-authored-by: Andreas Mueller <t3kcit@gmail.com> * addressing comments * add a component documentation menu to the website Co-authored-by: Andreas Mueller <t3kcit@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 6fb4eaa - Browse repository at this point
Copy the full SHA 6fb4eaaView commit details -
Adding NestedPolynomialFunction as another synthetic function (micros…
…oft#152) * Added nested polynomial function * Bug fix in ComponentConfigStore * Added a better comment for NestedPolynomialObjective * Fixed linter * Resolved my own comments Co-authored-by: Adam Smiechowski <adsmiech@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 1a313f4 - Browse repository at this point
Copy the full SHA 1a313f4View commit details
Commits on Oct 28, 2020
-
Fixed a bug that caused intermitted test failures (microsoft#153)
Co-authored-by: Adam Śmiechowski <adsmiech@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for d946242 - Browse repository at this point
Copy the full SHA d946242View commit details
Commits on Oct 31, 2020
-
OneHotEncoder Hypergrid Adapter integration in Regression-Enhanced Ra…
…ndom Forest Model (microsoft#148) * intial commit to confirm branch rename worked * found and fixed defects in CategoricalToOneHotEncodingAdapter for cases without categorical dims; worked through changes in RERF and RERF tests * removed unneeded comment/code * removed test for gradient; will replace this when the path for using this gradient is clear * added sklearn StandardScaler to RERF to reduce hat matrix condition number; eliminated unit tests for exact polynomial and gradient coeffs as these are relative to scaled X and are not knowable from unit tests; eliminated specific random seeds used in unit tests * addessing PR feedback * increased number of lasso fit iterations to address prediction inaccuracies when fitting heirarchical categorical objective * expect to have resolved flakey test by increasing training set size * addressing pylint * still trying to understand prediction accuracy miss Co-authored-by: Ed Thayer <edthaye@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 417ff56 - Browse repository at this point
Copy the full SHA 417ff56View commit details
Commits on Nov 4, 2020
-
Optimizer Evaluation Tools (microsoft#155)
Optimzier Evaluation Tools The bulk of this PR lives in two files: - OptimizerEvaluationReport.py - OptimizerEvaluator.py Their usage is presented in the OptimizerEvaluationTools.ipynb Notebook. The goal of this work is to produce a framework to efficiently compare optimizer configurations across a variety of synthetic objective functions.
Configuration menu - View commit details
-
Copy full SHA for 76100b0 - Browse repository at this point
Copy the full SHA 76100b0View commit details
Commits on Nov 7, 2020
-
use pytest to run python tests (microsoft#154)
* use pytest to run python tests * install pytest in windows python checks CI * install pytest in docker image * Update scripts/run-python-tests.sh Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> * some review comments * add back some options from the previous invocation * allow waiting for the grpc server to finish stopping and cleanup the client socket as well * rewrite unittest patterns to pytest patterns * no camelcase in teardown in pytest * more unittest to pytest conversion * remove empty inheritance * actually safe file * reviews by adam Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> Co-authored-by: Brian Kroth <bpkroth@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for fd8ef08 - Browse repository at this point
Copy the full SHA fd8ef08View commit details
Commits on Nov 12, 2020
-
Configuration menu - View commit details
-
Copy full SHA for ae5ab29 - Browse repository at this point
Copy the full SHA ae5ab29View commit details
Commits on Nov 13, 2020
-
Registering multiple objectives (microsoft#175)
This PR is a first step towards multi-objective optimization. It's really simple: the user can now define an objective space that's multi-dimensional. Subsequently the user can register observations that contain values for one or more of those objectives. Still, only one objective is optimized. Effectively, all those extra objectives are remembered, but ignored.
Configuration menu - View commit details
-
Copy full SHA for 00992a6 - Browse repository at this point
Copy the full SHA 00992a6View commit details
Commits on Nov 14, 2020
-
Add hypergrid adapter exposing SkLearn's PolynomialFeatures for adapt…
…ee continuous dimensions (microsoft#172) * Adding hypergrid adapter that explodes continuous dimensions to polynomial basis functions * Added point project/unproject tests; fixed some defects these tests exposed * added more tests; fixed more defects * addressing PR feedback; Recommendation to use @pytest not yet working, so that snippet is commented out - to be fixed before final checkin * cleaned up unit tests w/ @pytest.mark.parameterize; found one defect in the _test_polynomial_feature_values_are_as_expected Co-authored-by: Ed Thayer <edthaye@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 6685e37 - Browse repository at this point
Copy the full SHA 6685e37View commit details
Commits on Nov 17, 2020
-
Initial work to add gcc support (microsoft#176)
* Remove clang-tidy target It doesn't work as is since it attempts to redefine the target upon every sub-project include. At the moment we use uncrustify anyways. * Initial work to enable gcc support Started as a hack to workaround cmake's FetchContent not appearing to pass the CMAKE_COMPILER_ID through when building with CXX=clang++-10 This also fixes a few minor issues that gcc complained about. TODO: - [ ] Enable CI pipeline tests * allow overriding the compiler choice on the cli * remove cdecl * add gcc testing to ci pipeline * tweaks * include the cxx compiler in the name * exclude gcc from Ubuntu 16.04 * remove some old debugging code * remove need for -fdeclspec from clang also shorten the name of the attr macro * porting some changes from grlap over * change the style of initializer we use for std::array for reflection ... ... to something that gcc in Ubuntu 16.04 can use * enable gcc on 16.04 again * uncrustify fixups * tweak the job names * default to clang, but allow it to be overridden Co-authored-by: Greg Lapinski <grlap@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for ef5142d - Browse repository at this point
Copy the full SHA ef5142dView commit details
Commits on Nov 18, 2020
-
Create Nuget Packages for C# (microsoft#177)
This is the first step in a line of work aimed towards making MLOS more natively consumable by external projects. The basic idea here is that we use the standard .NET.Sdk method of packaging each of the relevant `.csproj`s as their own `.nupkg`. This let's us rely on `dotnet` to automatically track `PackageReference` and `ProjectReference` dependencies and generate them into the `.nuspec` that produces the `.nupkg`. Since `.nupkg` files are considered immutable, we have to create unique version numbers for each of them. To do this, I added [`GitVersionTask`](https://github.com/GitTools/GitVersion), but only for the OSS builds. Typically, GitVersion expects one to use GitFlow or GithubFlow workflows with develop and release branches. We are currently only using a Mainline workflow model, so the `GitVersion.yml` file is configure as such. The versions produced use SemVer by default, so when building on a branch, we get a version like the following: `Mlos.SettingsSystem.CodeGen.0.1.1-codegen-nuget-for-external-project-integration.33.nupkg` Where - the `0.1.1` portion is taken as the number of commits on the mainline branch since the last v0.1.0 tag found in the repo - the `-codegen-nuget-for-external-project-integration` is the name of the branch - and the `.33` is the number of commits since that branch point. In effect, we can use `git tag` to version our `x.y.z` releases, at least until we start adding `release/0.1.x` style servicing branches. For the moment, the `.nupkg` files are not signed and not published (e.g. to nuget.org). Instead, we produce them into the `target/pkg/$(Configuration)/` tree whenever either the new `Mlos.NetCore.Components.Packages.proj` is build or else when `dotnet pack` is run on some `.csproj`. The `ExternalIntegrationExample.SettingsRegistry.csproj` (which otherwise uses no Mlos build rules and acts as a standard `.NET.Sdk` example) is used for testing and is configured to pull from that tree (for local testing) as well. The next stage of work (separate PR) is to configure a set of `cmake` rules using `FetchContent` to grab the C++ components, build these packages and reference them when building the rest of a C++ project and its dependencies. TODO: - [x] Enable the test project and package building in CI - [x] for Windows - [x] for Linux - [x] Test VisualStudio build - [x] Verify CDPX/CoreXT builds (!473121) WONTFIX: - Leaving actual publishing/signing to nuget.org for a future PR
Configuration menu - View commit details
-
Copy full SHA for 9093352 - Browse repository at this point
Copy the full SHA 9093352View commit details
Commits on Nov 19, 2020
-
Windows build fixups for recent MSBuild tool versions (microsoft#184)
* recent msbuild auto imports those build/*.targets from PackageReferences * make sure to tell msbuild that we aren't attempting to reference the c# settings registry dll from c++ projects
Configuration menu - View commit details
-
Copy full SHA for d6d882f - Browse repository at this point
Copy the full SHA d6d882fView commit details -
Avoid some broken cmake package versions in Docker image build (micro…
…soft#185) * update comment * minor tweaks * rejectlist a couple of cmake package versions that are causing problems due to lacking dependencies
Configuration menu - View commit details
-
Copy full SHA for f8fbbec - Browse repository at this point
Copy the full SHA f8fbbecView commit details -
Computing Pareto (microsoft#181)
This PR introduces a simple algorithm for computing a pareto frontier, given an objectives dataframe and a corresponding OptimizationProblem. You will find the algorithm in ParetoFrontier. This class will later be extended to maintain a cache of the latest pareto, and to iteratively add to it as more observations roll in. It will be owned by a BayesianOptimizer.
Configuration menu - View commit details
-
Copy full SHA for 5c262ed - Browse repository at this point
Copy the full SHA 5c262edView commit details -
Configuration menu - View commit details
-
Copy full SHA for b2e16cb - Browse repository at this point
Copy the full SHA b2e16cbView commit details -
* start on context features * add draft of context usage * add draft of context in BO notebook * line length lint * use Optimizer.feature_space * allow computation of optimum in BayesianOptimization.optimum * be backwards compatible for now * add wrapper around surrogate model * add example of BO with context * fix sign issue, raise on name conflict * get dimension names for comparison * try to pacify pylint * update optimizer interface * one more for the interface * rework context notebook * whoops safe failure * refactor suggest into OptimizerBase, add maximize to public interface * intermediate checkin * update to mirror refactoring * minor fixups, use different function in notebook * go back to old join for feature space, remove maximize * slightly slimmer feature space * add context to BayesianOptimizer.register * move feature dataframe construciton to optimization problem * add greedy utility function to maximize utility on speculative optimum * fix use us sign in greedy utility function * undo spurious change * ensure register is consistent in OptimizerBase * Update source/Mlos.Python/mlos/Optimizers/OptimizationProblem.py Co-authored-by: Adam Śmiechowski <51930563+byte-sculptor@users.noreply.github.com> * Update source/Mlos.Python/mlos/Optimizers/OptimizationProblem.py Co-authored-by: Adam Śmiechowski <51930563+byte-sculptor@users.noreply.github.com> * rename best_predicted_within_context to best_speculative_within_context * rename config_values to feature_values * fix merge issues * check in changes for context notebook * make sure BayesianOptimizerProxy adheres to Optimizer interface * rename GreedyUtilityFunction to PredictedValueUtilityFunction * don't explicitly set n_new_samples_before_refit if using default * actually rename greedyutilityfunction * minor fixes for rename * pylint * change interface of predict to use parameter_values * typo * pyflake whitespace * fix suggest calls in grpc * undo support for context in gRPC and cheat a bit * hack around context in gRPC * fix some tests for change of get_all_observations * store parameters and context separately in BayesianOptimizer * remove some more weird projections * create features of concatenated observations lol * pass feature df to compute_goodness_of_fit * indentation fix * remove unused variable * start on testing * fix and test some error messages * add shape tests * add test for optimization problem * pylint * add tests for actual optimization * BayesianOptimizerProxy has no ExperimentDesigner * Update source/Mlos.Python/mlos/Grpc/BayesianOptimizerProxy.py Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> * Update source/Mlos.Python/mlos/Optimizers/BayesianOptimizer.py Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> * Update source/Mlos.Python/mlos/Optimizers/unit_tests/TestBayesianOptimizer.py Co-authored-by: Adam Śmiechowski <51930563+byte-sculptor@users.noreply.github.com> * Update source/Mlos.Python/mlos/Optimizers/unit_tests/TestOptimizationProblem.py Co-authored-by: Adam Śmiechowski <51930563+byte-sculptor@users.noreply.github.com> * Update source/Mlos.Python/mlos/Grpc/BayesianOptimizerProxy.py Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> * Update source/Mlos.Python/mlos/Grpc/BayesianOptimizerProxy.py Co-authored-by: Adam Śmiechowski <51930563+byte-sculptor@users.noreply.github.com> * line breaks in context notebook * remove trailing whitespace * remove some default values, fix for new interface for predict * add todo to handle context in OptimizerMicroservice * fix error message matching * fix phrasing of errors * some day... Co-authored-by: Adam Śmiechowski <51930563+byte-sculptor@users.noreply.github.com> Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 2623df7 - Browse repository at this point
Copy the full SHA 2623df7View commit details -
Integrate ContinuousToPolynomialFeaturesHypergridAdapter into RERF su…
…rrogate model. (microsoft#182) * integrated polynomial features adapter into RERF * cleaned up some lint catches * fixed defects in ContinuousToPolynomialFeatures Adapter plus improved unit tests for this adapter * addressing some PR feedback; changes to remedy flaky tests * lowered test threshold based on new observered ranges for unexplained_variance Co-authored-by: Ed Thayer <edthaye@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for c458598 - Browse repository at this point
Copy the full SHA c458598View commit details
Commits on Nov 20, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 2353fcf - Browse repository at this point
Copy the full SHA 2353fcfView commit details -
first shot at adding ~coveralls~ codecov integration (microsoft#187)
* first shot at adding coveralls integration * coverage output as xml * use github action * use codecov not coveralls * install pytest-cov * don't comment on PRs * add badge for codecov * Update .github/workflows/main.yml Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> * Update .github/workflows/main.yml Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com> Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for f7b76f2 - Browse repository at this point
Copy the full SHA f7b76f2View commit details -
Remove quotes from a docker label (microsoft#190)
Should fix this error that recently showed up in our docker/build-push-action step: "Error: Invalid Opening Quote: a quote is found inside a field at line 2" Co-authored-by: Andreas Mueller <andreasmuellerml@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 3ef980a - Browse repository at this point
Copy the full SHA 3ef980aView commit details
Commits on Nov 23, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 9854854 - Browse repository at this point
Copy the full SHA 9854854View commit details
Commits on Nov 30, 2020
-
Merged PR 450889: More comments in Net.Core projects
More comments in Net.Core projects
Configuration menu - View commit details
-
Copy full SHA for 7ce0afe - Browse repository at this point
Copy the full SHA 7ce0afeView commit details -
Merged PR 452994: More comments in Net.Core projects
More comments in Net.Core projects.
Configuration menu - View commit details
-
Copy full SHA for b6201d7 - Browse repository at this point
Copy the full SHA b6201d7View commit details -
Merged PR 452930: Move AssemblyInitialize to new ExperimentSession class
Move AssemblyInitialize to new ExperimentSession class. This is not complete implementation of ExperimentSessionManager. Only SmartCache project is updated. Small fixes related to shared memory cleanup.
Configuration menu - View commit details
-
Copy full SHA for 68655b5 - Browse repository at this point
Copy the full SHA 68655b5View commit details -
Merged PR 456497: Annotate C++ code using SAL 2.0
Annotate C++ code using SAL 2.0
Configuration menu - View commit details
-
Copy full SHA for ca974eb - Browse repository at this point
Copy the full SHA ca974ebView commit details -
Merged PR 461654: Chanage SAL annotation (_Must_inspect_result_)
Chanage SAL annotation _Check_return_ to _Must_inspect_result_
Configuration menu - View commit details
-
Copy full SHA for e08fc96 - Browse repository at this point
Copy the full SHA e08fc96View commit details -
Merged PR 465958: Removing unused file
Removing unused file
Configuration menu - View commit details
-
Copy full SHA for a0a8d14 - Browse repository at this point
Copy the full SHA a0a8d14View commit details -
Merged PR 466228: Remove unused method declaration
Remove unused method declaration
Configuration menu - View commit details
-
Copy full SHA for 6594a1f - Browse repository at this point
Copy the full SHA 6594a1fView commit details -
Merged PR 467425: Correct CodeGen namespace
Correct CodeGen namespace
Configuration menu - View commit details
-
Copy full SHA for c05486c - Browse repository at this point
Copy the full SHA c05486cView commit details -
Merged PR 467995: Rename StaticSingle to AlignedInstance (and fix cto…
…r issues). Rename StaticSingle to AlignedInstance (and fix ctor issues). Set ResolveAssemblyWarnOrErrorOnTargetArchitectureMismatch to None to avoid warning when building in VS.
Configuration menu - View commit details
-
Copy full SHA for b60739f - Browse repository at this point
Copy the full SHA b60739fView commit details -
Merged PR 470448: Updated compiler properties
Workaround build issues in VisualStudio
Configuration menu - View commit details
-
Copy full SHA for dfc97d8 - Browse repository at this point
Copy the full SHA dfc97d8View commit details -
Merged PR 456546: Exchange FileDescriptors using Unix domain socket
Exchange FileDescriptors using Unix domain socket
Configuration menu - View commit details
-
Copy full SHA for 3735b0d - Browse repository at this point
Copy the full SHA 3735b0dView commit details -
standardize on path separator character for linux shells
and other minor cleanup
Configuration menu - View commit details
-
Copy full SHA for a351551 - Browse repository at this point
Copy the full SHA a351551View commit details -
Configuration menu - View commit details
-
Copy full SHA for c2138cb - Browse repository at this point
Copy the full SHA c2138cbView commit details -
Configuration menu - View commit details
-
Copy full SHA for d0b7e30 - Browse repository at this point
Copy the full SHA d0b7e30View commit details -
Configuration menu - View commit details
-
Copy full SHA for e3904dd - Browse repository at this point
Copy the full SHA e3904ddView commit details
Commits on Dec 1, 2020
-
Update ctest wrappers for anonymous shared memory and other fixups (m…
…icrosoft#195) Specifically, fix the dependency logic for ctest fixtures to avoid some periodic random failures when multiple optimizers tried to start at once (colliding on the use of the port) and remove unnecessary dependencies (Mlos.UnitTest didn't need any of the optimizer fixtures).
Configuration menu - View commit details
-
Copy full SHA for dd4d428 - Browse repository at this point
Copy the full SHA dd4d428View commit details
Commits on Dec 2, 2020
-
External cmake project integration (microsoft#193)
Work to make it easier/possible to integrate the C++ components of MLOS with an external C++ project (e.g. LevelDB, MySql, etc.) via `cmake`. - Fixes some issues with our own `cmake` config assumptions. - Adds examples patterns and documentation for using in an "external project" (currently just a minimal example). - Hooks those into the CI pipelines. This branch was split off of the work from microsoft#177 (since merged) to build NuGet packages for some of the C# components, so many of the original commits are shared.
Configuration menu - View commit details
-
Copy full SHA for 9eef425 - Browse repository at this point
Copy the full SHA 9eef425View commit details -
Tweaks to Python Test CI pipelines (microsoft#198)
* Tweaks to Python Test CI pipelines - Increase the timeouts on the pytest runs. There have been some timeouts recently, due in part to the addition of code coverage tracking (microsoft#187). - Separate the pylint checks (short) from the pytest runs (long, flaky) so that we can have the docker image publish task proceed in nightly runs even if the pytest run was flaky. * Avoid duplicate ctest runs during Github CI runs Fixes microsoft#196 * pass the GITHUB_WORKFLOW env var through the docker process * fixup argument order
Configuration menu - View commit details
-
Copy full SHA for 27c39e7 - Browse repository at this point
Copy the full SHA 27c39e7View commit details
Commits on Dec 3, 2020
-
Use explicit codecov token to upload results (microsoft#199)
Attempting to fix the following error: ``` -> Pinging Codecov https://codecov.io/upload/v4?package=github-action-20201130-cc6d3fe&token=secret&branch=main&commit=27c39e71a07b063cc2c59a3eb72609a722d17510&build=397032896&build_url=http%3A%2F%2Fgithub.com%2Fmicrosoft%2FMLOS%2Factions%2Fruns%2F397032896&name=&tag=&slug=microsoft%2FMLOS&service=github-actions&flags=unittests&pr=&job=&cmd_args=Q,f,n,F,Z HTTP 400 ERROR: Tokenless uploads are only supported for public repositories on GitHub Actions that can be verified through the Actions API. Please use an upload token if your repository is private and specify it via the -t flag. You can find the token for this repository at the url below on codecov.io (login required): Repo token: https://codecov.io/gh/microsoft/MLOS/settings Documentation: https://docs.codecov.io/docs/about-the-codecov-bash-uploader#section-upload-token ``` See Also: https://github.com/microsoft/MLOS/runs/1489105180?check_suite_focus=true
Configuration menu - View commit details
-
Copy full SHA for 4e3c786 - Browse repository at this point
Copy the full SHA 4e3c786View commit details
Commits on Dec 9, 2020
-
Avoid broken licenseheaders package version (microsoft#202)
Avoids the following error: ``` Traceback (most recent call last): File "/opt/hostedtoolcache/Python/3.9.0/x64/bin/licenseheaders", line 8, in <module> sys.exit(main()) File "/opt/hostedtoolcache/Python/3.9.0/x64/lib/python3.9/site-packages/licenseheaders.py", line 612, in main arguments = parse_command_line(sys.argv) File "/opt/hostedtoolcache/Python/3.9.0/x64/lib/python3.9/site-packages/licenseheaders.py", line 304, in parse_command_line known_extensions = [ftype+":"+",".join(conf["extensions"]) for ftype, conf in typeSettings.items()] File "/opt/hostedtoolcache/Python/3.9.0/x64/lib/python3.9/site-packages/licenseheaders.py", line 304, in <listcomp> known_extensions = [ftype+":"+",".join(conf["extensions"]) for ftype, conf in typeSettings.items()] KeyError: 'extensions' Error: Process completed with exit code 1. ```
Configuration menu - View commit details
-
Copy full SHA for f99d421 - Browse repository at this point
Copy the full SHA f99d421View commit details -
Fixup cake cmake config (microsoft#203)
* Make sure cmake generated makefiles target the right build type config when invoked through cake * remove unnecessary dependency on netcore build for cake builds (it's handled internally by cmake as well)
Configuration menu - View commit details
-
Copy full SHA for ac800ab - Browse repository at this point
Copy the full SHA ac800abView commit details
Commits on Dec 10, 2020
-
Avoid broken markdown lint checker action version (microsoft#205)
* avoid broken markdown-link-check action version The following upstream change adds quotes around our list of folders to check which results in passing a bad path to find: gaurav-nelson/github-action-markdown-link-check@53c1dda
Configuration menu - View commit details
-
Copy full SHA for 6407de7 - Browse repository at this point
Copy the full SHA 6407de7View commit details
Commits on Dec 11, 2020
-
2020/december/hyperspheres as objective function2 (microsoft#204)
* Started moving hyperspheres to a separate objective function * Created a Hypersphere ObjectiveFunction and the tests are passing * Linter Co-authored-by: Adam Smiechowski <adsmiech@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 8c3e91e - Browse repository at this point
Copy the full SHA 8c3e91eView commit details -
Restore use of v1 release branch for markdown-lint-check action.
Upstream now has the fix for the issue we were running into: gaurav-nelson/github-action-markdown-link-check#88 https://github.com/gaurav-nelson/github-action-markdown-link-check/releases/tag/1.0.10 https://github.com/gaurav-nelson/github-action-markdown-link-check/releases/tag/v1
Configuration menu - View commit details
-
Copy full SHA for e98e2c0 - Browse repository at this point
Copy the full SHA e98e2c0View commit details -
Parallelizing pytest (microsoft#206)
1. Invokes pytest with a `-n auto` argument to make multiple tests run in parallel (num parallel jobs = num cores) 2. Installs pytest-xdist in all our pipelines to make the above possible 3. Shortens the duration of TestOptimizerEvaluator 4. Modifies the tests that use gRPC to attempt to start the service on 100 different ports before giving up. This is the easiest way to make sure that all tests requiring gRPC can run in parallel. The alternative would be to have them all talk to a single instance, but it would turn an easy parallelization problem into a hard one as we'd need to manage the lifetime of that single instance. 5. Relaxes the check in TestSmartCacheWithRemoteOptimizerV3.py as the current one was a bit too ambitious and lead to some flakiness. 6. Puts a band-aid on test_optimization_with_context. microsoft#207 hints at a long-term fix. Co-authored-by: Adam Śmiechowski <adsmiech@microsoft.com> Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 0ef1a92 - Browse repository at this point
Copy the full SHA 0ef1a92View commit details -
Fixups long haul test (microsoft#200)
* fixup glob for running long haul tests fixes microsoft#191 * need to wrap wildcards in shell
Configuration menu - View commit details
-
Copy full SHA for 143177d - Browse repository at this point
Copy the full SHA 143177dView commit details
Commits on Dec 15, 2020
-
Implementing a multi-objective regression model (microsoft#208)
This PR introduces a multi-objective random forest model, which is essentially a collection of single-objective random forests - each predicting a different objective. This model is not yet exposed as part of the optimizer. Co-authored-by: Adam Śmiechowski <adsmiech@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 4548c31 - Browse repository at this point
Copy the full SHA 4548c31View commit details -
Configuration menu - View commit details
-
Copy full SHA for ac4e328 - Browse repository at this point
Copy the full SHA ac4e328View commit details -
Comparison of two different ways to summarize decision tree predictio…
…ns (microsoft#197) * Comparison of two different ways to summarize decision tree predictions * removed two obsolete helper functions; corrected typo in formula in intro markdown cell * modified what's considered true observation summary stats to reflect duplication of obs across leaves Co-authored-by: Ed Thayer <edthaye@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for ceb3a77 - Browse repository at this point
Copy the full SHA ceb3a77View commit details
Commits on Jan 5, 2021
-
Workaround an issue uncovered by recent versions of sphinx (microsoft…
…#211) * Workaround an issue introduced by recent versions of sphinx With sphinx 3.4.0-3.4.2 (released in late December 2020), sphinx generates asserts by the Point module due to invalid keys (e.g. __sphinx_mock__) in the Point. Exception occurred: File "/src/MLOS/source/Mlos.Python/mlos/Spaces/Point.py", line 75, in __getitem__ raise KeyError(f"This Point does not have a value along dimension: {dimension_name}") KeyError: 'This Point does not have a value along dimension: __sphinx_mock__' * A better fix to the sphinx hasattr issue * fixup: raise an appropriate error, instead of returning None https://docs.python.org/3.7/reference/datamodel.html?highlight=__getattr__#object.__getattr__
Configuration menu - View commit details
-
Copy full SHA for 94a8710 - Browse repository at this point
Copy the full SHA 94a8710View commit details -
Configuration menu - View commit details
-
Copy full SHA for 68252f0 - Browse repository at this point
Copy the full SHA 68252f0View commit details
Commits on Jan 14, 2021
-
Integrating multi objective random forest into the optimizer framework (
microsoft#212) This is the next batch of code getting us towards multi-objective optimization. PAST: Previously we have been able to register multiple objectives with the optimizer, but the optimizer wouldn't use them at all. PRESENT: With this PR the optimizer will instantiate a regression model for each of the objectives, but it doesn't yet use the predictions for anything. FUTURE: The next steps will be to write a few multi-objective utility functions so that the optimizer can use them to suggest new configurations. The chief goal of this PR is to integrate the MultiObjectiveHomogeneousRandomForest into the framework. Note that there is no change to the Optimizer's interface. The prediction of the first objective is returned, and all the others are thrown out.
Configuration menu - View commit details
-
Copy full SHA for 0161cd6 - Browse repository at this point
Copy the full SHA 0161cd6View commit details
Commits on Jan 15, 2021
-
Disable old docker package registry (microsoft#213)
* add manual CI run support * disable use of old github docker package registry
Configuration menu - View commit details
-
Copy full SHA for 108cc03 - Browse repository at this point
Copy the full SHA 108cc03View commit details
Commits on Jan 28, 2021
-
Multi-Objective Optimization (microsoft#214)
This PR enables multi-objective optimization based on multi-objective homogeneous random forest surrogate model and a probability of improvement utility function. The core of this PR is in `MultiObjectiveProbabilityOfImprovementUtilityFunction.py` which uses Monte Carlo simulation to estimate the probability that a given configuration expands the pareto frontier. To support the above functionality, we need a way of determining whether a point is dominated or not. ParetoFrontier.is_dominated(...) does that trick. One way to track optimizer's progress is to track the change in the volume of the pareto frontier. Measuring that volume precisely appears hard, so we settle for a probably approximately correct answer produced by ParetoFrontier.approximate_pareto_volume() for now. All of the other changes revolve around: 1) Integrating the new UtilityFunction into the framework. 2) Persisting the ParetoFrontier within the Optimizer (needed by the utility function). 3) Testing. 4) Small fixes. **Testing** I added new unit tests for new functionality. I also ran multi-objective optimization on 3 different problems over the weekend to compare it to random search (10 repeats for both guided and random search). Firstly, nothing crashed. Secondly, the results are very promising even without any tuning. The graphs below show the comparison between average pareto volume vs. number of iterations for across 10 runs. 2 objectives: ![image](https://user-images.githubusercontent.com/51930563/105795949-952a7500-5f42-11eb-91c7-e4887ec9267f.png) ![image](https://user-images.githubusercontent.com/51930563/105795961-9d82b000-5f42-11eb-85cb-12730e081eea.png) 4 objectives: ![image](https://user-images.githubusercontent.com/51930563/105795872-6b714e00-5f42-11eb-83e2-8532e4444eff.png) ![image](https://user-images.githubusercontent.com/51930563/105795902-7a580080-5f42-11eb-84a8-299a01278f96.png) More work will be required to: 1) Update the OptimizerEvaluator and OptimizerEvaluationReport to capture multi-objective statistics. 2) Update the gRCP interface to support multi-objective predictions 3) Visualizations, more tests and usability improvements. 4) Other utility functions. But practically, this PR enables multi-objective optimization.
Configuration menu - View commit details
-
Copy full SHA for 9f189c4 - Browse repository at this point
Copy the full SHA 9f189c4View commit details
Commits on Jan 29, 2021
-
Adding Multi-Objective Optimization to Optimizer Evaluator (microsoft…
…#216) OptimizerEvaluator now tracks pareto over time, pareto volume over time and multi-objective goodness of fit metrics. OptimizerEvaluationReport can now both write itself to disk and read itself back into memory.
Configuration menu - View commit details
-
Copy full SHA for e4ab63e - Browse repository at this point
Copy the full SHA e4ab63eView commit details
Commits on Feb 18, 2021
-
Adding a new synthetic function (microsoft#217)
Adding EnvelopedWaves and MultiObjectiveEnvelopedWaves as two new synthetic functions with many useful properties. These are sine waves enveloped by another function, either linear, quadratic or another sine wave. An enveloped sine wave produces complexity for the optimizer that allows us evaluate its behavior on non-trivial problems. Simultaneously, sine waves have the following advantages over polynomials: 1. They have well known optima - even when we envelop the function with another sine wave, as long as we keep their frequencies harmonic, we can know exactly where the optimum is. 2. They cannot be well approximated by a polynomial (Taylor expansion is accurate only locally). 3. For multi-objective problems, we can manipulate the phase shift of each objective to control the shape of the pareto frontier. The way multi-objective enveloped waves work is that we pass the same parameters through 1 or more single-objective enveloped waves functions. One useful property is that we not only know where the optima for individual functions are (maxima of sine are easy to find), but we can also know and control the shape of the pareto frontier, by controlling the phase difference between the individual objectives. For example: a phase difference of 0, means that that the objective functions are overlaid on top of each other and their optima are exactly on top of each other, so the pareto frontier is a single, optimal point Alternatively, the phase difference of quarter-period, introduces a trade-off between the objectives where y0 = sin(x) and y1 = sin(x - math.pi / 2) = -cos(x) which yields a pareto frontier in a shape of a quarter-cirle. Yet another option is to use a phase difference of math.pi. This yields a trade-off between the objectives where: y0 = sin(x) and y1 = sin(x - math.pi) = -sin(x) = -y0 which yields a pareto frontier where a gain in one objective results in an equal loss in the other objective, so the shape of that frontier is a diagonal of a square.
Configuration menu - View commit details
-
Copy full SHA for 3b1e966 - Browse repository at this point
Copy the full SHA 3b1e966View commit details -
Added Enveloped Waves to the ObjectiveFunctionFactory (microsoft#218)
This PR makes it possible to create the new objective function via the ObjectiveFunctionFactory. Existing tests cover this addition.
Configuration menu - View commit details
-
Copy full SHA for 7b882e0 - Browse repository at this point
Copy the full SHA 7b882e0View commit details -
Adding start and end timestamps to the evaluation report (microsoft#219)
This really simple change adds a start time and an end time to the optimizer evaluation report, thus allowing us to also know how long the optimization process took.
Configuration menu - View commit details
-
Copy full SHA for 0191954 - Browse repository at this point
Copy the full SHA 0191954View commit details
Commits on Feb 19, 2021
-
Created a UtilityFunctionOptimizerFactory (microsoft#220)
Adding another factory to the project. We already have a BayesianOptimizerFactory, and an ObjectiveFunctionFactory, and with this PR we add a UtiliyFunctionOptimizerFactory. Some subsequent PRs will contain the remaining UtilityFunctionFactory, and RegressionModelFactory. This will help us unify our testing, make sure that all derived classes expose the same interfaces and exhibit sufficiently similar behavior.
Configuration menu - View commit details
-
Copy full SHA for dd56ab4 - Browse repository at this point
Copy the full SHA dd56ab4View commit details
Commits on Feb 25, 2021
-
Making glow worm optimizer use the context (microsoft#222)
I was working on Random Near Incumbent Optimizer and I noticed that GlowWormSwarmOptimizer was doing something really wrong with the context features. I don't really understand how/if it ever worked with context. So this is a quick fix, where we have the worms keep track of the parameter values (not feature values) and only constructing the features dataframe when calling the utility function. This way, the glow worm swarm optimizer can suggest optimal parameters for a given context, instead of suggesting optimal parameters and context. I really don't know how it ever worked. I suspect it was never able to compute the utility value, and always bailed on the first 'if' producing a de-facto random suggestion.
Configuration menu - View commit details
-
Copy full SHA for ee252cf - Browse repository at this point
Copy the full SHA ee252cfView commit details
Commits on Mar 5, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 1f50625 - Browse repository at this point
Copy the full SHA 1f50625View commit details -
Configuration menu - View commit details
-
Copy full SHA for ad50612 - Browse repository at this point
Copy the full SHA ad50612View commit details