Numpy compatible diagflat operator #15468

endvroy · 2019-07-05T03:20:17Z

Description

(Brief description on what this PR is about)

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
Changes are complete (i.e. I finished coding on this PR)
All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

Feature1, tests, (and when applicable, API doc)
Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

* Add numpy namespace and initial impl of np.sum (not complete) * Clean up * Fix import error * numpy sum * add test and backward data type support * add license to test_numpy_op.py * improve test to reduce flakiness * fix sanity build * extra numeric test and imperative test * add error message for initial argument

…PIs (#14758) * Infra of new ndarray and symbol types for numpy operators * Rename * Fix import problem * Refactor * Remove redundant code * Add docstring * More on numpy ndarray and symbol * Override unimplemented methdos for ndarray and _NumpySymbol * Fix built-in methods of ndarray and _NumpySymbol * Fix test and sanity check * Fix pylint * Address cr comments * Add unit tests for ndarray and _NumpySymbol * Add _true_divide * Fix gpu build * Add future import division * More correct way of checking if an output is from a np compat op * Fix gpu build * Fix output ndarray/symbol types with at least one new ndarray/symbol * Modify true_divide doc * Fix flaky copying zero-size arrays via gpus * Fix zero size in gluon hybridize and zeros/ones symbol not creating new symbol type * Fix doc

* Numpy Dot case 1-4 + case 3.5 forward and 0.5 backward * Backward computation and test coverage

* Add np transpose More ops and namespaces for submodules Add relu and sigmoid Add reshape Fix symbolic name mismatch Add maximum and minimum * Add convenience fluent method * Add ndarray.item() * Fix CI * Fix lint * Fix lint * Fix reshape gpu * Add example * Remove python notebook outputs * Remove notebook output * Add one more example

* Refactor * Initial refactoring * Fix notebook * Move numpy op check from backend to frontend * Add homogeneous ndarray check * Fix grouping inhomogeneous types of symbols * Improve error handling of different types of symbols as outputs * Fix test * Fix numpy test * Fix ci * Try to fix gpu ci failure

* Refactor notebook * notebook working with hybrid block * More refactoring * Remove unnecessary use_np_compat * Use class decorator to initialize numpy ndarrays in parameter.py * Clear notebook outputs * Improve np decorator * Remove npe op from optimizer * Fix CI * Fix functools.wraps issue in Python2 * Fix ci

* numpy stack * migrate to use_np_shape

* Unary Ops * new version of unit tests

* Add np_array semantics for Gluon Fix notebook Fix sanity Fix gluon deferred infer shape Add np.random.uniform Add random normal Add boolean comparison ops Add np.ndarray indexing Reformat test ndarray indexing Fix unit tests Add one more test of indexing Fix sanity Enable amp test Add np.arange Revert cython unit test to ctypes Delete unnecessary use_np_shape decorator from test Rebase with numpy branch support range as index Fix python2 range type check Add argmax Disable clojure test * Fix ci * Add np.linalg.norm for ord='fro' * Fix pylint

* Fix * Fix linear regression gluon * More fix * Fix pylint * Fix for chapter 4 * Add np.add mul div mod pow sub and shuffle * Fix model selection, underfitting, overfitting * Fix weight decay * Fix dropout * Fix * Fix chapter 4

* Add np array adapter decorator for layers * Fix performance regression caused by too many conversions between nd.NDArray and np.ndarray * Fix pylint * Fix test backward compatibility issue * Fix test_lambda

* Add np op doc * Fix several issues * Add a N-D dot b 2D support * Simplify array creation api * Add swapaxes * Fix rnn gluon * More fix * Fix pylint * Delete * Fix mp windows

* numpy split * numpy split * unit test * unit test

* Add npx batch_dot and topk * Text embedding uses numpy * Fix SoftmaxCrossEntropyLoss with np * Fix sentiment cnn * Fix pylint * Fix dot attention * Fix seq2seq attention * Add np.tile * Fix transformer * Fix ci * Fix ci and rebase

* Fix parameter initializer * Add np.save and np.load * Fix read-write * Fix lint

* numpy amax * weird cu file diff * fix the unit test error * fix gpu bug * minor fix * fix lint * remove scalar value check * fix the bug on unit test * fix the case () that breaks the kernel launch * add zero dimension unit test * revert the tuple change * use mshadow maximum * remove test zero * change the macro for now * change the cuda to use mashadow op * fix the broadcast_reduce_op_value.cu wrong kernel * add more logic in shape to detect the invalid situation * change back to type swtich * change to as_nd_ndarray * add missing @npx.use_np_shape * retrigger CI * address the comment * undo algorithm import * remove the numeric gradient check

* draft of multinomial * rename to more concise name * finish shape * complete the forward function * complete forward without handle 0 dimension & scalar * handle 0 dimension * add new line * fix lint * fix the build error * fix lint * finish unit test * change the registration * make multinomial support pvals as mx.ndarray * delete newline * fix lint error * support input as list, mx.ndarray, np.ndarray & unit test * fix lint * fix the include error * fix lint * refactor & pass the tensor instead of tuple to kernel * fix lint * updata the doc * address the comment

* draft * finish linspace implementation * finish linspace * delete newline * fix pylint * add more unit test * address comment * add more test case * disable too-many-arguments * resolve confliction * add ctx

* Add np.prod * Fix ndarray.reshape accepting positional integers as arguments * Rebase * Fix rebase error * Add np.ndarray.flatten * Fix * Add broadcast_to * Add meshgrid and broadcast_arrays * Fix sin, cos, sinh, cosh not supporting scalars * Add more unary ops supporting python scalars * Fix * Fix * Fix ci * Fix sanity

* Change op name style to lower case underscore * Add ops under image to npx * Add image submodule to npx * Fix split_and_load use np * Fix fine tuning * Fix bbox and anchor * Fix odd * Fix ssd and rcnn * Remove restriction on binary element-wise scalar * Fix gan * Fix sanity * Try to fix website build failure * Add npx.random.seed * Fix doc

haojin2 · 2019-12-10T00:48:01Z

diagflat has been added in #16813, closing this PR.

haojin2 and others added 30 commits June 3, 2019 23:10

Enable np op compat check with name prefix (#14897)

355254e

[numpy] Numpy dot (#14831)

7a12654

* Numpy Dot case 1-4 + case 3.5 forward and 0.5 backward * Backward computation and test coverage

numpy-compatible mean (#14859)

4325676

Change np_compat to np_shape

6e33cc1

Temporarily disable test_amp

877e0e9

Numpy-compatible stack (#15027)

9911675

* numpy stack * migrate to use_np_shape

Numpy Unary Ops (#15010)

ac92ec3

* Unary Ops * new version of unit tests

numpy concatenate (#15104)

aa5153f

[WIP][numpy] Fix for D2L Chapters 2/3/4 (#15139)

517451b

* Fix * Fix linear regression gluon * More fix * Fix pylint * Fix for chapter 4 * Add np.add mul div mod pow sub and shuffle * Fix model selection, underfitting, overfitting * Fix weight decay * Fix dropout * Fix * Fix chapter 4

[numpy] Fix d2l performance regression (#15173)

df47457

* Add np array adapter decorator for layers * Fix performance regression caused by too many conversions between nd.NDArray and np.ndarray * Fix pylint * Fix test backward compatibility issue * Fix test_lambda

Fix (#15188)

4e1274f

fix for chapter6 conv nn (#15224)

4ad56dc

[numpy] Fix d2l chapter8 (#15237)

fb10e28

* Add np op doc * Fix several issues * Add a N-D dot b 2D support * Simplify array creation api * Add swapaxes * Fix rnn gluon * More fix * Fix pylint * Delete * Fix mp windows

fix for ch11 (#15244)

a00a2ce

Numpy-compatible split (#15049)

93679fe

* numpy split * numpy split * unit test * unit test

[numpy] Fix d2l chapter 5 (#15264)

96520cb

* Fix parameter initializer * Add np.save and np.load * Fix read-write * Fix lint

Numpy compatible linspace (#15256)

2260667

* draft * finish linspace implementation * finish linspace * delete newline * fix pylint * add more unit test * address comment * add more test case * disable too-many-arguments * resolve confliction * add ctx

numpy-compatible cumsum (#15309)

f803dde

add doc for multinomial, dot, cumsum, clip, abs, exp, arctan (#15386)

4dfb7b9

endvroy added 21 commits July 15, 2019 11:00

added safeguard for zero-size arrays

0bdbe91

added check against array access overflow

fff7d09

added tests for forward

230d322

fixed array access error

9598847

fixed bugs in test

b9c8993

fixed bug in shape dim check

0165505

fixed bug in out-of-bound checking

b6f31c9

fixed bug in out-of-bound checking

87054c7

backward completed and tested

470ae2e

fixed bug in backward

00dde0d

added gpu support

3737f0a

reformatted

cfbf7f8

removed unused var

226d920

fixed ambiguous call of div in CI

22cd5ed

fixed bug on gpu

38bee62

several code style fixes

0c72766

fixed sanity problem of readability

16c7f3c

added doc for diagflat

9f0b54a

moved diagflat to init_op

f0dba73

removed redundant blank lines

d94d664

merged np_init.h

df05a4f

reminisce force-pushed the numpy branch from d8d6b3b to 7da1a11 Compare July 17, 2019 08:51

haojin2 force-pushed the numpy branch 4 times, most recently from a23120e to 07d7a94 Compare July 24, 2019 18:59

haojin2 force-pushed the numpy branch from 07d7a94 to 47f4cd3 Compare July 26, 2019 18:35

reminisce force-pushed the numpy branch from 47f4cd3 to 63bbb6c Compare August 2, 2019 08:27

haojin2 closed this Dec 10, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Numpy compatible diagflat operator #15468

Numpy compatible diagflat operator #15468

endvroy commented Jul 5, 2019

haojin2 commented Dec 10, 2019

Numpy compatible diagflat operator #15468

Numpy compatible diagflat operator #15468

Conversation

endvroy commented Jul 5, 2019

Description

Checklist

Essentials

Changes

Comments

haojin2 commented Dec 10, 2019