Support Combined indexing for __getitem__ and __setitem__

[zoooo0820]

PR types

New features

PR changes

APIs

Description

Pcard-66985

Background

At present, the indexing and assignment (i.e. __getitem__ / __setitem__) of Paddle are incomplete in combined indexing (basic-indexing and advanced-indexing both appeared). This brings great trouble to build some complex models.

• Problem 1: For __getitem__ , both dynamic and static mode are not supported combined indexing.

>>> x = paddle.randn((2,3,4))
>>> x[:,[0,1]]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/paddle/fluid/dygraph/tensor_patch_methods.py", line 747, in __getitem__
    return self._getitem_index_not_tensor(item)
ValueError: (InvalidArgument) When index contains a list, its length is excepted to 1, but received 2
  [Hint: Expected size == 1, but received size:2 != 1:1.] (at /paddle/Paddle/paddle/fluid/pybind/slice_utils.h:249)

• Problem 2: For __setitem__, combined indexing forward but not backward is supported in dynamic mode; both forward and backward are not supported in static mode.

>>> x = paddle.randn((2,2))
>>> x.stop_gradient=False
>>> y = x + 1
>>> y[:,[[0]]] = 10
>>> paddle.grad(y, x)
[Tensor(shape=[2, 2], dtype=float32, place=Place(gpu:0), stop_gradient=True,
       [[1., 1.],
        [1., 1.]])]

# the correct result should be 
[Tensor(shape=[2, 2], dtype=float32, place=Place(gpu:0), stop_gradient=True,
       [[0., 1.],
        [0., 1.]])]

>>> paddle.enable_static()
>>> x = paddle.randn((2,2))
>>> x[:, [[0]]] = 10
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/framework.py", line 2387, in __setitem__
    return _setitem_impl_(self, item, value)
  File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/variable_index.py", line 684, in _setitem_impl_
    .format(item))
IndexError: Valid index accept int or slice or ellipsis or list, but received [slice(None, None, None), [[0]]].

What this PR did

1. [New Feature] Support combined indexing for both __getitem__ and __setitem__, with autograd;
2. [Bug fixes] Fix wrong output shape when slices containing a stride in dynamic mode __getitem__; e.g. If x.shape=(2,3,4), x[0, ::2, 1].shape will be [2], while currently outputs [1,2,1], incorrectly
3. [Optimize] Change the strategy for advanced-indexing in __setitem__, using index_put instead of earlier gather_nd -> elementwise_sub -> scatter_nd_add or reshape-> elementwise_mul / reduce_sum -> stack -> scatter -> reshape.
4. Modify some unit tests since these case are supported and will not raise error in the future.

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[paddle-ci-bot]

Sorry to inform you that d6f9a2c's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

[jeff41404]

LGTM

[XiaoguangHu01]

LGTM