[NODE][REFLECTION] Support NDArray as field

[tqchen]

Support NDArray as a serializable and reflectable field of Node system. This is going to be useful to introduce constant node in the language.

Example Testcase

In C++

struct NDArrayWrapperNode : public ::tvm::Node {
  std::string name;
  tvm::runtime::NDArray array;

  void VisitAttrs(tvm::AttrVisitor* v) final {
    v->Visit("name", &name);
    v->Visit("array", &array);
  }

  static constexpr const char* _type_key = "NDArrayWrapper";
  TVM_DECLARE_NODE_TYPE_INFO(NDArrayWrapperNode, Node);
};

TVM_DEFINE_NODE_REF(NDArrayWrapper, NDArrayWrapperNode);

def test_ndarray_reflection():
    x = np.random.uniform(size=(10, 2)).astype("float32")
    xx = tvm.nd.array(x)
    xnode = tvm.make.node("NDArrayWrapper", name="xx", array=xx)
    xnode2 = tvm.make.node("NDArrayWrapper", name="x2", array=xx)
    assert xnode.array.same_as(xx)
    json_str = tvm.save_json([xnode, xnode2])
    json_dict = json.loads(json_str)
    b64_str = json_dict["b64ndarrays"][0]
    decoded = py_str(base64.b64encode(base64.b64decode(b64_str)))
    assert b64_str == decoded
    xlist = tvm.load_json(json_str)
    np.testing.assert_equal(xlist[0].array.asnumpy(), xx.asnumpy())
    assert xlist[1].array == xlist[0].array

[tqchen]

@jroesch can you review?

[jroesch]

Modulo a few nitpicks looks good to me, did you write the base64 encoding code yourself?

[jroesch]

Could we maybe expand on this note, we expect that after deserialization the user will reallocate the data with the appropriate device type and id?

[jroesch]

What is the purpose of this struct?

[jroesch]

Are these wrappers just so we get a 2-tuple like piece of data in Python?

[tqchen]

Yes, these are wrappers to get 2-tuple pieces of data, we put it in nnvm so they can be deprecated in future. Used to test the features introduced

[jroesch]

Could we capitalize this?

[jroesch]

👍

[tqchen]

@jroesch I have addressed your comments, please check again. base64 encoding was adapted from code that I wrote in previous projects, so we can keep things self-contained. The NDArrayWrapper is mainly used to test the introduced feature. It also simplifies the previous hack in nnvm.

We can phase it out later(that is why i didn't put it in tvm)

[jroesch]

👍 everything looked good, sorry took me a while to get back to it, didn't check again until now.