RFC: TensorFlow Canonical Type System

rfcs/20200211-tf-types.md

@@ -0,0 +1,293 @@
+# TensorFlow Canonical Type System
+
+| Status        | WIP       |
+:-------------- |:---------------------------------------------------- |
+| **RFC #**     | [208](https://github.com/tensorflow/community/pull/208)
+| **Author(s)** | Dan Moldovan (mdan@google.com) |
+| **Sponsor**   | Gaurav Jain (gjn@google.com)                 |
+| **Updated**   | 2020-03-04                                           |
+
+## Objective
+
+This RFC proposes a new TensorFlow module and namespace (`tf.types`) dedicated to storing implementation-free type definitions, similar to Java interfaces or C++ forward declarations. This module has no other dependencies inside TensorFlow, so any other internal module can depend on it to ensure interoperability without the risk of creating circular dependencies. These definitions can also be used by external users, for example in pytype annotations.
+The RFC focuses on the Python API, however the design should be reviewed with cross-language consistency in mind.
+
+## Motivation
+
+**Interoperability and composability**. A set of standard types that formalize an interface and decouples it from implementation ensures composability between components, especially when multiple implementations are involved.
+
+**Supports the [acyclic dependencies principle](https://en.wikipedia.org/wiki/Acyclic_dependencies_principle)**. In many instances, circular dependencies are caused between low-level complex components that need to compose (e.g. autograph needs to recognize datasets, and datasets need to use autograph). Interface extraction is a common pattern for breaking such cycles.
+
+**Supports pytype**. A set of static types that is consistent under Python’s `isinstance`/`issubclass` is required to support [PEP-484 type annotations](https://www.python.org/dev/peps/pep-0484/) in TensorFlow. This module can serve as the basis for that.
+
+**Helps formalize requirements for new APIs**. Having a formal, implementation-independent definition for things such as tensors, variables, iterables, iterators makes it easy to document and test compatibility between APIs.
+
+## User Benefit
+
+Application developers may use these canonical definitions for pytype annotations.
+
+Library developers can more easily define their API interfaces by referring to this namespace.
+
+Developers of modules internal to TensorFlow can use this module to avoid creating circular dependencies.
+
+## Design Proposal
+
+### The `tf.types` Namespace / Module
+
+All the declarations exposed under the `tf.types` namespace reside in the `python/types/*.py` module. These are [abstract base classes](https://docs.python.org/3.7/library/abc.html) with a bare minimum of method definitions and minimal or no implementation, which serve to formalize and document the contract of common types such as `Tensor`, `Variable`, etc.
+
+These definitions may be used as PEP 484 type hints, although in some cases they may be type- or shape- erased (for example, `tf.types.Tensor` may not necessarily be parametrized by `dtype` or `shape`). Note however that designs which parametrize on shape do exist, see for instance [tensorflow#31579](https://github.com/tensorflow/tensorflow/issues/31579).
+
+The type definitions are consistent with standard Python [subtyping mechanics](https://docs.python.org/3.8/library/typing.html#nominal-vs-structural-subtyping) such as instance checks or protocols (in versions prior to Python 3.8, it is difficult to simultaneously support both).
+
+### General Principles
+
+This module should not contain any implementation code. An advantage of that is that users exploring the implementation of specific types will not need to inspect this module. However, users who do not wish to inspect the code may visit the documentation of these generic types to better understand specifically what are the concrete subclasses of this type expected to do.
+
+The `tf.types` module may depend on external packages (such as `numpy`) _strictly for the purpose of defining type annotations and documentation_. No dependencies to other TensorFlow interfaces are allowed. Any dependencies on external packages which themselves depend on TensorFlow are expressly forbidden.
+
+Changes to definitions inside `tf.types` must be approved by TensorFlow leads, and typically should be accompanied by an RFC.
+
+All type declarations are based on PEP-484 and related specifications, and defined using [typing](https://docs.python.org/3/library/typing.html), with the aim of being compatible with static type checkers like [pytype](https://github.com/google/pytype), [mypy](http://mypy-lang.org/), [pyre](https://pyre-check.org/).
+
+It is recommended that internal and external type annotations, `isinstance` and `issubclass` checks use these types, eventually deprecating helpers like `tf.is_tensor`. However, concrete types continue to exist - for example, variables are instances of `tf.Variable`, which is now a subclass of `tf.types.Variable`.
+
+Class type definitions define a minimum of abstract methods and properties which are required for pytype compatibility.
+
+### Custom `Tensor` types and `tf.register_tensor_conversion_function`
+
+Custom objects can be used in standard TF operations using [tf.register_tensor_conversion_function](https://www.tensorflow.org/api_docs/python/tf/register_tensor_conversion_function). This dependency injection mechanism allows implicit casting from existing types such as list and tuple without modifying the type definition of these objects.
+
+```
+>>> class MyClass:
+...   pass
+>>> def conversion_func(value, dtype=None, name=None, as_ref=False):
+...  return tf.constant(1)
+>>> tf.register_tensor_conversion_function(MyClass, conversion_func)
+>>> obj = MyClass()
+>>> tf.convert_to_tensor(obj)
+<tf.Tensor: shape=(), dtype=int32, numpy=1>
+```
+
+However, `register_tensor_conversion_function` is not compatible with static type checking.
+
+As a side note, types which that can be converted to a [NumPy array](https://docs.scipy.org/doc/numpy/user/basics.dispatch.html#basics-dispatch) can leverage that mechanism instead, because TensorFlow supports implicit conversion from `ndarray`:
+
+```
+>>> class MyClass:
+...   def __array__(self):
+...     return np.array(1)
+>>> obj = MyClass()
+>>> tf.convert_to_tensor(obj)
+<tf.Tensor: shape=(), dtype=int32, numpy=1>
+```
+
+For true custom `Tensor` objects, we propose a protocol approach similar to NumPy’s, as alternative to `register_tensor_conversion_function`:
+
+```
+>>> class MyClass:
+...   def __tf_tensor__(self):
+...     return tf.constant(1)
+>>> obj = MyClass()
+>>> tf.convert_to_tensor(obj)
+<tf.Tensor: shape=(), dtype=int32, numpy=1>
+```
+
+Note that the mechanism above can be made compatible with static type checks using [typing.Protocol](https://www.python.org/dev/peps/pep-0544/#defining-a-protocol):
+
+```
+>>> class SupportsTensor(Protocol):
+...   def __tf_tensor__(self):
+...     pass
+>>> def f(x: SupportsTensor):
+...   pass
+>>> obj = MyClass()
+>>> f(obj)  # passes static type checks
+```
+
+Ultimately, `TensorLike` is to become a union of the protocol type along with any other types supported through legacy mechanisms:
+
+```
+TensorLike = Union[List, Tuple, tf.Tensor, SupportsTensor, ...]
+```
+
+The `Protocol` type is only standardized in Python 3.8. Backports exist through [typing_extensions](https://github.com/python/typing/tree/master/typing_extensions), although they still don’t support Python 3.5. Therefore, typing annotations will only be supported in 3.6+, and complete support is only available in 3.8+.
+
+Note that `Protocol` subtypes require [@runtime_checkable](https://www.python.org/dev/peps/pep-0544/#runtime-checkable-decorator-and-narrowing-types-by-isinstance) in order to be compatible with `isinstance`. However, that degrades the performance of `isinstance` in a way similar to `abc.ABCMeta`. For that reason, TensorFlow internal logic is encouraged to use the the more direct `hasattr` test for structural type checks of this kind.
+
+Although this RFC proposes the deprecation of `register_tensor_conversion_function`, it does not set a timeline for removing it. It remains an open question whether interim support for type annotations should be added to `register_tensor_conversion_function`.
+
+### Support for `tf.function`'s `input_signature`
+
+The type system listed here can be expanded to allow input signatures using type annotations, see for instance [this thread](https://github.com/tensorflow/tensorflow/issues/31579).
+
+Presently, the [input_signature](https://www.tensorflow.org/api_docs/python/tf/function) optional mechanism uses [tf.TensorSpec](https://www.tensorflow.org/api_docs/python/tf/TensorSpec) to describe the function arguments:
+
+```
+>>> @function(input_signature=[TensorSpec([3], dtype=int32)])
+... def f(x):
+...   tf.print(x)
+>>> f(constant([1, 2, 3]))
+[1 2 3]
+>>> f(constant([1, 2]))  # Shape mismatch
+ValueError: Python inputs incompatible with input_signature
+>>> f(constant([1.0, 2.0, 3.0]))  # DType mismatch
+ValueError: Python inputs incompatible with input_signature
+```
+
+It is expected however that some or all of this information will be repeated by the function's type annotations. Type annotations may be generic, for example by only specifying a dtype:
+
+```
+>>> @function(input_signature=[TensorSpec([3], dtype=int32)])
+... def f(x: Tensor[int32]):
+...   ...
+```
+
+In such cases, `tf.function` is expected to verify that the type annotation matches the `input_signature`.
+
+In the long term, this RFC recommends that type annotations fully replace the `input_signature`, so far as the Python type annotation system allows it:
+
+```
+>>> @function
+... def f(x: Tensor[int32, [3]]):
+...   ...
+```
+
+Internally. such type annotations will still be represented as `tf.TypeSpec` objects, ensuring backward compatbility.
+
+### Initial Type Hierarchy
+
+TensorFlow generally adopts an incremental development method. This RFC aims to remain consistent with that.
+
+Below are listed the major types presently used in TensorFlow. All types included in this list are subject to [normal compatibility rules](https://www.tensorflow.org/guide/versions), so they are unlikely to change in the future. It is therefore preferable to maintain a strict minimum of orthogonal declarations and carefully vet any additions.
+
+Most of these symbols will not be initially exported as public symbols. Only internal submodules will be able to use unexported types. The unexported types may be gradually exposed under `tf.types` or under `tf.types.experimental`.
+
+The initial type hierarchy is focused on V2 symbols. We expect to encounter places where these symbols would not be compatible with V1 code; in such cases, the V1 symbols will not be affected.
+
+#### Types created by this RFC
+
+These types will be added with the initial creation of the `tf.types` namespace.
+
+* Core tensor types
+
+  * `DType`
+  * `Shape`
+  * `Tensor` - generic dense tensor
+
+    * `Symbol` - the regular graph tensor
+    * `Value` - eager tensors
+
+  * `TensorLike` - any type that can be implicitly converted to `Tensor` (see for example https://github.com/tensorflow/addons/blob/master/tensorflow_addons/utils/types.py)
+
+  * `Variable`
+
+#### Potential types for subsequent implementation
+
+These types are raised for discussion by this RFC, but are not part of the original implementation, unless they are strictly required for consistency (to be determined during the initial submission).
+
+Many of these are expected to be required when breaking the cyclic dependencies that currently exist between submodules. However, it is hoped that opening them up for discussion early can help create a more coherent type system.
+
+* Container types
+
+  * `Composite` - low-level static structure (opaque to GraphDef/IR)
+  * `Module` - builder for structures of `Variables` (invisible to GraphDef/IR)
+  * `Optional` - basic programming construct, currently prototyped in `tf.data.experimental.Optional`; unlike `typing.Optional`, it doesn't include `None`
+  * `List` - superclass for `TensorArray`, `Queue`, etc. (opaque to GraphDef/IR)
+
+* Higher-level types
+  * `Dataset` - ETL pipeline
+  * `Iterator` - basic stateful programming construct
+  * `Iterable` - basic stateless programming construct
+  * `Function` - basic programming construct
+  * `Error` - superclass of all TF-specific errors
+
+ * Distributed types
+   * `DistributedDataset` - collective ETL
+   * `DistributedIterator` - collective iterator
+
+ * Low-level execution primitives
+   * `Graph` - GraphDef/IR program
+     * `FunctionGraph` - IR of a single concrete function
+
+#### Adding new types
+
+This module may contain public symbols, exported using `@tf_export`, and private (unexported) symbols. Private symbols are reserved exclusively for internal submodules. Only public types are subject to the normal compatibility guarantees.
+
+Private types should only be added here if more than one submodule requires them.
+
+Public types represent established, well-known types that are critical to the TensorFlow API. They may only be added with API owners approval. In general, a type should be thoroughly scrutinized before being made public. Prefer to err on the side of keeping it private, when in doubt. Ideally, new public types should be introduced using the RFC process.
+
+A good candidate for a public `tf.types` definition meets the following criteria:
+ * has at least two concrete implementations, and at least one is part of the core TensorFlow API
+ * represents a well-established programming abstraction (e.g. list, map, object)
+ * does not overlap with existing definitions
+ * is consistent with existing definitions
+ * is compatible with all applicable core APIs
+
+#### Detailed notes
+
+##### Optional
+
+`tf.types.Optional` is the [nullable type](https://en.wikipedia.org/wiki/Nullable_type) in TensorFlow.
+
+Example graph code:
+
+```
+  >>> ds = tf.data.Dataset.range(3)
+  >>> itr = iter(ds)
+  >>> opt_next = tf.data.experimental.get_next_as_optional(itr)
+  >>> tf.print(opt_next.has_value(), opt_next.get_value())
+  1 0
+```
+
+It is not fully equivalent with `typing.Optional`, as TensorFlow has no explicit type or value for `None`.
+
+### Alternatives Considered
+
+* N/A
+
+### Performance Implications
+
+* There is a potential performance concern if using `abc` for the abstract base types, which are about an order of magnitude slower for `isinstance` checks. The cost of `isinstance` may be non-negligible for eager execution or scaling to large graphs. In such cases, we may want to avoid using `abc`. See https://github.com/tensorflow/community/pull/208#discussion_r382494902.
+
+### Dependencies
+
+* None, by definition.
+
+### Engineering Impact
+
+* Engineering impact: Separate interfaces allow for faster loading times by reducing coupling between modules.
+* Maintenance: Minimal maintenance overhead since there is no functionality involved. The TensorFlow team and contributors will maintain the documentation up to date. Changes should be reviewed and approved by the TensorFlow team leads.
+
+### Platforms and Environments
+
+* Platforms: Python only, in the first stage. However, the type system should be aligned as much as possible with the core types in the TensorFlow runtime, and be language-independent as much as possible.
+* Execution environments: The type system is independent of platform. This also implies that no platform-specific types (such as `TPUTensor`) exist.
+
+### Best Practices
+
+* This set of type definitions support the acyclic dependencies principle, by requiring that implementations avoid lateral dependencies (e.g. with a linter rule).
+
+### Tutorials and Examples
+
+* As the design matures, we plan to showcase libraries that leverage this pattern.
+* Type annotations will be included in existing tutorials as definitions become final.
+
+### Compatibility
+
+* New minor version. Existing classes (`tf.Tensor`) will become subclasses of the new type interfaces.
+* Most subcomponents of TF (Lite, distributed, function, SavedModel) will depend on this new module, although their functionality is not impacted.
+* Libraries which depend on TensorFlow are encouraged to refer to `tf.types` definitions, rather than the concrete implementations for better future compatibility.
+
+### User Impact
+
+* Users will see a new `tf.types` module, that may be referenced from documentation and type annotations.
+
+
+## Questions and Discussion Topics
+
+* Whether to create entries for well-known types already exported like `tf.Tensor` and `tf.Variable`. Internally, superclasses for these types will be created, but the existing `tf.Tensor` can just point to those to avoid unnecessary duplication.
+* Single flat vs. hierarchical namespace - for example: `tf.types.distribute.Dataset`, or `tf.types.DistributedDataset`?
+* The inclusion of more specialized `Graph` types, such as `FuncGraph`, `CondBranchFuncGraph`, `WhileCondFuncGraph`, `WhileBodyFuncGraph`. It’s unclear where these should be defined, however internal submodules needs these subtypes to maintain acyclic dependencies.
+* `register_tensor_conversion_function` - should it support static type verifications (e.g. using the [ABCMeta.register](https://docs.python.org/3/library/abc.html#abc.ABCMeta.register) mechanism)?