[WIP][DRAFT] Is this the right track? Generating Dense Quantized op

[AndrewZhaoLuo]

• Add the qparam manager class Lily alluded to here

• Generate the dense layer according to math (TM) and using existing relay nodes. Have not checked for correctness of implementation but hopefully the idea is clear

• These relay generation functions return the new relay expression and the associated qparams for the output.

• The hot take is that simulated quantize and static quantize graphs are always going to be very similar the only difference is the actual internal casting between them. We try to capture this with:

  internal_accumulation_dtype: str = "float32" which represents the actual type being accumulated and

  simulated_accumulation_dtype: str = "int32" which represents the accumulated type being simulated

Things need to do / think about:
- per channel quantization
- what are the actual differences between simulated and static quantized graphs?
- organizing files in general
- a lot of other stuff...

[electriclilies]

Overall, looks pretty good to me, just a few nitpicky things :)

[electriclilies]

Passing in a name to this function is good, it will help avoid confusion.

Right now, your implementation saves every QParam ever created and adds it to a list. I'm not sure this is the exact behavior we want, since the purpose of the class saving the QParams is to let the C++ rewriter see the QParams that were created in the rewrite of the last pattern, not all the QParams. (We can get all the scales and zero point variables by doing relay.analysis.free_vars, so we don't have to store them all in a list).

However, I'm not 100% sure of the implementation details of that because it will have to interact with the FFI. For now, maybe we should just not save the QParams we've previously created, and we can add the functionality when we need it.

[electriclilies]

Units is a dense attribute, but it is not always set, so sometimes the caller will be able to pass in the units and sometimes not.

[electriclilies]

I believe that the units in dense is in_units, so the caller won't have access to out_units unless they look at the shape.

[electriclilies]

Could you change the file name to utils.py? I'm not a huge fan of common :) Also, not sure if tvm.relay.transform.quantization is the best namespace.. I personally prefertvm.relay.transform.quantize, but it's worth a discussion

[electriclilies]

One question to investigate is whether the constant folder will get rid of all the extra terms created by the non-zero zero point in the quantized inference version of the graph. If not, we may need to check the actual value of the zero point while generating the quantized relay for inference.
Not something we need to worry about right now, but it should be on our radar.