Hi,
which data types (e.g. uint8, int8, int16) does AIMET support as target for the quantization and where can information on this be found in the documentation?
I would like to quantize a model using uint8 as the target data type. Can this be done with AIMET?
Hi @schmittsy - For a simulation tool like AIMET the actual data type of the quantized values is not important. Perhaps your intention was to ask if AIMET supports both symmetric (usually associated with int8) and asymmetric (usually associated with uint8) quantization. If so, then the answer is yes.
To specify symmetric quantization, you would specify this in the “runtime config file”. E.g. here is the default config file: aimet/default_config.json at develop · quic/aimet (github.com)
If you see that file and look for “is_symmetric”, you can see that it is enabled. So just use this config file when creating the QuantizationSimModel. See API docs here: AIMET PyTorch Quantization SIM API — AI Model Efficiency Toolkit Documentation: ver tf-torch-cpu_1.22.2 (quic.github.io)
Look for the parameter called config_file.
Let me know if you need more clarifications.
Hi @akhobare,
Thanks a lot for your answer.
Yes, my intention was to ask if it is possible to quantize a model using asymmetric quantization (uint8 required by the target hardware) using AIMET. Sorry that this was not clear from my question.
I will check out the files you mentioned.