AI Model Efficiency Toolkit (AIMET) Forum

Combination of Adaround, Cross layer equalization and QAT

Dear AIMET researchers,

In the provided materials, it suggested not to do bias correction after adaround. I am wondering is it possible to combine cross layer equalization with adaround and QAT? If possible, what is the recommended order? Thank you for your attention!

Best Regards,
Edan

Hi @edan840216 We can combine CLE and AdaRound techniques. It is recommended we apply CLE with bias correction, then apply AdaRound. And, if the PTQ is not good enough after these are applied, take the best PTQ model as init and perform QAT. Please let us know if you have further questions.

Hi @quic_ssiddego, thank you for your reply. We will try this pipeline!

Best Regards,
Edan

1 Like