Combination of Adaround, Cross layer equalization and QAT

1

Dear AIMET researchers,

In the provided materials, it suggested not to do bias correction after adaround. I am wondering is it possible to combine cross layer equalization with adaround and QAT? If possible, what is the recommended order? Thank you for your attention!

Best Regards,
Edan

2

Hi @edan840216 We can combine CLE and AdaRound techniques. It is recommended we apply CLE with bias correction, then apply AdaRound. And, if the PTQ is not good enough after these are applied, take the best PTQ model as init and perform QAT. Please let us know if you have further questions.

3

Hi @quic_ssiddego, thank you for your reply. We will try this pipeline!

Best Regards,
Edan

1 Like