I am running AIMET APIs for SSVD compression using Pytorch. If I save the compressed model, and load the model again using torch.load, the model output quality is not good at all. Is it because the compression has changed the model architecture and hence we cannot use original model architecture to load? What is the standard practice for this scenario?
@amitdedhia - Sorry for the late response
Was the model accuracy good before saving? In general, after compression, depending on the compression ratio chosen, there might be a sharp drop in accuracy. You would need to fine-tune the model (15 epochs or so) to recover back the accuracy.
@akhobare yes, the compression accuracy was good before saving. I did finetune the model as well.
Are you still facing the same issue? Different models have different levels of compressibility - meaning some models like VGG16 can be compressed by 1/2 without losing too much accuracy post-fine-tuning, while others like Resnet18 are more sensitive and can be compressed less than that given the same accuracy drop target.
What model are you trying to compress? And how much compression ratio are you targeting?