AI Model Efficiency Toolkit (AIMET) Forum

Exception while running channel pruning using Pytorch

I have installed AIMET on google colab. I am trying to evaluate AIMET using Pytorch APIs. I could successfully run the SSVD compression. However on the same environment, the channel pruning API throws error. Can anyone help whats going wrong here?

Exception details:

2020-06-26 05:59:22,290 - CompRatioSelect - INFO - Analyzing compression ratio: 0.1 =====================>
/usr/local/lib/python3.6/dist-packages/aimet_torch/winnow/mask_propagation_winnower.py:96: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
dummy_input = torch.tensor(dummy_input).cuda() # pylint: disable=not-callable
/usr/local/lib/python3.6/dist-packages/torch/tensor.py:435: RuntimeWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won’t change the number of iterations executed (and might lead to errors or silently give incorrect results).
‘incorrect results).’, category=RuntimeWarning)

RuntimeError Traceback (most recent call last)
in ()
19 compress_scheme=CompressionScheme.channel_pruning,
20 cost_metric=CostMetric.mac,
—> 21 parameters=channel_pruning_params)

14 frames
/usr/local/lib/python3.6/dist-packages/aimet_torch/compress.py in compress_model(model, eval_callback, eval_iterations, input_shape, compress_scheme, cost_metric, parameters, trainer, visualization_url)
111 raise ValueError(“Compression scheme not supported: {}”.format(compress_scheme))
112
–> 113 compressed_layer_db, stats = algo.compress_model(cost_metric, trainer)
114 return compressed_layer_db.model, stats

/usr/local/lib/python3.6/dist-packages/aimet_common/compression_algo.py in compress_model(self, cost_metric, trainer)
86
87 # Find optimal compression ratios for each layer
—> 88 layer_comp_ratio_list, stats = self._comp_ratio_select_algo.select_per_layer_comp_ratios()
89
90 self._pickle_comp_ratio_list(layer_comp_ratio_list)

/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in select_per_layer_comp_ratios(self)
221
222 # Compute eval scores for each candidate comp-ratio in each layer
–> 223 eval_scores_dict = self._construct_eval_dict()
224
225 # Fit the scores to a monotonically increasing function

/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in _construct_eval_dict(self)
212 else:
213 # Create the eval scores dictionary
–> 214 eval_scores_dict = self._compute_eval_scores_for_all_comp_ratio_candidates()
215
216 # save the dictionary to file (in case the user wants to reuse the dictionary in the future)

/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in _compute_eval_scores_for_all_comp_ratio_candidates(self)
398
399 layer_wise_eval_scores = self._compute_layerwise_eval_score_per_comp_ratio_candidate(data_table,
–> 400 progress_bar, layer)
401 eval_scores_dict[layer.name] = layer_wise_eval_scores
402

/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in _compute_layerwise_eval_score_per_comp_ratio_candidate(self, tabular_progress_object, progress_bar, layer)
427 [LayerCompRatioPair(layer, comp_ratio)],
428 self._cost_metric,
–> 429 trainer=None)
430
431 eval_score = self._eval_func(pruned_layer_db.model, self._eval_iter, use_cuda=self._is_cuda)

/usr/local/lib/python3.6/dist-packages/aimet_torch/channel_pruning/channel_pruner.py in prune_model(self, layer_db, layer_comp_ratio_list, cost_metric, trainer)
249 # call the base class method
250 comp_layer_db = Pruner.prune_model(self, layer_db,
–> 251 layer_comp_ratio_list, cost_metric, trainer)
252
253 return comp_layer_db

/usr/local/lib/python3.6/dist-packages/aimet_common/pruner.py in prune_model(self, layer_db, layer_comp_ratio_list, cost_metric, trainer)
73
74 if comp_ratio is not None and comp_ratio < 1.0:
—> 75 self._prune_layer(layer_db, comp_layer_db, layer, comp_ratio, cost_metric)
76
77 # fine-tuning the layer while creating the final model

/usr/local/lib/python3.6/dist-packages/aimet_torch/channel_pruning/channel_pruner.py in _prune_layer(self, orig_layer_db, comp_layer_db, layer, comp_ratio, cost_metric)
216 def _prune_layer(self, orig_layer_db: LayerDatabase, comp_layer_db: LayerDatabase,
217 layer: Layer, comp_ratio: float, cost_metric: CostMetric):
–> 218 self._winnow_and_reconstruct_layer(orig_layer_db, comp_layer_db, layer, comp_ratio, True)
219
220 def calculate_compressed_cost(self, layer_db: LayerDatabase,

/usr/local/lib/python3.6/dist-packages/aimet_torch/channel_pruning/channel_pruner.py in _winnow_and_reconstruct_layer(self, orig_layer_db, comp_layer_db, layer, comp_ratio, perform_reconstruction)
178 [(layer.module, prune_indices)],
179 reshape=self._allow_custom_downsample_ops,
–> 180 in_place=True)
181
182 # 3) data sub sampling and reconstruction

/usr/local/lib/python3.6/dist-packages/aimet_torch/winnow/winnow.py in winnow_model(model, input_shape, list_of_modules_to_winnow, reshape, in_place, verbose)
68
69 mask_winnower = MaskPropagationWinnower(model, input_shape, list_of_modules_to_winnow, reshape,
—> 70 in_place, verbose)
71 new_model, ordered_modules_list = mask_winnower.propagate_masks_and_winnow()
72

/usr/local/lib/python3.6/dist-packages/aimet_torch/winnow/mask_propagation_winnower.py in init(self, model, input_shape, list_of_modules_to_winnow, reshape, in_place, verbose)
96 dummy_input = torch.tensor(dummy_input).cuda() # pylint: disable=not-callable
97
—> 98 self._graph = ConnectedGraph(self._model, (dummy_input,))
99 self.list_of_modules_to_winnow_with_names =
100 generate_and_add_module_winnow_list_with_names(model, self._list_of_modules_to_winnow)

/usr/local/lib/python3.6/dist-packages/aimet_torch/meta/connectedgraph.py in init(self, model, model_input)
200
201 self._generate_module_lookup_table(model)
–> 202 self._construct_graph(model, model_input)
203
204 # Map torch module types to normalized names to provide backward compatibility to

/usr/local/lib/python3.6/dist-packages/aimet_torch/meta/connectedgraph.py in _construct_graph(self, model, model_input)
303 “”"
304 if torch.version == ‘1.1.0’:
–> 305 trace = torch.jit.trace(model, model_input)
306 # Parse trace code to create ops and products
307 self._parse_trace_code(trace.code)

/usr/local/lib/python3.6/dist-packages/torch/jit/init.py in trace(func, example_inputs, optimize, check_trace, check_inputs, check_tolerance, _force_outplace, _module_class)
686 traced = _module_class(func, **executor_options)
687 traced._c._create_method_from_trace(‘forward’, func, example_inputs,
–> 688 var_lookup_fn, _force_outplace)
689 else:
690 name = getattr(func, ‘name’, ‘forward’)

RuntimeError: Only tensors or tuples of tensors can be output from traced functions (getNestedOutputTrace at /pytorch/torch/csrc/jit/tracer.cpp:200)
frame #0: std::function<std::string ()>::operator()() const + 0x11 (0x7fa40370c441 in /usr/local/lib/python3.6/dist-packages/torch/lib/libc10.so)
frame #1: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x2a (0x7fa40370bd7a in /usr/local/lib/python3.6/dist-packages/torch/lib/libc10.so)
frame #2: torch::jit::tracer::getNestedOutputTrace(std::shared_ptrtorch::jit::tracer::TracingState const&, c10::IValue const&) + 0x20d (0x7fa4029f173d in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch.so.1)
frame #3: torch::jit::tracer::exit(std::vector<c10::IValue, std::allocatorc10::IValue > const&) + 0x2f (0x7fa4029f180f in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch.so.1)
frame #4: + 0x447c13 (0x7fa44bfbdc13 in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so)
frame #5: + 0x459e92 (0x7fa44bfcfe92 in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so)
frame #6: + 0x130cfc (0x7fa44bca6cfc in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so)
frame #7: _PyCFunction_FastCallDict + 0x35c (0x5669ac in /usr/bin/python3)
frame #8: /usr/bin/python3() [0x50a5c3]
frame #9: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #10: /usr/bin/python3() [0x507d64]
frame #11: /usr/bin/python3() [0x509a90]
frame #12: /usr/bin/python3() [0x50a48d]
frame #13: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #14: /usr/bin/python3() [0x509758]
frame #15: /usr/bin/python3() [0x50a48d]
frame #16: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #17: /usr/bin/python3() [0x507d64]
frame #18: _PyFunction_FastCallDict + 0x2e2 (0x509042 in /usr/bin/python3)
frame #19: /usr/bin/python3() [0x594931]
frame #20: /usr/bin/python3() [0x549e5f]
frame #21: /usr/bin/python3() [0x5513d1]
frame #22: _PyObject_FastCallKeywords + 0x19c (0x5a9cbc in /usr/bin/python3)
frame #23: /usr/bin/python3() [0x50a5c3]
frame #24: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #25: /usr/bin/python3() [0x507d64]
frame #26: _PyFunction_FastCallDict + 0x357 (0x5090b7 in /usr/bin/python3)
frame #27: /usr/bin/python3() [0x594931]
frame #28: /usr/bin/python3() [0x549e5f]
frame #29: /usr/bin/python3() [0x5513d1]
frame #30: _PyObject_FastCallKeywords + 0x19c (0x5a9cbc in /usr/bin/python3)
frame #31: /usr/bin/python3() [0x50a5c3]
frame #32: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #33: /usr/bin/python3() [0x507d64]
frame #34: /usr/bin/python3() [0x509a90]
frame #35: /usr/bin/python3() [0x50a48d]
frame #36: _PyEval_EvalFrameDefault + 0x1226 (0x50cd96 in /usr/bin/python3)
frame #37: /usr/bin/python3() [0x509758]
frame #38: /usr/bin/python3() [0x50a48d]
frame #39: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #40: /usr/bin/python3() [0x509758]
frame #41: /usr/bin/python3() [0x50a48d]
frame #42: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #43: /usr/bin/python3() [0x509758]
frame #44: /usr/bin/python3() [0x50a48d]
frame #45: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #46: /usr/bin/python3() [0x507d64]
frame #47: /usr/bin/python3() [0x509a90]
frame #48: /usr/bin/python3() [0x50a48d]
frame #49: _PyEval_EvalFrameDefault + 0x1226 (0x50cd96 in /usr/bin/python3)
frame #50: /usr/bin/python3() [0x509758]
frame #51: /usr/bin/python3() [0x50a48d]
frame #52: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #53: /usr/bin/python3() [0x509758]
frame #54: /usr/bin/python3() [0x50a48d]
frame #55: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #56: /usr/bin/python3() [0x509758]
frame #57: /usr/bin/python3() [0x50a48d]
frame #58: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #59: /usr/bin/python3() [0x509758]
frame #60: /usr/bin/python3() [0x50a48d]
frame #61: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #62: /usr/bin/python3() [0x509758]
frame #63: /usr/bin/python3() [0x50a48d]

Interesting. Thanks for reporting @amitdedhia.

@quic_sundarr, anything jumps out at you from the stack trace?

@amitdedhia,

Before using AIMET for Channel Pruning, could you please do torch.onnx export of your model with verbose=True and let us know the result.

Sample code:
torch.onnx.export(your_model, <model_input_tensor>, “<model_name.onnx”, verbose=True)

Reference:
https://pytorch.org/docs/1.1.0/onnx.html?highlight=onnx#module-torch.onnx

Thanks.

@quic_sundarr
Sorry for delayed response. I was busy with other priority stuff.

To help you, I tried to install AIMET on google colab. However, I am getting error while installing. I was able to successfully install AIMET on google colab about a month back. So not sure what did I do wrong. Can you help me please on this?

The error is:
CMake Error at TrainingExtensions/torch/cmake_install.cmake:73 (file):
file INSTALL cannot find “/content/build/artifacts/site.py”.

I am attaching full trace. The error occurs while running the last installation step i.e. “make install” command.

Thank you…

Similar problem is faced by others too. This worked for me while building inside a docker container: https://github.com/quic/aimet/issues/108#issuecomment-655196783

Thanks @ishan This worked :+1:

@quic_sundarr
I tried with a different model. The above exception has gone. The channel pruning api ran well for quite long. However it failed again mid way. Here is the exception

RuntimeError: Given transposed=1, weight of size 1193 96 4 4, expected input[16, 1280, 10, 10] to have 1193 channels, but got 1280 channels instead

The below is more detail:

    [Dataset] Number of sample pairs: 70
2020-08-03 09:34:27,084 - CompRatioSelect - INFO - Layer backbone.features.18.0.1, comp_ratio 0.800000 ==> eval_score=0.012582
2020-08-03 09:34:27,085 - CompRatioSelect - INFO - Analyzing compression ratio: 0.9 =====================>
2020-08-03 09:34:33,991 - ChannelPruning - INFO - finished linear regression fit 
[Dataset] Checking file paths...
[Dataset] Number of sample pairs: 70
2020-08-03 09:34:39,162 - CompRatioSelect - INFO - Layer backbone.features.18.0.1, comp_ratio 0.900000 ==> eval_score=0.013433
2020-08-03 09:34:39,164 - CompRatioSelect - INFO - Analyzing compression ratio: 0.1 =====================>
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-65-7a498fadb2e5> in <module>()
     27                                                    compress_scheme=CompressionScheme.channel_pruning,
     28                                                    cost_metric=CostMetric.mac,
---> 29                                                    parameters=channel_pruning_params)

18 frames
/usr/local/lib/python3.6/dist-packages/aimet_torch/compress.py in compress_model(model, eval_callback, eval_iterations, input_shape, compress_scheme, cost_metric, parameters, trainer, visualization_url)
    111             raise ValueError("Compression scheme not supported: {}".format(compress_scheme))
    112 
--> 113         compressed_layer_db, stats = algo.compress_model(cost_metric, trainer)
    114         return compressed_layer_db.model, stats

/usr/local/lib/python3.6/dist-packages/aimet_common/compression_algo.py in compress_model(self, cost_metric, trainer)
     86 
     87         # Find optimal compression ratios for each layer
---> 88         layer_comp_ratio_list, stats = self._comp_ratio_select_algo.select_per_layer_comp_ratios()
     89 
     90         self._pickle_comp_ratio_list(layer_comp_ratio_list)

/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in select_per_layer_comp_ratios(self)
    221 
    222         # Compute eval scores for each candidate comp-ratio in each layer
--> 223         eval_scores_dict = self._construct_eval_dict()
    224 
    225         # Fit the scores to a monotonically increasing function

/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in _construct_eval_dict(self)
    212         else:
    213             # Create the eval scores dictionary
--> 214             eval_scores_dict = self._compute_eval_scores_for_all_comp_ratio_candidates()
    215 
    216             # save the dictionary to file (in case the user wants to reuse the dictionary in the future)

/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in _compute_eval_scores_for_all_comp_ratio_candidates(self)
    398 
    399             layer_wise_eval_scores = self._compute_layerwise_eval_score_per_comp_ratio_candidate(data_table,
--> 400                                                                                                  progress_bar, layer)
    401             eval_scores_dict[layer.name] = layer_wise_eval_scores
    402 

/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in _compute_layerwise_eval_score_per_comp_ratio_candidate(self, tabular_progress_object, progress_bar, layer)
    427                                                        [LayerCompRatioPair(layer, comp_ratio)],
    428                                                        self._cost_metric,
--> 429                                                        trainer=None)
    430 
    431             eval_score = self._eval_func(pruned_layer_db.model, self._eval_iter, use_cuda=self._is_cuda)

/usr/local/lib/python3.6/dist-packages/aimet_torch/channel_pruning/channel_pruner.py in prune_model(self, layer_db, layer_comp_ratio_list, cost_metric, trainer)
    249         # call the base class method
    250         comp_layer_db = Pruner.prune_model(self, layer_db,
--> 251                                            layer_comp_ratio_list, cost_metric, trainer)
    252 
    253         return comp_layer_db

/usr/local/lib/python3.6/dist-packages/aimet_common/pruner.py in prune_model(self, layer_db, layer_comp_ratio_list, cost_metric, trainer)
     73 
     74             if comp_ratio is not None and comp_ratio < 1.0:
---> 75                 self._prune_layer(layer_db, comp_layer_db, layer, comp_ratio, cost_metric)
     76 
     77             # fine-tuning the layer while creating the final model

/usr/local/lib/python3.6/dist-packages/aimet_torch/channel_pruning/channel_pruner.py in _prune_layer(self, orig_layer_db, comp_layer_db, layer, comp_ratio, cost_metric)
    216     def _prune_layer(self, orig_layer_db: LayerDatabase, comp_layer_db: LayerDatabase,
    217                      layer: Layer, comp_ratio: float, cost_metric: CostMetric):
--> 218         self._winnow_and_reconstruct_layer(orig_layer_db, comp_layer_db, layer, comp_ratio, True)
    219 
    220     def calculate_compressed_cost(self, layer_db: LayerDatabase,

/usr/local/lib/python3.6/dist-packages/aimet_torch/channel_pruning/channel_pruner.py in _winnow_and_reconstruct_layer(self, orig_layer_db, comp_layer_db, layer, comp_ratio, perform_reconstruction)
    185             orig_layer = orig_layer_db.find_layer_by_name(layer.name)
    186             self._data_subsample_and_reconstruction(orig_layer.module, layer.module, orig_layer_db.model,
--> 187                                                     comp_layer_db.model)
    188 
    189         # 4) update layer database

/usr/local/lib/python3.6/dist-packages/aimet_torch/channel_pruning/channel_pruner.py in _data_subsample_and_reconstruction(self, orig_layer, pruned_layer, orig_model, comp_model)
    115         """
    116         inp_data, out_data = DataSubSampler.get_sub_sampled_data(orig_layer, pruned_layer, orig_model, comp_model,
--> 117                                                                  self._data_loader, self._num_reconstruction_samples)
    118 
    119         WeightReconstructor.reconstruct_params_for_conv2d(pruned_layer, inp_data, out_data)

/usr/local/lib/python3.6/dist-packages/aimet_torch/data_subsampler.py in get_sub_sampled_data(cls, orig_layer, pruned_layer, orig_model, comp_model, data_loader, num_reconstruction_samples)
    276 
    277             DataSubSampler._forward_pass(orig_model, batch)
--> 278             DataSubSampler._forward_pass(comp_model, batch)
    279 
    280             input_data = np.vstack(pruned_layer_inp_data)

/usr/local/lib/python3.6/dist-packages/aimet_torch/data_subsampler.py in _forward_pass(model, batch)
    181         try:
    182             with torch.no_grad():
--> 183                 _ = model(*batch)
    184         except StopForwardException:
    185             pass

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    491             result = self._slow_forward(*input, **kwargs)
    492         else:
--> 493             result = self.forward(*input, **kwargs)
    494         for hook in self._forward_hooks.values():
    495             hook_result = hook(self, input, result)

/content/Human-Segmentation-PyTorch/models/UNet.py in forward(self, input)
    108         def forward(self, input):
    109                 x1, x2, x3, x4, x5 = self._run_backbone(input)
--> 110                 x = self.decoder1(x5, x4)
    111                 x = self.decoder2(x, x3)
    112                 x = self.decoder3(x, x2)

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    491             result = self._slow_forward(*input, **kwargs)
    492         else:
--> 493             result = self.forward(*input, **kwargs)
    494         for hook in self._forward_hooks.values():
    495             hook_result = hook(self, input, result)

/content/Human-Segmentation-PyTorch/models/UNet.py in forward(self, input, shortcut)
     21 
     22         def forward(self, input, shortcut):
---> 23                 x = self.deconv(input)
     24                 x = torch.cat([x, shortcut], dim=1)
     25                 x = self.block_unit(x)

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    491             result = self._slow_forward(*input, **kwargs)
    492         else:
--> 493             result = self.forward(*input, **kwargs)
    494         for hook in self._forward_hooks.values():
    495             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/conv.py in forward(self, input, output_size)
    794         return F.conv_transpose2d(
    795             input, self.weight, self.bias, self.stride, self.padding,
--> 796             output_padding, self.groups, self.dilation)
    797 
    798 

RuntimeError: Given transposed=1, weight of size 1193 96 4 4, expected input[16, 1280, 10, 10] to have 1193 channels, but got 1280 channels instead

I first applied SSVD compression on my uncompressed model. Then applied channel pruning. The input model to channel pruning (i.e. the SSVD compressed model) is as follows.

Sequential(
  (0): Sequential(
    (0): Sequential(
      (0): Conv2d(3, 1, kernel_size=(3, 1), stride=(2, 1), padding=(1, 0), bias=False)
      (1): Conv2d(1, 32, kernel_size=(1, 3), stride=(1, 2), padding=(0, 1), bias=False)
    )
    (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU6(inplace)
  )
  (1): InvertedResidual(
    (conv): Sequential(
      (0): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32, bias=False)
      (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Sequential(
        (0): Conv2d(32, 1, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(1, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (4): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (2): InvertedResidual(
    (conv): Sequential(
      (0): Sequential(
        (0): Conv2d(16, 1, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(1, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(96, 96, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=96, bias=False)
      (4): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(96, 1, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(1, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (3): InvertedResidual(
    (conv): Sequential(
      (0): Sequential(
        (0): Conv2d(24, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(2, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(144, 144, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=144, bias=False)
      (4): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(144, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(2, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (4): InvertedResidual(
    (conv): Sequential(
      (0): Conv2d(24, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(144, 144, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=144, bias=False)
      (4): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(144, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(2, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (5): InvertedResidual(
    (conv): Sequential(
      (0): Sequential(
        (0): Conv2d(32, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(2, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192, bias=False)
      (4): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(192, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(2, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (6): InvertedResidual(
    (conv): Sequential(
      (0): Sequential(
        (0): Conv2d(32, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(2, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192, bias=False)
      (4): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(192, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(2, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (7): InvertedResidual(
    (conv): Sequential(
      (0): Sequential(
        (0): Conv2d(32, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(2, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(192, 192, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=192, bias=False)
      (4): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(192, 4, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(4, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (8): InvertedResidual(
    (conv): Sequential(
      (0): Sequential(
        (0): Conv2d(64, 5, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(5, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
      (4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(384, 5, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(5, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (9): InvertedResidual(
    (conv): Sequential(
      (0): Sequential(
        (0): Conv2d(64, 5, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(5, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
      (4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(384, 5, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(5, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (10): InvertedResidual(
    (conv): Sequential(
      (0): Sequential(
        (0): Conv2d(64, 5, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(5, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
      (4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(384, 5, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(5, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (11): InvertedResidual(
    (conv): Sequential(
      (0): Sequential(
        (0): Conv2d(64, 5, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(5, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
      (4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(384, 7, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(7, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (12): InvertedResidual(
    (conv): Sequential(
      (0): Sequential(
        (0): Conv2d(96, 8, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(8, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576, bias=False)
      (4): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(576, 8, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(8, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (13): InvertedResidual(
    (conv): Sequential(
      (0): Sequential(
        (0): Conv2d(96, 8, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(8, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576, bias=False)
      (4): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(576, 8, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(8, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (14): InvertedResidual(
    (conv): Sequential(
      (0): Sequential(
        (0): Conv2d(96, 8, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(8, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(576, 576, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=576, bias=False)
      (4): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(576, 12, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(12, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (15): InvertedResidual(
    (conv): Sequential(
      (0): Sequential(
        (0): Conv2d(160, 13, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(13, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960, bias=False)
      (4): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(960, 13, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(13, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (16): InvertedResidual(
    (conv): Sequential(
      (0): Conv2d(160, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960, bias=False)
      (4): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(960, 13, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(13, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (17): InvertedResidual(
    (conv): Sequential(
      (0): Sequential(
        (0): Conv2d(160, 13, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(13, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU6(inplace)
      (3): Conv2d(960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960, bias=False)
      (4): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU6(inplace)
      (6): Sequential(
        (0): Conv2d(960, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): Conv2d(24, 320, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
      (7): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (18): Sequential(
    (0): Sequential(
      (0): Conv2d(320, 25, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (1): Conv2d(25, 1280, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (1): BatchNorm2d(1280, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU6(inplace)
  )
)

So what could be the issue here?
Let me know if you need any more details from me …

@amitdedhia

#1
To narrow down the problem, did you just try Channel Pruning on this model without doing SSVD?
If the problem still exists, we can focus on channel prubing.

#2
Is it possible for you to share the complete model?
Very often, these issues happen because of how the forward() functions of a model is written.
Without looking at the forward() functions it is difficult to debug or analyze.

Thanks

@quic_sundarr

I have shared the model files in another thread here: Exception while exporting Quantized model

Can you pick from it? - Thanks

I will try just Channel Pruning later (soon) and let you know the results

@amitdedhia Thanks for sharing the model. I looked at the model definition. So, the model you are testing is UNet and you used MobileNetV2 as the backbone. Correct? I will take a detailed look and update.

@amitdedhia, The UNet model’s forward() has the following line:

x = F.interpolate(x, size=input.shape[-2:], mode='bilinear', align_corners=True)

My first suspicion is that interpolate() may be causing the ONNX export error (other thread) as well as the Channel Pruning error. Please replace the interpolate() with an equivalent operation and try ONNX export. If ONNX export works successfully, then you can try the AIMET features.

@quic_sundarr Thank you for your response.

Instead of interpolate, can I use F.Upsample ? Do you think that may also cause the same issue? Or do you think it may be ok to use upsample()?

@amitdedhia Please try the Upsample. I am not sure about the behavior. Better to test it.

@quic_sundarr ONNX export does not work for bi-linear interpolation (F.interpolate) for torch v1.1.0. Pytorch team fixed it for later versions. Can I use latest version of Pytorch for AIMET libraries?

@amitdedhia Please do not use a version PyTorch other than version 1.1.0. You will encounter lot of errors.

You can try the following options one by one and check if any of them helps to do a successfulONNX export.

#1 Try interpolate with align_corners set to False.

2 Try interpolate with mode set to ‘nearest’

#3 Try Upsample mode set to’bilinear’ and ‘nearest’ with align_corners set to False
I had already suggested trying with Upsample. Did you try?

@quic_sundarr thanks for quick response.

No - I did not try yet the Upsample (was working on other task). I am going to try these above suggestions in this week.

@quic_sundarr

I tried all 4 approaches mentioned above and the result is same. Note that I did only channel pruning (no SSVD).

The error I am getting is: RuntimeError: Given transposed=1, weight of size 1191 96 4 4, expected input[16, 1280, 10, 10] to have 1191 channels, but got 1280 channels instead

The following is the model I used:

UNet(
  (backbone): MobileNetV2(
    (features): Sequential(
      (0): Sequential(
        (0): Conv2d(3, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU6(inplace)
      )
      (1): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32, bias=False)
          (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (4): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (2): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(16, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(96, 96, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=96, bias=False)
          (4): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(96, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (3): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(24, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(144, 144, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=144, bias=False)
          (4): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(144, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (4): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(24, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(144, 144, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=144, bias=False)
          (4): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(144, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (5): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(32, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192, bias=False)
          (4): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(192, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (6): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(32, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192, bias=False)
          (4): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(192, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (7): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(32, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(192, 192, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=192, bias=False)
          (4): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(192, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (8): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
          (4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (9): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
          (4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (10): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
          (4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (11): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
          (4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(384, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (12): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576, bias=False)
          (4): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(576, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (13): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576, bias=False)
          (4): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(576, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (14): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(576, 576, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=576, bias=False)
          (4): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(576, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (15): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(160, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960, bias=False)
          (4): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(960, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (16): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(160, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960, bias=False)
          (4): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(960, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (17): InvertedResidual(
        (conv): Sequential(
          (0): Conv2d(160, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU6(inplace)
          (3): Conv2d(960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960, bias=False)
          (4): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (5): ReLU6(inplace)
          (6): Conv2d(960, 320, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (7): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (18): Sequential(
        (0): Conv2d(320, 1280, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1280, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU6(inplace)
      )
    )
  )
  (decoder1): DecoderBlock(
    (deconv): ConvTranspose2d(1280, 96, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
    (block_unit): InvertedResidual(
      (conv): Sequential(
        (0): Conv2d(192, 1152, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU6(inplace)
        (3): Conv2d(1152, 1152, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=1152, bias=False)
        (4): BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (5): ReLU6(inplace)
        (6): Conv2d(1152, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (7): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
  )
  (decoder2): DecoderBlock(
    (deconv): ConvTranspose2d(96, 32, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
    (block_unit): InvertedResidual(
      (conv): Sequential(
        (0): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU6(inplace)
        (3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
        (4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (5): ReLU6(inplace)
        (6): Conv2d(384, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
  )
  (decoder3): DecoderBlock(
    (deconv): ConvTranspose2d(32, 24, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
    (block_unit): InvertedResidual(
      (conv): Sequential(
        (0): Conv2d(48, 288, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU6(inplace)
        (3): Conv2d(288, 288, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=288, bias=False)
        (4): BatchNorm2d(288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (5): ReLU6(inplace)
        (6): Conv2d(288, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (7): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
  )
  (decoder4): DecoderBlock(
    (deconv): ConvTranspose2d(24, 16, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
    (block_unit): InvertedResidual(
      (conv): Sequential(
        (0): Conv2d(32, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU6(inplace)
        (3): Conv2d(192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192, bias=False)
        (4): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (5): ReLU6(inplace)
        (6): Conv2d(192, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (7): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
  )
  (conv_last): Sequential(
    (0): Conv2d(16, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): Conv2d(3, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  )
  (upsample): Upsample(scale_factor=2.0, mode=bilinear)
) 

Clearly the issue is related to decoder1 object. I tried ignoring those layers from pruning like below.
auto_params = ChannelPruningParameters.AutoModeParams(greedy_params, modules_to_ignore=[comp_model.backbone.features[0][0], comp_model.backbone.features[18][0], comp_model.backbone.features[18][1], comp_model.decoder1.deconv, comp_model.conv_last[0], comp_model.conv_last[1]])

However it still did not work.

I am not sure whether this is issue in channel pruning code, or is it related to the model.

Please suggest…

@amitdedhia,

Please note that I am repeating below what I had recommended on August 7th:

"If ONNX export works successfully, then you can try the AIMET features."

All the workarounds I had suggested in my previous posts are for trying out if you are able to
do a successful onnx export.

This is a problem with the model with PyTorch 1.1.0
Please rewrite your model and make sure ONNX export succeeds.
If ONNX export works, then only you can use the Channel Pruning feature.

Thanks

@quic_sundarr, I guess you meant torch.jit.trace() instead of torch.onnx.export(). I am guessing they are very correlated, but AIMET depends on the former, correct?

@amitdedhia, just to clarify a bit more - AIMET needs to know the graph structure of the model to automatically make modifications related to channel pruning. The only way to know the graph structure of a PyTorch model is to run a trace through it. We run into these tracing issues occasionally with some model definitions, but a small tweak to the model definition is generally all that is required to get the trace to cooperate. The PyTorch forums help sometimes.