I have installed AIMET on google colab. I am trying to evaluate AIMET using Pytorch APIs. I could successfully run the SSVD compression. However on the same environment, the channel pruning API throws error. Can anyone help whats going wrong here?
Exception details:
2020-06-26 05:59:22,290 - CompRatioSelect - INFO - Analyzing compression ratio: 0.1 =====================>
/usr/local/lib/python3.6/dist-packages/aimet_torch/winnow/mask_propagation_winnower.py:96: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
dummy_input = torch.tensor(dummy_input).cuda() # pylint: disable=not-callable
/usr/local/lib/python3.6/dist-packages/torch/tensor.py:435: RuntimeWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won’t change the number of iterations executed (and might lead to errors or silently give incorrect results).
‘incorrect results).’, category=RuntimeWarning)
RuntimeError Traceback (most recent call last)
in ()
19 compress_scheme=CompressionScheme.channel_pruning,
20 cost_metric=CostMetric.mac,
—> 21 parameters=channel_pruning_params)
14 frames
/usr/local/lib/python3.6/dist-packages/aimet_torch/compress.py in compress_model(model, eval_callback, eval_iterations, input_shape, compress_scheme, cost_metric, parameters, trainer, visualization_url)
111 raise ValueError(“Compression scheme not supported: {}”.format(compress_scheme))
112
→ 113 compressed_layer_db, stats = algo.compress_model(cost_metric, trainer)
114 return compressed_layer_db.model, stats
/usr/local/lib/python3.6/dist-packages/aimet_common/compression_algo.py in compress_model(self, cost_metric, trainer)
86
87 # Find optimal compression ratios for each layer
—> 88 layer_comp_ratio_list, stats = self._comp_ratio_select_algo.select_per_layer_comp_ratios()
89
90 self._pickle_comp_ratio_list(layer_comp_ratio_list)
/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in select_per_layer_comp_ratios(self)
221
222 # Compute eval scores for each candidate comp-ratio in each layer
→ 223 eval_scores_dict = self._construct_eval_dict()
224
225 # Fit the scores to a monotonically increasing function
/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in _construct_eval_dict(self)
212 else:
213 # Create the eval scores dictionary
→ 214 eval_scores_dict = self._compute_eval_scores_for_all_comp_ratio_candidates()
215
216 # save the dictionary to file (in case the user wants to reuse the dictionary in the future)
/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in _compute_eval_scores_for_all_comp_ratio_candidates(self)
398
399 layer_wise_eval_scores = self._compute_layerwise_eval_score_per_comp_ratio_candidate(data_table,
→ 400 progress_bar, layer)
401 eval_scores_dict[layer.name] = layer_wise_eval_scores
402
/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in _compute_layerwise_eval_score_per_comp_ratio_candidate(self, tabular_progress_object, progress_bar, layer)
427 [LayerCompRatioPair(layer, comp_ratio)],
428 self._cost_metric,
→ 429 trainer=None)
430
431 eval_score = self._eval_func(pruned_layer_db.model, self._eval_iter, use_cuda=self._is_cuda)
/usr/local/lib/python3.6/dist-packages/aimet_torch/channel_pruning/channel_pruner.py in prune_model(self, layer_db, layer_comp_ratio_list, cost_metric, trainer)
249 # call the base class method
250 comp_layer_db = Pruner.prune_model(self, layer_db,
→ 251 layer_comp_ratio_list, cost_metric, trainer)
252
253 return comp_layer_db
/usr/local/lib/python3.6/dist-packages/aimet_common/pruner.py in prune_model(self, layer_db, layer_comp_ratio_list, cost_metric, trainer)
73
74 if comp_ratio is not None and comp_ratio < 1.0:
—> 75 self._prune_layer(layer_db, comp_layer_db, layer, comp_ratio, cost_metric)
76
77 # fine-tuning the layer while creating the final model
/usr/local/lib/python3.6/dist-packages/aimet_torch/channel_pruning/channel_pruner.py in _prune_layer(self, orig_layer_db, comp_layer_db, layer, comp_ratio, cost_metric)
216 def _prune_layer(self, orig_layer_db: LayerDatabase, comp_layer_db: LayerDatabase,
217 layer: Layer, comp_ratio: float, cost_metric: CostMetric):
→ 218 self._winnow_and_reconstruct_layer(orig_layer_db, comp_layer_db, layer, comp_ratio, True)
219
220 def calculate_compressed_cost(self, layer_db: LayerDatabase,
/usr/local/lib/python3.6/dist-packages/aimet_torch/channel_pruning/channel_pruner.py in _winnow_and_reconstruct_layer(self, orig_layer_db, comp_layer_db, layer, comp_ratio, perform_reconstruction)
178 [(layer.module, prune_indices)],
179 reshape=self._allow_custom_downsample_ops,
→ 180 in_place=True)
181
182 # 3) data sub sampling and reconstruction
/usr/local/lib/python3.6/dist-packages/aimet_torch/winnow/winnow.py in winnow_model(model, input_shape, list_of_modules_to_winnow, reshape, in_place, verbose)
68
69 mask_winnower = MaskPropagationWinnower(model, input_shape, list_of_modules_to_winnow, reshape,
—> 70 in_place, verbose)
71 new_model, ordered_modules_list = mask_winnower.propagate_masks_and_winnow()
72
/usr/local/lib/python3.6/dist-packages/aimet_torch/winnow/mask_propagation_winnower.py in init(self, model, input_shape, list_of_modules_to_winnow, reshape, in_place, verbose)
96 dummy_input = torch.tensor(dummy_input).cuda() # pylint: disable=not-callable
97
—> 98 self._graph = ConnectedGraph(self._model, (dummy_input,))
99 self.list_of_modules_to_winnow_with_names =
100 generate_and_add_module_winnow_list_with_names(model, self._list_of_modules_to_winnow)
/usr/local/lib/python3.6/dist-packages/aimet_torch/meta/connectedgraph.py in init(self, model, model_input)
200
201 self._generate_module_lookup_table(model)
→ 202 self._construct_graph(model, model_input)
203
204 # Map torch module types to normalized names to provide backward compatibility to
/usr/local/lib/python3.6/dist-packages/aimet_torch/meta/connectedgraph.py in _construct_graph(self, model, model_input)
303 “”"
304 if torch.version == ‘1.1.0’:
→ 305 trace = torch.jit.trace(model, model_input)
306 # Parse trace code to create ops and products
307 self._parse_trace_code(trace.code)
/usr/local/lib/python3.6/dist-packages/torch/jit/init.py in trace(func, example_inputs, optimize, check_trace, check_inputs, check_tolerance, _force_outplace, _module_class)
686 traced = _module_class(func, **executor_options)
687 traced._c._create_method_from_trace(‘forward’, func, example_inputs,
→ 688 var_lookup_fn, _force_outplace)
689 else:
690 name = getattr(func, ‘name’, ‘forward’)
RuntimeError: Only tensors or tuples of tensors can be output from traced functions (getNestedOutputTrace at /pytorch/torch/csrc/jit/tracer.cpp:200)
frame #0: std::function<std::string ()>::operator()() const + 0x11 (0x7fa40370c441 in /usr/local/lib/python3.6/dist-packages/torch/lib/libc10.so)
frame #1: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x2a (0x7fa40370bd7a in /usr/local/lib/python3.6/dist-packages/torch/lib/libc10.so)
frame #2: torch::jit::tracer::getNestedOutputTrace(std::shared_ptrtorch::jit::tracer::TracingState const&, c10::IValue const&) + 0x20d (0x7fa4029f173d in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch.so.1)
frame #3: torch::jit::tracer::exit(std::vector<c10::IValue, std::allocatorc10::IValue > const&) + 0x2f (0x7fa4029f180f in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch.so.1)
frame #4: + 0x447c13 (0x7fa44bfbdc13 in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so)
frame #5: + 0x459e92 (0x7fa44bfcfe92 in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so)
frame #6: + 0x130cfc (0x7fa44bca6cfc in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so)
frame #7: _PyCFunction_FastCallDict + 0x35c (0x5669ac in /usr/bin/python3)
frame #8: /usr/bin/python3() [0x50a5c3]
frame #9: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #10: /usr/bin/python3() [0x507d64]
frame #11: /usr/bin/python3() [0x509a90]
frame #12: /usr/bin/python3() [0x50a48d]
frame #13: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #14: /usr/bin/python3() [0x509758]
frame #15: /usr/bin/python3() [0x50a48d]
frame #16: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #17: /usr/bin/python3() [0x507d64]
frame #18: _PyFunction_FastCallDict + 0x2e2 (0x509042 in /usr/bin/python3)
frame #19: /usr/bin/python3() [0x594931]
frame #20: /usr/bin/python3() [0x549e5f]
frame #21: /usr/bin/python3() [0x5513d1]
frame #22: _PyObject_FastCallKeywords + 0x19c (0x5a9cbc in /usr/bin/python3)
frame #23: /usr/bin/python3() [0x50a5c3]
frame #24: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #25: /usr/bin/python3() [0x507d64]
frame #26: _PyFunction_FastCallDict + 0x357 (0x5090b7 in /usr/bin/python3)
frame #27: /usr/bin/python3() [0x594931]
frame #28: /usr/bin/python3() [0x549e5f]
frame #29: /usr/bin/python3() [0x5513d1]
frame #30: _PyObject_FastCallKeywords + 0x19c (0x5a9cbc in /usr/bin/python3)
frame #31: /usr/bin/python3() [0x50a5c3]
frame #32: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #33: /usr/bin/python3() [0x507d64]
frame #34: /usr/bin/python3() [0x509a90]
frame #35: /usr/bin/python3() [0x50a48d]
frame #36: _PyEval_EvalFrameDefault + 0x1226 (0x50cd96 in /usr/bin/python3)
frame #37: /usr/bin/python3() [0x509758]
frame #38: /usr/bin/python3() [0x50a48d]
frame #39: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #40: /usr/bin/python3() [0x509758]
frame #41: /usr/bin/python3() [0x50a48d]
frame #42: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #43: /usr/bin/python3() [0x509758]
frame #44: /usr/bin/python3() [0x50a48d]
frame #45: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #46: /usr/bin/python3() [0x507d64]
frame #47: /usr/bin/python3() [0x509a90]
frame #48: /usr/bin/python3() [0x50a48d]
frame #49: _PyEval_EvalFrameDefault + 0x1226 (0x50cd96 in /usr/bin/python3)
frame #50: /usr/bin/python3() [0x509758]
frame #51: /usr/bin/python3() [0x50a48d]
frame #52: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #53: /usr/bin/python3() [0x509758]
frame #54: /usr/bin/python3() [0x50a48d]
frame #55: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #56: /usr/bin/python3() [0x509758]
frame #57: /usr/bin/python3() [0x50a48d]
frame #58: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #59: /usr/bin/python3() [0x509758]
frame #60: /usr/bin/python3() [0x50a48d]
frame #61: _PyEval_EvalFrameDefault + 0x444 (0x50bfb4 in /usr/bin/python3)
frame #62: /usr/bin/python3() [0x509758]
frame #63: /usr/bin/python3() [0x50a48d]
Interesting. Thanks for reporting @amitdedhia.
@quic_sundarr, anything jumps out at you from the stack trace?
Before using AIMET for Channel Pruning, could you please do torch.onnx export of your model with verbose=True and let us know the result.
Sample code:
torch.onnx.export(your_model, <model_input_tensor>, “<model_name.onnx”, verbose=True)
Reference:
https://pytorch.org/docs/1.1.0/onnx.html?highlight=onnx#module-torch.onnx
Thanks.
@quic_sundarr
Sorry for delayed response. I was busy with other priority stuff.
To help you, I tried to install AIMET on google colab. However, I am getting error while installing. I was able to successfully install AIMET on google colab about a month back. So not sure what did I do wrong. Can you help me please on this?
The error is:
CMake Error at TrainingExtensions/torch/cmake_install.cmake:73 (file):
file INSTALL cannot find “/content/build/artifacts/site.py”.
I am attaching full trace. The error occurs while running the last installation step i.e. “make install” command.
Thank you…
Similar problem is faced by others too. This worked for me while building inside a docker container: Issue in installation with make install command · Issue #108 · quic/aimet · GitHub
Thanks @ishan This worked 
@quic_sundarr
I tried with a different model. The above exception has gone. The channel pruning api ran well for quite long. However it failed again mid way. Here is the exception
RuntimeError: Given transposed=1, weight of size 1193 96 4 4, expected input[16, 1280, 10, 10] to have 1193 channels, but got 1280 channels instead
The below is more detail:
[Dataset] Number of sample pairs: 70
2020-08-03 09:34:27,084 - CompRatioSelect - INFO - Layer backbone.features.18.0.1, comp_ratio 0.800000 ==> eval_score=0.012582
2020-08-03 09:34:27,085 - CompRatioSelect - INFO - Analyzing compression ratio: 0.9 =====================>
2020-08-03 09:34:33,991 - ChannelPruning - INFO - finished linear regression fit
[Dataset] Checking file paths...
[Dataset] Number of sample pairs: 70
2020-08-03 09:34:39,162 - CompRatioSelect - INFO - Layer backbone.features.18.0.1, comp_ratio 0.900000 ==> eval_score=0.013433
2020-08-03 09:34:39,164 - CompRatioSelect - INFO - Analyzing compression ratio: 0.1 =====================>
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-65-7a498fadb2e5> in <module>()
27 compress_scheme=CompressionScheme.channel_pruning,
28 cost_metric=CostMetric.mac,
---> 29 parameters=channel_pruning_params)
18 frames
/usr/local/lib/python3.6/dist-packages/aimet_torch/compress.py in compress_model(model, eval_callback, eval_iterations, input_shape, compress_scheme, cost_metric, parameters, trainer, visualization_url)
111 raise ValueError("Compression scheme not supported: {}".format(compress_scheme))
112
--> 113 compressed_layer_db, stats = algo.compress_model(cost_metric, trainer)
114 return compressed_layer_db.model, stats
/usr/local/lib/python3.6/dist-packages/aimet_common/compression_algo.py in compress_model(self, cost_metric, trainer)
86
87 # Find optimal compression ratios for each layer
---> 88 layer_comp_ratio_list, stats = self._comp_ratio_select_algo.select_per_layer_comp_ratios()
89
90 self._pickle_comp_ratio_list(layer_comp_ratio_list)
/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in select_per_layer_comp_ratios(self)
221
222 # Compute eval scores for each candidate comp-ratio in each layer
--> 223 eval_scores_dict = self._construct_eval_dict()
224
225 # Fit the scores to a monotonically increasing function
/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in _construct_eval_dict(self)
212 else:
213 # Create the eval scores dictionary
--> 214 eval_scores_dict = self._compute_eval_scores_for_all_comp_ratio_candidates()
215
216 # save the dictionary to file (in case the user wants to reuse the dictionary in the future)
/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in _compute_eval_scores_for_all_comp_ratio_candidates(self)
398
399 layer_wise_eval_scores = self._compute_layerwise_eval_score_per_comp_ratio_candidate(data_table,
--> 400 progress_bar, layer)
401 eval_scores_dict[layer.name] = layer_wise_eval_scores
402
/usr/local/lib/python3.6/dist-packages/aimet_common/comp_ratio_select.py in _compute_layerwise_eval_score_per_comp_ratio_candidate(self, tabular_progress_object, progress_bar, layer)
427 [LayerCompRatioPair(layer, comp_ratio)],
428 self._cost_metric,
--> 429 trainer=None)
430
431 eval_score = self._eval_func(pruned_layer_db.model, self._eval_iter, use_cuda=self._is_cuda)
/usr/local/lib/python3.6/dist-packages/aimet_torch/channel_pruning/channel_pruner.py in prune_model(self, layer_db, layer_comp_ratio_list, cost_metric, trainer)
249 # call the base class method
250 comp_layer_db = Pruner.prune_model(self, layer_db,
--> 251 layer_comp_ratio_list, cost_metric, trainer)
252
253 return comp_layer_db
/usr/local/lib/python3.6/dist-packages/aimet_common/pruner.py in prune_model(self, layer_db, layer_comp_ratio_list, cost_metric, trainer)
73
74 if comp_ratio is not None and comp_ratio < 1.0:
---> 75 self._prune_layer(layer_db, comp_layer_db, layer, comp_ratio, cost_metric)
76
77 # fine-tuning the layer while creating the final model
/usr/local/lib/python3.6/dist-packages/aimet_torch/channel_pruning/channel_pruner.py in _prune_layer(self, orig_layer_db, comp_layer_db, layer, comp_ratio, cost_metric)
216 def _prune_layer(self, orig_layer_db: LayerDatabase, comp_layer_db: LayerDatabase,
217 layer: Layer, comp_ratio: float, cost_metric: CostMetric):
--> 218 self._winnow_and_reconstruct_layer(orig_layer_db, comp_layer_db, layer, comp_ratio, True)
219
220 def calculate_compressed_cost(self, layer_db: LayerDatabase,
/usr/local/lib/python3.6/dist-packages/aimet_torch/channel_pruning/channel_pruner.py in _winnow_and_reconstruct_layer(self, orig_layer_db, comp_layer_db, layer, comp_ratio, perform_reconstruction)
185 orig_layer = orig_layer_db.find_layer_by_name(layer.name)
186 self._data_subsample_and_reconstruction(orig_layer.module, layer.module, orig_layer_db.model,
--> 187 comp_layer_db.model)
188
189 # 4) update layer database
/usr/local/lib/python3.6/dist-packages/aimet_torch/channel_pruning/channel_pruner.py in _data_subsample_and_reconstruction(self, orig_layer, pruned_layer, orig_model, comp_model)
115 """
116 inp_data, out_data = DataSubSampler.get_sub_sampled_data(orig_layer, pruned_layer, orig_model, comp_model,
--> 117 self._data_loader, self._num_reconstruction_samples)
118
119 WeightReconstructor.reconstruct_params_for_conv2d(pruned_layer, inp_data, out_data)
/usr/local/lib/python3.6/dist-packages/aimet_torch/data_subsampler.py in get_sub_sampled_data(cls, orig_layer, pruned_layer, orig_model, comp_model, data_loader, num_reconstruction_samples)
276
277 DataSubSampler._forward_pass(orig_model, batch)
--> 278 DataSubSampler._forward_pass(comp_model, batch)
279
280 input_data = np.vstack(pruned_layer_inp_data)
/usr/local/lib/python3.6/dist-packages/aimet_torch/data_subsampler.py in _forward_pass(model, batch)
181 try:
182 with torch.no_grad():
--> 183 _ = model(*batch)
184 except StopForwardException:
185 pass
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
491 result = self._slow_forward(*input, **kwargs)
492 else:
--> 493 result = self.forward(*input, **kwargs)
494 for hook in self._forward_hooks.values():
495 hook_result = hook(self, input, result)
/content/Human-Segmentation-PyTorch/models/UNet.py in forward(self, input)
108 def forward(self, input):
109 x1, x2, x3, x4, x5 = self._run_backbone(input)
--> 110 x = self.decoder1(x5, x4)
111 x = self.decoder2(x, x3)
112 x = self.decoder3(x, x2)
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
491 result = self._slow_forward(*input, **kwargs)
492 else:
--> 493 result = self.forward(*input, **kwargs)
494 for hook in self._forward_hooks.values():
495 hook_result = hook(self, input, result)
/content/Human-Segmentation-PyTorch/models/UNet.py in forward(self, input, shortcut)
21
22 def forward(self, input, shortcut):
---> 23 x = self.deconv(input)
24 x = torch.cat([x, shortcut], dim=1)
25 x = self.block_unit(x)
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
491 result = self._slow_forward(*input, **kwargs)
492 else:
--> 493 result = self.forward(*input, **kwargs)
494 for hook in self._forward_hooks.values():
495 hook_result = hook(self, input, result)
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/conv.py in forward(self, input, output_size)
794 return F.conv_transpose2d(
795 input, self.weight, self.bias, self.stride, self.padding,
--> 796 output_padding, self.groups, self.dilation)
797
798
RuntimeError: Given transposed=1, weight of size 1193 96 4 4, expected input[16, 1280, 10, 10] to have 1193 channels, but got 1280 channels instead
I first applied SSVD compression on my uncompressed model. Then applied channel pruning. The input model to channel pruning (i.e. the SSVD compressed model) is as follows.
Sequential(
(0): Sequential(
(0): Sequential(
(0): Conv2d(3, 1, kernel_size=(3, 1), stride=(2, 1), padding=(1, 0), bias=False)
(1): Conv2d(1, 32, kernel_size=(1, 3), stride=(1, 2), padding=(0, 1), bias=False)
)
(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
)
(1): InvertedResidual(
(conv): Sequential(
(0): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32, bias=False)
(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Sequential(
(0): Conv2d(32, 1, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(1, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(4): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(2): InvertedResidual(
(conv): Sequential(
(0): Sequential(
(0): Conv2d(16, 1, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(1, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(96, 96, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=96, bias=False)
(4): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(96, 1, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(1, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(3): InvertedResidual(
(conv): Sequential(
(0): Sequential(
(0): Conv2d(24, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(2, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(144, 144, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=144, bias=False)
(4): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(144, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(2, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(4): InvertedResidual(
(conv): Sequential(
(0): Conv2d(24, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(144, 144, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=144, bias=False)
(4): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(144, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(2, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(5): InvertedResidual(
(conv): Sequential(
(0): Sequential(
(0): Conv2d(32, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(2, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192, bias=False)
(4): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(192, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(2, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(6): InvertedResidual(
(conv): Sequential(
(0): Sequential(
(0): Conv2d(32, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(2, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192, bias=False)
(4): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(192, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(2, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(7): InvertedResidual(
(conv): Sequential(
(0): Sequential(
(0): Conv2d(32, 2, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(2, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(192, 192, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=192, bias=False)
(4): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(192, 4, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(4, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(8): InvertedResidual(
(conv): Sequential(
(0): Sequential(
(0): Conv2d(64, 5, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(5, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
(4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(384, 5, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(5, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(9): InvertedResidual(
(conv): Sequential(
(0): Sequential(
(0): Conv2d(64, 5, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(5, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
(4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(384, 5, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(5, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(10): InvertedResidual(
(conv): Sequential(
(0): Sequential(
(0): Conv2d(64, 5, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(5, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
(4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(384, 5, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(5, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(11): InvertedResidual(
(conv): Sequential(
(0): Sequential(
(0): Conv2d(64, 5, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(5, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
(4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(384, 7, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(7, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(12): InvertedResidual(
(conv): Sequential(
(0): Sequential(
(0): Conv2d(96, 8, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(8, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576, bias=False)
(4): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(576, 8, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(8, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(13): InvertedResidual(
(conv): Sequential(
(0): Sequential(
(0): Conv2d(96, 8, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(8, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576, bias=False)
(4): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(576, 8, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(8, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(14): InvertedResidual(
(conv): Sequential(
(0): Sequential(
(0): Conv2d(96, 8, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(8, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(576, 576, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=576, bias=False)
(4): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(576, 12, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(12, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(15): InvertedResidual(
(conv): Sequential(
(0): Sequential(
(0): Conv2d(160, 13, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(13, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960, bias=False)
(4): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(960, 13, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(13, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(16): InvertedResidual(
(conv): Sequential(
(0): Conv2d(160, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960, bias=False)
(4): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(960, 13, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(13, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(17): InvertedResidual(
(conv): Sequential(
(0): Sequential(
(0): Conv2d(160, 13, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(13, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960, bias=False)
(4): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Sequential(
(0): Conv2d(960, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(24, 320, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(7): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(18): Sequential(
(0): Sequential(
(0): Conv2d(320, 25, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): Conv2d(25, 1280, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(1): BatchNorm2d(1280, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
)
)
So what could be the issue here?
Let me know if you need any more details from me …
#1
To narrow down the problem, did you just try Channel Pruning on this model without doing SSVD?
If the problem still exists, we can focus on channel prubing.
#2
Is it possible for you to share the complete model?
Very often, these issues happen because of how the forward() functions of a model is written.
Without looking at the forward() functions it is difficult to debug or analyze.
Thanks
I have shared the model files in another thread here: Exception while exporting Quantized model - #5 by amitdedhia
Can you pick from it? - Thanks
I will try just Channel Pruning later (soon) and let you know the results
@amitdedhia Thanks for sharing the model. I looked at the model definition. So, the model you are testing is UNet and you used MobileNetV2 as the backbone. Correct? I will take a detailed look and update.
@amitdedhia, The UNet model’s forward() has the following line:
x = F.interpolate(x, size=input.shape[-2:], mode='bilinear', align_corners=True)
My first suspicion is that interpolate() may be causing the ONNX export error (other thread) as well as the Channel Pruning error. Please replace the interpolate() with an equivalent operation and try ONNX export. If ONNX export works successfully, then you can try the AIMET features.
@quic_sundarr Thank you for your response.
Instead of interpolate, can I use F.Upsample ? Do you think that may also cause the same issue? Or do you think it may be ok to use upsample()?
@amitdedhia Please try the Upsample. I am not sure about the behavior. Better to test it.
@quic_sundarr ONNX export does not work for bi-linear interpolation (F.interpolate) for torch v1.1.0. Pytorch team fixed it for later versions. Can I use latest version of Pytorch for AIMET libraries?
@amitdedhia Please do not use a version PyTorch other than version 1.1.0. You will encounter lot of errors.
You can try the following options one by one and check if any of them helps to do a successfulONNX export.
#1 Try interpolate with align_corners set to False.
2 Try interpolate with mode set to ‘nearest’
#3 Try Upsample mode set to’bilinear’ and ‘nearest’ with align_corners set to False
I had already suggested trying with Upsample. Did you try?
@quic_sundarr thanks for quick response.
No - I did not try yet the Upsample (was working on other task). I am going to try these above suggestions in this week.
I tried all 4 approaches mentioned above and the result is same. Note that I did only channel pruning (no SSVD).
The error I am getting is: RuntimeError: Given transposed=1, weight of size 1191 96 4 4, expected input[16, 1280, 10, 10] to have 1191 channels, but got 1280 channels instead
The following is the model I used:
UNet(
(backbone): MobileNetV2(
(features): Sequential(
(0): Sequential(
(0): Conv2d(3, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
)
(1): InvertedResidual(
(conv): Sequential(
(0): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32, bias=False)
(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
(4): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(2): InvertedResidual(
(conv): Sequential(
(0): Conv2d(16, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(96, 96, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=96, bias=False)
(4): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(96, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(3): InvertedResidual(
(conv): Sequential(
(0): Conv2d(24, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(144, 144, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=144, bias=False)
(4): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(144, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(4): InvertedResidual(
(conv): Sequential(
(0): Conv2d(24, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(144, 144, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=144, bias=False)
(4): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(144, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(5): InvertedResidual(
(conv): Sequential(
(0): Conv2d(32, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192, bias=False)
(4): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(192, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(6): InvertedResidual(
(conv): Sequential(
(0): Conv2d(32, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192, bias=False)
(4): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(192, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(7): InvertedResidual(
(conv): Sequential(
(0): Conv2d(32, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(192, 192, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=192, bias=False)
(4): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(192, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(8): InvertedResidual(
(conv): Sequential(
(0): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
(4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(9): InvertedResidual(
(conv): Sequential(
(0): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
(4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(10): InvertedResidual(
(conv): Sequential(
(0): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
(4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(11): InvertedResidual(
(conv): Sequential(
(0): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
(4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(384, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(12): InvertedResidual(
(conv): Sequential(
(0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576, bias=False)
(4): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(576, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(13): InvertedResidual(
(conv): Sequential(
(0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576, bias=False)
(4): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(576, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(14): InvertedResidual(
(conv): Sequential(
(0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(576, 576, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=576, bias=False)
(4): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(576, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(15): InvertedResidual(
(conv): Sequential(
(0): Conv2d(160, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960, bias=False)
(4): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(960, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(16): InvertedResidual(
(conv): Sequential(
(0): Conv2d(160, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960, bias=False)
(4): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(960, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(17): InvertedResidual(
(conv): Sequential(
(0): Conv2d(160, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960, bias=False)
(4): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(960, 320, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(18): Sequential(
(0): Conv2d(320, 1280, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(1280, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
)
)
)
(decoder1): DecoderBlock(
(deconv): ConvTranspose2d(1280, 96, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(block_unit): InvertedResidual(
(conv): Sequential(
(0): Conv2d(192, 1152, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(1152, 1152, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=1152, bias=False)
(4): BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(1152, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
)
(decoder2): DecoderBlock(
(deconv): ConvTranspose2d(96, 32, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(block_unit): InvertedResidual(
(conv): Sequential(
(0): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)
(4): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(384, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
)
(decoder3): DecoderBlock(
(deconv): ConvTranspose2d(32, 24, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(block_unit): InvertedResidual(
(conv): Sequential(
(0): Conv2d(48, 288, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(288, 288, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=288, bias=False)
(4): BatchNorm2d(288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(288, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
)
(decoder4): DecoderBlock(
(deconv): ConvTranspose2d(24, 16, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(block_unit): InvertedResidual(
(conv): Sequential(
(0): Conv2d(32, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace)
(3): Conv2d(192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192, bias=False)
(4): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace)
(6): Conv2d(192, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
)
(conv_last): Sequential(
(0): Conv2d(16, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(1): Conv2d(3, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
)
(upsample): Upsample(scale_factor=2.0, mode=bilinear)
)
Clearly the issue is related to decoder1 object. I tried ignoring those layers from pruning like below.
auto_params = ChannelPruningParameters.AutoModeParams(greedy_params, modules_to_ignore=[comp_model.backbone.features[0][0], comp_model.backbone.features[18][0], comp_model.backbone.features[18][1], comp_model.decoder1.deconv, comp_model.conv_last[0], comp_model.conv_last[1]])
However it still did not work.
I am not sure whether this is issue in channel pruning code, or is it related to the model.
Please suggest…
Please note that I am repeating below what I had recommended on August 7th:
“If ONNX export works successfully, then you can try the AIMET features.”
All the workarounds I had suggested in my previous posts are for trying out if you are able to
do a successful onnx export.
This is a problem with the model with PyTorch 1.1.0
Please rewrite your model and make sure ONNX export succeeds.
If ONNX export works, then only you can use the Channel Pruning feature.
Thanks
@quic_sundarr, I guess you meant torch.jit.trace() instead of torch.onnx.export(). I am guessing they are very correlated, but AIMET depends on the former, correct?
@amitdedhia, just to clarify a bit more - AIMET needs to know the graph structure of the model to automatically make modifications related to channel pruning. The only way to know the graph structure of a PyTorch model is to run a trace through it. We run into these tracing issues occasionally with some model definitions, but a small tweak to the model definition is generally all that is required to get the trace to cooperate. The PyTorch forums help sometimes.