[Quant] [PT2] Enable QAT Quantization flow in X86InductorQuantizer #111280
Conversation
[ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/111280
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit e73c3ab with merge base a126bbf ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
leslie-fang-intel
changed the title
ghstack-source-id: cfa1feb Pull Request resolved: pytorch#111280
…uantizer" **Summary** This PR enables PT2 QAT Quantization flow in `X86InductorQuantizer`. **Test Plan** ``` python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_with_quantizer_api python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_unary_with_quantizer_api python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d_relu ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler [ghstack-poisoned]
…uantizer" **Summary** This PR enables PT2 QAT Quantization flow in `X86InductorQuantizer`. **Test Plan** ``` python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_with_quantizer_api python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_unary_with_quantizer_api python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d_relu ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler [ghstack-poisoned]
|
Hi @jerryzh168 @andrewor14, Could you kindly help to take a look of this PR? |
| 4, | ||
| 17, |
There was a problem hiding this comment.
these numbers will be tricky to get right I feel, so ideally this can be structured differently, e.g. like specifying a list of nodes etc.
There was a problem hiding this comment.
Yean, it's tricky to make it correct. I add a following up PR to re-structure it: #112570 in this ghstack. After that, we only need to check the patterns we care in each test case such as: dequant promotion, QConv/Linear Unary and QConv Binary instead of all the patterns . Please help to take a look of that PR also @jerryzh168.
| if ( | ||
| conv_node.op != "call_function" | ||
| or conv_node.target != torch.ops.aten.conv2d.default | ||
| ): |
There was a problem hiding this comment.
just making sure, do you have plans to follow https://pytorch.org/tutorials/prototype/pt2e_quantizer.html#a-note-on-ir-for-pt2e-quantization-flow later?
There was a problem hiding this comment.
Did you mean use SubgraphMatcherWithNameNodeMap to match the pattern for quantizer? I think we can follow up the changes later, maybe after the changes in xnnpack_quantizer. BTW: as discussed in the slack, is there any plan to support functionalization for pre autograd aten IR? I think it can help us save lots of patterns to use SubgraphMatcherWithNameNodeMap .
There was a problem hiding this comment.
sg, yeah functionalization will be supported in 1-2 month I think
| self, model: torch.fx.GraphModule, config: QuantizationConfig | ||
| ): | ||
| # Annotate QAT Specific patterns | ||
| self._annotate_qat_conv2d_bn_unary(model, config) |
There was a problem hiding this comment.
do you need to add something to prepare_pt2e code to support this pattern? https://github.com/pytorch/pytorch/blob/main/torch/ao/quantization/quantize_pt2e.py#L178
we are also thinking about moving qat specific code to quantizer as well actually, we'll discuss a bit internally and get your feedback
There was a problem hiding this comment.
Yean, for now we only have relu as unary post op which I think is already supported in https://github.com/pytorch/pytorch/blob/main/torch/ao/quantization/quantize_pt2e.py#L178.
ghstack-source-id: 1e15fb5 Pull Request resolved: pytorch#111280
…uantizer" **Summary** This PR enables PT2 QAT Quantization flow in `X86InductorQuantizer`. **Test Plan** ``` python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_with_quantizer_api python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_unary_with_quantizer_api python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d_relu ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler [ghstack-poisoned]
…uantizer" **Summary** This PR enables PT2 QAT Quantization flow in `X86InductorQuantizer`. **Test Plan** ``` python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_with_quantizer_api python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_unary_with_quantizer_api python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d_relu ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler [ghstack-poisoned]
|
Hi @jerryzh168 @andrewor14, thanks for the comments. Please kindly help to take a look again. |
…uantizer" **Summary** This PR enables PT2 QAT Quantization flow in `X86InductorQuantizer`. **Test Plan** ``` python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_with_quantizer_api python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_unary_with_quantizer_api python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d_relu ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler [ghstack-poisoned]
ghstack-source-id: 70c4f4f Pull Request resolved: pytorch#111280
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
#111281) **Summary** This PR adds ConvBNAdd(ReLU) QAT Annotation into `X86InductorQuantizer`. **Test Plan** ``` python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_binary_with_quantizer_api python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_binary_unary_with_quantizer_api python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d_add python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d_add_relu ``` Pull Request resolved: #111281 Approved by: https://github.com/jgong5, https://github.com/jerryzh168 ghstack dependencies: #111280
…ytorch#111280) **Summary** This PR enables PT2 QAT Quantization flow in `X86InductorQuantizer`. **Test Plan** ``` python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_with_quantizer_api python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_unary_with_quantizer_api python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d_relu ``` Pull Request resolved: pytorch#111280 Approved by: https://github.com/jgong5, https://github.com/jerryzh168
pytorch#111281) **Summary** This PR adds ConvBNAdd(ReLU) QAT Annotation into `X86InductorQuantizer`. **Test Plan** ``` python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_binary_with_quantizer_api python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_binary_unary_with_quantizer_api python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d_add python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d_add_relu ``` Pull Request resolved: pytorch#111281 Approved by: https://github.com/jgong5, https://github.com/jerryzh168 ghstack dependencies: pytorch#111280
…ytorch#111280) **Summary** This PR enables PT2 QAT Quantization flow in `X86InductorQuantizer`. **Test Plan** ``` python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_with_quantizer_api python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_unary_with_quantizer_api python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d_relu ``` Pull Request resolved: pytorch#111280 Approved by: https://github.com/jgong5, https://github.com/jerryzh168
pytorch#111281) **Summary** This PR adds ConvBNAdd(ReLU) QAT Annotation into `X86InductorQuantizer`. **Test Plan** ``` python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_binary_with_quantizer_api python -m pytest test_x86inductor_quantizer.py -k test_qat_conv2d_binary_unary_with_quantizer_api python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d_add python -m pytest test_mkldnn_pattern_matcher.py -k test_qat_qconv2d_add_relu ``` Pull Request resolved: pytorch#111281 Approved by: https://github.com/jgong5, https://github.com/jerryzh168 ghstack dependencies: pytorch#111280
5 participants
Stack from ghstack (oldest at bottom):
Summary
This PR enables PT2 QAT Quantization flow in
X86InductorQuantizer.Test Plan
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler