[Quantization][PrivateUse1] Adding more support QuantizedPrivateuse1 backends #139860
Conversation
fmo-mt
requested review from
digantdesai,
jerryzh168,
jianyuh,
kimishpatel and
salilsdesai
as code owners
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/139860
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ✅ No FailuresAs of commit bbde42a with merge base c19c384 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
bdhirsh
added
the
triaged
| namespace native { | ||
|
|
||
| Tensor& quantize_tensor_per_tensor_affine( | ||
| TORCH_API Tensor& quantize_tensor_per_tensor_affine( |
There was a problem hiding this comment.
@jerryzh168 you might be a better reviewer here - do we want these API's to get public symbols?
There was a problem hiding this comment.
I think we do, and it would throw an error of undefined symbol if we don't.
PTAL @jerryzh168
There was a problem hiding this comment.
this is not a public function I think, what does "getting public symbols" entail here?
There was a problem hiding this comment.
this is not a public function I think, what does "getting public symbols" entail here?
it's not, but it should be if privateuse1 user like to implement quantization functions like torch.quantize_per_tensor, so we can reuse this native common functions.
| @@ -1,4 +1,5 @@ | |||
| #pragma once | |||
| #include <ATen/native/DispatchStub.h> | |||
There was a problem hiding this comment.
Well, as I comment, it's because that during implementing some operators for privateuse1 backend, If DispatchStub.h is not included, we can't use the two stubs in this header, like masked_fill_kernel_quantized_stub and index_put_kernel_quantized_stub, which would throw an error during compilation.
|
@bdhirsh @jerryzh168 Hi, can we make this proceed? I believe that many other |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
…backends (pytorch#139860) Here's are some explanations of this PR. 1. Changes in `aten/src/ATen/core/Tensor.cpp` and `c10/core/DispatchKey.cpp`: Support toString method for `QuantizedPrivateUse1` backend, make pytorch print out correct backend string for it. 2. Add header `DispatchStub.h` in `aten/src/ATen/native/quantized/IndexKernel.h`: If this header is not included, we can't utilize `masked_fill_kernel_quantized_stub` even we include this `IndexKernel.h` header, it would throw an error during compilation. 3. Add multiple `TORCH_API`s in `aten/src/ATen/native/quantized/AffineQuantizer.h`: these functions is useful for other privateuse1 backends supporting quantization functions, if these `TORCH_API` are missed, it would throw an error during runtime (undefined symbol) Pull Request resolved: pytorch#139860 Approved by: https://github.com/bdhirsh
5 participants
Here's are some explanations of this PR.
aten/src/ATen/core/Tensor.cppandc10/core/DispatchKey.cpp: Support toString method forQuantizedPrivateUse1backend, make pytorch print out correct backend string for it.DispatchStub.hinaten/src/ATen/native/quantized/IndexKernel.h: If this header is not included, we can't utilizemasked_fill_kernel_quantized_stubeven we include thisIndexKernel.hheader, it would throw an error during compilation.TORCH_APIs inaten/src/ATen/native/quantized/AffineQuantizer.h: these functions is useful for other privateuse1 backends supporting quantization functions, if theseTORCH_APIare missed, it would throw an error during runtime (undefined symbol)