[fp8]Support fp8 deep gemm in paddle #73092
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
你的PR提交成功，感谢你对开源项目的贡献! |
risemeup1
force-pushed
the
support_fp8_deep_gemm_in_paddle
branch
2 times, most recently
from
2dabc5f to
a5c4aea
Compare
risemeup1
changed the title
risemeup1
force-pushed
the
support_fp8_deep_gemm_in_paddle
branch
from
a5c4aea to
f618e85
Compare
SigureMo
reviewed
Jun 4, 2025
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| // The file has been adapted from DeepSeek DeepEP project |
There was a problem hiding this comment.
Suggested change
| // The file has been adapted from DeepSeek DeepEP project | |
| // The file has been adapted from DeepSeek DeepGEMM project |
|
|
||
| // The file has been adapted from DeepSeek DeepEP project | ||
| // Copyright (c) 2025 DeepSeek | ||
| // Licensed under the MIT License - https://github.com/deepseek-ai/DeepEP/blob/main/LICENSE |
There was a problem hiding this comment.
Suggested change
| // Licensed under the MIT License - https://github.com/deepseek-ai/DeepEP/blob/main/LICENSE | |
| // Licensed under the MIT License - https://github.com/deepseek-ai/DeepGEMM/blob/main/LICENSE |
后同
| // TODO: remove some useless computation for unaligned Ms | ||
| #pragma unroll | ||
| for (uint32_t local_idx = 0; local_idx < BLOCK_M / WAVE_BLOCK_M; ++ local_idx) { | ||
| auto m_offset = local_idx * WAVE_BLOCK_M; |
There was a problem hiding this comment.
为什么 fp8 目录没有 __init__.py？这样 fp8 目录不是一个 module，有潜在的 import 问题
另外，fp8 目录只有 deep_gemm 一个子目录是后续给其他模块预留的么？
Comment on lines
112
to
115
| # keys = {k: keys[k] for k in sorted(keys.keys())} | ||
| # signature = (name, f"{keys}") | ||
| # if signature in self.tuned: | ||
| # return self.tuned[signature] |
test/fp8/test_fp8_deep_gemm.py
Outdated
Comment on lines
86
to
89
| # x_np = np.full((num_groups, m, k), 3) | ||
| # y_np = np.full((num_groups, n, k), 2) | ||
| # x=paddle.to_tensor(x_np).astype(paddle.bfloat16) | ||
| # y=paddle.to_tensor(y_np).astype(paddle.bfloat16) |
Comment on lines
+223
to
+217
| test_gemm() | ||
| test_m_grouped_gemm_contiguous() | ||
| test_m_grouped_gemm_masked() |
python/paddle/incubate/__init__.py
Outdated
| 'gemm_fp8_fp8_bf16_nt', | ||
| 'm_grouped_gemm_fp8_fp8_bf16_nt_contiguous', | ||
| 'm_grouped_gemm_fp8_fp8_bf16_nt_masked', | ||
| ] |
There was a problem hiding this comment.
确认下，这些是公开 API 么？这些如果是公开 API，那就需要为每个 API
- 完善类型提示
- 完善文档
另外，如果是公开 API，目前这里也是不可行的，因为编译文档的环境这里应该不满足 cuda_version >= 12.9 and 90 in paddle.version.cuda_archs() 吧？这会导致官网无法渲染相关文档
python/paddle/incubate/__init__.py
Outdated
| m_grouped_gemm_fp8_fp8_bf16_nt_contiguous, | ||
| m_grouped_gemm_fp8_fp8_bf16_nt_masked, | ||
| ) | ||
| except ImportError: |
risemeup1
force-pushed
the
support_fp8_deep_gemm_in_paddle
branch
2 times, most recently
from
cb0cabb to
11654d3
Compare
risemeup1
force-pushed
the
support_fp8_deep_gemm_in_paddle
branch
from
11654d3 to
03c2430
Compare
SigureMo
approved these changes
Jun 5, 2025
Hongqing-work
approved these changes
Jun 5, 2025
zhangbo9674
approved these changes
Jun 5, 2025
|
/re-run approval |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Category
Execute Infrastructure
PR Types
Others
Description
支持paddle接入fp8 deepgemm
pcard-67164