[EP ABI] Check if nodes specified in GetCapability() have already been assigned #26156

adrianlizarraga · 2025-09-25T00:53:51Z

Description

Fixes segfault in PluginExecutionProvider::GetCapability() when the underlying OrtEp tries to claim nodes that have already been assigned to another EP.

Motivation and Context

Should log a warning (instead of crashing or throwing an exception) when a plugin EP tries to claim a node that is already assigned to another EP.

…d to other EP

onnxruntime/core/session/plugin_ep/ep_plugin_provider_interfaces.cc

onnxruntime/test/framework/ep_plugin_provider_test.cc

premature

…es.cc Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>

onnxruntime/core/session/plugin_ep/ep_plugin_provider_interfaces.cc

onnxruntime/test/framework/ep_plugin_provider_test.cc

…, continue processing

onnxruntime/core/session/plugin_ep/ep_plugin_provider_interfaces.cc

onnxruntime/test/framework/ep_plugin_provider_test.cc

…other EP. Otherwise, if need to log an error, exit immediately

skottmckay

onnxruntime/core/session/plugin_ep/ep_plugin_provider_interfaces.cc

snnn

Thanks

…n assigned (#26156) ### Description Fixes segfault in `PluginExecutionProvider::GetCapability()` when the underlying `OrtEp` tries to claim nodes that have already been assigned to another EP. ### Motivation and Context Should log a warning (instead of crashing or throwing an exception) when a plugin EP tries to claim a node that is already assigned to another EP. --------- Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>

### Description Adds the following commits to the `rel-1.23.1` branch for ORT 1.23.1: - add session_id_ to LogEvaluationStart/Stop, LogSessionCreationStart - main merge date: July 31, 1:05am - pr: #25590 - commit: e753643 - [build] fix WebAssembly build on macOS/arm64 - main merge date: Aug 5, 8:07am - pr: #25653 - commit: 53f152b - [CPU] MoE Kernel (#25958) - main merge date: Sept 10, 4:54pm - pr: #25958 - commit: 930e640 - [CPU] Block-wise QMoE kernel for CPU - main merge date: Sept 15, 8:32am - pr: #26009 - commit: 5d17734 - [C#] Implement missing APIs - main merge date: Sept 24, 10:50am - pr: #26101 - commit: 35dcab5 - Regenerate test model with ONNX IR < 12 - main merge date: Sept 24, 2:50pm - pr: #26149 - commit: 88f2652 - [CPU] Fix compilation errors because of unused variables - main merge date: Sept 25, 1:21pm - pr: #26147 - commit: 42fcd71 - [EP ABI] Check if nodes specified in GetCapability() have already been assigned - main merge date: Sept 26, 1:24am - pr: #26156 - commit: 67d3ba0 - [QNN EP] Add dynamic option to set HTP performance mode - main merge date: Sept 26, 11:55am - pr: #26135 - commit: 6cc40fd --------- Co-authored-by: xieofxie <xieofxie@126.com> Co-authored-by: hualxie <hualxie@microsoft.com> Co-authored-by: Yulong Wang <7679871+fs-eire@users.noreply.github.com> Co-authored-by: Akshay Sonawane <111780983+apsonawane@users.noreply.github.com> Co-authored-by: Dmitri Smirnov <yuslepukhin@users.noreply.github.com> Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com> Co-authored-by: quic-tirupath <quic_tirupath@quicinc.com> Co-authored-by: quic-ashwshan <quic_ashwshan@quicinc.com>

snnn · 2025-09-27T03:29:39Z

This PR has been cherry-picked into the rel-1.23.1 branch in PR #26182. Removing the release:1.23.1 label.

* ORT 1.23.1 cherrypick 1 [REDO] (microsoft#26140) ### Description Cherry-pick the following PRs into the ORT 1.23.1 branch: - Fix Attention GQA implementation on CPU - **MANUAL MERGE**: see microsoft#26057 - main merge date: Sept 15, 11:33am - pr: microsoft#25966 - commit: d530b29 - Address edge GetMemInfo edge cases - main merge date: Sept 16, 10:32am - pr: microsoft#26021 - commit: d251f3a - Implement new Python APIs - main merge date: Sept 17, 11:44am - pr: microsoft#25999 - commit: abc63e8 - MemcpyFromHost and MemcpyToHost support for plugin EPs - **MERGE CONFLICT** on file onnxruntime/test/optimizer/transpose_optimizer_test.cc. Conflicts with microsoft#25689 - main merge date: Sept 23, 10:42am - pr: microsoft#26088 - commit: 4545732 - [TRT RTX EP] Fix bug for generating the correct subgraph in GetCapability microsoft#26132 - main merge date: Sept 23, 8:54pm - pr: microsoft#26132 - commit: 72e56e7 ### Motivation and Context  --------- Co-authored-by: Dmitri Smirnov <yuslepukhin@users.noreply.github.com> Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com> Co-authored-by: Chi Lo <54722500+chilo-ms@users.noreply.github.com> * ORT 1.23.1 cherrypick 2 (microsoft#26182) ### Description Adds the following commits to the `rel-1.23.1` branch for ORT 1.23.1: - add session_id_ to LogEvaluationStart/Stop, LogSessionCreationStart - main merge date: July 31, 1:05am - pr: microsoft#25590 - commit: e753643 - [build] fix WebAssembly build on macOS/arm64 - main merge date: Aug 5, 8:07am - pr: microsoft#25653 - commit: 53f152b - [CPU] MoE Kernel (microsoft#25958) - main merge date: Sept 10, 4:54pm - pr: microsoft#25958 - commit: 930e640 - [CPU] Block-wise QMoE kernel for CPU - main merge date: Sept 15, 8:32am - pr: microsoft#26009 - commit: 5d17734 - [C#] Implement missing APIs - main merge date: Sept 24, 10:50am - pr: microsoft#26101 - commit: 35dcab5 - Regenerate test model with ONNX IR < 12 - main merge date: Sept 24, 2:50pm - pr: microsoft#26149 - commit: 88f2652 - [CPU] Fix compilation errors because of unused variables - main merge date: Sept 25, 1:21pm - pr: microsoft#26147 - commit: 42fcd71 - [EP ABI] Check if nodes specified in GetCapability() have already been assigned - main merge date: Sept 26, 1:24am - pr: microsoft#26156 - commit: 67d3ba0 - [QNN EP] Add dynamic option to set HTP performance mode - main merge date: Sept 26, 11:55am - pr: microsoft#26135 - commit: 6cc40fd --------- Co-authored-by: xieofxie <xieofxie@126.com> Co-authored-by: hualxie <hualxie@microsoft.com> Co-authored-by: Yulong Wang <7679871+fs-eire@users.noreply.github.com> Co-authored-by: Akshay Sonawane <111780983+apsonawane@users.noreply.github.com> Co-authored-by: Dmitri Smirnov <yuslepukhin@users.noreply.github.com> Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com> Co-authored-by: quic-tirupath <quic_tirupath@quicinc.com> Co-authored-by: quic-ashwshan <quic_ashwshan@quicinc.com> --------- Co-authored-by: Adrian Lizarraga <adlizarraga@microsoft.com> Co-authored-by: Dmitri Smirnov <yuslepukhin@users.noreply.github.com> Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com> Co-authored-by: Chi Lo <54722500+chilo-ms@users.noreply.github.com> Co-authored-by: xieofxie <xieofxie@126.com> Co-authored-by: hualxie <hualxie@microsoft.com> Co-authored-by: Yulong Wang <7679871+fs-eire@users.noreply.github.com> Co-authored-by: Akshay Sonawane <111780983+apsonawane@users.noreply.github.com> Co-authored-by: quic-tirupath <quic_tirupath@quicinc.com> Co-authored-by: quic-ashwshan <quic_ashwshan@quicinc.com>

adrianlizarraga added 4 commits September 24, 2025 17:52

Check if nodes specified in GetCapability() have already been assigne…

3e884f3

…d to other EP

Do not throw if only *some* nodes are already assigned to another EP

0d8bdf7

Add unit tests

e24a908

Update comment

cdf1052

adrianlizarraga requested review from edgchen1 and skottmckay September 25, 2025 07:22

adrianlizarraga marked this pull request as ready for review September 25, 2025 07:23

Add ORT_API_CALL

17ea760

edgchen1 reviewed Sep 25, 2025

View reviewed changes

onnxruntime/core/session/plugin_ep/ep_plugin_provider_interfaces.cc Outdated Show resolved Hide resolved

onnxruntime/test/framework/ep_plugin_provider_test.cc Outdated Show resolved Hide resolved

onnxruntime/test/framework/ep_plugin_provider_test.cc Outdated Show resolved Hide resolved

jywu-msft previously approved these changes Sep 25, 2025

View reviewed changes

Update onnxruntime/core/session/plugin_ep/ep_plugin_provider_interfac…

0010eab

…es.cc Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>

chilo-ms reviewed Sep 25, 2025

View reviewed changes

onnxruntime/core/session/plugin_ep/ep_plugin_provider_interfaces.cc Outdated Show resolved Hide resolved

adrianlizarraga added 2 commits September 25, 2025 11:33

Simplify testing and test more cases

3d54e3d

Add warning for kSingleAssignNode

03dcc93

adrianlizarraga requested review from chilo-ms and edgchen1 September 25, 2025 18:44

edgchen1 reviewed Sep 25, 2025

View reviewed changes

onnxruntime/core/session/plugin_ep/ep_plugin_provider_interfaces.cc Outdated Show resolved Hide resolved

Don't use captured structured bindings feature (C++20 ext)

3dc9942

edgchen1 reviewed Sep 25, 2025

View reviewed changes

onnxruntime/test/framework/ep_plugin_provider_test.cc Show resolved Hide resolved

adrianlizarraga added 2 commits September 25, 2025 12:34

Address comments: check assigned EP in case for AddSingleNode

9a473dc

When have to log an error, return immediately. When logging a warning…

c7c7e59

…, continue processing

edgchen1 reviewed Sep 25, 2025

View reviewed changes

onnxruntime/core/session/plugin_ep/ep_plugin_provider_interfaces.cc Show resolved Hide resolved

edgchen1 reviewed Sep 25, 2025

View reviewed changes

onnxruntime/test/framework/ep_plugin_provider_test.cc Outdated Show resolved Hide resolved

adrianlizarraga added 2 commits September 25, 2025 17:04

Consistency: skip node grouping if any node is already assigned to an…

369dcd3

…other EP. Otherwise, if need to log an error, exit immediately

remove unused code now

8782eb6

skottmckay approved these changes Sep 26, 2025

View reviewed changes

edgchen1 reviewed Sep 26, 2025

View reviewed changes

onnxruntime/core/session/plugin_ep/ep_plugin_provider_interfaces.cc Show resolved Hide resolved

edgchen1 approved these changes Sep 26, 2025

View reviewed changes

adrianlizarraga added the release:1.23.1 label Sep 26, 2025

snnn approved these changes Sep 26, 2025

View reviewed changes

adrianlizarraga merged commit 67d3ba0 into main Sep 26, 2025
102 of 104 checks passed

adrianlizarraga deleted the adrianl/ep-get-capability-handle-preassigned-nodes branch September 26, 2025 08:24

adrianlizarraga mentioned this pull request Sep 26, 2025

ORT 1.23.1 cherrypick 2 #26182

Merged

snnn removed the release:1.23.1 label Sep 27, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[EP ABI] Check if nodes specified in GetCapability() have already been assigned #26156

[EP ABI] Check if nodes specified in GetCapability() have already been assigned #26156

adrianlizarraga commented Sep 25, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

skottmckay left a comment

Uh oh!

Uh oh!

snnn left a comment •

edited

Loading

Uh oh!

Uh oh!

snnn commented Sep 27, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

[EP ABI] Check if nodes specified in GetCapability() have already been assigned #26156

[EP ABI] Check if nodes specified in GetCapability() have already been assigned #26156

Conversation

adrianlizarraga commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

skottmckay left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

snnn left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

snnn commented Sep 27, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

adrianlizarraga commented Sep 25, 2025 •

edited

Loading

snnn left a comment •

edited

Loading