[TRTLLM-8044][refactor] Rename data -> cache for cacheTransceiver #7659

Tabrizian · 2025-09-09T18:31:58Z

Refactor dataTransceiver classes

Summary by CodeRabbit

New Features
- Added multi-connection KV-cache transfer sessions with optional metric collection/export.
- Introduced asynchronous send/receive operations for cache transfers to improve throughput.
Refactor
- Streamlined cache transfer APIs and internal components for clearer responsibilities and improved stability.
- Harmonized naming in agent connection settings for clarity (non-breaking).
Tests
- Updated and expanded unit tests to cover the new cache transfer flows.
Chores
- Removed obsolete implementations and includes; updated build configuration to reflect the new architecture.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

run [--reuse-test (optional)pipeline-id --disable-fail-fast --skip-test --stage-list "A10-PyTorch-1, xxx" --gpu-type "A30, H100_PCIe" --test-backend "pytorch, cpp" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" --detailed-log --debug(experimental)]

Launch build/test pipelines. All previously running jobs will be killed.

--reuse-test (optional)pipeline-id (OPTIONAL) : Allow the new pipeline to reuse build artifacts and skip successful test stages from a specified pipeline or the last pipeline if no pipeline-id is indicated. If the Git commit ID has changed, this option will be always ignored. The DEFAULT behavior of the bot is to reuse build artifacts and successful test results from the last pipeline.

--disable-reuse-test (OPTIONAL) : Explicitly prevent the pipeline from reusing build artifacts and skipping successful test stages from a previous pipeline. Ensure that all builds and tests are run regardless of previous successes.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-PyTorch-1, xxx" (OPTIONAL) : Only run the specified test stages. Examples: "A10-PyTorch-1, xxx". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--test-backend "pytorch, cpp" (OPTIONAL) : Skip test stages which don't match the specified backends. Only support [pytorch, cpp, tensorrt, triton]. Examples: "pytorch, cpp" (does not run test stages with tensorrt or triton backend). Note: Does NOT update GitHub pipeline status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests in addition to running L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx".

--detailed-log (OPTIONAL) : Enable flushing out all logs to the Jenkins console. This will significantly increase the log volume and may slow down the job.

--debug (OPTIONAL) : Experimental feature. Enable access to the CI container for debugging purpose. Note: Specify exactly one stage in the stage-list parameter to access the appropriate container environment. Note: Does NOT update GitHub check status.

For guidance on mapping tests to stage names, see docs/source/reference/ci-overview.md
and the scripts/test_to_stage_mapping.py helper.

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

Tabrizian · 2025-09-09T18:33:41Z

/bot run --disable-fail-fast

tensorrt-cicd · 2025-09-09T18:43:52Z

PR_Github #18260 [ run ] triggered by Bot

coderabbitai · 2025-09-09T18:44:51Z

📝 Walkthrough

Walkthrough

Refactors batch-manager data transfer from DataResponder/DataRequester to CacheSender/CacheReceiver, removes DataTransceiverImpl, updates headers and includes, adds TransferSession and measurement utilities in cacheFormatter, adjusts AgentConnection buffer naming, switches UCX include to new transceiver header, and updates unit tests accordingly. CMake no longer builds dataTransceiverImpl.cpp.

Changes

Cohort / File(s)	Summary
Batch manager public API overhaul `cpp/tensorrt_llm/batch_manager/dataTransceiver.h`, `cpp/tensorrt_llm/batch_manager/cacheFormatter.h`	Replaces DataSender/DataReceiver/DataRequester/DataResponder with CacheSender/CacheReceiver; introduces TransferSession alias to kv_cache_manager, TransceiverTag, SizeType32, BaseCacheFormatter alias; adds new public APIs (sendAsync/receiveAsync, request-info exchange); adds measurement helpers and multi-connection TransferSession in cacheFormatter.
Batch manager implementation refactor `cpp/tensorrt_llm/batch_manager/dataTransceiver.cpp`, `cpp/tensorrt_llm/batch_manager/cacheTransceiver.cpp`	Reworks internals to use ConnectionManager and CacheSender/CacheReceiver; updates async send/recv paths, comm-state sourcing, and future containers; removes includes of old impl; aligns naming.
Removal of legacy impl `cpp/tensorrt_llm/batch_manager/dataTransceiverImpl.cpp`, `cpp/tensorrt_llm/batch_manager/dataTransceiverImpl.h`, `cpp/tensorrt_llm/batch_manager/CMakeLists.txt`	Deletes DataTransceiverImpl sources/headers and removes them from build.
CacheTransceiver header alignment `cpp/include/tensorrt_llm/batch_manager/cacheTransceiver.h`	Forward decls and private members renamed to CacheSender/CacheReceiver; futures container renamed. No public signature changes.
Executor agent connection rename `cpp/tensorrt_llm/executor/cache_transmission/agent_utils/connection.h`, `cpp/tensorrt_llm/executor/cache_transmission/agent_utils/connection.cpp`	Renames sender-state buffer field and parameter from mReceiverBufferDesc to mCacheReceiverBufferDesc; adjusts send to use new member.
Executor UCX include switch `cpp/tensorrt_llm/executor/cache_transmission/ucx_utils/connection.cpp`	Replaces include of dataTransceiverImpl.h with dataTransceiver.h.
Tests update `cpp/tests/unit_tests/multi_gpu/cacheTransceiverTest.cpp`, `cpp/tests/unit_tests/executor/ucxCommTest.cpp`	Updates tests to CacheSender/CacheReceiver, renames mocks and members, removes impl header includes.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant R as Receiver (CacheReceiver)
  participant S as Sender (CacheSender)
  participant M as ConnectionManager
  participant F as Formatter
  participant TS as TransferSession

  rect rgba(230,240,255,0.6)
  note over R,S: Request-info exchange (new)
  R->>M: sendRequestInfo(llmRequest)
  activate M
  M-->>S: deliver RequestInfo
  deactivate M
  S->>S: recvRequestInfo()
  S->>TS: create TransferSession(connections, DataContext,...)
  end

  rect rgba(235,255,235,0.6)
  note over S,R: Data transfer phase
  par Async send
    S->>R: data chunks via connections
  and Receive orchestration
    R->>TS: receiveSync(TS)
    S->>TS: sendSync(llmRequest)
  end
  end

  rect rgba(255,245,230,0.6)
  note over S: Completion/cleanup
  S->>S: update CommState / release session
  R->>R: finalize buffers/events
  end

sequenceDiagram
  autonumber
  participant BM as CacheTransceiver
  participant CS as CacheSender
  participant CR as CacheReceiver
  participant FQ as mSenderFutures

  BM->>CS: sendAsync(req)
  activate CS
  CS-->>BM: std::future<void>
  deactivate CS
  BM->>FQ: store future (mSenderFutures)

  BM->>CR: receiveAsync(req)
  activate CR
  CR-->>BM: std::future<void>
  deactivate CR

  loop poll
    BM->>FQ: checkContextTransferStatus()
    FQ-->>BM: completed/errored futures
    BM->>FQ: erase completed
  end

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Tip

👮 Agentic pre-merge checks are now available in preview!

Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.

Built-in checks – Quickly apply ready-made checks to enforce title conventions, require pull request descriptions that follow templates, validate linked issues for compliance, and more.
Custom agentic checks – Define your own rules using CodeRabbit’s advanced agentic capabilities to enforce organization-specific policies and workflows. For example, you can instruct CodeRabbit’s agent to verify that API documentation is updated whenever API schema files are modified in a PR. Note: Upto 5 custom checks are currently allowed during the preview period. Pricing for this feature will be announced in a few weeks.

Example:

reviews:
  pre_merge_checks:
    custom_checks:
      - name: "Undocumented Breaking Changes"
        mode: "warning"
        instructions: |
          Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).

Please share your feedback with us on this Discord post.

Pre-merge checks (3 warnings)

❌ Failed checks (3 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 4.23% which is insufficient. The required threshold is 80.00%.	You can run `@coderabbitai generate docstrings` to improve docstring coverage.
Title Check	⚠️ Warning	The current title "[TRTLLM-8044][refactor] Rename data -> cache for cacheTransceiver" only highlights renaming within the cacheTransceiver component, but this pull request renames data → cache across all DataTransceiver classes and updates related APIs and public interfaces, so it does not concisely summarize the main change.	Please revise the title to succinctly reflect the overall refactor of dataTransceiver classes—including renaming DataResponder/DataRequester to CacheSender/CacheReceiver and updating the public API—such as “[TRTLLM-8044][refactor] Rename DataTransceiver components to Cache* and update API.”
Description Check	⚠️ Warning	The pull request description retains the template guidance and headings but leaves the Description and Test Coverage sections empty, so it does not explain the issue, the implementation details, or the relevant tests that verify the changes.	Please fill in the Description section with a concise summary of the refactoring changes and their rationale, populate the Test Coverage section with the specific unit and integration tests that safeguard the new functionality, and remove or replace the unused template instructions.

✨ Finishing Touches

📝 Generate Docstrings

🧪 Generate unit tests

Create PR with unit tests
Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)

cpp/tensorrt_llm/executor/cache_transmission/agent_utils/connection.h (1)

187-193: Initialize mOffsetRatio to a safe default to avoid div-by-zero in send()

send() divides by mOffsetRatio.second; a default-initialized pair risks 0. Initialize to {0, 1}.

     struct SenderState
     {
-        MemoryDesc mCacheReceiverBufferDesc{nullptr, 0, 0};
-        int validSegmentIdx{0};
-        std::pair<size_t, size_t> mOffsetRatio;
+        MemoryDesc mCacheReceiverBufferDesc{nullptr, 0, 0};
+        int validSegmentIdx{0};
+        std::pair<size_t, size_t> mOffsetRatio{0, 1};
         SenderState() = default;
     };

cpp/tensorrt_llm/executor/cache_transmission/agent_utils/connection.cpp (1)

107-113: Precondition checks before offset computation (avoid UB and OOB writes)

Guard against zero denominator and ensure the computed region fits in dst buffer.

-    auto dstBaseDesc = mSenderState.mCacheReceiverBufferDesc;
-    auto offset = size / mSenderState.mOffsetRatio.second * mSenderState.mOffsetRatio.first;
+    auto dstBaseDesc = mSenderState.mCacheReceiverBufferDesc;
+    TLLM_CHECK_WITH_INFO(mSenderState.mOffsetRatio.second != 0, "Invalid offset ratio: denominator is 0. Call setSenderState() first.");
+    auto offset = size / mSenderState.mOffsetRatio.second * mSenderState.mOffsetRatio.first;
+    TLLM_CHECK_WITH_INFO(offset <= dstBaseDesc.getLen(), "Computed offset exceeds destination buffer length.");
+    TLLM_CHECK_WITH_INFO(size <= (dstBaseDesc.getLen() - offset), "Transfer size overflows destination buffer.");

cpp/tensorrt_llm/batch_manager/cacheTransceiver.cpp (1)

170-176: Windows build break: dlerror() is POSIX-only

This TLLM_CHECK_WITH_INFO uses dlerror() even on _WIN32 path. Use GetLastError() on Windows, dlerror() elsewhere.

-        TLLM_CHECK_WITH_INFO(
-            mWrapperLibHandle != nullptr, "UCX wrapper library is not open correctly. error : %s", dlerror());
+    #if defined(_WIN32)
+        TLLM_CHECK_WITH_INFO(
+            mWrapperLibHandle != nullptr, "UCX wrapper library failed to open. error: %lu", GetLastError());
+    #else
+        TLLM_CHECK_WITH_INFO(
+            mWrapperLibHandle != nullptr, "UCX wrapper library failed to open. error: %s", dlerror());
+    #endif

🧹 Nitpick comments (13)

cpp/tensorrt_llm/executor/cache_transmission/agent_utils/connection.cpp (1)

165-170: Parameter naming nit: avoid member-style prefix on parameters

The parameter mCacheReceiverBufferDesc uses the member-style 'm' prefix; prefer cacheReceiverBufferDesc for parameters. Consider also passing by const&.
cpp/tensorrt_llm/executor/cache_transmission/ucx_utils/connection.cpp (3)
117-121: Avoid magic 0xFFFFFFFF tag mask; introduce a named constant

Replace repeated 0xFFFFFFFF with a file-scope constexpr per guidelines.
@@
-#include "tensorrt_llm/executor/cache_transmission/ucx_utils/connection.h"
+#include "tensorrt_llm/executor/cache_transmission/ucx_utils/connection.h"
+
+namespace {
+constexpr uint64_t kTAG_MASK = 0xFFFFFFFFull;
+} // namespace
@@
-    uint64_t tag
-        = ((mSendTagPrefix & 0xFFFFFFFF) << 32) | static_cast<uint64_t>(batch_manager::TransceiverTag::kID_TAG);
+    uint64_t tag = ((mSendTagPrefix & kTAG_MASK) << 32)
+        | (static_cast<uint64_t>(batch_manager::TransceiverTag::kID_TAG) & kTAG_MASK);
@@
-    uint64_t sendTag = ((mSendTagPrefix & 0xFFFFFFFF) << 32) | (static_cast<uint64_t>(ctx.getTag()) & (0xFFFFFFFF));
+    uint64_t sendTag = ((mSendTagPrefix & kTAG_MASK) << 32) | (static_cast<uint64_t>(ctx.getTag()) & kTAG_MASK);
@@
-    uint64_t recvTag = ((mRecvTagPrefix & 0xFFFFFFFF) << 32) | (static_cast<uint64_t}(ctx.getTag()) & (0xFFFFFFFF));
+    uint64_t recvTag = ((mRecvTagPrefix & kTAG_MASK) << 32) | (static_cast<uint64_t>(ctx.getTag()) & kTAG_MASK);
Also applies to: 149-151, 174-176

49-61: Avoid tight busy-wait loops on UCX requests

The while (!req->isCompleted()) spin-loops can burn CPU. Add a small backoff or yield.
-            while (!recvRequest->isCompleted())
-                ;
+            while (!recvRequest->isCompleted())
+            {
+                std::this_thread::yield();
+            }
@@
-            while (!sendRequest->isCompleted())
-                ;
+            while (!sendRequest->isCompleted())
+            {
+                std::this_thread::yield();
+            }
@@
-            while (!sendRequest->isCompleted())
-                ;
+            while (!sendRequest->isCompleted())
+            {
+                std::this_thread::yield();
+            }
@@
-            while (!recvRequest->isCompleted())
-                ;
+            while (!recvRequest->isCompleted())
+            {
+                std::this_thread::yield();
+            }
Also applies to: 69-79

92-95: Portable logging format for 64-bit IDs

If mConnectionId types are 64-bit, %lu is non-portable (Windows long is 32-bit). Prefer PRIu64 or cast to unsigned long long and use %llu.
- "..., mConnectionId: %lu, mConnectionIdInPeer: %lu,fromRequester: %d", mConnectionId, mConnectionIdInPeer, mFromRequester);
+ "..., mConnectionId: %llu, mConnectionIdInPeer: %llu,fromRequester: %d",
+ static_cast<unsigned long long>(mConnectionId),
+ static_cast<unsigned long long>(mConnectionIdInPeer),
+ mFromRequester);
Apply similarly to the other TLLM_LOG_DEBUG lines in this file.

Also applies to: 100-104, 109-113, 129-132, 142-144, 159-162, 168-171, 183-186
cpp/include/tensorrt_llm/batch_manager/cacheTransceiver.h (1)
1-18: Header hygiene: use include guards per guidelines and update year

Replace #pragma once with a TRTLLM_CACHETRANSCEIVER_H guard and update the header year to 2025.
-#pragma once
+#ifndef TRTLLM_CACHETRANSCEIVER_H
+#define TRTLLM_CACHETRANSCEIVER_H
@@
- * Copyright (c) 2023-2024, NVIDIA CORPORATION.  All rights reserved.
+ * Copyright (c) 2023-2025, NVIDIA CORPORATION.  All rights reserved.
@@
-#include <memory>
+#include <memory>
+
+#endif // TRTLLM_CACHETRANSCEIVER_H
Note: place the #endif at the end of the file.
cpp/tensorrt_llm/batch_manager/cacheTransceiver.cpp (1)
23-35: Prefer constexpr over macro for library name

Replace UCX_WRAPPER_LIB_NAME macro with a constexpr per guidelines.
-#define UCX_WRAPPER_LIB_NAME "tensorrt_llm_ucx_wrapper"
+namespace {
+constexpr char kUCX_WRAPPER_LIB_NAME[] = "tensorrt_llm_ucx_wrapper";
+} // namespace
@@
-        mWrapperLibHandle = dllOpen(UCX_WRAPPER_LIB_NAME);
+        mWrapperLibHandle = dllOpen(kUCX_WRAPPER_LIB_NAME);
cpp/tests/unit_tests/multi_gpu/cacheTransceiverTest.cpp (1)
160-160: Naming inconsistency: Prefer camel case for class names.

The class name should follow the CamelCase convention as specified in the coding guidelines. Consider renaming to MockCacheSender.
-class MockCacheSender : public CacheSender
+class MockCacheSender : public CacheSender
cpp/tensorrt_llm/batch_manager/dataTransceiver.cpp (1)
705-705: Minor: Inconsistent format specifier in log message.

The log message uses %ld for request IDs which should be the appropriate format for the actual type.
-TLLM_LOG_ERROR("Exception in CacheReceiver request(): request id:%ld , request context id:%ld : %s",
+TLLM_LOG_ERROR("Exception in CacheReceiver request(): request id:%zu , request context id:%zu : %s",
cpp/tensorrt_llm/batch_manager/cacheFormatter.h (1)
235-235: Consider adding error handling for rank retrieval.

The MPI rank retrieval could potentially fail in non-MPI environments.
-auto rank = mpi::MpiComm::world().getRank();
+auto rank = 0;
+if (mpi::MpiComm::world().isInitialized())
+{
+    rank = mpi::MpiComm::world().getRank();
+}
cpp/tensorrt_llm/batch_manager/dataTransceiver.h (4)
18-34: Replace #pragma once with required include guard and trim unused/heavy includes

Guidelines require explicit include guards. Also, several headers appear unused in this interface header; prefer lighter includes to reduce compile-time coupling.

Apply:
-#pragma once
+#ifndef TRTLLM_DATATRANSCEIVER_H
+#define TRTLLM_DATATRANSCEIVER_H
-#include <fstream>
+#include <iosfwd>
 #include <future>
-#include <map>
 #include <string>
 
-#include "tensorrt_llm/batch_manager/cacheFormatter.h"
 #include "tensorrt_llm/batch_manager/llmRequest.h"
-#include "tensorrt_llm/common/envUtils.h"
-#include "tensorrt_llm/common/logger.h"
 #include "tensorrt_llm/executor/cacheCommunicator.h"
 #include "tensorrt_llm/executor/dataTransceiverState.h"
-#include "tensorrt_llm/executor/serializeUtils.h"
-#include "tensorrt_llm/runtime/cudaEvent.h"
-#include "tensorrt_llm/runtime/utils/mpiUtils.h"
+#include "tensorrt_llm/batch_manager/cacheFormatter.h"
And add at EOF:
+ #endif // TRTLLM_DATATRANSCEIVER_H
119-128: Align Doxygen style with repo convention

Guidelines call for //! for single-line docs and //!< for members. Consider switching from /// @brief for consistency.

Example:
-    /// @brief Asynchronously respond to the request and send data.
+    //! Asynchronously respond to the request and send data.
Also applies to: 137-144, 149-154

172-177: Add brief Doxygen for sendRequestInfo/receiveSync

These two public methods lack doc comments while others have them. Add short one-liners for consistency.
-    TransferSession sendRequestInfo(LlmRequest const& llmRequest);
+    //! Send request metadata and acquire a transfer session for receive side.
+    TransferSession sendRequestInfo(LlmRequest const& llmRequest);
 
-    void receiveSync(TransferSession& session);
+    //! Blocking receive that unformats data into llmRequest via session.
+    void receiveSync(TransferSession& session);
38-41: Track TODO: consider moving transport aliases/types into a dedicated namespace

Open an issue to plan the proposed tensorrt_llm::transmission namespace so we don’t leave a lingering TODO in public headers.

Want me to file a tracking issue with a lightweight migration plan?

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d49374b and b685dfe.

📒 Files selected for processing (13)

cpp/include/tensorrt_llm/batch_manager/cacheTransceiver.h (2 hunks)
cpp/tensorrt_llm/batch_manager/CMakeLists.txt (0 hunks)
cpp/tensorrt_llm/batch_manager/cacheFormatter.h (3 hunks)
cpp/tensorrt_llm/batch_manager/cacheTransceiver.cpp (8 hunks)
cpp/tensorrt_llm/batch_manager/dataTransceiver.cpp (16 hunks)
cpp/tensorrt_llm/batch_manager/dataTransceiver.h (3 hunks)
cpp/tensorrt_llm/batch_manager/dataTransceiverImpl.cpp (0 hunks)
cpp/tensorrt_llm/batch_manager/dataTransceiverImpl.h (0 hunks)
cpp/tensorrt_llm/executor/cache_transmission/agent_utils/connection.cpp (2 hunks)
cpp/tensorrt_llm/executor/cache_transmission/agent_utils/connection.h (2 hunks)
cpp/tensorrt_llm/executor/cache_transmission/ucx_utils/connection.cpp (1 hunks)
cpp/tests/unit_tests/executor/ucxCommTest.cpp (0 hunks)
cpp/tests/unit_tests/multi_gpu/cacheTransceiverTest.cpp (11 hunks)

💤 Files with no reviewable changes (4)

cpp/tests/unit_tests/executor/ucxCommTest.cpp
cpp/tensorrt_llm/batch_manager/dataTransceiverImpl.cpp
cpp/tensorrt_llm/batch_manager/CMakeLists.txt
cpp/tensorrt_llm/batch_manager/dataTransceiverImpl.h

🧰 Additional context used

📓 Path-based instructions (7)

**/*.{h,hpp,hh,hxx,cpp,cxx,cc,cu,cuh}