KEMBAR78
Issue in python binding for InferenceRequest for GptMananger · Issue #528 · NVIDIA/TensorRT-LLM · GitHub
Skip to content

Issue in python binding for InferenceRequest for GptMananger #528

@bfontain

Description

@bfontain

Hi,

Currently streaming with the python bindings for GptManager doesn't work as there is a bug in the python binding for InferenceRequest: toTrtLlm() is missing copying the streaming property from the binding object to the native InferenceRequest object.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions