tensorrt_llm/evaluate/interface.py (1)

37-39: Use typing.Dict for 3.8 compatibility; pass kwargs safely.

Project targets Python 3.8+. Replace PEP 585 generics with typing.Dict to avoid 3.9+ requirement. Logic forwarding chat_template_kwargs looks good.

Apply this diff:

-from typing import Any, Iterable, List, Optional, Union
+from typing import Any, Iterable, List, Optional, Union, Dict
@@
-                 system_prompt: Optional[str] = None,
-                 chat_template_kwargs: Optional[dict[str, Any]] = None):
+                 system_prompt: Optional[str] = None,
+                 chat_template_kwargs: Optional[Dict[str, Any]] = None):
@@
-        return llm.tokenizer.apply_chat_template(messages,
-                                                 tokenize=False,
-                                                 add_generation_prompt=True,
-                                                 **(self.chat_template_kwargs
-                                                    or {}))
+        return llm.tokenizer.apply_chat_template(
+            messages,
+            tokenize=False,
+            add_generation_prompt=True,
+            **(self.chat_template_kwargs or {}),
+        )

As per coding guidelines

Also applies to: 45-46, 69-71

tensorrt_llm/evaluate/mmlu.py (1)

24-24: Adopt Dict for Python 3.8; improve JSON option parsing (lint-safe, better errors).

• Replace dict[...] with typing.Dict for 3.8 compatibility.
• The lambda callback triggers ARG005 and gives poor error messages on bad JSON. Use a small parser function and raise click.BadParameter.

Apply this diff:

+import json
@@
-from typing import Any, Iterable, List, Optional, Union
+from typing import Any, Iterable, List, Optional, Union, Dict
@@
-                 system_prompt: Optional[str] = None,
-                 chat_template_kwargs: Optional[dict[str, Any]] = None):
+                 system_prompt: Optional[str] = None,
+                 chat_template_kwargs: Optional[Dict[str, Any]] = None):
@@
-    @click.option(
-        "--chat_template_kwargs",
-        type=str,
-        default=None,
-        callback=lambda ctx, param, value: json.loads(value) if value else None,
-        help=
-        'Chat template kwargs as JSON string, e.g., \'{"thinking_budget": 0}\'')
+    def _parse_json_option(ctx, param, value):
+        if not value:
+            return None
+        try:
+            return json.loads(value)
+        except json.JSONDecodeError as e:
+            raise click.BadParameter(f"Invalid JSON: {e.msg}") from e
+    @click.option(
+        "--chat_template_kwargs",
+        type=str,
+        default=None,
+        callback=_parse_json_option,
+        help='Chat template kwargs as JSON string, e.g., \'{"thinking_budget": 0}\'')
@@
-                         system_prompt=system_prompt,
-                         chat_template_kwargs=chat_template_kwargs)
+                         system_prompt=system_prompt,
+                         chat_template_kwargs=chat_template_kwargs)

As per coding guidelines

Also applies to: 38-38, 141-147, 303-309, 335-342

tensorrt_llm/evaluate/lm_eval.py (2)

52-61: Use typing.Dict for Python 3.8 compatibility across constructors.

Replace PEP 585 dict[...] with typing.Dict to keep 3.8 support.

Apply this diff:

-                 streaming: bool = False,
-                 chat_template_kwargs: Optional[dict[str, Any]] = None):
+                 streaming: bool = False,
+                 chat_template_kwargs: Optional[Dict[str, Any]] = None):
@@
-                 max_images: int = 999,
-                 chat_template_kwargs: Optional[dict[str, Any]] = None):
+                 max_images: int = 999,
+                 chat_template_kwargs: Optional[Dict[str, Any]] = None):
@@
-                 is_multimodal: bool = False,
-                 chat_template_kwargs: Optional[dict[str, Any]] = None):
+                 is_multimodal: bool = False,
+                 chat_template_kwargs: Optional[Dict[str, Any]] = None):

As per coding guidelines

Also applies to: 148-169, 303-332

---

480-485: Prefer a shared JSON parser over inline lambdas (fixes lint, better UX).

Define a single _parse_json_option (returns None or parsed dict; raises click.BadParameter on invalid JSON) and reuse it for all --chat_template_kwargs options to avoid ARG005 and improve errors.

I can provide a consolidated diff to add the helper and update these options if desired.

Also applies to: 538-544, 588-594, 638-644, 684-689

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro