Refactor the way we handle outputs for new llamas and new models #39120

ArthurZucker · 2025-06-30T09:28:05Z

What does this PR do?

Fixes # (issue)

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline,
Pull Request section?
Was this discussed/approved via a Github issue or the forum? Please add a link
to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

HuggingFaceDocBuilderDev · 2025-06-30T12:36:03Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

LysandreJik

Okay looks great!

vasqu

Just a few smaller things that I noticed

vasqu · 2025-06-30T14:54:10Z

src/transformers/utils/generic.py

+    cumulative_seqlens_q: Optional["torch.LongTensor"]
+    cumulative_seqlens_k: Optional["torch.LongTensor"]


We should keep it consistent with the flash attention naming imo even if it's less explicit, see

transformers/src/transformers/modeling_flash_attention_utils.py

Lines 524 to 525 in ea0ea39

cu_seqlens_q=cu_seqlens_q,

cu_seqlens_k=cu_seqlens_k,

vasqu · 2025-06-30T15:00:18Z

src/transformers/utils/generic.py

+        return_dict = self.config.use_return_dict if hasattr(self, "config") else True
+        if "return_dict" in kwargs:
+            return_dict = kwargs.get("return_dict", self.config.use_return_dict)


Suggested change

return_dict = self.config.use_return_dict if hasattr(self, "config") else True

if "return_dict" in kwargs:

return_dict = kwargs.get("return_dict", self.config.use_return_dict)

return_dict = self.config.use_return_dict if hasattr(self, "config") else False

if "return_dict" in kwargs:

return_dict = kwargs.get("return_dict", return_dict)

Not sure about the default value but the default on get could cause issues if the config is None (which shouldnt happen but you never know)

src/transformers/utils/generic.py

src/transformers/models/llama/modeling_llama.py

src/transformers/utils/generic.py

chenhengqi · 2025-08-13T01:10:00Z

See https://huggingface.co/microsoft/Phi-4-mini-instruct/blob/main/modeling_phi3.py#L38

* initial comment * test * initial conversion for outline * intermediate commit for configuration * chore:init files for sam2 * adding arbitary undefined config * check * add vision * make style * init sam2 base model * Fix imports * Linting * chore:sam to sam2 classes * Linting * Add sam2 to models.__init__ * chore:match prompt encoder with sam2 code * chore:prepare kwargs for mask decoder * Add image/video predictors * Add CUDA kernel * Add output classes * linting * Add logging info * tmp commit * docs for sam2 * enable image processing * check difference of original SAM2 - difference is the order of ToTensor() - please see https://pytorch.org/vision/main/_modules/torchvision/transforms/functional.html#resize * enable promptencoder of sam2 * fix promprencoder * Confirmed that PromptEncoder is exactly same (Be aware of bfloat16 and float32 difference) * Confirmed that ImageEncoder is exactly same (Be aware the linting of init) * Confirmed that MaskDecoder is exactly same (TO DO: lint variable name) * SamModel is now available (Need more chore for name) * make fix-copies * make style * make CI happy * Refactor VisionEncoder and PostioinEmbedding * TO DO : fix the image_embeddings and sparse_embeddings part * pure image inference done * reusable features fix and make style * styling * refactor memoryattention * tmp * tmp * refactor memoryencoder TO DO : convert and inference the video pipeline * TO DO : fix the image_encoder shape * conversion finish TO DO: need to check video inference * make style * remove video model * lint * change * python utils/check_docstringspy --check_all * python utils/check_config_attributes.py * remove copies for sam2promptencoder due to configuration * change __init__.py * remove tensorflow version * fix that to not use direct comparison * make style * add missing import * fix image_embedding_size * refactor Sam2 Attention * add fully working video inference (refactoring todo) * clarify _prepare_memory_conditioned_features * simplify modeling code, remove unused paths * use one model * use auto_docstring * refactor rope embeddings * nit * not using multimask when several points given * add all sam2.1 * add video tmp * add Sam2VideoSessionState + fast image proc + video proc * remove init_states from model * fix batch inference * add image integration tests * uniformize modeling code with other sam models and use modular * pass vision tests an most model tests * All tests passing * add offloading inference state and video to cpu * fix inference from image embedding and existing mask * fix multi_boxes mask inference * Fix batch images + batch boxes inference * improve processing for image inference * add support for mask generation pipeline * add support for get_connected_components post processing in mask generation * add fast image processor sam, image processor tests and use modular for sam2 image processor * fix mistake in sam after #39120 * fix init weights * refactor convert * add integration tests for video + other improvements * add needed missing docstrings * Improve docstrings and * improve inference speed by avoiding cuda sync * add test * skip test for vision_model * minor fix for vision_model * fix vision_model by adding sam2model and change the torch dependencies * remove patch_size * remove image_embedding_size * fix patch_size * fix test * make style * Separate hieradet and vision encoder in sam2 * fixup * review changes part 1 * remove MemoryEncoderConfig and MemoryAttentionConfig * pass q_stride instead of q_pool module * add inference on streamed videos * explicitely process streamed frames * nit * Improve docstrings in Sam2Model * update sam2 modeling with better gestion of inference state and cache, and separate Sam2Model and Sam2VideoModel * improve video inference api * change inference_state to inference_session * use modular for Sam2Model * fix convert sam2 hf * modular * Update src/transformers/models/sam2/video_processing_sam2.py Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * fix minor config * fix attention loading error * update modeling tests to use hub checkpoints * Use CI A10 runner for integration tests values + higher tolerance for video integration tests * PR review part 1 * fix doc * nit improvements * enforce one input format for points, labels and boxes * nit * last few nits from PR review * fix style * fix the input type * fix docs * add sam2 model as conversion script * improve sam2 doc * nit fixes + optimization * split sam2 and sam2_video in two models * PR review part 1 * fix None for default slow processor of sam2 * remove unecessary code path in sam2_video * refactor/simplify RoPE * replace embedding module list with embedding matrix * fix tests * remove kernel * nit * use lru_cache for sine_pos_embeddings * reorder sam2_video methods * simplify sam2_video * PR review part 1 * simplify sam2 video a lot * more simplification * update integration tests with updated conftest * more explicit config for hieradet * do post_processing outside of sam2 video model * Improve Sam2VideoVisionRotaryEmbedding * fix tests * update docs and fix mask2former/oneformer * avoid unnecessary reshapes/permute * fix device concatenating points * small dtype fix * PR review * nit * fix style and finish up doc * fix style * fix docstrings * fix modular --------- Co-authored-by: RUFFY-369 <prakarshkaushik369@gmail.com> Co-authored-by: Haitham Khedr <haithamkhedr@meta.com> Co-authored-by: sangbum choi <sangbumchoi@sangbumui-MacBookAir.local> Co-authored-by: yonigozlan <yoni.gozlan@huggingface.co> Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>

* unpin `torchcodec==0.5.0` and use `torch 2.8` on daily CI (#40072) fix Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * fix: resolve triton version check compatibility on windows (#39986) * fix: resolve triton version check compatibility on windows * style: remove trailing space * fix: fix typo --------- Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com> * [qwen-vl] fix beam search with videos (#39726) * fix * fix copies * [gemma3] update conversion key mapping (#39778) update conversion key mapping * fix: move super().__init__ after vision_config init in Mistral3Config (#40063) fix: move super().__init__ after vision_config init in Mistral3Config (#40062) * Remove deprecated cache-related objects (#40035) remove them * guard on model.eval when using torch.compile + FSDP2 (#37413) guard on model.eval Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> * Fix repo consistency (#40077) fix * added Textnet fast image processor (#39884) * feat: add fast image processor implementation for TextNet model * chore: override to_dict method to TextNetImageProcessorFast for slow processor compatibility tests * chore: update init method * chore: coding and style checks * chore: fixed code quality issue * chore: override resize to handle size_divisor, move all preprocessing logic to child class * fix: autoImageProcessor issue for textnet * chore: cleanup * simplify resize --------- Co-authored-by: yonigozlan <yoni.gozlan@huggingface.co> * Fix `time_spent ` in `notification_service.py`. (#40081) fix Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * chore: standardize DeBERTa model card (#37409) * chore: standardize DeBERTa model card * Apply suggestions from code review in docs Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * fix: Update deberta.md with code cleanup suggestions * Update docs/source/en/model_doc/deberta.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/deberta.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update deberta.md * Update deberta.md --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * [`GPT Big Code`] Fix attention scaling (#40041) * fix * update integration tests * fmt * add regression test * feat: extract rev in attn_implementation kernels via @ (#40009) * feat: extract rev in attn_implementation kernels via @ * fix: adjust for ruff * fix: update regex and add explanatory comment * fix: move attn_implementation kernel doc * fix: remove extra line * Update notification service MI325 (#40078) add mi325 to amd_daily_ci_workflows * Fix PerceptionLM image preprocessing for non-tiled image input. (#40006) * Fix PerceptionLM image preprocessing for non-tiled image input. * Add test for single tile vanilla image processing. * ruff format * recover missing test skip * Simplify test. * minor test name fix * Revert FA2 kwargs construction (#40029) * revert * use imports * went way too high in imports level * style * [fix] batch inference for llava_onevision (#40021) * [fix] llava onevision batch inference * style * cannot pass inconsistent list & handle text-only case * [docs] Zero Shot Object Detection Task (#40096) * refactor zsod task docs * keeping the image guided od section * Apply suggestions from code review Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * Update docs/source/en/tasks/zero_shot_object_detection.md Co-authored-by: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com> --------- Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> Co-authored-by: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com> * Update Glm4V processor and add tests (#39988) * update GLm4V and add tests * Update tests/models/glm4v/test_processor_glm4v.py Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> * remove min/max pixels for BC * fix video tests --------- Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> * Add glm4.5&&glm4.5V doc (#40095) * Docs: GLM-4-MoE & GLM-4V-MoE pages * Docs: polish GLM-4V-MoE intro, remove placeholders; pin image * Docs --------- Co-authored-by: wujiahan <lambert@gmail.com> * Causal loss for `ForConditionalGeneration` (#39973) * feat: add ForConditionalGeneration loss to LOSS_MAPPING * consistent spelling of "recognized" * Audio encodings now match conv2d weight dtype in Gemma3nAudioSSCPConvBlock (#39743) audio encodings now match conv weight dtype in Gemma3nAudioSSCPConvBlock * New DynamicSlidingWindowLayer & associated Cache (#40039) * start adding the layer * style * improve * modular * fix * fix * improve * generate integration * comment * remove old one * remove * fix * fix * fix * fix all recompiles * fix * doc * fix * add text config check * fix encoderdecoder cache * add it for all models with sliding/hybrid support * revert * start fixing * prophetnet * fsmt * fix ddp_data * add test for mistral * improve mistral test and add gemma2 test * docstrings * Enable SIM rules (#39806) * Enable SIM rules Signed-off-by: cyy <cyyever@outlook.com> * More fixes Signed-off-by: cyy <cyyever@outlook.com> --------- Signed-off-by: cyy <cyyever@outlook.com> * feat: add `is_fast` to ImageProcessor (#39603) * feat: add `is_fast` to ImageProcessor * test_image_processing_common.py 업데이트 Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> * feat: add missing BaseImageProcessorFast import * fix: `issubclass` for discriminating subclass of BaseImageProcessorFast --------- Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co> * Re-apply make style (#40106) make style * Replace `logger.warning` with `logger.warning_once` in `GradientCheckpointingLayer` (#40091) * Fix regression in mllama vision encoder (#40083) fix mllama vision encoder Signed-off-by: Isotr0py <2037008807@qq.com> * Switch the order of args in StaticCache (for BC and future logic) (#40100) * switch order for BC and future logic * in generate as well * Fix Qwen3 MoE GGUF architecture mismatch (#39976) * fix qwen3moe gguf architecture * Fix Qwen3Moe GGUF loading --------- Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com> Co-authored-by: Jinuk Kim <jusjinuk@snu.ac.kr> * Fix error on importing unavailable torch.distributed (#40038) Currently model_debugging_utils.py would have an unguarded `import torch.distributed.tensor`. This PR ensures that the distributed module is available before including its tensor module. * Default to dequantize if cpu in device_map for mxfp4 (#39993) * default to dq if cpu * an other check * style * revert some changes * [`Flash Attention`] Fix flash attention integration (#40002) * fix flash attention * i got a stroke reading that comment * change dropout kwarg back to before * rename _fa3... as it's used for multiple variants and should work as fallback instead * simplify imports and support kwargs for fa * style * fix comments order * small fix * skip kernels test (causes cuda illegal memories w/o cleanup), fix fa test in general esp for models like bart * style * allow fullgraph by preloading on init * make globals "private" * ci pls be happy * change skip conditions based on backend flag (indicating missing mask interface) * move globals support to a function to prepare kwargs * style * generalize supported kwargs * small change to doc * fix * add comments * style * revert prep during generate * style * revert weird style changes * add fa kwarg prep during generate with fixes back * how did this even happen * how * add comment * [trainer] ensure special tokens in model configs are aligned with tokenizer at train time (#38441) * tmp commit * add test * make fixup * reset warns/info in test * Fix Causality Handling in Flash Attention to Support Bidirectional Attention (#39707) Fix the is_causal logic to enable bidirectional attention Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * [docs] Add reference to HF-maintained `custom_generate` collections (#39894) decoding -> generation; add collections * Add model card for MobileViT (#40033) * Add model card for MobileViT * Update docs/source/en/model_doc/mobilevit.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/mobilevit.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/mobilevit.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/mobilevit.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/mobilevit.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update mobilevit.md * Update mobilevit.md * Update mobilevit.md * Update docs/source/en/model_doc/mobilevit.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/mobilevit.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update mobilevit.md * Update mobilevit.md * Update mobilevit.md * Update mobilevit.md --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * remove sequence parallel in llama4 (#40084) * 🌐 [i18n-KO] Translated `tiny_agents.md` to Korean (#39913) * docs: ko: tiny_agents.md * feat: nmt draft * fix: manual edits * fix: manual edits * [bugfix] Fix tensor device in Idefics2, Idefics3, and SmolVLM (#39975) * [bugfix] ensure correct tensor device in Idefics2, Idefics3, and SmolVLM models * to cuda * changed xLSTMRMSNorm to RMSNorm (#40113) * changed xLSTMRMS.. to RMS... * fix linter error --------- Co-authored-by: Nikita <nikita@Nikitas-MacBook-Pro.local> * Fix QuantoQuantizedCache import issues (#40109) * fix quantoquantized * [serve] allow array `content` inputs for LLMs (#39829) fix bug; add tests * `decoding_method` argument in generate (#40085) * factor out expand inputs * callable arg * improve docs, add test * Update docs/source/en/generation_strategies.md Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> --------- Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> * Collated reports (#40080) * Add initial collated reports script and job definition * provide commit hash for this run. Also use hash in generated artifact name. Json formatting * tidy * Add option to upload collated reports to hf hub * Add glob pattern for test report folders * Fix glob * Use machine_type as path filter instead of glob. Include machine_type in collated report * DOCS: Add missing space in SECURITY.md (#40087) * [trainer] handle case where EOS token is None in `generation_config` (#40127) * handle case where EOS token is None in gen config * update eli5 dataset * Fix hidden torchvision>=0.15 dependency issue (#39928) * use pil_torch_interpolation_mapping for NEAREST/NEAREST_EXACT * fix min torchvision version * use InterpolationMode directly * remove unused is_torchvision_greater_or_equal, * nit * 🌐 [i18n-KO] Translated `main_classes/processors.md` to Korean (#39519) * docs: ko: processors.md * feat: nmt draft * fix: manual edits * Update docs/source/ko/main_classes/processors.md Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com> * Update docs/source/ko/main_classes/processors.md Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com> --------- Co-authored-by: TaskerJang <bymyself103@naver.com> Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com> * 🌐 [i18n-KO] Translated `jamba.md` to Korean (#39890) * docs: ko: jamba.md * feat: nmt draft * fix: manual edits * fix: resolve suggestion Co-authored-by: Minseo Kim <75977640+luckyvickyricky@users.noreply.github.com> --------- Co-authored-by: Minseo Kim <75977640+luckyvickyricky@users.noreply.github.com> * 🌐 [i18n-KO] Translated `main_classes/optimizer_schedules.md` to Korean (#39713) * docs: ko: main_classes/optimizer_schedules * feat: nmt draft * fix: improve TOC anchors and expressions in optimizer_schedules - Add TOC anchors to all section headers - Fix terminology and improve Korean expressions * fix: Correct translation of 'weight decay fixed' to '가중치 감쇠가 적용된' Changed '가중치 감쇠가 수정된' to '가중치 감쇠가 적용된' for more accurate translation of 'weight decay fixed' in the context of optimization. * fix: Use more natural Korean inheritance expression Changed '에서 상속받는' to '을 상속받는' to follow natural Korean grammar patterns for inheritance terminology. * fix: Use consistent '미세 조정' translation for 'finetuned models' Changed '파인튜닝된' to '미세 조정된 모델' to follow the established translation glossary for 'finetuned models' terminology. * 🚨🚨 [generate] ignore `cache_implementation="hybrid"` hub defaults (#40135) * working? * fix tests * 🌐 [i18n-KO] Translated `gpt2.md` to Korean (#39808) * docs: ko: bamba.md * feat: nmt draft * fix: manual edits * docs: ko: gpt2.md * feat: nmt draft * fix: manual edits * Remove bamba.md from docs/source/ko/model_doc/ * Update _toctree.yml * 🌐 [i18n-KO] Translated `optimizers.md` to Korean (#40011) * docs: ko: optimizers.md * feat: optimizers draft * fix: manual edits * docs: ko: update optimizers.md * Update docs/source/ko/optimizers.md Co-authored-by: Minseo Kim <75977640+luckyvickyricky@users.noreply.github.com> * Update docs/source/ko/optimizers.md Co-authored-by: Minseo Kim <75977640+luckyvickyricky@users.noreply.github.com> * Update docs/source/ko/optimizers.md Co-authored-by: Jaehyeon Shin <108786184+skwh54@users.noreply.github.com> * docs: ko: final updates to optimizers and toctree --------- Co-authored-by: Minseo Kim <75977640+luckyvickyricky@users.noreply.github.com> Co-authored-by: Jaehyeon Shin <108786184+skwh54@users.noreply.github.com> * 🌐 [i18n-KO] Translated grounding-dino.md to Korean (#39861) * docs: ko: grounding-dino.md * feat: nmt draft * fix: manual edits * Update docs/source/ko/model_doc/grounding-dino.md Co-authored-by: Kim Juwon <81630351+Kim-Ju-won@users.noreply.github.com> * Update docs/source/ko/model_doc/grounding-dino.md Co-authored-by: Kim Juwon <81630351+Kim-Ju-won@users.noreply.github.com> * Update docs/source/ko/model_doc/grounding-dino.md Co-authored-by: Kim Juwon <81630351+Kim-Ju-won@users.noreply.github.com> * docs: add AP explanation for better readability --------- Co-authored-by: TaskerJang <bymyself103@naver.com> Co-authored-by: Kim Juwon <81630351+Kim-Ju-won@users.noreply.github.com> Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com> * 🚨 Use lru_cache for sine pos embeddings MaskFormer (#40007) * use lru_cache for sine pos embeddings maskformer * fix calls to pos embed * change maxsize to 1 * 🌐 [i18n-KO] Translated `pipelines.md` to Korean (#39577) * docs: ko: pipelines.md * feat: gpt draft * Update docs/source/ko/main_classes/pipelines.md Co-authored-by: Yijun Lee <119404328+yijun-lee@users.noreply.github.com> * Update docs/source/ko/main_classes/pipelines.md Co-authored-by: Yijun Lee <119404328+yijun-lee@users.noreply.github.com> * Update docs/source/ko/main_classes/pipelines.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/ko/main_classes/pipelines.md Co-authored-by: Yijun Lee <119404328+yijun-lee@users.noreply.github.com> * Update docs/source/ko/main_classes/pipelines.md Co-authored-by: Yijun Lee <119404328+yijun-lee@users.noreply.github.com> * Update _toctree.yml * Update _toctree.yml 번역 문서 수정 * Update pipelines.md ToC 수정 * Update pipelines.md --------- Co-authored-by: xhaktm <tnwjd318@hs.ac.kr> Co-authored-by: Yijun Lee <119404328+yijun-lee@users.noreply.github.com> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * gpt oss is important (#40139) * Fix Janus (#40140) fix * Add Segment Anything 2 (SAM2) (#32317) * initial comment * test * initial conversion for outline * intermediate commit for configuration * chore:init files for sam2 * adding arbitary undefined config * check * add vision * make style * init sam2 base model * Fix imports * Linting * chore:sam to sam2 classes * Linting * Add sam2 to models.__init__ * chore:match prompt encoder with sam2 code * chore:prepare kwargs for mask decoder * Add image/video predictors * Add CUDA kernel * Add output classes * linting * Add logging info * tmp commit * docs for sam2 * enable image processing * check difference of original SAM2 - difference is the order of ToTensor() - please see https://pytorch.org/vision/main/_modules/torchvision/transforms/functional.html#resize * enable promptencoder of sam2 * fix promprencoder * Confirmed that PromptEncoder is exactly same (Be aware of bfloat16 and float32 difference) * Confirmed that ImageEncoder is exactly same (Be aware the linting of init) * Confirmed that MaskDecoder is exactly same (TO DO: lint variable name) * SamModel is now available (Need more chore for name) * make fix-copies * make style * make CI happy * Refactor VisionEncoder and PostioinEmbedding * TO DO : fix the image_embeddings and sparse_embeddings part * pure image inference done * reusable features fix and make style * styling * refactor memoryattention * tmp * tmp * refactor memoryencoder TO DO : convert and inference the video pipeline * TO DO : fix the image_encoder shape * conversion finish TO DO: need to check video inference * make style * remove video model * lint * change * python utils/check_docstringspy --check_all * python utils/check_config_attributes.py * remove copies for sam2promptencoder due to configuration * change __init__.py * remove tensorflow version * fix that to not use direct comparison * make style * add missing import * fix image_embedding_size * refactor Sam2 Attention * add fully working video inference (refactoring todo) * clarify _prepare_memory_conditioned_features * simplify modeling code, remove unused paths * use one model * use auto_docstring * refactor rope embeddings * nit * not using multimask when several points given * add all sam2.1 * add video tmp * add Sam2VideoSessionState + fast image proc + video proc * remove init_states from model * fix batch inference * add image integration tests * uniformize modeling code with other sam models and use modular * pass vision tests an most model tests * All tests passing * add offloading inference state and video to cpu * fix inference from image embedding and existing mask * fix multi_boxes mask inference * Fix batch images + batch boxes inference * improve processing for image inference * add support for mask generation pipeline * add support for get_connected_components post processing in mask generation * add fast image processor sam, image processor tests and use modular for sam2 image processor * fix mistake in sam after #39120 * fix init weights * refactor convert * add integration tests for video + other improvements * add needed missing docstrings * Improve docstrings and * improve inference speed by avoiding cuda sync * add test * skip test for vision_model * minor fix for vision_model * fix vision_model by adding sam2model and change the torch dependencies * remove patch_size * remove image_embedding_size * fix patch_size * fix test * make style * Separate hieradet and vision encoder in sam2 * fixup * review changes part 1 * remove MemoryEncoderConfig and MemoryAttentionConfig * pass q_stride instead of q_pool module * add inference on streamed videos * explicitely process streamed frames * nit * Improve docstrings in Sam2Model * update sam2 modeling with better gestion of inference state and cache, and separate Sam2Model and Sam2VideoModel * improve video inference api * change inference_state to inference_session * use modular for Sam2Model * fix convert sam2 hf * modular * Update src/transformers/models/sam2/video_processing_sam2.py Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * fix minor config * fix attention loading error * update modeling tests to use hub checkpoints * Use CI A10 runner for integration tests values + higher tolerance for video integration tests * PR review part 1 * fix doc * nit improvements * enforce one input format for points, labels and boxes * nit * last few nits from PR review * fix style * fix the input type * fix docs * add sam2 model as conversion script * improve sam2 doc * nit fixes + optimization * split sam2 and sam2_video in two models * PR review part 1 * fix None for default slow processor of sam2 * remove unecessary code path in sam2_video * refactor/simplify RoPE * replace embedding module list with embedding matrix * fix tests * remove kernel * nit * use lru_cache for sine_pos_embeddings * reorder sam2_video methods * simplify sam2_video * PR review part 1 * simplify sam2 video a lot * more simplification * update integration tests with updated conftest * more explicit config for hieradet * do post_processing outside of sam2 video model * Improve Sam2VideoVisionRotaryEmbedding * fix tests * update docs and fix mask2former/oneformer * avoid unnecessary reshapes/permute * fix device concatenating points * small dtype fix * PR review * nit * fix style and finish up doc * fix style * fix docstrings * fix modular --------- Co-authored-by: RUFFY-369 <prakarshkaushik369@gmail.com> Co-authored-by: Haitham Khedr <haithamkhedr@meta.com> Co-authored-by: sangbum choi <sangbumchoi@sangbumui-MacBookAir.local> Co-authored-by: yonigozlan <yoni.gozlan@huggingface.co> Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * [docs] Fix ko toctree (#40138) Update _toctree.yml * Remove an old badly designed test (#40142) remove it * updated visualBERT modelcard (#40057) * updated visualBERT modelcard * fix: Review for VisualBERT card * 🌐 [i18n-KO] Translated `gemma3.md` to Korean (#39865) * docs: ko: gemma3.md * feat: nmt draft * fix: manual edits * fix: resolve suggestions Co-authored-by: Chaewon Song <chaewon1019@ewhain.net> * fix: resolve suggestions --------- Co-authored-by: Chaewon Song <chaewon1019@ewhain.net> * Fix quantized cache with only cache_implementation in generate (#40144) * fix args * comment * Add pytest marker: `torch_compile_test` and `torch_export_test` (#39950) * new marker * trigger CI * update --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Update Dockerfiles to install packages inside a virtual environment (#39098) * Removed un-necessary virtual environment creation in Dockerfiles. * Updated Dockerfiles to install packages in a virtual environment. * use venv's python * update * build and trigger * trigger * build and trigger * build and trigger * build and trigger * build and trigger * build and trigger * build and trigger * update * update * update * update --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Create self-scheduled-amd-mi355-caller.yml (#40134) * [Cohere2Vision] remove unused arg (#40103) * remove unused arg * remove the arg from test as well * [efficientloftr] fix bugs and follow original cross attn implementation strictly (#40141) * fix: changed is_causal to be False * fix: Added original cross attention bug * fix: fixed the way bordel removal is computed * fix: added missing normalization on coarse features * test: fixed integration tests --------- Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * Fix CI: Use correct import in SAM for torchvision InterpolationMode (#40160) fix ci * [Continous Batching] set head_dim when config.head_dim is None (#40159) * set head_dim when config.head_dim is None * use model's actual TP setting * Replace `self.tokenizer` by `self.processing_class` (#40119) * [FA2] Fix it finally - revert fa kwargs preparation (#40161) revert * [bugfix] fix flash-attention2 unavailable error for Ascend NPU (#40151) * [bugfix] fix flash-attention2 unavailable error for Ascend NPU * remove redundant apply_rotary_emb usage * fix ruff check error * pad_input and unpad_input use same implementation as fa2 * rollback redundant codes * fix ruff check error * optimize fa2 judgement logic * Fix docs typo (#40167) * DINOv3 model * working version * linter revert * linter revert * linter revert * fix init * remove flex and add convert to hf script * DINOv3 convnext * working version of convnext * adding to auto * Dinov3 -> DINOv3 * PR feedback * complete convert checkpoint * fix assertion * bf16 -> fp32 * add fast image processor * fixup * change conversion script * Use Pixtral attention * minor renaming * simplify intermediates capturing * refactor DINOv3ViTPatchEmbeddings * Refactor DINOv3ViTEmbeddings * [WIP] rope: remove unused params * [WIP] rope: rename period -> inv_freq for consistency * [WIP] rope: move augs * change inv_freq init (not persistent anymore) * [WIP] rope: move coords to init * rope - done! * use default LayerScale * conversion: truncate expected outputs * remove commented code * Refactor MLP layers * nit * clean up config params * nit docs * simplify embeddings * simplify compile compat lru_cache * fixup * dynamic patch coords * move augmentation * Fix docs * fixup and type hints * fix output capturing * fix tests * fixup * fix auto mappings * Add draft docs * fix dtype cast issue * add push to hub * add image processor tests * fixup * add modular * update modular * convert and test convnext * update conversion script * update prefix * Update LayerNorm * refactor DINOv3ConvNextLayer * rename * refactor convnext model * fix doc check * fix docs * fix convnext config * tmp fix for check docstring * remove unused arg * fix tests * (nit) change init * standardize gated MLP * clear namings and sat493m * fix tensors on different devices * revert linter * pr * pr feedbak ruff format * missing headers * fix code snippet and collection link in docs * DINOv3 description * fix checkpoints in tests * not doc fixes in configs * output_hidden_states * x -> features * remove sequential --------- Co-authored-by: Cijo Jose <cijose@meta.com> * build: Add fast image processor tvp (#39529) * build: add TvpImageProcessorFast - Introduced TvpImageProcessorFast to enhance image processing capabilities. - Updated image processing auto registration to include the new fast processor. - Modified tests to accommodate both TvpImageProcessor and TvpImageProcessorFast, ensuring comprehensive coverage for both classes. * fix: TvpImageProcessorFast with new resize method and update processing logic * build: add TvpImageProcessorFast * refactor: clean up whitespace and formatting in TvpImageProcessorFast and related tests - Removed unnecessary whitespace and ensured consistent formatting in image_processing_tvp_fast.py. - Updated import order in test_image_processing_tvp.py for clarity. - Minor adjustments to maintain code readability and consistency. * fix: Enhance TvpFastImageProcessorKwargs and update documentation - Added TvpFastImageProcessorKwargs class to define valid kwargs for TvpImageProcessorFast. - Updated the documentation in tvp.md to include the new class and its parameters. - Refined the image processing logic in image_processing_tvp_fast.py for better handling of padding and resizing. - Improved test cases in test_image_processing_tvp.py to ensure compatibility with the new processing logic and tensor inputs. * fix: tested now with python 3.9 * fix: remove tvp kwargs from docs * simplify processing * remove import and fix tests --------- Co-authored-by: yonigozlan <yoni.gozlan@huggingface.co> * Add GptOssForSequenceClassification for GPT-OSS models (#40043) * Add GptOssForSequenceClassification * Tiny fix * make fixup * trigger CI rerun * Check config type instead --------- Co-authored-by: Yuefeng Zhan <yuefzh@microsoft.com> * Standardize BARTpho model card: badges, new examples, fixed broken im… (#40051) * Standardize BARTpho model card: badges, new examples, fixed broken image section, and links (#36979)Update bartpho.md * Update bartpho.md Removed non-required/unsupported sections: Quantization, Attention visualizer, and Resources (plus stray tokenizer header). Added code snippets which were suggested * Update bartpho.md Updated with necessary tags * Update bartpho.md * Update bartpho.md --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Add dates to the model docs (#39320) * added dates to the models with a single hf papers link * added the dates for models with multiple papers * half of no_papers models done * rest of no_papers models also done, only the exceptions left * added copyright disclaimer to sam_hw, cohere, cohere2 + dates * some more fixes, hf links + typo * some new models + a rough script * the script looks robust, changed all paper links to hf * minor change to handle technical reports along with blogs * ran make fixup to remove the white space * refactor * Pin torch to 2.7.1 on CircleCI for now (#40174) * fix * fix --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Update dynamic attnt setter for multimodals (#39908) * update * fix the test for DepthPro * PR comments * wait, I didn't delete this in prev commit? * fix * better way --------- Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co> Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com> * [MINOR:TYPO] Update base.py (#40169) * [MINOR:TYPO] Update base.py All other occurrences in the docs use lowercase. (https://github.com/search?q=repo%3Ahuggingface%2Ftransformers%20translation_XX_to_YY&type=code) Also, using uppercase doesn't work: tested with "translation_EN_to_FR" which doesn't work and instead returns: `ValueError: The task does not provide any default models for options ('EN', 'FR')` It might be a good idea to allow for uppercase, but that's for another issue. * [MINOR:TYPO] Update __init__.py * make model doc device agnostic (#40143) * make model doc device agnostic Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * Update align.md * Update aya_vision.md * Update byt5.md * refine Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * Update granitevision.md * Update src/transformers/pytorch_utils.py Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * add doc Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * 3 more Signed-off-by: Yao, Matrix <matrix.yao@intel.com> --------- Signed-off-by: Yao, Matrix <matrix.yao@intel.com> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * fix to avoid modifying a view in place (#40162) * fix to avoid modifying a view in place * add backward test in tensor parallel * add test to test_modelig_gpt_oss.py * linting * Fix fsdp for generic-task models (#40191) * remove abc inheritance * add fast test * Add repr to EncoderDecoderCache (#40195) * add repr * oups * Fix typos (#40175) Signed-off-by: cyy <cyyever@outlook.com> * Remove _prepare_flash_attention_from_position_ids (#40069) Signed-off-by: cyy <cyyever@outlook.com> * Avoid CUDA stream sync (#40060) Signed-off-by: cyy <cyyever@outlook.com> * Fix various Pylint warnings (#40107) Tidy code Signed-off-by: cyy <cyyever@outlook.com> * Update: add type hints to check_tokenizers.py (#40094) * Update check_tokenizers.py chore(typing): add type hints to check_tokenizers script - Annotate params/returns for helper functions - Keep tokenizer instances as `Any` to avoid runtime coupling - Make `check_LTR_mark` return `bool` explicitly (no behavior change) * Update check_tokenizers.py chore(typing): replace Any with PreTrainedTokenizerBase in check_tokenizers.py - Use transformers.tokenization_utils_base.PreTrainedTokenizerBase for `slow` and `fast` params - Covers both PreTrainedTokenizer and PreTrainedTokenizerFast - Exposes required methods (encode, decode, encode_plus, tokenize) - Removes generic Any typing while staying implementation-agnostic * Benchmarking improvements (#39768) * Start revamping benchmarking * Start refactoring benchmarking * Use Pandas for CSV * import fix * Remove benchmark files * Remove sample data * Address review comments * Add X-Codec model (#38248) * add working x-codec * nit * fix styling + copies * fix docstring * fix docstring and config attribute * Update args + config * update convertion script * update docs + cleanup * Ruff fix * fix doctrings * Fix GPT-OSS `swiglu_limit` not passed in for MXFP4 (#40197) Add swiglu_limit = 7.0 * docs: Update LayoutLM model card according to new standardized format (#40129) * docs: Update LayoutLM model card with standardized format * Apply suggestions from code review This commit incorporates all suggestions provided in the recent review. Further changes will be committed separately to address remaining comments. Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Address remaining review comments * Address few more review comments: 1. remove transformer-cli section 2. put resources after notes 3. change API refs to 2nd level header * Update layoutlm.md * Update layoutlm.md --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Revert "Pin torch to 2.7.1 on CircleCI for now" + Final fix for `too long with no output` (#40201) * Revert "Pin torch to 2.7.1 on CircleCI for now (#40174)" This reverts commit 31b6e6e1dac0d32f74ec5cd6b3c1868534ccd7b5. * fix --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Use correct `model_input_names` for PixtralImageProcessor (#40226) add image_sizes to model_input_names * fix error vocab_size at Qwen2_5_VLForConditionalGeneration loss_function (#40130) * fix error vocab_size at Qwen2_5_VLForConditionalGeneration loss_function Signed-off-by: luoxiaoc <xiaochuan.luo@intel.com> * fix similar errer at qwen2_vl and do make fix-copies Signed-off-by: luoxiaoc <xiaochuan.luo@intel.com> * pass in kwargs for loss_func at qwen2_vl and qwen2_5_vl Signed-off-by: luoxiaoc <xiaochuan.luo@intel.com> * Apply style fixes --------- Signed-off-by: luoxiaoc <xiaochuan.luo@intel.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * [SAM 2] Change checkpoints in docs and tests (#40213) * change checkpoints in docs and tests * add notebook * Fix more typos (#40212) Signed-off-by: cyy <cyyever@outlook.com> * Fix ESM token_dropout crash when using inputs_embeds instead of input_ids (#40181) * fix: Error after calling ESM model with input embeddings not input ids * propagate changes to other models * AMD scheduled CI ref env file (#40243) * Reference env-file to be used in docker running the CI * Disable MI300 CI for now * Add Ovis2 model and processor implementation (#37088) * Add Ovis2 model and processor implementation * Apply style fixes * Add unit tests for Ovis2 image processing and processor * Refactor image processing functions for clarity and efficiency * Add Ovis2 ImageProcessorFast * Refactor Ovis2 code * Refactor Ovis2 model components and update processor functionality * Fix repo consistency issues for Ovis2: docstring, config cleanup * Update Ovis2 model integration tests * Update Ovis2 configuration and processing classes for improved documentation * Remove duplicate entry for 'ovis2' in VLM_CLASS_NAMES * Fix conflict * Fix import order * Update image processor class names * Update Ovis2 model structure * Refactor Ovis2 configuration * Fix typos * Refactor Ovis2 model classes and remove unused code * Fix typos * Refactor Ovis2 model initialization * Fiix typos * Remove Ovis2 model mapping from MODEL_MAPPING_NAMES in modeling_auto.py * Add license and update type hints * Refactor token function and update docstring handling * Add license * Add Ovis2 model support and update documentation * Refactor Ovis2 model structure and enhance multimodal capabilities * Update Ovis2 weight mapping for consistency and clarity in key patterns * Remove unused 'grids' parameter from Ovis2 model and Update processing logic to handle image grids more efficiently. * Refactor Ovis2 model test structure to include Ovis2Model * Add optional disable_grouping param to Ovis2ImageProcessorFast * Refactor type hints in Ovis2 modules * Add licensing information in Ovis2 modules and tests * Refactor Ovis2 model by removing unused methods * Refactor Ovis2 model tests by renaming test classes and removing skipped tests * Refactor Ovis2 model output classes * Refactor Ovis2 weight conversion and Update model embedding classes * Refactor Ovis2 model imports and remove unused functions * Enhance vision configuration extraction in Ovis2 weight conversion * Refactor Ovis2 model's forward method to remove interpolation option * Update Ovis2 model documentation * Refactor Ovis2 model input handling and tokenizer configuration * Update return type hints in Ovis2 model * Remove commented-out code * fix config for tests and remove key mappings * Update tokenizer configuration to use add_special_tokens method * skip torchscript * Fix image placeholder generation in Ovis2Processor * Refactor Ovis2 model to rename visual_table to visual_embeddings_table * Enhance Ovis2 model by adding vision_feature_select_strategy parameter * Refactor Ovis2 model weights conversion and architecture * Refactor Ovis2 model by removing vision_feature_select_strategy parameter * Update Ovis2 model examples * Refactor Ovis2 model * Update Ovis2 model * Update Ovis2 model configuration * Refactor Ovis2 model test setup * Refactor flash attention support * Refactor * Fix typo * Refactor * Refactor model classes * Update expected output in Ovis2 * Refactor docstrings * Fix * Fix * Fix * Update input in tests * Fix * Fix get_decoder method * Refactor * Refactor Ovis2 * Fix * Fix * Fix test * Add get_placeholder_mask * Refactor Ovis2 model tests * Fix * Refactor * Fix * Fix * Fix Ovis2 test --------- Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com> * Fix more pylint warnings (#40204) Fix pylint warnings Signed-off-by: cyy <cyyever@outlook.com> * 🚨 Always return Cache objects in modelings (to align with generate) (#39765) * watch the world burn * fix models, pipelines * make the error a warning * remove kwargs and return_legacy_cache * fix reformer * remove transpose_for_scores call in ESM-2 (#40210) * remove transpose_for_scores call Signed-off-by: Peter St. John <pstjohn@nvidia.com> * fix copied evolla code Signed-off-by: Peter St. John <pstjohn@nvidia.com> --------- Signed-off-by: Peter St. John <pstjohn@nvidia.com> * Add `chat_template` (`jinja2`) as an extra dependency (#40128) * add jinja2 as a dependency * Make jinja2 a core dependency in install_requires - Add jinja2 to install_requires list in setup.py for automatic installation - Add jinja2 to runtime version checks in dependency_versions_check.py - Resolves issue where pip install transformers doesn't install jinja2 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Make jinja2 a core dependency in install_requires * Make jinja2 an extra dependency instead of adding a core dep --------- Co-authored-by: Claude <noreply@anthropic.com> * [typing] fix type annotation error in DepthPro model image processor (#40238) * fix type annotation error in DepthPro model image processor * fix * run make fix-copies * [serve] guard imports (#39825) guard imports * [`CI`] Fix repo consistency (#40249) * fix * doc --------- Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com> * Fixes for EncoderDecoderCache (#40008) * Add expectation to t5 for rocm 9.4 * Made EncoderDecoderCache compatible with nn.DataParallel * Fixed t5gemma EncoderDecoderCache * Added todos in autoformer * Ruff * Init is self-contained * Review compliance * Fixed kwargs init of EncoderDecoderCache * fix: Catch correct ConnectionError for additional_chat_templates (#39874) * fix: Catch correct ConnectionError for additional_chat_templates * fix: don't catch timeout * fix: formatting * Model card for NLLB (#40074) * initializing branch and draft PR * updated model card .md file * minor * minor * Update docs/source/en/model_doc/nllb.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * resolving comments + adding visuals * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * NllbTokenizerFast and NllbTokenizer added * endline * minor * Update nllb.md --------- Co-authored-by: Sahil Kabir <sahilkabir@Sahils-MacBook-Pro.local> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Correct typo and update notes in docs Readme (#40234) * Correct typo and update notes in docs readme * Update docs/README.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/README.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Fix benchmark workflow (#40254) Correct init_db.sql path Co-authored-by: Akos Hadnagy <akoshuggingface@mi325x8-123.atl1.do.cpe.ice.amd.com> * docs: Update OLMo model card (#40233) * Updated OLMo model card * Update OLMo description Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Fix typo Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Fix cli typo Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Fix cli example Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Add bitsandbytes info Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Skip broken tests (#40157) skip these tests * Remove MI300 CI (#40270) Remove MI300 CI (in history if we need it back) * set inputs_embeds to None while generate to avoid audio encoder forward in generation process (#40248) * set inputs_embeds to None while generate to avoid audio encoder forward in generation process * set input_features to none instead --------- Co-authored-by: lvyuanjun.lyj <lvyuanjun.lyj@alibaba-inc.com> * [detection] fix attention mask for RT-DETR-based models (#40269) * Fix get_contrastive_denoising_training_group attention * Add bool attention_mask conversion * Fix slow static cache export tests (#40261) * 🚨🚨 Switch default compilation to fullgraph=False (#40137) * switch default * docstring * docstring * rework tests and remove outdated restrictions * simplify * we need a check for static cache * fix * rename var * fix * revert * style * rename test * Fix setting attention for multimodal models (#39984) * fix * use non-explicit `None` * keep previously set attn if exists * [detection] fix correct `k_proj` weight and bias slicing in D-FINE (#40257) Fix: correct k_proj weight and bias conversion in D-FINE * Add Kosmos-2.5 (#31711) Add Microsoft Kosmos-2.5 --------- Co-authored-by: kirp@umich.edu <tic-top> Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Skipping pytree registration in case fsdp is enabled (#40075) * Skipping pytree registration in case fsdp is enabled * Beauty changes * Beauty changes * Moved the is_fsdp_available function to import utils * Moved is_fsdp_available to integrations.fsdp * Skipping pytree registration in case fsdp is enabled * Beauty changes * Beauty changes * Moved the is_fsdp_available function to import utils * Moved is_fsdp_available to integrations.fsdp * Added pytree registration inside dynamic cache class * Making ci/cd lords happy * Adding a check if DynamicCache is already a leaf * Adding try/catch for multiple initializations of DynamicCache in test suites * Moving dynamic cache pytree registration to executorch * Adding try catch back * Update image_processing_perception_lm_fast.py to allow for proper override of vision_input_type (#40252) * Update image_processing_perception_lm_fast.py Allow for a proper override of vision_input_type in hf fast image processor, otherwise we need to resort to manually setting the attribute. * Update processing_perception_lm.py to match kwargs vision input type * Update image_processing_perception_lm_fast.py kwargs to signature args * fix which routing method (#40283) * Fix chat CLI GPU loading and request_id validation issues (#40230) (#40232) * Fix chat CLI GPU loading and request_id validation issues (#40230) This commit addresses two critical bugs in the transformers chat CLI: 1. **GPU Loading Issue**: Changed default device from "cpu" to "auto" in ChatArguments - Chat CLI now automatically uses GPU when available instead of defaulting to CPU - Matches the behavior of the underlying serving infrastructure 2. **Request ID Validation Error**: Added request_id field to TransformersCompletionCreateParamsStreaming schema - Fixes "Unexpected keys in the request: {'request_id'}" error on second message - Allows request_id to be properly sent and validated by the server Both fixes target the exact root causes identified in issue #40230: - Users will now get GPU acceleration by default when available - Chat sessions will no longer break after the second message * Remove unrelated request_id field from TransformersCompletionCreateParamsStreaming * docs(layoutlm): add missing `id=usage` to `<hfoptions>` tag in LayoutLM model card (#40273) docs(layoutlm): add missing 'id=usage' to <hfoptions> tag in LayoutLM model card * Standardize RAG model card (#40222) * Standardize RAG model card Update rag.md to follow the new Hugging Face model card template: - Added friendly overview in plain language - Added pipeline and AutoModel usage examples - Included quantization example with BitsAndBytesConfig - Added notes and resources sections - Removed abstract and FlashAttention badge * Standardize RAG model card Update rag.md to follow the new Hugging Face model card template: - Added friendly overview in plain language - Added AutoModel usage example - Included quantization example with BitsAndBytesConfig * docs: Update TrOCR model card to new format (#40240) * docs: Update TrOCR model card to new format * Updated Sugegestions * Update model card for gpt neox japanese (#39862) * Update GPT-NeoX-Japanese model card * Apply suggestions from code review * Update gpt_neox_japanese.md --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * SmolVLM and InternVL: Ensure pixel values are converted to the correct dtype for fp16/bf16 (#40121) * Ensure pixel values are converted to the correct dtype for fp16/bf16 * add to modular * Standardize BertGeneration model card (#40250) * Standardize BertGeneration model card: new format, usage examples, quantization * Update docs/source/en/model_doc/bert-generation.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/bert-generation.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/bert-generation.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/bert-generation.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/bert-generation.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/bert-generation.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/bert-generation.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Apply reviewer feedback: update code examples * Add missing code example --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Adjust ROCm test output expectations (#40279) Adjust ROCm output expectations * SmolVLM test fixes (#40275) * Fix SmolVLM tests * Add the proper CUDA expectations as well * Split 'A10 and A100 expectations * Ruff --------- Co-authored-by: Akos Hadnagy <akoshuggingface@mi325x8-123.atl1.do.cpe.ice.amd.com> * make model docs device agnostic (2) (#40256) * doc cont. Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * more models Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * Update docs/source/en/quicktour.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/quicktour.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/quicktour.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/quicktour.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update mixtral.md --------- Signed-off-by: Yao, Matrix <matrix.yao@intel.com> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * [3/3] make docs device agnostic, all en docs for existing models done (#40298) docs to device agnostic cont. Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * Add MetaCLIP 2 (#39826) * First draft * Make fixup * Use eos_token_id * Improve tests * Update clip * Make fixup * Fix processor tests * Add conversion script * Update docs * Update tokenization_auto * Make fixup * Use check_model_inputs * Rename to lowercase * Undo CLIP changes * Address comment * Convert all checkpoints * Update auto files * Rename checkpoints * Allow to be able to run `torch.compile` tests with `fullgraph=True` (#40164) * fix * address comment --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * [`FA`] Fix dtype in varlen with position ids (#40295) fix * [docs] delete more TF/Flax docs (#40289) * delete some TF docs * update documentation checks to ignore tf/flax * a few more removals * nit * Update utils/check_repo.py Co-authored-by: Matt <Rocketknight1@users.noreply.github.com> --------- Co-authored-by: Matt <Rocketknight1@users.noreply.github.com> * Clean up X-Codec. (#40271) * Clean up xcodec addition. * Clean up config. * Switch to fixtures test. * Small stuff. * Remove OTel SDK dependencies (#40305) * Fix GOT-OCR2 and Cohere2Vision image processor patches caculation (#40312) fix got-ocr patches caculation Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn> * [`fix`] Pass adamw optimizer parameters to StableAdamW (#40184) * fix: pass adamw optimizer parameters to StableAdamW * add test for stable_adamw initialization with trainer arguments * address copilot suggestion * fix: update weight_decay handling in stable_adamw kwargs --------- Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> * chore: fix typo in `find_executable_batch_size` to match new 0.9 ratio (#40206) * :rotating_light: [`Flash Attention`] Fix sliding window size (#40163) * swa fix * add comment, make fix symmetrical * modify fa inference test to force swa correctness check * fixup comment * Remove unnecessary contiguous calls for modern torch (#40315) * Add support for Florence-2 (#38188) * init * add modular * fixup * update configuration * add processing file * update auto files * update * update modular * green setup_and_quality ci * it works * fix some tests * commit florence2 * update test * make test cases done - 16 left * style * fix few test cases * fix some tests * fix init test * update florence2 vision style * hope is green * fix init test * fix init * update modular * refactor vision module * fix: channel attention use dynamic scale * update modular * update * update attention mask * update * fix naming * Update src/transformers/models/florence2/processing_florence2.py Co-authored-by: Matt <Rocketknight1@users.noreply.github.com> * spatial block works * more beautiful * more more beautiful * merge main * merge main and fixup * fix typing hint * update modeling * fix eager matches sdpa * fix style * fix compile test - all green * remove florence2 language * remove Florence2LanguageModel things * fix style * update florence2 model * override prepare encoder_decoder for generation * add weight conversion script * rewrite channel attention to use sdpa * eleminate 1 tranpose op * support fa2 * fix quality check * chore: reformat `test_modeling_florence2.py` * some refactor for processor * some refactor for processor * update naming convention and remove BC * make it pass the test * fix: correct Embedding Cosine * update comments and docstring * support input_embeds * support input embeds ideally * fix style * fix style * fix style again :D * add test prcoessor * refactor processor and add test for processor * reformat test processor * make fixup * fix schema check * remove image_token * ensure image token in tokenizer and fix integration tests * fix processor test * add more integration tests for large model and rename test_processor to test_processing * test_assisted_decoding_sample should pass * update doc and make model work with image text to text pipeline * docs: add sdpa bagde * resolve cyril's comments * fix import torch error * add helper get_placeholder_mask * inherit from llava * florence2 may not _supports_attention_backend because of bart ... * move florence2 model card to multimodal * let base model always return_dict * fix style * tiny update doc * set _checkpoint_conversion_mapping = {} * fix code quality * support flex and compile graph and move external func to internal func * remove condition because it always true * remove window funcs * move post processor config out * fix ci * new intro to trigger test * remove `kernel_size` argument --------- Co-authored-by: ducviet00-h2 <viet.d.hoang@h2corporation.jp> Co-authored-by: Matt <Rocketknight1@users.noreply.github.com> * Qwen2.5-Omni test fixes (#40307) Updated expectations, and mp tests * Add back `_tp_plan` attribute (#39944) * Update modeling_utils.py * make sure we update with the module's plan * use public api * oups * update * fix failing test * Update src/transformers/integrations/tensor_parallel.py * Update src/transformers/integrations/tensor_parallel.py * fix * make the API more friendly! * fix tests * fix styling --------- Co-authored-by: Arthur Zucker <arthur.zucker@gmail.com> Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * byebye torch 2.1 (#40317) * Bump minimum torch version to 2.2 * Remove is_torch_greater_or_equal_than_2_2 * update versions table * Deprecate is_torch_sdpa_available (except for backward compat), remove require_torch_sdpa * No more `natten` (#40287) get rid off natten Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * [`GPT OSS`] Refactor the tests as it was not properly checking the outputs (#40288) * it was long due! * use the official kernel * more permissive * update the kernel as well * mmm should it be this? * up pu * fixup * Update test_modeling_gpt_oss.py * style * start with 20b * Update CI with nightly torch workflow file (#40306) * fix nightly ci * Apply suggestions from code review Co-authored-by: ivarflakstad <69173633+ivarflakstad@users.noreply.github.com> --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> Co-authored-by: ivarflakstad <69173633+ivarflakstad@users.noreply.github.com> * Fix: Apply `get_placeholder_mask` in Ovis2 (#40280) * Refactor special image mask * Refactor get_placeholder_mask method * Revert "Refactor special image mask" This reverts commit 9eb1828ae930329656d6f323a510c5e6033e1f85. * Fix * Revert "Refactor get_placeholder_mask method" This reverts commit 07aad6484bb08d6351d5b605e9db574d28edcd15. * Update notification service amd_daily_ci_workflows definition (#40314) * One cache class to rule them all (#40276) * remove all classes * fix generate * start replacing everywhere * finish removing everywhere * typo * typo * fix * typo * remove num_layers=1 * CI * fix all docstrings * review * style * Fix chunked attention mask with left-padding (#40324) * add fix * add test * raise proper warning for older versions * fix * fix and add 2nd test * fix for flex and torch 2.5 * [docs] remove flax references from `/en/model_doc` (#40311) * 1st commit * all models up to D * all models up to G * all models up to M * all remaining models * Fix qwen-omni processor text only mode (#40336) * Fix qwen-omni processor text only mode * remove try except --------- Co-authored-by: yuekaiz <yuekaiz@mgmt1-login.cm.cluster> * Change Qwen2RMSNorm to RMSNorm from PyTorch (#40066) * Unify Qwen2RMSNorm definitions and use RMSNorm from PyTorch Signed-off-by: cyy <cyyever@outlook.com> * subclass RMSNorm Signed-off-by: cyy <cyyever@outlook.com> --------- Signed-off-by: cyy <cyyever@outlook.com> * Add DeepseekV3ForSequenceClassification for Deepseek V3 models (#40200) * Add Sequence Classification Support for Deepseek v3 model DeepseekV3ForSequenceClassification * After run make fixup * Fix deprecation warning version (#40343) fix * Add missing arguments to class constructors (#40068) * Add missing arguments Signed-off-by: cyy <cyyever@outlook.com> * Fix typos Signed-off-by: cyy <cyyever@outlook.com> * More fixes Signed-off-by: cyy <cyyever@outlook.com> --------- Signed-off-by: cyy <cyyever@outlook.com> * [docs] remove TF references from `/en/model_doc` (#40344) * models up to F * models up to M * all models * Fix: Only call Trainer.align_special_tokens if model has "config" attribute (#40322) * Only call Trainer.align_special_tokens if model has "config" attribute * Add efficient test for training a model without model.config * Reformat * add type hints (#40319) * add basic type hints to import module * run make fixup * remove optional * fixes --------- Co-authored-by: Matt <Rocketknight1@users.noreply.github.com> * Fix an infinite loop bug in recursive search of relative imports (#40326) Fix bug in recursive search of relative imports * Fix links in Glm4vMoe configuration classes to point to the correct H… (#40310) * Fix links in Glm4vMoe configuration classes to point to the correct Hugging Face model repository * run fixup to update links in Glm4vMoe configuration classes to point to the correct Hugging Face model repository * T5 test and target device fixes (#40313) * Fix cache setup related issues * Fix target-device-related issues * Ruff * Address review comments * Update `test_spm_converter_bytefallback_warning` (#40284) fff Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * (small) fix conditional for input_ids and input_embeds in marian (#40045) * (small) fix conditional for input_ids and input_embeds in marian * address comment * Fix attention vizualizer (#40285) * make visualizer rely on create causal mask * format * fixup * fixup * read token * read token, duh * what is up with that token * small tests? * adjust * try with flush * normalize for ANSI * buffer shenanigans * [ModernBert] Prevent the attention mask from being None in ModernBertForSequenceClassification (#35991) * [ModernBert] Prevent the attention mask from being None in ModernBertForSequenceClassification * fix the modular conversion * Clean up XCodec and other codecs (#40348) * Clean up xcodec addition. * Clean up config. * Switch to fixtures test. * Small stuff. * Polish XCodec and standardize across codecs. * Update src/transformers/models/xcodec/modeling_xcodec.py Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> * Format and fix test. * Update tol. --------- Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> * [serve] add cors warnings (#40112) * add cors warnings * Update src/transformers/commands/serving.py Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com> * Update src/transformers/commands/serving.py Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * Apply suggestions from code review * make fixup --------- Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com> Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * [detection] use consistent dtype for Conditional and DAB DETR positional embeddings (#40300) fix: use consistent dtype for sine positional embeddings * Remove more PyTorch 2.2 compatible code (#40337) Signed-off-by: cyy <cyyever@outlook.com> * [`FA`] Fix some model tests (#40350) * fix * cleanup, revert aimv2 fa changes * fix aria * i searched a long time but the cross dependency is for the recent models so... * this was something... evolla * fix modernbert decoder + make fa test more robust * nit * Qwen2.5-VL test fixes for ROCm (#40308) * [generate] handle support for cache classes when num enc layers != num dec layers (#40277) * handle support for cache classes when num enc layers != num dec layers * handle overwrites * one more corner case * Update src/transformers/generation/utils.py * Update src/transformers/generation/utils.py * Apply suggestions from code review * handle corner case :o * [4/N]more docs to device agnostic (#40355) * more docs to device agnostic Signed-off-by: YAO Matrix <matrix.yao@intel.com> * more Signed-off-by: YAO Matrix <matrix.yao@intel.com> * 1 Signed-off-by: YAO Matrix <matrix.yao@intel.com> * 2 Signed-off-by: YAO Matrix <matrix.yao@intel.com> * Update vitpose.md * Update camembert.md * Update camembert.md --------- Signed-off-by: YAO Matrix <matrix.yao@intel.com> * DOCS: Clarification on the use of `label_names` as an argument to TrainingArguments (#40353) * Update trainer.md * Update trainer.md Removed the detail about label_names argument usage from the tip/ warning section * Update training_args.py Added the label_names usage clarification in the docstring * Update trainer.md --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * HunYuan opensource (#39606) * merge opensource_hunyuan * add head_dim * fix assertion error * fix seen_tokens * ready_for_upstream (merge request !17) Squash merge branch 'ready_for_upstream' into 'main' * fix configuration type&docstring * fix style * ready_for_upstream (merge request !18) Squash merge branch 'ready_for_upstream' into 'main' * add doc * fix testcode * fix configuration type&docstring * rename base model * remove assert * update * remove tiktoken * update * fix moe and code style (#3) * update * fix format * update * revert makefile * fix moe config * fix numel() * remove prepare_inputs_for_generation * fix kv_seq_len * add docs/toctree * remove unused paramter&add licence * add licence * remove unused paramter * fix code * dense modular update import fix fix use mistralmodel fix qknorm add sliding_window make style fix dense done hunyuan moe fix import fix modular fixup fixup * update model path * fix mlp_bias * fix modular * Fix modeling (#5) …

* initial comment * test * initial conversion for outline * intermediate commit for configuration * chore:init files for sam2 * adding arbitary undefined config * check * add vision * make style * init sam2 base model * Fix imports * Linting * chore:sam to sam2 classes * Linting * Add sam2 to models.__init__ * chore:match prompt encoder with sam2 code * chore:prepare kwargs for mask decoder * Add image/video predictors * Add CUDA kernel * Add output classes * linting * Add logging info * tmp commit * docs for sam2 * enable image processing * check difference of original SAM2 - difference is the order of ToTensor() - please see https://pytorch.org/vision/main/_modules/torchvision/transforms/functional.html#resize * enable promptencoder of sam2 * fix promprencoder * Confirmed that PromptEncoder is exactly same (Be aware of bfloat16 and float32 difference) * Confirmed that ImageEncoder is exactly same (Be aware the linting of init) * Confirmed that MaskDecoder is exactly same (TO DO: lint variable name) * SamModel is now available (Need more chore for name) * make fix-copies * make style * make CI happy * Refactor VisionEncoder and PostioinEmbedding * TO DO : fix the image_embeddings and sparse_embeddings part * pure image inference done * reusable features fix and make style * styling * refactor memoryattention * tmp * tmp * refactor memoryencoder TO DO : convert and inference the video pipeline * TO DO : fix the image_encoder shape * conversion finish TO DO: need to check video inference * make style * remove video model * lint * change * python utils/check_docstringspy --check_all * python utils/check_config_attributes.py * remove copies for sam2promptencoder due to configuration * change __init__.py * remove tensorflow version * fix that to not use direct comparison * make style * add missing import * fix image_embedding_size * refactor Sam2 Attention * add fully working video inference (refactoring todo) * clarify _prepare_memory_conditioned_features * simplify modeling code, remove unused paths * use one model * use auto_docstring * refactor rope embeddings * nit * not using multimask when several points given * add all sam2.1 * add video tmp * add Sam2VideoSessionState + fast image proc + video proc * remove init_states from model * fix batch inference * add image integration tests * uniformize modeling code with other sam models and use modular * pass vision tests an most model tests * All tests passing * add offloading inference state and video to cpu * fix inference from image embedding and existing mask * fix multi_boxes mask inference * Fix batch images + batch boxes inference * improve processing for image inference * add support for mask generation pipeline * add support for get_connected_components post processing in mask generation * add fast image processor sam, image processor tests and use modular for sam2 image processor * fix mistake in sam after huggingface#39120 * fix init weights * refactor convert * add integration tests for video + other improvements * add needed missing docstrings * Improve docstrings and * improve inference speed by avoiding cuda sync * add test * skip test for vision_model * minor fix for vision_model * fix vision_model by adding sam2model and change the torch dependencies * remove patch_size * remove image_embedding_size * fix patch_size * fix test * make style * Separate hieradet and vision encoder in sam2 * fixup * review changes part 1 * remove MemoryEncoderConfig and MemoryAttentionConfig * pass q_stride instead of q_pool module * add inference on streamed videos * explicitely process streamed frames * nit * Improve docstrings in Sam2Model * update sam2 modeling with better gestion of inference state and cache, and separate Sam2Model and Sam2VideoModel * improve video inference api * change inference_state to inference_session * use modular for Sam2Model * fix convert sam2 hf * modular * Update src/transformers/models/sam2/video_processing_sam2.py Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * fix minor config * fix attention loading error * update modeling tests to use hub checkpoints * Use CI A10 runner for integration tests values + higher tolerance for video integration tests * PR review part 1 * fix doc * nit improvements * enforce one input format for points, labels and boxes * nit * last few nits from PR review * fix style * fix the input type * fix docs * add sam2 model as conversion script * improve sam2 doc * nit fixes + optimization * split sam2 and sam2_video in two models * PR review part 1 * fix None for default slow processor of sam2 * remove unecessary code path in sam2_video * refactor/simplify RoPE * replace embedding module list with embedding matrix * fix tests * remove kernel * nit * use lru_cache for sine_pos_embeddings * reorder sam2_video methods * simplify sam2_video * PR review part 1 * simplify sam2 video a lot * more simplification * update integration tests with updated conftest * more explicit config for hieradet * do post_processing outside of sam2 video model * Improve Sam2VideoVisionRotaryEmbedding * fix tests * update docs and fix mask2former/oneformer * avoid unnecessary reshapes/permute * fix device concatenating points * small dtype fix * PR review * nit * fix style and finish up doc * fix style * fix docstrings * fix modular --------- Co-authored-by: RUFFY-369 <prakarshkaushik369@gmail.com> Co-authored-by: Haitham Khedr <haithamkhedr@meta.com> Co-authored-by: sangbum choi <sangbumchoi@sangbumui-MacBookAir.local> Co-authored-by: yonigozlan <yoni.gozlan@huggingface.co> Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>

* Update notification service MI325 (#40078) add mi325 to amd_daily_ci_workflows * Fix PerceptionLM image preprocessing for non-tiled image input. (#40006) * Fix PerceptionLM image preprocessing for non-tiled image input. * Add test for single tile vanilla image processing. * ruff format * recover missing test skip * Simplify test. * minor test name fix * Revert FA2 kwargs construction (#40029) * revert * use imports * went way too high in imports level * style * [fix] batch inference for llava_onevision (#40021) * [fix] llava onevision batch inference * style * cannot pass inconsistent list & handle text-only case * [docs] Zero Shot Object Detection Task (#40096) * refactor zsod task docs * keeping the image guided od section * Apply suggestions from code review Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * Update docs/source/en/tasks/zero_shot_object_detection.md Co-authored-by: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com> --------- Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> Co-authored-by: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com> * Update Glm4V processor and add tests (#39988) * update GLm4V and add tests * Update tests/models/glm4v/test_processor_glm4v.py Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> * remove min/max pixels for BC * fix video tests --------- Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> * Add glm4.5&&glm4.5V doc (#40095) * Docs: GLM-4-MoE & GLM-4V-MoE pages * Docs: polish GLM-4V-MoE intro, remove placeholders; pin image * Docs --------- Co-authored-by: wujiahan <lambert@gmail.com> * Causal loss for `ForConditionalGeneration` (#39973) * feat: add ForConditionalGeneration loss to LOSS_MAPPING * consistent spelling of "recognized" * Audio encodings now match conv2d weight dtype in Gemma3nAudioSSCPConvBlock (#39743) audio encodings now match conv weight dtype in Gemma3nAudioSSCPConvBlock * New DynamicSlidingWindowLayer & associated Cache (#40039) * start adding the layer * style * improve * modular * fix * fix * improve * generate integration * comment * remove old one * remove * fix * fix * fix * fix all recompiles * fix * doc * fix * add text config check * fix encoderdecoder cache * add it for all models with sliding/hybrid support * revert * start fixing * prophetnet * fsmt * fix ddp_data * add test for mistral * improve mistral test and add gemma2 test * docstrings * Enable SIM rules (#39806) * Enable SIM rules Signed-off-by: cyy <cyyever@outlook.com> * More fixes Signed-off-by: cyy <cyyever@outlook.com> --------- Signed-off-by: cyy <cyyever@outlook.com> * feat: add `is_fast` to ImageProcessor (#39603) * feat: add `is_fast` to ImageProcessor * test_image_processing_common.py 업데이트 Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> * feat: add missing BaseImageProcessorFast import * fix: `issubclass` for discriminating subclass of BaseImageProcessorFast --------- Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co> * Re-apply make style (#40106) make style * Replace `logger.warning` with `logger.warning_once` in `GradientCheckpointingLayer` (#40091) * Fix regression in mllama vision encoder (#40083) fix mllama vision encoder Signed-off-by: Isotr0py <2037008807@qq.com> * Switch the order of args in StaticCache (for BC and future logic) (#40100) * switch order for BC and future logic * in generate as well * Fix Qwen3 MoE GGUF architecture mismatch (#39976) * fix qwen3moe gguf architecture * Fix Qwen3Moe GGUF loading --------- Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com> Co-authored-by: Jinuk Kim <jusjinuk@snu.ac.kr> * Fix error on importing unavailable torch.distributed (#40038) Currently model_debugging_utils.py would have an unguarded `import torch.distributed.tensor`. This PR ensures that the distributed module is available before including its tensor module. * Default to dequantize if cpu in device_map for mxfp4 (#39993) * default to dq if cpu * an other check * style * revert some changes * [`Flash Attention`] Fix flash attention integration (#40002) * fix flash attention * i got a stroke reading that comment * change dropout kwarg back to before * rename _fa3... as it's used for multiple variants and should work as fallback instead * simplify imports and support kwargs for fa * style * fix comments order * small fix * skip kernels test (causes cuda illegal memories w/o cleanup), fix fa test in general esp for models like bart * style * allow fullgraph by preloading on init * make globals "private" * ci pls be happy * change skip conditions based on backend flag (indicating missing mask interface) * move globals support to a function to prepare kwargs * style * generalize supported kwargs * small change to doc * fix * add comments * style * revert prep during generate * style * revert weird style changes * add fa kwarg prep during generate with fixes back * how did this even happen * how * add comment * [trainer] ensure special tokens in model configs are aligned with tokenizer at train time (#38441) * tmp commit * add test * make fixup * reset warns/info in test * Fix Causality Handling in Flash Attention to Support Bidirectional Attention (#39707) Fix the is_causal logic to enable bidirectional attention Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * [docs] Add reference to HF-maintained `custom_generate` collections (#39894) decoding -> generation; add collections * Add model card for MobileViT (#40033) * Add model card for MobileViT * Update docs/source/en/model_doc/mobilevit.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/mobilevit.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/mobilevit.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/mobilevit.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/mobilevit.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update mobilevit.md * Update mobilevit.md * Update mobilevit.md * Update docs/source/en/model_doc/mobilevit.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/mobilevit.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update mobilevit.md * Update mobilevit.md * Update mobilevit.md * Update mobilevit.md --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * remove sequence parallel in llama4 (#40084) * 🌐 [i18n-KO] Translated `tiny_agents.md` to Korean (#39913) * docs: ko: tiny_agents.md * feat: nmt draft * fix: manual edits * fix: manual edits * [bugfix] Fix tensor device in Idefics2, Idefics3, and SmolVLM (#39975) * [bugfix] ensure correct tensor device in Idefics2, Idefics3, and SmolVLM models * to cuda * changed xLSTMRMSNorm to RMSNorm (#40113) * changed xLSTMRMS.. to RMS... * fix linter error --------- Co-authored-by: Nikita <nikita@Nikitas-MacBook-Pro.local> * Fix QuantoQuantizedCache import issues (#40109) * fix quantoquantized * [serve] allow array `content` inputs for LLMs (#39829) fix bug; add tests * `decoding_method` argument in generate (#40085) * factor out expand inputs * callable arg * improve docs, add test * Update docs/source/en/generation_strategies.md Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> --------- Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> * Collated reports (#40080) * Add initial collated reports script and job definition * provide commit hash for this run. Also use hash in generated artifact name. Json formatting * tidy * Add option to upload collated reports to hf hub * Add glob pattern for test report folders * Fix glob * Use machine_type as path filter instead of glob. Include machine_type in collated report * DOCS: Add missing space in SECURITY.md (#40087) * [trainer] handle case where EOS token is None in `generation_config` (#40127) * handle case where EOS token is None in gen config * update eli5 dataset * Fix hidden torchvision>=0.15 dependency issue (#39928) * use pil_torch_interpolation_mapping for NEAREST/NEAREST_EXACT * fix min torchvision version * use InterpolationMode directly * remove unused is_torchvision_greater_or_equal, * nit * 🌐 [i18n-KO] Translated `main_classes/processors.md` to Korean (#39519) * docs: ko: processors.md * feat: nmt draft * fix: manual edits * Update docs/source/ko/main_classes/processors.md Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com> * Update docs/source/ko/main_classes/processors.md Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com> --------- Co-authored-by: TaskerJang <bymyself103@naver.com> Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com> * 🌐 [i18n-KO] Translated `jamba.md` to Korean (#39890) * docs: ko: jamba.md * feat: nmt draft * fix: manual edits * fix: resolve suggestion Co-authored-by: Minseo Kim <75977640+luckyvickyricky@users.noreply.github.com> --------- Co-authored-by: Minseo Kim <75977640+luckyvickyricky@users.noreply.github.com> * 🌐 [i18n-KO] Translated `main_classes/optimizer_schedules.md` to Korean (#39713) * docs: ko: main_classes/optimizer_schedules * feat: nmt draft * fix: improve TOC anchors and expressions in optimizer_schedules - Add TOC anchors to all section headers - Fix terminology and improve Korean expressions * fix: Correct translation of 'weight decay fixed' to '가중치 감쇠가 적용된' Changed '가중치 감쇠가 수정된' to '가중치 감쇠가 적용된' for more accurate translation of 'weight decay fixed' in the context of optimization. * fix: Use more natural Korean inheritance expression Changed '에서 상속받는' to '을 상속받는' to follow natural Korean grammar patterns for inheritance terminology. * fix: Use consistent '미세 조정' translation for 'finetuned models' Changed '파인튜닝된' to '미세 조정된 모델' to follow the established translation glossary for 'finetuned models' terminology. * 🚨🚨 [generate] ignore `cache_implementation="hybrid"` hub defaults (#40135) * working? * fix tests * 🌐 [i18n-KO] Translated `gpt2.md` to Korean (#39808) * docs: ko: bamba.md * feat: nmt draft * fix: manual edits * docs: ko: gpt2.md * feat: nmt draft * fix: manual edits * Remove bamba.md from docs/source/ko/model_doc/ * Update _toctree.yml * 🌐 [i18n-KO] Translated `optimizers.md` to Korean (#40011) * docs: ko: optimizers.md * feat: optimizers draft * fix: manual edits * docs: ko: update optimizers.md * Update docs/source/ko/optimizers.md Co-authored-by: Minseo Kim <75977640+luckyvickyricky@users.noreply.github.com> * Update docs/source/ko/optimizers.md Co-authored-by: Minseo Kim <75977640+luckyvickyricky@users.noreply.github.com> * Update docs/source/ko/optimizers.md Co-authored-by: Jaehyeon Shin <108786184+skwh54@users.noreply.github.com> * docs: ko: final updates to optimizers and toctree --------- Co-authored-by: Minseo Kim <75977640+luckyvickyricky@users.noreply.github.com> Co-authored-by: Jaehyeon Shin <108786184+skwh54@users.noreply.github.com> * 🌐 [i18n-KO] Translated grounding-dino.md to Korean (#39861) * docs: ko: grounding-dino.md * feat: nmt draft * fix: manual edits * Update docs/source/ko/model_doc/grounding-dino.md Co-authored-by: Kim Juwon <81630351+Kim-Ju-won@users.noreply.github.com> * Update docs/source/ko/model_doc/grounding-dino.md Co-authored-by: Kim Juwon <81630351+Kim-Ju-won@users.noreply.github.com> * Update docs/source/ko/model_doc/grounding-dino.md Co-authored-by: Kim Juwon <81630351+Kim-Ju-won@users.noreply.github.com> * docs: add AP explanation for better readability --------- Co-authored-by: TaskerJang <bymyself103@naver.com> Co-authored-by: Kim Juwon <81630351+Kim-Ju-won@users.noreply.github.com> Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com> * 🚨 Use lru_cache for sine pos embeddings MaskFormer (#40007) * use lru_cache for sine pos embeddings maskformer * fix calls to pos embed * change maxsize to 1 * 🌐 [i18n-KO] Translated `pipelines.md` to Korean (#39577) * docs: ko: pipelines.md * feat: gpt draft * Update docs/source/ko/main_classes/pipelines.md Co-authored-by: Yijun Lee <119404328+yijun-lee@users.noreply.github.com> * Update docs/source/ko/main_classes/pipelines.md Co-authored-by: Yijun Lee <119404328+yijun-lee@users.noreply.github.com> * Update docs/source/ko/main_classes/pipelines.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/ko/main_classes/pipelines.md Co-authored-by: Yijun Lee <119404328+yijun-lee@users.noreply.github.com> * Update docs/source/ko/main_classes/pipelines.md Co-authored-by: Yijun Lee <119404328+yijun-lee@users.noreply.github.com> * Update _toctree.yml * Update _toctree.yml 번역 문서 수정 * Update pipelines.md ToC 수정 * Update pipelines.md --------- Co-authored-by: xhaktm <tnwjd318@hs.ac.kr> Co-authored-by: Yijun Lee <119404328+yijun-lee@users.noreply.github.com> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * gpt oss is important (#40139) * Fix Janus (#40140) fix * Add Segment Anything 2 (SAM2) (#32317) * initial comment * test * initial conversion for outline * intermediate commit for configuration * chore:init files for sam2 * adding arbitary undefined config * check * add vision * make style * init sam2 base model * Fix imports * Linting * chore:sam to sam2 classes * Linting * Add sam2 to models.__init__ * chore:match prompt encoder with sam2 code * chore:prepare kwargs for mask decoder * Add image/video predictors * Add CUDA kernel * Add output classes * linting * Add logging info * tmp commit * docs for sam2 * enable image processing * check difference of original SAM2 - difference is the order of ToTensor() - please see https://pytorch.org/vision/main/_modules/torchvision/transforms/functional.html#resize * enable promptencoder of sam2 * fix promprencoder * Confirmed that PromptEncoder is exactly same (Be aware of bfloat16 and float32 difference) * Confirmed that ImageEncoder is exactly same (Be aware the linting of init) * Confirmed that MaskDecoder is exactly same (TO DO: lint variable name) * SamModel is now available (Need more chore for name) * make fix-copies * make style * make CI happy * Refactor VisionEncoder and PostioinEmbedding * TO DO : fix the image_embeddings and sparse_embeddings part * pure image inference done * reusable features fix and make style * styling * refactor memoryattention * tmp * tmp * refactor memoryencoder TO DO : convert and inference the video pipeline * TO DO : fix the image_encoder shape * conversion finish TO DO: need to check video inference * make style * remove video model * lint * change * python utils/check_docstringspy --check_all * python utils/check_config_attributes.py * remove copies for sam2promptencoder due to configuration * change __init__.py * remove tensorflow version * fix that to not use direct comparison * make style * add missing import * fix image_embedding_size * refactor Sam2 Attention * add fully working video inference (refactoring todo) * clarify _prepare_memory_conditioned_features * simplify modeling code, remove unused paths * use one model * use auto_docstring * refactor rope embeddings * nit * not using multimask when several points given * add all sam2.1 * add video tmp * add Sam2VideoSessionState + fast image proc + video proc * remove init_states from model * fix batch inference * add image integration tests * uniformize modeling code with other sam models and use modular * pass vision tests an most model tests * All tests passing * add offloading inference state and video to cpu * fix inference from image embedding and existing mask * fix multi_boxes mask inference * Fix batch images + batch boxes inference * improve processing for image inference * add support for mask generation pipeline * add support for get_connected_components post processing in mask generation * add fast image processor sam, image processor tests and use modular for sam2 image processor * fix mistake in sam after #39120 * fix init weights * refactor convert * add integration tests for video + other improvements * add needed missing docstrings * Improve docstrings and * improve inference speed by avoiding cuda sync * add test * skip test for vision_model * minor fix for vision_model * fix vision_model by adding sam2model and change the torch dependencies * remove patch_size * remove image_embedding_size * fix patch_size * fix test * make style * Separate hieradet and vision encoder in sam2 * fixup * review changes part 1 * remove MemoryEncoderConfig and MemoryAttentionConfig * pass q_stride instead of q_pool module * add inference on streamed videos * explicitely process streamed frames * nit * Improve docstrings in Sam2Model * update sam2 modeling with better gestion of inference state and cache, and separate Sam2Model and Sam2VideoModel * improve video inference api * change inference_state to inference_session * use modular for Sam2Model * fix convert sam2 hf * modular * Update src/transformers/models/sam2/video_processing_sam2.py Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * fix minor config * fix attention loading error * update modeling tests to use hub checkpoints * Use CI A10 runner for integration tests values + higher tolerance for video integration tests * PR review part 1 * fix doc * nit improvements * enforce one input format for points, labels and boxes * nit * last few nits from PR review * fix style * fix the input type * fix docs * add sam2 model as conversion script * improve sam2 doc * nit fixes + optimization * split sam2 and sam2_video in two models * PR review part 1 * fix None for default slow processor of sam2 * remove unecessary code path in sam2_video * refactor/simplify RoPE * replace embedding module list with embedding matrix * fix tests * remove kernel * nit * use lru_cache for sine_pos_embeddings * reorder sam2_video methods * simplify sam2_video * PR review part 1 * simplify sam2 video a lot * more simplification * update integration tests with updated conftest * more explicit config for hieradet * do post_processing outside of sam2 video model * Improve Sam2VideoVisionRotaryEmbedding * fix tests * update docs and fix mask2former/oneformer * avoid unnecessary reshapes/permute * fix device concatenating points * small dtype fix * PR review * nit * fix style and finish up doc * fix style * fix docstrings * fix modular --------- Co-authored-by: RUFFY-369 <prakarshkaushik369@gmail.com> Co-authored-by: Haitham Khedr <haithamkhedr@meta.com> Co-authored-by: sangbum choi <sangbumchoi@sangbumui-MacBookAir.local> Co-authored-by: yonigozlan <yoni.gozlan@huggingface.co> Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * [docs] Fix ko toctree (#40138) Update _toctree.yml * Remove an old badly designed test (#40142) remove it * updated visualBERT modelcard (#40057) * updated visualBERT modelcard * fix: Review for VisualBERT card * 🌐 [i18n-KO] Translated `gemma3.md` to Korean (#39865) * docs: ko: gemma3.md * feat: nmt draft * fix: manual edits * fix: resolve suggestions Co-authored-by: Chaewon Song <chaewon1019@ewhain.net> * fix: resolve suggestions --------- Co-authored-by: Chaewon Song <chaewon1019@ewhain.net> * Fix quantized cache with only cache_implementation in generate (#40144) * fix args * comment * Add pytest marker: `torch_compile_test` and `torch_export_test` (#39950) * new marker * trigger CI * update --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Update Dockerfiles to install packages inside a virtual environment (#39098) * Removed un-necessary virtual environment creation in Dockerfiles. * Updated Dockerfiles to install packages in a virtual environment. * use venv's python * update * build and trigger * trigger * build and trigger * build and trigger * build and trigger * build and trigger * build and trigger * build and trigger * update * update * update * update --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Create self-scheduled-amd-mi355-caller.yml (#40134) * [Cohere2Vision] remove unused arg (#40103) * remove unused arg * remove the arg from test as well * [efficientloftr] fix bugs and follow original cross attn implementation strictly (#40141) * fix: changed is_causal to be False * fix: Added original cross attention bug * fix: fixed the way bordel removal is computed * fix: added missing normalization on coarse features * test: fixed integration tests --------- Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * Fix CI: Use correct import in SAM for torchvision InterpolationMode (#40160) fix ci * [Continous Batching] set head_dim when config.head_dim is None (#40159) * set head_dim when config.head_dim is None * use model's actual TP setting * Replace `self.tokenizer` by `self.processing_class` (#40119) * [FA2] Fix it finally - revert fa kwargs preparation (#40161) revert * [bugfix] fix flash-attention2 unavailable error for Ascend NPU (#40151) * [bugfix] fix flash-attention2 unavailable error for Ascend NPU * remove redundant apply_rotary_emb usage * fix ruff check error * pad_input and unpad_input use same implementation as fa2 * rollback redundant codes * fix ruff check error * optimize fa2 judgement logic * Fix docs typo (#40167) * DINOv3 model * working version * linter revert * linter revert * linter revert * fix init * remove flex and add convert to hf script * DINOv3 convnext * working version of convnext * adding to auto * Dinov3 -> DINOv3 * PR feedback * complete convert checkpoint * fix assertion * bf16 -> fp32 * add fast image processor * fixup * change conversion script * Use Pixtral attention * minor renaming * simplify intermediates capturing * refactor DINOv3ViTPatchEmbeddings * Refactor DINOv3ViTEmbeddings * [WIP] rope: remove unused params * [WIP] rope: rename period -> inv_freq for consistency * [WIP] rope: move augs * change inv_freq init (not persistent anymore) * [WIP] rope: move coords to init * rope - done! * use default LayerScale * conversion: truncate expected outputs * remove commented code * Refactor MLP layers * nit * clean up config params * nit docs * simplify embeddings * simplify compile compat lru_cache * fixup * dynamic patch coords * move augmentation * Fix docs * fixup and type hints * fix output capturing * fix tests * fixup * fix auto mappings * Add draft docs * fix dtype cast issue * add push to hub * add image processor tests * fixup * add modular * update modular * convert and test convnext * update conversion script * update prefix * Update LayerNorm * refactor DINOv3ConvNextLayer * rename * refactor convnext model * fix doc check * fix docs * fix convnext config * tmp fix for check docstring * remove unused arg * fix tests * (nit) change init * standardize gated MLP * clear namings and sat493m * fix tensors on different devices * revert linter * pr * pr feedbak ruff format * missing headers * fix code snippet and collection link in docs * DINOv3 description * fix checkpoints in tests * not doc fixes in configs * output_hidden_states * x -> features * remove sequential --------- Co-authored-by: Cijo Jose <cijose@meta.com> * build: Add fast image processor tvp (#39529) * build: add TvpImageProcessorFast - Introduced TvpImageProcessorFast to enhance image processing capabilities. - Updated image processing auto registration to include the new fast processor. - Modified tests to accommodate both TvpImageProcessor and TvpImageProcessorFast, ensuring comprehensive coverage for both classes. * fix: TvpImageProcessorFast with new resize method and update processing logic * build: add TvpImageProcessorFast * refactor: clean up whitespace and formatting in TvpImageProcessorFast and related tests - Removed unnecessary whitespace and ensured consistent formatting in image_processing_tvp_fast.py. - Updated import order in test_image_processing_tvp.py for clarity. - Minor adjustments to maintain code readability and consistency. * fix: Enhance TvpFastImageProcessorKwargs and update documentation - Added TvpFastImageProcessorKwargs class to define valid kwargs for TvpImageProcessorFast. - Updated the documentation in tvp.md to include the new class and its parameters. - Refined the image processing logic in image_processing_tvp_fast.py for better handling of padding and resizing. - Improved test cases in test_image_processing_tvp.py to ensure compatibility with the new processing logic and tensor inputs. * fix: tested now with python 3.9 * fix: remove tvp kwargs from docs * simplify processing * remove import and fix tests --------- Co-authored-by: yonigozlan <yoni.gozlan@huggingface.co> * Add GptOssForSequenceClassification for GPT-OSS models (#40043) * Add GptOssForSequenceClassification * Tiny fix * make fixup * trigger CI rerun * Check config type instead --------- Co-authored-by: Yuefeng Zhan <yuefzh@microsoft.com> * Standardize BARTpho model card: badges, new examples, fixed broken im… (#40051) * Standardize BARTpho model card: badges, new examples, fixed broken image section, and links (#36979)Update bartpho.md * Update bartpho.md Removed non-required/unsupported sections: Quantization, Attention visualizer, and Resources (plus stray tokenizer header). Added code snippets which were suggested * Update bartpho.md Updated with necessary tags * Update bartpho.md * Update bartpho.md --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Add dates to the model docs (#39320) * added dates to the models with a single hf papers link * added the dates for models with multiple papers * half of no_papers models done * rest of no_papers models also done, only the exceptions left * added copyright disclaimer to sam_hw, cohere, cohere2 + dates * some more fixes, hf links + typo * some new models + a rough script * the script looks robust, changed all paper links to hf * minor change to handle technical reports along with blogs * ran make fixup to remove the white space * refactor * Pin torch to 2.7.1 on CircleCI for now (#40174) * fix * fix --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Update dynamic attnt setter for multimodals (#39908) * update * fix the test for DepthPro * PR comments * wait, I didn't delete this in prev commit? * fix * better way --------- Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co> Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com> * [MINOR:TYPO] Update base.py (#40169) * [MINOR:TYPO] Update base.py All other occurrences in the docs use lowercase. (https://github.com/search?q=repo%3Ahuggingface%2Ftransformers%20translation_XX_to_YY&type=code) Also, using uppercase doesn't work: tested with "translation_EN_to_FR" which doesn't work and instead returns: `ValueError: The task does not provide any default models for options ('EN', 'FR')` It might be a good idea to allow for uppercase, but that's for another issue. * [MINOR:TYPO] Update __init__.py * make model doc device agnostic (#40143) * make model doc device agnostic Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * Update align.md * Update aya_vision.md * Update byt5.md * refine Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * Update granitevision.md * Update src/transformers/pytorch_utils.py Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * add doc Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * 3 more Signed-off-by: Yao, Matrix <matrix.yao@intel.com> --------- Signed-off-by: Yao, Matrix <matrix.yao@intel.com> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * fix to avoid modifying a view in place (#40162) * fix to avoid modifying a view in place * add backward test in tensor parallel * add test to test_modelig_gpt_oss.py * linting * Fix fsdp for generic-task models (#40191) * remove abc inheritance * add fast test * Add repr to EncoderDecoderCache (#40195) * add repr * oups * Fix typos (#40175) Signed-off-by: cyy <cyyever@outlook.com> * Remove _prepare_flash_attention_from_position_ids (#40069) Signed-off-by: cyy <cyyever@outlook.com> * Avoid CUDA stream sync (#40060) Signed-off-by: cyy <cyyever@outlook.com> * Fix various Pylint warnings (#40107) Tidy code Signed-off-by: cyy <cyyever@outlook.com> * Update: add type hints to check_tokenizers.py (#40094) * Update check_tokenizers.py chore(typing): add type hints to check_tokenizers script - Annotate params/returns for helper functions - Keep tokenizer instances as `Any` to avoid runtime coupling - Make `check_LTR_mark` return `bool` explicitly (no behavior change) * Update check_tokenizers.py chore(typing): replace Any with PreTrainedTokenizerBase in check_tokenizers.py - Use transformers.tokenization_utils_base.PreTrainedTokenizerBase for `slow` and `fast` params - Covers both PreTrainedTokenizer and PreTrainedTokenizerFast - Exposes required methods (encode, decode, encode_plus, tokenize) - Removes generic Any typing while staying implementation-agnostic * Benchmarking improvements (#39768) * Start revamping benchmarking * Start refactoring benchmarking * Use Pandas for CSV * import fix * Remove benchmark files * Remove sample data * Address review comments * Add X-Codec model (#38248) * add working x-codec * nit * fix styling + copies * fix docstring * fix docstring and config attribute * Update args + config * update convertion script * update docs + cleanup * Ruff fix * fix doctrings * Fix GPT-OSS `swiglu_limit` not passed in for MXFP4 (#40197) Add swiglu_limit = 7.0 * docs: Update LayoutLM model card according to new standardized format (#40129) * docs: Update LayoutLM model card with standardized format * Apply suggestions from code review This commit incorporates all suggestions provided in the recent review. Further changes will be committed separately to address remaining comments. Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Address remaining review comments * Address few more review comments: 1. remove transformer-cli section 2. put resources after notes 3. change API refs to 2nd level header * Update layoutlm.md * Update layoutlm.md --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Revert "Pin torch to 2.7.1 on CircleCI for now" + Final fix for `too long with no output` (#40201) * Revert "Pin torch to 2.7.1 on CircleCI for now (#40174)" This reverts commit 31b6e6e1dac0d32f74ec5cd6b3c1868534ccd7b5. * fix --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Use correct `model_input_names` for PixtralImageProcessor (#40226) add image_sizes to model_input_names * fix error vocab_size at Qwen2_5_VLForConditionalGeneration loss_function (#40130) * fix error vocab_size at Qwen2_5_VLForConditionalGeneration loss_function Signed-off-by: luoxiaoc <xiaochuan.luo@intel.com> * fix similar errer at qwen2_vl and do make fix-copies Signed-off-by: luoxiaoc <xiaochuan.luo@intel.com> * pass in kwargs for loss_func at qwen2_vl and qwen2_5_vl Signed-off-by: luoxiaoc <xiaochuan.luo@intel.com> * Apply style fixes --------- Signed-off-by: luoxiaoc <xiaochuan.luo@intel.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * [SAM 2] Change checkpoints in docs and tests (#40213) * change checkpoints in docs and tests * add notebook * Fix more typos (#40212) Signed-off-by: cyy <cyyever@outlook.com> * Fix ESM token_dropout crash when using inputs_embeds instead of input_ids (#40181) * fix: Error after calling ESM model with input embeddings not input ids * propagate changes to other models * AMD scheduled CI ref env file (#40243) * Reference env-file to be used in docker running the CI * Disable MI300 CI for now * Add Ovis2 model and processor implementation (#37088) * Add Ovis2 model and processor implementation * Apply style fixes * Add unit tests for Ovis2 image processing and processor * Refactor image processing functions for clarity and efficiency * Add Ovis2 ImageProcessorFast * Refactor Ovis2 code * Refactor Ovis2 model components and update processor functionality * Fix repo consistency issues for Ovis2: docstring, config cleanup * Update Ovis2 model integration tests * Update Ovis2 configuration and processing classes for improved documentation * Remove duplicate entry for 'ovis2' in VLM_CLASS_NAMES * Fix conflict * Fix import order * Update image processor class names * Update Ovis2 model structure * Refactor Ovis2 configuration * Fix typos * Refactor Ovis2 model classes and remove unused code * Fix typos * Refactor Ovis2 model initialization * Fiix typos * Remove Ovis2 model mapping from MODEL_MAPPING_NAMES in modeling_auto.py * Add license and update type hints * Refactor token function and update docstring handling * Add license * Add Ovis2 model support and update documentation * Refactor Ovis2 model structure and enhance multimodal capabilities * Update Ovis2 weight mapping for consistency and clarity in key patterns * Remove unused 'grids' parameter from Ovis2 model and Update processing logic to handle image grids more efficiently. * Refactor Ovis2 model test structure to include Ovis2Model * Add optional disable_grouping param to Ovis2ImageProcessorFast * Refactor type hints in Ovis2 modules * Add licensing information in Ovis2 modules and tests * Refactor Ovis2 model by removing unused methods * Refactor Ovis2 model tests by renaming test classes and removing skipped tests * Refactor Ovis2 model output classes * Refactor Ovis2 weight conversion and Update model embedding classes * Refactor Ovis2 model imports and remove unused functions * Enhance vision configuration extraction in Ovis2 weight conversion * Refactor Ovis2 model's forward method to remove interpolation option * Update Ovis2 model documentation * Refactor Ovis2 model input handling and tokenizer configuration * Update return type hints in Ovis2 model * Remove commented-out code * fix config for tests and remove key mappings * Update tokenizer configuration to use add_special_tokens method * skip torchscript * Fix image placeholder generation in Ovis2Processor * Refactor Ovis2 model to rename visual_table to visual_embeddings_table * Enhance Ovis2 model by adding vision_feature_select_strategy parameter * Refactor Ovis2 model weights conversion and architecture * Refactor Ovis2 model by removing vision_feature_select_strategy parameter * Update Ovis2 model examples * Refactor Ovis2 model * Update Ovis2 model * Update Ovis2 model configuration * Refactor Ovis2 model test setup * Refactor flash attention support * Refactor * Fix typo * Refactor * Refactor model classes * Update expected output in Ovis2 * Refactor docstrings * Fix * Fix * Fix * Update input in tests * Fix * Fix get_decoder method * Refactor * Refactor Ovis2 * Fix * Fix * Fix test * Add get_placeholder_mask * Refactor Ovis2 model tests * Fix * Refactor * Fix * Fix * Fix Ovis2 test --------- Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com> * Fix more pylint warnings (#40204) Fix pylint warnings Signed-off-by: cyy <cyyever@outlook.com> * 🚨 Always return Cache objects in modelings (to align with generate) (#39765) * watch the world burn * fix models, pipelines * make the error a warning * remove kwargs and return_legacy_cache * fix reformer * remove transpose_for_scores call in ESM-2 (#40210) * remove transpose_for_scores call Signed-off-by: Peter St. John <pstjohn@nvidia.com> * fix copied evolla code Signed-off-by: Peter St. John <pstjohn@nvidia.com> --------- Signed-off-by: Peter St. John <pstjohn@nvidia.com> * Add `chat_template` (`jinja2`) as an extra dependency (#40128) * add jinja2 as a dependency * Make jinja2 a core dependency in install_requires - Add jinja2 to install_requires list in setup.py for automatic installation - Add jinja2 to runtime version checks in dependency_versions_check.py - Resolves issue where pip install transformers doesn't install jinja2 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Make jinja2 a core dependency in install_requires * Make jinja2 an extra dependency instead of adding a core dep --------- Co-authored-by: Claude <noreply@anthropic.com> * [typing] fix type annotation error in DepthPro model image processor (#40238) * fix type annotation error in DepthPro model image processor * fix * run make fix-copies * [serve] guard imports (#39825) guard imports * [`CI`] Fix repo consistency (#40249) * fix * doc --------- Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com> * Fixes for EncoderDecoderCache (#40008) * Add expectation to t5 for rocm 9.4 * Made EncoderDecoderCache compatible with nn.DataParallel * Fixed t5gemma EncoderDecoderCache * Added todos in autoformer * Ruff * Init is self-contained * Review compliance * Fixed kwargs init of EncoderDecoderCache * fix: Catch correct ConnectionError for additional_chat_templates (#39874) * fix: Catch correct ConnectionError for additional_chat_templates * fix: don't catch timeout * fix: formatting * Model card for NLLB (#40074) * initializing branch and draft PR * updated model card .md file * minor * minor * Update docs/source/en/model_doc/nllb.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * resolving comments + adding visuals * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md suggestion Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/nllb.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * NllbTokenizerFast and NllbTokenizer added * endline * minor * Update nllb.md --------- Co-authored-by: Sahil Kabir <sahilkabir@Sahils-MacBook-Pro.local> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Correct typo and update notes in docs Readme (#40234) * Correct typo and update notes in docs readme * Update docs/README.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/README.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Fix benchmark workflow (#40254) Correct init_db.sql path Co-authored-by: Akos Hadnagy <akoshuggingface@mi325x8-123.atl1.do.cpe.ice.amd.com> * docs: Update OLMo model card (#40233) * Updated OLMo model card * Update OLMo description Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Fix typo Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Fix cli typo Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Fix cli example Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Add bitsandbytes info Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Skip broken tests (#40157) skip these tests * Remove MI300 CI (#40270) Remove MI300 CI (in history if we need it back) * set inputs_embeds to None while generate to avoid audio encoder forward in generation process (#40248) * set inputs_embeds to None while generate to avoid audio encoder forward in generation process * set input_features to none instead --------- Co-authored-by: lvyuanjun.lyj <lvyuanjun.lyj@alibaba-inc.com> * [detection] fix attention mask for RT-DETR-based models (#40269) * Fix get_contrastive_denoising_training_group attention * Add bool attention_mask conversion * Fix slow static cache export tests (#40261) * 🚨🚨 Switch default compilation to fullgraph=False (#40137) * switch default * docstring * docstring * rework tests and remove outdated restrictions * simplify * we need a check for static cache * fix * rename var * fix * revert * style * rename test * Fix setting attention for multimodal models (#39984) * fix * use non-explicit `None` * keep previously set attn if exists * [detection] fix correct `k_proj` weight and bias slicing in D-FINE (#40257) Fix: correct k_proj weight and bias conversion in D-FINE * Add Kosmos-2.5 (#31711) Add Microsoft Kosmos-2.5 --------- Co-authored-by: kirp@umich.edu <tic-top> Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Skipping pytree registration in case fsdp is enabled (#40075) * Skipping pytree registration in case fsdp is enabled * Beauty changes * Beauty changes * Moved the is_fsdp_available function to import utils * Moved is_fsdp_available to integrations.fsdp * Skipping pytree registration in case fsdp is enabled * Beauty changes * Beauty changes * Moved the is_fsdp_available function to import utils * Moved is_fsdp_available to integrations.fsdp * Added pytree registration inside dynamic cache class * Making ci/cd lords happy * Adding a check if DynamicCache is already a leaf * Adding try/catch for multiple initializations of DynamicCache in test suites * Moving dynamic cache pytree registration to executorch * Adding try catch back * Update image_processing_perception_lm_fast.py to allow for proper override of vision_input_type (#40252) * Update image_processing_perception_lm_fast.py Allow for a proper override of vision_input_type in hf fast image processor, otherwise we need to resort to manually setting the attribute. * Update processing_perception_lm.py to match kwargs vision input type * Update image_processing_perception_lm_fast.py kwargs to signature args * fix which routing method (#40283) * Fix chat CLI GPU loading and request_id validation issues (#40230) (#40232) * Fix chat CLI GPU loading and request_id validation issues (#40230) This commit addresses two critical bugs in the transformers chat CLI: 1. **GPU Loading Issue**: Changed default device from "cpu" to "auto" in ChatArguments - Chat CLI now automatically uses GPU when available instead of defaulting to CPU - Matches the behavior of the underlying serving infrastructure 2. **Request ID Validation Error**: Added request_id field to TransformersCompletionCreateParamsStreaming schema - Fixes "Unexpected keys in the request: {'request_id'}" error on second message - Allows request_id to be properly sent and validated by the server Both fixes target the exact root causes identified in issue #40230: - Users will now get GPU acceleration by default when available - Chat sessions will no longer break after the second message * Remove unrelated request_id field from TransformersCompletionCreateParamsStreaming * docs(layoutlm): add missing `id=usage` to `<hfoptions>` tag in LayoutLM model card (#40273) docs(layoutlm): add missing 'id=usage' to <hfoptions> tag in LayoutLM model card * Standardize RAG model card (#40222) * Standardize RAG model card Update rag.md to follow the new Hugging Face model card template: - Added friendly overview in plain language - Added pipeline and AutoModel usage examples - Included quantization example with BitsAndBytesConfig - Added notes and resources sections - Removed abstract and FlashAttention badge * Standardize RAG model card Update rag.md to follow the new Hugging Face model card template: - Added friendly overview in plain language - Added AutoModel usage example - Included quantization example with BitsAndBytesConfig * docs: Update TrOCR model card to new format (#40240) * docs: Update TrOCR model card to new format * Updated Sugegestions * Update model card for gpt neox japanese (#39862) * Update GPT-NeoX-Japanese model card * Apply suggestions from code review * Update gpt_neox_japanese.md --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * SmolVLM and InternVL: Ensure pixel values are converted to the correct dtype for fp16/bf16 (#40121) * Ensure pixel values are converted to the correct dtype for fp16/bf16 * add to modular * Standardize BertGeneration model card (#40250) * Standardize BertGeneration model card: new format, usage examples, quantization * Update docs/source/en/model_doc/bert-generation.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/bert-generation.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/bert-generation.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/bert-generation.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/bert-generation.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/bert-generation.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/model_doc/bert-generation.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Apply reviewer feedback: update code examples * Add missing code example --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Adjust ROCm test output expectations (#40279) Adjust ROCm output expectations * SmolVLM test fixes (#40275) * Fix SmolVLM tests * Add the proper CUDA expectations as well * Split 'A10 and A100 expectations * Ruff --------- Co-authored-by: Akos Hadnagy <akoshuggingface@mi325x8-123.atl1.do.cpe.ice.amd.com> * make model docs device agnostic (2) (#40256) * doc cont. Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * more models Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * Update docs/source/en/quicktour.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/quicktour.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/quicktour.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/quicktour.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update mixtral.md --------- Signed-off-by: Yao, Matrix <matrix.yao@intel.com> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * [3/3] make docs device agnostic, all en docs for existing models done (#40298) docs to device agnostic cont. Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * Add MetaCLIP 2 (#39826) * First draft * Make fixup * Use eos_token_id * Improve tests * Update clip * Make fixup * Fix processor tests * Add conversion script * Update docs * Update tokenization_auto * Make fixup * Use check_model_inputs * Rename to lowercase * Undo CLIP changes * Address comment * Convert all checkpoints * Update auto files * Rename checkpoints * Allow to be able to run `torch.compile` tests with `fullgraph=True` (#40164) * fix * address comment --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * [`FA`] Fix dtype in varlen with position ids (#40295) fix * [docs] delete more TF/Flax docs (#40289) * delete some TF docs * update documentation checks to ignore tf/flax * a few more removals * nit * Update utils/check_repo.py Co-authored-by: Matt <Rocketknight1@users.noreply.github.com> --------- Co-authored-by: Matt <Rocketknight1@users.noreply.github.com> * Clean up X-Codec. (#40271) * Clean up xcodec addition. * Clean up config. * Switch to fixtures test. * Small stuff. * Remove OTel SDK dependencies (#40305) * Fix GOT-OCR2 and Cohere2Vision image processor patches caculation (#40312) fix got-ocr patches caculation Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn> * [`fix`] Pass adamw optimizer parameters to StableAdamW (#40184) * fix: pass adamw optimizer parameters to StableAdamW * add test for stable_adamw initialization with trainer arguments * address copilot suggestion * fix: update weight_decay handling in stable_adamw kwargs --------- Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> * chore: fix typo in `find_executable_batch_size` to match new 0.9 ratio (#40206) * :rotating_light: [`Flash Attention`] Fix sliding window size (#40163) * swa fix * add comment, make fix symmetrical * modify fa inference test to force swa correctness check * fixup comment * Remove unnecessary contiguous calls for modern torch (#40315) * Add support for Florence-2 (#38188) * init * add modular * fixup * update configuration * add processing file * update auto files * update * update modular * green setup_and_quality ci * it works * fix some tests * commit florence2 * update test * make test cases done - 16 left * style * fix few test cases * fix some tests * fix init test * update florence2 vision style * hope is green * fix init test * fix init * update modular * refactor vision module * fix: channel attention use dynamic scale * update modular * update * update attention mask * update * fix naming * Update src/transformers/models/florence2/processing_florence2.py Co-authored-by: Matt <Rocketknight1@users.noreply.github.com> * spatial block works * more beautiful * more more beautiful * merge main * merge main and fixup * fix typing hint * update modeling * fix eager matches sdpa * fix style * fix compile test - all green * remove florence2 language * remove Florence2LanguageModel things * fix style * update florence2 model * override prepare encoder_decoder for generation * add weight conversion script * rewrite channel attention to use sdpa * eleminate 1 tranpose op * support fa2 * fix quality check * chore: reformat `test_modeling_florence2.py` * some refactor for processor * some refactor for processor * update naming convention and remove BC * make it pass the test * fix: correct Embedding Cosine * update comments and docstring * support input_embeds * support input embeds ideally * fix style * fix style * fix style again :D * add test prcoessor * refactor processor and add test for processor * reformat test processor * make fixup * fix schema check * remove image_token * ensure image token in tokenizer and fix integration tests * fix processor test * add more integration tests for large model and rename test_processor to test_processing * test_assisted_decoding_sample should pass * update doc and make model work with image text to text pipeline * docs: add sdpa bagde * resolve cyril's comments * fix import torch error * add helper get_placeholder_mask * inherit from llava * florence2 may not _supports_attention_backend because of bart ... * move florence2 model card to multimodal * let base model always return_dict * fix style * tiny update doc * set _checkpoint_conversion_mapping = {} * fix code quality * support flex and compile graph and move external func to internal func * remove condition because it always true * remove window funcs * move post processor config out * fix ci * new intro to trigger test * remove `kernel_size` argument --------- Co-authored-by: ducviet00-h2 <viet.d.hoang@h2corporation.jp> Co-authored-by: Matt <Rocketknight1@users.noreply.github.com> * Qwen2.5-Omni test fixes (#40307) Updated expectations, and mp tests * Add back `_tp_plan` attribute (#39944) * Update modeling_utils.py * make sure we update with the module's plan * use public api * oups * update * fix failing test * Update src/transformers/integrations/tensor_parallel.py * Update src/transformers/integrations/tensor_parallel.py * fix * make the API more friendly! * fix tests * fix styling --------- Co-authored-by: Arthur Zucker <arthur.zucker@gmail.com> Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * byebye torch 2.1 (#40317) * Bump minimum torch version to 2.2 * Remove is_torch_greater_or_equal_than_2_2 * update versions table * Deprecate is_torch_sdpa_available (except for backward compat), remove require_torch_sdpa * No more `natten` (#40287) get rid off natten Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * [`GPT OSS`] Refactor the tests as it was not properly checking the outputs (#40288) * it was long due! * use the official kernel * more permissive * update the kernel as well * mmm should it be this? * up pu * fixup * Update test_modeling_gpt_oss.py * style * start with 20b * Update CI with nightly torch workflow file (#40306) * fix nightly ci * Apply suggestions from code review Co-authored-by: ivarflakstad <69173633+ivarflakstad@users.noreply.github.com> --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> Co-authored-by: ivarflakstad <69173633+ivarflakstad@users.noreply.github.com> * Fix: Apply `get_placeholder_mask` in Ovis2 (#40280) * Refactor special image mask * Refactor get_placeholder_mask method * Revert "Refactor special image mask" This reverts commit 9eb1828ae930329656d6f323a510c5e6033e1f85. * Fix * Revert "Refactor get_placeholder_mask method" This reverts commit 07aad6484bb08d6351d5b605e9db574d28edcd15. * Update notification service amd_daily_ci_workflows definition (#40314) * One cache class to rule them all (#40276) * remove all classes * fix generate * start replacing everywhere * finish removing everywhere * typo * typo * fix * typo * remove num_layers=1 * CI * fix all docstrings * review * style * Fix chunked attention mask with left-padding (#40324) * add fix * add test * raise proper warning for older versions * fix * fix and add 2nd test * fix for flex and torch 2.5 * [docs] remove flax references from `/en/model_doc` (#40311) * 1st commit * all models up to D * all models up to G * all models up to M * all remaining models * Fix qwen-omni processor text only mode (#40336) * Fix qwen-omni processor text only mode * remove try except --------- Co-authored-by: yuekaiz <yuekaiz@mgmt1-login.cm.cluster> * Change Qwen2RMSNorm to RMSNorm from PyTorch (#40066) * Unify Qwen2RMSNorm definitions and use RMSNorm from PyTorch Signed-off-by: cyy <cyyever@outlook.com> * subclass RMSNorm Signed-off-by: cyy <cyyever@outlook.com> --------- Signed-off-by: cyy <cyyever@outlook.com> * Add DeepseekV3ForSequenceClassification for Deepseek V3 models (#40200) * Add Sequence Classification Support for Deepseek v3 model DeepseekV3ForSequenceClassification * After run make fixup * Fix deprecation warning version (#40343) fix * Add missing arguments to class constructors (#40068) * Add missing arguments Signed-off-by: cyy <cyyever@outlook.com> * Fix typos Signed-off-by: cyy <cyyever@outlook.com> * More fixes Signed-off-by: cyy <cyyever@outlook.com> --------- Signed-off-by: cyy <cyyever@outlook.com> * [docs] remove TF references from `/en/model_doc` (#40344) * models up to F * models up to M * all models * Fix: Only call Trainer.align_special_tokens if model has "config" attribute (#40322) * Only call Trainer.align_special_tokens if model has "config" attribute * Add efficient test for training a model without model.config * Reformat * add type hints (#40319) * add basic type hints to import module * run make fixup * remove optional * fixes --------- Co-authored-by: Matt <Rocketknight1@users.noreply.github.com> * Fix an infinite loop bug in recursive search of relative imports (#40326) Fix bug in recursive search of relative imports * Fix links in Glm4vMoe configuration classes to point to the correct H… (#40310) * Fix links in Glm4vMoe configuration classes to point to the correct Hugging Face model repository * run fixup to update links in Glm4vMoe configuration classes to point to the correct Hugging Face model repository * T5 test and target device fixes (#40313) * Fix cache setup related issues * Fix target-device-related issues * Ruff * Address review comments * Update `test_spm_converter_bytefallback_warning` (#40284) fff Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * (small) fix conditional for input_ids and input_embeds in marian (#40045) * (small) fix conditional for input_ids and input_embeds in marian * address comment * Fix attention vizualizer (#40285) * make visualizer rely on create causal mask * format * fixup * fixup * read token * read token, duh * what is up with that token * small tests? * adjust * try with flush * normalize for ANSI * buffer shenanigans * [ModernBert] Prevent the attention mask from being None in ModernBertForSequenceClassification (#35991) * [ModernBert] Prevent the attention mask from being None in ModernBertForSequenceClassification * fix the modular conversion * Clean up XCodec and other codecs (#40348) * Clean up xcodec addition. * Clean up config. * Switch to fixtures test. * Small stuff. * Polish XCodec and standardize across codecs. * Update src/transformers/models/xcodec/modeling_xcodec.py Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> * Format and fix test. * Update tol. --------- Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> * [serve] add cors warnings (#40112) * add cors warnings * Update src/transformers/commands/serving.py Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com> * Update src/transformers/commands/serving.py Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * Apply suggestions from code review * make fixup --------- Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com> Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * [detection] use consistent dtype for Conditional and DAB DETR positional embeddings (#40300) fix: use consistent dtype for sine positional embeddings * Remove more PyTorch 2.2 compatible code (#40337) Signed-off-by: cyy <cyyever@outlook.com> * [`FA`] Fix some model tests (#40350) * fix * cleanup, revert aimv2 fa changes * fix aria * i searched a long time but the cross dependency is for the recent models so... * this was something... evolla * fix modernbert decoder + make fa test more robust * nit * Qwen2.5-VL test fixes for ROCm (#40308) * [generate] handle support for cache classes when num enc layers != num dec layers (#40277) * handle support for cache classes when num enc layers != num dec layers * handle overwrites * one more corner case * Update src/transformers/generation/utils.py * Update src/transformers/generation/utils.py * Apply suggestions from code review * handle corner case :o * [4/N]more docs to device agnostic (#40355) * more docs to device agnostic Signed-off-by: YAO Matrix <matrix.yao@intel.com> * more Signed-off-by: YAO Matrix <matrix.yao@intel.com> * 1 Signed-off-by: YAO Matrix <matrix.yao@intel.com> * 2 Signed-off-by: YAO Matrix <matrix.yao@intel.com> * Update vitpose.md * Update camembert.md * Update camembert.md --------- Signed-off-by: YAO Matrix <matrix.yao@intel.com> * DOCS: Clarification on the use of `label_names` as an argument to TrainingArguments (#40353) * Update trainer.md * Update trainer.md Removed the detail about label_names argument usage from the tip/ warning section * Update training_args.py Added the label_names usage clarification in the docstring * Update trainer.md --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * HunYuan opensource (#39606) * merge opensource_hunyuan * add head_dim * fix assertion error * fix seen_tokens * ready_for_upstream (merge request !17) Squash merge branch 'ready_for_upstream' into 'main' * fix configuration type&docstring * fix style * ready_for_upstream (merge request !18) Squash merge branch 'ready_for_upstream' into 'main' * add doc * fix testcode * fix configuration type&docstring * rename base model * remove assert * update * remove tiktoken * update * fix moe and code style (#3) * update * fix format * update * revert makefile * fix moe config * fix numel() * remove prepare_inputs_for_generation * fix kv_seq_len * add docs/toctree * remove unused paramter&add licence * add licence * remove unused paramter * fix code * dense modular update import fix fix use mistralmodel fix qknorm add sliding_window make style fix dense done hunyuan moe fix import fix modular fixup fixup * update model path * fix mlp_bias * fix modular * Fix modeling (#5) * fix attention * use llamamodel * fix code * Fix qk (#6) * fix qk_norm * fix * fix modual * Fix moe (#7) * fix some moe code * fix einsum * try top1 * use top1 * Fix rotary (#8) * fix rotary * fix modeling * fix modular * fix testcode * remove A13B unit test * Fix moe v1 (#9) fix moe & gate * Fix gate norm (#10) * add norm_topk_prob * Fix testcase (#11) * fix&skip test * Fix testcase (#12) * skip testcase * Fix norm topk (#13) * hardcode norm_topk_prob * fix testcase --------- Co-authored-by: pridejcyang <pridejcyang@tencent.com> Co-authored-by: Mingji Han <mingjihan@tencent.com> * Fix idefics3 vision embeddings indices dtype (#40360) fix idefics3 vision embeddings Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn> * wav2vec2 fixes (#40341) * Changed datasets to avoid a datasets error * Changed back split to test * Change multimodal data links to HF hub (#40309) change multimodal data links to HF hub * [pipelines] add support to `skip_special_tokens` in the main text generation pipelines (#40356) * add support to skip_special_tokens in pipelines * add test * rm redundant * ⚠️⚠️ Use `dtype` instead of `torch_dtype` everywhere! (#39782) * update everywhere * style * pipelines * switch it everywhere in tests * switch it everywhere in docs * switch in converters everywhere * update in examples * update in model docstrings * style * warnings * style * Update configuration_utils.py * fix * Update configuration_utils.py * fixes and add first test * add pipeline tests * Update test_pipelines_common.py * add config test * Update test_modeling_common.py * add new ones * post rebase * add new * post rebase adds * [processor] move commonalities to mixin (#40339) * move commonalities to mixin * revert - unrelated * fix copies * fix style * comments * [configuration] allow to overwrite kwargs from subconfigs (#40241) allow to overwrite kwargs from subconfigs * fix(example): align parameter names with the latest function definition for gdino (#40369) * Addiing ByteDance Seed Seed-OSS (#40272) add seed oss * Add GptOssForTokenClassification for GPT-OSS models (#40190) * Add GptOssForTokenClassification for GPT-OSS models * After run make fixup * Bug Fix: Dynamically set return_lse flag in FlexAttention (#40352) * bug fix - return_lse dynamically set * addressed compatibility with return type - flex_attention_forward * rename variables * revert changes to commits * Chat Template Doc Fixes (#40173) * draft commit * draft commit * Fixup chat_extras too * Update conversations.md * Update the toctree and titles * Update the writing guide! …

…gingface#39120) * just update 2 files * update other models as well just making fix-copies * also add the changes needed to modeling utils * put this on the pretrained model instead * nits and fixes * update generic, fix to use config value * update other modelings * use transformers kwargs instead * update * update * update other models * update * updates * update * update * update * fix * finally * very small nits * this fixes more tests * fix other models as well! * update modularqwen2 * update models based on qwen2 * update * update * remove the **flash stuff in favor of noraml kwargs * update * propagate gemma? * remove output attentions * propagate * support cross attention edge case * same * test this * fixes * more fix * update * update * fix conflicts * update * fix emu3 * fix emu3 * move the fix a bit * quel enfer * some fixes, loss_kwargs should never had been * finish fixing gemma3n * fix small lm3 * fix another one * fix csm now * fux csm and mistral * fix mistral now * small fixes * fix janusss * only for some models * fixup * phix phi3 * more fixes? * dose this fix it? * update * holy shit it was just graph breaks * protect torch * updates * fix samhq? * fix moonshine * more moonshine fixes, 3 failures left! * nits * generic needs to support more * more fixes to moonshine! * fix cross attention outputs! * fix csm! * nits * fix stupid kosmos2 * current updates * fixes * use output recorder? * nicer! * a little bit of magic * update * fix protect * fix * small fixes * protect import * fix a bunch of more models * fix fixups * fix some of the last ones * nit * partly fix phi * update * fix import path * make something that is fullgraph compatible just to be sure * typing was wrong on llama so the rest was wrong as well * fucking ugly but at least it is still exportable * syle * supposed to fix moonshine, it still breaks * fix some default * fix the last bits of sam * update samhq * more fixes to am hq * nit * fix all output+hidden states and output_attentions! * fix? * fix diffllama * updates to fix initialization on the sam pips * ups there was a bug * fix the last sam hq test * fix gotocr * fix gotocr2! * fixes * skip stupid tests * there was one left :) * fixup * fix fix copies issues with this test file * fix copies for sam_hq * rm some comments * skip 2 more failing tests * fix * fix everything * Apply suggestions from code review Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com> * add more doc! * fix public init * fix modular qwen3 --------- Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com>

fix

…gingface#39120) * just update 2 files * update other models as well just making fix-copies * also add the changes needed to modeling utils * put this on the pretrained model instead * nits and fixes * update generic, fix to use config value * update other modelings * use transformers kwargs instead * update * update * update other models * update * updates * update * update * update * fix * finally * very small nits * this fixes more tests * fix other models as well! * update modularqwen2 * update models based on qwen2 * update * update * remove the **flash stuff in favor of noraml kwargs * update * propagate gemma? * remove output attentions * propagate * support cross attention edge case * same * test this * fixes * more fix * update * update * fix conflicts * update * fix emu3 * fix emu3 * move the fix a bit * quel enfer * some fixes, loss_kwargs should never had been * finish fixing gemma3n * fix small lm3 * fix another one * fix csm now * fux csm and mistral * fix mistral now * small fixes * fix janusss * only for some models * fixup * phix phi3 * more fixes? * dose this fix it? * update * holy shit it was just graph breaks * protect torch * updates * fix samhq? * fix moonshine * more moonshine fixes, 3 failures left! * nits * generic needs to support more * more fixes to moonshine! * fix cross attention outputs! * fix csm! * nits * fix stupid kosmos2 * current updates * fixes * use output recorder? * nicer! * a little bit of magic * update * fix protect * fix * small fixes * protect import * fix a bunch of more models * fix fixups * fix some of the last ones * nit * partly fix phi * update * fix import path * make something that is fullgraph compatible just to be sure * typing was wrong on llama so the rest was wrong as well * fucking ugly but at least it is still exportable * syle * supposed to fix moonshine, it still breaks * fix some default * fix the last bits of sam * update samhq * more fixes to am hq * nit * fix all output+hidden states and output_attentions! * fix? * fix diffllama * updates to fix initialization on the sam pips * ups there was a bug * fix the last sam hq test * fix gotocr * fix gotocr2! * fixes * skip stupid tests * there was one left :) * fixup * fix fix copies issues with this test file * fix copies for sam_hq * rm some comments * skip 2 more failing tests * fix * fix everything * Apply suggestions from code review Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com> * add more doc! * fix public init * fix modular qwen3 --------- Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com>

fix

…gingface#39120) * just update 2 files * update other models as well just making fix-copies * also add the changes needed to modeling utils * put this on the pretrained model instead * nits and fixes * update generic, fix to use config value * update other modelings * use transformers kwargs instead * update * update * update other models * update * updates * update * update * update * fix * finally * very small nits * this fixes more tests * fix other models as well! * update modularqwen2 * update models based on qwen2 * update * update * remove the **flash stuff in favor of noraml kwargs * update * propagate gemma? * remove output attentions * propagate * support cross attention edge case * same * test this * fixes * more fix * update * update * fix conflicts * update * fix emu3 * fix emu3 * move the fix a bit * quel enfer * some fixes, loss_kwargs should never had been * finish fixing gemma3n * fix small lm3 * fix another one * fix csm now * fux csm and mistral * fix mistral now * small fixes * fix janusss * only for some models * fixup * phix phi3 * more fixes? * dose this fix it? * update * holy shit it was just graph breaks * protect torch * updates * fix samhq? * fix moonshine * more moonshine fixes, 3 failures left! * nits * generic needs to support more * more fixes to moonshine! * fix cross attention outputs! * fix csm! * nits * fix stupid kosmos2 * current updates * fixes * use output recorder? * nicer! * a little bit of magic * update * fix protect * fix * small fixes * protect import * fix a bunch of more models * fix fixups * fix some of the last ones * nit * partly fix phi * update * fix import path * make something that is fullgraph compatible just to be sure * typing was wrong on llama so the rest was wrong as well * fucking ugly but at least it is still exportable * syle * supposed to fix moonshine, it still breaks * fix some default * fix the last bits of sam * update samhq * more fixes to am hq * nit * fix all output+hidden states and output_attentions! * fix? * fix diffllama * updates to fix initialization on the sam pips * ups there was a bug * fix the last sam hq test * fix gotocr * fix gotocr2! * fixes * skip stupid tests * there was one left :) * fixup * fix fix copies issues with this test file * fix copies for sam_hq * rm some comments * skip 2 more failing tests * fix * fix everything * Apply suggestions from code review Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com> * add more doc! * fix public init * fix modular qwen3 --------- Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com>

fix

…gingface#39120) * just update 2 files * update other models as well just making fix-copies * also add the changes needed to modeling utils * put this on the pretrained model instead * nits and fixes * update generic, fix to use config value * update other modelings * use transformers kwargs instead * update * update * update other models * update * updates * update * update * update * fix * finally * very small nits * this fixes more tests * fix other models as well! * update modularqwen2 * update models based on qwen2 * update * update * remove the **flash stuff in favor of noraml kwargs * update * propagate gemma? * remove output attentions * propagate * support cross attention edge case * same * test this * fixes * more fix * update * update * fix conflicts * update * fix emu3 * fix emu3 * move the fix a bit * quel enfer * some fixes, loss_kwargs should never had been * finish fixing gemma3n * fix small lm3 * fix another one * fix csm now * fux csm and mistral * fix mistral now * small fixes * fix janusss * only for some models * fixup * phix phi3 * more fixes? * dose this fix it? * update * holy shit it was just graph breaks * protect torch * updates * fix samhq? * fix moonshine * more moonshine fixes, 3 failures left! * nits * generic needs to support more * more fixes to moonshine! * fix cross attention outputs! * fix csm! * nits * fix stupid kosmos2 * current updates * fixes * use output recorder? * nicer! * a little bit of magic * update * fix protect * fix * small fixes * protect import * fix a bunch of more models * fix fixups * fix some of the last ones * nit * partly fix phi * update * fix import path * make something that is fullgraph compatible just to be sure * typing was wrong on llama so the rest was wrong as well * fucking ugly but at least it is still exportable * syle * supposed to fix moonshine, it still breaks * fix some default * fix the last bits of sam * update samhq * more fixes to am hq * nit * fix all output+hidden states and output_attentions! * fix? * fix diffllama * updates to fix initialization on the sam pips * ups there was a bug * fix the last sam hq test * fix gotocr * fix gotocr2! * fixes * skip stupid tests * there was one left :) * fixup * fix fix copies issues with this test file * fix copies for sam_hq * rm some comments * skip 2 more failing tests * fix * fix everything * Apply suggestions from code review Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com> * add more doc! * fix public init * fix modular qwen3 --------- Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com>

fix

…gingface#39120) * just update 2 files * update other models as well just making fix-copies * also add the changes needed to modeling utils * put this on the pretrained model instead * nits and fixes * update generic, fix to use config value * update other modelings * use transformers kwargs instead * update * update * update other models * update * updates * update * update * update * fix * finally * very small nits * this fixes more tests * fix other models as well! * update modularqwen2 * update models based on qwen2 * update * update * remove the **flash stuff in favor of noraml kwargs * update * propagate gemma? * remove output attentions * propagate * support cross attention edge case * same * test this * fixes * more fix * update * update * fix conflicts * update * fix emu3 * fix emu3 * move the fix a bit * quel enfer * some fixes, loss_kwargs should never had been * finish fixing gemma3n * fix small lm3 * fix another one * fix csm now * fux csm and mistral * fix mistral now * small fixes * fix janusss * only for some models * fixup * phix phi3 * more fixes? * dose this fix it? * update * holy shit it was just graph breaks * protect torch * updates * fix samhq? * fix moonshine * more moonshine fixes, 3 failures left! * nits * generic needs to support more * more fixes to moonshine! * fix cross attention outputs! * fix csm! * nits * fix stupid kosmos2 * current updates * fixes * use output recorder? * nicer! * a little bit of magic * update * fix protect * fix * small fixes * protect import * fix a bunch of more models * fix fixups * fix some of the last ones * nit * partly fix phi * update * fix import path * make something that is fullgraph compatible just to be sure * typing was wrong on llama so the rest was wrong as well * fucking ugly but at least it is still exportable * syle * supposed to fix moonshine, it still breaks * fix some default * fix the last bits of sam * update samhq * more fixes to am hq * nit * fix all output+hidden states and output_attentions! * fix? * fix diffllama * updates to fix initialization on the sam pips * ups there was a bug * fix the last sam hq test * fix gotocr * fix gotocr2! * fixes * skip stupid tests * there was one left :) * fixup * fix fix copies issues with this test file * fix copies for sam_hq * rm some comments * skip 2 more failing tests * fix * fix everything * Apply suggestions from code review Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com> * add more doc! * fix public init * fix modular qwen3 --------- Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com>

fix

…gingface#39120) * just update 2 files * update other models as well just making fix-copies * also add the changes needed to modeling utils * put this on the pretrained model instead * nits and fixes * update generic, fix to use config value * update other modelings * use transformers kwargs instead * update * update * update other models * update * updates * update * update * update * fix * finally * very small nits * this fixes more tests * fix other models as well! * update modularqwen2 * update models based on qwen2 * update * update * remove the **flash stuff in favor of noraml kwargs * update * propagate gemma? * remove output attentions * propagate * support cross attention edge case * same * test this * fixes * more fix * update * update * fix conflicts * update * fix emu3 * fix emu3 * move the fix a bit * quel enfer * some fixes, loss_kwargs should never had been * finish fixing gemma3n * fix small lm3 * fix another one * fix csm now * fux csm and mistral * fix mistral now * small fixes * fix janusss * only for some models * fixup * phix phi3 * more fixes? * dose this fix it? * update * holy shit it was just graph breaks * protect torch * updates * fix samhq? * fix moonshine * more moonshine fixes, 3 failures left! * nits * generic needs to support more * more fixes to moonshine! * fix cross attention outputs! * fix csm! * nits * fix stupid kosmos2 * current updates * fixes * use output recorder? * nicer! * a little bit of magic * update * fix protect * fix * small fixes * protect import * fix a bunch of more models * fix fixups * fix some of the last ones * nit * partly fix phi * update * fix import path * make something that is fullgraph compatible just to be sure * typing was wrong on llama so the rest was wrong as well * fucking ugly but at least it is still exportable * syle * supposed to fix moonshine, it still breaks * fix some default * fix the last bits of sam * update samhq * more fixes to am hq * nit * fix all output+hidden states and output_attentions! * fix? * fix diffllama * updates to fix initialization on the sam pips * ups there was a bug * fix the last sam hq test * fix gotocr * fix gotocr2! * fixes * skip stupid tests * there was one left :) * fixup * fix fix copies issues with this test file * fix copies for sam_hq * rm some comments * skip 2 more failing tests * fix * fix everything * Apply suggestions from code review Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com> * add more doc! * fix public init * fix modular qwen3 --------- Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com>

fix

…gingface#39120) * just update 2 files * update other models as well just making fix-copies * also add the changes needed to modeling utils * put this on the pretrained model instead * nits and fixes * update generic, fix to use config value * update other modelings * use transformers kwargs instead * update * update * update other models * update * updates * update * update * update * fix * finally * very small nits * this fixes more tests * fix other models as well! * update modularqwen2 * update models based on qwen2 * update * update * remove the **flash stuff in favor of noraml kwargs * update * propagate gemma? * remove output attentions * propagate * support cross attention edge case * same * test this * fixes * more fix * update * update * fix conflicts * update * fix emu3 * fix emu3 * move the fix a bit * quel enfer * some fixes, loss_kwargs should never had been * finish fixing gemma3n * fix small lm3 * fix another one * fix csm now * fux csm and mistral * fix mistral now * small fixes * fix janusss * only for some models * fixup * phix phi3 * more fixes? * dose this fix it? * update * holy shit it was just graph breaks * protect torch * updates * fix samhq? * fix moonshine * more moonshine fixes, 3 failures left! * nits * generic needs to support more * more fixes to moonshine! * fix cross attention outputs! * fix csm! * nits * fix stupid kosmos2 * current updates * fixes * use output recorder? * nicer! * a little bit of magic * update * fix protect * fix * small fixes * protect import * fix a bunch of more models * fix fixups * fix some of the last ones * nit * partly fix phi * update * fix import path * make something that is fullgraph compatible just to be sure * typing was wrong on llama so the rest was wrong as well * fucking ugly but at least it is still exportable * syle * supposed to fix moonshine, it still breaks * fix some default * fix the last bits of sam * update samhq * more fixes to am hq * nit * fix all output+hidden states and output_attentions! * fix? * fix diffllama * updates to fix initialization on the sam pips * ups there was a bug * fix the last sam hq test * fix gotocr * fix gotocr2! * fixes * skip stupid tests * there was one left :) * fixup * fix fix copies issues with this test file * fix copies for sam_hq * rm some comments * skip 2 more failing tests * fix * fix everything * Apply suggestions from code review Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com> * add more doc! * fix public init * fix modular qwen3 --------- Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com>

fix

* initial comment * test * initial conversion for outline * intermediate commit for configuration * chore:init files for sam2 * adding arbitary undefined config * check * add vision * make style * init sam2 base model * Fix imports * Linting * chore:sam to sam2 classes * Linting * Add sam2 to models.__init__ * chore:match prompt encoder with sam2 code * chore:prepare kwargs for mask decoder * Add image/video predictors * Add CUDA kernel * Add output classes * linting * Add logging info * tmp commit * docs for sam2 * enable image processing * check difference of original SAM2 - difference is the order of ToTensor() - please see https://pytorch.org/vision/main/_modules/torchvision/transforms/functional.html#resize * enable promptencoder of sam2 * fix promprencoder * Confirmed that PromptEncoder is exactly same (Be aware of bfloat16 and float32 difference) * Confirmed that ImageEncoder is exactly same (Be aware the linting of init) * Confirmed that MaskDecoder is exactly same (TO DO: lint variable name) * SamModel is now available (Need more chore for name) * make fix-copies * make style * make CI happy * Refactor VisionEncoder and PostioinEmbedding * TO DO : fix the image_embeddings and sparse_embeddings part * pure image inference done * reusable features fix and make style * styling * refactor memoryattention * tmp * tmp * refactor memoryencoder TO DO : convert and inference the video pipeline * TO DO : fix the image_encoder shape * conversion finish TO DO: need to check video inference * make style * remove video model * lint * change * python utils/check_docstringspy --check_all * python utils/check_config_attributes.py * remove copies for sam2promptencoder due to configuration * change __init__.py * remove tensorflow version * fix that to not use direct comparison * make style * add missing import * fix image_embedding_size * refactor Sam2 Attention * add fully working video inference (refactoring todo) * clarify _prepare_memory_conditioned_features * simplify modeling code, remove unused paths * use one model * use auto_docstring * refactor rope embeddings * nit * not using multimask when several points given * add all sam2.1 * add video tmp * add Sam2VideoSessionState + fast image proc + video proc * remove init_states from model * fix batch inference * add image integration tests * uniformize modeling code with other sam models and use modular * pass vision tests an most model tests * All tests passing * add offloading inference state and video to cpu * fix inference from image embedding and existing mask * fix multi_boxes mask inference * Fix batch images + batch boxes inference * improve processing for image inference * add support for mask generation pipeline * add support for get_connected_components post processing in mask generation * add fast image processor sam, image processor tests and use modular for sam2 image processor * fix mistake in sam after #39120 * fix init weights * refactor convert * add integration tests for video + other improvements * add needed missing docstrings * Improve docstrings and * improve inference speed by avoiding cuda sync * add test * skip test for vision_model * minor fix for vision_model * fix vision_model by adding sam2model and change the torch dependencies * remove patch_size * remove image_embedding_size * fix patch_size * fix test * make style * Separate hieradet and vision encoder in sam2 * fixup * review changes part 1 * remove MemoryEncoderConfig and MemoryAttentionConfig * pass q_stride instead of q_pool module * add inference on streamed videos * explicitely process streamed frames * nit * Improve docstrings in Sam2Model * update sam2 modeling with better gestion of inference state and cache, and separate Sam2Model and Sam2VideoModel * improve video inference api * change inference_state to inference_session * use modular for Sam2Model * fix convert sam2 hf * modular * Update src/transformers/models/sam2/video_processing_sam2.py Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * fix minor config * fix attention loading error * update modeling tests to use hub checkpoints * Use CI A10 runner for integration tests values + higher tolerance for video integration tests * PR review part 1 * fix doc * nit improvements * enforce one input format for points, labels and boxes * nit * last few nits from PR review * fix style * fix the input type * fix docs * add sam2 model as conversion script * improve sam2 doc * add rough necessarry changes * first working edgetam * fix issue with object pointers * Use modular as much as possible * nit fixes + optimization * refactor spatial perceiver * cleanup after merge * add working edgetam * improve perceiver resampler code * simplify/unify rope attention logic * Improve comments in apply_rotary_pos_emb_2d * add working tests * fix test timmwrapper * add docs * make fixup * nits * fix modular * fix modular * PR review part 1 * split apply_rotary_pos_emb_2d * add granularity to _prepare_memory_conditioned_features * add dates to doc * add separate mlp for memory attention * Fix memory on wrong device * store processed frames in dict * update checkpoints in tests * update dates --------- Co-authored-by: sangbumchoi <danielsejong55@gmail.com> Co-authored-by: RUFFY-369 <prakarshkaushik369@gmail.com> Co-authored-by: Sangbum Daniel Choi <34004152+SangbumChoi@users.noreply.github.com> Co-authored-by: Haitham Khedr <haithamkhedr@meta.com> Co-authored-by: sangbum choi <sangbumchoi@sangbumui-MacBookAir.local> Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>

* initial comment * test * initial conversion for outline * intermediate commit for configuration * chore:init files for sam2 * adding arbitary undefined config * check * add vision * make style * init sam2 base model * Fix imports * Linting * chore:sam to sam2 classes * Linting * Add sam2 to models.__init__ * chore:match prompt encoder with sam2 code * chore:prepare kwargs for mask decoder * Add image/video predictors * Add CUDA kernel * Add output classes * linting * Add logging info * tmp commit * docs for sam2 * enable image processing * check difference of original SAM2 - difference is the order of ToTensor() - please see https://pytorch.org/vision/main/_modules/torchvision/transforms/functional.html#resize * enable promptencoder of sam2 * fix promprencoder * Confirmed that PromptEncoder is exactly same (Be aware of bfloat16 and float32 difference) * Confirmed that ImageEncoder is exactly same (Be aware the linting of init) * Confirmed that MaskDecoder is exactly same (TO DO: lint variable name) * SamModel is now available (Need more chore for name) * make fix-copies * make style * make CI happy * Refactor VisionEncoder and PostioinEmbedding * TO DO : fix the image_embeddings and sparse_embeddings part * pure image inference done * reusable features fix and make style * styling * refactor memoryattention * tmp * tmp * refactor memoryencoder TO DO : convert and inference the video pipeline * TO DO : fix the image_encoder shape * conversion finish TO DO: need to check video inference * make style * remove video model * lint * change * python utils/check_docstringspy --check_all * python utils/check_config_attributes.py * remove copies for sam2promptencoder due to configuration * change __init__.py * remove tensorflow version * fix that to not use direct comparison * make style * add missing import * fix image_embedding_size * refactor Sam2 Attention * add fully working video inference (refactoring todo) * clarify _prepare_memory_conditioned_features * simplify modeling code, remove unused paths * use one model * use auto_docstring * refactor rope embeddings * nit * not using multimask when several points given * add all sam2.1 * add video tmp * add Sam2VideoSessionState + fast image proc + video proc * remove init_states from model * fix batch inference * add image integration tests * uniformize modeling code with other sam models and use modular * pass vision tests an most model tests * All tests passing * add offloading inference state and video to cpu * fix inference from image embedding and existing mask * fix multi_boxes mask inference * Fix batch images + batch boxes inference * improve processing for image inference * add support for mask generation pipeline * add support for get_connected_components post processing in mask generation * add fast image processor sam, image processor tests and use modular for sam2 image processor * fix mistake in sam after huggingface#39120 * fix init weights * refactor convert * add integration tests for video + other improvements * add needed missing docstrings * Improve docstrings and * improve inference speed by avoiding cuda sync * add test * skip test for vision_model * minor fix for vision_model * fix vision_model by adding sam2model and change the torch dependencies * remove patch_size * remove image_embedding_size * fix patch_size * fix test * make style * Separate hieradet and vision encoder in sam2 * fixup * review changes part 1 * remove MemoryEncoderConfig and MemoryAttentionConfig * pass q_stride instead of q_pool module * add inference on streamed videos * explicitely process streamed frames * nit * Improve docstrings in Sam2Model * update sam2 modeling with better gestion of inference state and cache, and separate Sam2Model and Sam2VideoModel * improve video inference api * change inference_state to inference_session * use modular for Sam2Model * fix convert sam2 hf * modular * Update src/transformers/models/sam2/video_processing_sam2.py Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * fix minor config * fix attention loading error * update modeling tests to use hub checkpoints * Use CI A10 runner for integration tests values + higher tolerance for video integration tests * PR review part 1 * fix doc * nit improvements * enforce one input format for points, labels and boxes * nit * last few nits from PR review * fix style * fix the input type * fix docs * add sam2 model as conversion script * improve sam2 doc * add rough necessarry changes * first working edgetam * fix issue with object pointers * Use modular as much as possible * nit fixes + optimization * refactor spatial perceiver * cleanup after merge * add working edgetam * improve perceiver resampler code * simplify/unify rope attention logic * Improve comments in apply_rotary_pos_emb_2d * add working tests * fix test timmwrapper * add docs * make fixup * nits * fix modular * fix modular * PR review part 1 * split apply_rotary_pos_emb_2d * add granularity to _prepare_memory_conditioned_features * add dates to doc * add separate mlp for memory attention * Fix memory on wrong device * store processed frames in dict * update checkpoints in tests * update dates --------- Co-authored-by: sangbumchoi <danielsejong55@gmail.com> Co-authored-by: RUFFY-369 <prakarshkaushik369@gmail.com> Co-authored-by: Sangbum Daniel Choi <34004152+SangbumChoi@users.noreply.github.com> Co-authored-by: Haitham Khedr <haithamkhedr@meta.com> Co-authored-by: sangbum choi <sangbumchoi@sangbumui-MacBookAir.local> Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>

* initial comment * test * initial conversion for outline * intermediate commit for configuration * chore:init files for sam2 * adding arbitary undefined config * check * add vision * make style * init sam2 base model * Fix imports * Linting * chore:sam to sam2 classes * Linting * Add sam2 to models.__init__ * chore:match prompt encoder with sam2 code * chore:prepare kwargs for mask decoder * Add image/video predictors * Add CUDA kernel * Add output classes * linting * Add logging info * tmp commit * docs for sam2 * enable image processing * check difference of original SAM2 - difference is the order of ToTensor() - please see https://pytorch.org/vision/main/_modules/torchvision/transforms/functional.html#resize * enable promptencoder of sam2 * fix promprencoder * Confirmed that PromptEncoder is exactly same (Be aware of bfloat16 and float32 difference) * Confirmed that ImageEncoder is exactly same (Be aware the linting of init) * Confirmed that MaskDecoder is exactly same (TO DO: lint variable name) * SamModel is now available (Need more chore for name) * make fix-copies * make style * make CI happy * Refactor VisionEncoder and PostioinEmbedding * TO DO : fix the image_embeddings and sparse_embeddings part * pure image inference done * reusable features fix and make style * styling * refactor memoryattention * tmp * tmp * refactor memoryencoder TO DO : convert and inference the video pipeline * TO DO : fix the image_encoder shape * conversion finish TO DO: need to check video inference * make style * remove video model * lint * change * python utils/check_docstringspy --check_all * python utils/check_config_attributes.py * remove copies for sam2promptencoder due to configuration * change __init__.py * remove tensorflow version * fix that to not use direct comparison * make style * add missing import * fix image_embedding_size * refactor Sam2 Attention * add fully working video inference (refactoring todo) * clarify _prepare_memory_conditioned_features * simplify modeling code, remove unused paths * use one model * use auto_docstring * refactor rope embeddings * nit * not using multimask when several points given * add all sam2.1 * add video tmp * add Sam2VideoSessionState + fast image proc + video proc * remove init_states from model * fix batch inference * add image integration tests * uniformize modeling code with other sam models and use modular * pass vision tests an most model tests * All tests passing * add offloading inference state and video to cpu * fix inference from image embedding and existing mask * fix multi_boxes mask inference * Fix batch images + batch boxes inference * improve processing for image inference * add support for mask generation pipeline * add support for get_connected_components post processing in mask generation * add fast image processor sam, image processor tests and use modular for sam2 image processor * fix mistake in sam after #39120 * fix init weights * refactor convert * add integration tests for video + other improvements * add needed missing docstrings * Improve docstrings and * improve inference speed by avoiding cuda sync * add test * skip test for vision_model * minor fix for vision_model * fix vision_model by adding sam2model and change the torch dependencies * remove patch_size * remove image_embedding_size * fix patch_size * fix test * make style * Separate hieradet and vision encoder in sam2 * fixup * review changes part 1 * remove MemoryEncoderConfig and MemoryAttentionConfig * pass q_stride instead of q_pool module * add inference on streamed videos * explicitely process streamed frames * nit * Improve docstrings in Sam2Model * update sam2 modeling with better gestion of inference state and cache, and separate Sam2Model and Sam2VideoModel * improve video inference api * change inference_state to inference_session * use modular for Sam2Model * fix convert sam2 hf * modular * Update src/transformers/models/sam2/video_processing_sam2.py Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * fix minor config * fix attention loading error * update modeling tests to use hub checkpoints * Use CI A10 runner for integration tests values + higher tolerance for video integration tests * PR review part 1 * fix doc * nit improvements * enforce one input format for points, labels and boxes * nit * last few nits from PR review * fix style * fix the input type * fix docs * add sam2 model as conversion script * improve sam2 doc * add rough necessarry changes * first working edgetam * fix issue with object pointers * Use modular as much as possible * nit fixes + optimization * refactor spatial perceiver * cleanup after merge * add working edgetam * improve perceiver resampler code * simplify/unify rope attention logic * Improve comments in apply_rotary_pos_emb_2d * add working tests * fix test timmwrapper * add docs * make fixup * nits * fix modular * fix modular * PR review part 1 * split apply_rotary_pos_emb_2d * add granularity to _prepare_memory_conditioned_features * add dates to doc * add separate mlp for memory attention * Fix memory on wrong device * store processed frames in dict * update checkpoints in tests * update dates --------- Co-authored-by: sangbumchoi <danielsejong55@gmail.com> Co-authored-by: RUFFY-369 <prakarshkaushik369@gmail.com> Co-authored-by: Sangbum Daniel Choi <34004152+SangbumChoi@users.noreply.github.com> Co-authored-by: Haitham Khedr <haithamkhedr@meta.com> Co-authored-by: sangbum choi <sangbumchoi@sangbumui-MacBookAir.local> Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>

* Update expected values for one more `test_speculative_generation` after #40949 (#40967) fix Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * FIX(trainer): ensure final checkpoint is saved when resuming training (#40347) * fix(trainer): ensure final checkpoint is saved when resuming training * add test * make style && slight fix of test * make style again * move test code to test_trainer * remove outdated test file * Apply style fixes --------- Co-authored-by: rangehow <rangehow@foxmail.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> * Add new model LFM2-VL (#40624) * Add LFM2-VL support * add tests * linting, formatting, misc review changes * add siglip2 to auto config and instantiate it in lfm2-vl configuration * decouple image processor from processor * remove torch import from configuration * replace | with Optional * remove layer truncation from modeling file * fix copies * update everything * fix test case to use tiny model * update the test cases * fix finally the image processor and add slow tests * fixup * typo in docs * fix tests * the doc name uses underscore * address comments from Yoni * delete tests and unsuffling * relative import * do we really handle imports better now? * fix test * slow tests * found a bug in ordering + slow tests * fix copies * dont run compile test --------- Co-authored-by: Anna <anna@liquid.ai> Co-authored-by: Anna Banaszak <48625325+ankke@users.noreply.github.com> * Fix outdated version checks of accelerator (#40969) * Fix outdated version checks of accelerator Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix outdated version checks of accelerator Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Use `skip_predictor=True` in vjepa2 `get_vision_features` (#40966) use skip_predictor in vjepa2 `get_vision_features` * [Trainer] Fix DP loss (#40799) * fix * style * Fix fp16 * style --------- Co-authored-by: Matej Sirovatka <54212263+S1ro1@users.noreply.github.com> * [timm_wrapper] better handling of "Unknown model" exception in timm (#40951) * fix(timm): Add exception handling for unknown Gemma3n model * nit: Let’s cater to this specific issue * nit: Simplify error handling * Fix Issue #39030: AutoTokenizer.from_pretrained does not propagate token (#40956) * fix merge conflicts * change token typing --------- Co-authored-by: Ubuntu <ubuntu@ip-172-31-27-253.ec2.internal> * [tests] Really use small models in all fast tests (#40945) * start * xcodec * chameleon * start * layoutlm2 * layoutlm * remove skip * oups * timm_wrapper * add default * doc * consistency * Add captured actual outputs to CI artifacts (#40965) * fix * fix * Remove `# TODO: ???` as it make me `???` * fix * fix * fix --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Revert change in `compile_friendly_resize` (#40645) fix * Track the CI (model) jobs that don't produce test output files (process being killed etc.) (#40981) * fix * fix --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Remove `set_model_tester_for_less_flaky_tests` (#40982) remove * Benchmarking v2 GH workflows (#40716) * WIP benchmark v2 workflow * Container was missing * Change to sandbox branch name * Wrong place for image name * Variable declarations * Remove references to file logging * Remove unnecessary step * Fix deps install * Syntax * Add workdir * Add upload feature * typo * No need for hf_transfer * Pass in runner * Runner config * Runner config * Runner config * Runner config * Runner config * mi325 caller * Name workflow runs properly * Copy-paste error * Add final repo IDs and schedule * Review comments * Remove wf params * Remove parametrization from worfkflow files * Fix callers * Change push trigger to pull_request + label * Add back schedule event * Push to the same dataset * Simplify parameter description * ENH: Enable readline support for transformers chat (#40911) ENH Enable readline support for chat This small change enables GNU readline support for the transformers chat command. This includes, among others: - advanced navigation and editing: ctrl + a ctrl + e alt + b alt + f ctrl + k alt + d etc. - navigate and search history: arrow up/down ctrl + p ctrl + n ctrl + r - undo: ctrl + _ - clear screen: ctrl + l Implementation Although it may look strange, just importing readline is enough to enable it in Python, see: https://docs.python.org/3/library/functions.html#input As readline is not available on some platforms (https://docs.python.org/3/library/readline.html), the import is guarded. Readline should work on Linux, MacOS, and with WSL, I'm not sure about Windows though. Ideally, someone can give it a try. It's possible that Windows users would have to install pyreadline (https://pypi.org/project/pyreadline3/). * [testing] test `num_hidden_layers` being small in model tester (#40992) fix Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * blt wip (#38579) * blt wip * cpu version * cpu friendly with full entropy model (real time patching) * adding config file instead of args file * enable MPS * refactoring unused code * single config class in config file * inherit from PreTrainedModel * refactor LMTransformer --> BLTPatcher * add conversion script * load from new checkpoing with form_pretrained * fixed demo from_pretrained * clean up * clean a few comments * cleanup folder * clean up dir * cleaned up modeling further * rename classes * adding transformers Attention class and RotaryEmbedding class * exchanged blt modules for transformers modules: attention, rotary_emb, create_causal_mask, etc * seperate out patcher config, update modeling and conversion script * rename vars to be more transformers-like * rm unused functions * adding cross attention from transformers * pass arg * rename weights * updated conversion script * overwritten commit! fixing PR * apply feedback * adding BLTRMSNorm like Llama * add repeat_kv and eager_attention_forward copied from * BLTMLP identical to MllamTextMLP * clean up some args' * more like mllama, but busier inits * BLTTransformerLayer config * decoder, encoder, global configs * wip working on modular file * cleaning up patch and configs * clean up patcher helpers * clean up patcher helpers further * clean up * some config renaming * clean up unused configs * clean up configs * clean up configs * update modular * clean * update demo * config more like mllama, seperated subconfigs from subdicts * read from config instead of self args * update demo file * model weights to causal lm weights * missed file * added tied weights keys * BLTForCausalLM * adding files after add-new-model-like * update demo * working on tests * first running integration tests * added integration tests * adding tokenization tests, integration tests, and cleaned up tokenization file, + ruff * tokenizer clean up * modular file * fixing rebase * ruff * adding correct basemodel output and updating config with checkpoint vals (for testing) * BLTModelTests git status * enabling inputs_embeds, although won't be equal to input_ids since need ids for patching logic * fix sdpa == causal tests * fix small model test and some gradient checkpointing * skip training GC tests * fix test * updated modular * update modular * ruff * adding modular + modeling * modular * more modern is_casual check * cleaning up modular * more modular reduction * ruff * modular fix * fix styling * return 2 * return 2 * fix some tests * fix bltcrossattention after modular break * some fixes / feedback * try cache generate fix * try cache generate fix * fix generate tests * attn_impl workaround * refactoring to use recent TransformersKwargs changes * fix hidden_states shape test * refactor to new outputs * simplify outputs a bit * rm unneeded decoderlayer overwriting * rename blt * forgot tokenizer test renamed * Reorder * Reorder * working on modular * updates from modular * new modular * ruff and such * update pretrainedmodel modular * using cohere2 apply_rotary_pos_emb * small changes * apply feedback r2 * fix cross_attention * apply more feedback * update modeling fix * load submodules from pretrainedmodel * set initializer_range to subconfigs * rm cross_attnetion_states pass when not needed * add 7b projection layer support * check repo * make copies * lost cohere2 rotate_half * ruff * copies? * don't tie weights for submodules * tie weights setting * check docstrings * apply feedback * rebase * rebased modeling * update docs * applying feedback * few more fixes * fix can_record_outputs * fast tokenizer * no more modulelist * tok auto * rm tokenizersss * fix docs * ruff * fix after rebase * fix test, configs are not subscriptable --------- Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-168-30.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-161-103.ec2.internal> Co-authored-by: Lysandre <hi@lysand.re> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-174-36.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-164-45.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-173-121.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-160-103.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-161-178.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-162-79.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-169-239.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-167-111.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-160-100.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-161-153.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-166-15.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-165-131.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-161-138.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-174-215.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-172-142.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-172-147.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-164-0.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-163-58.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-165-202.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-166-244.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-174-186.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-160-192.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-162-14.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-171-249.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-164-75.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-161-78.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-163-134.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-162-180.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-175-241.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-160-225.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-167-9.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-168-34.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-166-68.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-167-175.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-170-160.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-168-95.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-172-73.ec2.internal> * [`RMSNorm`] Fix rms norm init for models that center around 1 (#40796) * fix * fixup inits * oops * fixup gemma * fixup modular order * how does this keep happen lol * vaultgemma is new i forgot * remove init check * Make `EfficientLoFTRModelTest` faster (#41000) * fix * fix * fix --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Fix typoes in src and tests (#40845) Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix more dates in model cards and wrong modalities in _toctree.yml (#40955) * Fix model cards and modalities in toctree * fix new models * RUFF fix on CI scripts (#40805) Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * fix dict like init for ModelOutput (#41002) * fix dict like init * style * [tests] update `test_left_padding_compatibility` (and minimize overwrites) (#40980) * update test (and overwrites) * better test comment * 0 as a default for * Patch more `unittest.case.TestCase.assertXXX` methods (#41008) fix Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * 🚨 [lightglue] fix: matches order changed because of early stopped indices (#40859) * fix: bug that made early stop change order of matches * fix: applied code suggestion Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * fix: applied code suggestion to modular * fix: integration tests --------- Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * Fix `PhimoeIntegrationTest` (#41007) * fix * fix * fix * fix * fix --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Fix Glm4v test (#41011) fix * Update after #41007 (#41014) * fix * fix --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Fix benchmark runner argument name (#41012) * Adding support for Qwen3Omni (#41025) * Add Qwen3Omni * make fix-copies, import properly * nit * fix wrong setup. Why was audio_token_id renamed ? * upds * more processing fixes * yup * fix more generation tests * down to 1? * fix import issue * style, update check repo * up * fix quality at my best * final quality? * fix doc building * FINAL COMMIT: SKIP IMPORTANT BUT FAILING TESTS FOR MERGE * SKIP THE TEMPLATE ONE --------- Co-authored-by: lvyuanjun.lyj <lvyuanjun.lyj@alibaba-inc.com> Co-authored-by: Arthur <arthur.zucker@gmail.com> * Making compute_loss_func always take priority in Trainer (#40632) * logger warn, if-else logic improved * redundant if condition fix * Modify Qwen3Omni parameter name since VL changed it (#41045) Modify parameter name since VL changed it Co-authored-by: lvyuanjun.lyj <lvyuanjun.lyj@alibaba-inc.com> * Fix Qwen video tests (#41049) fix test * [testing] Fix `qwen2_audio` (#41018) * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Fix typing of tuples (#41028) * Fix tuple typing Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * More fixes Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * More fixes Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Remove optax (#41030) Remove optax dep Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix typos in English/Chinese documentation (#41031) * Fix typos and formatting in English docs Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix typos and formatting in Chinese docs Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Use torch.autocast (#40975) * Use torch.autocast Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Format code Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * docs: improved RoPE function Docstrings (#41004) * docs: improved RoPE functuon docstrings * Update src/transformers/modeling_rope_utils.py Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> --------- Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> * Fix condition for emitting warning when generation exceeds max model length (#40775) correct warning when generation exceeds max model length Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com> * Fix outdated torch version check (#40925) Update torch minimum version check to 2.2 Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Add Whole Word Masking and Padding Strategy to DataCollatorForLanguageModeling (#39485) * Add whole word masking * Vectorize whole word masking functions * Unit test whole word masking * Remove support for TF in whole word masking * [testing] Fix `seed_oss` (#41052) * fix * fix * fix * fix * fix * fix * Update tests/models/seed_oss/test_modeling_seed_oss.py Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> * fix --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> * Remove repeated import (#40937) * Remove repeated import Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix conflict Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Simplify unnecessary Optional typing (#40839) Remove Optional Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Add write token for uploading benchmark results to the Hub (#41047) * Separate write token for Hub upload * Address review comments * Address review comments * Ci utils (#40978) * Add CI reports dir to gitignore * Add utils to run local CI * Review compliance * Style * License * Fix CI jobs being all red 🔴 (false positive) (#41059) fix Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Update quantization CI (#41068) * fix * new everything * fix * [i18n-bn] Add Bengali language README file (#40935) * [i18n-bn] Add Bengali language README file and update links in existing language files * Update Bengali README for clarity and consistency in model descriptions * Improve documentation and errors in Mamba2-based models (#41063) * fix bug in Mamba2 docs * correct 'because on of' issue * link to other Mamba2 model types * github URL is not changed * update error message in generated files * Update team member list for some CI workflows (#41094) * update list * update list --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * fix crash when using chat to send 2+ request to gptoss (#40536) Signed-off-by: Wang, Yi <yi.a.wang@intel.com> * Minor addition, no split modules for VideoMAEE (#41051) * added no split modules * fixed typo --------- Co-authored-by: Raushan Turganbay <raushan@huggingface.co> * Switch to `python:3.10-slim` for CircleCI docker images (#41067) fix Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Fix argument name in benchmarking script (#41086) * Fix argument name in benchmarking script * Adjust vars * Fix typos in documentation (#41087) Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix typing (#40788) * Fix optional typing Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix optional typing Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix schema typing Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix typing * Fix typing * Fix typing * Fix typing * Use np.ndarray Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix typing Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Format code Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Use np.ndarray Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Improve typing Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix quote string of np.ndarray Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * More fixes Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix code * Format Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Remove unused arguments (#40916) * Fix unused arguments Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * More fixes Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * fix wrong height and width when read video use torchvision (#41091) * docs: Fix Tool Use links and remove dead RAG links (#41104) docs: Fix tool use links. Remove dead RAG links. Fix style * [tests] gpt2 + `CausalLMModelTester` (#41003) * tmp commit * tmp commit * tmp commit * rm old GPT2ModelTester * nit bug * add facilities for encoder-decoder tests; add comments on ALL overwrites/extra fns * vision_encoder_decoder * Fix `_get_test_info` for inherited tests (#41106) * fix _get_test_info * fix patched * add comment * ruff --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Remove bad test skips (#41109) * remove bad skips * remove more * fix inits * Format empty lines and white space in markdown files. (#41100) * Remove additional white space and empty lines from markdown files Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Add empty lines around code Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Update ruff to 0.13.1 + target Python 3.10 + apply fixes (#37809) Update ruff to 0.13.1 target it to Python 3.10 and apply its fixes Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com> * Support loading LFM2 GGUF (#41111) * add gguf config mapping for lfm2 * add lfm2 tensor process to unsqueeze conv weights * adjust values from gguf config to HF config * add test for lfm2 gguf * ruff --------- Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> * [torchao safetensors] integrate torchao safetensors support with transformers (#40735) * enable torchao safetensors * enable torchao safetensors support * add more version checking * [Qwen3-next] Fix dimension mismatch in torch_chunk_gated_delta_rule and torch_recurrent_gated_delta_rule (#40963) (#41036) * fix mismatched dims for qwen3 next * propagate changes * chore: renamed tot_heads to total_sequence_length * Apply suggestion from @vasqu Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> * minor fix to modular qwen3 next file --------- Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> * Fix the error where a keyword argument appearing before *args (#41099) Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix broken `` expressions in markdown files (#41113) Fix broken expressions in markdown files Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Remove self-assignment (#41062) * Remove self-assignment Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Update src/transformers/integrations/flash_paged.py Co-authored-by: Matt <Rocketknight1@users.noreply.github.com> * Clear pass Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Clear pass Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Clear pass Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> Co-authored-by: Matt <Rocketknight1@users.noreply.github.com> * Fixed MXFP4 model storage issue (#41118) * Fixed loading LongT5 from legacy checkpoints (#40724) * Fixed loading LongT5 from legacy checkpoints * Adapted the fix to work with missing lm_head * dummy commit (#41133) * dummy commit, nothing interesting * dummy commit, nothing interesting * dummy commit, nothing interesting * dummy commit, nothing interesting --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Fix loading logic flaw with regards to unexpected and missing keys (#40850) * Unexpected keys should be ignored at load with device map * remove them all * fix logic flaw * fix * simplify * style * fix * revert caching allocator change * add other test * add nice doc --------- Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com> * Fix: align Qwen2.5-VL inference rope index with training by passing s… (#41153) Fix: align Qwen2.5-VL inference rope index with training by passing second_per_grid_ts * Fix single quotes in markdown (#41154) Fix typos Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * extend gemma3n integration ut cases on XPU (#41071) Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * Add Parakeet (#39062) * first commit Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * update to handle masking for bs>1 Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * Add tests and docs Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * update model ids Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * update docs and improve style Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * update librosa location Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * import guard torch too Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * ruff code checks fix Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * ruff format check Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * updated to parakeet names Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * update script Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * Add tokenizer decoding Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * Remove other model dependency Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * clean tests Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * fix tests Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * linting Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * fix ruff lint warnings Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * move to seperate folders Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * add parakeet ctc model code Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * simplify encoder structure Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * update documentation Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * add parakeet to toctree Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * fix tests Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * add parakeet doc Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * Address comments Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * Update featurizer to compute lens directly Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * fix ruff tests Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * fix encoding format Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * fix minor ctc decoding Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * revert modular_model_converter.py changes * revert check_config_attributes.py changes * refactor: fastconformer & parakeet_ctc -> parakeet * modeling update * test update * propagate feature extractor updates * propagate doc changes * propagate doc changes * propagate tokenization changes * propagate conversion changes * remove fastconformer tests * remove modular * update processor * update processor * tset update * diverse fixes * 100% macthing greedy batched * Update conversion script. * Refactor docs. * Reafactor auto loading. * Refactor and fix tokenization and processing. * Update integration test. * Modeling fixes: - ensure correct attention mask shape - ensure layer drop returns valid output - correct blank token ID when computing CTC loss * Format and repo consistency. * Update model doc. * Fix feature extraction tests. * Fix (most) tokenizer tests. * Add pipeline example. * Fixes * Use eager_attention_forward from Llama. * Small tweaks. * Replace Sequential with ModuleList * Add check if not all layers copied * Clean tokenizer. * Standardize FastSpeech2ConformerConvolutionModule for Parakeet. * Switch to modular for modeling and processing. * Add processor tests. * Fix modeling tests. * Formating and docstrings. * Add `return_attention_mask` like other feature extractors. * clean up after merging main. * nits on modeling * configuration update * nit * simplification: use PretrainedTokenizerFast, simplify processor * add dtype arg to mel_filter_bank * feature extraction: simplify! * modeling update * change to ParakeetTokenizerFast * correct attention mask handling * auto update * proc update * test update * feature extraction fixes * modeling update * conversion script update * udpate tests feature integration * update tokenization and tests * processor tests * revert audio_utils * config docstring update * blank_token -> pad_token * modeling udpate * doc update * fix tests * fix test * fix tests * address review comments * add comment * add comment * explicitly not support flash * atttention straightforward masking * fix * tokenizer update: skipping blank tokens by default * doc update * fix max_positions_embeddings handling * nits * change atol faeture extraction integration tests * doc update + fix loss * doc update * nit * update integration test for A10 * repo id name * nit --------- Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Co-authored-by: Eustache Le Bihan <eulebihan@gmail.com> Co-authored-by: eustlb <94853470+eustlb@users.noreply.github.com> Co-authored-by: Eric B <ebezzam@gmail.com> * Fix format of compressed_tensors.md (#41155) * Fix table format Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix format Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Simplify and improve model loading logic (#41103) * remove unexpected keys from inputs (they have nothing to do there) * remove input * simplify a lot init * fix * fix check for non-persistent buffer * revert because too many old and bad models... * remove comment * type hint * make it a real test * remove model_to_load -> always use the same model * typo * remove legacy offload_folder (we never waste that memory anymore) * do not change prefix anymore * change very bad function name * create adjust method * remove useless method * restrict * BC * remove unused method * CI * remove unused args * small fix * fix * CI * CI * avoid too many loops * fix regex * cleaner * typo * fix * fix * Force new vision models addition to include a fast image processor (#40802) * add test * fix test and change cutoff date * Add documentation to test * Add language specifiers to code blocks of markdown files (#41114) * Add language specifiers to code blocks of markdown files Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Update docs/source/en/model_doc/qwen3_omni_moe.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/chat_templating_writing.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/chat_templating_writing.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update docs/source/en/chat_templating_writing.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * More fixes Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Update nemotron.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update phimoe.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Update README.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Fix syntax error Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * Improve `add_dates` script (#41167) * utils/add_dates.py * put lfm2-vl in correct category * Fix flash-attn for paged_attention when no kernels (#41078) * Fix non-kernels flash attention paged implementation * Cover all cases * Style * Update src/transformers/integrations/flash_paged.py Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com> * Apply style fixes --------- Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Remove data from examples (#41168) Remove telemetry * Enable fa in amd docker (#41069) * Add FA to docker * Use caching mechanism for qwen2_5 * Fix a typo in important models list * Partial fixes for gemma3 * Added a commit ID for FA repo * Detailled the expectation storage format * Rebase fix * Apply style fixes --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * handle flash slow tests (#41072) * handle flash slow tests * update patch mask to 1/0 for flash * don't skip flash * flash * raise tols * rm flash support :( * nits --------- Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-173-7.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-171-230.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-168-95.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-166-214.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-163-147.ec2.internal> * Modernbert fix (#41056) * Add FA to docker * Fixed padding for mdernbert * Fixed logits and hidden states extraction in ModernBertForMultipleChoice * Added a test for ModernBertForMultipleChoice * fixes * More fixes and GREEN CI * consistency * moar consistency * CI Runners - move amd runners mi355 and 325 to runner group (#41193) * Update CI workflows to use devmi355 branch * Add workflow trigger for AMD scheduled CI caller * Remove unnecessary blank line in workflow YAML * Add trigger for workflow_run on main branch * Update workflow references from devmi355 to main * Change runner_scale_set to runner_group in CI config * [XPU] Add MXFP4 support for XPU (#41117) * XPU supports gpt-oss MXFP4 * Complete MXFP4 UT file and comment information * Complete MXFP4 UT file and comment information * Fix code style * Fix code style --------- Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> * [tests] `CausalLMTester` automatically infers other test classes from `base_model_class` 🐛 🔫 (#41066) * halfway through the models * update test checks * refactor all * another one * use tuples * more deletions * solve bad inheritance patterns * type * PR ready? * automatic model class inference from the base class * vaultgemma * make fixup * make fixup * rebase with gpt2 * make fixup :'( * gpt2 is special * More typing fixes (#41102) * Fix noqa Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * fix typing Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Use np.ndarray Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * More fixes Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * remove noqa Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix chars Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * More fixes Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * enable flex attention ut cases on XPU (#40989) * enable flex attention ut cases on XPU Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * fix style Signed-off-by: Yao, Matrix <matrix.yao@intel.com> --------- Signed-off-by: Yao, Matrix <matrix.yao@intel.com> Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> * fix(trainer): Avoid moving model with device_map (#41032) * fix(trainer): Avoid moving model with device_map When a model is loaded with `device_map="auto"` and is too large to fit on a single GPU, `accelerate` will offload some layers to the CPU or disk. The `Trainer` would previously attempt to move the entire model to the specified device, causing a `RuntimeError` because a model dispatched with `accelerate` hooks cannot be moved. This commit fixes the issue by adding a check in `_move_model_to_device` to see if the model has an `hf_device_map` attribute. If it does, the device placement is assumed to be handled by `accelerate`, and the `model.to(device)` call is skipped. A regression test is added to ensure the `Trainer` can be initialized with a model that has a `hf_device_map` that simulates offloading without raising an error. * Added the logger warning for the move model --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Fix attention sink implementation in flex attention (#41083) * Fix attention sink implementation in flex attention * fix dim * fix * Remove print * raisae error when return_lse is False yet s_aux is providewd * Clean test files for merge * Update src/transformers/integrations/flex_attention.py Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * force return lse * Add to doc --------- Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * Separate docker images for Nvidia and AMD in benchmarking (#41119) Separate docker images for Nvidia and AMD * Make quantizers good citizens loading-wise (#41138) * fix param_needs_quantization * rewrite most hqq * clean * fix * comment * remove it from exception of safetensors * start on bnb 4bits * post-rebase fix * make bnb4 bit a good citizen * remove forgotten print * make bnb 8bits a good citizen * better hqq * fix * clean * remove state dict from signature * switch method * make torchao a good citizen * fixes * fix torchao * add check * typo * [`Kernels Attention`] Change fallback logic to error out on explicit kernels request and include FA3 (#41010) * fix * be more strict * change logic to include fa3 * fix the case where nothing is requested * modify old tests + add kernels related tests * style * Add EdgeTAM (#39800) * initial comment * test * initial conversion for outline * intermediate commit for configuration * chore:init files for sam2 * adding arbitary undefined config * check * add vision * make style * init sam2 base model * Fix imports * Linting * chore:sam to sam2 classes * Linting * Add sam2 to models.__init__ * chore:match prompt encoder with sam2 code * chore:prepare kwargs for mask decoder * Add image/video predictors * Add CUDA kernel * Add output classes * linting * Add logging info * tmp commit * docs for sam2 * enable image processing * check difference of original SAM2 - difference is the order of ToTensor() - please see https://pytorch.org/vision/main/_modules/torchvision/transforms/functional.html#resize * enable promptencoder of sam2 * fix promprencoder * Confirmed that PromptEncoder is exactly same (Be aware of bfloat16 and float32 difference) * Confirmed that ImageEncoder is exactly same (Be aware the linting of init) * Confirmed that MaskDecoder is exactly same (TO DO: lint variable name) * SamModel is now available (Need more chore for name) * make fix-copies * make style * make CI happy * Refactor VisionEncoder and PostioinEmbedding * TO DO : fix the image_embeddings and sparse_embeddings part * pure image inference done * reusable features fix and make style * styling * refactor memoryattention * tmp * tmp * refactor memoryencoder TO DO : convert and inference the video pipeline * TO DO : fix the image_encoder shape * conversion finish TO DO: need to check video inference * make style * remove video model * lint * change * python utils/check_docstringspy --check_all * python utils/check_config_attributes.py * remove copies for sam2promptencoder due to configuration * change __init__.py * remove tensorflow version * fix that to not use direct comparison * make style * add missing import * fix image_embedding_size * refactor Sam2 Attention * add fully working video inference (refactoring todo) * clarify _prepare_memory_conditioned_features * simplify modeling code, remove unused paths * use one model * use auto_docstring * refactor rope embeddings * nit * not using multimask when several points given * add all sam2.1 * add video tmp * add Sam2VideoSessionState + fast image proc + video proc * remove init_states from model * fix batch inference * add image integration tests * uniformize modeling code with other sam models and use modular * pass vision tests an most model tests * All tests passing * add offloading inference state and video to cpu * fix inference from image embedding and existing mask * fix multi_boxes mask inference * Fix batch images + batch boxes inference * improve processing for image inference * add support for mask generation pipeline * add support for get_connected_components post processing in mask generation * add fast image processor sam, image processor tests and use modular for sam2 image processor * fix mistake in sam after #39120 * fix init weights * refactor convert * add integration tests for video + other improvements * add needed missing docstrings * Improve docstrings and * improve inference speed by avoiding cuda sync * add test * skip test for vision_model * minor fix for vision_model * fix vision_model by adding sam2model and change the torch dependencies * remove patch_size * remove image_embedding_size * fix patch_size * fix test * make style * Separate hieradet and vision encoder in sam2 * fixup * review changes part 1 * remove MemoryEncoderConfig and MemoryAttentionConfig * pass q_stride instead of q_pool module * add inference on streamed videos * explicitely process streamed frames * nit * Improve docstrings in Sam2Model * update sam2 modeling with better gestion of inference state and cache, and separate Sam2Model and Sam2VideoModel * improve video inference api * change inference_state to inference_session * use modular for Sam2Model * fix convert sam2 hf * modular * Update src/transformers/models/sam2/video_processing_sam2.py Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * fix minor config * fix attention loading error * update modeling tests to use hub checkpoints * Use CI A10 runner for integration tests values + higher tolerance for video integration tests * PR review part 1 * fix doc * nit improvements * enforce one input format for points, labels and boxes * nit * last few nits from PR review * fix style * fix the input type * fix docs * add sam2 model as conversion script * improve sam2 doc * add rough necessarry changes * first working edgetam * fix issue with object pointers * Use modular as much as possible * nit fixes + optimization * refactor spatial perceiver * cleanup after merge * add working edgetam * improve perceiver resampler code * simplify/unify rope attention logic * Improve comments in apply_rotary_pos_emb_2d * add working tests * fix test timmwrapper * add docs * make fixup * nits * fix modular * fix modular * PR review part 1 * split apply_rotary_pos_emb_2d * add granularity to _prepare_memory_conditioned_features * add dates to doc * add separate mlp for memory attention * Fix memory on wrong device * store processed frames in dict * update checkpoints in tests * update dates --------- Co-authored-by: sangbumchoi <danielsejong55@gmail.com> Co-authored-by: RUFFY-369 <prakarshkaushik369@gmail.com> Co-authored-by: Sangbum Daniel Choi <34004152+SangbumChoi@users.noreply.github.com> Co-authored-by: Haitham Khedr <haithamkhedr@meta.com> Co-authored-by: sangbum choi <sangbumchoi@sangbumui-MacBookAir.local> Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * Fix EXAONE-4.0 dummy id (#41089) * Fix EXAONE-4.0 dummy id * Fix exaone4 dummy (#1) * fix * fix * fix * fix * fix --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> --------- Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com> Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Fix 8bit bnb loading (#41200) * Fix 8bit * oups forgot the case where it is not prequantized * Fix docker quantization (#41201) * launch docker * remove gptq for now * run tests * Revert "run tests" This reverts commit f85718ce3a21d5937bf7405b8925c125c67d1a3e. * revert * Embed interactive timeline in docs (#41015) * embed timeline in docs (test web componentand Iframe) * test scaling * test multiple scales * compensate scale in width * set correct syle and scale * remove bottom space created by scale * add timeline as a separate page * reformulate docs after review * [docs] Fix links (#41110) fix * Remove unnecessary Optional typing (#41198) Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * docs/examples(speech): pin CTC commands to Hub datasets; add Windows notes (#41027) * examples(speech): load Common Voice from Hub; remove deprecated dataset-script references (Windows-friendly notes) * docs/examples(speech): pin CTC streaming & other CTC commands to Hub datasets; add Windows notes * make style * examples(speech): align DataTrainingArguments help with datasets docs; minor wording fixes * docs/examples(speech): address review remove Hub subsection & Whisper tip; align dataset help text * style: apply ruff/black/usort/codespell on examples/speech-recognition * Apply style fixes * Update examples/pytorch/speech-recognition/README.md * update doc to match load_dataset --------- Co-authored-by: Eustache Le Bihan <eulebihan@gmail.com> Co-authored-by: eustlb <94853470+eustlb@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Fix Qwen3-Omni audio_token_id serialization issue (#41192) Fix Qwen3-Omni audio_token_id serialization by overriding parent's attribute_map - Override attribute_map in Qwen3OmniMoeThinkerConfig to prevent inheritance of incorrect mapping - Parent class maps audio_token_id -> audio_token_index, but implementation uses audio_token_id directly - Fixes issue where custom audio_token_id values were not preserved during save_pretrained/from_pretrained cycles Fixes #41191 * Wait for main process in _save_checkpoint to ensure best checkpoint exists (#40923) * Update trainer.py * fix * fix format * move barrier, delete redundant * Avoid assumption that model has config attribute in deepspeed (#41207) Avoid assumption that model has config in deepspeed * Trainer: Pass `num_items_in_batch` to `compute_loss` in `prediction_step` (#41183) * Add num_items_in_batch computation to predict_step. * address comments. * Fix test cases. * fixup --------- Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> * [ESM] add accepts_loss_kwargs=False to EsmPreTrainedModel (#41006) add accepts_loss_kwargs=False to EsmPreTrainedModel Signed-off-by: Peter St. John <pstjohn@nvidia.com> Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> * Align pull request template to bug report template (#41220) The only difference is that I don't users to https://discuss.huggingface.co/ for hub issues. * [generate] cache missing custom generate file (#41216) * cache missing custom generate file * make fixup * Remove old Python code (#41226) Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Adapt to the SDPA interface to enable the NPU to call FlashAttentionScore (#41143) Adapt to the SDPA interface to enable the NPU to call FlashAttentionScore. Co-authored-by: frozenleaves <frozen@Mac.local> * update code owners (#41221) Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Unify is_torchvision_v2_available with is_torchvision_available (#41227) Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix typing of train_args (#41142) * Fix typing Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix fsdp typing Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix sliding window attn mask (#41228) * Fix sliding window attn mask * Clearer test * Apply style fixes * If Picasso made ascii drawings he would have made this --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Revert "Fix DeepSpeed mixed precision precedence over Accelerate defaults" (#41124) * Revert "Fix DeepSpeed mixed precision precedence over Accelerate defaults (#3…" This reverts commit df67cd35f0ca1a1cbf7147b2576db31b16200cf4. * fix * [docs] Fix tp_plan (#41205) remove manual * Fix white space in documentation (#41157) * Fix white space Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Revert changes Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix autodoc Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * fix qwen text config (#41158) * fix qwen text config * fix tests * fix one more test * address comments * Video processor accepts single frames on cuda (#41218) * fix * why was is np if input is in torch * Use math.log2 (#41241) Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * fix TrainerIntegrationDeepSpeed UT failures (#41236) Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * [repo utils] Update `models_to_deprecate.py` (#41231) * update models_to_deprecate * exclude this file * handle typos and aliases * don't commit files * PR suggestions; make fixup * Use removeprefix and removesuffix (#41240) * Use removeprefix and removesuffix Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * More fixes Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix pylint warnings (#41222) * Remove unused variables Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Remove reimported packages Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix code Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix pylint warnings Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Simplify Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Remove all instances of `is_safetensors_available` (#41233) * safetensors is a core dep * fix * ok * simplify branching * keep it for now --------- Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com> * FP-Quant NVFP4 and Python 3.9 support (#39876) * quartet * quartet qat -> quartet * format * bf16 backward * interfaces * forward_method * quartet -> fp_quant * style * List -> list * list typing * fixed format and annotations * test_fp_quant * docstrings and default dtypes * better docstring and removed noop checks * docs * pseudoquantization support to test on non-blackwell * pseudoquant * Pseudoquant docs * Update docs/source/en/quantization/fp_quant.md Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> * Update docs/source/en/quantization/fp_quant.md * Update docs/source/en/quantization/fp_quant.md * Update src/transformers/utils/quantization_config.py Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com> * Update tests/quantization/fp_quant_integration/test_fp_quant.py Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com> * Update tests/quantization/fp_quant_integration/test_fp_quant.py Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> * small test fixes * dockerfile update * spec link * removed `_process_model_after_weight_loading` * toctree * nvfp4 * nvfp4 tests * FP-Quant version bumped * nvfp4 default and docs update * trainable * cpu if pseudoquant * proper group size selection * gsr * qutlass requirement version bumo * Upstream docker copy * docs update --------- Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com> * [`FA3`] Fix masking and loading logic in same process (#41217) fix loading and fa3 masking * [t5gemma] fix `get_text_config` and related fixes (#40939) * tmp commit * t5gemma fixes * Don't convert to `safetensors` on the fly if the call is from testing (#41194) * don't convert * disable * Update src/transformers/modeling_utils.py Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co> * fix * disable * disable * disable --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co> * Resolve remote custom module path warnings (#41243) * add peft team members to issue/pr template (#41262) * add * Update .github/PULL_REQUEST_TEMPLATE.md Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com> --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com> * docs: update bitsandbytes platform support (#41266) * add more activation kernels, follow up (#40944) * add more activation kernels * fixing style * fix version * fix asr pipeline ut failures (#41275) * fix asr pipeline ut failures Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * make style Signed-off-by: Yao, Matrix <matrix.yao@intel.com> --------- Signed-off-by: Yao, Matrix <matrix.yao@intel.com> * Use regex defailed flags (#41264) Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix multi-video timestamp bug in Qwen-3-VL and GLM4V (#41229) * fix multi-video timestamp bug in qwen3vl,glm4v * run make fix-copies to sync modular files * run make fix-copies to sync modular files --------- Co-authored-by: UBT <daqin.luo@ubtrobot.com> * Fix binding of video frames to video placeholder in `InternVL` model (#41237) * Fix binding video frames to video placeholder in prompt Signed-off-by: Daniel Bershatsky <daniel.bershatsky@gmail.com> * Add test on binding video frames to prompt Signed-off-by: Daniel Bershatsky <daniel.bershatsky@gmail.com> * Fix code style issues Signed-off-by: Daniel Bershatsky <daniel.bershatsky@gmail.com> * Fix broken tests on `InternVLProcessor` Signed-off-by: Daniel Bershatsky <daniel.bershatsky@gmail.com> * Add `return_tensors` to video processor defaults Signed-off-by: Daniel Bershatsky <daniel.bershatsky@gmail.com> --------- Signed-off-by: Daniel Bershatsky <daniel.bershatsky@gmail.com> * Deprecate Trackio environment variables and deploy to Spaces by default (#40950) * allow prive space id for trackio * complete docstring * Deprecate environment variables for Trackio integration; use TrainingArguments instead and deploy by default * style * Enhance documentation for Trackio Space ID in TrainingArguments * Allow private Space id for Trackio (#40948) * allow prive space id for trackio * complete docstring * fix async client for transformers chat (#41255) * fix-client * fix * Unify is_torchvision_v2_available with is_torchvision_available (#41259) Fix is_torchvision_v2_available Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Use max/min (#41280) Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Biogptlogits (#41270) added logits slicing to BioGpt for seq classifier Signed-off-by: Aviral <aviralkamaljain@gmail.com> * Fix unnecessary single-item container checks (#41279) Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> * Fix pylint generator warnings (#41258) Fix pylint generator warnings Signed-off-by: cyy <cyyever@outlook.com> * feat: use `aws-highcpu-32-priv` for amd docker img build (#41285) * feat: use `aws-highcpu-32-priv` for amd docker img build * feat: add `workflow_dispatch` event to docker build CI * Add processor and intergration test for qwen3vl (#41277) * support aux loss in qwen3vlmoe * update qwen3vl processor test! * add integration tests for qwen3vl-30a3 * remove duplicated decorator * code clean * fix consistency * do not inherit from nn.Linear for better quantization * pass check * Remove `test_initialization` (#41261) remove it * Remove some previous team members from allow list of triggering Github Actions (#41263) * delete * delete --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Build doc in 2 jobs: `en` and `other languages` (#41290) * separate * separate --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * Fix mxfp4 dequantization (#41292) fix * [`Flex Attn`] Fix lse x attention sinks logic (#41249) fix * FIX: Bug in PEFT integration delete_adapter method (#41252) The main content of this PR is to fix a bug in the delete_adapter method of the PeftAdapterMixin. Previously, it did not take into account auxiliary modules from PEFT, e.g. those added by modules_to_save. This PR fixes this oversight. Note that the PR uses a new functionality from PEFT that exposes integration functions like delete_adapter. Those will be contained in the next PEFT release, 0.18.0 (yet unreleased). Therefore, the bug is only fixed when users have a PEFT version fullfilling this requirement. I ensured that with old PEFT versions, the integration still works the same as previously. The newly added test for this is skipped if the PEFT version is too low. (Note: I tested locally with that the test will pass with PEFT 0.18.0) While working on this, I also cleaned up the following: - The active_adapter property has been deprecated for more than 2 years (#26407). It is safe to remove it now. - There were numerous small errors or outdated pieces of information in the docstrings, which have been addressed. When PEFT < 0.18.0 is used, although we cannot delete modules_to_save, we can still detect them and warn about it. * Italian translation for README.md (#41269) chore: add Italian translation for README.md * Fix README.md error when installing from source (#41303) * download and use HF Hub Cache (#41181) use hub cache Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> * fix some merge issues * [test_all] * [test-all] --------- Signed-off-by: Yuanyuan Chen <cyyever@outlook.com> Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com> Signed-off-by: Wang, Yi <yi.a.wang@intel.com> Signed-off-by: Yao, Matrix <matrix.yao@intel.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: Peter St. John <pstjohn@nvidia.com> Signed-off-by: Daniel Bershatsky <daniel.bershatsky@gmail.com> Signed-off-by: Aviral <aviralkamaljain@gmail.com> Signed-off-by: cyy <cyyever@outlook.com> Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com> Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> Co-authored-by: Rangehow <88258534+rangehow@users.noreply.github.com> Co-authored-by: rangehow <rangehow@foxmail.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> Co-authored-by: Raushan Turganbay <raushan@huggingface.co> Co-authored-by: Anna <anna@liquid.ai> Co-authored-by: Anna Banaszak <48625325+ankke@users.noreply.github.com> Co-authored-by: Yuanyuan Chen <cyyever@outlook.com> Co-authored-by: Hamish Scott <41787553+hamishs@users.noreply.github.com> Co-authored-by: Matej Sirovatka <54212263+S1ro1@users.noreply.github.com> Co-authored-by: Harshal Janjani <75426551+harshaljanjani@users.noreply.github.com> Co-authored-by: Branden <brandenkmurray@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-27-253.ec2.internal> Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co> Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> Co-authored-by: Ákos Hadnagy <akos@ahadnagy.com> Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com> Co-authored-by: Ita Zaporozhets <31893021+itazap@users.noreply.github.com> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-168-30.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-161-103.ec2.internal> Co-authored-by: Lysandre <hi@lysand.re> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-174-36.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-164-45.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-173-121.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-160-103.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-161-178.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-162-79.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-169-239.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-167-111.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-160-100.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-161-153.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-166-15.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-165-131.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-161-138.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-174-215.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-172-142.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-172-147.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-164-0.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-163-58.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-165-202.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-166-244.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-174-186.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-160-192.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-162-14.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-171-249.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-164-75.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-161-78.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-163-134.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-162-180.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-175-241.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-160-225.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-167-9.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-168-34.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-166-68.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-167-175.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-170-160.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-168-95.ec2.internal> Co-authored-by: ita.zaporozhets@huggingface.co <ita_zaporozhets@ip-26-0-172-73.ec2.internal> Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> Co-authored-by: StevenBucaille <steven.bucaille@gmail.com> Co-authored-by: BakerBunker <17872844+BakerBunker@users.noreply.github.com> Co-authored-by: lvyuanjun.lyj <lvyuanjun.lyj@alibaba-inc.com> Co-authored-by: Ayush <ayushtanwar1729@gmail.com> Co-authored-by: Ryan Mullins <ryan@ryanmullins.org> Co-authored-by: Yannick Schnider <Yannick.Schnider1@ibm.com> Co-authored-by: Ralph Gleaton <70818603+rjgleaton@users.noreply.github.com> Co-authored-by: Rémi Ouazan <83456801+remi-or@users.noreply.github.com> Co-authored-by: Saidur Rahman Pulok <59414463+saidurpulok@users.noreply.github.com> Co-authored-by: Nick Doiron <ndoiron@mapmeld.com> Co-authored-by: Wang, Yi <yi.a.wang@intel.com> Co-authored-by: Duygu Altinok <duygu.altinok12@gmail.com>…

* initial comment * test * initial conversion for outline * intermediate commit for configuration * chore:init files for sam2 * adding arbitary undefined config * check * add vision * make style * init sam2 base model * Fix imports * Linting * chore:sam to sam2 classes * Linting * Add sam2 to models.__init__ * chore:match prompt encoder with sam2 code * chore:prepare kwargs for mask decoder * Add image/video predictors * Add CUDA kernel * Add output classes * linting * Add logging info * tmp commit * docs for sam2 * enable image processing * check difference of original SAM2 - difference is the order of ToTensor() - please see https://pytorch.org/vision/main/_modules/torchvision/transforms/functional.html#resize * enable promptencoder of sam2 * fix promprencoder * Confirmed that PromptEncoder is exactly same (Be aware of bfloat16 and float32 difference) * Confirmed that ImageEncoder is exactly same (Be aware the linting of init) * Confirmed that MaskDecoder is exactly same (TO DO: lint variable name) * SamModel is now available (Need more chore for name) * make fix-copies * make style * make CI happy * Refactor VisionEncoder and PostioinEmbedding * TO DO : fix the image_embeddings and sparse_embeddings part * pure image inference done * reusable features fix and make style * styling * refactor memoryattention * tmp * tmp * refactor memoryencoder TO DO : convert and inference the video pipeline * TO DO : fix the image_encoder shape * conversion finish TO DO: need to check video inference * make style * remove video model * lint * change * python utils/check_docstringspy --check_all * python utils/check_config_attributes.py * remove copies for sam2promptencoder due to configuration * change __init__.py * remove tensorflow version * fix that to not use direct comparison * make style * add missing import * fix image_embedding_size * refactor Sam2 Attention * add fully working video inference (refactoring todo) * clarify _prepare_memory_conditioned_features * simplify modeling code, remove unused paths * use one model * use auto_docstring * refactor rope embeddings * nit * not using multimask when several points given * add all sam2.1 * add video tmp * add Sam2VideoSessionState + fast image proc + video proc * remove init_states from model * fix batch inference * add image integration tests * uniformize modeling code with other sam models and use modular * pass vision tests an most model tests * All tests passing * add offloading inference state and video to cpu * fix inference from image embedding and existing mask * fix multi_boxes mask inference * Fix batch images + batch boxes inference * improve processing for image inference * add support for mask generation pipeline * add support for get_connected_components post processing in mask generation * add fast image processor sam, image processor tests and use modular for sam2 image processor * fix mistake in sam after huggingface#39120 * fix init weights * refactor convert * add integration tests for video + other improvements * add needed missing docstrings * Improve docstrings and * improve inference speed by avoiding cuda sync * add test * skip test for vision_model * minor fix for vision_model * fix vision_model by adding sam2model and change the torch dependencies * remove patch_size * remove image_embedding_size * fix patch_size * fix test * make style * Separate hieradet and vision encoder in sam2 * fixup * review changes part 1 * remove MemoryEncoderConfig and MemoryAttentionConfig * pass q_stride instead of q_pool module * add inference on streamed videos * explicitely process streamed frames * nit * Improve docstrings in Sam2Model * update sam2 modeling with better gestion of inference state and cache, and separate Sam2Model and Sam2VideoModel * improve video inference api * change inference_state to inference_session * use modular for Sam2Model * fix convert sam2 hf * modular * Update src/transformers/models/sam2/video_processing_sam2.py Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * fix minor config * fix attention loading error * update modeling tests to use hub checkpoints * Use CI A10 runner for integration tests values + higher tolerance for video integration tests * PR review part 1 * fix doc * nit improvements * enforce one input format for points, labels and boxes * nit * last few nits from PR review * fix style * fix the input type * fix docs * add sam2 model as conversion script * improve sam2 doc * add rough necessarry changes * first working edgetam * fix issue with object pointers * Use modular as much as possible * nit fixes + optimization * refactor spatial perceiver * cleanup after merge * add working edgetam * improve perceiver resampler code * simplify/unify rope attention logic * Improve comments in apply_rotary_pos_emb_2d * add working tests * fix test timmwrapper * add docs * make fixup * nits * fix modular * fix modular * PR review part 1 * split apply_rotary_pos_emb_2d * add granularity to _prepare_memory_conditioned_features * add dates to doc * add separate mlp for memory attention * Fix memory on wrong device * store processed frames in dict * update checkpoints in tests * update dates --------- Co-authored-by: sangbumchoi <danielsejong55@gmail.com> Co-authored-by: RUFFY-369 <prakarshkaushik369@gmail.com> Co-authored-by: Sangbum Daniel Choi <34004152+SangbumChoi@users.noreply.github.com> Co-authored-by: Haitham Khedr <haithamkhedr@meta.com> Co-authored-by: sangbum choi <sangbumchoi@sangbumui-MacBookAir.local> Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>

ShaohonChen · 2025-10-08T03:35:11Z

Hello! I noticed that while using Transformers, the hidden_states return previously available in BaseModelOutputWithPast seems to have been removed starting from this PR. I recall that this variable was used to return the hidden states of each decoder layer in a transformer network. I’d like to know why this return value was removed—was it due to performance considerations or other reasons? I also noticed that this return value still exists in CausalLMOutputs. When developing models based on Transformers in the future, should we still implement this return? I’d appreciate some clarification. Thank you!🙏 @ArthurZucker

ArthurZucker · 2025-10-08T11:58:52Z

Hey! Its not removed! If you set output_hidden_states you will still get them!

* initial comment * test * initial conversion for outline * intermediate commit for configuration * chore:init files for sam2 * adding arbitary undefined config * check * add vision * make style * init sam2 base model * Fix imports * Linting * chore:sam to sam2 classes * Linting * Add sam2 to models.__init__ * chore:match prompt encoder with sam2 code * chore:prepare kwargs for mask decoder * Add image/video predictors * Add CUDA kernel * Add output classes * linting * Add logging info * tmp commit * docs for sam2 * enable image processing * check difference of original SAM2 - difference is the order of ToTensor() - please see https://pytorch.org/vision/main/_modules/torchvision/transforms/functional.html#resize * enable promptencoder of sam2 * fix promprencoder * Confirmed that PromptEncoder is exactly same (Be aware of bfloat16 and float32 difference) * Confirmed that ImageEncoder is exactly same (Be aware the linting of init) * Confirmed that MaskDecoder is exactly same (TO DO: lint variable name) * SamModel is now available (Need more chore for name) * make fix-copies * make style * make CI happy * Refactor VisionEncoder and PostioinEmbedding * TO DO : fix the image_embeddings and sparse_embeddings part * pure image inference done * reusable features fix and make style * styling * refactor memoryattention * tmp * tmp * refactor memoryencoder TO DO : convert and inference the video pipeline * TO DO : fix the image_encoder shape * conversion finish TO DO: need to check video inference * make style * remove video model * lint * change * python utils/check_docstringspy --check_all * python utils/check_config_attributes.py * remove copies for sam2promptencoder due to configuration * change __init__.py * remove tensorflow version * fix that to not use direct comparison * make style * add missing import * fix image_embedding_size * refactor Sam2 Attention * add fully working video inference (refactoring todo) * clarify _prepare_memory_conditioned_features * simplify modeling code, remove unused paths * use one model * use auto_docstring * refactor rope embeddings * nit * not using multimask when several points given * add all sam2.1 * add video tmp * add Sam2VideoSessionState + fast image proc + video proc * remove init_states from model * fix batch inference * add image integration tests * uniformize modeling code with other sam models and use modular * pass vision tests an most model tests * All tests passing * add offloading inference state and video to cpu * fix inference from image embedding and existing mask * fix multi_boxes mask inference * Fix batch images + batch boxes inference * improve processing for image inference * add support for mask generation pipeline * add support for get_connected_components post processing in mask generation * add fast image processor sam, image processor tests and use modular for sam2 image processor * fix mistake in sam after huggingface#39120 * fix init weights * refactor convert * add integration tests for video + other improvements * add needed missing docstrings * Improve docstrings and * improve inference speed by avoiding cuda sync * add test * skip test for vision_model * minor fix for vision_model * fix vision_model by adding sam2model and change the torch dependencies * remove patch_size * remove image_embedding_size * fix patch_size * fix test * make style * Separate hieradet and vision encoder in sam2 * fixup * review changes part 1 * remove MemoryEncoderConfig and MemoryAttentionConfig * pass q_stride instead of q_pool module * add inference on streamed videos * explicitely process streamed frames * nit * Improve docstrings in Sam2Model * update sam2 modeling with better gestion of inference state and cache, and separate Sam2Model and Sam2VideoModel * improve video inference api * change inference_state to inference_session * use modular for Sam2Model * fix convert sam2 hf * modular * Update src/transformers/models/sam2/video_processing_sam2.py Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com> * fix minor config * fix attention loading error * update modeling tests to use hub checkpoints * Use CI A10 runner for integration tests values + higher tolerance for video integration tests * PR review part 1 * fix doc * nit improvements * enforce one input format for points, labels and boxes * nit * last few nits from PR review * fix style * fix the input type * fix docs * add sam2 model as conversion script * improve sam2 doc * add rough necessarry changes * first working edgetam * fix issue with object pointers * Use modular as much as possible * nit fixes + optimization * refactor spatial perceiver * cleanup after merge * add working edgetam * improve perceiver resampler code * simplify/unify rope attention logic * Improve comments in apply_rotary_pos_emb_2d * add working tests * fix test timmwrapper * add docs * make fixup * nits * fix modular * fix modular * PR review part 1 * split apply_rotary_pos_emb_2d * add granularity to _prepare_memory_conditioned_features * add dates to doc * add separate mlp for memory attention * Fix memory on wrong device * store processed frames in dict * update checkpoints in tests * update dates --------- Co-authored-by: sangbumchoi <danielsejong55@gmail.com> Co-authored-by: RUFFY-369 <prakarshkaushik369@gmail.com> Co-authored-by: Sangbum Daniel Choi <34004152+SangbumChoi@users.noreply.github.com> Co-authored-by: Haitham Khedr <haithamkhedr@meta.com> Co-authored-by: sangbum choi <sangbumchoi@sangbumui-MacBookAir.local> Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>

ArthurZucker added 16 commits June 30, 2025 11:25

just update 2 files

7433c44

update other models as well just making fix-copies

37b4ef0

also add the changes needed to modeling utils

7f113b4

put this on the pretrained model instead

abf9d39

nits and fixes

eb6747b

update generic, fix to use config value

0f1d7e0

update other modelings

e437edd

use transformers kwargs instead

96aabd7

update

63df15b

update

98f402c

update other models

a7e0ce2

update

c9bb39e

updates

cb5da53

update

0dc0826

update

fca73ad

update

98739ba

ArthurZucker added 5 commits June 30, 2025 14:36

fix

124cd82

finally

4a14287

very small nits

ea87eb7

this fixes more tests

8c66f4d

fix other models as well!

3caf7d7

ArthurZucker marked this pull request as ready for review June 30, 2025 12:58

LysandreJik approved these changes Jun 30, 2025

View reviewed changes

ArthurZucker added 5 commits June 30, 2025 15:22

update modularqwen2

113219b

update models based on qwen2

e7705c9

update

a74974d

update

3fb6b71

remove the **flash stuff in favor of noraml kwargs

7266aaf

vasqu approved these changes Jun 30, 2025

View reviewed changes

vasqu mentioned this pull request Aug 22, 2025

Bug Fix: Dynamically set return_lse flag in FlexAttention #40352

Merged

5 tasks

dacorvo mentioned this pull request Aug 25, 2025

Add support for SmolLM3 models huggingface/optimum-neuron#934

Merged

zaristei pushed a commit to zaristei/transformers that referenced this pull request Sep 9, 2025

Fix broken SAM after huggingface#39120 (huggingface#39289)

6dbbcd2

fix

zaristei pushed a commit to zaristei/transformers that referenced this pull request Sep 9, 2025

Fix broken SAM after huggingface#39120 (huggingface#39289)

4c7b307

fix

zaristei pushed a commit to zaristei/transformers that referenced this pull request Sep 9, 2025

Fix broken SAM after huggingface#39120 (huggingface#39289)

6d49005

fix

zaristei pushed a commit to zaristei/transformers that referenced this pull request Sep 9, 2025

Fix broken SAM after huggingface#39120 (huggingface#39289)

08a81d8

fix

zaristei pushed a commit to zaristei/transformers that referenced this pull request Sep 9, 2025

Fix broken SAM after huggingface#39120 (huggingface#39289)

768e250

fix

zaristei pushed a commit to zaristei/transformers that referenced this pull request Sep 9, 2025

Fix broken SAM after huggingface#39120 (huggingface#39289)

a7bf294

fix

zaristei pushed a commit to zaristei/transformers that referenced this pull request Sep 9, 2025

Fix broken SAM after huggingface#39120 (huggingface#39289)

a4cf4f7

fix

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Refactor the way we handle outputs for new llamas and new models #39120

Refactor the way we handle outputs for new llamas and new models #39120

Uh oh!

ArthurZucker commented Jun 30, 2025

Uh oh!

HuggingFaceDocBuilderDev commented Jun 30, 2025

Uh oh!

LysandreJik left a comment

Uh oh!

vasqu left a comment

Uh oh!

vasqu Jun 30, 2025

Uh oh!

vasqu Jun 30, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chenhengqi commented Aug 13, 2025

Uh oh!

ShaohonChen commented Oct 8, 2025

Uh oh!

ArthurZucker commented Oct 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

		cumulative_seqlens_q: Optional["torch.LongTensor"]
		cumulative_seqlens_k: Optional["torch.LongTensor"]

Refactor the way we handle outputs for new llamas and new models #39120

Refactor the way we handle outputs for new llamas and new models #39120

Uh oh!

Conversation

ArthurZucker commented Jun 30, 2025

What does this PR do?

Before submitting

Who can review?

Uh oh!

HuggingFaceDocBuilderDev commented Jun 30, 2025

Uh oh!

LysandreJik left a comment

Choose a reason for hiding this comment

Uh oh!

vasqu left a comment

Choose a reason for hiding this comment

Uh oh!

vasqu Jun 30, 2025

Choose a reason for hiding this comment

Uh oh!

vasqu Jun 30, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chenhengqi commented Aug 13, 2025

Uh oh!

ShaohonChen commented Oct 8, 2025

Uh oh!

ArthurZucker commented Oct 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants