KEMBAR78
Fix Gemma3 and Gemma2 flops computation. by gagika · Pull Request #2009 · AI-Hypercomputer/maxtext · GitHub
Skip to content

Fix Gemma3 and Gemma2 flops computation. #2009

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 23, 2025
Merged

Fix Gemma3 and Gemma2 flops computation. #2009

merged 1 commit into from
Jul 23, 2025

Conversation

@gagika
Copy link
Collaborator

@gagika gagika commented Jul 22, 2025

Description

  • Add correct local attention flops for Gemma3
  • Change both gemma2 and gemma3 to use causal attention flops (total attention flops / 2).

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed.

@gagika gagika force-pushed the gemma3-flops branch 2 times, most recently from a72f72d to 4760e6a Compare July 22, 2025 20:29
@gagika gagika marked this pull request as ready for review July 22, 2025 20:50
gobbleturk
gobbleturk approved these changes Jul 23, 2025
View reviewed changes
Copy link
Collaborator

@gobbleturk gobbleturk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks gagik!

@copybara-service copybara-service bot merged commit 6da8ede into main Jul 23, 2025
20 checks passed
@copybara-service copybara-service bot deleted the gemma3-flops branch July 23, 2025 19:42
This was referenced Aug 1, 2025
Open
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gobbleturk gobbleturk gobbleturk approved these changes

@bvandermoon bvandermoon bvandermoon approved these changes

@khatwanimohit khatwanimohit Awaiting requested review from khatwanimohit khatwanimohit is a code owner

@vipannalla vipannalla Awaiting requested review from vipannalla vipannalla is a code owner

@RissyRan RissyRan Awaiting requested review from RissyRan RissyRan is a code owner

@richjames0 richjames0 Awaiting requested review from richjames0 richjames0 is a code owner

@shralex shralex Awaiting requested review from shralex shralex is a code owner

@yangyuwei yangyuwei Awaiting requested review from yangyuwei

@SurbhiJainUSC SurbhiJainUSC Awaiting requested review from SurbhiJainUSC SurbhiJainUSC is a code owner

@hengtaoguo hengtaoguo Awaiting requested review from hengtaoguo hengtaoguo is a code owner

@A9isha A9isha Awaiting requested review from A9isha A9isha is a code owner

@aireenmei aireenmei Awaiting requested review from aireenmei aireenmei is a code owner

@lukebaumann lukebaumann Awaiting requested review from lukebaumann

@shauryagup shauryagup Awaiting requested review from shauryagup

@RoshaniN RoshaniN Awaiting requested review from RoshaniN

@michelle-yooh michelle-yooh Awaiting requested review from michelle-yooh

Assignees

@gobbleturk gobbleturk

@NuojCheng NuojCheng

Labels

pull ready

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@gagika @gobbleturk @bvandermoon @NuojCheng