KEMBAR78
Round result of sqrt and BigMath methods by tompng · Pull Request #427 · ruby/bigdecimal · GitHub
Skip to content

Conversation

@tompng
Copy link
Member

@tompng tompng commented Sep 12, 2025

BigMath method no longer returns inaccurate extra digits. Fixes #354

# first 52 digits of sin(1)
BigMath.sin(1, 1000).round(52)
=> 0.8414709848078965066525023216302989996225630607983711e0

# Before
BigMath.sin(1, 20)
=> 0.8414709848078965066525023216302989993032982180559195e0
     ^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
     Required digits(20)  accurate digits    noise digits

# After
BigMath.sin(1, 20)
=> 0.84147098480789650665

Too many precisions may result in slow calculation.

BigDecimal(2).sqrt(50).sqrt(4)
=> 0.1189207115002721066717499970560475915292972092463817413019002224719158698970398979e1 # before
=> 0.1189e1 # after

BigMath.exp was already rounding the result. Now other methods aligns with BigMath.exp.

@tompng tompng force-pushed the omit_extra_precision branch from 70c5313 to 3de9153 Compare September 12, 2025 18:22
BigMath methods no longer returns extra digits which may be inaccurate
@tompng tompng force-pushed the omit_extra_precision branch from 3de9153 to 1cd7277 Compare September 12, 2025 18:30
@tompng tompng merged commit 44a2bb6 into ruby:master Sep 12, 2025
81 checks passed
@tompng tompng deleted the omit_extra_precision branch September 12, 2025 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BigDecimal#sqrt precision too high

1 participant