-
-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Using std::setprecision to get the right precision for double to string conversion. #8302
Conversation
@@ -23,7 +27,9 @@ class CastVisitor : public boost::static_visitor<std::string> { | |||
} | |||
|
|||
std::string operator()(const double& d) const { | |||
std::string s{boost::lexical_cast<std::string>(d)}; | |||
std::ostringstream ss; | |||
ss << std::setprecision(std::numeric_limits<double>::digits10 + 1) << d; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it 16 because that's the case for most double
s?
Double values have between 15 and 18 digits of precision, with most double values having at least 16 significant digits. (source)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SQLite supports 15 significant digits. So we set precision to 15 significant digits plus 1 insignificant (estimate). The previous approach was using more digits, leading to bad results in some cases.
This pattern is used in many other places:
https://github.com/search?q=%22std%3A%3Asetprecision%28std%3A%3Anumeric_limits%3Cdouble%3E%3A%3Adigits10+%2B+1%29%22&type=code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, left one question.
Maybe @directionless , could you review this fix? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good enough
Fixes #8301
boost::lexical_cast was not using the right precision for double-to-string conversion.
Switched to using std::ostringstream.