Welcome to OnlineConversion.com

Significant digits

Significant digits
by Svet on 02/13/04 at 04:45:45


Firsty - a great site, went into my favorites on first visit. I've got a question for the devs - what is the accuracy (in significant digits) of your conversion factors?

For most purposes 7 dps default (single data type?) you have is great, but for sensitive calculations I need about 15 (double). I've cheated so far by putting in a large value to convert and extracting all the significant digits I need - but that is limited by the conversion factors.

Could you recommend someplace with high-accuracy conversion factors perchance?

Thank you and my apologies if the info exists elsewhere.


Re: Significant digits
by Robert Fogt on 02/13/04 at 19:54:55

Well JavaScript is capable of many more significant digits, but the problem is the rounding errors.

Computers do not work well with Real numbers. So a very good math function library is needed for 7+ significant digit accuracy.

You can find math libraries such as that in C/C++, Pascal, VB, etc, but not in JavaScript.

Lots of times with JavaScript I get results such as 15.9999999999999 instead of 16. So limiting to 7 accurate digits seemed the best idea instead of the alternative.

Though if I do find a decent math function library in JavaScript, rest assured I would snatch that up quick as can be. I too would love to see the conversions into 15+ digits.

Go Back | Archive Index


Did you find us useful?

Please consider supporting the site with a small donation.

click here for more information


BookMark Us

It may come in handy.

Check out our Conversion Software for Windows.

Can't find something?
Try searching.

Are you bored?
Try the Fun Stuff.

Was this site helpful?
Link to Us | Donate