Skip to main content
Unix & Linux

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

10
  • Nice explanation and solution. But I don't understand what's wrong when you take 0.5973 times 10000. Can you give me more detail? Thank you. Commented Jul 9, 2014 at 23:19
  • 1
    I’m not sure what you’re asking. (1) What’s happening? I’m not sure, exactly. What nobody else has said is that people typically use decimal (base 10) to represent numbers, but computers use binary (base 2). So, when a computer reads a decimal number, it converts it to binary, and then converts back to decimal on output. For example, ½ is 0.5 in decimal, 0.1 in binary; ¾ is 0.75 in decimal, 0.11 in binary. And some simple fractions cannot be represented exactly in decimal; e.g., ⅓ is 0.33333333…. When you truncate to a finite number of digits, you get an approximation (i.e., rounding error). … Commented Jul 9, 2014 at 23:51
  • … Well, the only fractions that can be represented exactly in binary are fractions of the form (some integer) ∕ (a power of 2), so, while a simple fraction like ⅕ can be represented exactly in decimal (as 0.2), it cannot be represented exactly in binary. Now, it happens to work out that some decimal numbers of the form 0.nnnn, when converted to binary at a certain precision (number of bits) and multiplied by 10000 at that same precision, come out to the original nnnn. And some don’t. I can’t explain why. … Commented Jul 9, 2014 at 23:52
  • … But I guess you might also be asking (2) why does it matter? It’s because we’re trying to get the exact results that you want. So if we have the input 0.5973, -0.5971, and -0.0002, and we multiply each (converted to binary) number by 10000, we get 5973.00000000000090949470, -5971, and -2. So the sum is 0.00000000000090949470. And that’s why we need to round the numbers to the nearest integer after we’ve multiplied by 10000. Commented Jul 9, 2014 at 23:53
  • Why does only 0.5973 become 5973.000000 but not also -0.5971 and -0.0002 when we multiply them with 10000? Does that always happen or it's just an example? Also, I think we should round the numbers before we divide them back, not after they are multiplied by 10000 because adding 0.5 many times can be dangerous. Commented Jul 10, 2014 at 0:27