Have you performed simple arithmetic operations like 0.1 + 0.2? You might have gotten something strange: 0.1 + 0.2 = 0.30000000000000004.

  • luciole (he/him)@beehaw.org
    link
    fedilink
    arrow-up
    14
    ·
    3 days ago

    Floating-point arithmetic is important to understand at least vaguely since it’s a pretty leaky abstraction. Fortunately, we don’t need a “✨Member-only story” on Medium to get acquainted with the underlying concepts.

  • karlhungus@lemmy.ca
    link
    fedilink
    arrow-up
    11
    arrow-down
    1
    ·
    3 days ago

    Ugh, i thought this was a question, not a link. So i spent time googling for a good tutorial on floats (because I didn’t click the link)…

    Now i hate myself, and this post.

  • RagingHungryPanda@lemm.ee
    link
    fedilink
    arrow-up
    8
    arrow-down
    1
    ·
    3 days ago

    It’s how CPUs do floating point calculations. It’s not just javascript. Long story short, a float is stored in the format of one bit for the +/-, some bits for a base value (mantissa), and some bits for the exponent. As a result, some numbers aren’t quite representable exactly.

  • thingsiplay@beehaw.org
    link
    fedilink
    arrow-up
    1
    ·
    3 days ago

    If you are adding 0.1 + 0.2, then it means you can cut off anything after the first digit (after the dot off course). Because the rest of the 0.1 is only 0 and the rest of 0.2 is 0. That can help with rounding errors on floating point calculations. I don’t program JavaScript, so no idea what the best way to go about it would be.