• needanke@feddit.org
    link
    fedilink
    arrow-up
    16
    arrow-down
    1
    ·
    edit-2
    2 days ago

    (I don’t think that was your teachers point at all, but) couldn’t the different formulas have produced different rounding errors due to floating point percision?

      • gandalf_der_12te@discuss.tchncs.de
        link
        fedilink
        arrow-up
        3
        arrow-down
        1
        ·
        2 days ago

        yeah because excel does rounding stuff automatically for you

        try entering 0.1 + 0.2 - 0.1 - 0.2 == 0.0 in any programming language of your choice and see what happens.

    • Blackmist@feddit.uk
      link
      fedilink
      English
      arrow-up
      14
      ·
      2 days ago

      Doubtful, but if anything mine would be more accurate. Fewer calculation steps to lose precision on. I think most spreadsheet software fudges floating point precision anyway. A computer programmer may accept that 0.1+0.2 is not 0.3 but an accountant or mathematician would not be having it.

      I think she was just shit at maths tbh. As a kid you sort of assume all the teachers know more than you about every subject, and that’s not the case at all.