Float numbers inaccurate in maya?

I was running this code and went OoO

import maya.cmds as cmds

rmpv = cmds.shadingNode("remapValue", n = "test", au = True)
cmds.setAttr(rmpv+".inputMin", -11)
cmds.setAttr(rmpv+".inputMax", 11)
cmds.setAttr(rmpv+".outputMin", 0.2)
cmds.setAttr(rmpv+".outputMax", 1.8)

when I realised that maya turned the values of 0.2 and 0.8 into 0.200000003 and 1.7999999523

I was able to make the numbers exact by choosing 0.5 and 1.5 instead. Or 1.25 or 0.125 work as well.

Can someone explain to me why maya does this?

I read something about " Float values in Python are represented as 64-bit double-precision values". What does that mean? I’m just worried now that my number inputs on some nodes are getting sliiiiightly distorted by 0.000000000something. Should I be worried? Does maya handle decimal numbers in weird ways?

Ah yes. Floating point precision. Long story short, decimal numbers will almost always be slightly off, but that’s just how numbers are stored in the computer. It’s not a big deal. Go right ahead and use values like 0.2 or 1.8 in Maya.

The computer stores binary numbers. In binary (base 2) each digit position is worth half of the one to its left. So 1/2 can be represented exactly, so can 1/4, 1/8, 1/16, 1/32 etc… But 1/10 CANNOT be exactly represented in binary for the same reason we can’t exactly represent 1/3 in decimal (base 10). We write 0.33333(repeating)… but at some point we have to decide how many digits (in this case 3’s) we write down. No matter how many 3’s we write, we’re always going to be a bit off.

Python uses 64 bit floating point numbers. That means they decided to write down exactly 64 binary digits (bits) to represent a number. But they use scientific notation. So a number like -3/32 is represented like (-3) * 2 ^ (-5). One of those 64 bits stores that the value is negative. Some of those 64 bits store that -5 part (11 bits), and the rest store the 3 part (52 bits).

The most common place where this really matters is checking if two floating point numbers are equal. In Python, the best practice is to use math.isclose(a, b) instead of a == b
And there are certainly other ways to store decimal numbers, but they’re not as common. You might run into them from time to time, but the float numbers in Python and Maya work like this.

3 Likes

woah, this is going a bit over my head, but it’s good to know anyway! Thank you kindly :slight_smile:

I remember watching a video on this that explained it pretty well a while back. I looked it up the other day but didn’t have a chance to post these. This might help explain how floating points work for you.

2 Likes