Strange numerics

wschadow

I stumbled across this issue

var dnozzle1 = heat.heaters[1].model.deadTime
var dnozzle2 = (10.0* heat.heaters[1].model.deadTime) / 10.0

echo {var.dnozzle1}^"  "^{var.dnozzle2}

Rounding is done in a mysterious way.

o_lampe

@wschadow Do you know for sure, that the true value is 5.7 ? It might just be truncated in a different way when echo prints OM-values directly?

wschadow

@o_lampe here is another example:

var enozzle1 = heat.heaters[1].model.coolingExp
var enozzle2 = (10.0 * heat.heaters[1].model.coolingExp) / 10.0 

echo {var.enozzle1}^"  "^{var.enozzle2}

According to the documentation the default value is 1.35. There is something wrong with the assignment / rounding of variables.

wschadow

To me this looks like a bug in the assignment / conversion / rounding:

var enozzle1 = 1.0*heat.heaters[1].model.coolingExp
var enozzle2 = 1.00000001*heat.heaters[1].model.coolingExp
var enozzle3 = (10.0 * heat.heaters[1].model.coolingExp) / 10.0 

echo {var.enozzle1}^"  "^{var.enozzle2}^"  "^{var.enozzle3}

dc42

@wschadow each object model value has a default number of decimal places, which is the number of decimal places that is provided when you enquire of that value. This is done because if we represented all floating point values to the full 7 decimal places, the volume of text returned by the object model to DWC, PanelDue and other user interfaces would be a lot larger for no gain. Also it would look silly if user messages gave e.g a selected temperature as 205.5000 rather than 205.5.

If you perform maths on object model values then the result is always provided to 7 decimal places.

wschadow

@dc42 ok, thanks for the explanation. Of course it makes to display only the significant digits or a certain number of digits. The problem here is that too few digits are displayed. When I write these variable to a file I get wrong numbers (1.4 will be written). When reading them and assigning them to variables in the object model, they are set to the wrong values.

wschadow

@dc42 This brings me also to another request: https://github.com/Duet3D/PanelDueFirmware/issues/317

The object model seems to change the number of digits for x and y depending on the number it cuts off trailing zeros, which is fine. DWC and Paneldue show for x and y only one digit after the ".". It would be very helpful, e.g. the x y offset calibration of an ODEX machine, to display to digits. There is enough room for that.

wschadow created this issue in Duet3D/PanelDueFirmware

Strange numerics

closed Two digits for the display of x and y coordinates #317