9-factor delta calibration in FW 1.16

[roboduet]

David, number of calibration factors are increasing in latest firmware. I'm not sure that my idea will be the best solution, but it may be reasonable to create bit mask (bit field?) for each calibration factor so users will be able to specify what exactly they want to calibrate? It can also simplify programming (instead of checking number of factors in variable you can just check corresponding bit in bitfield).

For example:
Bit 0 - 0x01 - endstop 1
Bit 1 - 0x02 - endstop 2
Bit 2 - 0x04 - endstop 3
Bit 3 - 0x08 - delta radius
Bit 4 - 0x10 - delta rod length

and so on. So if user want to calibrate just delta radius and delta rod length he can specify 0x18 in calibration macro. This bit arithmetic is complex for users, maybe it's possible to specify string with parameter letters instead of number (for example G30 Pxyzrl where x y z - endstops, r - radius, l - rod length or just Pxyz if user want to calibrate endstops only)

I'm thinking about possibility to play, for example, with bed tilt compensation or delta rod length without changing other parameters using autocalibration. Also it's not possible for current firmware to use autocalibration, for example, for diagonal rod length without affecting X and Y tower position offsets.

[dc42]

Yes that would be a possibility.

Unfortunately, the bed tilt compensation I implemented in 1.16 isn't numerically stable, on my delta at least. I think this is because bed tilt can be compensated fairly well by adjusting the endstops and tower position corrections. But the rods on my delta are still at 30 degrees to the horizontal at the extremities, and on a machine that goes down to 20 degrees it might work better.

[roboduet]

I just wanted to eliminate possible misunderstandings (as you understand English is not native to me) - I want to choose what exactly I need to calibrate by pointing factors I want in G30 parameter.

For example, adjusting bed tilt compensation without affecting endstop positions.

[dc42]

Yes I understood that.

[peridot]

How does bed tilt compensation differ from endstop offsets?

That is, I understand that endstop offsets induce a vertical shift combined with a shear, while bed tilt compensation should presumably induce a rotation (and homed height gets you a vertical shift). So the transformations are different. But their effects on the Z=0 plane are identical, so how can you calibrate both?

If you want to specify which things to calibrate - which is a nice idea - is it really necessary to use bitfields? How about a comma-separated list of keywords? For example: G30 Sstops.radius,rod,angular,tilt

If numerical stability is an issue, there are numerical algorithms that can do something sensible with degenerate parameters. If I understand the code it treats the problem as a linear one defined by the derivatives (Gauss-Newton) and uses Gauss-Jordan to solve the normal equations. Using a singular value decomposition makes Gauss-Newton more stable, or Levenberg-Marquardt addresses the nonlinearities in a way that also stabilizes the linear parts of the problem. I'd be happy to take a swing at implementing one of these if you're interested.

[roboduet]

@peridot:

If you want to specify which things to calibrate - which is a nice idea - is it really necessary to use bitfields? How about a comma-separated list of keywords? For example: G30 Sstops.radius,rod,angular,tilt

Yes, as I said before, bitfields are very complex for users. That's why I suggested to use textual representation. But in slightly shorter form:

@roboduet:

(for example G30 Pxyzrl where x y z - endstops, r - radius, l - rod length or just Pxyz if user want to calibrate endstops only)

Your proposal is even more readable.

[dc42]

@peridot:

How does bed tilt compensation differ from endstop offsets?

That is, I understand that endstop offsets induce a vertical shift combined with a shear, while bed tilt compensation should presumably induce a rotation (and homed height gets you a vertical shift). So the transformations are different. But their effects on the Z=0 plane are identical, so how can you calibrate both?

The effects on the Z=0 plane are not identical. The difference is most pronounced when the effector is near the edge of the bed opposite a tower.

@peridot:

If numerical stability is an issue, there are numerical algorithms that can do something sensible with degenerate parameters. If I understand the code it treats the problem as a linear one defined by the derivatives (Gauss-Newton) and uses Gauss-Jordan to solve the normal equations. Using a singular value decomposition makes Gauss-Newton more stable, or Levenberg-Marquardt addresses the nonlinearities in a way that also stabilizes the linear parts of the problem. I'd be happy to take a swing at implementing one of these if you're interested.

That would be great! I am not a mathematician by training so I am not familiar with alternative techniques. I was planning to replace Gauss-Jordan by QR decomposition when I can get my head round it. I changed the maths from single to double precision in the firmware build for the Duet WiFi, but it doesn't appear to have helped significantly.

[dc42]

@roboduet:

@peridot:

If you want to specify which things to calibrate - which is a nice idea - is it really necessary to use bitfields? How about a comma-separated list of keywords? For example: G30 Sstops.radius,rod,angular,tilt

Yes, as I said before, bitfields are very complex for users. That's why I suggested to use textual representation. But in slightly shorter form:

@roboduet:

(for example G30 Pxyzrl where x y z - endstops, r - radius, l - rod length or just Pxyz if user want to calibrate endstops only)

Your proposal is even more readable.

G30 S0:1:2:6:7 would be compatible with the existing parser.

[peridot]

@dc42:

@peridot:

How does bed tilt compensation differ from endstop offsets?

That is, I understand that endstop offsets induce a vertical shift combined with a shear, while bed tilt compensation should presumably induce a rotation (and homed height gets you a vertical shift). So the transformations are different. But their effects on the Z=0 plane are identical, so how can you calibrate both?

The effects on the Z=0 plane are not identical. The difference is most pronounced when the effector is near the edge of the bed opposite a tower.

Clearly I need to work on my simulator. I still want to understand the impacts of various delta misalignments, and now I have a better sensor, so I should be able to check the simulation against reality. I still think that the calibration parameters are being used to level the print plane by adjusting different parameters than the real misalignment. So additional parameters with an orthogonal effect are helpful.

@dc42:

@peridot:

If numerical stability is an issue, there are numerical algorithms that can do something sensible with degenerate parameters. If I understand the code it treats the problem as a linear one defined by the derivatives (Gauss-Newton) and uses Gauss-Jordan to solve the normal equations. Using a singular value decomposition makes Gauss-Newton more stable, or Levenberg-Marquardt addresses the nonlinearities in a way that also stabilizes the linear parts of the problem. I'd be happy to take a swing at implementing one of these if you're interested.

That would be great! I am not a mathematician by training so I am not familiar with alternative techniques. I was planning to replace Gauss-Jordan by QR decomposition when I can get my head round it. I changed the maths from single to double precision in the firmware build for the Duet WiFi, but it doesn't appear to have helped significantly.

I'll take a look at it. I have a lot of experience with numerical code, but very little with programming in a real-time environment. First step will be setting up a development environment that can compile a new firmware blob.

It might also help to take a look at the derivative matrix and see how it looks numerically. (Basically ask the calibration routines to spit it out and use a real computer to look at the singular value decomposition.)

[dc42]

If you enable Move debug then it spits out the matrices to the USB port at each step, but currently only to 3 decimal places. Here is a typical output:

[[language]]
Height errors:
  0.008 0.141 -0.113 -0.126 -0.017 0.119 -0.195 -0.138 0.006 0.151 0.133 0.025 -0.084 -0.019 -0.032 0.007
 
Derivative matrix:
  0.042   0.035   0.923   0.167   0.069  -0.058   0.000  -0.933
 -0.071   0.260   0.812   0.167  -0.085  -0.402  -0.467  -0.808
 -0.096   0.541   0.555   0.168  -0.001  -0.628  -0.808  -0.467
 -0.071   0.799   0.271   0.173   0.083  -0.490  -0.933   0.000
  0.043   0.913   0.044   0.178  -0.071   0.005  -0.808   0.467
  0.270   0.801  -0.071   0.175  -0.419   0.499  -0.467   0.808
  0.551   0.542  -0.093   0.197  -0.619   0.608   0.000   0.899
  0.790   0.270  -0.060   0.216  -0.462   0.386   0.437   0.757
  0.887   0.053   0.060   0.223  -0.005   0.077   0.749   0.432
  0.786  -0.065   0.279   0.213   0.450  -0.065   0.871   0.000
  0.547  -0.097   0.550   0.192   0.604   0.001   0.773  -0.446
  0.268  -0.075   0.807   0.169   0.411   0.088   0.463  -0.802
  0.176   0.168   0.656   0.457   0.116  -0.111   0.000  -0.467
  0.177   0.644   0.179   0.463  -0.117   0.002  -0.404   0.233
  0.610   0.190   0.200   0.485  -0.002   0.106   0.349   0.201
  0.336   0.327   0.338   0.563   0.000   0.000   0.000  -0.000
 
Normal matrix
  3.346   0.935   0.965   1.464  -0.031   0.982   2.396   1.401   0.007
  0.935   3.449   0.922   1.364  -0.989   0.004  -2.618   1.506   0.243
  0.965   0.922   3.562   1.378   0.973  -0.969  -0.025  -3.032  -0.116
  1.464   1.364   1.378   1.397  -0.018   0.044   0.054   0.063   0.062
 -0.031  -0.989   0.973  -0.018   1.559  -0.780   1.106  -1.956  -0.265
  0.982   0.004  -0.969   0.044  -0.780   1.609   1.163   1.952   0.038
  2.396  -2.618  -0.025   0.054   1.106   1.163   5.219  -0.086  -0.288
  1.401   1.506  -3.032   0.063  -1.956   1.952  -0.086   5.336   0.310
 
Solved matrix
  3.346   0.000   0.000   0.000   0.000   0.000   0.000   0.000  -1.129
  0.000  -3.288   0.000   0.000   0.000   0.000   0.000   0.000   1.662
  0.000   0.000  -3.679   0.000   0.000   0.000   0.000   0.000  -2.934
  0.000   0.000   0.000   0.288   0.000   0.000   0.000   0.000   0.022
  0.000   0.000   0.000   0.000   0.716   0.000   0.000   0.000  -0.032
  0.000   0.000   0.000   0.000   0.000   0.559   0.000   0.000  -0.139
  0.000   0.000   0.000   0.000   0.000   0.000   0.007   0.000  -0.001
  0.000   0.000   0.000   0.000   0.000   0.000   0.000   0.007   0.006
 
Solution:
 -0.337  -0.505   0.797   0.078  -0.045  -0.249  -0.072   0.815
 

This was with a bed that was deliberately tilted by -0.4deg in the X direction. The last two diagonal coefficients in the diagonalised matrix are very small compared to the others, which is what makes me think the matrix is ill-conditioned. Although I don't believe that bed tilt can be perfectly compensated for by adjusting the endstop offsets, I think that a combination of endstop offset adjustment and tower position adjustment can do a quite a good job of compensating for a tilted bed in respect of height errors.

The derivative matrix probably isn't all that accurate because it is calculated by bumping, not analytically.

[peridot]

Aargh! USB. Also three decimal places. Is it feasible to make all that debugging info go to the web interface too?

You're right, though, that the question is: what changes in the measured values cannot be accounted for with all the other parameters, but can be accounted for with tilt parameters? As long as there's something, they do provide assistance, but if most of their effect is soaked up by endstop offsets and tower position adjustments, then you're stuck with a highly covariant set of parameters. Even if the numerics can be made to behave, manual adjustment is going to be annoying. The worst case is if the effect on the z=0 plane is identical but the effect on print geometry is different; then you need both for accurate printing but you can't measure their effects by probing the bed.