MPG Pendant for Duet 2 using PanelDue input.

[CJD]

One of my previous statements was wrong. I initially though that a software reboot following an M112 Emergency Stop did not restore the web interface but in fact it does. It just takes longer than I was expecting. So it does look like for some reason the HTTP protocol is being disabled.

I am just adding checksums to the commands to see if that makes any difference.

Colin.

[CJD]

Adding checksums to the commands made no difference. So I restricted the rate of commands to no more than 10 per second, increasing the movement size as necessary. This seems to have improved the situation. I have not seen any failures yet. Unfortunately the jogging is now a bit jerky as variable size movements coupled with acceleration and deceleration don't produce smooth movement.
It seems to be a usable work-around but still not the smooth motion I would really like.

Colin.

[dc42]

Maybe 10 commands/second is lower than it needs to be?

[CJD]

Yes, it is still not clear what is happening. This is only a temporary work around. For my own peace of mind I would like to find the root cause. More investigation to come when I get the time.

[CJD]

This is a long post but I think I have made some progress.....

After a lot of testing today I think I can see what is going wrong.
Even after reducing the rate of commands sent through the paneldue port I could still reproduce the web interface hangup.

I then enabled debugging of the http and gcodes on the USB port using M111.

When it was working I saw the following typical debug output.

HTTP connection accepted
HTTP req, command words { GET /rr_status HTTP/1.1 }, parameters { type=1 }
Sending JSON reply, length 547
aux: G0 X0.50
aux: G0 X0.50
aux: G0 X1.00

By repeatedly sending commands over the PanelDue port I recorded the following when the web interface crashed.

HTTP connection accepted
HTTP req, command words { GET /rr_status HTTP/1.1 }, parameters { type=2 }
aux: G0 X1.00
aux: G0 X0.50
aux: G0 X1.00
aux: G0 X1.00
aux: G0 X1.00
aux: G0 X1.50
aux: G0 X1.50
aux: G0 X1.50
aux: G0 X1.00
aux: G0 X0.50
Ran out of output buffers at ../src/Networking/HttpResponder.cpp(980)

when I closed the Web browser I saw

HTTP connection accepted
HTTP req, command words { GET /rr_disconnect HTTP/1.1 }, parameters { }
Sending JSON reply, length 9
Ran out of output buffers at ../src/Networking/HttpResponder.cpp(980)
Can't send anymore

So it is definitely an output buffer problem.

I could then see the Web interface trying to reconnect but it was failing because the "Ran out of output buffers" message kept appearing.

It looked like the output buffers were never being reset.

Sending an M122 command from the USB port gave the following.

serial: m122
=== Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 2.05 running on Duet WiFi 1.02 or later
Board ID: 08DGM-9T6BU-FG3SJ-6J1FJ-3SS6K-98Z3H
Used output buffers: 22 of 24 (24 max)
=== RTOS ===
Static ram: 25712
Dynamic ram: 92844 of which 0 recycled
Exception stack ram used: 456
Never used ram: 12060
Tasks: NETWORK(ready,404) HEAT(blocked,1232) MAIN(running,4080) IDLE(ready,160)
Owned mutexes:
=== Platform ===
Last reset 00:11:30 ago, cause: software
Last software reset time unknown, reason: Heat task stuck, spinning module GCodes, available RAM 11428 bytes (slot 2)
Software reset code 0x00a3 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0041f80f BFAR 0xe000ed38 SP 0x200049fc Task 0x4e49414d
Stack: 0042bf17 0042bfda 01000000 00000000 00000000 00000000

The used output buffers line would tend to say that 22 buffers were in use even though the machine was idle.

Following up on this clue I reset and repeated the exercise whilst regularly doing M122 commands.
I could see the used buffer count gradually increasing. So it appears that something in the firmware is consuming buffers and not releasing them. I believe that this is the root cause of the problem. (And maybe also similar problems others have had with output buffers).

By accident I found that if I used my G0 commands to move the machine to its maximum limit the stepper motor stopped moving as expected. If I then continued to send G0 commands the used buffer count increased quite rapidly until the web interface crashed.
The only reason I can see for this is that the machine would have been sending 'Movement out of range' error messages after each command and I was probably sending my G0 commands at the same time.

I have a theory which might explain this behavior:-

If a new G command is received while the firmware is trying to send a response to the previous command I suspect the response could be being abandoned without freeing up the associated output buffer.

I tried to rewrite my Pendant code to wait for an "ok" response after each command to try to avoid this condition but it appears that the PanelDue port does not send responses to anything except an M408 Json status request.

Is there any way to get the PanelDue port to output the 'ok' after each command?

Colin.

[dc42]

You must periodically read the responses using M408, otherwises responses will pile up in the output buffer. But I'll implement a timeout too in the next RRF3 build.

[CJD]

Thanks for that. I hadn’t realised that was necessary, I assumed if I did not need the responses I did no have to read them.

Not much time to devote to this at the moment, hopefully I can get time to look at it soon. When it is working reliably I will publish the design for the pendant on github.

Colin.

[CJD]

Thanks, that fixed my problem.

For anyone else trying to use the PanelDue port for their own hardware the summary is this....

The PanelDue port does not send responses to commands in the same way that the USB port does. Responses are queued up in the output buffer and must be regularly retrieved using the M408 command with a sequence number.

You must issue the M408 command regularly even if you are not using the responses or the firmware will run out of output buffers and begin behaving strangely.

If you are not interested in the responses then just sending
M408 S0 R1 every few seconds and ignoring the response will flush out the buffers.

[CJD]

if anyone is interested in making one the Arduino sketch and schematic for this pendent can be found at
https://github.com/g4eml/JogBox

Colin.

[wesc]

@CJD are you using a COTS pendant or one of your own design?

[CJD]

@wesc

I am using my own design. However the Arduino code could probably be adapted to use any pendant with switches and rotary encoder.

Colin.