Modbus Spindle Control

gloomyandy

@NineMile How recent is the version of the stm32 source base you are using? I've added some of the low level UART changes that may help the timing issues (in coren2g), but I have not yet merged in the very latest changes in which I think David has adjusted the timing.

NineMile

@gloomyandy ~August 18th afternoon for both RepRapFirmware and CoreN2G - git reports both up to date with origin (your branches).

I've applied David's changes myself to test but it didn't seem to resolve the issue

I still have the oscilloscope hooked up but haven't had a chance to test yet, should be able to get both successful and unsuccessful traces a bit later.

gloomyandy

@NineMile Yep that should have the UART changes present in it. They should allow the packet to be constructed and then sent all in one continuous burst (which may not have been happening without those changes). However I've not been able to test them. I'll be updating things later today with the latest changes (there is a lot happening in 3.6 at the moment so it take s a while to update and test things).

Of course the other thing is that there is no knowing how well these various devices actually follow the standards!

oliof

@dc42 will we get events for ModBus unexpectedly disconnecting to allow for safe disengagement?

dc42

@oliof no you won't, because the only way to tell if a Modbus device is responding is to send a command to write or read a register and check whether the M260.1 or M261.1 command returns success - and you can do that yourself.

When we start using Modbus to implement critical functions without using M260.1 or M261.1 such as changing spindle speed, then we may introduce events to handle failure of those operations.

NineMile

Just wanted to update to say I've been using daemon.g to control my VFD over modbus for a while now and it feels like it's working well. The error messages not being suppressed when writing to a variable is a bit of a pain but aside from that, it feels like a relatively robust way to do it.

For clarity, I still have the spindle configured to use enable / direction / PWM pins but these are not connected. I read the spindle state in daemon.g and send the relevant Modbus requests to start it and control the speed.

I've had a 2 second comms timeout set on the VFD so if there's no Modbus status check within 2 seconds the VFD will stop automatically.

There's a lot of functionality I could add to this to e-stop / pause / whatever if the VFD reports the spindle stopped or at high load or is completely unresponsive, but for the moment I'm pretty happy with the behaviour.

NineMile

So I've been thinking about how spindle control could be integrated directly into RRF in a manner that wouldn't require a HAL to be written in C++ for each type of VFD, and the approach that seems to make sense thus far is to implement a modbus device type that can be assigned to an address on a configured aux port, and then assigned to a spindle of type modbus.

The modbus device type stores a limited number of read and write commands - reads are run on a configurable interval and populate data into the object model, one or more write commands can be configured to act as the actions to take to run a spindle forward, in reverse or to stop it. It would also be possible to run stored commands manually (think of like how M261.1 currently works but rather than specifying all the details you just pick the command number to run).

The nice thing about abstracting the modbus device out is that it could also work to monitor and manage non VFD devices using stored commands.

; Configure Aux 1 to Modbus / UART
M575 P1 B57600 S7

; Create Modbus device 2 at station 80 on Aux 1,
; waiting a minimum of 10ms between requests.
; Some modbus devices have defined processing
; delays between requests that should not be violated
M610 P1 I2 A80 D10

; Assign read commands to device 2. Maximum of 5?
; NOTE: Commands are all named, this is purely for
; readability purposes in the object model.

; Read 5, 16 bit registers from 0x1001 every 500ms.
; These values would be available in the object model.
; P0 is the command index that can be used later
; to refer to this command.
M611.1 I2 P0 S"spindleState" R4097 B5 D500

; Read 1, 16 bit register from 0x1003 every 250ms.
; Multiply the value by a factor of 60 to convert
; this to RPM
M611.1 I2 P1 S"spindleSpeed" R4099 B1 D250 F60

; Read 1, 16 bit register from 0x101B every 500ms.
M611.1 I2 P2 S"spindlePower" R4123 B1 D500

...

; Assign write commands to device 2. Maximum of 5?

; Reset
M611.2 I2 P0 S"reset" R4353 B38550

; Run Forward
M611.2 I2 P1 S"runForward" R4097 B2

; Run Reverse
M611.2 I2 P2 S"runReverse" R4097 B4

; Stop
M611.2 I2 P3 S"stop" R4097 B0

; Emergency Stop
M611.2 I2 P4 S"emergencyStop" R4097 B127

; Set RPM
; This command is special as it does not
; define a B-value - the B argument must
; be given if this command is run manually,
; and would be passed from the spindle control
; code.
; The F argument is a scale factor to adjust
; the input value into the right unit for the
; VFD, in this case from RPM to Hz.
M611.2 I2 P5 S"setRPM" R4098 F0.01667

; Create Spindle 0 as a Modbus spindle,
; using device 2.
M950 P0 T2 I2

; At this point the spindle will exist and RRF
; will start polling the Modbus device for each
; of the read registers defined. It will not
; be controllable because no write commands have
; been assigned to the spindle yet.

; F is the command number(s) to run in order
; to start the spindle. Assign write command
; 1 to start the spindle forwards.
; R is the command number(s) to run in order
; to run the spindle in reverse.
; S is the command number(s) to run in order
; to stop the spindle.
; V is the command number(s) to run in order
; to change the speed of the spindle
M611.3 P0 F1 R2 S3 V5

; For a device that need writes to multiple
; registers to start the spindle, you might
; implement them like the below

; Enable
M611.2 I2 P1 S"enableSpindle" R4097 B1

; Stop
M611.2 I2 P2 S"stop" R4097 B0

; Direction field is part of a bitmask,
; bit 1 set indicates run in reverse

; Forward
M611.2 I2 P3 S"setForward" R4098 B0

; Reverse
M611.2 I2 P4 S"setReverse" R4098 B1

; Configure spindle commands
; For forward, run forward and enable
; For reverse, run reverse and enable
; For stop, run stop etc
M611.3 P0 F3:1 R4:1 S2...

I realise this is somewhat complex from a configuration perspective but it means adding support for any modbus-rtu capable device should be doable in configuration only.

If anyone has any suggestions on this, whether it's too complex or whether there's a way to allow this to support non modbus compliant systems as well (e.g. HuanYang VFDs) then I'd love to hear it.

I'm willing to take a stab at implementing this approach if people think this is reasonable.

gloomyandy

@NineMile Would this work with the VFD discussed over on discord (the HY02D223B I think)?

Personally I'd prefer to extend RRF via external scripts to handle the various spindle operations. I don't think changes in speed or other spindle changes happen so often that doing this would be an issue. So basically have RRF call out to a script (with parameters if needed) for every spindle operation (start, stop, change in speed etc). I'd pair that with adding/extending the existing modbus read/write commands to offer a way to write just a set of bytes with modbus framing and crc. Looking at the LinuxCNC source it seems there are a number of VFDs (like the HY02D223B) which seem to use modbus like communication (including the "normal" modbus crc calculation), but which do not follow the register/coil model of modbus-rtu. So maybe having a way to read/write an arbitrary set of bytes with the required framing and crc would allow scripts to handle the protocol formatting? One advantage of this approach is that in theory you could use the same mechanism to talk to a VFD that does not use modbus style packets at all (though this would require any crc to be calculated by the script).

NineMile

@gloomyandy said in Modbus Spindle Control:

@NineMile Would this work with the VFD discussed over on discord (the HY02D223B I think)?

Personally I'd prefer to extend RRF via external scripts to handle the various spindle operations. I don't think changes in speed or other spindle changes happen so often that doing this would be an issue. So basically have RRF call out to a script (with parameters if needed) for every spindle operation (start, stop, change in speed etc). I'd pair that with adding/extending the existing modbus read/write commands to offer a way to write just a set of bytes with modbus framing and crc. Looking at the LinuxCNC source it seems there are a number of VFDs (like the HY02D223B) which seem to use modbus like communication (including the "normal" modbus crc calculation), but which do not follow the register/coil model of modbus-rtu. So maybe having a way to read/write an arbitrary set of bytes with the required framing and crc would allow scripts to handle the protocol formatting? One advantage of this approach is that in theory you could use the same mechanism to talk to a VFD that does not use modbus style packets at all (though this would require any crc to be calculated by the script).

I like this approach as well, although Jay pointed out that using system macros or whatever would involve SD card reads for every spindle control action.

I did notice when reducing my daemon.g loop delay below about 500ms there were sometimes instances where the machine would halt for a short period between movements, which could be an issue with retrieving information from the VFD on a regular basis.

One of the benefits of integrating the reads at least into the main spinloop would be that we could fire events on a loss of communication.

You're right though, since the HY02D223B doesn't specifically support modbus-rtu then this would be an issue. It might be possible to solve that by allowing the "Modbus device" (bad name for this I know) to use either Modbus or raw UART, with the raw UART setting having options for unCRC'd, CRC16'd etc.

gloomyandy

@NineMile I guess the question is how often do you actually need to issue spindle control commands and how often do you actually need to read data from the spindle? Unless there is some sort of closed loop control going on I would hope the answer would be not very often. Even with the reads being handled by daemon.g I would hope that if there is a loss of communication you could raise an event. if there is some sort of safety issue here that needs a fast response I'm not sure that handling that via modbus is a very good solution. Don't these devices typically have some sort of "alarm" function/signal that can be hooked up to an input pin?

I don't really like the idea of having some sort of "HAL" built into RRF, it makes adding support for new devices much harder and adds extra code (so more flash space and stuff to maintain), that has zero benefit for the majority of RRF users.

An interesting question is that if we went this way should we keep the existing modbus gcode commands (that handle the various register/coil commands) or drop them and simply have a modbus read/write command that does the minimal framing (timing + <packet content> + crc) and leave it up to scripts to construct the "packet content"? At the moment we only support a subset of the modbus-rtu commands listed in this document: https://www.modbus.org/docs/Modbus_Application_Protocol_V1_1b3.pdf so the current implementation may need expanding?

NineMile

@gloomyandy said in Modbus Spindle Control:

would hope that if there is a loss of communication you could raise an event. if there is some sort of safety issue here that needs a fast response I'm not sure that handling that via modbus is a very good solution. Don't these devices typically have some sort of "alarm" function/signal that can be hooked up to an input pin?

There's usually a configuration on the VFD side that can be used to trigger an alarm if there's been no communication in a particular amount of time, with additional configuration to control how the VFD responds to that. For example on my Shihlin SL3 I have it configured to alarm and stop if there's no comms from RRF in 2 seconds.

I think this could be done from the RRF side by tracking the last successful read from the VFD and either pausing or emergency stopping if it's been too long.

@gloomyandy said in Modbus Spindle Control:

An interesting question is that if we went this way should we keep the existing modbus gcode commands (that handle the various register/coil commands) or drop them and simply have a modbus read/write command that does the minimal framing (timing + <packet content> + crc) and leave it up to scripts to construct the "packet content"? At the moment we only support a subset of the modbus-rtu commands listed in this document: https://www.modbus.org/docs/Modbus_Application_Protocol_V1_1b3.pdf so the current implementation may need expanding?

If you discount CRC, the only real difference between the Modbus gcodes and the UART gcodes is that the Modbus ones restrict the functions that can be called. It might make more sense to add a CRC option to the UART commands and let the caller build the packet contents in whatever fashion they want rather than having commands specifically for modbus-rtu.

If we want to make it easier to build particular packet formats then maybe we could just add meta gcode functions instead (we could actually do that for CRC if we don't want to add an option to the UART command itself). Maybe just remove M260.1 and M261.1 entirely and move that functionality into meta gcode functions for packet building / parsing?

Actually the more I think about it the more I think you're right that macros are the right approach. It means people could implement entirely custom spindle control systems, and from the spindle control side it would just need an additional spindle type that calls system macro files if they exist

gloomyandy

@NineMile said in Modbus Spindle Control:

f you discount CRC, the only real difference between the Modbus gcodes and the UART gcodes is that the Modbus ones restrict the functions that can be called. It might make more sense to add a CRC option to the UART commands and let the caller build the packet contents in whatever fashion they want rather than having commands specifically for modbus-rtu.

I don't think that is really the case. If you take a look at the modbus write funtion here: https://github.com/Duet3D/RepRapFirmware/blob/3.6-dev/src/Comms/AuxDevice.cpp#L190

You will see there are extra calls around the actual write operations. My understanding is that these work with the low level UART code (in coreN2G) to ensure that the modbus data is written as a single "burst" of bytes (avoiding possible issues with task switches between writing bytes) and that there is a sufficiently long period of "no data" between packets (in effect modbus uses no data as a packet delimiter). None of that is present in the standard UART write code (this is what my comments about "timing" referred to above). Oh and there is also the handling of the RS485 transceiver read/write pin which is needed if that does not do auto switching. So at the very least we would need to identify if the UART is in "modbus" mode or not. There is then also the handling of "read" operations which are really a write followed by a read, again I think that this has timing considerations that don't really apply to the normal UART read.

So I think there might still be a case for having modbus specific read/write operations but I'd be tempted to try and make them more generic ones that simply read/write a bunch of bytes and apply the timing and crc to them, but leave the formatting and decoding of those bytes up to the user in the form of a script?

That's my take on this, I'd be interested to hear @dc42 take on the matter. To help with that, do you have any feel for how often spindle commands are actually issued and what sort of polling might be needed for status reporting? Can this status polling perhaps be optimised by for instance only doing some of it immediately after issuing a command?

NineMile

@gloomyandy said in Modbus Spindle Control:

I don't think that is really the case. If you take a look at the modbus write funtion here: https://github.com/Duet3D/RepRapFirmware/blob/3.6-dev/src/Comms/AuxDevice.cpp#L190

Ah yep, good point on the timing concerns. So the user on Discord who has used the UART functions to implement the HY style "Modbus" comms might just be getting lucky with timing / silent durations that work for that particular VFD but with no guarantee of working on others.

@gloomyandy said in Modbus Spindle Control:

So I think there might still be a case for having modbus specific read/write operations but I'd be tempted to try and make them more generic ones that simply read/write a bunch of bytes and apply the timing and crc to them, but leave the formatting and decoding of those bytes up to the user in the form of a script?

I think there's definitely scope for this. I wanted to use the 0x08 Modbus function before to autoscan for devices on the bus, but this is not currently possible using the Modbus read / write functions as that function code is not implemented.

Reducing these functions to taking a station address and a data field (everything between address and check fields in the below image) would work well (ignore ASCII mode I wanted to include the headers).

@gloomyandy said in Modbus Spindle Control:

That's my take on this, I'd be interested to hear @dc42 take on the matter. To help with that, do you have any feel for how often spindle commands are actually issued and what sort of polling might be needed for status reporting? Can this status polling perhaps be optimised by for instance only doing some of it immediately after issuing a command?

I would say under normal circumstances, control commands would be executed on a similar cadence to tool changes. For each machining operation in a gcode file, you would usually stop the spindle, run a tool change, set the spindle RPM and then start the spindle.

In my particular use case, I also want to be able to monitor the output speed of the VFD on a regular basis so I can pause or abort the job if the VFD triggers an error - it would be quite important to do that quickly, but I think polling the VFD every half a second or so is enough.

From a control perspective, the worst case scenario I can think of right now is my implementation of variable spindle speed control - it actively changes the rpm of the spindle up and down constantly to avoid resonance frequencies, and right now that happens at the same rate as polling the VFD (every 500ms).

From a start / stop perspective, I think the worst case might be something like the rapidchange toolchanger, which starts and stops the spindle in both directions within a couple of seconds.

So I think at the absolute worst case we're looking at half a second between spindle control commands or status checks.

Edit: I don't think status checks can be optimised to only check after commands. Imagine if you start a spindle and then run a 10 min machining op, and 8 mins in the VFD overheats and shuts down. RRF wouldn't know about that and could potentially keep running the toolpath while the spindle is stopped, which would cause all sorts of issues including potential broken tools. I do think the status checking needs to be running at all times while a spindle is running.

gloomyandy

@NineMile said in Modbus Spindle Control:

Edit: I don't think status checks can be optimised to only check after commands. Imagine if you start a spindle and then run a 10 min machining op, and 8 mins in the VFD overheats and shuts down. RRF wouldn't know about that and could potentially keep running the toolpath while the spindle is stopped, which would cause all sorts of issues including potential broken tools. I do think the status checking needs to be running at all times while a spindle is running.

As I said above I'm not a big fan of using modbus for serious errors, personally I'd say you should be using some sort of trigger from a hardware fault signal for that. But even in this case I suspect that the reality is that a second (or possibly) more would be fast enough to handle that situation.

My thoughts on checking for a response after a command was that perhaps you can use a slower polling rate during "normal operations" and only increase that rate if you need to after say issuing a command. But really that is just detail and probably depends a lot on the actual use case.

I'm curious in the table above, it lists a "start" value, but I don't see any mention of that either in the spec I linked to or the code. Where is that table from?

NineMile

@gloomyandy said in Modbus Spindle Control:

As I said above I'm not a big fan of using modbus for serious errors, personally I'd say you should be using some sort of trigger from a hardware fault signal for that. But even in this case I suspect that the reality is that a second (or possibly) more would be fast enough to handle that situation.

I agree with this to a point, but I'd take a guess that at the moment, the majority of spindles that are being run from RRF will be running without any feedback from the VFD at all - because just getting them connected up and running with direction control is complex enough, let alone working out how to get feedback inputs and writing RRF macros to process them properly.

The best thing about Modbus control is only needing to run 2 wires to the VFD, and getting 2-way communication with minimum complexity on the hardware side.

IMO It's much easier to write spindle control macros for a particular VFD type once that include some level of machine and vfd protection using Modbus feedback than it is to walk people through the hardware and VFD setup to implement hardware-based alarming and failure recovery. Especially with all the different types of VFDs out there that all seem to work slightly differently on the hardware side as well.

@gloomyandy said in Modbus Spindle Control:

I'm curious in the table above, it lists a "start" value, but I don't see any mention of that either in the spec I linked to or the code. Where is that table from?

I mentioned this briefly in the gcode I posted for the other style of control - this is from the VFD manual for the Shihlin SL3 which regardless of modbus-rtu spec, specifies a quiet time of 10ms between requests to give the VFD processor enough time to respond. If you send a request before this 10ms timer is up the request errors out.

Attaching the SL3 manual here because I think it's probably one of the most verbose VFD manuals I've come across that explains the Modbus control approach.

20220609_SL3_User_Manual_V1.03-041.pdf

NineMile

@gloomyandy said in Modbus Spindle Control:

So I think there might still be a case for having modbus specific read/write operations but I'd be tempted to try and make them more generic ones that simply read/write a bunch of bytes and apply the timing and crc to them, but leave the formatting and decoding of those bytes up to the user in the form of a script?

Looking at the Modbus gcodes again this evening (M260.1 and M261.1) and it occurs to me that if the user has the ability to send arbitrary data then there's no distinction between these 2 commands.

It seems like this lower-level "raw" behaviour could be achieved with a single gcode that takes raw bytes and expects a particular number of bytes in response, something like this:

; Purely hypothetical request that expects 3 data bytes returned from the 3 byte data request.
; var.modbusRaw would be a vector of bytes
M261.3 B3 D{10,12,14} V"modbusRaw"

The underlying implementation would send the station address and deal with CRC, but everything to do with the data would be up to the user.

dc42

@NineMile the Modbus protocol specifies what response is expected to each command. RRF checks those responses. We could ads a generic command but as the response to such a command would be undefined, we would not be able to check it.

Appending the CRC would only be appropriate if the non-Modbus device used the same CRC rules as Modbus.

gloomyandy

@dc42 I had a quick look at the linuxCNC HAL code for the VFD that the user over on discord was using. That looks like it uses the same crc16 and seed as modbus but has a different payload. I get the impression from a quick scan of the linuxCNC stuff that this is fairly common, devices which claim to be "modbus" use the modbus signal timing. framing and crc but do not use the modbus register/coil model and associated commands. But I'm certainly no expert on this.

NineMile

@gloomyandy said in Modbus Spindle Control:

@dc42 I had a quick look at the linuxCNC HAL code for the VFD that the user over on discord was using. That looks like it uses the same crc16 and seed as modbus but has a different payload. I get the impression from a quick scan of the linuxCNC stuff that this is fairly common, devices which claim to be "modbus" use the modbus signal timing. framing and crc but do not use the modbus register/coil model and associated commands. But I'm certainly no expert on this.

This is my understanding as well. Theres at least a couple quite common Chinese VFD's which advertise Modbus support but don't follow the modbus-rtu data format - the Huanyang one encountered on Discord seems to be the most common though.

For my own understanding of the RRF codebase, I spent a bit of time looking at how custom spindle control might be implemented using DoFileMacro.

It seems like adjusting spindle state and rpm would probably need to move to some type of reconciliation based system - where direct calls to spindle->SetRpm() and spindle->SetState() would store values that would then be applied later while spinning a GCodeBuffer.

Applying the settings could be simply setting pin output states for a standard spindle config, like what happens now, or calling DoFileMacro to execute a user-provided control system.

However - there's areas in the code where spindle control actions appear to be taken from outside any particular GCodeBuffer (Tool::Activate() for example, called from direct LCD requests). These would not map cleanly onto just switching into a spindle control state in the current GCodeBuffer where the request was made.

My initial thought was to use the Daemon buffer for all spindle control, but a long-running daemon macro could impact spindle control, and vice-versa.

Does it make sense to run the spindle control actions in their own GCodeBuffer instance and synchronise it with the other buffers?

For example - when File buffer calls M3 to start the spindle, we change the pending spindle settings, notify that the spindle needs to be reconciled and then don't allow the File channel to continue until the Spindle buffer reports the settings have been applied?

Any thoughts on this appreciated, or if I'm missing any prior art that might indicate a smarter way to do this?

I'm going to be at SMRRF in a couple weekends so I'd be happy to chat about this in person if you're going to be there (David / Andy)?

gloomyandy

@NineMile I'm also planning to be at SMRRF, probably on the Sunday.