Duet3D Logo Duet3D
    • Tags
    • Documentation
    • Order
    • Register
    • Login

    Reboots/crashes - RRF ≤3.5.0-rc1

    Scheduled Pinned Locked Moved Unsolved
    Beta Firmware
    13
    137
    9.5k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Exerqtorundefined
      Exerqtor @dc42
      last edited by

      @dc42 said in Reboots/crashes - RRF 3.5.0-rc1:

      @Exerqtor please try the firmware at https://www.dropbox.com/scl/fo/1r49rb2x3jsn1az0jdu1j/h?rlkey=xke2yefix0hyp5dioxa58pgtp&dl=0 and post a M122 after any crash.

      Hopefully this shines some more light on it:

      M122
      === Diagnostics ===
      RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-rc.1+ (2023-12-04 12:12:17) running on Duet 3 Mini5plus WiFi (standalone mode)
      Board ID: XNHXF-HR6KL-K65J0-409N2-K9W1Z-RV2MZ
      Used output buffers: 10 of 40 (40 max)
      === RTOS ===
      Static ram: 102844
      Dynamic ram: 123360 of which 0 recycled
      Never used RAM 12356, free system stack 188 words
      Tasks: NETWORK(1,ready,18.8%,239) HEAT(3,nWait,0.0%,336) Move(4,nWait,0.0%,343) CanReceiv(6,nWait,0.0%,797) CanSender(5,nWait,0.0%,336) CanClock(7,delaying,0.0%,350) TMC(4,nWait,0.7%,114) MAIN(1,running,79.3%,670) IDLE(0,ready,0.3%,29) AIN(4,delaying,0.8%,264), total 100.0%
      Owned mutexes:
      === Platform ===
      Last reset 00:26:31 ago, cause: software
      Last software reset at 2023-12-05 09:59, reason: HardFault imprec ibus, Gcodes spinning, available RAM 10968, slot 1
      Software reset code 0x4063 HFSR 0x40000000 CFSR 0x00000500 ICSR 0x00000803 BFAR 0xe000ed38 SP 0x20011f48 Task NETW Freestk 466 ok
      Stack: 0d362e30 20032dec 000001bf 00000001 0d362e30 20032dec 000043ae 010f0200 00000001 200117a4 2002bed0 000001bf 000001bf 00000004 0002dd0f 2002bed4 2002be01 000001bf 2002c138 20031db0 2002c150 2001e888 a5a5a5a5 a5a5a5a5 0002ff15 00000000 00000000
      Error status: 0x04
      Aux0 errors 0,0,0
      MCU revision 3, ADC conversions started 1591912, completed 1591912, timed out 0, errs 0
      MCU temperature: min 33.2, current 34.0, max 36.2
      Supply voltage: min 22.0, current 24.1, max 27.0, under voltage events: 0, over voltage events: 0, power good: yes
      Heap OK, handles allocated/used 99/33, heap memory allocated/used/recyclable 2048/1172/736, gc cycles 74
      Events: 0 queued, 0 completed
      Driver 0: standstill, SG min 0, read errors 0, write errors 1, ifcnt 227, reads 18224, writes 13, timeouts 0, DMA errors 0, CC errors 0
      Driver 1: standstill, SG min 16, read errors 0, write errors 1, ifcnt 227, reads 18224, writes 13, timeouts 0, DMA errors 0, CC errors 0
      Driver 2: standstill, SG min 2, read errors 0, write errors 1, ifcnt 150, reads 18224, writes 13, timeouts 0, DMA errors 0, CC errors 0
      Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 149, reads 18223, writes 13, timeouts 0, DMA errors 0, CC errors 0
      Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 149, reads 18224, writes 13, timeouts 0, DMA errors 0, CC errors 0
      Driver 5: not present
      Driver 6: not present
      Date/time: 2023-12-05 10:26:25
      Cache data hit count 2710632535
      Slowest loop: 9.77ms; fastest: 0.13ms
      === Storage ===
      Free file entries: 18
      SD card 0 detected, interface speed: 22.5MBytes/sec
      SD card longest read time 7.1ms, write time 4.5ms, max retries 0
      === Move ===
      DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, ebfmin 0.00, ebfmax 0.00
      no step interrupt scheduled
      Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
      === DDARing 0 ===
      Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
      === DDARing 1 ===
      Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
      === Heat ===
      Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
      Heater 1 is on, I-accum = 0.0
      === GCodes ===
      Movement locks held by null, null
      HTTP is idle in state(s) 0
      Telnet is idle in state(s) 0
      File is idle in state(s) 0
      USB is idle in state(s) 0
      Aux is idle in state(s) 0
      Trigger is idle in state(s) 0
      Queue is idle in state(s) 0
      LCD is idle in state(s) 0
      SBC is idle in state(s) 0
      Daemon is doing "G4 S1" in state(s) 0 0, running macro
      Aux2 is idle in state(s) 0
      Autopause is idle in state(s) 0
      File2 is idle in state(s) 0
      Queue2 is idle in state(s) 0
      Q0 segments left 0, axes/extruders owned 0x0000803
      Code queue 0 is empty
      Q1 segments left 0, axes/extruders owned 0x0000000
      Code queue 1 is empty
      === Filament sensors ===
      in 0 notIn 0
      Extruder 0 sensor: no filament
      === CAN ===
      Messages queued 14337, received 32674, lost 0, errs 1, boc 0
      Longest wait 2ms for reply type 6053, peak Tx sync delay 260, free buffers 26 (min 25), ts 7960/7959/0
      Tx timeouts 0,0,0,0,0,0
      === Network ===
      Slowest loop: 7.91ms; fastest: 0.00ms
      Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
      HTTP sessions: 3 of 8
      === WiFi ===
      Interface state: active
      Module is connected to access point 
      Failed messages: pending 0, notrdy 0, noresp 0
      Firmware version 2.1beta4
      MAC address c4:5b:be:ce:91:93
      Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 42972
      WiFi IP address 192.168.10.x
      Signal strength -49dBm, channel 6, mode 802.11n, reconnections 0
      Clock register 00002001
      Socket states: 0 0 0 0 0 0 0 0
      
      dc42undefined 1 Reply Last reply Reply Quote 0
      • dc42undefined
        dc42 administrators @Exerqtor
        last edited by

        @Exerqtor interesting, that one is a different sort of reset but in the same function. Please send M122 P500 S0, then wait for it to reset again and send the M122 report as usual.

        Duet WiFi hardware designer and firmware engineer
        Please do not ask me for Duet support via PM or email, use the forum
        http://www.escher3d.com, https://miscsolutions.wordpress.com

        Exerqtorundefined 1 Reply Last reply Reply Quote 1
        • Exerqtorundefined
          Exerqtor @dc42
          last edited by

          @dc42 said in Reboots/crashes - RRF 3.5.0-rc1:

          @Exerqtor interesting, that one is a different sort of reset but in the same function. Please send M122 P500 S0, then wait for it to reset again and send the M122 report as usual.

          Here you go ( i did a print or two after sending M122 P500 S0, hope that don't mess anything up):

          === Diagnostics ===
          RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-rc.1+ (2023-12-04 12:12:17) running on Duet 3 Mini5plus WiFi (standalone mode)
          Board ID: XNHXF-HR6KL-K65J0-409N2-K9W1Z-RV2MZ
          Used output buffers: 6 of 40 (40 max)
          === RTOS ===
          Static ram: 102844
          Dynamic ram: 123360 of which 0 recycled
          Never used RAM 12356, free system stack 196 words
          Tasks: NETWORK(1,ready,19.1%,239) HEAT(3,nWait,0.0%,353) Move(4,nWait,0.0%,357) CanReceiv(6,nWait,0.0%,797) CanSender(5,nWait,0.0%,336) CanClock(7,delaying,0.0%,350) TMC(4,nWait,0.7%,108) MAIN(1,running,79.0%,670) IDLE(0,ready,0.3%,29) AIN(4,delaying,0.8%,264), total 100.0%
          Owned mutexes: WiFi(NETWORK)
          === Platform ===
          Last reset 01:09:12 ago, cause: software
          Last software reset at 2023-12-06 08:16, reason: AssertionFailed, Gcodes spinning, available RAM 5824, slot 2
          Software reset code 0x4123 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00000000 BFAR 0xe000ed38 SP 0x20011fbc Task NETW Freestk 495 ok
          Stack: 0000090f 000af3c8 0002de29 2002bed4 2002be01 000001b0 2002c100 20031550 2002c118 2001e888 a5a5a5a5 a5a5a5a5 312e303d 00000000 00000000 00000000 2003155c 00000800 20035a68 2002c100 20018678 2002bf9d 20018678 2001e888 000300af 00000000 00000000
          Error status: 0x04
          Aux0 errors 0,0,0
          MCU revision 3, ADC conversions started 4152839, completed 4152837, timed out 0, errs 0
          MCU temperature: min 33.8, current 34.0, max 36.9
          Supply voltage: min 22.5, current 24.1, max 26.5, under voltage events: 0, over voltage events: 0, power good: yes
          Heap OK, handles allocated/used 99/33, heap memory allocated/used/recyclable 2048/1248/812, gc cycles 193
          Events: 0 queued, 0 completed
          Driver 0: standstill, SG min 16, read errors 0, write errors 1, ifcnt 89, reads 21918, writes 13, timeouts 0, DMA errors 0, CC errors 0
          Driver 1: standstill, SG min 248, read errors 0, write errors 1, ifcnt 88, reads 21918, writes 13, timeouts 0, DMA errors 0, CC errors 0
          Driver 2: standstill, SG min 2, read errors 0, write errors 1, ifcnt 52, reads 21917, writes 13, timeouts 0, DMA errors 0, CC errors 0
          Driver 3: standstill, SG min 2, read errors 0, write errors 1, ifcnt 51, reads 21917, writes 13, timeouts 0, DMA errors 0, CC errors 0
          Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 50, reads 21918, writes 13, timeouts 0, DMA errors 0, CC errors 0
          Driver 5: not present
          Driver 6: not present
          Date/time: 2023-12-06 09:25:54
          Cache data hit count 4294967295
          Slowest loop: 12.98ms; fastest: 0.13ms
          === Storage ===
          Free file entries: 18
          SD card 0 detected, interface speed: 22.5MBytes/sec
          SD card longest read time 6.9ms, write time 4.8ms, max retries 0
          === Move ===
          DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, ebfmin 0.00, ebfmax 0.00
          no step interrupt scheduled
          Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
          === DDARing 0 ===
          Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
          === DDARing 1 ===
          Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
          === Heat ===
          Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
          Heater 1 is on, I-accum = 0.0
          === GCodes ===
          Movement locks held by null, null
          HTTP is idle in state(s) 0
          Telnet is idle in state(s) 0
          File is idle in state(s) 0
          USB is idle in state(s) 0
          Aux is idle in state(s) 0
          Trigger is idle in state(s) 0
          Queue is idle in state(s) 0
          LCD is idle in state(s) 0
          SBC is idle in state(s) 0
          Daemon is doing "G4 S1" in state(s) 0 0, running macro
          Aux2 is idle in state(s) 0
          Autopause is idle in state(s) 0
          File2 is idle in state(s) 0
          Queue2 is idle in state(s) 0
          Q0 segments left 0, axes/extruders owned 0x0000803
          Code queue 0 is empty
          Q1 segments left 0, axes/extruders owned 0x0000000
          Code queue 1 is empty
          === Filament sensors ===
          in 0 notIn 0
          Extruder 0 sensor: no filament
          === CAN ===
          Messages queued 37382, received 85164, lost 0, errs 1, boc 0
          Longest wait 2ms for reply type 6053, peak Tx sync delay 270, free buffers 26 (min 25), ts 20763/20762/0
          Tx timeouts 0,0,0,0,0,0
          === Network ===
          Slowest loop: 5.88ms; fastest: 0.00ms
          Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
          HTTP sessions: 2 of 8
          === WiFi ===
          Interface state: active
          Module is connected to access point 
          Failed messages: pending 0, notrdy 0, noresp 0
          Firmware version 2.1beta4
          MAC address c4:5b:be:ce:91:93
          Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 42968
          WiFi IP address 192.168.10.x
          Signal strength -49dBm, channel 6, mode 802.11n, reconnections 0
          Clock register 00002001
          Socket states: 0 0 0 0 0 0 0 0
          
          dc42undefined 1 Reply Last reply Reply Quote 0
          • dc42undefined
            dc42 administrators @Exerqtor
            last edited by

            @Exerqtor thanks, that's back to normal. Please install the firmware from https://www.dropbox.com/scl/fo/pj3pphcoi373dtlcbsrao/h?rlkey=w3hwi2n2l51bywl0773j9uo6a&dl=0 (ignore all the other files there, you only need the .uf2 file) and again send M122 P500 S0 after it starts up. That command will persist until the next reset, so no problem if you do several prints.

            Duet WiFi hardware designer and firmware engineer
            Please do not ask me for Duet support via PM or email, use the forum
            http://www.escher3d.com, https://miscsolutions.wordpress.com

            Exerqtorundefined 2 Replies Last reply Reply Quote 0
            • Exerqtorundefined
              Exerqtor @dc42
              last edited by

              @dc42
              "Normal" lol. Ok i'll install it once the current print finishes, and add M122 P500 S0 at the bottom of config.g.

              1 Reply Last reply Reply Quote 0
              • Exerqtorundefined
                Exerqtor @dc42
                last edited by Exerqtor

                @dc42 said in Reboots/crashes - RRF 3.5.0-rc1:

                @Exerqtor thanks, that's back to normal. Please install the firmware from https://www.dropbox.com/scl/fo/pj3pphcoi373dtlcbsrao/h?rlkey=w3hwi2n2l51bywl0773j9uo6a&dl=0 (ignore all the other files there, you only need the .uf2 file) and again send M122 P500 S0 after it starts up. That command will persist until the next reset, so no problem if you do several prints.

                Here we have one:

                M122
                === Diagnostics ===
                RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-rc.1+ (2023-12-06 12:09:26) running on Duet 3 Mini5plus WiFi (standalone mode)
                Board ID: XNHXF-HR6KL-K65J0-409N2-K9W1Z-RV2MZ
                Used output buffers: 1 of 40 (40 max)
                === RTOS ===
                Static ram: 102844
                Dynamic ram: 123360 of which 0 recycled
                Never used RAM 12356, free system stack 194 words
                Tasks: NETWORK(2,nWait,18.3%,228) HEAT(3,nWait,0.0%,355) Move(4,nWait,0.0%,357) CanReceiv(6,nWait,0.1%,797) CanSender(5,nWait,0.0%,336) CanClock(7,delaying,0.0%,350) TMC(4,nWait,0.7%,108) MAIN(1,running,79.8%,670) IDLE(0,ready,0.3%,29) AIN(4,delaying,0.9%,264), total 100.0%
                Owned mutexes: WiFi(NETWORK)
                === Platform ===
                Last reset 00:05:36 ago, cause: software
                Last software reset at 2023-12-07 02:46, reason: AssertionFailed, Gcodes spinning, available RAM 5440, slot 0
                Software reset code 0x4123 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00000000 BFAR 0xe000ed38 SP 0x20011fbc Task NETW Freestk 495 ok
                Stack: 00000913 000af408 0002de53 2002bed4 2002be01 000001b0 2002c100 20031598 2002c118 2001e888 a5a5a5a5 a5a5a5a5 312e303d 00000000 00000000 00000000 200315a4 00000800 20035ab0 2002c100 20018678 2002bf9d 20018678 2001e888 000300db 00000000 00000000
                Error status: 0x04
                Aux0 errors 0,0,0
                MCU revision 3, ADC conversions started 336854, completed 336854, timed out 0, errs 0
                MCU temperature: min 35.1, current 35.7, max 38.4
                Supply voltage: min 22.4, current 24.1, max 26.6, under voltage events: 0, over voltage events: 0, power good: yes
                Heap OK, handles allocated/used 99/33, heap memory allocated/used/recyclable 2048/640/204, gc cycles 16
                Events: 0 queued, 0 completed
                Driver 0: standstill, SG min 0, read errors 0, write errors 1, ifcnt 207, reads 17692, writes 13, timeouts 0, DMA errors 0, CC errors 0
                Driver 1: standstill, SG min 0, read errors 0, write errors 1, ifcnt 206, reads 17691, writes 13, timeouts 0, DMA errors 0, CC errors 0
                Driver 2: standstill, SG min 0, read errors 0, write errors 1, ifcnt 111, reads 17691, writes 13, timeouts 0, DMA errors 0, CC errors 0
                Driver 3: standstill, SG min 2, read errors 0, write errors 1, ifcnt 112, reads 17691, writes 13, timeouts 0, DMA errors 0, CC errors 0
                Driver 4: standstill, SG min 2, read errors 0, write errors 1, ifcnt 109, reads 17692, writes 13, timeouts 0, DMA errors 0, CC errors 0
                Driver 5: not present
                Driver 6: not present
                Date/time: 2023-12-07 02:51:52
                Cache data hit count 560391083
                Slowest loop: 12.95ms; fastest: 0.14ms
                === Storage ===
                Free file entries: 18
                SD card 0 detected, interface speed: 22.5MBytes/sec
                SD card longest read time 6.9ms, write time 4.6ms, max retries 0
                === Move ===
                DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, ebfmin 0.00, ebfmax 0.00
                no step interrupt scheduled
                Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
                === DDARing 0 ===
                Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
                === DDARing 1 ===
                Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
                === Heat ===
                Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
                Heater 1 is on, I-accum = 0.0
                === GCodes ===
                Movement locks held by null, null
                HTTP is idle in state(s) 0
                Telnet is idle in state(s) 0
                File is idle in state(s) 0
                USB is idle in state(s) 0
                Aux is idle in state(s) 0
                Trigger is idle in state(s) 0
                Queue is idle in state(s) 0
                LCD is idle in state(s) 0
                SBC is idle in state(s) 0
                Daemon is doing "G4 S1" in state(s) 0 0, running macro
                Aux2 is idle in state(s) 0
                Autopause is idle in state(s) 0
                File2 is idle in state(s) 0
                Queue2 is idle in state(s) 0
                Q0 segments left 0, axes/extruders owned 0x0000803
                Code queue 0 is empty
                Q1 segments left 0, axes/extruders owned 0x0000000
                Code queue 1 is empty
                === Filament sensors ===
                in 0 notIn 0
                Extruder 0 sensor: no filament
                === CAN ===
                Messages queued 3040, received 6936, lost 0, errs 1, boc 0
                Longest wait 2ms for reply type 6031, peak Tx sync delay 253, free buffers 26 (min 25), ts 1684/1683/0
                Tx timeouts 0,0,0,0,0,0
                === Network ===
                Slowest loop: 8.87ms; fastest: 0.00ms
                Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
                HTTP sessions: 3 of 8
                === WiFi ===
                Interface state: active
                Module is connected to access point 
                Failed messages: pending 0, notrdy 0, noresp 0
                Firmware version 2.1beta4
                MAC address c4:5b:be:ce:91:93
                Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 42948
                WiFi IP address 192.168.10.x
                Signal strength -46dBm, channel 6, mode 802.11n, reconnections 0
                Clock register 00002001
                Socket states: 0 0 0 0 0 0 0 0
                
                Exerqtorundefined 1 Reply Last reply Reply Quote 0
                • Exerqtorundefined
                  Exerqtor @Exerqtor
                  last edited by

                  And another:

                  M122
                  === Diagnostics ===
                  RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-rc.1+ (2023-12-06 12:09:26) running on Duet 3 Mini5plus WiFi (standalone mode)
                  Board ID: XNHXF-HR6KL-K65J0-409N2-K9W1Z-RV2MZ
                  Used output buffers: 1 of 40 (40 max)
                  === RTOS ===
                  Static ram: 102844
                  Dynamic ram: 122128 of which 0 recycled
                  Never used RAM 13588, free system stack 198 words
                  Tasks: NETWORK(1,ready,18.3%,203) HEAT(3,nWait,0.0%,353) Move(4,nWait,0.0%,357) CanReceiv(6,nWait,0.1%,797) CanSender(5,nWait,0.0%,336) CanClock(7,delaying,0.0%,350) TMC(4,nWait,0.7%,116) MAIN(1,running,79.7%,670) IDLE(0,ready,0.4%,29) AIN(4,delaying,0.8%,264), total 100.0%
                  Owned mutexes: WiFi(NETWORK)
                  === Platform ===
                  Last reset 00:03:15 ago, cause: software
                  Last software reset at 2023-12-07 06:59, reason: AssertionFailed, Gcodes spinning, available RAM 13540, slot 0
                  Software reset code 0x4123 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00000000 BFAR 0xe000ed38 SP 0x20011fbc Task NETW Freestk 495 ok
                  Stack: 00000913 000af408 0002de53 2002bed4 2002be01 000001a8 2002c100 20031d28 2002c118 2001e888 a5a5a5a5 a5a5a5a5 362e303d 00000000 00000000 00000000 20031d34 00000800 20035a28 2002c100 20018678 2002bf9d 20018678 2001e888 000300db 00000000 00000000
                  Error status: 0x04
                  Aux0 errors 0,0,0
                  MCU revision 3, ADC conversions started 195496, completed 195496, timed out 0, errs 0
                  MCU temperature: min 34.2, current 34.8, max 37.3
                  Supply voltage: min 22.6, current 24.1, max 26.4, under voltage events: 0, over voltage events: 0, power good: yes
                  Heap OK, handles allocated/used 99/29, heap memory allocated/used/recyclable 2048/1524/1128, gc cycles 8
                  Events: 0 queued, 0 completed
                  Driver 0: standstill, SG min 0, read errors 0, write errors 1, ifcnt 249, reads 10257, writes 13, timeouts 0, DMA errors 0, CC errors 0
                  Driver 1: standstill, SG min 0, read errors 0, write errors 1, ifcnt 248, reads 10257, writes 13, timeouts 0, DMA errors 0, CC errors 0
                  Driver 2: standstill, SG min 0, read errors 0, write errors 1, ifcnt 153, reads 10257, writes 13, timeouts 0, DMA errors 0, CC errors 0
                  Driver 3: standstill, SG min 2, read errors 0, write errors 1, ifcnt 154, reads 10256, writes 13, timeouts 0, DMA errors 0, CC errors 0
                  Driver 4: standstill, SG min 2, read errors 0, write errors 1, ifcnt 151, reads 10257, writes 13, timeouts 0, DMA errors 0, CC errors 0
                  Driver 5: not present
                  Driver 6: not present
                  Date/time: 2023-12-07 07:03:01
                  Cache data hit count 327929753
                  Slowest loop: 8.69ms; fastest: 0.14ms
                  === Storage ===
                  Free file entries: 18
                  SD card 0 detected, interface speed: 22.5MBytes/sec
                  SD card longest read time 3.8ms, write time 3.7ms, max retries 0
                  === Move ===
                  DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, ebfmin 0.00, ebfmax 0.00
                  no step interrupt scheduled
                  Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
                  === DDARing 0 ===
                  Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
                  === DDARing 1 ===
                  Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
                  === Heat ===
                  Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
                  Heater 1 is on, I-accum = 0.0
                  === GCodes ===
                  Movement locks held by null, null
                  HTTP is idle in state(s) 0
                  Telnet is idle in state(s) 0
                  File is idle in state(s) 0
                  USB is idle in state(s) 0
                  Aux is idle in state(s) 0
                  Trigger is idle in state(s) 0
                  Queue is idle in state(s) 0
                  LCD is idle in state(s) 0
                  SBC is idle in state(s) 0
                  Daemon is doing "G4 S1" in state(s) 0 0, running macro
                  Aux2 is idle in state(s) 0
                  Autopause is idle in state(s) 0
                  File2 is idle in state(s) 0
                  Queue2 is idle in state(s) 0
                  Q0 segments left 0, axes/extruders owned 0x0000803
                  Code queue 0 is empty
                  Q1 segments left 0, axes/extruders owned 0x0000000
                  Code queue 1 is empty
                  === Filament sensors ===
                  in 0 notIn 0
                  Extruder 0 sensor: no filament
                  === CAN ===
                  Messages queued 1768, received 4041, lost 0, errs 1, boc 0
                  Longest wait 2ms for reply type 6053, peak Tx sync delay 345, free buffers 26 (min 25), ts 977/976/0
                  Tx timeouts 0,0,0,0,0,0
                  === Network ===
                  Slowest loop: 6.88ms; fastest: 0.00ms
                  Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
                  HTTP sessions: 3 of 8
                  === WiFi ===
                  Interface state: active
                  Module is connected to access point 
                  Failed messages: pending 0, notrdy 0, noresp 0
                  Firmware version 2.1beta4
                  MAC address c4:5b:be:ce:91:93
                  Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 42948
                  WiFi IP address 192.168.10.x
                  Signal strength -46dBm, channel 6, mode 802.11n, reconnections 0
                  Clock register 00002001
                  Socket states: 0 0 0 0 0 0 0 0 
                  
                  dc42undefined 2 Replies Last reply Reply Quote 0
                  • dc42undefined
                    dc42 administrators @Exerqtor
                    last edited by dc42

                    @Exerqtor thanks for those. I'll have another firmware build for you to test shortly.

                    The corrupt data appears to be 4 characters of ASCII text. In two cases it is =0.1 and in one case =0.6 . Does that text appear in the file you printed, or in any of your macro files that the print may be invoking?

                    Duet WiFi hardware designer and firmware engineer
                    Please do not ask me for Duet support via PM or email, use the forum
                    http://www.escher3d.com, https://miscsolutions.wordpress.com

                    Exerqtorundefined 1 Reply Last reply Reply Quote 0
                    • dc42undefined
                      dc42 administrators @Exerqtor
                      last edited by

                      @Exerqtor please try the firmware at https://www.dropbox.com/scl/fo/y34newtid8z5zp7qa5ted/h?rlkey=i65uvg992k74vrgygg4rp4g98&dl=0.

                      Duet WiFi hardware designer and firmware engineer
                      Please do not ask me for Duet support via PM or email, use the forum
                      http://www.escher3d.com, https://miscsolutions.wordpress.com

                      Exerqtorundefined 2 Replies Last reply Reply Quote 0
                      • Exerqtorundefined
                        Exerqtor @dc42
                        last edited by Exerqtor

                        @dc42 said in Reboots/crashes - RRF 3.5.0-rc1:

                        @Exerqtor thanks for those. I'll have another firmware build for you to test shortly.

                        The corrupt data appears to be 4 characters of ASCII text. In two cases it is =0.1 and in one case =0.6 . Does that text appear in the file you printed, or in any of your macro files that the print may be invoking?

                        Sounds like progress 😁

                        But no, all the reports I've posted since we uncovered the crashes being CPU-WiFi transfere related I've turned off the WiFi module when I print. So all the crashes has happened with the printer in an idle state (unless otherwise mentioned).

                        @dc42 said in Reboots/crashes - RRF 3.5.0-rc1:

                        @Exerqtor please try the firmware at https://www.dropbox.com/scl/fo/y34newtid8z5zp7qa5ted/h?rlkey=i65uvg992k74vrgygg4rp4g98&dl=0.

                        Ok, I'll download and install it straight away!

                        Exerqtorundefined 1 Reply Last reply Reply Quote 0
                        • Exerqtorundefined
                          Exerqtor @Exerqtor
                          last edited by Exerqtor

                          @Exerqtor said in Reboots/crashes - RRF 3.5.0-rc1:

                          @dc42 said in Reboots/crashes - RRF 3.5.0-rc1:

                          @Exerqtor thanks for those. I'll have another firmware build for you to test shortly.

                          The corrupt data appears to be 4 characters of ASCII text. In two cases it is =0.1 and in one case =0.6 . Does that text appear in the file you printed, or in any of your macro files that the print may be invoking?

                          Sounds like progress 😁

                          But no, all the reports I've posted since we uncovered the crashes being CPU-WiFi transfere related I've turned off the WiFi module when I print. So all the crashes has happened with the printer in an idle state (unless otherwise mentioned).

                          I guess what daemon.g is doing still applies while the printer is idle, so here is how that looks as of now (just grabbed it straight of the printer):

                          ; /sys/daemon.g  v2.4
                          ; Used to execute regular tasks, the firmware executes it and once the end of file is reached it waits. If the file is not found it waits and then looks for it again.
                          
                          ;---/
                          ; -/--/--/--/--/--/--/--/--/--/--/--/--/--/--/--/--
                          ; THIS MACRO ONLY WORKS WITH RRF 3.5.0b1 AND LATER!!
                          ;--/--/--/--/--/--/--/--/--/--/--/--/--/--/--/--/--
                          ;-/
                          
                          ; Loop, to be able to turn on/off daemon.g
                          while global.RunDaemon
                            ; Stuff goes here         
                          
                            ; Resume after filament change
                            if exists(global.FilamentCHG)
                              if global.FilamentCHG = true && state.status="paused"
                                M24                                                                      ; Resume the pause automatically now that the manual filament change is done
                                set global.FilamentCHG = false
                          
                            ; Refresh chamber lights status
                            if exists(global.chamber_leds)
                              set global.chamber_leds = state.gpOut[0].pwm * 100
                          
                            ;Check sb_leds status
                            var SB_LEDS = true                                                           ; Turn on(true) / off(false) the Stealthburner led "system"                                                          
                            if var.SB_LEDS
                              if fileexists("/sys/lib/led/sb_leds-state.g")
                                M98 P"/sys/lib/led/sb_leds-state.g"                                      ; Check if global.sb_leds has changed since last run/loop
                          
                            ; Print acceleration & input shaping control
                            if global.accel_control
                              
                              ; Low accel on the whole first layer so that it gets good adhesion
                              if global.layer_number = 1
                                M204 P500                                                                ; Set printing acceleration(mm/s�)
                              
                              ; Everything else
                              else
                                if global.line_type = "External perimeter"
                                  if move.printingAcceleration != global.low_accel
                                    M204 P{global.low_accel}                                             ; Set printing acceleration(mm/s�)
                                  ;if move.shaping.type != global.input_shaper
                                    ;M593 P{global.input_shaper}                                          ; Configure input shaping
                                
                                elif global.line_type = "Perimeter"
                                  if move.printingAcceleration != 2000
                                    M204 P2000                                                           ; Set printing acceleration(mm/s�)
                                  ;if move.shaping.type != global.input_shaper
                                    ;M593 P{global.input_shaper}                                          ; Configure input shaping
                                
                                elif global.line_type = "Solid infill"
                                  if move.printingAcceleration != 2000
                                    M204 P2000                                                           ; Set printing acceleration(mm/s�)
                                  ;if move.shaping.type != "none"
                                    ;M593 P"none"                                                         ; Configure input shaping
                                
                                elif global.line_type = "Top solid infill"
                                  if move.printingAcceleration != global.low_accel
                                    M204 P{global.low_accel}                                             ; Set printing acceleration(mm/s�)
                                
                                ; When none of the above line extrusion types
                                else
                                  if move.printingAcceleration != global.def_print_accel
                                    M204 P{global.def_print_accel}                                       ; Set printing acceleration(mm/s�)
                                  ;if move.shaping.type != "none"
                                    ;M593 P"none"                                                         ; Disable input shaping
                            ; -----------------
                            ; Daemon loop delay
                            G4 S1                                                                        ; Delay running again or next command for at least 1 second
                          

                          And sb_leds-state.g:

                          ; /sys/lib/led/sb_leds-state.g  v2.0
                          ; Created by sb_leds.g to store the current/active LED colors 
                          ; Called by daemon.g to check if global.sb_leds status has changed since last run
                          
                          var sb_leds = "boot"
                          
                          if var.sb_leds = global.sb_leds
                            ; Same status, do nothing
                          else
                            ; New status, change colors
                            M98 P"/sys/lib/led/sb_leds.g"
                          

                          And while the printer is idle global.sb_leds don't change value.

                          So as far as i can see it shouldn't be any =0.1 or =0.6 anywhere.

                          1 Reply Last reply Reply Quote 0
                          • droftartsundefined droftarts referenced this topic
                          • Exerqtorundefined
                            Exerqtor @dc42
                            last edited by Exerqtor

                            EDIT:

                            Started a new thread to keep this one on topic.

                            1 Reply Last reply Reply Quote 0
                            • Exerqtorundefined
                              Exerqtor @dc42
                              last edited by

                              @dc42 said in Reboots/crashes - RRF 3.5.0-rc1:

                              @Exerqtor please try the firmware at https://www.dropbox.com/scl/fo/y34newtid8z5zp7qa5ted/h?rlkey=i65uvg992k74vrgygg4rp4g98&dl=0.

                              First M122 report after a crash on that build (also in a idle state when crashing) :

                              M122
                              === Diagnostics ===
                              RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-rc.1+ (2023-12-07 09:49:06) running on Duet 3 Mini5plus WiFi (standalone mode)
                              Board ID: XNHXF-HR6KL-K65J0-409N2-K9W1Z-RV2MZ
                              Used output buffers: 11 of 40 (40 max)
                              === RTOS ===
                              Static ram: 102844
                              Dynamic ram: 122152 of which 0 recycled
                              Never used RAM 13564, free system stack 186 words
                              Tasks: NETWORK(1,ready,1687.6%,203) HEAT(3,nWait,2.1%,330) Move(4,nWait,0.2%,351) CanReceiv(6,nWait,4.3%,774) CanSender(5,nWait,0.0%,336) CanClock(7,delaying,0.6%,350) TMC(4,nWait,60.6%,108) MAIN(1,running,1092.0%,670) IDLE(0,ready,21.0%,29) AIN(4,delaying,73.1%,264), total 2941.7%
                              Owned mutexes: WiFi(NETWORK)
                              === Platform ===
                              Last reset 04:49:41 ago, cause: software
                              Last software reset at 2023-12-08 01:28, reason: AssertionFailed, Platform spinning, available RAM 12356, slot 2
                              Software reset code 0x4120 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00489000 BFAR 0xe000ed38 SP 0x20011fbc Task NETW Freestk 495 ok
                              Stack: 00000913 000af408 0002de51 2002bed4 2002be01 000001af 2002c100 20031d10 2002c118 2001e888 a5a5a5a5 a5a5a5a5 0d362e30 00000000 00000000 00000000 20031d1c 00000800 20035a10 2002c100 20018678 2002bf9d 20018678 2001e888 000300db 00000000 00000000
                              Error status: 0x04
                              Aux0 errors 0,0,0
                              MCU revision 3, ADC conversions started 17381709, completed 17381709, timed out 0, errs 0
                              MCU temperature: min 34.2, current 34.8, max 38.2
                              Supply voltage: min 22.7, current 24.1, max 26.4, under voltage events: 0, over voltage events: 0, power good: yes
                              Heap OK, handles allocated/used 99/29, heap memory allocated/used/recyclable 2048/492/96, gc cycles 783
                              Events: 0 queued, 0 completed
                              Driver 0: standstill, SG min 2, read errors 0, write errors 1, ifcnt 106, reads 62817, writes 13, timeouts 0, DMA errors 0, CC errors 0
                              Driver 1: standstill, SG min 0, read errors 0, write errors 1, ifcnt 106, reads 62817, writes 13, timeouts 0, DMA errors 0, CC errors 0
                              Driver 2: standstill, SG min 2, read errors 0, write errors 1, ifcnt 222, reads 62816, writes 13, timeouts 1, DMA errors 0, CC errors 0, failedOp 0x6c
                              Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 222, reads 62816, writes 13, timeouts 0, DMA errors 0, CC errors 0
                              Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 219, reads 62817, writes 13, timeouts 0, DMA errors 0, CC errors 0
                              Driver 5: not present
                              Driver 6: not present
                              Date/time: 2023-12-08 06:17:56
                              Cache data hit count 4294967295
                              Slowest loop: 12.85ms; fastest: 0.14ms
                              === Storage ===
                              Free file entries: 18
                              SD card 0 detected, interface speed: 22.5MBytes/sec
                              SD card longest read time 7.2ms, write time 4.5ms, max retries 0
                              === Move ===
                              DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, ebfmin 0.00, ebfmax 0.00
                              no step interrupt scheduled
                              Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
                              === DDARing 0 ===
                              Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
                              === DDARing 1 ===
                              Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
                              === Heat ===
                              Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
                              Heater 1 is on, I-accum = 0.0
                              === GCodes ===
                              Movement locks held by null, null
                              HTTP is idle in state(s) 0
                              Telnet is idle in state(s) 0
                              File is idle in state(s) 0
                              USB is idle in state(s) 0
                              Aux is idle in state(s) 0
                              Trigger is idle in state(s) 0
                              Queue is idle in state(s) 0
                              LCD is idle in state(s) 0
                              SBC is idle in state(s) 0
                              Daemon is doing "G4 S1" in state(s) 0 0, running macro
                              Aux2 is idle in state(s) 0
                              Autopause is idle in state(s) 0
                              File2 is idle in state(s) 0
                              Queue2 is idle in state(s) 0
                              Q0 segments left 0, axes/extruders owned 0x0000803
                              Code queue 0 is empty
                              Q1 segments left 0, axes/extruders owned 0x0000000
                              Code queue 1 is empty
                              === Filament sensors ===
                              in 0 notIn 0
                              Extruder 0 sensor: no filament
                              === CAN ===
                              Messages queued 156442, received 356364, lost 0, errs 0, boc 0
                              Longest wait 2ms for reply type 6042, peak Tx sync delay 275, free buffers 26 (min 25), ts 86907/86906/0
                              Tx timeouts 0,0,0,0,0,0
                              === Network ===
                              Slowest loop: 24.24ms; fastest: 0.00ms
                              Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
                              HTTP sessions: 3 of 8
                              === WiFi ===
                              Interface state: active
                              Module is connected to access point 
                              Failed messages: pending 0, notrdy 0, noresp 0
                              Firmware version 2.1beta4
                              MAC address c4:5b:be:ce:91:93
                              Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 42964
                              WiFi IP address 192.168.10.x
                              Signal strength -54dBm, channel 6, mode 802.11n, reconnections 0
                              Clock register 00002001
                              Socket states: 0 0 0 0 0 0 0 0
                              
                              droftartsundefined 1 Reply Last reply Reply Quote 0
                              • droftartsundefined
                                droftarts administrators @Exerqtor
                                last edited by

                                @Exerqtor do you get crashes if you disable daemon.g?

                                Ian

                                Bed-slinger - Mini5+ WiFi/1LC | RRP Fisher v1 - D2 WiFi | Polargraph - D2 WiFi | TronXY X5S - 6HC/Roto | CNC router - 6HC | Tractus3D T1250 - D2 Eth

                                Exerqtorundefined 1 Reply Last reply Reply Quote 0
                                • Exerqtorundefined
                                  Exerqtor @droftarts
                                  last edited by

                                  @droftarts said in Reboots/crashes - RRF 3.5.0-rc1:

                                  @Exerqtor do you get crashes if you disable daemon.g?

                                  Ian

                                  As in setting the global to false disable, or have a empty daemon.g?

                                  I can try both, i just set the global to false (i'm pretty sure we tested this earlier, but it's been such a long time since this all started I have a hard time keeping up).

                                  dc42undefined 1 Reply Last reply Reply Quote 0
                                  • dc42undefined
                                    dc42 administrators @Exerqtor
                                    last edited by

                                    @Exerqtor better to have a daemon.g file with just a very long G4 command in it.

                                    Duet WiFi hardware designer and firmware engineer
                                    Please do not ask me for Duet support via PM or email, use the forum
                                    http://www.escher3d.com, https://miscsolutions.wordpress.com

                                    Exerqtorundefined 1 Reply Last reply Reply Quote 0
                                    • Exerqtorundefined
                                      Exerqtor @dc42
                                      last edited by

                                      @dc42 said in Reboots/crashes - RRF 3.5.0-rc1:

                                      @Exerqtor better to have a daemon.g file with just a very long G4 command in it.

                                      In general you mean? Or could elaborate on that a little bit?

                                      I have alot of crap in daemon.g currently, it's mostly disabled since i ditched what i was trying to do with it. Just haven't gotten around to cleaning it up yet.

                                      Guess i could do that now 🤷‍♂️

                                      dc42undefined 1 Reply Last reply Reply Quote 0
                                      • dc42undefined
                                        dc42 administrators @Exerqtor
                                        last edited by dc42

                                        @Exerqtor what I meant was that a daemon.g file with just a long G4 delay in it would result in less S card access than no daemon.g at all. The maximum value for a G4 delay command is about 2 million seconds.

                                        Conversely, if you try reducing the G4 S1 delay in your current daemon.g file (e.g. try G4 P10) then this will result in more SD card reads. It would be interesting to see if that results in more frequent resets. Please try that.

                                        Another thing you could do at the same time is to pad out the end of daemon.g with a few long comment lines something like this:

                                        ;abcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcd

                                        This might indicate whether the data that overwrites the return address comes from the residue of the sector of daemon.g that is currently past the end of the file. The sector size is 512 bytes so the number of comment characters in total shod be at least that.

                                        Duet WiFi hardware designer and firmware engineer
                                        Please do not ask me for Duet support via PM or email, use the forum
                                        http://www.escher3d.com, https://miscsolutions.wordpress.com

                                        Exerqtorundefined 1 Reply Last reply Reply Quote 0
                                        • Exerqtorundefined
                                          Exerqtor @dc42
                                          last edited by Exerqtor

                                          @dc42
                                          Gotcha!

                                          I just did the much needed cleanup in my daemon.g, but I've now added what you said mentioned.

                                          This is how it looks as of now:

                                          ; /sys/daemon.g  v2.5
                                          ; Used to execute regular tasks, the firmware executes it and once the end of file is reached it waits. If the file is not found it waits and then looks for it again.
                                          
                                          ;---/
                                          ; -/--/--/--/--/--/--/--/--/--/--/--/--/--/--/--/--
                                          ; THIS MACRO ONLY WORKS WITH RRF 3.5.0b1 AND LATER!!
                                          ;--/--/--/--/--/--/--/--/--/--/--/--/--/--/--/--/--
                                          ;-/
                                          
                                          ; Loop, to be able to turn on/off daemon.g
                                          while global.RunDaemon
                                            ; Stuff goes below this line
                                            ; ---------------------------------------------------------------------------         
                                          
                                            ; Resume after filament change
                                            if exists(global.FilamentCHG)
                                              if global.FilamentCHG = true && state.status="paused"
                                                M24                                                                      ; Resume the pause automatically now that the manual filament change is done
                                                set global.FilamentCHG = false
                                          
                                            ; --------------------
                                          
                                            ; Refresh chamber lights status
                                            if exists(global.chamber_leds)
                                              set global.chamber_leds = state.gpOut[0].pwm * 100
                                          
                                            ; --------------------
                                          
                                            ;Check sb_leds status
                                            var SB_LEDS = true                                                           ; Turn on(true) / off(false) the Stealthburner led "system"                                                          
                                            if var.SB_LEDS
                                              if fileexists("/sys/lib/led/sb_leds-state.g")
                                                M98 P"/sys/lib/led/sb_leds-state.g"                                      ; Check if global.sb_leds has changed since last run/loop
                                          
                                            ; ---------------------------------------------------------------------------
                                            ; Daemon loop delay
                                            G4 P10                                                                        ; Delay running again or next command for at least 1 second (was S1)
                                          
                                          ;abcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcd
                                          ;abcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcd
                                          ;abcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcd
                                          ;abcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcd
                                          ;abcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcdbcdabcdabcdabcd
                                          
                                          dc42undefined 1 Reply Last reply Reply Quote 0
                                          • dc42undefined
                                            dc42 administrators @Exerqtor
                                            last edited by

                                            @Exerqtor thanks, let's see if that resets more often than before.

                                            Duet WiFi hardware designer and firmware engineer
                                            Please do not ask me for Duet support via PM or email, use the forum
                                            http://www.escher3d.com, https://miscsolutions.wordpress.com

                                            Exerqtorundefined 2 Replies Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Unless otherwise noted, all forum content is licensed under CC-BY-SA