@tinker It might be possible that the slight delay you need to enter a password is sufficient for the resources on the Duet to free up. That's the only explanation I can think of. NS_ERROR_CONNECTION_REFUSED would be a sign that the Duet is still overloaded (because of too few free sockets) when DWC attempts to load the object model.
If you need FTP and/or Telnet access from time to time, perhaps consider creating macros to turn that on and off on demand, but start with FTP/Telnet disabled. That should make the initial load of DWC on a Duet 2 Ethernet or Duet Maestro more stable.