Duet Web Control 2.0.0-RC3 is ready

dc42 · 26 Dec 2018, 14:07

@sigxcpu, I have come to the same conclusion that there are problems triggered by the increased number of concurrent connections that DWC2 uses. The problems are worse when there is a password set, but I get them occasionally even with no password. I've asked chrishamm if he can reduce the number of concurrent connections used. When I next work on DuetWiFiServer, I'll review the way that multiple concurrent connections are handled.

wilriker · 26 Dec 2018, 14:42

After using DWC2 a lot today, I have found some issues (some personal others breaking changes)

I looks nice but wastes a lot of space especially on smaller resultion screens, e.g. there now are only 3 axis jog buttons per direction whereas there were 4 before. Changing CSS in dev tools to unhide one extra column of buttons use the space much more efficient without being too dense
It misses some information, e.g. end times based on remaining time estimates, detailed versions currently installed, more details on sliced file being printed (not that important)
Sometimes when switching away form job status and returning, the layer time graph stays empty until page is reloaded in browser

And my personal deal breaker: I cannot access DWC from remote anymore. I always get CORS request failed.
My setup for remote access:

I have a small server in my network that can be accessed from anywhere via SSH.
This server runs a nginx that has a host duet.localhost as a proxy pass setup and only listens on loopback interface.
I then connect from my smartphone to my server via SSH and do a port forwarding -L8080:localhost:80.
I access http://duet.localhost:8080 on my smartphone's browser and get DWC - and actually the page loads but then quits with the above error and does not load any status

EDIT: I just checked and duet.localhost:8080/reprap.htm works as always.

EDIT2: what I really do like is that the Macros widget has directories listed first now.

Scachi · 26 Dec 2018, 14:58

For a file Filament usage is shown as:
2731.8999999999996
when I hover the mouse over this value this information shows up (slides down):
$display(props.item[header.value],1,'mm');

gnydick · 26 Dec 2018, 17:25

@dc42 browsers have a limit of the number of concurrent connections to the same domain (IP address in this case).

All the more reason to simplify.

gnydick · 26 Dec 2018, 17:30

@wilriker didn't work

sigxcpu · 26 Dec 2018, 20:19

@dc42 said in Duet Web Control 2.0.0-RC2 is ready:

@sigxcpu, I have come to the same conclusion that there are problems triggered by the increased number of concurrent connections that DWC2 uses. The problems are worse when there is a password set, but I get them occasionally even with no password. I've asked chrishamm if he can reduce the number of concurrent connections used. When I next work on DuetWiFiServer, I'll review the way that multiple concurrent connections are handled.

@gnydick said in [Duet Web Control 2.0.0-RC2 is ready]

@dc42 browsers have a limit of the number of concurrent connections to the same domain (IP address in this case).

All the more reason to simplify.

It is the other way around. Browsers have a too high limit of concurrent connections per domain.
Most of modern browsers have 6, which is too much compared to Duet's 4.

For example, I've recompiled the firmware with 6 HTTP acceptors instead of 4. It reloads DWC2 most of the times, with very few exceptions (i've had DWC1 open in another brower so that ate a connection).
Unfortunately, I am pretty sure that will kill the firmware during printing because increasing to 8 kills it in a boot loop.

The proper fix here should be to implement TCP connection backlogging, instead of refusing them. OK, we serve 4 simultaneout, but we can keep more intents in the backlog before accepting them. I don't know how much memory a backlogged connection eats.

I don't think users care too much about the UI load time, but we do care about it to load reliably.
Another fix is in DWC2 to compact all JS files in a single one and all CSS files in a single one.

Maybe there is a web tool to compact all these in a single "archive" whatever that is. I am a backend developer, therefore I'm not versed in all of these.

gnydick · 27 Dec 2018, 01:19

@sigxcpu That's exactly my point. The web page code should be simplified. It shouldn't be all ajax-y for an embedded web host (with limited resources -- 4 concurrent connections) .

There definitely are minifying tools fro JS and CSS.

Ultimately, the interface is way too complex for the task. Read my rant up the thread.

Danal · 27 Dec 2018, 01:40

This post is deleted!

brandonh · 27 Dec 2018, 09:55

The new DWC looks good to me. Installed super-quick and looks more modern. I'm very happy to see X axis time labels for the heater diagram - it sounds mundane but this is really useful. Good work!

Minor issues I saw:

The developer console shows an error upon page load:
GET http://bigkossel.local/rr_download?name=0%3A%2Fsys%2Fdwc2defaults.json 404 (file not found)
G-code Jobs shows "No jobs" until loaded; would be good to show "loading"
Clicking the back button for Gcode Jobs after going to a folder takes me to the Settings tab and system folder, oddly.

Suggestions:

When I'm in a printing Status view, the thing I care about are the estimated print times. These could have the font vastly larger. In this view, speed graphing would be AWESOME for debugging.

sigxcpu · 27 Dec 2018, 09:59

@gnydick said in Duet Web Control 2.0.0-RC2 is ready:

@sigxcpu That's exactly my point. The web page code should be simplified. It shouldn't be all ajax-y for an embedded web host (with limited resources -- 4 concurrent connections) .

There definitely are minifying tools fro JS and CSS.

Ultimately, the interface is way too complex for the task. Read my rant up the thread.

I think "ajax-y" is the right way to do it here because after loading the assets, it just polls for status, which is thin.
Initial loading is the issue here because it tries to load the assets like from a "normal" web server, with parallel connections up to 6.
I see that there are 2 JS, 2 CSS and one "font" besides the index page.
The 2+2 are loaded right after the index page but sometimes one of them fails so I would assume that the browser optimizes loading and initiates an asset download before the index loading is finished and connection reused.
I am pretty sure that's the difference between:

it loads => index page connection is finished and reused, therefore there are 4 conns available
it fails to load => index page is lingering a bit more, so one of the 4 assets is failing to load

Maybe another thing that will help is to enable caching because I always see all the things loading, never a 304.
I don't know how complicated would be but HTTP/1.1 does have connection keep alive. Browser asks for it but the server reponds with Connection: close explicitly. Though this means up to 6 connections anyway.

Overall, my grip is not with the complexity of the JS app, but it is obvious that the way it loads is completely incompatible with the embedded webserver.

burtoogle · 27 Dec 2018, 10:18

Here's a suggestion that may possibly help. I developed an HTML/Javascript GUI for my PB-4 balancer (https://smartavionics.co.uk/pb4/pb4.html). It's based on jquery + bootstrap and the web server is a WiFi module with minimal resources. What I have done to speed up the initial load is make use of a javascript library called basket.js (https://addyosmani.com/basket.js/) that caches files in the browser local storage. So I just load the initial HTML and basket.js from the WiFi module and then everything else jquery, bootstrap, application code all gets loaded from the basket. Works extremely well.

burtoogle · 27 Dec 2018, 10:21

Another thing, to actually do the data transfer (vibration data, spectra, commands, status, etc.) between the browser and the balancer HW, I use a websocket and that works very well also.

sigxcpu · 27 Dec 2018, 10:34

True, but the problem here is the initial/cold/first load. Speeding that up will be helpful if it actually loads reliably.

burtoogle · 27 Dec 2018, 10:53

@sigxcpu said in Duet Web Control 2.0.0-RC2 is ready:

True, but the problem here is the initial/cold/first load. Speeding that up will be helpful if it actually loads reliably.

Understood. However, when using basket.js you are in control as to when a resource is loaded so you don't get the multiple concurrent connections that cause problems with low-powered hosts.

dc42 · 27 Dec 2018, 10:56

For example, I've recompiled the firmware with 6 HTTP acceptors instead of 4. It reloads DWC2 most of the times, with very few exceptions (i've had DWC1 open in another brower so that ate a connection).
Unfortunately, I am pretty sure that will kill the firmware during printing because increasing to 8 kills it in a boot loop.

My guess is that with 8 you are running short of memory. Each HTTP responder needs around 2K of memory.

The proper fix here should be to implement TCP connection backlogging, instead of refusing them. OK, we serve 4 simultaneout, but we can keep more intents in the backlog before accepting them.

It already does. From https://github.com/dc42/DuetWiFiSocketServer/blob/dev/src/Listener.cpp, around line 150:

 	}
	p->listeningPcb = tcp_listen_with_backlog(tempPcb, Backlog);
	if (p->listeningPcb == nullptr)
	{

The constant Backlog is set to 8.

fotomas · 27 Dec 2018, 13:03

Bug report:

Duet Web Control 2.0.0-RC2 / RepRapFirmware 1.23

Sending G-Code using a mouse click on the "send" button sends previous command. It seems to always be one step behind. Two mouse clicks on send button makes the intended command execute. This behavior occurs on both places where a G-Command can be executed.

Sending a command using Enter key works.

gnydick · 27 Dec 2018, 15:22

@sigxcpu yeah, the polling via Ajax is expected. But I was assuming there are a lot more elements than that that are async.

Like I said, it's just too complicated. It's not a control panel, it's a beauty show piece, and until they realize it, it's not going to be really efficient.

whosrdaddy · 27 Dec 2018, 18:47

Small warning for those who are running the duet on a non standard port (ie port <> 80), DWC2.0 (RC2) will not function.
I opened an issue on the Github page so @chrishamm can fix this small bug

BLV · 27 Dec 2018, 19:13

Could you share please some screenshots of the new interface?

sigxcpu · 27 Dec 2018, 20:12

@dc42 said in Duet Web Control 2.0.0-RC2 is ready:

For example, I've recompiled the firmware with 6 HTTP acceptors instead of 4. It reloads DWC2 most of the times, with very few exceptions (i've had DWC1 open in another brower so that ate a connection).
Unfortunately, I am pretty sure that will kill the firmware during printing because increasing to 8 kills it in a boot loop.

My guess is that with 8 you are running short of memory. Each HTTP responder needs around 2K of memory.

The proper fix here should be to implement TCP connection backlogging, instead of refusing them. OK, we serve 4 simultaneout, but we can keep more intents in the backlog before accepting them.

It already does. From https://github.com/dc42/DuetWiFiSocketServer/blob/dev/src/Listener.cpp, around line 150:
 	}
	p->listeningPcb = tcp_listen_with_backlog(tempPcb, Backlog);
	if (p->listeningPcb == nullptr)
	{
 
The constant Backlog is set to 8.

Then something is broken in backlog implementation because the Nth+1 connection gets TCP RST instead of waiting. Here is how to find out N:

 bash-3.2$ ab -r -v 0 -dSq -n 10 -c 3 http://192.168.27.8:80/index.html
...
Concurrency Level:      3
Time taken for tests:   0.185 seconds
Complete requests:      10
Failed requests:        0
...
 
bash-3.2$ ab -r -v 0 -dSq -n 10 -c 4 http://192.168.27.8:80/index.html
...
Concurrency Level:      4
Time taken for tests:   0.123 seconds
Complete requests:      10
Failed requests:        0
 
bash-3.2$ ab -r -v 0 -dSq -n 10 -c 5 http://192.168.27.8:80/index.html
...
Concurrency Level:      5
Time taken for tests:   0.162 seconds
Complete requests:      10
Failed requests:        9
   (Connect: 0, Receive: 1, Length: 8, Exceptions: 0)

	}
	p->listeningPcb = tcp_listen_with_backlog(tempPcb, Backlog);
	if (p->listeningPcb == nullptr)
	{

	bash-3.2$ ab -r -v 0 -dSq -n 10 -c 3 http://192.168.27.8:80/index.html
	...
	Concurrency Level: 3
	Time taken for tests: 0.185 seconds
	Complete requests: 10
	Failed requests: 0
	...

	bash-3.2$ ab -r -v 0 -dSq -n 10 -c 4 http://192.168.27.8:80/index.html
	...
	Concurrency Level: 4
	Time taken for tests: 0.123 seconds
	Complete requests: 10
	Failed requests: 0

	bash-3.2$ ab -r -v 0 -dSq -n 10 -c 5 http://192.168.27.8:80/index.html
	...
	Concurrency Level: 5
	Time taken for tests: 0.162 seconds
	Complete requests: 10
	Failed requests: 9
	(Connect: 0, Receive: 1, Length: 8, Exceptions: 0)