Too many header key-value pairs
-
I've got a bit of a complex setup where I'm exposing DWC to the internet by reverse proxying through traefik with authelia as a forwardauth middleware. The issue that I'm having is when I set authelia as a middleware and try to access my duet I just get the error "Your Duet rejected the HTTP request: too many header key-value pairs". I use it for several other services as well with no problem.
What's interesting is it only does this with chrome but loads DWC fine on firefox. Also, when I disable the authelia middleware and just expose the duet through traefik with no other layer it loads DWC no problem.
Honestly I don't know from what point to begin to troubleshoot this. I'm not sure if this is an issue with DWC, chrome, traefik, authelia, or my configuration somewhere. In the first place I don't fully understand what this error means, and it seems to not be standard, unless my goggle-fu is failing me. Any advice is appreciated.
-
@Masterjuggler Looks like your middleware adds too many HTTP headers to your client HTTP request. I don't know your middleware, but somehow you need to reconfigure your middleware and reduce the number of HTTP header key/value pairs to 30 (which is already quite a lot). Ideally, it would only forward the original headers from the client and nothing else.
-
@chrishamm do you know why middleware would add key/value pairs to GET requests? Isn't there a danger of a clash with key/value pairs already used? Or perhaps there are some reserved key names?I guess a Wireshark trace would show us what is being added, but I don't know whether it is possible to get a Wireshark trace to display packets that don't involve the PC that Wireshark is running on.
-
@chrishamm It sounds like this is something inherent to the way DWC is set up? If that's the case though, why would it be fine in firefox and only throw this error in chrome? I've been using this middleware with about 15 other services I'm exposing and haven't had any issues so far.
-
@Masterjuggler Ar those other services running on an embedded processor with like 128 kilobytes of RAM?
The Duet is a pretty specialized machine and "normal" (read wasteful. profligate bloatware) software development practices do not necessarily apply.
I worked with a vendor once that wanted to send me a 4 gigabyte JSON file to grab some values out of and when I mentioned I had a TOTAL ram of 96 kilobytes, he called me a liar on a Zoom call. Well...... We eventually chose a different data vendor and they lost out on a million dollar a year contract.
So, this is a special machine that has very strict limits and "It works with other PC-based applications" isn't really going to apply here.
Can you grab the GET packet and post it so we can see all the headers your application is adding to the stream please? That would be incredibly helpful and you might be able to help the Duet team make an even better product.
-
@Masterjuggler That may be but RRF running on an embedded processor has limited RAM. Version 3.5 of DWC adds a single HTTP header to HTTP requests, older versions don't add anything at all. Everything else originates from your browser+middleware. When I send HTTP requests to my printer the number of HTTP headers is 13, which is still pretty far away from the limit of 30 headers. Perhaps some kind of browser extension adds more headers, I don't know - try to connect to your Duet via middleware from a private/incongnito tab and see if that makes a difference.
I'm afraid I cannot say what else is added without a dump of a GET request or a Wireshark trace between Duet and middleware.
@dc42 I think some reverse proxies add X-Forward-* headers but I can't say much more about that.
-
@chrishamm @Masterjuggler if you can find out what headers your middleware is adding (perhaps by using Wireshark, or by asking the supplier of the middleware) then we may be able to add a filter to ignore the ones that the Duet doesn't care about, to avoid filling up the table.
-
@dc42 I'll have to figure out a proper method of tracking down exactly what headers are being added. I'm a little bit in information overload right now lol, learning this stuff as I go.
After some playing around with the configuration I did find that removing authelia's default authResponseHeaders of Remote-User, Remote-Groups, Remote-Name, and Remote-Email allows DWC to load on chrome. That's only 4 headers and still means that a lot are being added somewhere along the chain, so I'll still try to figure out what's going on when I have time, but at least right now it's not a big problem. Unless I need those headers for some reason, I don't fully understand what they do tbh.
-
@Masterjuggler I've created this issue https://github.com/Duet3D/RepRapFirmware/issues/870 to track this. Please tell us which Duet, firmware version and DWC version you are using.
-
If anyone is interested, Authelia is open-source, and there are many ways to contact them listed in this github page.
Maybe jumping on a discord server or Matrix Space (whatever that is) and asking this question could provide an answer and maybe even a solution.
-
@dc42 I'm using a Duet 2 WiFi with firmware & DWC version 3.4.5. No expansion boards or RPI. To expand on my setup a bit, I'm running an ubuntu headless server with docker containers for traefik 2.10.1 and authelia. I'm then exposing via cloudflare tunnel (cloudflared) to avoid having to forward a port from my router. It's possible headers are being added at multiple stages through this chain, I was focusing on authelia because that was the straw that broke the camel's back so to speak.
As mentioned, removing authelia's four default authResponseHeaders (taken from here) reduced the number of headers such that DWC now loads in chrome with authelia in place, though I don't understand the system well enough to know if that is secure or if it causes any other problems.
-
@Masterjuggler I don't think removing those extra headers has any effect - they're ignored by RRF anyway. As previously said, a Wireshark dump taken between Duet and middleware may help us further figure out what and how many headers need to be handled in your config scenario.
-
This post is deleted!