uWebsockets inconsistently prematurely closes connection while reading response headers with nginx ingress and ssl passthrough #1859
-
|
Hello! Context - I have a setup in kubernetes comprised of some uWebsockets (C++) applications in pods behind an nginx ingress controller (basically a reverse proxy). SSL pass through is enabled so that SSL termination happens in uwebsockets rather than nginx. This is for security purposes, and I also suspect it would be faster than terminating at nginx if setup properly. Normally, when I use curl or the js fetch api, everything works great (even when absolutely flooded with requests). However, if I were to make a request from a browser (ie just put the url into the address bar), it works fine about 40% of the time, and then suddenly starts to return 502 bad gateway errors. This is an issue because we are serving images that get loaded via the src tag in html images in our frontend, and most of these requests end up in 502 errors. After further investigation, it appears that uwebsockets closes the connection while nginx is trying to read the reply, the nginx logs are as follows: <server.address> is our server address, just a standard domain (Edit: Just a quick note, this even happens with small responses, (like json) not just for large responses like images) Could it be that this has something to do with http versions / ALPN? I am aware of uNetworking/uWebSockets.js#835, and I have already tried the proxy_http_version 1.1; thing but it changes nothing (which is to be expected as ssl-passthrough is enabled). I would really appreciate some help as we cannot progress further without solving this issue :( if anyone wants more testing / logs just let me know! |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
Welp, after many weeks of troubles, I have finally fixed it. It is not an error with uWebsockets at all (it almost never is), for some reason nginx was trying http instead of https sometimes to the pods, which are only listening for https (key from the logs: All I did was add the annotation It works flawlessly now! (I can't believe I missed an entire days worth of sleep for one line of config) |
Beta Was this translation helpful? Give feedback.
Welp, after many weeks of troubles, I have finally fixed it. It is not an error with uWebsockets at all (it almost never is), for some reason nginx was trying http instead of https sometimes to the pods, which are only listening for https (key from the logs:
... upstream: "http://192.16 ...)All I did was add the annotation
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"to my ingress.yaml and re-applied it.It works flawlessly now! (I can't believe I missed an entire days worth of sleep for one line of config)