Recently we came across following error when mobile app was trying to upload images using backend API.
The backend API was developed using Ruby on Rails and was powered using NGINX HTTP server along with unicorn application server.
Notice the the http response received was
502 which is documented in
RFC as shown below.
10.5.3 502 Bad Gateway The server, while acting as a gateway or proxy, received an invalid response from the upstream server it accessed in attempting to fulfill the request.
Debugging the problem
From the first look it seemed that the
client_max_body_size parameter for nginx was too less,
which might cause the request to fail if the uploaded filesize is greater than the allowed limit.
But that was not the case with our NGINX configuration. The NGINX was configured to accept file size upto 250 megabytes, while the file being uploaded was around 6 megabytes.
In our Rails code we are forcing all traffic to use HTTPS by having
production.rb to have following setting.
What this setting does is, it forces HTTPS by redirecting HTTP requests to their HTTPS counterparts.
So any request accessing
http://domain.com/path will be redirected to
Forcing all traffic to HTTPS still does not explain the vague error
sendfile() failed (32: Broken pipe) while sending request to upstream and why the
HTTP 502 error which means
Here is how our
nginx.conf is configured.
NGINX is configured to accept both HTTP and HTTPS requests. This is a pretty standard setting so that application can support both HTTP and HTTPS. What it means is that nginx will accept both HTTP and HTTPS request. It would not force HTTP request to be HTTPS. It would forward HTTP request to rails backend api.
Here is what happens when you upload a file
When we upload a file then NGINX takes that request and passes that request to Rails. NGINX expects a response from Rails only when NGINX is done sending the whole request.
Let’s see what happens when a user visits login page using HTTP. NGINX takes the request and passes the request to Rails. When NGINX is done handing over the request to Rails then NGINX expects a response from Rails. However in this case rather than sending 200 Rails sends a redirect over HTTPS. So now NGINX takes up the request again over HTTPS and then hands over request to Rails. Rails processes the request and returns 200. Everything works out.
Now let’s see what happens when a file is uploaded over HTTP.
User uploads a file over HTTP. NGINX takes up the request and starts sending the file to Rails. More details about the file upload process can be see at RFC. The data is sent to server in chunks. When first chunk is sent to server then server notices that data is being sent over HTTP and it immediately sends a response that data should be sent over HTTPS.
In the meantime on the client side , client is still pushing rest of the data to the server. Client expects a response from the server only when it is done pushing all the data. However in this case server sends a response when client is not expecting it.
NGINX at this time is all confused and thinks something has gone wrong
with the gateway and returns
HTTP/1.1 502 Bad Gateway and aborts the
upload operation. And that how we get the
So next time you see an error like this make sure that you are using
https to upload the file if server is enforcing