I think I have found a bug in net/http.rb as distributed with Ruby 1.8.4.
I'm using s3sync to upload files to Amazon S3, which uses the
send_request_with_body_stream method to send file data.
Before doing this, it sets the Content-Length header to the current file
size. However, the file can grow (or shrink) before the upload finishes, and
send_request_with_body_stream makes no attempt to deal with this. It simply
trusts the caller to set Content-Length exactly right, and reads and writes
as many bytes as it can get from the file.
This can cause it to break the HTTP protocol by trying to write more bytes
than the server is expecting. The server may interpret subsequent data as
new commands, possibly leading to a security breach if this was used as a
well-timed attack.
The best solution would be to use Chunked encoding, as this can deal with
the file shrinking or growing. However, Amazon S3 does not support chunked
encoding, so we need another solution.
In case the file grows, I'd suggest not sending any more bytes than the
Content-Length header says. In other words, replacing the following code
around line 1523:
while s = f.read(1024)
sock.write s
end
with something like this:
len = content_length()
while len > 0
chunk = len
if (chunk > 1024) then chunk = 1024 end
s = f.read(chunk)
sock.write(s)
len -= s.length
end
This change appears to fix the problem for me. However it does nothing to address shrinking files, which can cause similar
problems. I'd suggest padding the file data with zeroes in this case, or throwing an exception that causes the upload
to fail (or a higher layer to retry). |