Is a client uploading a file a slow client from unicorn's point of view?

Laas Toom laas at toom.ee
Tue Oct 9 23:06:55 UTC 2012


On 09.10.2012, at 23:03, Eric Wong <normalperson at yhbt.net> wrote:

>> 2) make the upload progress available, so e.g. AJAX-powered upload forms can show progressbar, which is really neat. No need for Flash-based uploaders.
> 
> It does?  I'm not seeing it in the documentation, but I know there's
> a separate upload progress module, though:
> http://wiki.nginx.org/HttpUploadProgressModule

I must have mixed them up then. :-)

>> I have a Rails app that accepts media uploads. All the processing happens in background, front-end handles only the actual upload and stores it to disk.
>> But with uploads as large as 1.4 GB, I've seen Rails response times > 200 secs. This starts to give timeouts in weird places.
> 
> Yikes.  I assume you're constrained by disk I/O there?

Might be. Additionally, the disk is SAN, so network activity there too.

> For some of the large file situations under Linux, I find it beneficial
> to lower the dirty_*ratio/*bytes drastically to avoid large, sudden
> bursts of disk activity and instead favor smaller writes.  I get lower
> throughput, but more consistent performance.

I shall look into it when I get to fixing this issue.


>> Afterwards it will only handle out the file location and Rails can
>> complete it's work a lot faster, freeing up workers.
>> 
>> Unicorn won't even see the file and Rails has the responsibility to
>> delete the file if it's invalid.
> 
> I think the only problem with this approach is it won't work well on
> setups where nginx is on separate machines from unicorn.  Shared
> storage would be required, but that ends up adding to network I/O,
> too...

But won't (almost) the same network I/O be evident anyway, because of nginx transferring the data to Unicorn over network (as they are on different machines)?

Thanks for clarifying,
Laas



More information about the mongrel-unicorn mailing list