Is a client uploading a file a slow client from unicorn's point of view?
laas at toom.ee
Tue Oct 9 23:06:55 UTC 2012
On 09.10.2012, at 23:03, Eric Wong <normalperson at yhbt.net> wrote:
>> 2) make the upload progress available, so e.g. AJAX-powered upload forms can show progressbar, which is really neat. No need for Flash-based uploaders.
> It does? I'm not seeing it in the documentation, but I know there's
> a separate upload progress module, though:
I must have mixed them up then. :-)
>> I have a Rails app that accepts media uploads. All the processing happens in background, front-end handles only the actual upload and stores it to disk.
>> But with uploads as large as 1.4 GB, I've seen Rails response times > 200 secs. This starts to give timeouts in weird places.
> Yikes. I assume you're constrained by disk I/O there?
Might be. Additionally, the disk is SAN, so network activity there too.
> For some of the large file situations under Linux, I find it beneficial
> to lower the dirty_*ratio/*bytes drastically to avoid large, sudden
> bursts of disk activity and instead favor smaller writes. I get lower
> throughput, but more consistent performance.
I shall look into it when I get to fixing this issue.
>> Afterwards it will only handle out the file location and Rails can
>> complete it's work a lot faster, freeing up workers.
>> Unicorn won't even see the file and Rails has the responsibility to
>> delete the file if it's invalid.
> I think the only problem with this approach is it won't work well on
> setups where nginx is on separate machines from unicorn. Shared
> storage would be required, but that ends up adding to network I/O,
But won't (almost) the same network I/O be evident anyway, because of nginx transferring the data to Unicorn over network (as they are on different machines)?
Thanks for clarifying,
More information about the mongrel-unicorn