[Mongrel] Uploading Large (100mb+) files

Rogelio Samour rogelio.samour at gmail.com
Wed Nov 29 11:13:37 EST 2006


  Great job on merb! And the myriad of other projects you're involved in!

Quick question:
- I followed your instructions checked out the latest merb trunk (rev 99)
- rake installed (after installing all it's dependencies... do you mind
listing those on the README somewhere? :)
- cd'd to the sample_app, and proceeded to upload a 500mb iso image
- the problem: it spins its wheels until it's finished and then it shows:

       File uploaded successfully "UPLOADED: {"commit"=>"Upload",
"action"=>"upload", "id"=>nil,

"controller"=>"files", "upload_id"=>"1164816252",




- so it's uploading fine but it's not showing the progress of the upload.
- what am I doing wrong?

thanks again!



here's what my terminal shows:
{"action"=>"progress", "id"=>nil, "controller"=>"files",
"upload_id"=>"1164816252"}(this one several times)
skip mutex (this one several times)
{"action"=>"progress", "id"=>nil, "controller"=>"files",
"upload_id"=>"1164816252"}(this one several times)
=> fh[type] = application/x-cd-image
skip mutex (this one several times)
{"action"=>"progress", "id"=>nil, "controller"=>"files",
"upload_id"=>"1164816252"}(this one several times)
{"commit"=>"Upload", "action"=>"upload", "id"=>nil, "controller"=>"files",
"upload_id"=>"1164816252", "data"=>{"name"=>"data",
"type"=>"application/x-cd-image", "tempfile"=>#<File:/tmp/Merb7817.0>,
skip mutex

On 11/28/06, Ezra Zygmuntowicz <ezmobius at gmail.com> wrote:
>         There is an example merb controller for use with the
> mongrel_upload_progress if  anyone is interested in using a small
> merb app for their upload app server. We have a number of customers
> doing this and some of them are handling a ton of file uploads all
> the time. Merb can also be a nice secure download manager. The Files
> controller in the example app in the merb svn shows how easy it is to
> do:
> http://svn.devjavu.com/merb/examples/sample_app/dist/app/controllers/
> files.rb
>         You can do a svn checkout of the merb svn here:
> $ svn co http://svn.devjavu.com/merb
> And build it and play with the example app like this:
> $ cd merb
> $ sudo rake install
> $ cd examples/sample_app
> $ merb -p 4000
>         Then go to http://localhost:4000/files/start in your browser to
> see
> an example of the upload progress. Sice it is running locally you
> shoudl try to upload a file at least 100Mb or bigger or else it will
> happen too fast and you won't see the progress bar.
>         The nice thing about the way merb works is that it doesn't use
> cgi.rb for mime parsing and it is written in a more thread safe way
> then actionpack. So multiple concurrent requests can be processed and
> have their mime parsing done all at the same time. Merb only does a
> mutex around the smallest possible place where you might be calling
> ActiveRecord code. With the rails dispatcher, the mutex surrounds the
> entire routing code and mime parsing and your controler code. Merb
> only locks around the action in your controller code. It does route
> recognition and mime parsing outside the mutex in a multi threaded
> way. This allows us to use ActiveRecord in single threaded mode which
> is better then AR's multi threaded mode for performance. So we use
> the smallest sized lock we can in order to still support this, but we
> do all the heavyweight routing and mime parsing concurrently.
>         I promised Zed I would do a write up about using a merb app for a
> dedicated upload server in conjunction with a rails app. I will try
> to get something written this week and posted. Until then I am open
> to and questions about merb and you can file tickets for support or
> feature requests here:
> http://merb.devjavu.com/
> http://merb.devjavu.com/projects/merb/register
>         Zed and I hope to come up with a solution so that we can use the
> upload progress without reparsing the mime twice per file. Right now
> mongrel streams the upload to a tmp file and increments the progress
> bar all the way to 100% before it hands control to your rails or merb
> controller to deal with the file. But then rails or merb has to parse
> the mime boundaries of the entire file again. So you may notice with
> big files that right when the progress bar makes it to 95-100% it
> will pause while rails or meb parses the mime again into another
> tmpfile. This is obviously not an optimal way to do this. Zed has
> started an awesome fast C mime carver and I will be working with him
> to integrate this so uploads can be parsed once with the fast C mimer
> and then just handed off to the next handler with no need to reparse.
> Cheers-
> -- Ezra Zygmuntowicz
> -- Lead Rails Evangelist
> -- ez at engineyard.com
> -- Engine Yard, Serious Rails Hosting
> -- (866) 518-YARD (9273)
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20061129/65a54e43/attachment-0001.html 

More information about the Mongrel-users mailing list