[Mongrel] Uploading Large (100mb+) files

Rogelio Samour rogelio.samour at gmail.com
Wed Nov 29 12:19:09 EST 2006


Ezra:

  Doh! The @upstatus var is never set because of the following:
Params: {"action"=>"progress", "id"=>nil, "controller"=>"files",
"upload_id"=>"1164816252"}
Routing to controller: Files action: progress
Parsing HTTP Input took: 0.038226 seconds
uninitialized constant Mongrel::Uploads - (NameError)
/usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:100:in
`const_missing'
/home/rsamour/public_html/merb/examples/sample_app/dist/app/controllers/files.rb:22:in
`progress'

The problem stems from the files.rb file (line 22 and 23). I know this is
straight from the mup example the
docs<http://mongrel.rubyforge.org/docs/upload_progress.html>

I'm missing a require somewhere... or the example missing it? hmmm

-rjs-

p.s. I do have the mup gem installed.




On 11/29/06, Rogelio Samour <rogelio.samour at gmail.com> wrote:
>
> Ezra,
>
>   Great job on merb! And the myriad of other projects you're involved in!
>
> Quick question:
> - I followed your instructions checked out the latest merb trunk (rev 99)
> - rake installed (after installing all it's dependencies... do you mind
> listing those on the README somewhere? :)
> - cd'd to the sample_app, and proceeded to upload a 500mb iso image
> - the problem: it spins its wheels until it's finished and then it
> shows:
>
>        File uploaded successfully "UPLOADED: {"commit"=>"Upload",
> "action"=>"upload", "id"=>nil,
>
> "controller"=>"files", "upload_id"=>"1164816252",
>
>            "data"=>{"name"=>"data",
>
> "type"=>"application/x-cd-image",
>
> "tempfile"=>#,
>
> "filename"=>" debian-31r2-i386-netinst.iso"}}
> - so it's uploading fine but it's not showing the progress of the upload.
> - what am I doing wrong?
>
> thanks again!
>
> cheers,
>
> -rjs-
>
> here's what my terminal shows:
> {"action"=>"progress", "id"=>nil, "controller"=>"files",
> "upload_id"=>"1164816252"}(this one several times)
> skip mutex (this one several times)
> {"action"=>"progress", "id"=>nil, "controller"=>"files",
> "upload_id"=>"1164816252"} (this one several times)
> => fh[type] = application/x-cd-image
> skip mutex (this one several times)
> {"action"=>"progress", "id"=>nil, "controller"=>"files",
> "upload_id"=>"1164816252"} (this one several times)
> {"commit"=>"Upload", "action"=>"upload", "id"=>nil, "controller"=>"files",
> "upload_id"=>"1164816252", "data"=>{"name"=>"data",
> "type"=>"application/x-cd-image", "tempfile"=>#<File:/tmp/Merb7817.0>,
> "filename"=>" debian-31r2-i386-netinst.iso"}}
> skip mutex
>
>
>
> On 11/28/06, Ezra Zygmuntowicz < ezmobius at gmail.com > wrote:
> >
> >
> >         There is an example merb controller for use with the
> > mongrel_upload_progress if  anyone is interested in using a small
> > merb app for their upload app server. We have a number of customers
> > doing this and some of them are handling a ton of file uploads all
> > the time. Merb can also be a nice secure download manager. The Files
> > controller in the example app in the merb svn shows how easy it is to
> > do:
> >
> > http://svn.devjavu.com/merb/examples/sample_app/dist/app/controllers/
> > files.rb
> >
> >         You can do a svn checkout of the merb svn here:
> >
> > $ svn co http://svn.devjavu.com/merb
> >
> > And build it and play with the example app like this:
> >
> > $ cd merb
> > $ sudo rake install
> > $ cd examples/sample_app
> > $ merb -p 4000
> >
> >         Then go to http://localhost:4000/files/start in your browser to
> > see
> > an example of the upload progress. Sice it is running locally you
> > shoudl try to upload a file at least 100Mb or bigger or else it will
> > happen too fast and you won't see the progress bar.
> >
> >         The nice thing about the way merb works is that it doesn't use
> > cgi.rb for mime parsing and it is written in a more thread safe way
> > then actionpack. So multiple concurrent requests can be processed and
> > have their mime parsing done all at the same time. Merb only does a
> > mutex around the smallest possible place where you might be calling
> > ActiveRecord code. With the rails dispatcher, the mutex surrounds the
> > entire routing code and mime parsing and your controler code. Merb
> > only locks around the action in your controller code. It does route
> > recognition and mime parsing outside the mutex in a multi threaded
> > way. This allows us to use ActiveRecord in single threaded mode which
> > is better then AR's multi threaded mode for performance. So we use
> > the smallest sized lock we can in order to still support this, but we
> > do all the heavyweight routing and mime parsing concurrently.
> >
> >         I promised Zed I would do a write up about using a merb app for
> > a
> > dedicated upload server in conjunction with a rails app. I will try
> > to get something written this week and posted. Until then I am open
> > to and questions about merb and you can file tickets for support or
> > feature requests here:
> >
> > http://merb.devjavu.com/
> > http://merb.devjavu.com/projects/merb/register
> >
> >         Zed and I hope to come up with a solution so that we can use the
> >
> > upload progress without reparsing the mime twice per file. Right now
> > mongrel streams the upload to a tmp file and increments the progress
> > bar all the way to 100% before it hands control to your rails or merb
> > controller to deal with the file. But then rails or merb has to parse
> > the mime boundaries of the entire file again. So you may notice with
> > big files that right when the progress bar makes it to 95-100% it
> > will pause while rails or meb parses the mime again into another
> > tmpfile. This is obviously not an optimal way to do this. Zed has
> > started an awesome fast C mime carver and I will be working with him
> > to integrate this so uploads can be parsed once with the fast C mimer
> > and then just handed off to the next handler with no need to reparse.
> >
> > Cheers-
> >
> > -- Ezra Zygmuntowicz
> > -- Lead Rails Evangelist
> > -- ez at engineyard.com
> > -- Engine Yard, Serious Rails Hosting
> > -- (866) 518-YARD (9273)
> >
> >
> > _______________________________________________
> > Mongrel-users mailing list
> > Mongrel-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/mongrel-users
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20061129/4892344c/attachment.html 


More information about the Mongrel-users mailing list