Support Requests: Browse | Submit New | Admin

[#8162] Multi Threading Performance

Date:
2007-01-26 01:14
Priority:
3
Submitted By:
Nobody
Assigned To:
Marcel Molina Jr. (marcel)
Category:
None
State:
Open
Summary:
Multi Threading Performance

Detailed description
Hey,

I just started using Ruby on Rails and have started to use different Gems such as this one as well as backgroundrb.  

I'm getting errors with aws-s3 when storing objects on s3.  I only have issues when 2 different threads try to store
something on s3 at the same time.  This is a newb question, but I'll ask it anyways:  how could i solve that?  I need
to allow 2 different threads to store something on s3 at the same time.  What happens now is one fails and the other
just continues.

If you care to help or this is the wrong place for this, just delete the request.  


This is my test script:

require '/usr/local/lib/ruby/gems/1.8/gems/aws-s3-0.3.0/lib/aws/s3'

class D7Obj < AWS::S3::S3Object
    set_current_bucket_to "<some_bucket>"
end 

class S3Handler

      AWS::S3::Base.establish_connection!(
        :access_key_id     => '<some_key>',
        :secret_access_key => '<some_secret>',
        :persistent => true
      )
    
    def connected
      if AWS::S3::Base.connected? == true
        return true
      else
        return false
      end
    end
        
    def save_file(file, filename, access_level = "public-read")  
        return D7Obj.store(filename, file, :access => :public_read) unless connected == false
    end
    
  end
    
  s3 = S3Handler.new


threads = []
5.times { |i|
  threads << Thread.new(i) {
  	s3 = S3Handler.new
  	s3.save_file(open('pics/2.jpg'), "#{i}.jpg")
  	puts("finished saving file #{i}.jpg")
  }
}
threads.each {|t| t.join }




I get this error:

/usr/local/lib/ruby/1.8/net/protocol.rb:133:in `sysread': end of file reached (EOFError)
        from test2.rb:40:in `join'
        from test2.rb:40
        from test2.rb:40:in `each'
        from test2.rb:40

Add A Comment: Notepad

Please login


Followup

Message
Date: 2007-01-26 17:28
Sender: Tom F

thx for getting back to me Marcel.  Much appreciated.  A new
day, a new challenge...I'll have to try to find a way to handle
this yet.
Date: 2007-01-26 13:59
Sender: Marcel Molina Jr.

Indeed, AWS::S3 isn't currently thread safe.

Attached Files:

Name Description Download
No Files Currently Attached

Changes:

Field Old Value Date By
assigned_tonone2007-01-26 13:59marcel