From bunny-amqp-devel at rubyforge.org Fri Aug 7 20:22:07 2009 From: bunny-amqp-devel at rubyforge.org (Bunny AMQP client mailing list) Date: Sat, 8 Aug 2009 01:22:07 +0100 Subject: [bunny-amqp-devel] [ANN] New Bunny release v0.5.1 Message-ID: <499614D1-76E7-4187-9CFD-2050767F0BF1@gmail.com> I thought it was time to unleash the latest release of Bunny. It should have been v0.5.0 but I messed up the versioning update :-\ Highlights of this release - * User can choose whether to instantiate an AMQP 0-8 or 0-9-1 spec version of the Bunny client with the :spec argument. The default will be 0-8. * New Bunny::Channel class enables easy channel creation. * New :frame_max and :channel_max parameters in Bunny::Client.new method. * New client methods tx_select, tx_commit and tx_rollback implement transaction functionality. A little explanation might be in order re. the :spec argument. If you omit the :spec argument in a Bunny#new call you will instantiate an AMQP 0-8 version of Bunny which works with the current stable release of RabbitMQ. If you do something like this - Bunny.new(:spec => '0.9') Then you will instantiate an AMQP 0-9-1 version of Bunny. The string in the method call doesn't have to be '0.9'. It can be anything that isn't '08'. The AMQP 0-9-1 version of Bunny only works with development versions of RabbitMQ. So it's sort of experimental, but by all means give it a run if you're feeling brave and let me know how you get on. As usual if you have any problems with this release please raise an issue on Github - http://github.com/celldee/bunny Thank you for your support From bunny-amqp-devel at rubyforge.org Tue Aug 11 11:49:36 2009 From: bunny-amqp-devel at rubyforge.org (Bunny AMQP client mailing list) Date: Tue, 11 Aug 2009 16:49:36 +0100 Subject: [bunny-amqp-devel] Queue#subscribe and heartbeat monitoring Message-ID: <65107EF2-F069-4FF2-BDA4-505DFA285C97@gmail.com> I've been making changes to Bunny - 1. Added a timeout parameter to the Queue#subscribe method. 2. Added a hearbeat parameter to the Client#new method. If anybody is interested in using either of these they are in the master branch in GitHub (http://github.com/celldee/bunny). I will create a couple of gists on GitHub to illustrate the use of these new parameters. Should you use/test them and find problems, room for improvement or just want to tell me that they work - please feel free. Regards, Chris From bunny-amqp-devel at rubyforge.org Wed Aug 12 09:38:59 2009 From: bunny-amqp-devel at rubyforge.org (Bunny AMQP client mailing list) Date: Wed, 12 Aug 2009 06:38:59 -0700 (PDT) Subject: [bunny-amqp-devel] Handling connection errors In-Reply-To: <3de9aeb50908111808q59d65a39j851a0058fb2cc12c@mail.gmail.com> References: <3de9aeb50908110900r466f4fbfj86b1013be36b92e3@mail.gmail.com> <7218d2b60908111100t227f7facnb00a9b5659f96289@mail.gmail.com> <3de9aeb50908111808q59d65a39j851a0058fb2cc12c@mail.gmail.com> Message-ID: <28e15d91-0fb3-4f9d-b8d8-841af9960757@b15g2000yqd.googlegroups.com> Hi Mark, On Aug 12, 2:08?am, Bunny AMQP client mailing list wrote: > Thanks, Dan, I'll check over there. ?Did my rescue and retry code look > reasonable, or can you think of anything else I should do for more robust > error handling? > --Mark Thanks for using Bunny. As I said over in the RabbitMQ group, I've made a change or two re. heartbeat that you're welcome to try, however, my work is not yet done. I have to incorporate some useful heartbeat logic into the Bunny API, because at the moment you would have to code that logic yourself. If you have any ideas about how that logic might be implemented I'd be glad to hear them. As far as your rescue and retry code goes, I'll take a better look at it to see if I can come up with any suggestions. I haven't really done anything with Thread.current in the past so I'm not sure if that has any bearing on your problem. I did stumble across this article - http://coderrr.wordpress.com/2008/04/10/lets-stop-polluting-the-threadcurrent-hash/ - again I'm not sure if there's anything in there that might be useful. Regards, Chris > > On Wed, Aug 12, 2009 at 2:00 AM, Bunny AMQP client mailing list < > > bunny-amqp-de... at rubyforge.org> wrote: > > Mark,What I believe you're seeing is this: When you start an AMQP client > > (any one) and connect to RabbitMQ, it starts a new Erlang process to handle > > that connection. If you lose that connection and start a new one, you'll get > > a new Erlang process. I'm not at all familiar with RabbitMQ's process for > > handling this or if there's a tunable parameter you could set to kill these > > abandoned processes more quickly. You might want to ask on the RabbitMQ list > > about this. > > > Thanks, > > Dan > > > On Tue, Aug 11, 2009 at 10:00 AM, Bunny AMQP client mailing list < > > bunny-amqp-de... at rubyforge.org> wrote: > > >> I'm using Bunny in a Rails application (using Passenger/nginx and Rails > >> 2.3.2 on CentOS 5) to send messages to RabbitMQ on a separate server for > >> offline processing. ?This application is being deployed in a few different > >> locations and it's been working great except in one place. > >> The client software does something like this: > > >> retries = 4 > >> begin > >> ? b = (Thread.current[:bunny] ||= Bunny(:host => HOSTNAME)) > >> ? b.start unless b.status == :connected > >> ? q = (Thread.current[qname] ||= b.queue(qname)) > >> ? q.publish(message) > >> rescue Exception => e > >> ? logger.info("Error: #{e.message}") > >> ? Thread.current[:bunny] = nil > >> ? Thread.current[qname] = nil > >> ? retry if (retries -= 1) >=0 > >> end > > >> This works fine on most systems and I never see the "Error". ?On one > >> system, though, I'm seeing periodic "Connection reset by perr" and "Broken > >> pipe". ?On the rabbitmq server, using lsof, ?I'm seeing a slow but > >> continuously growing number of sockets with state "connected" that are > >> apparently not connected to anything on the client side. ?After about one > >> day, the number of sockets grows to about 900, while there are only 15 > >> instances of the application on the client. ?The number of sockets on the > >> client grows too, but much more slowly (a total of around 50, with some app > >> instances showing 4-5 sockets). > > >> I've compared these two problem servers to working servers at other > >> locations and I can't figure out what the difference could be. ?I've checked > >> all network kernel variables, checked the network interfaces for errors, > >> checked the versions of all Rails gems, checked the versions of rabbitmq and > >> erlang, and so on. > > >> Questions: > > >> 1. Is my error code handling appropriate? ?If I rescue an exception on a > >> bunny connection, is there something I should do with either the connection > >> or the queue variables to ensure they're closed? > > >> 2. Any suggestions on where to look next? ?I ran tshark but didn't see > >> anything unusual (or I didn't know what to look for) > > >> Thanks, > >> Mark > > >> _______________________________________________ > >> bunny-amqp-devel mailing list > >> bunny-amqp-de... at rubyforge.org > >>http://rubyforge.org/mailman/listinfo/bunny-amqp-devel > > > _______________________________________________ > > bunny-amqp-devel mailing list > > bunny-amqp-de... at rubyforge.org > >http://rubyforge.org/mailman/listinfo/bunny-amqp-devel > > > > _______________________________________________ > bunny-amqp-devel mailing list > bunny-amqp-de... at rubyforge.orghttp://rubyforge.org/mailman/listinfo/bunny-amqp-devel From bunny-amqp-devel at rubyforge.org Thu Aug 13 11:34:13 2009 From: bunny-amqp-devel at rubyforge.org (Bunny AMQP client mailing list) Date: Thu, 13 Aug 2009 16:34:13 +0100 Subject: [bunny-amqp-devel] Channels and connections Message-ID: Hi, Up until now there has been a 1:1 relationship between channels and connections in Bunny. I think that it would be useful to be able to use multiple channels with a connection, therefore I am implementing a simple way to achieve this. This is what I have in mind. 1. The Bunny::Channel class will have an array named frame_buffer. 2. When Bunny::Client does a next_frame call it will first check to see if there are any frames in the current Channel frame_buffer. If there are any, it will return the first one. If not, it will do what it does now to get the next_frame. If that frame has the same channel number as the current Channel, the frame will be returned. Otherwise, the frame will be stored in the frame_buffer of the relevant Channel and the next frame will be read. I think that this will work. Of course, let me know if you have a better way to do this. Any thoughts/questions on the subject would be appreciated. Regards, Chris From bunny-amqp-devel at rubyforge.org Thu Aug 13 12:47:03 2009 From: bunny-amqp-devel at rubyforge.org (Bunny AMQP client mailing list) Date: Thu, 13 Aug 2009 09:47:03 -0700 (PDT) Subject: [bunny-amqp-devel] Handling connection errors In-Reply-To: <3de9aeb50908121739p333d99adu471eba0e6e60be3b@mail.gmail.com> References: <3de9aeb50908110900r466f4fbfj86b1013be36b92e3@mail.gmail.com> <7218d2b60908111100t227f7facnb00a9b5659f96289@mail.gmail.com> <3de9aeb50908111808q59d65a39j851a0058fb2cc12c@mail.gmail.com> <28e15d91-0fb3-4f9d-b8d8-841af9960757@b15g2000yqd.googlegroups.com> <3de9aeb50908121739p333d99adu471eba0e6e60be3b@mail.gmail.com> Message-ID: Mark On Aug 13, 1:39?am, Bunny AMQP client mailing list wrote: > Chris, > Thanks for writing Bunny and making it available. ?Until I discovered a few > months ago I'd been using STOMP, but this required building RabbitMQ with > the STOMP adapter and made deployments messier. > > I think most objections to Thread.current are that it's a global variable so > (a) you could end up stepping on someone's toes (especially if it's in a > library), and (b) it's just not a good design practice. ?In my case, I'm > using it only as a cache within a single model of the main application. > > I was worried more about whether I need to do something to clean up after a > broken connection. ?Right now, I'm keeping (caching) two variables: an > instance of the Bunny object and an instance of a queue. ?If there's an > exception, the only thing I know how to do is throw those objects away and > create new ones. ?What I'm thinking about now is adding something like this > inside the rescue block > > begin > ? [...] > rescue > ? Thread.current[:bunny].stop if Thread.current[:bunny] rescue > ? [...] > end > > This attempt to stop and close the current Bunny object and if that fails, > the rescue line modifier catches it and lets me keep going. ?Hope this makes > sense. > > --Mark > Yup, it does make sense. I'd be interested to hear if it solves your problem. Cheers, Chris > On Wed, Aug 12, 2009 at 9:38 PM, Bunny AMQP client mailing list < > > bunny-amqp-de... at rubyforge.org> wrote: > > Hi Mark, > > > On Aug 12, 2:08 am, Bunny AMQP client mailing list > de... at rubyforge.org> wrote: > > > Thanks, Dan, I'll check over there. ?Did my rescue and retry code look > > > reasonable, or can you think of anything else I should do for more robust > > > error handling? > > > --Mark > > > Thanks for using Bunny. As I said over in the RabbitMQ group, I've > > made a change or two re. heartbeat that you're welcome to try, > > however, my work is not yet done. I have to incorporate some useful > > heartbeat logic into the Bunny API, because at the moment you would > > have to code that logic yourself. If you have any ideas about how that > > logic might be implemented I'd be glad to hear them. > > > As far as your rescue and retry code goes, I'll take a better look at > > it to see if I can come up with any suggestions. I haven't really done > > anything with Thread.current in the past so I'm not sure if that has > > any bearing on your problem. I did stumble across this article - > > >http://coderrr.wordpress.com/2008/04/10/lets-stop-polluting-the-threa... > > - again I'm not sure if there's anything in there that might be > > useful. > > > Regards, > > > Chris > > > > On Wed, Aug 12, 2009 at 2:00 AM, Bunny AMQP client mailing list < > > > > bunny-amqp-de... at rubyforge.org> wrote: > > > > Mark,What I believe you're seeing is this: When you start an AMQP > > client > > > > (any one) and connect to RabbitMQ, it starts a new Erlang process to > > handle > > > > that connection. If you lose that connection and start a new one, > > you'll get > > > > a new Erlang process. I'm not at all familiar with RabbitMQ's process > > for > > > > handling this or if there's a tunable parameter you could set to kill > > these > > > > abandoned processes more quickly. You might want to ask on the RabbitMQ > > list > > > > about this. > > > > > Thanks, > > > > Dan > > > > > On Tue, Aug 11, 2009 at 10:00 AM, Bunny AMQP client mailing list < > > > > bunny-amqp-de... at rubyforge.org> wrote: > > > > >> I'm using Bunny in a Rails application (using Passenger/nginx and > > Rails > > > >> 2.3.2 on CentOS 5) to send messages to RabbitMQ on a separate server > > for > > > >> offline processing. ?This application is being deployed in a few > > different > > > >> locations and it's been working great except in one place. > > > >> The client software does something like this: > > > > >> retries = 4 > > > >> begin > > > >> ? b = (Thread.current[:bunny] ||= Bunny(:host => HOSTNAME)) > > > >> ? b.start unless b.status == :connected > > > >> ? q = (Thread.current[qname] ||= b.queue(qname)) > > > >> ? q.publish(message) > > > >> rescue Exception => e > > > >> ? logger.info("Error: #{e.message}") > > > >> ? Thread.current[:bunny] = nil > > > >> ? Thread.current[qname] = nil > > > >> ? retry if (retries -= 1) >=0 > > > >> end > > > > >> This works fine on most systems and I never see the "Error". ?On one > > > >> system, though, I'm seeing periodic "Connection reset by perr" and > > "Broken > > > >> pipe". ?On the rabbitmq server, using lsof, ?I'm seeing a slow but > > > >> continuously growing number of sockets with state "connected" that are > > > >> apparently not connected to anything on the client side. ?After about > > one > > > >> day, the number of sockets grows to about 900, while there are only 15 > > > >> instances of the application on the client. ?The number of sockets on > > the > > > >> client grows too, but much more slowly (a total of around 50, with > > some app > > > >> instances showing 4-5 sockets). > > > > >> I've compared these two problem servers to working servers at other > > > >> locations and I can't figure out what the difference could be. ?I've > > checked > > > >> all network kernel variables, checked the network interfaces for > > errors, > > > >> checked the versions of all Rails gems, checked the versions of > > rabbitmq and > > > >> erlang, and so on. > > > > >> Questions: > > > > >> 1. Is my error code handling appropriate? ?If I rescue an exception on > > a > > > >> bunny connection, is there something I should do with either the > > connection > > > >> or the queue variables to ensure they're closed? > > > > >> 2. Any suggestions on where to look next? ?I ran tshark but didn't see > > > >> anything unusual (or I didn't know what to look for) > > > > >> Thanks, > > > >> Mark > > > > >> _______________________________________________ > > > >> bunny-amqp-devel mailing list > > > >> bunny-amqp-de... at rubyforge.org > > > >>http://rubyforge.org/mailman/listinfo/bunny-amqp-devel > > > > > _______________________________________________ > > > > bunny-amqp-devel mailing list > > > > bunny-amqp-de... at rubyforge.org > > > >http://rubyforge.org/mailman/listinfo/bunny-amqp-devel > > > > _______________________________________________ > > > bunny-amqp-devel mailing list > > > bunny-amqp-de... at rubyforge.orghttp:// > > rubyforge.org/mailman/listinfo/bunny-amqp-devel > > _______________________________________________ > > bunny-amqp-devel mailing list > > bunny-amqp-de... at rubyforge.org > >http://rubyforge.org/mailman/listinfo/bunny-amqp-devel > > > > _______________________________________________ > bunny-amqp-devel mailing list > bunny-amqp-de... at rubyforge.orghttp://rubyforge.org/mailman/listinfo/bunny-amqp-devel From bunny-amqp-devel at rubyforge.org Mon Aug 17 09:34:04 2009 From: bunny-amqp-devel at rubyforge.org (Bunny AMQP client mailing list) Date: Mon, 17 Aug 2009 14:34:04 +0100 Subject: [bunny-amqp-devel] [ANN] New release of Bunny v0.5.2 Message-ID: <26BB9D6B-52AC-4AD9-9D72-61FD08F2BBAD@gmail.com> Hi, There is a new release of Bunny available from the usual outlets. It contains the following highlights that you may or may not find useful - Add :heartbeat parameter to Client#new method and some rudimentary heartbeat logic Add :timeout parameter to Queue#subscribe method Add Client#returned_message method to handle undeliverable returned messages Tidy up the connection logic so that appropriate error messages are output If you have any issues please report them on GitHub. If you have any questions etc. I'll be happy to try to answer them. Thank you for using Bunny. Regards, Chris