From rogboone at yahoo.com  Thu Feb  7 12:11:11 2008
From: rogboone at yahoo.com (Roggie Boone)
Date: Thu, 7 Feb 2008 09:11:11 -0800 (PST)
Subject: [Swiftiply-users] [Newbie] swiftiply not starting -- error
Message-ID: <402718.6475.qm@web32513.mail.mud.yahoo.com>

I've just installed Swiftiply 0.6.1.1 (via gem) on  a fresh
Ubuntu 7.10 server.  When I try to start it, I get an error
about NilClass and the "split" function.  I've seen this error
posted elsewhere (via a google search), but never saw a 
solution posted.  Anyone have any ideas?  Thanks in
advance.

Current  gems:

eventmachine (0.10.0)
mongrel (1.1.3)
rails (2.0.2)
swiftiply (0.6.1.1)

I get this error when I try to start swiftiply (my config is below
as well);

# swiftiply -c /etc/swiftiply.conf

/var/lib/gems/1.8/gems/swiftiply-0.6.1.1/src/swiftcore/Swiftiply.rb:717:in `em_config': private method `split' called for nil:NilClass (NoMethodError)
        from /var/lib/gems/1.8/gems/swiftiply-0.6.1.1/src/swiftcore/Swiftiply.rb:708:in `each'
        from /var/lib/gems/1.8/gems/swiftiply-0.6.1.1/src/swiftcore/Swiftiply.rb:708:in `em_config'
        from /var/lib/gems/1.8/gems/swiftiply-0.6.1.1/src/swiftcore/Swiftiply.rb:676:in `each'
        from /var/lib/gems/1.8/gems/swiftiply-0.6.1.1/src/swiftcore/Swiftiply.rb:676:in `em_config'
        from /var/lib/gems/1.8/gems/swiftiply-0.6.1.1/src/swiftcore/Swiftiply.rb:666:in `each'
        from /var/lib/gems/1.8/gems/swiftiply-0.6.1.1/src/swiftcore/Swiftiply.rb:666:in `em_config'
        from /var/lib/gems/1.8/gems/swiftiply-0.6.1.1/src/swiftcore/Swiftiply.rb:632:in `run'
        from /var/lib/gems/1.8/gems/eventmachine-0.10.0/lib/eventmachine.rb:1086:in `call'
        from /var/lib/gems/1.8/gems/eventmachine-0.10.0/lib/eventmachine.rb:1086:in `event_callback'
        from /var/lib/gems/1.8/gems/eventmachine-0.10.0/lib/eventmachine.rb:224:in `run_machine'
        from /var/lib/gems/1.8/gems/eventmachine-0.10.0/lib/eventmachine.rb:224:in `run'
        from /var/lib/gems/1.8/gems/swiftiply-0.6.1.1/src/swiftcore/Swiftiply.rb:629:in `run'
        from /var/lib/gems/1.8/gems/swiftiply-0.6.1.1/bin/swiftiply:139:in `run'
        from /var/lib/gems/1.8/gems/swiftiply-0.6.1.1/bin/swiftiply:149
        from /var/lib/gems/1.8/bin/swiftiply:16:in `load'
        from /var/lib/gems/1.8/bin/swiftiply:16
PID 6736



Here is my swiftiply.conf (cluster address is the IP of the server,
but I've taken it out for this posting):

cluster_address: xxx.xxx.xx.xx
cluster_port: 80
daemonize: true
epoll: true
epoll_descriptors: 20000
map:
  - incoming:
    - localhost
outgoing: 127.0.0.1:30000
default: true
docroot: /opt/www/myapp/current
cache_directory: public
redeployable: true



       
---------------------------------
Looking for last minute shopping deals?  Find them fast with Yahoo! Search.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/swiftiply-users/attachments/20080207/d59701d5/attachment.html 

From wyhaines at gmail.com  Thu Feb  7 13:14:35 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Thu, 7 Feb 2008 11:14:35 -0700
Subject: [Swiftiply-users] [Newbie] swiftiply not starting -- error
In-Reply-To: <402718.6475.qm@web32513.mail.mud.yahoo.com>
References: <402718.6475.qm@web32513.mail.mud.yahoo.com>
Message-ID: <f4cd26df0802071014r35390fcdm63ebf5c65088fdc1@mail.gmail.com>

On Feb 7, 2008 10:11 AM, Roggie Boone <rogboone at yahoo.com> wrote:
> Here is my swiftiply.conf (cluster address is the IP of the server,
> but I've taken it out for this posting):
>
> cluster_address: xxx.xxx.xx.xx
> cluster_port: 80
> daemonize: true
> epoll: true
> epoll_descriptors: 20000
> map:
>   - incoming:
>     - localhost
> outgoing: 127.0.0.1:30000
> default: true
> docroot: /opt/www/myapp/current
> cache_directory: public
> redeployable: true


Just on first glance, if the spacing has been preserved correctly
there, that may be the problem.  Whitespace with this style of config
file (YAML file) is syntactically significant.

cluster_address: xxx.xxx.xx.xx
cluster_port: 80
daemonize: true
epoll: true
epoll_descriptors: 20000
map:
  - incoming:
    - localhost
    outgoing: 127.0.0.1:30000
    docroot: /opt/www/myapp/current
    redeployable: true
    cache_directory: public


A couple of notes/questions that occur to me, too, when looking at this.

1) Are your incoming requests really going to be for

Host: localhost

2) Understand that request redeployablity, when enabled, means that
you need to make sure your app won't do bad things if it happens.
i.e. if you have a request being processed, and that processing
changes values in the database in a non-atomic way.  Multiple separate
updates to the db without a db-side transaction, for example, and the
process dies or is killed in the middle of said processing, what
happens when another backend gets the redeployed request, and the
sequence of events starts over?  The ability to make requests
redeployable so that they aren't lost due to hardware or software
failure can be useful, but it can also be dangerous if you don't think
through how you are using it.


Hope this helps.

Kirk Haines

From jnicoll at gnexp.com  Thu Feb  7 13:35:26 2008
From: jnicoll at gnexp.com (Jeremy Nicoll)
Date: Thu, 07 Feb 2008 11:35:26 -0700
Subject: [Swiftiply-users] [Newbie] swiftiply not starting -- error
In-Reply-To: <f4cd26df0802071014r35390fcdm63ebf5c65088fdc1@mail.gmail.com>
References: <402718.6475.qm@web32513.mail.mud.yahoo.com>
	<f4cd26df0802071014r35390fcdm63ebf5c65088fdc1@mail.gmail.com>
Message-ID: <47AB4F6E.7010702@gnexp.com>

    To take care of the 2nd potential issue, you can use databases that 
use transactions (InnoDB in MYSQL is one), and then wrap everything that 
you want to happen in a transaction like so in Ruby on Rails (if that's 
what you are using):

@user = User.find(params[:id])
address = @user.address  # A custom relation set up in the User model.

User.transaction do
    @user.update_attributes!(params[:user])
    address.update_attributes!(params[:address])
end

    This way, if the process fails before the last transaction for any 
reason (it dies, one of the records is invalid, etc), nothing happens to 
your database because the "COMMIT" sql command has not been run yet.

--
Jeremy Nicoll

Kirk Haines wrote:
> On Feb 7, 2008 10:11 AM, Roggie Boone <rogboone at yahoo.com> wrote:
>   
>> Here is my swiftiply.conf (cluster address is the IP of the server,
>> but I've taken it out for this posting):
>>
>> cluster_address: xxx.xxx.xx.xx
>> cluster_port: 80
>> daemonize: true
>> epoll: true
>> epoll_descriptors: 20000
>> map:
>>   - incoming:
>>     - localhost
>> outgoing: 127.0.0.1:30000
>> default: true
>> docroot: /opt/www/myapp/current
>> cache_directory: public
>> redeployable: true
>>     
>
>
> Just on first glance, if the spacing has been preserved correctly
> there, that may be the problem.  Whitespace with this style of config
> file (YAML file) is syntactically significant.
>
> cluster_address: xxx.xxx.xx.xx
> cluster_port: 80
> daemonize: true
> epoll: true
> epoll_descriptors: 20000
> map:
>   - incoming:
>     - localhost
>     outgoing: 127.0.0.1:30000
>     docroot: /opt/www/myapp/current
>     redeployable: true
>     cache_directory: public
>
>
> A couple of notes/questions that occur to me, too, when looking at this.
>
> 1) Are your incoming requests really going to be for
>
> Host: localhost
>
> 2) Understand that request redeployablity, when enabled, means that
> you need to make sure your app won't do bad things if it happens.
> i.e. if you have a request being processed, and that processing
> changes values in the database in a non-atomic way.  Multiple separate
> updates to the db without a db-side transaction, for example, and the
> process dies or is killed in the middle of said processing, what
> happens when another backend gets the redeployed request, and the
> sequence of events starts over?  The ability to make requests
> redeployable so that they aren't lost due to hardware or software
> failure can be useful, but it can also be dangerous if you don't think
> through how you are using it.
>
>
> Hope this helps.
>
> Kirk Haines
> _______________________________________________
> Swiftiply-users mailing list
> Swiftiply-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/swiftiply-users
>
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/swiftiply-users/attachments/20080207/23b0b6ab/attachment.html 

From jro at codegrinder.com  Tue Feb 12 15:22:15 2008
From: jro at codegrinder.com (Jason Rohwedder)
Date: Tue, 12 Feb 2008 14:22:15 -0600
Subject: [Swiftiply-users] Swiftiply process died - 0.6.1.1
Message-ID: <731bcaae0802121222x35e21f9dmd471c8f13075243e@mail.gmail.com>

I had the swifiply process die on me spontaneously earlier today.  If I were
to run it in the foreground will I be able to see any stack trace of what
tripped up if it were to happen again?  I'm not seeing any logging options
in the source or the documentation so far.  Thanks!
-jro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/swiftiply-users/attachments/20080212/7a4ec255/attachment.html 

From wyhaines at gmail.com  Tue Feb 12 15:32:40 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Tue, 12 Feb 2008 13:32:40 -0700
Subject: [Swiftiply-users] Swiftiply process died - 0.6.1.1
In-Reply-To: <731bcaae0802121222x35e21f9dmd471c8f13075243e@mail.gmail.com>
References: <731bcaae0802121222x35e21f9dmd471c8f13075243e@mail.gmail.com>
Message-ID: <f4cd26df0802121232m23d21580q5cdc5f650dddcc98@mail.gmail.com>

On Feb 12, 2008 1:22 PM, Jason Rohwedder <jro at codegrinder.com> wrote:
> I had the swifiply process die on me spontaneously earlier today.  If I were
> to run it in the foreground will I be able to see any stack trace of what
> tripped up if it were to happen again?  I'm not seeing any logging options
> in the source or the documentation so far.  Thanks!

What EM version/Swiftiply version?  What platform?  And are you using
the epoll support?


Thanks,

Kirk Haines

From jro at codegrinder.com  Tue Feb 12 16:14:41 2008
From: jro at codegrinder.com (Jason Rohwedder)
Date: Tue, 12 Feb 2008 15:14:41 -0600
Subject: [Swiftiply-users] Swiftiply process died - 0.6.1.1
In-Reply-To: <f4cd26df0802121247s97f53d9y54cb50108497f3ea@mail.gmail.com>
References: <731bcaae0802121222x35e21f9dmd471c8f13075243e@mail.gmail.com>
	<f4cd26df0802121232m23d21580q5cdc5f650dddcc98@mail.gmail.com>
	<731bcaae0802121239p75d89452s900411c6d00229f8@mail.gmail.com>
	<f4cd26df0802121247s97f53d9y54cb50108497f3ea@mail.gmail.com>
Message-ID: <731bcaae0802121314r717e2f0ay1fbbc85234f0f8d@mail.gmail.com>

On 2/12/08, Kirk Haines <wyhaines at gmail.com> wrote:
> On Feb 12, 2008 1:39 PM, Jason Rohwedder <jro at codegrinder.com> wrote:
>
>  > Kirk,
>  > from gems:
>  >  eventmachine (0.10.0)
>  >  swiftiply (0.6.1.1)
>  >
>  > Red Hat Enterprise Linux ES release 4 (Nahant Update 6) (32bit)
>  >
>  > And we're using epoll settings:
>  >  epoll: true
>  >  epoll_descriptors: 20000
>  >
>  > Thanks!
>
>
> It's almost certainly an uncatchable EM bug.  It has been fixed in the
>  EM trunk, and there is a 0.11 release coming soon that will have it
>  fixed.
>
>  If you can, the short term fix is to build EM from trunk and run that one.
>
Kirk,

Thanks for the tip.  Do you happen to know what bug # or svn commit I
can reference for that bug?  There are a lot of changes between 0.10.0
and trunk.  I'd like to get an idea of what's changing and if we could
just apply that patch to the current stable.  Thanks

-jason

From jro at codegrinder.com  Tue Feb 12 17:38:02 2008
From: jro at codegrinder.com (Jason Rohwedder)
Date: Tue, 12 Feb 2008 16:38:02 -0600
Subject: [Swiftiply-users] Swiftiply process died - 0.6.1.1
In-Reply-To: <731bcaae0802121314r717e2f0ay1fbbc85234f0f8d@mail.gmail.com>
References: <731bcaae0802121222x35e21f9dmd471c8f13075243e@mail.gmail.com>
	<f4cd26df0802121232m23d21580q5cdc5f650dddcc98@mail.gmail.com>
	<731bcaae0802121239p75d89452s900411c6d00229f8@mail.gmail.com>
	<f4cd26df0802121247s97f53d9y54cb50108497f3ea@mail.gmail.com>
	<731bcaae0802121314r717e2f0ay1fbbc85234f0f8d@mail.gmail.com>
Message-ID: <731bcaae0802121438r2d1df8d3x8911a000e2b27b2f@mail.gmail.com>

On 2/12/08, Jason Rohwedder <jro at codegrinder.com> wrote:
> On 2/12/08, Kirk Haines <wyhaines at gmail.com> wrote:
>
> > On Feb 12, 2008 1:39 PM, Jason Rohwedder <jro at codegrinder.com> wrote:
>  >
>  >  > Kirk,
>  >  > from gems:
>  >  >  eventmachine (0.10.0)
>  >  >  swiftiply (0.6.1.1)
>  >  >
>  >  > Red Hat Enterprise Linux ES release 4 (Nahant Update 6) (32bit)
>  >  >
>  >  > And we're using epoll settings:
>  >  >  epoll: true
>  >  >  epoll_descriptors: 20000
>  >  >
>  >  > Thanks!
>  >
>  >
>  > It's almost certainly an uncatchable EM bug.  It has been fixed in the
>  >  EM trunk, and there is a 0.11 release coming soon that will have it
>  >  fixed.
>  >
>  >  If you can, the short term fix is to build EM from trunk and run that one.
>  >
>
> Kirk,
>
>  Thanks for the tip.  Do you happen to know what bug # or svn commit I
>  can reference for that bug?  There are a lot of changes between 0.10.0
>  and trunk.  I'd like to get an idea of what's changing and if we could
>  just apply that patch to the current stable.  Thanks
>

Just as a follow up .. It happened again and I was able to capture the
error message

terminate called after throwing an instance of 'std::runtime_error'
what(): unable to delete epoll event: Bad file descriptor

From chetan.muneshwar at betterlabs.net  Wed Feb 13 01:29:58 2008
From: chetan.muneshwar at betterlabs.net (Chetan Muneshwar - BetterLabs)
Date: Wed, 13 Feb 2008 11:59:58 +0530
Subject: [Swiftiply-users] Swiftiply-users Digest, Vol 8, Issue 1
In-Reply-To: <mailman.344.1202855892.11700.swiftiply-users@rubyforge.org>
References: <mailman.344.1202855892.11700.swiftiply-users@rubyforge.org>
Message-ID: <f34743be0802122229j543b694cyaf13d92ca4a4a65@mail.gmail.com>

hello friends
Everything is fine swiftiply events based mechanism rocking

but i have one problem of" bad file descriptor error" and swift stop working
i need good solution for that


there is no any controlling utility for swift to monitor

here what i done through shell script
7001 = port for swiftiply
7002,7003,7004= port for mongrel cluster

1) for finding pid:
[
#! /bin/bash
PROCNAME=$1
PIDS=`ps -efa | grep $PROCNAME | grep -v grep | awk '{ print $2 }'`
for ff in $PIDS
do
   echo "$ff"
done
]

2) its how i control swift processes

#! /bin/bash
TIMESTAMP=$(date +%Y-%m-%d.%H%M)
mongo=$(pii /etc/SWT/7001.dp.yml)

if [ "$mongo" ==  "" ] ;then
                        myswift dp 3
                       php /root/sendmessage.php " new pid are $mongo"
"92323423403"
                else
                       echo "running $TIMESTAMP" >>/etc/chkdp

fi


3) this is how i check for memory usage to avoid bad flie desriptor error
          check for events ( having same port identity) to maintain
memory usage threshold and kindly restart swift events

#! /bin/bash
p1=$(pii /etc/SWT/7001.myapp.yml)
s1=$(pii 45002)
swidp=$( ps -o vsz $s1 |cut -d " " -f 4 )
echo $swidp >/tmp/$$$12rt
myarr=(`cat /tmp/$$$12rt`)

for ((  i = 0 ;  i < ${#myarr[*]};  i++  ))
                do

                        if [ `echo ${myarr[i]}` -gt 150000 ] ;then
                            kill -9  $p1
                            dpsif
                else
                                echo "well no P"
                        fi
                done

rm -rf /tmp/$$$12rt



4) At the same time i used mongrel &swiftply events for application to run
    apache mod_proxy balancer serving to both swift cluster port and mongrel
    port  in round robbin & byrequest .



5)  Tts how i control mongrels_clusters

[
#! /bin/bash

p1=$(pii mongrel.7002.pid)
virmem1=$(ps -o vsz $p1 |cut -d " " -f 4)
echo $virmem1
if [ $virmem1 -gt 150000 ] ;then
        kill -9 $p1
        dpstart
else
        echo "hi"
fi
p2=$(pii mongrel.7003.pid)
virmem2=$(ps -o vsz $p2 |cut -d " " -f 4)
echo $virmem2
if [ $virmem2 -gt 150000 ] ;then
        kill -9 $p2
        dpstart
else
        echo "hi"
fi
p3=$(pii mongrel.7004.pid)
virmem3=$(ps -o vsz $p3 |cut -d " " -f 4)
echo $virmem3
if [ $virmem3 -gt 150000 ] ;then
        kill -9 $p3
        dpstart
else
        echo "hi"
fi
]


6)

#! /bin/bash
p1=$(pii mongrel.7002.pid)
p2=$(pii mongrel.7003.pid)
p3=$(pii mongrel.7004.pid)
echo $p1 $p2 $p3



                if [ "$p1" ==  ""  ] ;then
                echo "probelm $p1 not running "
                        kill -9 $p1 $p2 $p3
                        cd /home/app/dp/current/
                        rm -rf /home/app/dp/current/log/mongrel.700*.pid
                        mongrel_rails cluster::start
                else
                        echo "hi $p1 running"
                fi



                if [ "$p2" ==  ""  ] ;then
                echo "probelm $p1 not running "
                        kill -9 $p1 $p2 $p3
                        cd /home/app/dp/current/
                        rm -rf /home/app/dp/current/log/mongrel.700*.pid
                        mongrel_rails cluster::start
                else
                        echo "hi $p2 running"
                fi


                if [ "$p3" ==  ""  ] ;then
                echo "probelm $p3 not running "
                        kill -9 $p1 $p2 $p3
                        cd /home/app/dp/current/
                        rm -rf /home/app/dp/current/log/mongrel.700*.pid
                        mongrel_rails cluster::start
                else
                        echo "hi $p3 running"
                fi



p1=$(pii mongrel.7002.pid)
p2=$(pii mongrel.7003.pid)
p3=$(pii mongrel.7004.pid)
echo $p1 $p2 $p3

]

7) My swift.yml
[
cluster_address: dp.com
cluster_port: 7001
daemonize: true
map:
  - incoming:
    - dp.com:80
    outgoing: 127.0.0.1:45002
    epoll: true
    epoll_descriptors: 20000
    redeployable: true
    redeployable: 114168
    default: true
    docroot: /home/app/dp/current/public

]


8)   apache proxy setting in virual host

<VirtualHost *:80>
  ServerName dp.com
  ServerAlias  www.dp.com
  DocumentRoot /home/app/dp/current/public/
  <Directory "/home/app/dp/current/public/">
     Options ExecCGI Indexes FollowSymLinks
     AllowOverride all
     Order allow,deny
     Allow from all
   </Directory>
  # Configure the cluster member proxy
  <Proxy balancer://dp_cluster>
    BalancerMember http://dp.com:7001
    BalancerMember http://127.0.0.1:7002
    BalancerMember http://127.0.0.1:7003
    BalancerMember http://127.0.0.1:7004
 </Proxy>
ProxyPass /images !
ProxyPass /stylesheets !
  RewriteEngine On
  RewriteRule ^([^.]+)$ $1.html [QSA]
  # All dynamic requests get sent to the cluster
  RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
  RewriteRule ^/(.*)$ balancer://dp_cluster%{REQUEST_URI} [P,QSA,L]
  # Deflate for clients that support it.
  AddOutputFilterByType DEFLATE text/html text/plain text/xml
  BrowserMatch ^Mozilla/4 gzip-only-text/html
  BrowserMatch ^Mozilla/4\.0[678] no-gzip
  BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

</VirtualHost>



ALL is control via cronjobs accorinding to sutable interval

playing with procees ids is always dangerous


for monitoring god gem & monit are good one



have nice day ChetanM

From wyhaines at gmail.com  Wed Feb 13 10:13:44 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Wed, 13 Feb 2008 08:13:44 -0700
Subject: [Swiftiply-users] Swiftiply-users Digest, Vol 8, Issue 1
In-Reply-To: <f34743be0802122229j543b694cyaf13d92ca4a4a65@mail.gmail.com>
References: <mailman.344.1202855892.11700.swiftiply-users@rubyforge.org>
	<f34743be0802122229j543b694cyaf13d92ca4a4a65@mail.gmail.com>
Message-ID: <f4cd26df0802130713k3a3da082w1b5473dacb5ce7e6@mail.gmail.com>

On Feb 12, 2008 11:29 PM, Chetan Muneshwar - BetterLabs
<chetan.muneshwar at betterlabs.net> wrote:
> hello friends
> Everything is fine swiftiply events based mechanism rocking
>
> but i have one problem of" bad file descriptor error" and swift stop working
> i need good solution for that

Ugh.  This has been reported to me several times in recent days.  It
is an uncatchable error coming from a bug in EM.  The bug is fixed in
the EM source trunk.  For an immediate fix, look at upgrading to the
trunk version of EM.  That version is becoming 0.11 soon.  I've had it
running for about 20 sites for a few weeks now with no problems that I
have seen.


Kirk Haines