From evan at cloudbur.st  Tue Feb 12 05:41:33 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Tue, 12 Feb 2008 05:41:33 -0500
Subject: [Mongrel-development] Mongrel trac has happened
Message-ID: <b6f68fc60802120241g481626ck59858791cc091790@mail.gmail.com>

Hey all,

I ended up setting up Trac on my own server, since the point of the EY
slice was to have someone manage it and nobody over there actually
ever did that.

http://mongrel.cloudbur.st/

The swimming dog is public domain. The brown-eyed dog didn't fit the
color scheme, but we can find him a home somewhere.

I still have to get some static stuff like the RDoc up, and the Atom
feed, but otherwise I think it's ready. It syncs down from the master
Rubyforge SVN. If this looks ok, I will finish those two things and
ask Tom Copeland to point the subdomain to us. I ported the content
from the existing site, but didn't format it all. This will make it
way easier for users to update the documentation for us.

Register on the Trac and then email me your username, and I'll make
you an admin.

Thanks

Evan

-- 
Evan Weaver
Cloudburst, LLC

From zedshaw at zedshaw.com  Thu Feb 14 01:17:39 2008
From: zedshaw at zedshaw.com (Zed A. Shaw)
Date: Thu, 14 Feb 2008 01:17:39 -0500
Subject: [Mongrel-development] Mongrel trac has happened
In-Reply-To: <b6f68fc60802120241g481626ck59858791cc091790@mail.gmail.com>
References: <b6f68fc60802120241g481626ck59858791cc091790@mail.gmail.com>
Message-ID: <20080214011739.917cd6a5.zedshaw@zedshaw.com>

On Tue, 12 Feb 2008 05:41:33 -0500
"Evan Weaver" <evan at cloudbur.st> wrote:

> Hey all,
> 
> I still have to get some static stuff like the RDoc up, and the Atom
> feed, but otherwise I think it's ready. It syncs down from the master
> Rubyforge SVN. If this looks ok, I will finish those two things and
> ask Tom Copeland to point the subdomain to us. I ported the content
> from the existing site, but didn't format it all. This will make it
> way easier for users to update the documentation for us.

Hmm, it seems you moved the original content from svn as well:

http://mongrel.rubyforge.org/svn/trunk/

Three requests I have before you guys go forward with this:

1) Put the original site webgen pages back into svn so that anyone can
find the original content and reconstruct the website.  It's kind of
really really not cool to make many people's hard work just disappear
like that.

2) New content that originates from content written by a contributor
has their attribution in a way that can't be altered by the wiki.  This
is important since they need the credit for the work they did.

3) Put this move to a vote on the mailing list so that people who use
mongrel know it's not unilateral.  I'm sure everyone is ok with it, but
it's good to make sure you hear people out on it.

This is kind of important to me since the web site was probably the
most fun part of doing mongrel, and I don't want all my great jokes and
stuff to disappear.  You don't have to use it, but at least perserve
the original stuff so that others can access it easily.

Thanks.

-- 
Zed A. Shaw
- Hate: http://savingtheinternetwithhate.com/
- Good: http://www.zedshaw.com/
- Evil: http://yearofevil.com/

From evan at cloudbur.st  Wed Feb 13 02:13:13 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Wed, 13 Feb 2008 02:13:13 -0500
Subject: [Mongrel-development] Mongrel trac has happened
In-Reply-To: <20080214011739.917cd6a5.zedshaw@zedshaw.com>
References: <b6f68fc60802120241g481626ck59858791cc091790@mail.gmail.com>
	<20080214011739.917cd6a5.zedshaw@zedshaw.com>
Message-ID: <b6f68fc60802122313t777681bagb63373aac7d6d408@mail.gmail.com>

On Feb 14, 2008 1:17 AM, Zed A. Shaw <zedshaw at zedshaw.com> wrote:
> Three requests I have before you guys go forward with this:

I copied the old site/ out of the stable tag into web/ where it's more obvious.

We can give someone admin status if they want to take ownership of a
page. Trac tracks all wiki changes anyway so it's not hard to tell who
did what.

I raised the issue of getting a Trac/wiki on the users list months ago
and most people agreed it was a good decision.

Evan

-- 
Evan Weaver
Cloudburst, LLC

From luislavena at gmail.com  Wed Feb 13 06:08:22 2008
From: luislavena at gmail.com (Luis Lavena)
Date: Wed, 13 Feb 2008 09:08:22 -0200
Subject: [Mongrel-development] Mongrel trac has happened
In-Reply-To: <20080214011739.917cd6a5.zedshaw@zedshaw.com>
References: <b6f68fc60802120241g481626ck59858791cc091790@mail.gmail.com>
	<20080214011739.917cd6a5.zedshaw@zedshaw.com>
Message-ID: <71166b3b0802130308s25b8f864t7042213ae6de6ea6@mail.gmail.com>

On Feb 14, 2008 4:17 AM, Zed A. Shaw <zedshaw at zedshaw.com> wrote:
>
> Hmm, it seems you moved the original content from svn as well:
>
> http://mongrel.rubyforge.org/svn/trunk/
>
> Three requests I have before you guys go forward with this:
>
> 1) Put the original site webgen pages back into svn so that anyone can
> find the original content and reconstruct the website.  It's kind of
> really really not cool to make many people's hard work just disappear
> like that.
>

The site content is still there:

http://mongrel.rubyforge.org/svn/web/

What it's missing there is the webgen Rake tasks (these should be
stripped from main Rakefile).

> 3) Put this move to a vote on the mailing list so that people who use
> mongrel know it's not unilateral.  I'm sure everyone is ok with it, but
> it's good to make sure you hear people out on it.

This was raised and discussed a few months back when "to use or not
devjavu or lighthouse services" -- on both cases you agreeded depend
on someones setup/application to host vital information about Mongrel
wasn't good.

> This is kind of important to me since the web site was probably the
> most fun part of doing mongrel, and I don't want all my great jokes and
> stuff to disappear.  You don't have to use it, but at least perserve
> the original stuff so that others can access it easily.

Too bad we didn't registered a domain name when this started :P

In that way we can have the site and a pointer to trac, both at the same time.

I just noticed that some of the docs about win32 services aren't up to
date (mea culpa).

-- 
Luis Lavena
Multimedia systems
-
A common mistake that people make when trying to design
something completely foolproof is to underestimate
the ingenuity of complete fools.
Douglas Adams

From evan at cloudbur.st  Fri Feb 22 13:40:02 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Fri, 22 Feb 2008 13:40:02 -0500
Subject: [Mongrel-development] Mongrel 1.1.4
Message-ID: <b6f68fc60802221040h3a3f2b99i134cd78d56973a6f@mail.gmail.com>

Hi all,

Mongrel 1.1.4 is ready:
svn://rubyforge.org/var/svn/mongrel/tags/rel_1-1-4 . It fixes the
camping handler, adds a test for the path traversal security flaw, and
corrects the treatment of the @throttle parameter.

Please try it out, or at least audit the commits, which is easy to do
on the Trac now.

If I don't hear anything by next Friday I will go ahead and make the
release. After 1.1.4 I am going to push towards 1.2 which you can see
described on the Roadmap: http://mongrel.rubyforge.org/roadmap .

Thanks

Evan

-- 
Evan Weaver
Cloudburst, LLC

From luislavena at gmail.com  Sat Feb 23 06:55:19 2008
From: luislavena at gmail.com (Luis Lavena)
Date: Sat, 23 Feb 2008 09:55:19 -0200
Subject: [Mongrel-development] Mongrel 1.1.4
In-Reply-To: <b6f68fc60802221040h3a3f2b99i134cd78d56973a6f@mail.gmail.com>
References: <b6f68fc60802221040h3a3f2b99i134cd78d56973a6f@mail.gmail.com>
Message-ID: <71166b3b0802230355p74557801w6ebd24df7ca90481@mail.gmail.com>

On Fri, Feb 22, 2008 at 4:40 PM, Evan Weaver <evan at cloudbur.st> wrote:
> Hi all,
>
>  Mongrel 1.1.4 is ready:
>  svn://rubyforge.org/var/svn/mongrel/tags/rel_1-1-4 . It fixes the
>  camping handler, adds a test for the path traversal security flaw, and
>  corrects the treatment of the @throttle parameter.
>
>  Please try it out, or at least audit the commits, which is easy to do
>  on the Trac now.
>
>  If I don't hear anything by next Friday I will go ahead and make the
>  release. After 1.1.4 I am going to push towards 1.2 which you can see
>  described on the Roadmap: http://mongrel.rubyforge.org/roadmap .
>

Excellent news Evan, we should put a ticket to simplify the gem
versioning thing (the constant and the built in one in the C
extension).
There are too many places to make mistakes about that, and should be simpler :-)

I'll try to hang out later during the weekend on #mongrel-dev FYI.

Regards,

-- 
Luis Lavena
Multimedia systems
-
A common mistake that people make when trying to design
something completely foolproof is to underestimate
the ingenuity of complete fools.
Douglas Adams

From zedshaw at zedshaw.com  Sun Feb 24 14:38:37 2008
From: zedshaw at zedshaw.com (Zed A. Shaw)
Date: Sun, 24 Feb 2008 14:38:37 -0500
Subject: [Mongrel-development] Mongrel 1.1.4
In-Reply-To: <b6f68fc60802221040h3a3f2b99i134cd78d56973a6f@mail.gmail.com>
References: <b6f68fc60802221040h3a3f2b99i134cd78d56973a6f@mail.gmail.com>
Message-ID: <20080224143837.ec6363c5.zedshaw@zedshaw.com>

On Fri, 22 Feb 2008 13:40:02 -0500
"Evan Weaver" <evan at cloudbur.st> wrote:

> Hi all,
> 
> Mongrel 1.1.4 is ready:
> svn://rubyforge.org/var/svn/mongrel/tags/rel_1-1-4 . It fixes the
> camping handler, adds a test for the path traversal security flaw, and
> corrects the treatment of the @throttle parameter.

I'll try it out under JRuby with our Rails project and let you know.
One thing I'll send you a patch for is the fact that under jruby
mongrel tries to trap("USR1") but that's not supported in Java so it
blows up hard.  I got the fix in yesterday, but I'll reapply it to
1.1.4 and send it on.

-- 
Zed A. Shaw
- Hate: http://savingtheinternetwithhate.com/
- Good: http://www.zedshaw.com/
- Evil: http://yearofevil.com/

From normalperson at yhbt.net  Mon Feb 25 03:01:27 2008
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 25 Feb 2008 00:01:27 -0800
Subject: [Mongrel-development] [PATCH] avoid needless syscall when
	num_processors limit is reached
Message-ID: <20080225080111.GA11848@soma>

Since we're going to close the socket immediately after the
num_processors limit is reached, there's no point in calling
setsockopt(2) to enable TCP_CORK on it.  Instead, only enable
TCP_CORK for connections we are able to handle.

While we're at it, avoid calling worker_list.length twice in the
connection rejected case and instead just set num_workers to
@workers.list.length once.  I'm assuming the original caching of
worker_list = @workers.list to avoid having the log message
display a different number due to a race condition; and this
preserves that functionality while avoiding an extra method call.

---
 This patch is against
 svn://rubyforge.org/var/svn/mongrel/branches/stable_1-1 r981

 lib/mongrel.rb |   16 +++++++---------
 1 files changed, 7 insertions(+), 9 deletions(-)

diff --git a/lib/mongrel.rb b/lib/mongrel.rb
index d99c56d..cfdfc49 100644
--- a/lib/mongrel.rb
+++ b/lib/mongrel.rb
@@ -270,18 +270,16 @@ module Mongrel
           while true
             begin
               client = @socket.accept
-  
-              if defined?($tcp_cork_opts) and $tcp_cork_opts
-                client.setsockopt(*$tcp_cork_opts) rescue nil
-              end
-  
-              worker_list = @workers.list
-  
-              if worker_list.length >= @num_processors
-                STDERR.puts "Server overloaded with #{worker_list.length} processors (#@num_processors max). Dropping connection."
+
+              num_workers = @workers.list.length
+              if num_workers >= @num_processors
+                STDERR.puts "Server overloaded with #{num_workers} processors (#@num_processors max). Dropping connection."
                 client.close rescue nil
                 reap_dead_workers("max processors")
               else
+                if defined?($tcp_cork_opts) and $tcp_cork_opts
+                  client.setsockopt(*$tcp_cork_opts) rescue nil
+                end
                 thread = Thread.new(client) {|c| process_client(c) }
                 thread[:started_on] = Time.now
                 @workers.add(thread)
-- 
Eric Wong

From normalperson at yhbt.net  Thu Feb 28 20:53:09 2008
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 28 Feb 2008 17:53:09 -0800
Subject: [Mongrel-development] [PATCH] http11: ~6% performance increase in
	header parsing
Message-ID: <20080229015309.GA9080@untitled>

Allocate one string object and avoid appending to it causing it
to be resized.  Additionally, optimize the string toupper copy
so that it's done in a single pass.

Also, use an inline, locale-independent toupper() implementation
which should be more predictable for users with exotic locales
(HTTP header names are always ASCII).

The following test script was used:
require 'mongrel'
include Mongrel

parser = HttpParser.new
req = "GET /ruby HTTP/1.1\r\n" \
      "User-Agent: curl/7.12.3\r\n" \
      "Host: bogomips.org\r\n" \
      "Accept: */*\r\n" \
      "\r\n".freeze
hash = Hash.new
100000.times do
  parser.execute(hash, req, 0)
  parser.reset
  hash.clear
end
---
 ext/http11/http11.c |   30 +++++++++++++++++++++++-------
 1 files changed, 23 insertions(+), 7 deletions(-)

diff --git a/ext/http11/http11.c b/ext/http11/http11.c
index b0cd42e..90d9b40 100644
--- a/ext/http11/http11.c
+++ b/ext/http11/http11.c
@@ -7,7 +7,6 @@
 #include <assert.h>
 #include <string.h>
 #include "http11_parser.h"
-#include <ctype.h>
 
 static VALUE mMongrel;
 static VALUE cHttpParser;
@@ -62,7 +61,9 @@ DEF_MAX_LENGTH(HEADER, (1024 * (80 + 32)));
 
 void http_field(void *data, const char *field, size_t flen, const char *value, size_t vlen)
 {
-  char *ch, *end;
+  char *ch;
+  const char *fch;
+  int i;
   VALUE req = (VALUE)data;
   VALUE v = Qnil;
   VALUE f = Qnil;
@@ -71,15 +72,30 @@ void http_field(void *data, const char *field, size_t flen, const char *value, s
   VALIDATE_MAX_LENGTH(vlen, FIELD_VALUE);
 
   v = rb_str_new(value, vlen);
-  f = rb_str_dup(global_http_prefix);
-  f = rb_str_buf_cat(f, field, flen); 
 
-  for(ch = RSTRING(f)->ptr, end = ch + RSTRING(f)->len; ch < end; ch++) {
-    if(*ch == '-') {
+  /*
+   * using rb_str_new(NULL, len) here is faster than rb_str_buf_new(len)
+   * in my testing, because: there's no minimum allocation length (and
+   * no check for it, either), RSTRING(f)->len does not need to be
+   * written twice, and and RSTRING(f)->ptr[len] will already be
+   * null-terminated for us.
+   */
+  f = rb_str_new(NULL, RSTRING(global_http_prefix)->len + flen);
+  memcpy(RSTRING(f)->ptr,
+         RSTRING(global_http_prefix)->ptr,
+         RSTRING(global_http_prefix)->len);
+
+  i = (int)flen;
+  fch = field;
+  ch = RSTRING(f)->ptr + RSTRING(global_http_prefix)->len;
+  while(--i >= 0) {
+    if(*fch == '-') {
       *ch = '_';
     } else {
-      *ch = toupper(*ch);
+      *ch = (*fch >= 'a' && *fch <= 'z') ? (*fch & ~0x20) : *fch;
     }
+    ++ch;
+    ++fch;
   }
 
   rb_hash_aset(req, f, v);
-- 
Eric Wong

From filipe at icewall.org  Thu Feb 28 21:30:50 2008
From: filipe at icewall.org (Filipe)
Date: Thu, 28 Feb 2008 23:30:50 -0300 (BRT)
Subject: [Mongrel-development] [PATCH] http11: ~6% performance increase
 in	header parsing
In-Reply-To: <20080229015309.GA9080@untitled>
References: <20080229015309.GA9080@untitled>
Message-ID: <Pine.LNX.4.64.0802282330000.3722@sophie.icewall.org>


Hey Eric, thanks!

Someone is testing patches from Eric? (this one and one earlier this
week).

If not i can test and merge it to head if ok :)

cheers,

filipe {
  @              icewall.org
  GPG         1024D/A6BA423E
  http://filipe.icewall.org/
}


On Thu, 28 Feb 2008, Eric Wong wrote:

> Allocate one string object and avoid appending to it causing it
> to be resized.  Additionally, optimize the string toupper copy
> so that it's done in a single pass.
>
> Also, use an inline, locale-independent toupper() implementation
> which should be more predictable for users with exotic locales
> (HTTP header names are always ASCII).
>
> The following test script was used:
> require 'mongrel'
> include Mongrel
>
> parser = HttpParser.new
> req = "GET /ruby HTTP/1.1\r\n" \
>      "User-Agent: curl/7.12.3\r\n" \
>      "Host: bogomips.org\r\n" \
>      "Accept: */*\r\n" \
>      "\r\n".freeze
> hash = Hash.new
> 100000.times do
>  parser.execute(hash, req, 0)
>  parser.reset
>  hash.clear
> end
> ---
> ext/http11/http11.c |   30 +++++++++++++++++++++++-------
> 1 files changed, 23 insertions(+), 7 deletions(-)
>
> diff --git a/ext/http11/http11.c b/ext/http11/http11.c
> index b0cd42e..90d9b40 100644
> --- a/ext/http11/http11.c
> +++ b/ext/http11/http11.c
> @@ -7,7 +7,6 @@
> #include <assert.h>
> #include <string.h>
> #include "http11_parser.h"
> -#include <ctype.h>
>
> static VALUE mMongrel;
> static VALUE cHttpParser;
> @@ -62,7 +61,9 @@ DEF_MAX_LENGTH(HEADER, (1024 * (80 + 32)));
>
> void http_field(void *data, const char *field, size_t flen, const char *value, size_t vlen)
> {
> -  char *ch, *end;
> +  char *ch;
> +  const char *fch;
> +  int i;
>   VALUE req = (VALUE)data;
>   VALUE v = Qnil;
>   VALUE f = Qnil;
> @@ -71,15 +72,30 @@ void http_field(void *data, const char *field, size_t flen, const char *value, s
>   VALIDATE_MAX_LENGTH(vlen, FIELD_VALUE);
>
>   v = rb_str_new(value, vlen);
> -  f = rb_str_dup(global_http_prefix);
> -  f = rb_str_buf_cat(f, field, flen);
>
> -  for(ch = RSTRING(f)->ptr, end = ch + RSTRING(f)->len; ch < end; ch++) {
> -    if(*ch == '-') {
> +  /*
> +   * using rb_str_new(NULL, len) here is faster than rb_str_buf_new(len)
> +   * in my testing, because: there's no minimum allocation length (and
> +   * no check for it, either), RSTRING(f)->len does not need to be
> +   * written twice, and and RSTRING(f)->ptr[len] will already be
> +   * null-terminated for us.
> +   */
> +  f = rb_str_new(NULL, RSTRING(global_http_prefix)->len + flen);
> +  memcpy(RSTRING(f)->ptr,
> +         RSTRING(global_http_prefix)->ptr,
> +         RSTRING(global_http_prefix)->len);
> +
> +  i = (int)flen;
> +  fch = field;
> +  ch = RSTRING(f)->ptr + RSTRING(global_http_prefix)->len;
> +  while(--i >= 0) {
> +    if(*fch == '-') {
>       *ch = '_';
>     } else {
> -      *ch = toupper(*ch);
> +      *ch = (*fch >= 'a' && *fch <= 'z') ? (*fch & ~0x20) : *fch;
>     }
> +    ++ch;
> +    ++fch;
>   }
>
>   rb_hash_aset(req, f, v);
> -- 
> Eric Wong
> _______________________________________________
> Mongrel-development mailing list
> Mongrel-development at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-development
>

From evan at cloudbur.st  Thu Feb 28 23:29:39 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Thu, 28 Feb 2008 20:29:39 -0800
Subject: [Mongrel-development] [PATCH] http11: ~6% performance increase
	in header parsing
In-Reply-To: <Pine.LNX.4.64.0802282330000.3722@sophie.icewall.org>
References: <20080229015309.GA9080@untitled>
	<Pine.LNX.4.64.0802282330000.3722@sophie.icewall.org>
Message-ID: <b6f68fc60802282029r647c6022p8c63b4a2ef8f9f3f@mail.gmail.com>

That would be great; I haven't gotten to it. Can you merge it against
branches/stable_1-2?

Also make sure to add a test if it changes any API or adds any functionality.

Thanks

Evan

On Thu, Feb 28, 2008 at 6:30 PM, Filipe <filipe at icewall.org> wrote:
>
>  Hey Eric, thanks!
>
>  Someone is testing patches from Eric? (this one and one earlier this
>  week).
>
>  If not i can test and merge it to head if ok :)
>
>  cheers,
>
>  filipe {
>   @              icewall.org
>   GPG         1024D/A6BA423E
>   http://filipe.icewall.org/
>
>
> }
>
>
>  On Thu, 28 Feb 2008, Eric Wong wrote:
>
>  > Allocate one string object and avoid appending to it causing it
>  > to be resized.  Additionally, optimize the string toupper copy
>  > so that it's done in a single pass.
>  >
>  > Also, use an inline, locale-independent toupper() implementation
>  > which should be more predictable for users with exotic locales
>  > (HTTP header names are always ASCII).
>  >
>  > The following test script was used:
>  > require 'mongrel'
>  > include Mongrel
>  >
>  > parser = HttpParser.new
>  > req = "GET /ruby HTTP/1.1\r\n" \
>  >      "User-Agent: curl/7.12.3\r\n" \
>  >      "Host: bogomips.org\r\n" \
>  >      "Accept: */*\r\n" \
>  >      "\r\n".freeze
>  > hash = Hash.new
>  > 100000.times do
>  >  parser.execute(hash, req, 0)
>  >  parser.reset
>  >  hash.clear
>  > end
>  > ---
>  > ext/http11/http11.c |   30 +++++++++++++++++++++++-------
>  > 1 files changed, 23 insertions(+), 7 deletions(-)
>  >
>  > diff --git a/ext/http11/http11.c b/ext/http11/http11.c
>  > index b0cd42e..90d9b40 100644
>  > --- a/ext/http11/http11.c
>  > +++ b/ext/http11/http11.c
>  > @@ -7,7 +7,6 @@
>  > #include <assert.h>
>  > #include <string.h>
>  > #include "http11_parser.h"
>  > -#include <ctype.h>
>  >
>  > static VALUE mMongrel;
>  > static VALUE cHttpParser;
>  > @@ -62,7 +61,9 @@ DEF_MAX_LENGTH(HEADER, (1024 * (80 + 32)));
>  >
>  > void http_field(void *data, const char *field, size_t flen, const char *value, size_t vlen)
>  > {
>  > -  char *ch, *end;
>  > +  char *ch;
>  > +  const char *fch;
>  > +  int i;
>  >   VALUE req = (VALUE)data;
>  >   VALUE v = Qnil;
>  >   VALUE f = Qnil;
>  > @@ -71,15 +72,30 @@ void http_field(void *data, const char *field, size_t flen, const char *value, s
>  >   VALIDATE_MAX_LENGTH(vlen, FIELD_VALUE);
>  >
>  >   v = rb_str_new(value, vlen);
>  > -  f = rb_str_dup(global_http_prefix);
>  > -  f = rb_str_buf_cat(f, field, flen);
>  >
>  > -  for(ch = RSTRING(f)->ptr, end = ch + RSTRING(f)->len; ch < end; ch++) {
>  > -    if(*ch == '-') {
>  > +  /*
>  > +   * using rb_str_new(NULL, len) here is faster than rb_str_buf_new(len)
>  > +   * in my testing, because: there's no minimum allocation length (and
>  > +   * no check for it, either), RSTRING(f)->len does not need to be
>  > +   * written twice, and and RSTRING(f)->ptr[len] will already be
>  > +   * null-terminated for us.
>  > +   */
>  > +  f = rb_str_new(NULL, RSTRING(global_http_prefix)->len + flen);
>  > +  memcpy(RSTRING(f)->ptr,
>  > +         RSTRING(global_http_prefix)->ptr,
>  > +         RSTRING(global_http_prefix)->len);
>  > +
>  > +  i = (int)flen;
>  > +  fch = field;
>  > +  ch = RSTRING(f)->ptr + RSTRING(global_http_prefix)->len;
>  > +  while(--i >= 0) {
>  > +    if(*fch == '-') {
>  >       *ch = '_';
>  >     } else {
>  > -      *ch = toupper(*ch);
>  > +      *ch = (*fch >= 'a' && *fch <= 'z') ? (*fch & ~0x20) : *fch;
>  >     }
>  > +    ++ch;
>  > +    ++fch;
>  >   }
>  >
>  >   rb_hash_aset(req, f, v);
>  > --
>  > Eric Wong
>  > _______________________________________________
>  > Mongrel-development mailing list
>  > Mongrel-development at rubyforge.org
>  > http://rubyforge.org/mailman/listinfo/mongrel-development
>  >
>  _______________________________________________
>  Mongrel-development mailing list
>  Mongrel-development at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-development
>



-- 
Evan Weaver
Cloudburst, LLC

From filipe at icewall.org  Thu Feb 28 23:52:59 2008
From: filipe at icewall.org (Filipe)
Date: Fri, 29 Feb 2008 01:52:59 -0300 (BRT)
Subject: [Mongrel-development] [PATCH] http11: ~6% performance increase
 in header parsing
In-Reply-To: <b6f68fc60802282029r647c6022p8c63b4a2ef8f9f3f@mail.gmail.com>
References: <20080229015309.GA9080@untitled>
	<Pine.LNX.4.64.0802282330000.3722@sophie.icewall.org>
	<b6f68fc60802282029r647c6022p8c63b4a2ef8f9f3f@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0802290152440.3722@sophie.icewall.org>


Ok, will do it. Will let you know when done!

filipe {
  @              icewall.org
  GPG         1024D/A6BA423E
  http://filipe.icewall.org/
}


On Thu, 28 Feb 2008, Evan Weaver wrote:

> That would be great; I haven't gotten to it. Can you merge it against
> branches/stable_1-2?
>
> Also make sure to add a test if it changes any API or adds any functionality.
>
> Thanks
>
> Evan
>
> On Thu, Feb 28, 2008 at 6:30 PM, Filipe <filipe at icewall.org> wrote:
>>
>>  Hey Eric, thanks!
>>
>>  Someone is testing patches from Eric? (this one and one earlier this
>>  week).
>>
>>  If not i can test and merge it to head if ok :)
>>
>>  cheers,
>>
>>  filipe {
>>   @              icewall.org
>>   GPG         1024D/A6BA423E
>>   http://filipe.icewall.org/
>>
>>
>> }
>>
>>
>>  On Thu, 28 Feb 2008, Eric Wong wrote:
>>
>> > Allocate one string object and avoid appending to it causing it
>> > to be resized.  Additionally, optimize the string toupper copy
>> > so that it's done in a single pass.
>> >
>> > Also, use an inline, locale-independent toupper() implementation
>> > which should be more predictable for users with exotic locales
>> > (HTTP header names are always ASCII).
>> >
>> > The following test script was used:
>> > require 'mongrel'
>> > include Mongrel
>> >
>> > parser = HttpParser.new
>> > req = "GET /ruby HTTP/1.1\r\n" \
>> >      "User-Agent: curl/7.12.3\r\n" \
>> >      "Host: bogomips.org\r\n" \
>> >      "Accept: */*\r\n" \
>> >      "\r\n".freeze
>> > hash = Hash.new
>> > 100000.times do
>> >  parser.execute(hash, req, 0)
>> >  parser.reset
>> >  hash.clear
>> > end
>> > ---
>> > ext/http11/http11.c |   30 +++++++++++++++++++++++-------
>> > 1 files changed, 23 insertions(+), 7 deletions(-)
>> >
>> > diff --git a/ext/http11/http11.c b/ext/http11/http11.c
>> > index b0cd42e..90d9b40 100644
>> > --- a/ext/http11/http11.c
>> > +++ b/ext/http11/http11.c
>> > @@ -7,7 +7,6 @@
>> > #include <assert.h>
>> > #include <string.h>
>> > #include "http11_parser.h"
>> > -#include <ctype.h>
>> >
>> > static VALUE mMongrel;
>> > static VALUE cHttpParser;
>> > @@ -62,7 +61,9 @@ DEF_MAX_LENGTH(HEADER, (1024 * (80 + 32)));
>> >
>> > void http_field(void *data, const char *field, size_t flen, const char *value, size_t vlen)
>> > {
>> > -  char *ch, *end;
>> > +  char *ch;
>> > +  const char *fch;
>> > +  int i;
>> >   VALUE req = (VALUE)data;
>> >   VALUE v = Qnil;
>> >   VALUE f = Qnil;
>> > @@ -71,15 +72,30 @@ void http_field(void *data, const char *field, size_t flen, const char *value, s
>> >   VALIDATE_MAX_LENGTH(vlen, FIELD_VALUE);
>> >
>> >   v = rb_str_new(value, vlen);
>> > -  f = rb_str_dup(global_http_prefix);
>> > -  f = rb_str_buf_cat(f, field, flen);
>> >
>> > -  for(ch = RSTRING(f)->ptr, end = ch + RSTRING(f)->len; ch < end; ch++) {
>> > -    if(*ch == '-') {
>> > +  /*
>> > +   * using rb_str_new(NULL, len) here is faster than rb_str_buf_new(len)
>> > +   * in my testing, because: there's no minimum allocation length (and
>> > +   * no check for it, either), RSTRING(f)->len does not need to be
>> > +   * written twice, and and RSTRING(f)->ptr[len] will already be
>> > +   * null-terminated for us.
>> > +   */
>> > +  f = rb_str_new(NULL, RSTRING(global_http_prefix)->len + flen);
>> > +  memcpy(RSTRING(f)->ptr,
>> > +         RSTRING(global_http_prefix)->ptr,
>> > +         RSTRING(global_http_prefix)->len);
>> > +
>> > +  i = (int)flen;
>> > +  fch = field;
>> > +  ch = RSTRING(f)->ptr + RSTRING(global_http_prefix)->len;
>> > +  while(--i >= 0) {
>> > +    if(*fch == '-') {
>> >       *ch = '_';
>> >     } else {
>> > -      *ch = toupper(*ch);
>> > +      *ch = (*fch >= 'a' && *fch <= 'z') ? (*fch & ~0x20) : *fch;
>> >     }
>> > +    ++ch;
>> > +    ++fch;
>> >   }
>> >
>> >   rb_hash_aset(req, f, v);
>> > --
>> > Eric Wong
>> > _______________________________________________
>> > Mongrel-development mailing list
>> > Mongrel-development at rubyforge.org
>> > http://rubyforge.org/mailman/listinfo/mongrel-development
>> >
>>  _______________________________________________
>>  Mongrel-development mailing list
>>  Mongrel-development at rubyforge.org
>>  http://rubyforge.org/mailman/listinfo/mongrel-development
>>
>
>
>
> -- 
> Evan Weaver
> Cloudburst, LLC
> _______________________________________________
> Mongrel-development mailing list
> Mongrel-development at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-development
>

From ry at tinyclouds.org  Fri Feb 29 05:03:27 2008
From: ry at tinyclouds.org (ry dahl)
Date: Fri, 29 Feb 2008 11:03:27 +0100
Subject: [Mongrel-development] [PATCH] http11: ~6% performance increase
	in header parsing
In-Reply-To: <20080229015309.GA9080@untitled>
References: <20080229015309.GA9080@untitled>
Message-ID: <3ae7f4480802290203v50023b57se867e531c162f066@mail.gmail.com>

This aught to use RSTRING_PTR(str) and RSTRING_LEN(str) e.g.
    for(ch = RSTRING_PTR(f), end = ch + RSTRING_LEN(f); ch < end; ch++) {
for 1.9 compatibility.

ry

On Fri, Feb 29, 2008 at 2:53 AM, Eric Wong <normalperson at yhbt.net> wrote:
> Allocate one string object and avoid appending to it causing it
>  to be resized.  Additionally, optimize the string toupper copy
>  so that it's done in a single pass.
>
>  Also, use an inline, locale-independent toupper() implementation
>  which should be more predictable for users with exotic locales
>  (HTTP header names are always ASCII).
>
>  The following test script was used:
>  require 'mongrel'
>  include Mongrel
>
>  parser = HttpParser.new
>  req = "GET /ruby HTTP/1.1\r\n" \
>       "User-Agent: curl/7.12.3\r\n" \
>       "Host: bogomips.org\r\n" \
>       "Accept: */*\r\n" \
>       "\r\n".freeze
>  hash = Hash.new
>  100000.times do
>   parser.execute(hash, req, 0)
>   parser.reset
>   hash.clear
>  end
>  ---
>   ext/http11/http11.c |   30 +++++++++++++++++++++++-------
>   1 files changed, 23 insertions(+), 7 deletions(-)
>
>  diff --git a/ext/http11/http11.c b/ext/http11/http11.c
>  index b0cd42e..90d9b40 100644
>  --- a/ext/http11/http11.c
>  +++ b/ext/http11/http11.c
>  @@ -7,7 +7,6 @@
>   #include <assert.h>
>   #include <string.h>
>   #include "http11_parser.h"
>  -#include <ctype.h>
>
>   static VALUE mMongrel;
>   static VALUE cHttpParser;
>  @@ -62,7 +61,9 @@ DEF_MAX_LENGTH(HEADER, (1024 * (80 + 32)));
>
>   void http_field(void *data, const char *field, size_t flen, const char *value, size_t vlen)
>   {
>  -  char *ch, *end;
>  +  char *ch;
>  +  const char *fch;
>  +  int i;
>    VALUE req = (VALUE)data;
>    VALUE v = Qnil;
>    VALUE f = Qnil;
>  @@ -71,15 +72,30 @@ void http_field(void *data, const char *field, size_t flen, const char *value, s
>    VALIDATE_MAX_LENGTH(vlen, FIELD_VALUE);
>
>    v = rb_str_new(value, vlen);
>  -  f = rb_str_dup(global_http_prefix);
>  -  f = rb_str_buf_cat(f, field, flen);
>
>  -  for(ch = RSTRING(f)->ptr, end = ch + RSTRING(f)->len; ch < end; ch++) {
>  -    if(*ch == '-') {
>  +  /*
>  +   * using rb_str_new(NULL, len) here is faster than rb_str_buf_new(len)
>  +   * in my testing, because: there's no minimum allocation length (and
>  +   * no check for it, either), RSTRING(f)->len does not need to be
>  +   * written twice, and and RSTRING(f)->ptr[len] will already be
>  +   * null-terminated for us.
>  +   */
>  +  f = rb_str_new(NULL, RSTRING(global_http_prefix)->len + flen);
>  +  memcpy(RSTRING(f)->ptr,
>  +         RSTRING(global_http_prefix)->ptr,
>  +         RSTRING(global_http_prefix)->len);
>  +
>  +  i = (int)flen;
>  +  fch = field;
>  +  ch = RSTRING(f)->ptr + RSTRING(global_http_prefix)->len;
>  +  while(--i >= 0) {
>  +    if(*fch == '-') {
>        *ch = '_';
>      } else {
>  -      *ch = toupper(*ch);
>  +      *ch = (*fch >= 'a' && *fch <= 'z') ? (*fch & ~0x20) : *fch;
>      }
>  +    ++ch;
>  +    ++fch;
>    }
>
>    rb_hash_aset(req, f, v);
>  --
>  Eric Wong
>  _______________________________________________
>  Mongrel-development mailing list
>  Mongrel-development at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-development
>

From ry at tinyclouds.org  Fri Feb 29 05:21:40 2008
From: ry at tinyclouds.org (ry dahl)
Date: Fri, 29 Feb 2008 11:21:40 +0100
Subject: [Mongrel-development] [PATCH] http11: ~6% performance increase
	in header parsing
In-Reply-To: <3ae7f4480802290203v50023b57se867e531c162f066@mail.gmail.com>
References: <20080229015309.GA9080@untitled>
	<3ae7f4480802290203v50023b57se867e531c162f066@mail.gmail.com>
Message-ID: <3ae7f4480802290221o2929dd9ev154859dd5475808c@mail.gmail.com>

while i'm nitpicking...:P

#define ASCII_UPPER(ch) ('a' <= ch && ch <= 'z' ? ch - 'a' + 'A' : ch)

    memcpy( RSTRING_PTR(f)
          , RSTRING_PTR(global_http_prefix)
          , RSTRING_LEN(global_http_prefix)
          );
    for(i = 0; i < length; i++) {
      ch = RSTRING_PTR(f) + RSTRING_LEN(global_http_prefix) + i;
      if(field[i] == '-') {
        *ch = '_';
      } else {
        *ch = ASCII_UPPER(field[i]);
      }
    }


On Fri, Feb 29, 2008 at 11:03 AM, ry dahl <ry at tinyclouds.org> wrote:
> This aught to use RSTRING_PTR(str) and RSTRING_LEN(str) e.g.
>     for(ch = RSTRING_PTR(f), end = ch + RSTRING_LEN(f); ch < end; ch++) {
>  for 1.9 compatibility.
>
>  ry
>
>
>
>  On Fri, Feb 29, 2008 at 2:53 AM, Eric Wong <normalperson at yhbt.net> wrote:
>  > Allocate one string object and avoid appending to it causing it
>  >  to be resized.  Additionally, optimize the string toupper copy
>  >  so that it's done in a single pass.
>  >
>  >  Also, use an inline, locale-independent toupper() implementation
>  >  which should be more predictable for users with exotic locales
>  >  (HTTP header names are always ASCII).
>  >
>  >  The following test script was used:
>  >  require 'mongrel'
>  >  include Mongrel
>  >
>  >  parser = HttpParser.new
>  >  req = "GET /ruby HTTP/1.1\r\n" \
>  >       "User-Agent: curl/7.12.3\r\n" \
>  >       "Host: bogomips.org\r\n" \
>  >       "Accept: */*\r\n" \
>  >       "\r\n".freeze
>  >  hash = Hash.new
>  >  100000.times do
>  >   parser.execute(hash, req, 0)
>  >   parser.reset
>  >   hash.clear
>  >  end
>  >  ---
>  >   ext/http11/http11.c |   30 +++++++++++++++++++++++-------
>  >   1 files changed, 23 insertions(+), 7 deletions(-)
>  >
>  >  diff --git a/ext/http11/http11.c b/ext/http11/http11.c
>  >  index b0cd42e..90d9b40 100644
>  >  --- a/ext/http11/http11.c
>  >  +++ b/ext/http11/http11.c
>  >  @@ -7,7 +7,6 @@
>  >   #include <assert.h>
>  >   #include <string.h>
>  >   #include "http11_parser.h"
>  >  -#include <ctype.h>
>  >
>  >   static VALUE mMongrel;
>  >   static VALUE cHttpParser;
>  >  @@ -62,7 +61,9 @@ DEF_MAX_LENGTH(HEADER, (1024 * (80 + 32)));
>  >
>  >   void http_field(void *data, const char *field, size_t flen, const char *value, size_t vlen)
>  >   {
>  >  -  char *ch, *end;
>  >  +  char *ch;
>  >  +  const char *fch;
>  >  +  int i;
>  >    VALUE req = (VALUE)data;
>  >    VALUE v = Qnil;
>  >    VALUE f = Qnil;
>  >  @@ -71,15 +72,30 @@ void http_field(void *data, const char *field, size_t flen, const char *value, s
>  >    VALIDATE_MAX_LENGTH(vlen, FIELD_VALUE);
>  >
>  >    v = rb_str_new(value, vlen);
>  >  -  f = rb_str_dup(global_http_prefix);
>  >  -  f = rb_str_buf_cat(f, field, flen);
>  >
>  >  -  for(ch = RSTRING(f)->ptr, end = ch + RSTRING(f)->len; ch < end; ch++) {
>  >  -    if(*ch == '-') {
>  >  +  /*
>  >  +   * using rb_str_new(NULL, len) here is faster than rb_str_buf_new(len)
>  >  +   * in my testing, because: there's no minimum allocation length (and
>  >  +   * no check for it, either), RSTRING(f)->len does not need to be
>  >  +   * written twice, and and RSTRING(f)->ptr[len] will already be
>  >  +   * null-terminated for us.
>  >  +   */
>  >  +  f = rb_str_new(NULL, RSTRING(global_http_prefix)->len + flen);
>  >  +  memcpy(RSTRING(f)->ptr,
>  >  +         RSTRING(global_http_prefix)->ptr,
>  >  +         RSTRING(global_http_prefix)->len);
>  >  +
>  >  +  i = (int)flen;
>  >  +  fch = field;
>  >  +  ch = RSTRING(f)->ptr + RSTRING(global_http_prefix)->len;
>  >  +  while(--i >= 0) {
>  >  +    if(*fch == '-') {
>  >        *ch = '_';
>  >      } else {
>  >  -      *ch = toupper(*ch);
>  >  +      *ch = (*fch >= 'a' && *fch <= 'z') ? (*fch & ~0x20) : *fch;
>  >      }
>  >  +    ++ch;
>  >  +    ++fch;
>  >    }
>  >
>  >    rb_hash_aset(req, f, v);
>  >  --
>  >  Eric Wong
>  >  _______________________________________________
>  >  Mongrel-development mailing list
>  >  Mongrel-development at rubyforge.org
>  >  http://rubyforge.org/mailman/listinfo/mongrel-development
>  >
>

From evan at cloudbur.st  Fri Feb 29 13:16:34 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Fri, 29 Feb 2008 13:16:34 -0500
Subject: [Mongrel-development] Mongrel 1.1.4
In-Reply-To: <20080224143837.ec6363c5.zedshaw@zedshaw.com>
References: <b6f68fc60802221040h3a3f2b99i134cd78d56973a6f@mail.gmail.com>
	<20080224143837.ec6363c5.zedshaw@zedshaw.com>
Message-ID: <b6f68fc60802291016pdec7318q8ca67edb3ab1596e@mail.gmail.com>

1.1.4 is going live real soon now, since I haven't heard any further updates.

Evan

On Sun, Feb 24, 2008 at 2:38 PM, Zed A. Shaw <zedshaw at zedshaw.com> wrote:
> On Fri, 22 Feb 2008 13:40:02 -0500
>
> "Evan Weaver" <evan at cloudbur.st> wrote:
>
>
> > Hi all,
>  >
>  > Mongrel 1.1.4 is ready:
>  > svn://rubyforge.org/var/svn/mongrel/tags/rel_1-1-4 . It fixes the
>  > camping handler, adds a test for the path traversal security flaw, and
>  > corrects the treatment of the @throttle parameter.
>
>  I'll try it out under JRuby with our Rails project and let you know.
>  One thing I'll send you a patch for is the fact that under jruby
>  mongrel tries to trap("USR1") but that's not supported in Java so it
>  blows up hard.  I got the fix in yesterday, but I'll reapply it to
>  1.1.4 and send it on.
>
>  --
>  Zed A. Shaw
>  - Hate: http://savingtheinternetwithhate.com/
>  - Good: http://www.zedshaw.com/
>  - Evil: http://yearofevil.com/
>
>
> _______________________________________________
>  Mongrel-development mailing list
>  Mongrel-development at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-development
>



-- 
Evan Weaver
Cloudburst, LLC

From evan at cloudbur.st  Fri Feb 29 14:31:58 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Fri, 29 Feb 2008 14:31:58 -0500
Subject: [Mongrel-development] Mongrel 1.1.4
In-Reply-To: <b6f68fc60802291016pdec7318q8ca67edb3ab1596e@mail.gmail.com>
References: <b6f68fc60802221040h3a3f2b99i134cd78d56973a6f@mail.gmail.com>
	<20080224143837.ec6363c5.zedshaw@zedshaw.com>
	<b6f68fc60802291016pdec7318q8ca67edb3ab1596e@mail.gmail.com>
Message-ID: <b6f68fc60802291131hc0dbe58sff73e36dd3fb8f5d@mail.gmail.com>

Luis,

Can you verify that the filename for the win32 build is correct? Gems
seems to have changed it again:
http://rubyforge.org/frs/?group_id=1306 .

Now gems outputs mongrel-1.1.4-x86-mswin32-60.gem, whereas before we
had mongrel-1.1.3-i386-mswin32.gem, and before that it was
mongrel-1.1.2-mswin32.gem .

Evan

On Fri, Feb 29, 2008 at 1:16 PM, Evan Weaver <evan at cloudbur.st> wrote:
> 1.1.4 is going live real soon now, since I haven't heard any further updates.
>
>  Evan
>
>
>
>  On Sun, Feb 24, 2008 at 2:38 PM, Zed A. Shaw <zedshaw at zedshaw.com> wrote:
>  > On Fri, 22 Feb 2008 13:40:02 -0500
>  >
>  > "Evan Weaver" <evan at cloudbur.st> wrote:
>  >
>  >
>  > > Hi all,
>  >  >
>  >  > Mongrel 1.1.4 is ready:
>  >  > svn://rubyforge.org/var/svn/mongrel/tags/rel_1-1-4 . It fixes the
>  >  > camping handler, adds a test for the path traversal security flaw, and
>  >  > corrects the treatment of the @throttle parameter.
>  >
>  >  I'll try it out under JRuby with our Rails project and let you know.
>  >  One thing I'll send you a patch for is the fact that under jruby
>  >  mongrel tries to trap("USR1") but that's not supported in Java so it
>  >  blows up hard.  I got the fix in yesterday, but I'll reapply it to
>  >  1.1.4 and send it on.
>  >
>  >  --
>  >  Zed A. Shaw
>  >  - Hate: http://savingtheinternetwithhate.com/
>  >  - Good: http://www.zedshaw.com/
>  >  - Evil: http://yearofevil.com/
>  >
>  >
>  > _______________________________________________
>  >  Mongrel-development mailing list
>  >  Mongrel-development at rubyforge.org
>  >  http://rubyforge.org/mailman/listinfo/mongrel-development
>  >
>
>
>
>  --
>  Evan Weaver
>  Cloudburst, LLC
>



-- 
Evan Weaver
Cloudburst, LLC

From luislavena at gmail.com  Fri Feb 29 14:57:57 2008
From: luislavena at gmail.com (Luis Lavena)
Date: Fri, 29 Feb 2008 17:57:57 -0200
Subject: [Mongrel-development] Mongrel 1.1.4
In-Reply-To: <b6f68fc60802291131hc0dbe58sff73e36dd3fb8f5d@mail.gmail.com>
References: <b6f68fc60802221040h3a3f2b99i134cd78d56973a6f@mail.gmail.com>
	<20080224143837.ec6363c5.zedshaw@zedshaw.com>
	<b6f68fc60802291016pdec7318q8ca67edb3ab1596e@mail.gmail.com>
	<b6f68fc60802291131hc0dbe58sff73e36dd3fb8f5d@mail.gmail.com>
Message-ID: <71166b3b0802291157j332755a3y4cf8ddd35a3131da@mail.gmail.com>

On Fri, Feb 29, 2008 at 5:31 PM, Evan Weaver <evan at cloudbur.st> wrote:
> Luis,
>
>  Can you verify that the filename for the win32 build is correct? Gems
>  seems to have changed it again:
>  http://rubyforge.org/frs/?group_id=1306 .
>
>  Now gems outputs mongrel-1.1.4-x86-mswin32-60.gem, whereas before we
>  had mongrel-1.1.3-i386-mswin32.gem, and before that it was
>  mongrel-1.1.2-mswin32.gem .
>

That's due two changes:

1) previous versions (with mswin32 as platform) was using incorrectly
the #platform Gem::Specification), latest version used the CURRENT
platform instead.

2) The new naming scheme came from RubyGems 1.0.1, but I prefer build
these with 0.9.4 to keep compatibility with users that didn't upgraded
to latest ruby or latest rubygems.

In any case, most of the users doing Rails already have RubyGems
1.0.1, so no damage will happen with new schema.

Regards,
-- 
Luis Lavena
Multimedia systems
-
A common mistake that people make when trying to design
something completely foolproof is to underestimate
the ingenuity of complete fools.
Douglas Adams

From normalperson at yhbt.net  Fri Feb 29 17:27:32 2008
From: normalperson at yhbt.net (Eric Wong)
Date: Fri, 29 Feb 2008 14:27:32 -0800
Subject: [Mongrel-development] [PATCH] http11: ~6% performance increase
	in header parsing
In-Reply-To: <3ae7f4480802290221o2929dd9ev154859dd5475808c@mail.gmail.com>
References: <20080229015309.GA9080@untitled>
	<3ae7f4480802290203v50023b57se867e531c162f066@mail.gmail.com>
	<3ae7f4480802290221o2929dd9ev154859dd5475808c@mail.gmail.com>
Message-ID: <20080229222731.GA13406@untitled>

ry dahl <ry at tinyclouds.org> wrote:
> while i'm nitpicking...:P
> 
> #define ASCII_UPPER(ch) ('a' <= ch && ch <= 'z' ? ch - 'a' + 'A' : ch)
> 
>     memcpy( RSTRING_PTR(f)
>           , RSTRING_PTR(global_http_prefix)
>           , RSTRING_LEN(global_http_prefix)
>           );
>     for(i = 0; i < length; i++) {
>       ch = RSTRING_PTR(f) + RSTRING_LEN(global_http_prefix) + i;
>       if(field[i] == '-') {
>         *ch = '_';
>       } else {
>         *ch = ASCII_UPPER(field[i]);
>       }
>     }

The ASCII_UPPER helps readability, thanks.

I haven't had a chance to benchmark your for loop vs my while loop I
wrote, but given the wide variety of compilers, versions and options, I
expect my while loop to be slightly faster in general.

Additionally, reordering the if/else condition to put the more common
case first didn't seem to help in my case (gcc 4.1.1-21 from Debian,
-O2), so I left it alone.

However, I haven't tried __builtin_expect() here, either (fast_xs did
benefit from __builtin_expect() in my tests).

Perhaps rolling the checks together would be fastest, eliminating
a redundant check for '-' if it's lowercase, still:

  *ch = ('a' <= *fch && *fch <= 'z') ?
        (*fch - 'a' + 'A') :
        (*fch != '-' ? *fch : '_');

Not that other inefficiencies drastically overshadow the increases
we get here :)

> On Fri, Feb 29, 2008 at 11:03 AM, ry dahl <ry at tinyclouds.org> wrote:
> > This aught to use RSTRING_PTR(str) and RSTRING_LEN(str) e.g.
> >     for(ch = RSTRING_PTR(f), end = ch + RSTRING_LEN(f); ch < end; ch++) {
> >  for 1.9 compatibility.

Sure.  I have a partial set of compatibility macros from fast_xs if we
don't have them already so <=1.8.5 doesn't break:

http://bogomips.org/ruby/fast_xs-0.6/ext/fast_xs/ruby_1_9_compat.h

> >  On Fri, Feb 29, 2008 at 2:53 AM, Eric Wong <normalperson at yhbt.net> wrote:
> >  > Allocate one string object and avoid appending to it causing it
> >  >  to be resized.  Additionally, optimize the string toupper copy
> >  >  so that it's done in a single pass.
> >  >
> >  >  Also, use an inline, locale-independent toupper() implementation
> >  >  which should be more predictable for users with exotic locales
> >  >  (HTTP header names are always ASCII).
> >  >
> >  >  The following test script was used:
> >  >  require 'mongrel'
> >  >  include Mongrel
> >  >
> >  >  parser = HttpParser.new
> >  >  req = "GET /ruby HTTP/1.1\r\n" \
> >  >       "User-Agent: curl/7.12.3\r\n" \
> >  >       "Host: bogomips.org\r\n" \
> >  >       "Accept: */*\r\n" \
> >  >       "\r\n".freeze
> >  >  hash = Hash.new
> >  >  100000.times do
> >  >   parser.execute(hash, req, 0)
> >  >   parser.reset
> >  >   hash.clear
> >  >  end
> >  >  ---
> >  >   ext/http11/http11.c |   30 +++++++++++++++++++++++-------
> >  >   1 files changed, 23 insertions(+), 7 deletions(-)
> >  >
> >  >  diff --git a/ext/http11/http11.c b/ext/http11/http11.c
> >  >  index b0cd42e..90d9b40 100644
> >  >  --- a/ext/http11/http11.c
> >  >  +++ b/ext/http11/http11.c
> >  >  @@ -7,7 +7,6 @@
> >  >   #include <assert.h>
> >  >   #include <string.h>
> >  >   #include "http11_parser.h"
> >  >  -#include <ctype.h>
> >  >
> >  >   static VALUE mMongrel;
> >  >   static VALUE cHttpParser;
> >  >  @@ -62,7 +61,9 @@ DEF_MAX_LENGTH(HEADER, (1024 * (80 + 32)));
> >  >
> >  >   void http_field(void *data, const char *field, size_t flen, const char *value, size_t vlen)
> >  >   {
> >  >  -  char *ch, *end;
> >  >  +  char *ch;
> >  >  +  const char *fch;
> >  >  +  int i;
> >  >    VALUE req = (VALUE)data;
> >  >    VALUE v = Qnil;
> >  >    VALUE f = Qnil;
> >  >  @@ -71,15 +72,30 @@ void http_field(void *data, const char *field, size_t flen, const char *value, s
> >  >    VALIDATE_MAX_LENGTH(vlen, FIELD_VALUE);
> >  >
> >  >    v = rb_str_new(value, vlen);
> >  >  -  f = rb_str_dup(global_http_prefix);
> >  >  -  f = rb_str_buf_cat(f, field, flen);
> >  >
> >  >  -  for(ch = RSTRING(f)->ptr, end = ch + RSTRING(f)->len; ch < end; ch++) {
> >  >  -    if(*ch == '-') {
> >  >  +  /*
> >  >  +   * using rb_str_new(NULL, len) here is faster than rb_str_buf_new(len)
> >  >  +   * in my testing, because: there's no minimum allocation length (and
> >  >  +   * no check for it, either), RSTRING(f)->len does not need to be
> >  >  +   * written twice, and and RSTRING(f)->ptr[len] will already be
> >  >  +   * null-terminated for us.
> >  >  +   */
> >  >  +  f = rb_str_new(NULL, RSTRING(global_http_prefix)->len + flen);
> >  >  +  memcpy(RSTRING(f)->ptr,
> >  >  +         RSTRING(global_http_prefix)->ptr,
> >  >  +         RSTRING(global_http_prefix)->len);
> >  >  +
> >  >  +  i = (int)flen;
> >  >  +  fch = field;
> >  >  +  ch = RSTRING(f)->ptr + RSTRING(global_http_prefix)->len;
> >  >  +  while(--i >= 0) {
> >  >  +    if(*fch == '-') {
> >  >        *ch = '_';
> >  >      } else {
> >  >  -      *ch = toupper(*ch);
> >  >  +      *ch = (*fch >= 'a' && *fch <= 'z') ? (*fch & ~0x20) : *fch;
> >  >      }
> >  >  +    ++ch;
> >  >  +    ++fch;
> >  >    }
> >  >
> >  >    rb_hash_aset(req, f, v);
> >  >  --
> >  >  Eric Wong
> >  >  _______________________________________________
> >  >  Mongrel-development mailing list
> >  >  Mongrel-development at rubyforge.org
> >  >  http://rubyforge.org/mailman/listinfo/mongrel-development
> >  >
> >
> _______________________________________________
> Mongrel-development mailing list
> Mongrel-development at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-development

-- 
Eric Wong

From normalperson at yhbt.net  Fri Feb 29 17:30:54 2008
From: normalperson at yhbt.net (Eric Wong)
Date: Fri, 29 Feb 2008 14:30:54 -0800
Subject: [Mongrel-development] [PATCH] http11: ~6% performance increase
	in	header parsing
In-Reply-To: <Pine.LNX.4.64.0802282330000.3722@sophie.icewall.org>
References: <20080229015309.GA9080@untitled>
	<Pine.LNX.4.64.0802282330000.3722@sophie.icewall.org>
Message-ID: <20080229223054.GB13406@untitled>

Filipe <filipe at icewall.org> wrote:
> 
> Hey Eric, thanks!
 
You're welcome :)

> Someone is testing patches from Eric? (this one and one earlier this
> week).
> 
> If not i can test and merge it to head if ok :)

I'd also like the patch I submitted for ticket #14 looked at (and
hopefully in 1.1.4), too:

  http://mongrel.rubyforge.org/ticket/14

I actually consider the patch for ticket #14 to be more important than
either of the patches I submitted here.

-- 
Eric Wong

From evan at cloudbur.st  Fri Feb 29 17:36:35 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Fri, 29 Feb 2008 17:36:35 -0500
Subject: [Mongrel-development] [PATCH] http11: ~6% performance increase
	in header parsing
In-Reply-To: <20080229222731.GA13406@untitled>
References: <20080229015309.GA9080@untitled>
	<3ae7f4480802290203v50023b57se867e531c162f066@mail.gmail.com>
	<3ae7f4480802290221o2929dd9ev154859dd5475808c@mail.gmail.com>
	<20080229222731.GA13406@untitled>
Message-ID: <b6f68fc60802291436w6ac62626i1f19f52a4a3a9e2c@mail.gmail.com>

We will need to target 1.2 for these since 1.1.4 is already out. That
makes more sense anyway since performance work is on the roadmap for
1.2.

Also, please, if you can make sure the changes are exercised by some
test. You may have to add it because our coverage is not totally
stellar in all places. If not, though add a ticket about the lack of
the test and assign it to me.

Evan

On Fri, Feb 29, 2008 at 5:27 PM, Eric Wong <normalperson at yhbt.net> wrote:
> ry dahl <ry at tinyclouds.org> wrote:
>
> > while i'm nitpicking...:P
>  >
>  > #define ASCII_UPPER(ch) ('a' <= ch && ch <= 'z' ? ch - 'a' + 'A' : ch)
>  >
>  >     memcpy( RSTRING_PTR(f)
>  >           , RSTRING_PTR(global_http_prefix)
>  >           , RSTRING_LEN(global_http_prefix)
>  >           );
>  >     for(i = 0; i < length; i++) {
>  >       ch = RSTRING_PTR(f) + RSTRING_LEN(global_http_prefix) + i;
>  >       if(field[i] == '-') {
>  >         *ch = '_';
>  >       } else {
>  >         *ch = ASCII_UPPER(field[i]);
>  >       }
>  >     }
>
>  The ASCII_UPPER helps readability, thanks.
>
>  I haven't had a chance to benchmark your for loop vs my while loop I
>  wrote, but given the wide variety of compilers, versions and options, I
>  expect my while loop to be slightly faster in general.
>
>  Additionally, reordering the if/else condition to put the more common
>  case first didn't seem to help in my case (gcc 4.1.1-21 from Debian,
>  -O2), so I left it alone.
>
>  However, I haven't tried __builtin_expect() here, either (fast_xs did
>  benefit from __builtin_expect() in my tests).
>
>  Perhaps rolling the checks together would be fastest, eliminating
>  a redundant check for '-' if it's lowercase, still:
>
>   *ch = ('a' <= *fch && *fch <= 'z') ?
>         (*fch - 'a' + 'A') :
>         (*fch != '-' ? *fch : '_');
>
>  Not that other inefficiencies drastically overshadow the increases
>  we get here :)
>
>
>  > On Fri, Feb 29, 2008 at 11:03 AM, ry dahl <ry at tinyclouds.org> wrote:
>  > > This aught to use RSTRING_PTR(str) and RSTRING_LEN(str) e.g.
>  > >     for(ch = RSTRING_PTR(f), end = ch + RSTRING_LEN(f); ch < end; ch++) {
>  > >  for 1.9 compatibility.
>
>  Sure.  I have a partial set of compatibility macros from fast_xs if we
>  don't have them already so <=1.8.5 doesn't break:
>
>  http://bogomips.org/ruby/fast_xs-0.6/ext/fast_xs/ruby_1_9_compat.h
>
>
>
>  > >  On Fri, Feb 29, 2008 at 2:53 AM, Eric Wong <normalperson at yhbt.net> wrote:
>  > >  > Allocate one string object and avoid appending to it causing it
>  > >  >  to be resized.  Additionally, optimize the string toupper copy
>  > >  >  so that it's done in a single pass.
>  > >  >
>  > >  >  Also, use an inline, locale-independent toupper() implementation
>  > >  >  which should be more predictable for users with exotic locales
>  > >  >  (HTTP header names are always ASCII).
>  > >  >
>  > >  >  The following test script was used:
>  > >  >  require 'mongrel'
>  > >  >  include Mongrel
>  > >  >
>  > >  >  parser = HttpParser.new
>  > >  >  req = "GET /ruby HTTP/1.1\r\n" \
>  > >  >       "User-Agent: curl/7.12.3\r\n" \
>  > >  >       "Host: bogomips.org\r\n" \
>  > >  >       "Accept: */*\r\n" \
>  > >  >       "\r\n".freeze
>  > >  >  hash = Hash.new
>  > >  >  100000.times do
>  > >  >   parser.execute(hash, req, 0)
>  > >  >   parser.reset
>  > >  >   hash.clear
>  > >  >  end
>  > >  >  ---
>  > >  >   ext/http11/http11.c |   30 +++++++++++++++++++++++-------
>  > >  >   1 files changed, 23 insertions(+), 7 deletions(-)
>  > >  >
>  > >  >  diff --git a/ext/http11/http11.c b/ext/http11/http11.c
>  > >  >  index b0cd42e..90d9b40 100644
>  > >  >  --- a/ext/http11/http11.c
>  > >  >  +++ b/ext/http11/http11.c
>  > >  >  @@ -7,7 +7,6 @@
>  > >  >   #include <assert.h>
>  > >  >   #include <string.h>
>  > >  >   #include "http11_parser.h"
>  > >  >  -#include <ctype.h>
>  > >  >
>  > >  >   static VALUE mMongrel;
>  > >  >   static VALUE cHttpParser;
>  > >  >  @@ -62,7 +61,9 @@ DEF_MAX_LENGTH(HEADER, (1024 * (80 + 32)));
>  > >  >
>  > >  >   void http_field(void *data, const char *field, size_t flen, const char *value, size_t vlen)
>  > >  >   {
>  > >  >  -  char *ch, *end;
>  > >  >  +  char *ch;
>  > >  >  +  const char *fch;
>  > >  >  +  int i;
>  > >  >    VALUE req = (VALUE)data;
>  > >  >    VALUE v = Qnil;
>  > >  >    VALUE f = Qnil;
>  > >  >  @@ -71,15 +72,30 @@ void http_field(void *data, const char *field, size_t flen, const char *value, s
>  > >  >    VALIDATE_MAX_LENGTH(vlen, FIELD_VALUE);
>  > >  >
>  > >  >    v = rb_str_new(value, vlen);
>  > >  >  -  f = rb_str_dup(global_http_prefix);
>  > >  >  -  f = rb_str_buf_cat(f, field, flen);
>  > >  >
>  > >  >  -  for(ch = RSTRING(f)->ptr, end = ch + RSTRING(f)->len; ch < end; ch++) {
>  > >  >  -    if(*ch == '-') {
>  > >  >  +  /*
>  > >  >  +   * using rb_str_new(NULL, len) here is faster than rb_str_buf_new(len)
>  > >  >  +   * in my testing, because: there's no minimum allocation length (and
>  > >  >  +   * no check for it, either), RSTRING(f)->len does not need to be
>  > >  >  +   * written twice, and and RSTRING(f)->ptr[len] will already be
>  > >  >  +   * null-terminated for us.
>  > >  >  +   */
>  > >  >  +  f = rb_str_new(NULL, RSTRING(global_http_prefix)->len + flen);
>  > >  >  +  memcpy(RSTRING(f)->ptr,
>  > >  >  +         RSTRING(global_http_prefix)->ptr,
>  > >  >  +         RSTRING(global_http_prefix)->len);
>  > >  >  +
>  > >  >  +  i = (int)flen;
>  > >  >  +  fch = field;
>  > >  >  +  ch = RSTRING(f)->ptr + RSTRING(global_http_prefix)->len;
>  > >  >  +  while(--i >= 0) {
>  > >  >  +    if(*fch == '-') {
>  > >  >        *ch = '_';
>  > >  >      } else {
>  > >  >  -      *ch = toupper(*ch);
>  > >  >  +      *ch = (*fch >= 'a' && *fch <= 'z') ? (*fch & ~0x20) : *fch;
>  > >  >      }
>  > >  >  +    ++ch;
>  > >  >  +    ++fch;
>  > >  >    }
>  > >  >
>  > >  >    rb_hash_aset(req, f, v);
>  > >  >  --
>  > >  >  Eric Wong
>  > >  >  _______________________________________________
>  > >  >  Mongrel-development mailing list
>  > >  >  Mongrel-development at rubyforge.org
>  > >  >  http://rubyforge.org/mailman/listinfo/mongrel-development
>  > >  >
>  > >
>  > _______________________________________________
>  > Mongrel-development mailing list
>  > Mongrel-development at rubyforge.org
>  > http://rubyforge.org/mailman/listinfo/mongrel-development
>
>  --
>
>
> Eric Wong
>  _______________________________________________
>  Mongrel-development mailing list
>  Mongrel-development at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-development
>



-- 
Evan Weaver
Cloudburst, LLC

From evan at cloudbur.st  Fri Feb 29 17:41:13 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Fri, 29 Feb 2008 17:41:13 -0500
Subject: [Mongrel-development] [PATCH] avoid needless syscall when
	num_processors limit is reached
In-Reply-To: <20080225080111.GA11848@soma>
References: <20080225080111.GA11848@soma>
Message-ID: <b6f68fc60802291441k5a414ae6ve96dfbc88c9da090@mail.gmail.com>

Is there already a ticket for this? I'm not seeing it.

Evan

On Mon, Feb 25, 2008 at 3:01 AM, Eric Wong <normalperson at yhbt.net> wrote:
> Since we're going to close the socket immediately after the
>  num_processors limit is reached, there's no point in calling
>  setsockopt(2) to enable TCP_CORK on it.  Instead, only enable
>  TCP_CORK for connections we are able to handle.
>
>  While we're at it, avoid calling worker_list.length twice in the
>  connection rejected case and instead just set num_workers to
>  @workers.list.length once.  I'm assuming the original caching of
>  worker_list = @workers.list to avoid having the log message
>  display a different number due to a race condition; and this
>  preserves that functionality while avoiding an extra method call.
>
>  ---
>   This patch is against
>   svn://rubyforge.org/var/svn/mongrel/branches/stable_1-1 r981
>
>   lib/mongrel.rb |   16 +++++++---------
>   1 files changed, 7 insertions(+), 9 deletions(-)
>
>  diff --git a/lib/mongrel.rb b/lib/mongrel.rb
>  index d99c56d..cfdfc49 100644
>  --- a/lib/mongrel.rb
>  +++ b/lib/mongrel.rb
>  @@ -270,18 +270,16 @@ module Mongrel
>            while true
>              begin
>                client = @socket.accept
>  -
>  -              if defined?($tcp_cork_opts) and $tcp_cork_opts
>  -                client.setsockopt(*$tcp_cork_opts) rescue nil
>  -              end
>  -
>  -              worker_list = @workers.list
>  -
>  -              if worker_list.length >= @num_processors
>  -                STDERR.puts "Server overloaded with #{worker_list.length} processors (#@num_processors max). Dropping connection."
>  +
>  +              num_workers = @workers.list.length
>  +              if num_workers >= @num_processors
>  +                STDERR.puts "Server overloaded with #{num_workers} processors (#@num_processors max). Dropping connection."
>                  client.close rescue nil
>                  reap_dead_workers("max processors")
>                else
>  +                if defined?($tcp_cork_opts) and $tcp_cork_opts
>  +                  client.setsockopt(*$tcp_cork_opts) rescue nil
>  +                end
>                  thread = Thread.new(client) {|c| process_client(c) }
>                  thread[:started_on] = Time.now
>                  @workers.add(thread)
>  --
>  Eric Wong
>  _______________________________________________
>  Mongrel-development mailing list
>  Mongrel-development at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-development
>



-- 
Evan Weaver
Cloudburst, LLC