[typo] Slow spam checking makes Baby Jesus cry
scott at sigkill.org
Mon Aug 21 11:09:34 EDT 2006
On 8/20/06, Piers Cawley <pdcawley at bofh.org.uk> wrote:
> Dominic Mitchell <dom at happygiraffe.net> writes:
> > Piers Cawley wrote:
> >> Piers Cawley <pdcawley at bofh.org.uk> writes:
> >>> Piers Cawley <pdcawley at bofh.org.uk> writes:
> >>>> The catch is, IPSocket.getaddress, which is what we use for DNS
> >>>> lookups, appears to be be a blocking call, which with the nature of
> >>>> Ruby threads, means it'll *still* hold up processing during the
> >>>> lookup.
> >>>> Thoughts?
> >>> Hmm... resolv.rb may be our friend. And (bonus points!) it's in the
> >>> standard library.
> >> And, following further work on using threads, I'd just like to say
> >> that threading is weird and is currently doing my head in. Background
> >> classification sort of works, but only very approximately.
> > Would it not be possible to take the same approach as mod_perl and
> > schedule some code to run immediately after the current request, but
> > before the next one (a cleanup handler)? It would still mean that
> > particular fastcgi process was unavailable, but it would mean that you
> > could return to the user quicker.
> Sadly not. Some of the spam checks I've seen in my logs have taken the
> best part of a minute. Tie up a couple of dispatchers like that and
> your performance is screwed.
> > Unfortunately, a quick look in railties at the dispatcher shows that
> > rails doesn't offer anything like this. Damn.
> Monkey patching the dispatcher to allow it isn't exactly hard. But see above.
This brings up a reasonably obvious question that I haven't seen asked
yet--why not turn off RBL and like Akismet take care of it? Akismet
seems pretty fast, and I've been happy with its error rate so far.
I've never been a bit RBL fan, for the usual reasons--all RBL lists
seem to eventually decend into filtering for spam politics more then
spam itself--ISP X doesn't have a good abuse policy, so we're going to
block all of their addresses to force a change. I'd much rather block
based on content then politics :-).
More information about the Typo-list