[sup-talk] sup-sync and xapian memory usage

[Rich Lane]

Excerpts from Andrew Pimlott's message of Mon Sep 07 13:04:50 -0400 2009:
> Using the xapian index, things are different.  It starts at 32M and
> steadily climbs to 77M after ~3500 messages, or around 1M every 100
> messages.  It does seem to climb faster at first and then more slowly.
> Either xapian is keeping a cache (but some searches suggest it doesn't),
> it's leaking memory, or it's allocating memory in a way that the the
> allocator can't reclaim the VM space.  Any ideas?

Xapian keeps writes buffered in memory. Try setting the environment
variable XAPIAN_FLUSH_THRESHOLD to a smaller value (the default is 10000
documents) and see if that helps.