> I've been fussing with this problem for a few days and I'm beginning to
> think it's a problem (bug?) in JacORB.  I have a client that requests an
> object with a specified number of structures in it. The IDL for the
> structure is shown below. If I request fewer than 520K of these objects
> (message size of 4160016), everything works as expected. Once I request
> any larger, the server becomes very, very slow. Eventually, the message
> will be returned (sometimes after several minutes). If I put debug
> statements in the write method of my Helper method I see it getting very,
> very slow. The stack trace is shown below when I interrupted execution.

I've seen this as well (in JacORB 1.3.1) and it's due to a design
limitation in the buffer management code.

BufferManager has a pool of buffers of various sizes, up to a limit of
around 4M.  If you request a buffer that's over 4M, it falls back to
creating a buffer that's exactly the requested size.

In CDROutputStream, before each primitive type or primitive array is
written, it checks that the current buffer is large enough to hold the
value that's about to be written.  If it isn't, it requests a larger buffer
from the buffer manager - the request is for a buffer that's just large
enough to hold the existing buffer contents plus the new value.

You can probably see how this is going...

Suppose you're trying to write out 5M octets, one at a time (i.e., not as a
primitive array).
After writing 1, the current buffer size is 32 (smallest managed buffer)
After writing 32, the current buffer size is 32
After writing 33, the current buffer size is 64 (32 bytes copied from
                  previous buffer, 1 new byte, 31 empty bytes)
After writing 65, the current buffer size is 128
...
After writing 2M, the current buffer size is 2M
After writing 2M+1, the current buffer size is 4M
...
After writing 4M, the current buffer size is 4M

So far, so good, because the number of buffer changes is only 17 so far.
The amortised cost is O(N).  Now look at what happens next:

After writing 4M+1, the current buffer size is 4M+1
After writing 4M+2, the current buffer size is 4M+2
...

Every new byte causes a new buffer to be allocated that is ONE BYTE larger
than the previous buffer.  All the contents of the old buffer get copied
into it.  This behaviour is O(N^2).

A proper solution to this involves rethinking the buffer manager's
allocation strategy: how it should deal with buffers that are over the
limit that it feels comfortable keeping around in case of future use.
Ideally it should allocate larger buffers in some geometric progression,
but then free them as quickly as possible.

-wjr