Take the 2-minute tour ×

What is the sensible buffer size to use when transferring (irrespective of UP/DOWN) large files (3-4 gigs) with Java?

byte buf[] = new byte[1024]


byte buf[] = new byte[5 * 1024 * 1024]

Sometimes even if you use a large buffer and pass to a read(byte array[]) method this doesn't guarantee you that you will get a full 5 me buffer. In my tests I have observed that the average size is usually 1.5kb per read() invocation. Does this make sense performance wise? I'd be glad if someone could point me to a resource discussing the issue in greater details.

share|improve this question
add comment

2 Answers

up vote 8 down vote accepted

It sounds like you're reading from a network connection (TCP?)

1500 bytes is the default Ethernet MTU, which explains why you're typically getting 1.5KB per read. The MTU can often be increased to 9KB by configuring the network stacks to use jumbo frames.

With this in mind, there is almost certainly no point in making buf larger than 9KB. Using a smaller buffer (say over 1KB) may or may not negatively impact performance.

In any case, the only way to get a definitive answer is through benchmarking various buffer sizes.

share|improve this answer
Hmz, good point. I haven't really factored in the the MTU of the TCP stack. –  LordDoskias Sep 5 '11 at 13:48
Using larger buffers will reduce the number of switches between Java code and native code and may have performance impact, independent of the network packet size. –  jarnbjo Sep 5 '11 at 13:50
As shows my experience 8-100KB buffer is good enough. –  umbr Sep 5 '11 at 14:43
Actually if you're using IPv6 you may also run into IPv6's jumbo frames and those are up to 4gb large (not that you'd use such a large frame today; though keep in mind that jumbo frames in IPv4 aren't really a standard so I'm sure you may find larger frames there as well). So I wouldn't hardcode it if that was really something to worry. Afaik Java SDK classes' IO buffers are by default 8kb large. –  Voo Sep 5 '11 at 16:58
add comment

In my research & testing, 8K is the optimal buffer size when reading from a socket on Linux with Java 6. If you allocate a buffer that is bigger than 8K it will simply be a waste of space. I've read that the native call Java makes uses an 8K buffer and this is why 8K is optimal but I've lose my reference. There is a bug pointing to this fact but it is not conclusive evidence: http://bugs.sun.com/view_bug.do?bug_id=6444633

That said, try experimenting on the platform you're interested in deploying on and you will find the optimal buffer size. If you're lazy, 8K is a great default.

share|improve this answer
add comment

Your Answer


