com.danga.MemCached

Class NativeHandler

java.lang.Object

com.danga.MemCached.NativeHandler

public class NativeHandler
extends java.lang.Object

Handle encoding standard Java types directly which can result in significant memory savings: Currently the Memcached driver for Java supports the setSerialize() option. This can increase performance in some situations but has a few issues: Code that performs class casting will throw ClassCastExceptions when setSerialize is enabled. For example: mc.set( "foo", new Integer( 1 ) ); Integer output = (Integer)mc.get("foo"); Will work just file when setSerialize is true but when its false will just throw a ClassCastException. Also internally it doesn't support Boolean and since toString is called wastes a lot of memory and causes additional performance issue. For example an Integer can take anywhere from 1 byte to 10 bytes. Due to the way the memcached slab allocator works it seems like a LOT of wasted memory to store primitive types as serialized objects (from a performance and memory perspective). In our applications we have millions of small objects and wasted memory would become a big problem. For example a Serialized Boolean takes 47 bytes which means it will fit into the 64byte LRU. Using 1 byte means it will fit into the 8 byte LRU thus saving 8x the memory. This also saves the CPU performance since we don't have to serialize bytes back and forth and we can compute the byte[] value directly. One problem would be when the user calls get() because doing so would require the app to know the type of the object stored as a bytearray inside memcached (since the user will probably cast). If we assume the basic types are interned we could use the first byte as the type with the remaining bytes as the value. Then on get() we could read the first byte to determine the type and then construct the correct object for it. This would prevent the ClassCastException I talked about above. We could remove the setSerialize() option and just assume that standard VM types are always internd in this manner. mc.set( "foo", new Boolean.TRUE ); Boolean b = (Boolean)mc.get( "foo" ); And the type casts would work because internally we would create a new Boolean to return back to the client. This would reduce memory footprint and allow for a virtual implementation of the Externalizable interface which is much faster than Serialzation. Currently the memory improvements would be: java.lang.Boolean - 8x performance improvement (now just two bytes) java.lang.Integer - 16x performance improvement (now just 5 bytes) Most of the other primitive types would benefit from this optimization. java.lang.Character being another obvious example. I know it seems like I'm being really picky here but for our application I'd save 1G of memory right off the bat. We'd go down from 1.152G of memory used down to 144M of memory used which is much better IMO. http://java.sun.com/docs/books/tutorial/native1.1/integrating/types.html

Author:

Kevin A. Burton, Greg Whalin

Field Summary

Fields

Modifier and Type	Field and Description
private static org.apache.log4j.Logger	log

Constructor Summary

Constructors

Constructor and Description
NativeHandler()

Method Summary

Modifier and Type	Method and Description
static java.lang.Object	decode(byte[] b, int flag) Decodes byte array using memcache flag to determine type.
protected static java.lang.Boolean	decodeBoolean(byte[] b)
protected static java.lang.Byte	decodeByte(byte[] b)
protected static byte[]	decodeByteArr(byte[] b)
protected static java.lang.Character	decodeCharacter(byte[] b)
protected static java.util.Date	decodeDate(byte[] b)
protected static java.lang.Double	decodeDouble(byte[] b)
protected static java.lang.Float	decodeFloat(byte[] b)
protected static java.lang.Integer	decodeInteger(byte[] b)
protected static java.lang.Long	decodeLong(byte[] b)
protected static java.lang.Short	decodeShort(byte[] b)
protected static java.lang.String	decodeString(byte[] b)
protected static java.lang.StringBuffer	decodeStringBuffer(byte[] b)
protected static java.lang.StringBuilder	decodeStringBuilder(byte[] b)
protected static byte[]	encode(java.lang.Boolean value)
protected static byte[]	encode(java.lang.Byte value)
protected static byte[]	encode(byte[] value)
protected static byte[]	encode(java.lang.Character value)
protected static byte[]	encode(java.util.Date value)
protected static byte[]	encode(double value)
protected static byte[]	encode(float value)
protected static byte[]	encode(int value)
protected static byte[]	encode(long value)
static byte[]	encode(java.lang.Object value) Encodes supported types
protected static byte[]	encode(java.lang.Short value)
protected static byte[]	encode(java.lang.String value)
protected static byte[]	encode(java.lang.StringBuffer value)
protected static byte[]	encode(java.lang.StringBuilder value)
protected static byte[]	getBytes(int value)
protected static byte[]	getBytes(long value)
static int	getMarkerFlag(java.lang.Object value) Returns the flag for marking the type of the byte array.
static boolean	isHandled(java.lang.Object value) Detemine of object can be natively serialized by this class.
protected static int	toInt(byte[] b) This works by taking each of the bit patterns and converting them to ints taking into account 2s complement and then adding them..
protected static long	toLong(byte[] b)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

private static org.apache.log4j.Logger log

Constructor Detail

NativeHandler

public NativeHandler()

Method Detail

isHandled

public static boolean isHandled(java.lang.Object value)

Detemine of object can be natively serialized by this class.

Parameters:

value - Object to test.

Returns:

true/false

getMarkerFlag

public static int getMarkerFlag(java.lang.Object value)

Returns the flag for marking the type of the byte array.

Parameters:

value - Object we are storing.

Returns:

int marker

encode

public static byte[] encode(java.lang.Object value)
                     throws java.lang.Exception

Encodes supported types

Parameters:

value - Object to encode.

Returns:

byte array

Throws:

java.lang.Exception - If fail to encode.

encode

protected static byte[] encode(java.lang.Byte value)

encode

protected static byte[] encode(java.lang.Boolean value)

encode

protected static byte[] encode(int value)

encode

protected static byte[] encode(long value)
                        throws java.lang.Exception

Throws:

java.lang.Exception

encode

protected static byte[] encode(java.util.Date value)

encode

protected static byte[] encode(java.lang.Character value)

encode

protected static byte[] encode(java.lang.String value)
                        throws java.lang.Exception

Throws:

java.lang.Exception

encode

protected static byte[] encode(java.lang.StringBuffer value)
                        throws java.lang.Exception

Throws:

java.lang.Exception

encode

protected static byte[] encode(float value)
                        throws java.lang.Exception

Throws:

java.lang.Exception

encode

protected static byte[] encode(java.lang.Short value)
                        throws java.lang.Exception

Throws:

java.lang.Exception

encode

protected static byte[] encode(double value)
                        throws java.lang.Exception

Throws:

java.lang.Exception

encode

protected static byte[] encode(java.lang.StringBuilder value)
                        throws java.lang.Exception

Throws:

java.lang.Exception

encode

protected static byte[] encode(byte[] value)

getBytes

protected static byte[] getBytes(long value)

getBytes

protected static byte[] getBytes(int value)

decode

public static java.lang.Object decode(byte[] b,
                                      int flag)
                               throws java.lang.Exception

Decodes byte array using memcache flag to determine type.

Parameters:

b -

marker -

Returns:

Throws:

java.lang.Exception

decodeByte

protected static java.lang.Byte decodeByte(byte[] b)

decodeBoolean

protected static java.lang.Boolean decodeBoolean(byte[] b)

decodeInteger

protected static java.lang.Integer decodeInteger(byte[] b)

decodeLong

protected static java.lang.Long decodeLong(byte[] b)
                                    throws java.lang.Exception

Throws:

java.lang.Exception

decodeCharacter

protected static java.lang.Character decodeCharacter(byte[] b)

decodeString

protected static java.lang.String decodeString(byte[] b)
                                        throws java.lang.Exception

Throws:

java.lang.Exception

decodeStringBuffer

protected static java.lang.StringBuffer decodeStringBuffer(byte[] b)
                                                    throws java.lang.Exception

Throws:

java.lang.Exception

decodeFloat

protected static java.lang.Float decodeFloat(byte[] b)
                                      throws java.lang.Exception

Throws:

java.lang.Exception

decodeShort

protected static java.lang.Short decodeShort(byte[] b)
                                      throws java.lang.Exception

Throws:

java.lang.Exception

decodeDouble

protected static java.lang.Double decodeDouble(byte[] b)
                                        throws java.lang.Exception

Throws:

java.lang.Exception

decodeDate

protected static java.util.Date decodeDate(byte[] b)

decodeStringBuilder

protected static java.lang.StringBuilder decodeStringBuilder(byte[] b)
                                                      throws java.lang.Exception

Throws:

java.lang.Exception

decodeByteArr

protected static byte[] decodeByteArr(byte[] b)

toInt

protected static int toInt(byte[] b)

This works by taking each of the bit patterns and converting them to ints taking into account 2s complement and then adding them..

Parameters:

b -

Returns:

toLong

protected static long toLong(byte[] b)