avro-netty
所属分类：网络编程
开发工具：Java
文件大小：0KB
下载次数：0
上传日期：2015-12-01 22:44:02
上 传 者：
sh-1993
说明： 使用Netty ByteBuf实现的Avro编码器解码器
(Avro Encoder Decoder implemented using Netty ByteBuf)
文件列表:
.travis.yml (93, 2015-12-01)
LICENSE.txt (11358, 2015-12-01)
pom.xml (4624, 2015-12-01)
src/ (0, 2015-12-01)
src/main/ (0, 2015-12-01)
src/main/java/ (0, 2015-12-01)
src/main/java/com/ (0, 2015-12-01)
src/main/java/com/github/ (0, 2015-12-01)
src/main/java/com/github/milenkovicm/ (0, 2015-12-01)
src/main/java/com/github/milenkovicm/avro/ (0, 2015-12-01)
src/main/java/com/github/milenkovicm/avro/generic/ (0, 2015-12-01)
src/main/java/com/github/milenkovicm/avro/generic/NettyGenericDatumReader.java (1872, 2015-12-01)
src/main/java/com/github/milenkovicm/avro/generic/NettyGenericDatumWriter.java (1208, 2015-12-01)
src/main/java/com/github/milenkovicm/avro/generic/package-info.java (128, 2015-12-01)
src/main/java/com/github/milenkovicm/avro/io/ (0, 2015-12-01)
src/main/java/com/github/milenkovicm/avro/io/ByteBufDecoder.java (6825, 2015-12-01)
src/main/java/com/github/milenkovicm/avro/io/ByteBufEncoder.java (8138, 2015-12-01)
src/main/java/com/github/milenkovicm/avro/io/package-info.java (101, 2015-12-01)
src/main/java/org/ (0, 2015-12-01)
src/main/java/org/apache/ (0, 2015-12-01)
src/main/java/org/apache/avro/ (0, 2015-12-01)
src/main/java/org/apache/avro/io/ (0, 2015-12-01)
src/main/java/org/apache/avro/io/ByteBufResolvingDecoder.java (2097, 2015-12-01)
src/main/java/org/apache/avro/io/package-info.java (133, 2015-12-01)
src/test/ (0, 2015-12-01)
src/test/java/ (0, 2015-12-01)
src/test/java/com/ (0, 2015-12-01)
src/test/java/com/github/ (0, 2015-12-01)
src/test/java/com/github/milenkovicm/ (0, 2015-12-01)
src/test/java/com/github/milenkovicm/avro/ (0, 2015-12-01)
src/test/java/com/github/milenkovicm/avro/AbstractTest.java (2583, 2015-12-01)
src/test/java/com/github/milenkovicm/avro/DataHelper.java (1925, 2015-12-01)
src/test/java/com/github/milenkovicm/avro/Helper.java (7420, 2015-12-01)
src/test/java/com/github/milenkovicm/avro/MainLoop.java (1382, 2015-12-01)
src/test/java/com/github/milenkovicm/avro/generic/ (0, 2015-12-01)
src/test/java/com/github/milenkovicm/avro/generic/NettyGenericDatumReaderTest.java (4733, 2015-12-01)
src/test/java/com/github/milenkovicm/avro/generic/NettyGenericDatumWriterTest.java (1915, 2015-12-01)
src/test/java/com/github/milenkovicm/avro/io/ (0, 2015-12-01)
... ...
# Avro Netty ByteBuf Encoder/Decoder [](https://travis-ci.org/milenkovicm/avro-netty)
Simple [Apache Avro](avro.apache.org) `Encoder/Decoder` implemented using [netty.io](netty.io) byte buffers ( `io.netty.buffer.ByteBuf` ). The main motivation for this implementation is to utilize Netty's powerful `ButeBuf` pooling.
## How to use?
To use `Encoder` just create new instance:
```java
// GenericRecord record - record to encode
final ByteBuf buffer = ByteBufAllocator.DEFAULT.buffer(1024);
final ByteBufEncoder encoder = new ByteBufEncoder();
encoder.setBuffer(buffer); // buffer to be used to store serialized object
final DatumWriter writer = new GenericDatumWriter(record.getSchema());
try {
writer.write(record, encoder);
final ByteBuf result = encoder.getBuffer(); // return the buffer with the serialized object
// do something with result, but don't forget to release it
} catch (final Exception e) {
throw new RuntimeException(e);
}
```
`Decoder` has similar api:
```java
// ByteBuf buffer - buffer to decode
final ByteBufDecoder decoder = new ByteBufDecoder();
decoder.setBuffer(buffer);
final GenericDatumReader reader = new GenericDatumReader(record.getSchema());
try {
final GenericRecord record = reader.read(null, decoder);
// do something with record
} catch (final IOException e) {
throw new RuntimeException(e);
}
```
### Encoder/Decoder Note
To get best performance from decoder don't forget to use pooled buffers and turn leak detection off!
```bash
$java -Dio.netty.leakDetectionLevel=DISABLED -Dio.netty.allocator.type=pooled ...
```
Encoder/Decoder is not thread safe, developer should ensure thread safety if needed.
## NettyGenericDatumWriter
Default `DatumWriter` implementation `GenericDatumWriter` expects `java.nio.ByteBuffer` for byte array properties, which may not be optimal for all use cases especially if powerful netty pooled buffer allocator is used. `NettyGenericDatumWriter` has ability to write `io.netty.buffer.ByteBuf` if used for byte array properties.
```java
byte[] value = new byte[] { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };
final GenericRecord record = new GenericData.Record(E_BYTES.SCHEMA$);
// use ByteBuf for byte array propertie
record.put("f_value", ByteBufAllocator.DEFAULT.buffer(value.length).writeBytes(value));
final ByteBuf buffer = ByteBufAllocator.DEFAULT.buffer(64);
final ByteBufEncoder encoder = new ByteBufEncoder();
encoder.setBuffer(buffer);
final DatumWriter writer = new NettyGenericDatumWriter(record.getSchema());
writer.write(record, encoder);
```
Note that if you use `ByteBuf` as a generic record property you'll have to release it manually, `NettyGenericDatumWriter` wont release it for you.
schema used:
```
{"namespace": "com.github.milenkovicm.avro.test.event",
"type": "record",
"name": "E_BYTES",
"fields": [
{"name": "f_value", "type": "bytes"}
]
}
```
## NettyGenericDatumReader
Default `DatumReader` implementation `GenericDatumReader` uses `java.nio.ByteBuffer` for byte array properties, which may not be optimal for all use cases especially if powerful netty pooled buffer allocator is used. `NettyGenericDatumReader` has ability to read byte arrays as `io.netty.buffer.ByteBuf`. Be careful as `ByteBuf` properties should be released when not needed.
```java
final ByteBufDecoder decoder = new ByteBufDecoder();
decoder.setBuffer(buffer);
final GenericDatumReader reader = new NettyGenericDatumReader(schema);
GenericRecord record = reader.read(null, decoder);
```
## Don't forget!
Don't forget two things:
1. to release `ByteBuf` when not needed anymore.
2. implementations are not thread safe.
that's it!
近期下载者：
相关文件：
收藏者：