A simple, go-like Go Kafka consumer. Use it like follows:
package main
import (
"fmt"
"github.com/nuance/kafka"
)
func main() {
r, _ := kafka.OpenConsumer("localhost:1234", "topic", 0,
kafka.OFF_NEWEST, kafka.DefaultReaderOptions())
buf := make([]byte, 1024)
for _, err := r.Read(buf); err == nil; _, err = r.Read(buf) {
fmt.Printf("'%s'", string(buf))
}
}
The kafka module exposes an io.ReadCloser-compatible interface:
type Consumer interface {
Seek(offset int64)
Read(buf []byte) (int, error)
GetOffset() int64
Close() error
Offsets(base int64, num int32) ([]int64, error)
}
Read will always read exactly one message or fail. Message fetches, however, are batched behind the scene, so calls to Read do not correspond 1:1 with network requests.
This API is subject to change. I'd like to eventually add support for publishers, and probably move the Offsets call off of the reader interface.