Iceberg Ruby

Apache Iceberg for Ruby

Installation

Add this line to your application’s Gemfile:

gem "iceberg"

Getting Started

Create a client for an Iceberg catalog

catalog = Iceberg::RestCatalog.new(uri: "http://localhost:8181")

Create a namespace

catalog.create_namespace("main")

Create a table

catalog.create_table("main.events") do |t|
  t.bigint "id"
  t.float "value"
end

Or

df = Polars::DataFrame.new({"id" => [1, 2], "value" => [3.0, 4.0]})
table = catalog.create_table("main.events", schema: df.schema)
table.append(df)

Load a table

table = catalog.load_table("main.events")

Query a table

table.to_polars.collect

Catalog Types

REST

Iceberg::RestCatalog.new(
  uri: "http://localhost:8181"
)

SQL

Iceberg::SqlCatalog.new(
  uri: "postgres://localhost:5432/iceberg",
  warehouse: "s3://my-bucket"
)

Memory

Iceberg::MemoryCatalog.new(
  warehouse: "/tmp/warehouse"
)

Namespaces

List namespaces

catalog.list_namespaces

Create a namespace

catalog.create_namespace("main")

Check if a namespace exists

catalog.namespace_exists?("main")

Get the properties of a namespace

catalog.namespace_properties("main")

Update a namespace

catalog.update_namespace("main", properties: {})

Drop a namespace

catalog.drop_namespace("main")

Tables

List tables

catalog.list_tables("main")

Create a table

catalog.create_table("main.events") do |t|
  t.integer "id"
  t.float "value"
end

Load a table

catalog.load_table("main.events")

Check if a table exists

catalog.table_exists?("main.events")

Rename a table

catalog.rename_table("main.events", "main.events2")

Register a table

catalog.register_table("main.events", "metadata.json")

Drop a table

catalog.drop_table("main.events")

Static Tables

Load a static table

Iceberg::StaticTable.new("metadata.json")

History

View the changelog

Contributing

Everyone is encouraged to help improve this project. Here are a few ways you can help:

• Report bugs
• Fix bugs and submit pull requests
• Write, clarify, or fix documentation
• Suggest or add new features

To get started with development:

git clone https://github.com/ankane/iceberg-ruby.git
cd iceberg-ruby
bundle install
bundle exec rake compile

# memory catalog
bundle exec rake test:memory

# REST catalog
docker run -p 8181:8181 apache/iceberg-rest-fixture
bundle exec rake test:rest

# SQL catalog
createdb iceberg_ruby_test
bundle exec rake test:sql