Skip to content

Commit 1b7c4b6

Browse files
committed
Finish 0.6.0 blog post
Change-Id: Iee16aac42268ab0d08320fd4931f267382f67c78
1 parent a78cb94 commit 1b7c4b6

File tree

2 files changed

+49
-8
lines changed

2 files changed

+49
-8
lines changed

site/_posts/2017-08-16-0.6.0-release.md

Lines changed: 47 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -32,12 +32,54 @@ memory format remains stable since the 0.3.x release.
3232
See the [Install Page][2] to learn how to get the libraries for your
3333
platform. The [complete changelog][5] is also available.
3434

35-
## Plasma Object Store
35+
## Plasma Shared Memory Object Store
3636

37+
This release includes the [Plasma Store][7], which you can read more about in
38+
the linked blog post. This system was originally developed as part of the [Ray
39+
Project][8] at the [UC Berkeley RISELab][9]. We recognized that Plasma would be
40+
highly valuable to the Arrow community as a tool for shared memory management
41+
and zero-copy deserialization. Additionally, we believe we will be able to
42+
develop a stronger software stack through sharing of IO and buffer management
43+
code.
3744

45+
The Plasma store is a server application which runs as a separate process. A
46+
reference C++ client, with Python bindings, is made available in this
47+
release. Clients can be developed in Java or other languages in the future to
48+
enable simple sharing of complex datasets through shared memory.
3849

39-
## Arrow Format Additions
50+
## Arrow Format Addition: Map type
4051

52+
We added a Map logical type to represent ordered and unordered maps
53+
in-memory. This corresponds to the `MAP` logical type annotation in the Parquet
54+
format (where maps are represented as repeated structs).
55+
56+
Map is represented as a list of structs. It is the first example of a logical
57+
type whose physical representation is a nested type. We have not yet created
58+
implementations of Map containers in any of the implementations, but this can
59+
be done in a future release.
60+
61+
As an example, the Python data:
62+
63+
```
64+
data = [{'a': 1, 'bb': 2, 'cc': 3}, {'dddd': 4}]
65+
```
66+
67+
Could be represented in an Arrow `Map<String, Int32>` as:
68+
69+
```
70+
Map<String, Int32> = List<Struct<keys: String, values: Int32>>
71+
is_valid: [true, true]
72+
offsets: [0, 3, 4]
73+
values: Struct<keys: String, values: Int32>
74+
children:
75+
- keys: String
76+
is_valid: [true, true, true, true]
77+
offsets: [0, 1, 3, 5, 9]
78+
data: abbccdddd
79+
- values: Int32
80+
is_valid: [true, true, true, true]
81+
data: [1, 2, 3, 4]
82+
```
4183
## Python Changes
4284

4385
Some highlights of Python development outside of bug fixes and general API
@@ -55,10 +97,6 @@ improvements include:
5597

5698
## Toward Arrow 1.0.0 and Beyond
5799

58-
We have much work ahead of us to build out Arrow integrations in other data
59-
systems to improve their processing performance and interoperability with other
60-
systems.
61-
62100
We are still discussing the roadmap to 1.0.0 release on the [developer mailing
63101
list][6]. The focus of the 1.0.0 release will likely be memory format stability
64102
and hardening integration tests across the remaining data types implemented in
@@ -69,3 +107,6 @@ Java and C++. Please join the discussion there.
69107
[3]: http://github.com/apache/parquet-cpp
70108
[5]: http://arrow.apache.org/release/0.6.0.html
71109
[6]: http://mail-archives.apache.org/mod_mbox/arrow-dev/
110+
[7]: http://arrow.apache.org/blog/2017/08/08/plasma-in-memory-object-store/
111+
[8]: https://ray-project.github.io/ray/
112+
[9]: https://rise.cs.berkeley.edu/

site/index.html

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,10 @@ <h1>Apache Arrow</h1>
77
<p class="lead">Powering Columnar In-Memory Analytics</p>
88
<p>
99
<a class="btn btn-lg btn-success" href="mailto:dev-subscribe@arrow.apache.org" role="button">Join Mailing List</a>
10-
<a class="btn btn-lg btn-primary" href="{{ site.baseurl }}/install/" role="button">Install (0.5.0 Release - July 23, 2017)</a>
10+
<a class="btn btn-lg btn-primary" href="{{ site.baseurl }}/install/" role="button">Install (0.6.0 Release - August 14, 2017)</a>
1111
</p>
1212
</div>
13-
<h4><strong>Latest News</strong>: <a href="{{ site.baseurl }}/blog/">Apache Arrow 0.5.0 release</a></h4>
13+
<h4><strong>Latest News</strong>: <a href="{{ site.baseurl }}/blog/">Apache Arrow 0.6.0 release</a></h4>
1414
<div class="row">
1515
<div class="col-lg-4">
1616
<h2>Fast</h2>

0 commit comments

Comments
 (0)