Tasks for parquet-mr 1.6.0 release

[asfimport]

In order to get release 1.6 of parquet-mr out, we have to do some updates.

For Apache policy:

1. -Update the header in all Parquet source files (see PARQUET-72 for examples)-
2. -Add DISCLAIMER and KEYS files (copy from parquet-format)-
3. Add the Apache Rat maven plugin to check source file license headers and make sure it passes
4. For the overall project, go through any external source that is bundled and update the root LICENSE and NOTICE files
5. -For each module that produces a binary artifact, go through the content pulled in by the maven-shade-plugin or the maven-assembly-plugin and update a LICENSE and NOTICE file for that content (see PARQUET-109 for an example).-

We also need to update the build (see PR #11):

1. Inherit from the ASF pom to pull in Apache release settings (will be done for org.apache groupId)
2. Update metadata, like SCM, mailing lists, and JIRA links
3. -Go through modules and remove any LICENSE and NOTICE files that are included by shading or assembly (relevant content should be included in parquet-mr LICENSE and NOTICE)-

Reporter: Ryan Blue / @rdblue
Assignee: Ryan Blue / @rdblue

Related issues:

• Parquet thrift can write unions that have 0 or more than 1 set value (is blocked by)

_{Note: This issue was originally created as PARQUET-111. Please see the migration documentation for further details.}

[asfimport]

Ryan Blue / @rdblue:
Okay, I've posted a work-in-progress pull request with some of these changes. Mainly, I'm trying to get the file headers and NOTICE files in order right now. That means replacing all of the current headers with the approved Apache header. That page notes:

If the source file is submitted with a copyright notice included in it, the copyright owner (or owner's agent) must either:

1. remove such notices, or
2. move them to the NOTICE file associated with each applicable project release, or
3. provide written permission for the ASF to make such removal or relocation of the notices.

There are 3 existing headers:

• "Copyright 2012 Twitter, Inc." everywhere
• "Copyright 2013 ARRIS, Inc." in parquet-tools
• "Copyright 2013 Lukas Nalezenec." in parquet-protobuf

I've replaced the Twitter headers and updated NOTICE to reflect the software grant to ASF. @julienledem, please have a look at make sure this is correct. We can add a note about where it was initially developed, if you would like to preserve that.

For parquet-tools and parquet-protobuf, I've moved the Copyright and license header into NOTICE. The documentation states that this can be done by ASF with written permission. I think we will need [~wesleypeck] and [~lukas.nalezenec] to +1 the changes to move forward as written confirmation that we can move the copyright lines to NOTICE.

I've also made a few changes to NOTICE:

• Moved the bit packing section for Daniel Lemire's work to LICENSE because it was not contributed to Parquet. I think this now follows the rules for third party code.
• Removed the notes about dependence on other Apache projects. If code or binary artifacts are included in builds, then this will need to be included in the binary LICENSE files (see PARQUET-109).

[asfimport]

Ryan Blue / @rdblue:
I think PR #109 is ready to be reviewed. Thanks!

[asfimport]

Ryan Blue / @rdblue:
Thanks [~lukas.nalezenec] for your +1!

I've separated out the parquet-tools changes into PR #114 and merged the rest of PR #109, since I don't want to rebase such a gigantic patch. I think all we need is for [~wesleypeck]'s approval and we should be able to close this. Then we'll be able to plan the first rc.

[asfimport]

Ryan Blue / @rdblue:
ARRIS got back to us via e-mail to confirm that they are okay with moving the copyright headers to the NOTICE file:

Sorry for the delay in response. We are OK to move the ARRIS copyrights header into the NOTICE file.

Sushma Annareddy

Sr. Manager, Software Engineering

ARRIS
o: +1 508-870-1151
c: +1 508-579-1678
e: sannareddy@arrisi.com
w: www.arrisi.com

Follow our blog: www.arriseverywhere.com

This electronic transmission (and any attached document) is for the sole use of the individual or entity to whom it is addressed. It is confidential and may be attorney-client privileged. In any event the Sender reserves, to the fullest extent, any "legal advice privilege". Any further distribution or copying of this message is strictly prohibited. If you received this message in error, please notify the Sender immediately and destroy the attached message (and all attached documents).

[asfimport]

Ryan Blue / @rdblue:
I've merged PR #114 with the header updates for parquet-tools. I think that the Apache requirements for a parquet-mr release are now satisfied so I'll close this issue.

[asfimport]

Julien Le Dem / @julienledem:
Awesome! Thanks @rdblue!