Decrease the default page size to 64k

[asfimport]

Impala uses 64k for a page size and recommends smaller page sizes over the current 1MB default. We're considering an absolute minimum row group size of 1MB, so it makes sense to dial the page size down a bit.

Reporter: Ryan Blue / @rdblue

_{Note: This issue was originally created as PARQUET-216. Please see the migration documentation for further details.}

[asfimport]

Julien Le Dem / @julienledem:
We should do some measurement on our existing datasets. (CC @isnotinvain)
Converting one file of each should be enough.
Things to look at:

• file size
• read speed
• write speed

[asfimport]

Daniel Weeks / @danielcweeks:
We could add something like this to the PARQUET-165 (still outstanding) and have runs with different sizes as part of the benchmark.

[asfimport]

Nezih Yigitbasi / @nezihyigitbasi:
There are already some benchmarks for a few different page sizes in that PR. I will be happy to coordinate with you to add more and then rebase & get it merged.

[asfimport]

Jacques Nadeau / @jacques-n:
I think we need substantially more information on this before a decision should be made. We haven't even scratched the possibilities with regards to page pruning. By making the pages very small, we substantially reduce the effectiveness of zonemap-like algorithms. I'm currently experimenting/prototyping moving page headers to the footer and adding advanced statistics to the page metadata. This will likely allow vast improvements over current patterns but is only reasonable in the case that pages are reasonably large.

I think a lot more work should go into trying to leverage the current default page sizes because they are based on sound algorithmic grounding. Only after we've explored that should we consider moving to a new size. In the meantime, people can always change the option if it is helpful for their particular workload.