Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ADAM-651] Hive-style partitioning of parquet files by genomic position #1911

Closed
wants to merge 6 commits into from
Closed

[ADAM-651] Hive-style partitioning of parquet files by genomic position #1911

wants to merge 6 commits into from

Commits on Feb 13, 2018

  1. hive-style partitioning of parquet files by genomic position 

    removed addChrPrefix parameter

Address PR comments - part 1

Address PR comments - part 2

fix nits

Rebased

Address review comments - part 3

address reviewer comments - white space and redundant asserts

fixed isPartitioned

fixed codacy issue with return in isPartitioned

made writePartitionedParquetFlag private

updated overlap calc in refereceRegionsToDatasetQueryStirng

try2

add filter in genomicRDD

move filterOverlapping to alignment record

another try

trying as filterDatasetByOverlappingRegions

move filter mapq out of AlignmentRecord

clean up, ready to retest with Mango

feb 12th morning

added GenotypeRDD filterByOverlappingRegion

support dataset region filter in all types

updated ADAMContext to use filterByOverlap and added docs

fix doc indent by mvn

removed public referenceRegionQueryStrin function form ADAMContext
    @jpdna
    Paschall authored and jpdna committed Feb 13, 2018
    Configuration menu
    Copy the full SHA
    137ac72 View commit details
    Browse the repository at this point in the history

Commits on Feb 17, 2018

  1. Save partition size to isPartitioned flag file and load as metadata

    @jpdna
    jpdna committed Feb 17, 2018
    Configuration menu
    Copy the full SHA
    c85bdae View commit details
    Browse the repository at this point in the history

  2. Added CLI option to save as parquet dataset

    @jpdna
    jpdna committed Feb 17, 2018
    Configuration menu
    Copy the full SHA
    f69adb5 View commit details
    Browse the repository at this point in the history

  3. nit

    @jpdna
    jpdna committed Feb 17, 2018
    Configuration menu
    Copy the full SHA
    8b63059 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into hive_partitioned_v7

    @jpdna
    jpdna authored Feb 17, 2018
    Configuration menu
    Copy the full SHA
    211072b View commit details
    Browse the repository at this point in the history

  5. fixed missing extends typoe tin DataBoundAlignmentRecordRDD

    @jpdna
    jpdna committed Feb 17, 2018
    Configuration menu
    Copy the full SHA
    5c29dde View commit details
    Browse the repository at this point in the history