Observational Heritability
Code and notebooks for running observational heritability analysis and generating figures for the manuscript. The data required to run the notebooks and the examples are available at the following URL.
http://riftehr.tatonettilab.org/
SOLARStrap requires that SOLAR is installed. You can do so from the following link:
https://www.nitrc.org/frs/?group_id=558
Estimating observational heritability using SOLARStrap requires four data files. The files for Rhinitis are provided as an example at the supporting website. Retrieve them with wget http://riftehr.tatonettilab.org/h2o_data.tar.gz.
- Trait file
- Patient demographics
- Family identifiers
- Pedigree file
- Covariates file (optional)
- Household file (optional)
Note, the script is dependent on column order and not column title. For example in the trait file below, the first column needs to include the patient identifier, it does not need to be titled ptid though (pid, individual_id etc are all fine.)
The trait file is a gzipped tab delimited file containing three columns: ptid, pheno, and value. The ptid is a local identifier for the patient. The pheno is a unique identifier for the trait (e.g. an ICD9/10 code, a LOINC code, or a local identifier). The value is either 1 or0 if the trait is dichotomous or the quantitative value if the trait is quantiative (unknown patients should be excluded from the trait file, but can be included in the pedigree and family file). Note that all individuals in the pedigree that are not in the trait file will be assigned NA as the trait value.
More than one trait can be coded in a single trait file. In this case SOLARStrap will run heritability estimates for all traits in the file.
gzcat example/rhinitis/67_trait_data.txt.gz | head
ptid pheno value
3000213365 67 1
3000267263 67 1
3000127670 67 1
3000084624 67 1
3000347920 67 1
3000179426 67 1
3000197653 67 1
3000272716 67 1
3000360729 67 1
The patient demographics file is a gzipped tab delimited file containing 6 columns: ptid, sex, birth_decade, race_code, ethnic_code, and age. If sex,race_code, or age is unknown it is coded as NULL. If ethnic_code is unknown it should be coded as U. Birth decade is not used at this time. race_code may be one of NA or NULL (unknown), W (white), or B (black). ethnic_code represents self-reporting of hispanic or not hispanic -- H, S, O will all be mapped to hispanic, other codes to non-hispanic. The race_code and the ethnic_code are used together to group patients and families into self-reported race/ethnicity groups. An ethnic_code of hispanic overrides the reported race and a race_code will be mapped to hispanic. Note that this logic is built for the demographic diversity of New York City. Uses in other areas may require different logic. In this case you may want to edit the load_demographics method in h2o_utility.py.
ptid sex birth_decade race_code ethnic_code age
3000149927 F NULL W N 68
3000191095 M NULL B N 75
3000276396 M NULL W N 63
3000181834 F NULL W H 55
3000130794 F NULL W H 60
3000341624 F NULL O D 13
3000267077 M NULL W H 56
3000089601 F NULL W H 52
3000234386 F NULL B N 45
3000176550 M NULL B N 70
The family identifiers file is a gzipped tab delimited file containing two columns: famid and ptid. All patients in the family file must also be in the pedigree.
famid ptid
30186335 3000313768
30186335 3000313771
30186335 3000313769
30186335 3000313770
30186335 3000313767
30186334 3000313763
30186334 3000313762
30186334 3000313765
30186334 3000313764
30186334 3000313766
30186337 3000313773
The pedigree file is a gzipped tab delimited file containing X columns: family_id, individual_id, father_id, mother_id, and own_ancestor. The own_ancestor column is now deprecated but must, for now, still be present. This column can be safely set to 0 for every row.
family_id individual_id father_id mother_id own_ancestor
30000851 3000068718 3000370355 3000068717 0
30000851 3000068717 3000370356 3000370357 0
30000851 3000068716 3000370355 3000068717 0
30000854 3000068719 3000370358 3000370359 0
30000854 3000068721 3000370360 3000068720 0
30000854 3000068720 3000370361 3000370362 0
30000854 3000370358 0 0 0
30000854 3000370359 0 0 0
30000854 3000370361 0 0 0
The covariates file is a gzipped tab delimited file containing at least 2 columns: the individual_id and at least one column with the covariate (more than 1 covariate column allowed)
ptid covariate_1
3000149927 1
3000191095 0
3000276396 0
3000181834 1
The household file is a gzipped tab delimited file containing at least 2 columns: the individual_id and the individual's household_id. Unknown individuals can just be excluded from the file or given an empty string value. SOLARStrap will assign a unique household identifier for these patients. Do NOT use for these patients NULL. If no household file is provided but an ACE model is selected, SOLARStrap will use the patient's mother_id as their household_id.
ptid household_id
3000149927 10456
3000191095 57623
3000276396 10456
3000181834 83215
There are five required arguments when running SOLARStrap, trait, demog, fam, ped, and type. The first four are paths to the trait, demographics, family ids, and pedigree files (as described above). type is either D for dichotomous or Q for quantitative. Here is an example using rhinitis. You will be prompted to make a ./working directory, do so with mkdir working.
python src/solarStrap_heritability.py trait=example/rhinitis/67_trait_data.txt.gz demog=example/rhinitis/patient_demog_data_with_age.txt.gz fam=example/rhinitis/family_ids.txt.gz ped=example/rhinitis/west_generic_pedigree_file.txt.gz type=D
The remaining arguments are optional. They are
-`cov` file path to Covariates file
-`hhid` file path to Household file
- `ace` Set to `yes` to run estimates modeling the household effect. The mother id will be used as the household id. Default is `no`.
- `verbose` Set to `yes` to see a lot more output. Default is `no`.
- 'nfam` The number of families to be sampled in each iteration, can be a float (proportion of total families) or an integer. Default is `0.15` (15%).
- `samples` The number of iterations to run. Default is `200`.
- `buildonly` Set to yes to only build the directories, do not run SOLAR. Default is `no`.
- `proband` Set to yes to use a proband. Default is `yes`.
- `sd` The path to the working directory. Default is `./working`.
-`h2c2coprocess` Only used when modeling household effect (ace is yes). If `no`, the final h2 and c2 are selected separately where the median c2 and median h2 are selected from the models that have a significant c2 estimate or h2 estimate respectively. Default is 'yes'. `no` should generally just be used when estimating c2 and there is no expected h2.
-`outputfams` Default is 'no'. Include a list of included families for each iteration of SOLAR.
In addition to the error and log messages that are printed to the screen, SOLARStrap will produce two results files: (1) a file of aggregated results (these are the h2o estimates) and (2) a log of all of the heritability estimates for each iteration. The former is aggregated version of the latter.
These files will be placed in the working directory provided to SOLARStrap when run (default is ./working). In addition, when SOLARStrap is run it will print out the path where these files will be saved and their names. For example:
python src/solarStrap_heritability.py trait=example/rhinitis/67_trait_data.txt.gz demog=example/rhinitis/patient_demog_data_with_age.txt.gz fam=example/rhinitis/family_ids.txt.gz ped=example/rhinitis/west_generic_pedigree_file.txt.gz type=D verbose=yes ace=yes
SolarStrap v 1.0 - Estimate heritability and shared environment of disease using observational data.
-----------------------------------------------------------------------------
Summary results will be saved in ./working/F6ZF3_solar_strap_results.csv
Results from each bootstrap will be saved at ./working/F6ZF3_solar_strap_allruns.csv
If you run solar in verbose mode you will see a log of the h2 estimates from SOLAR if SOLARStrap is running successfully.
Number of families with case: 3686
Trait Ethnicity NFam Samp AE h2 err pval ACE h2 err pval Sample AFP
67 ALL 552 1 1.00 nan 2.90e-06 0.86 0.09 1.06e-04 1.3207
67 ALL 552 2 0.17 0.09 1.73e-01 0.10 0.31 2.96e-01 1.2609
67 ALL 552 3 1.00 nan 2.90e-06 0.87 0.09 3.58e-05 1.2373
67 ALL 552 4 1.00 nan 1.50e-06 0.83 0.09 7.70e-05 1.2790
67 ALL 552 5 1.00 nan 3.20e-06 0.77 0.10 9.62e-05 1.2627
67 ALL 552 6 1.00 nan 1.04e-08 0.86 0.06 4.60e-06 1.2880
67 ALL 552 7 1.00 nan 1.34e-09 0.87 0.10 5.00e-07 1.2754
67 ALL 552 8 1.00 nan 1.30e-06 0.77 0.07 8.95e-05 1.2681
...