Phenotype preprocessing#
Prepare molecular-phenotype matrices for xQTL analysis by imputing missing values when needed, adding genomic annotations, and formatting data for chromosome-, region-, TAD-, or sample-specific analyses.
Miniprotocol Timing#
This is the total duration for the standard toy-data route; module-specific timings appear on their respective pages.
Timing: <12 min (on the toy dataset)
Overview#
This mini-protocol walks through how molecular-phenotype matrices are prepared for downstream xQTL analysis. Each step calls a workflow from phenotype_imputation.ipynb, gene_annotation.ipynb, or phenotype_formatting.ipynb.
The commands are selectable routes rather than one mandatory chain. Impute only matrices containing missing values. Choose the annotation workflow matching gene, protein, LeafCutter, or psichomics identifiers, then choose the formatting workflow required by the downstream analysis unit. GCT sample extraction and BAM subsetting are independent utilities.
Steps#
Choose a route before running commands; the 12 commands are not one mandatory chain.
Analysis goal |
Commands to run, in order |
Inputs |
|---|---|---|
Gene-expression or protein phenotype by chromosome |
2 → 7 |
|
Phenotype with missing values by chromosome |
1 → 2 → 7 |
|
Retrieve gene coordinates from Ensembl BioMart |
3 |
|
LeafCutter clusters mapped to genes |
4 |
|
LeafCutter isoforms annotated for QTL analysis |
5 |
|
psichomics isoforms annotated for QTL analysis |
6 |
|
Partition a GCT matrix by chromosome |
8 |
|
Partition a BED phenotype by predefined regions |
2 → 9 |
|
Define TAD-based phenotype regions |
2 → 10 |
|
Extract selected samples from a GCT matrix |
11 |
|
Subset BAM files to selected genomic regions |
12 |
|
Run only the row matching the intended analysis goal, following its commands in numerical order. Other imputation methods are available through phenotype_imputation.ipynb and its Command Interface.
1. Impute missing phenotype values#
What it does: Use generalized empirical Bayes matrix factorization to complete the example protein matrix.
sos run pipeline/phenotype_imputation.ipynb gEBMF \
--phenoFile input/proteomics/protocol_example.protein.missing.bed.gz \
--cwd output/phenotype_imputation_uf \
--num_factor 30
2. Add genomic coordinates to gene or protein phenotypes#
What it does: Join phenotype identifiers to a supplied coordinate annotation and write a coordinate-aware BED matrix.
sos run pipeline/gene_annotation.ipynb annotate_coord \
--cwd output/gene_annotation \
--phenoFile input/rnaseq/protocol_example.rnaseq.bed.gz \
--coordinate-annotation input/reference_data/Homo_sapiens.GRCh38.103.chr.reformatted.collapse_only.gene.ERCC.gtf \
--phenotype-id-column gene_id
3. Retrieve gene coordinates from Ensembl BioMart#
What it does: Query the selected Ensembl release when a local coordinate annotation is unavailable.
sos run pipeline/gene_annotation.ipynb annotate_coord_biomart \
--cwd output/gene_annotation \
--phenoFile input/rnaseq/protocol_example.rnaseq.gene_ID.tsv \
--ensembl-version 115
4. Map LeafCutter clusters to genes#
What it does: Assign LeafCutter clusters to genes using splice-site overlap with the gene annotation.
sos run pipeline/gene_annotation.ipynb map_leafcutter_cluster_to_gene \
--cwd output/gene_annotation \
--phenoFile input/rnaseq/protocol_example.leafcutter.phenotype.bed.gz \
--intron-count input/rnaseq/protocol_example.leafcutter.intron_count.tsv \
--coordinate-annotation input/reference_data/Homo_sapiens.GRCh38.103.chr.gtf \
--map-stra site
5. Annotate LeafCutter isoforms#
What it does: Convert LeafCutter intron-cluster phenotypes into annotated isoform features for QTL analysis.
sos run pipeline/gene_annotation.ipynb annotate_leafcutter_isoforms \
--cwd output/gene_annotation \
--phenoFile input/rnaseq/protocol_example.leafcutter.phenotype.bed.gz \
--intron-count input/rnaseq/protocol_example.leafcutter.intron_count.tsv \
--coordinate-annotation input/reference_data/Homo_sapiens.GRCh38.103.chr.gtf \
--map-stra site
6. Annotate psichomics isoforms#
What it does: Add genomic and gene annotations to psichomics-derived splicing phenotypes.
sos run pipeline/gene_annotation.ipynb annotate_psichomics_isoforms \
--cwd output/gene_annotation \
--phenoFile input/rnaseq/protocol_example.psichomics.phenotype.tsv \
--coordinate-annotation input/reference_data/Homo_sapiens.GRCh38.103.chr.gtf
7. Partition a BED phenotype by chromosome#
What it does: Split a coordinate-annotated BED phenotype into chromosome-specific files.
sos run pipeline/phenotype_formatting.ipynb phenotype_by_chrom \
--cwd output/phenotype_uf \
--phenoFile output/gene_annotation/protocol_example.rnaseq.bed.bed.gz \
--name protocol_example \
--chrom chr22
8. Partition a GCT phenotype by chromosome#
What it does: Split a coordinate-aware GCT matrix into chromosome-specific GCT files.
sos run pipeline/phenotype_formatting.ipynb phenotype_by_chrom_gct \
--cwd output/phenotype_gct \
--phenoFile input/rnaseq/protocol_example.rnaseq.gene_tpm.gct.gz \
--chrom chr21 chr22
9. Partition a phenotype by predefined regions#
What it does: Extract phenotype features falling within each region in a supplied region list.
sos run pipeline/phenotype_formatting.ipynb phenotype_by_region \
--cwd output/phenotype_by_region \
--phenoFile output/gene_annotation/protocol_example.rnaseq.bed.bed.gz \
--region-list input/reference_data/TAD/protocol_example_protein.enhanced_cis_chr22.bed
10. Define TAD-based phenotype regions#
What it does: Assign phenotype features to TAD windows and generate a region list for downstream analysis.
sos run pipeline/phenotype_formatting.ipynb phenotype_annotate_by_tad \
--cwd output/phenotype_by_region \
--phenoFile output/gene_annotation/protocol_example.rnaseq.bed.bed.gz \
--TAD-list input/reference_data/TAD/TADB_enhanced_cis.bed \
--phenotype-per-tad 2
11. Extract selected samples from a GCT matrix#
What it does: Retain only samples listed in a supplied keep file.
sos run pipeline/phenotype_formatting.ipynb gct_extract_samples \
--cwd output/phenotype_gct \
--phenoFile input/rnaseq/protocol_example.rnaseq.gene_tpm.gct.gz \
--keep-samples input/colocboost/keep_samples.txt
12. Subset BAM files by genomic region#
What it does: Extract selected chromosomes or regions from every BAM listed in the input manifest.
sos run pipeline/phenotype_formatting.ipynb bam_subsetting \
--cwd output/bam_subset \
--phenoFile input/rnaseq/bam_file_list.txt \
--region chr21 chr22
Output Files#
Route or step |
Products and relative paths |
|---|---|
Step 1 |
|
Step 2 |
|
Step 3 |
|
Step 4 |
|
Step 5 |
|
Step 6 |
|
Step 7 |
|
Step 8 |
|
Step 9 |
|
Step 10 |
|
Step 11 |
|
Step 12 |
|
Anticipated Results#
The selected route produces a molecular-phenotype matrix with the genomic annotation and layout required by its downstream xQTL analysis. Standard gene-expression routes typically end with chromosome-specific BED files; splicing routes end with gene- or isoform-annotated phenotypes; region and TAD routes end with per-region files and their manifest.
Continue with covariate preprocessing and then the appropriate association-testing workflow.
Command interface#
List the workflows and parameters available in each module used by this mini-protocol.
sos run pipeline/phenotype_imputation.ipynb -h
sos run pipeline/gene_annotation.ipynb -h
sos run pipeline/phenotype_formatting.ipynb -h