Phenotype data imputation

Phenotype data imputation#

Imputes missing values in a molecular phenotype matrix, so downstream models see a complete matrix.

Overview#

Molecular phenotype matrices are rarely complete: a feature can be unmeasured in a sample for technical reasons, and the models downstream generally require a full matrix. Dropping every feature with a missing entry throws away real signal, so this step imputes the gaps instead.

Empirical Bayes matrix factorisation is the primary method here, EBMF for a single matrix and gEBMF when the data are grouped, following Qi et al., medRxiv 2023. Seven alternatives are also provided - missForest, missXGBoost, k-nearest neighbours, SoftImpute, mean imputation and limit-of-detection imputation - so a method can be matched to the assay.

When to run it. After phenotype QC and normalisation, before covariate preprocessing and the association scan.

Input#

--phenoFile: the molecular phenotype matrix in BED layout, the first four columns being #chr, start, end and ID and the rest one column per sample, with missing entries written as NA. Example input/proteomics/protocol_example.protein.missing.bed.gz, a toy matrix with roughly 10% of values set to missing so the methods have gaps to fill:
```
#chr   start   end     ID                            SAMPLE_001          SAMPLE_002
chr12  752578  752579  chr12_ENSG00000060237_Q9H4A3  0.238966360190167   -0.611171227886468
chr12  990508  990509  chr12_ENSG00000082805_Q8IUD2  NA                  -1.86313205860919
```
--num_factor: the number of factors for the EBMF and gEBMF models. It must be smaller than the number of samples; the toy set has 60, so the examples use 30.
--cwd: the directory outputs are written to.
--save_flash: off by default. When set, gEBMF also writes the fitted flashier factor object.
--no-qc-prior-to-impute: leaves quality control on by default. Passing it skips QC, which also means the QC matrix is not available to the output.

Output#

{cwd}/{phenoFile}.imputed.bed.gz - the imputed matrix in the same BED layout as the input, bgzipped and tabix-indexed, with a .tbi beside it. Example output/phenotype_imputation_uf/protocol_example.protein.missing.bed.imputed.bed.gz, where the gaps shown in the input preview above are filled:
```
#chr	start	end	ID	SAMPLE_001	SAMPLE_002	SAMPLE_003	SAMPLE_004	SAMPLE_005	SAMPLE_006	SAMP
chr12	752578	752579	chr12_ENSG00000060237_Q9H4A3	0.238966360190167	-0.611171227886468	1.
chr12	990508	990509	chr12_ENSG00000082805_Q8IUD2	-0.762712503460405	-1.86313205860919	1.
```
{cwd}/{phenoFile}.gEBMF_factors.rds - the saved flashier factor object from a gEBMF fit, written only when --save_flash True is given. It is a binary R object, so it has no text preview; load it with readRDS. Example output/phenotype_imputation_uf/protocol_example.protein.missing.bed.gEBMF_factors.rds.

Minimal Working Example#

Eight imputation methods plus a filter that drops features too sparse to impute. Pick one method. Two practical notes for the toy data: --num_factor for the EBMF and gEBMF factor models must be smaller than the number of samples, which is 60 here, so the examples use --num_factor 30; and leave quality control enabled, that is do not pass --no-qc-prior-to-impute, so the QC matrix used for the output is available to every method.

Grouped Empirical Bayes Matrix Factorization (gEBMF)#

Grouped Empirical Bayes Matrix Factorization. Extends EBMF by fitting factors within row groups (here, by chromosome), so structure shared across molecular features in the same group is borrowed when filling in missing values. This is the recommended default for the protocol.