Post-APA calling: Imputation and QC

Fills the missing values in a PDUI matrix and quantile-normalises the result.

Overview

Alternative polyadenylation is quantified as PDUI, the fraction of transcripts using the distal poly(A) site. DaPars leaves gaps wherever a gene had too little coverage in a sample to call that fraction, and downstream models need a complete matrix.

This step imputes those gaps with the impute package and quantile-normalises the filled matrix so samples are on a common scale. A second, optional workflow renames the sample columns from DaPars internal IDs to the names used elsewhere in the study.

When to run it. After APA calling, before the association scan.

Input

• --cwd: the DaPars working directory, holding one apa_{chr}/ subdirectory per chromosome with the raw Dapars_result_result_temp.{chr}.txt PDUI matrix in it. Genes are rows, samples are columns. Required.

• --chrlist: the chromosomes to process, one or more values. Required.

• --match, used by APArename only: a two-column table mapping the DaPars project IDs to the sample names to use instead. Example input/covariate/protocol_example.apa_matchtable.txt:

  ProjID     SampleID
  Sample001  SampleA
  

• --modular-script-dir: the code/script directory holding the analysis scripts.

• --thread, --ncore, --job-size, --walltime, --mem: job resources.

• --container: the image to run in.

Output

• {cwd}/apa_{chr}/Dapars_result_impute_{chr}.bed - the imputed, quantile-normalised PDUI matrix for one chromosome, from APAimpute.

• {cwd}/apa_{chr}/Dapars_result_impute_renamed_{chr}.bed - the same matrix with sample columns relabelled, from APArename.

• {cwd}/Dapars_allchrom.bed and {cwd}/Dapars_allchrom_renamed.bed - the per-chromosome results concatenated, the second bgzipped and tabix-indexed as Dapars_allchrom_renamed.bed.gz with a .tbi. Example output/apa/Dapars_allchrom_renamed.bed.gz, 4 columns and 1 rows:

  #chr  start  end  Gene
  

Minimal Working Example

Two workflows: APAimpute fills and normalises the matrix, and APArename optionally relabels its columns. Run the first; run the second only if the DaPars sample IDs need mapping to your own names.

Impute and normalise

Timing: TBD (on toy dataset)

sos run pipeline/apa_impute.ipynb APAimpute \
    --cwd output/apa \
    --chrlist chr22

Rename the sample columns

Optionally, rename the sample columns of the imputed PDUI matrix using a match table (maps the Dapars sample IDs to your desired sample names):

Timing: TBD (on toy dataset)

sos run pipeline/apa_impute.ipynb APArename \
    --cwd output/apa \
    --chrlist chr22 \
    --match input/covariate/protocol_example.apa_matchtable.txt

Command Interface

sos run pipeline/apa_impute.ipynb -h

ERROR: Notebook JSON is invalid: %s
usage: sos run code/SoS/molecular_phenotypes/QC/apa_impute.ipynb
               [workflow_name | -t targets] [options] [workflow_options]
  workflow_name:        Single or combined workflows defined in this script
  targets:              One or more targets to generate
  options:              Single-hyphen sos parameters (see "sos run -h" for details)
  workflow_options:     Double-hyphen workflow-specific parameters

Workflows:
  APAimpute
  APArename

Global Workflow Options:
  --walltime 40h
  --mem 32G
  --ncore 16 (as int)
  --cwd VAL (as path, required)
  --modular-script-dir code/script (as path)
  --thread 8 (as int)
  --job-size 1 (as int)
  --container ''

Sections
  APAimpute:
    Workflow Options:
      --chrlist VAL VAL ... (as type, required)
  APArename_1:
    Workflow Options:
      --match VAL (as path, required)
      --chrlist VAL VAL ... (as type, required)
  APArename_2:
    Workflow Options:
      --match VAL (as path, required)
      --chrlist VAL VAL ... (as type, required)
  APArename_3:
    Workflow Options:
      --match 'input/covariate/protocol_example.apa_matchtable.txt'
  APArename_4:

Workflow implementation

[global]
parameter: walltime = '40h'
parameter: mem = '32G'
parameter: ncore = 16
parameter: cwd = path
parameter: modular_script_dir = path('code/script')  # override with --modular-script-dir
parameter: thread = 8
parameter: job_size = 1
parameter: container = ''

[APAimpute]
parameter: chrlist = list
input: [f'{cwd}/apa_{x}/Dapars_result_result_temp.{x}.txt' for x in chrlist]
output: [f'{cwd}/apa_{x}/Dapars_result_impute_{x}.bed' for x in chrlist]
task: trunk_workers = 1, trunk_size = job_size, walltime = walltime, mem = mem, cores = ncore
bash: expand= "${ }"
    Rscript ${modular_script_dir}/molecular_phenotypes/QC/apa_impute.R --step impute \
        --cwd ${cwd} \
        --chrlist "${chrlist}"

[APArename_1]
parameter: match = path
parameter: chrlist = list
input: [f'{cwd}/apa_{x}/Dapars_result_impute_{x}.bed' for x in chrlist], group_by = 1
output: [f'{cwd}/apa_{x}/Dapars_result_impute_renamed_{x}.bed' for x in chrlist], group_by = 1
bash: expand= "${ }"
    Rscript ${modular_script_dir}/molecular_phenotypes/QC/apa_impute.R --step rename \
        --input "${_input}" \
        --match "${match}" \
        --output "${_output}" \
        --update-end

[APArename_2]
parameter: match = path
parameter: chrlist = list
input: [f'{cwd}/apa_{x}/Dapars_result_impute_renamed_{x}.bed' for x in chrlist], group_by = 1
output: [f'{cwd}/apa_{x}/Dapars_result_impute_renamed_{x}.bed.gz' for x in chrlist], group_by = 1
bash: expand= "${ }"
    Rscript ${modular_script_dir}/molecular_phenotypes/QC/apa_impute.R --step bgzip_index \
        --input "${_input}" \
        --output "${_output}"

[APArename_3]
parameter: match = "input/covariate/protocol_example.apa_matchtable.txt" #path
input: f'{cwd}/Dapars_allchrom.bed'
output: f'{cwd}/Dapars_allchrom_renamed.bed'
bash: expand= "${ }"
    Rscript ${modular_script_dir}/molecular_phenotypes/QC/apa_impute.R --step rename \
        --input "${_input}" \
        --match "${match}" \
        --output "${_output}"

[APArename_4]
output: f'{_input}.gz', f'{_input}.gz.tbi'
bash: expand = "${ }"
    Rscript ${modular_script_dir}/molecular_phenotypes/QC/apa_impute.R --step bgzip_index \
        --input "${_input}" \
        --output "${_output[0]}"