Generating a Uniform Allele Distribution Dataset¶

This notebook outlines the process for generating synthetic Adaptive Immune Receptor Repertoire (AIRR) sequences using GenAIRR. The focus is on leveraging GenAIRR's modular pipeline structure to create a dataset with uniformly distributed allele combinations. The dataset can be used for benchmarking and analyzing various bioinformatics tools and methods.

Initial Setup and Imports¶

In this section, we import the necessary modules from GenAIRR and related Python libraries.

In [ ]:

from GenAIRR import Pipeline, steps, HUMAN_IGH_OGRDB
from GenAIRR.mutation import Uniform
import pandas as pd
import random
from itertools import product

# Use the built-in heavy chain data config
data_config_builtin = HUMAN_IGH_OGRDB

from GenAIRR import Pipeline, steps, HUMAN_IGH_OGRDB from GenAIRR.mutation import Uniform import pandas as pd import random from itertools import product # Use the built-in heavy chain data config data_config_builtin = HUMAN_IGH_OGRDB

Preparing Allele Lists¶

We extract all possible alleles from the DataConfig to create lists of V, D, and J alleles. This helps in forming combinations for the dataset.

In [ ]:

V_alleles = [i for j in data_config_builtin.v_alleles for i in data_config_builtin.v_alleles[j]]
D_alleles = [i for j in data_config_builtin.d_alleles for i in data_config_builtin.d_alleles[j]]
J_alleles = [i for j in data_config_builtin.j_alleles for i in data_config_builtin.j_alleles[j]]

V_alleles = [i for j in data_config_builtin.v_alleles for i in data_config_builtin.v_alleles[j]] D_alleles = [i for j in data_config_builtin.d_alleles for i in data_config_builtin.d_alleles[j]] J_alleles = [i for j in data_config_builtin.j_alleles for i in data_config_builtin.j_alleles[j]]

Generating Allele Combinations¶

We use Python's itertools.product to create all possible combinations of V-D, D-J, and V-D-J alleles.

This approach ensures that each possible combination of alleles is represented in the dataset, allowing for comprehensive benchmarking.

In [ ]:

random.shuffle(V_alleles)
random.shuffle(D_alleles)
random.shuffle(J_alleles)

# Generate uniform combinations
V_D_combinations = list(product(V_alleles, D_alleles))
D_J_combinations = list(product(D_alleles, J_alleles))
V_D_J_combinations = list(product(V_alleles, D_alleles, J_alleles))

all_combinations = V_D_combinations + D_J_combinations + V_D_J_combinations

random.shuffle(V_alleles) random.shuffle(D_alleles) random.shuffle(J_alleles) # Generate uniform combinations V_D_combinations = list(product(V_alleles, D_alleles)) D_J_combinations = list(product(D_alleles, J_alleles)) V_D_J_combinations = list(product(V_alleles, D_alleles, J_alleles)) all_combinations = V_D_combinations + D_J_combinations + V_D_J_combinations

In [ ]:

print(f"Number of V-D combinations: {len(V_D_combinations):,}")
print(f"Number of D-J combinations: {len(D_J_combinations):,}")
print(f"Number of V-D-J combinations: {len(V_D_J_combinations):,}")
print(f"Total number of combinations: {len(all_combinations):,}")

print(f"Number of V-D combinations: {len(V_D_combinations):,}") print(f"Number of D-J combinations: {len(D_J_combinations):,}") print(f"Number of V-D-J combinations: {len(V_D_J_combinations):,}") print(f"Total number of combinations: {len(all_combinations):,}")

Setting Up the Augmentation Pipeline¶

In this step, we set up a GenAIRR simulation pipeline. We iterate over the list of allele combinations and dynamically update the first step of the pipeline to enforce specific V/D/J alleles for each sequence.

In [ ]:

from tqdm.auto import tqdm

# Define pipeline steps (the first step will be replaced in the loop)
pipeline_steps = [
    steps.SimulateSequence(Uniform(min_mutation_rate=0.003, max_mutation_rate=0.25)),
    steps.FixVPositionAfterTrimmingIndexAmbiguity(),
    steps.FixDPositionAfterTrimmingIndexAmbiguity(),
    steps.FixJPositionAfterTrimmingIndexAmbiguity(),
    steps.CorrectForVEndCut(),
    steps.CorrectForDTrims(),
    steps.DistillMutationRate(),
    steps.CorruptSequenceBeginning(
        probability=0.7,
        event_weights=(0.4, 0.4, 0.2),
        nucleotide_add_coefficient=210,
        nucleotide_remove_coefficient=310,
        nucleotide_add_after_remove_coefficient=50,
    ),
    steps.EnforceSequenceLength(max_length=576),
    steps.InsertNs(n_ratio=0.02, probability=0.5),
    steps.ShortDValidation(),
    steps.InsertIndels(probability=0.5, max_indels=5, insertion_probability=0.5, deletion_probability=0.5),
]

simulated_sequences = []
n_repeats = 5

# Below can be replaced / repeated for any specific combinations
for V, D in tqdm(V_D_combinations):
    sim_step = steps.SimulateSequence(
        Uniform(min_mutation_rate=0.003, max_mutation_rate=0.25),
        productive=False,
        specific_v=V,
        specific_d=D
    )
    pipeline_steps[0] = sim_step
    pipeline = Pipeline(config=HUMAN_IGH_OGRDB, steps=pipeline_steps)
    for _ in range(n_repeats):
        simulated_sequences.append(pipeline.execute().get_dict())

from tqdm.auto import tqdm # Define pipeline steps (the first step will be replaced in the loop) pipeline_steps = [ steps.SimulateSequence(Uniform(min_mutation_rate=0.003, max_mutation_rate=0.25)), steps.FixVPositionAfterTrimmingIndexAmbiguity(), steps.FixDPositionAfterTrimmingIndexAmbiguity(), steps.FixJPositionAfterTrimmingIndexAmbiguity(), steps.CorrectForVEndCut(), steps.CorrectForDTrims(), steps.DistillMutationRate(), steps.CorruptSequenceBeginning( probability=0.7, event_weights=(0.4, 0.4, 0.2), nucleotide_add_coefficient=210, nucleotide_remove_coefficient=310, nucleotide_add_after_remove_coefficient=50, ), steps.EnforceSequenceLength(max_length=576), steps.InsertNs(n_ratio=0.02, probability=0.5), steps.ShortDValidation(), steps.InsertIndels(probability=0.5, max_indels=5, insertion_probability=0.5, deletion_probability=0.5), ] simulated_sequences = [] n_repeats = 5 # Below can be replaced / repeated for any specific combinations for V, D in tqdm(V_D_combinations): sim_step = steps.SimulateSequence( Uniform(min_mutation_rate=0.003, max_mutation_rate=0.25), productive=False, specific_v=V, specific_d=D ) pipeline_steps[0] = sim_step pipeline = Pipeline(config=HUMAN_IGH_OGRDB, steps=pipeline_steps) for _ in range(n_repeats): simulated_sequences.append(pipeline.execute().get_dict())

In [ ]:

print(f"Simulated Sequences : {len(simulated_sequences):,}")

print(f"Simulated Sequences : {len(simulated_sequences):,}")

In [ ]:

pd.DataFrame(simulated_sequences)

pd.DataFrame(simulated_sequences)