Downstream Datasets

This module provides some templated dataset processing classes inherited from the abstract OmniDataset class, which is used to handle datasets in the OmniGenBench framework.

Categories:

• OmniDatasetForSequenceClassification: A dataset class for sequence classification tasks.

• OmniDatasetForRegression: A dataset class for sequence regression tasks.

• OmniDatasetForTokenClassification: A dataset class for token (nucleotide) classification tasks.

• OmniDatasetForTokenRegression: A dataset class for token (nucleotide) regression tasks.

OmniDataset

Specialized dataset classes for OmniGenome framework.

This module provides specialized dataset classes for various genomic tasks, inheriting from the abstract OmniDataset. These classes handle data preparation for token classification, sequence classification, token regression, and sequence regression, integrating with tokenizers and managing metadata.

class omnigenbench.src.dataset.omni_dataset.OmniDatasetForSequenceClassification(data_source, tokenizer, max_length=None, **kwargs)

Bases: OmniDataset

Dataset class for sequence classification tasks in genomics.

This class extends OmniDataset to prepare input sequences and their corresponding sequence-level labels. It’s designed for tasks where the entire sequence needs to be classified into one of several categories.

Variables:

• metadata – Dictionary containing dataset metadata including library information

• label2id – Mapping from label strings to integer IDs

prepare_input(instance, **kwargs)

Prepare a single data instance for sequence classification.

This method handles both string sequences and dictionary instances containing sequence and label information. It tokenizes the input sequence and prepares sequence-level labels for classification.

Parameters:

• instance – A single data instance. Can be a string representing the sequence or a dictionary with ‘seq’/’sequence’ and ‘labels’/’label’ keys.

• **kwargs – Additional keyword arguments for tokenization, such as ‘padding’ and ‘truncation’.

Returns:

dict –

A dictionary of tokenized inputs, including ‘input_ids’, ‘attention_mask’,

and ‘labels’ (tensor of sequence-level labels).

Raises:

Exception – If the input instance format is unknown or if a dictionary instance does not contain a ‘label’ or ‘labels’ key, or if the label is not an integer.

class omnigenbench.src.dataset.omni_dataset.OmniDatasetForSequenceRegression(data_source, tokenizer, max_length=None, **kwargs)

Bases: OmniDataset

Dataset class for sequence regression tasks in genomics.

This class extends OmniDataset to prepare input sequences and their corresponding sequence-level regression targets. It’s designed for tasks where the entire sequence needs to be assigned a continuous value.

Variables:

metadata – Dictionary containing dataset metadata including library information

prepare_input(instance, **kwargs)

Prepare a single data instance for sequence regression.

This method handles both string sequences and dictionary instances containing sequence and regression target information. It tokenizes the input sequence and prepares sequence-level regression targets.

Parameters:

• instance – A single data instance. Can be a string representing the sequence or a dictionary with ‘seq’/’sequence’ and ‘labels’/’label’ keys.

• **kwargs – Additional keyword arguments for tokenization, such as ‘padding’ and ‘truncation’.

Returns:

dict –

A dictionary of tokenized inputs, including ‘input_ids’, ‘attention_mask’,

and ‘labels’ (tensor of sequence-level regression targets).

Raises:

Exception – If the input instance format is unknown or if a dictionary instance does not contain a ‘label’ or ‘labels’ key.

class omnigenbench.src.dataset.omni_dataset.OmniDatasetForTokenClassification(data_source, tokenizer, max_length=None, **kwargs)

Bases: OmniDataset

Dataset class for token-level classification tasks in genomics.

This class extends OmniDataset to support tokenizing genomic sequences and aligning them with token-level labels for tasks like sequence tagging.

Variables:

• metadata (dict) – Dataset metadata including library name, version, and task type.

• label2id (dict) – Mapping from label strings to integer IDs used for training.

prepare_input(instance, **kwargs)

Prepare a single data instance for token classification.

This method handles both string sequences and dictionary instances containing sequence and label information. It tokenizes the input sequence and prepares token-level labels for classification.

Parameters:

• instance – A single data instance. Can be a string representing the sequence or a dictionary with ‘seq’/’sequence’ and ‘labels’/’label’ keys.

• **kwargs – Additional keyword arguments for tokenization, such as ‘padding’ and ‘truncation’.

Returns:

dict –

A dictionary of tokenized inputs, including ‘input_ids’, ‘attention_mask’,

and ‘labels’ (tensor of token-level labels).

Raises:

Exception – If the input instance format is unknown or if a dictionary instance does not contain a ‘seq’ or ‘sequence’ key.

class omnigenbench.src.dataset.omni_dataset.OmniDatasetForTokenRegression(data_source, tokenizer, max_length=None, **kwargs)

Bases: OmniDataset

Dataset class for token regression tasks in genomics.

This class extends OmniDataset to prepare input sequences and their corresponding token-level regression targets. It’s designed for tasks where each token in a sequence needs to be assigned a continuous value.

Variables:

metadata – Dictionary containing dataset metadata including library information

prepare_input(instance, **kwargs)

Prepare a single data instance for token regression.

This method handles both string sequences and dictionary instances containing sequence and regression target information. It tokenizes the input sequence and prepares token-level regression targets.

Parameters:

• instance – A single data instance. Can be a string representing the sequence or a dictionary with ‘seq’/’sequence’ and ‘labels’/’label’ keys.

• **kwargs – Additional keyword arguments for tokenization, such as ‘padding’ and ‘truncation’.

Returns:

dict –

A dictionary of tokenized inputs, including ‘input_ids’, ‘attention_mask’,

and ‘labels’ (tensor of token-level regression targets).

Raises:

Exception – If the input instance format is unknown or if a dictionary instance does not contain a ‘seq’ or ‘sequence’ key.