API Reference¶

This page documents the public Python API for corpusgen.

Top-Level Functions¶

These are the primary entry points, importable directly from corpusgen.

evaluate¶

evaluate ¶

evaluate(sentences: list[str], language: str = 'en-us', target_phonemes: list[str] | str | None = None, unit: str = 'phoneme') -> EvaluationReport

Evaluate a corpus of sentences for phoneme coverage.

This is the primary user-facing API. It phonemizes the input sentences, tracks coverage against a target phoneme inventory, and returns a structured report.

Parameters:

Name	Type	Description	Default
`sentences`	`list[str]`	List of text sentences to evaluate.	required
`language`	`str`	Language code for G2P conversion (e.g., 'en-us', 'fr-fr').	`'en-us'`
`target_phonemes`	`list[str] \| str \| None`	Target phoneme inventory to measure coverage against. If None, the inventory is derived from all unique phonemes found in the corpus (resulting in 100% coverage — useful for inventory discovery). If the string `"phoible"`, the target is automatically fetched from the PHOIBLE database for the given language (requires cached PHOIBLE data).	`None`
`unit`	`str`	Coverage unit type — "phoneme", "diphone", or "triphone".	`'phoneme'`

Returns:

Type	Description
`EvaluationReport`	EvaluationReport with coverage metrics, per-sentence details,
`EvaluationReport`	phoneme counts, and source provenance.

Raises:

Type	Description
`ValueError`	If unit is not one of "phoneme", "diphone", "triphone".

select_sentences¶

select_sentences ¶

select_sentences(candidates: list[str], language: str = 'en-us', target_phonemes: list[str] | str | None = None, unit: str = 'phoneme', algorithm: str = 'greedy', max_sentences: int | None = None, target_coverage: float = 1.0, candidate_phonemes: list[list[str]] | None = None, weights: dict[str, float] | None = None, **algorithm_kwargs: Any) -> SelectionResult

Select sentences from candidates for maximal phoneme coverage.

This is the primary user-facing API for corpus selection. It handles G2P conversion, target inventory resolution, and algorithm dispatch.

Parameters:

Name	Type	Description	Default
`candidates`	`list[str]`	List of candidate sentences (raw text).	required
`language`	`str`	Language code for G2P conversion (e.g., 'en-us', 'fr-fr'). Ignored if candidate_phonemes is provided.	`'en-us'`
`target_phonemes`	`list[str] \| str \| None`	Target phoneme inventory. If None, derived from all unique phonemes in candidates. If `"phoible"`, fetched from the PHOIBLE database for the given language.	`None`
`unit`	`str`	Coverage unit type — "phoneme", "diphone", or "triphone".	`'phoneme'`
`algorithm`	`str`	Selection algorithm — "greedy", "celf", "stochastic", "ilp", "distribution", or "nsga2".	`'greedy'`
`max_sentences`	`int \| None`	Maximum number of sentences to select (budget). None means no limit.	`None`
`target_coverage`	`float`	Stop when this coverage fraction is reached.	`1.0`
`candidate_phonemes`	`list[list[str]] \| None`	Pre-phonemized candidates. If provided, G2P is skipped entirely. Must have same length as candidates.	`None`
`weights`	`dict[str, float] \| None`	Optional mapping from unit to weight for marginal gain. If None, all units are weighted equally (1.0).	`None`
`**algorithm_kwargs`	`Any`	Passed to the algorithm constructor (e.g., epsilon and seed for stochastic, target_distribution for distribution, population_size/n_generations for nsga2).	`{}`

Returns:

Type	Description
`SelectionResult`	SelectionResult with selected sentences and coverage metrics.

Raises:

Type	Description
`ValueError`	If algorithm, unit, or inputs are invalid.
`ImportError`	If the requested algorithm needs an uninstalled dependency.

get_inventory¶

get_inventory ¶

get_inventory(language: str, source: str | None = None) -> Inventory

Get a PHOIBLE phoneme inventory for a language.

Accepts either an espeak-ng voice code (e.g., 'en-us', 'fr-fr') or an ISO 639-3 / Glottocode identifier (e.g., 'eng', 'stan1293'). Tries espeak mapping first, then falls back to direct PHOIBLE lookup.

Parameters:

Name	Type	Description	Default
`language`	`str`	espeak-ng code, ISO 639-3 code, or Glottocode.	required
`source`	`str \| None`	Optional PHOIBLE source filter (e.g., 'spa', 'upsid').	`None`

Returns:

Type	Description
`Inventory`	An Inventory object with phonemes, features, and metadata.

Raises:

Type	Description
`FileNotFoundError`	If the cached PHOIBLE CSV is unavailable. Download it with `PhoibleDataset().download()` before calling this function.
`RuntimeError`	If the default PHOIBLE cache does not match corpusgen's pinned, checksum-verified revision.
`KeyError`	If the language identifier is not found.

Data Models¶

Inventory¶

Inventory `dataclass` ¶

Inventory(inventory_id: int, language_name: str, iso639_3: str, glottocode: str, specific_dialect: str | None, source: str, segments: list[Segment])

A single phonological inventory from one PHOIBLE source.

Represents a complete phoneme inventory for a language/dialect as documented by a specific source. Preserves all PHOIBLE metadata including segments, allophones, distinctive features, and provenance.

Attributes:

Name	Type	Description
`inventory_id`	`int`	PHOIBLE InventoryID.
`language_name`	`str`	Human-readable language name.
`iso639_3`	`str`	ISO 639-3 language code.
`glottocode`	`str`	Glottolog code.
`specific_dialect`	`str \| None`	Dialect specification, or None.
`source`	`str`	PHOIBLE source identifier (e.g., 'spa', 'upsid', 'ph').
`segments`	`list[Segment]`	List of Segment objects in this inventory.

phonemes `property` ¶

phonemes: list[str]

All IPA symbols in segment order.

consonants `property` ¶

consonants: list[str]

IPA symbols for consonant segments.

vowels `property` ¶

vowels: list[str]

IPA symbols for vowel segments.

tones `property` ¶

tones: list[str]

IPA symbols for tone segments.

marginal_phonemes `property` ¶

marginal_phonemes: list[str]

IPA symbols for marginal segments.

non_marginal_phonemes `property` ¶

non_marginal_phonemes: list[str]

IPA symbols for non-marginal segments.

consonant_segments `property` ¶

consonant_segments: list[Segment]

Consonant Segment objects.

vowel_segments `property` ¶

vowel_segments: list[Segment]

Vowel Segment objects.

tone_segments `property` ¶

tone_segments: list[Segment]

Tone Segment objects.

marginal_segments `property` ¶

marginal_segments: list[Segment]

Marginal Segment objects.

non_marginal_segments `property` ¶

non_marginal_segments: list[Segment]

Non-marginal Segment objects.

all_allophones `property` ¶

all_allophones: dict[str, list[str]]

Map each phoneme to its allophone list.

size `property` ¶

size: int

Total number of segments.

consonant_count `property` ¶

consonant_count: int

Number of consonant segments.

vowel_count `property` ¶

vowel_count: int

Number of vowel segments.

tone_count `property` ¶

tone_count: int

Number of tone segments.

has_tones `property` ¶

has_tones: bool

Whether this inventory includes tone segments.

marginal_count `property` ¶

marginal_count: int

Number of marginal segments.

segments_with_feature ¶

segments_with_feature(feature: str, value: str) -> list[Segment]

Return segments matching a single feature constraint.

Parameters:

Name	Type	Description	Default
`feature`	`str`	Distinctive feature name (e.g., 'nasal', 'labial').	required
`value`	`str`	Required value ('+', '-', or '0').	required

Returns:

Type	Description
`list[Segment]`	List of matching Segment objects.

Raises:

Type	Description
`ValueError`	If feature name or value is invalid.

segments_with_features ¶

segments_with_features(constraints: dict[str, str]) -> list[Segment]

Return segments matching multiple feature constraints.

Parameters:

Name	Type	Description	Default
`constraints`	`dict[str, str]`	Dict of {feature_name: required_value}.	required

Returns:

Type	Description
`list[Segment]`	List of Segment objects matching ALL constraints.

Raises:

Type	Description
`ValueError`	If any feature name or value is invalid.

to_dict ¶

to_dict() -> dict[str, Any]

Export as a plain Python dict.

Returns:

Type	Description
`dict[str, Any]`	Dict with all inventory metadata and segment data,
`dict[str, Any]`	suitable for JSON serialization.

Segment¶

Segment `dataclass` ¶

Segment(phoneme: str, segment_class: str, marginal: bool, allophones: list[str], features: dict[str, str], glyph_id: str)

A single phonological segment with full PHOIBLE metadata.

Hashable so segments are usable in sets and as dict keys.

Attributes:

Name	Type	Description
`phoneme`	`str`	IPA symbol (e.g., 'p', 'tʃ', 'ɛ̃', '˥˩').
`segment_class`	`str`	One of 'consonant', 'vowel', 'tone'.
`marginal`	`bool`	Whether the segment is marginal in this inventory.
`allophones`	`list[str]`	List of allophonic variants (IPA strings).
`features`	`dict[str, str]`	Dict of 38 distinctive features, each '+', '-', or '0'.
`glyph_id`	`str`	Unicode glyph ID string from PHOIBLE.

PhoibleDataset¶

PhoibleDataset ¶

PhoibleDataset(cache_dir: Path | None = None)

Manages access to the PHOIBLE phonological inventory database.

Loads the PHOIBLE CSV (105K+ segment rows, 3,020 inventories, 2,186 languages) and provides efficient query methods.

Data is parsed lazily on first query, or explicitly via load().

Parameters:

Name	Type	Description	Default
`cache_dir`	`Path \| None`	Directory to store/read the cached phoible.csv. Defaults to `~/.corpusgen/`.	`None`

cache_dir `property` ¶

cache_dir: Path

Directory where phoible.csv is cached.

csv_path `property` ¶

csv_path: Path

Full path to the cached phoible.csv file.

csv_exists `property` ¶

csv_exists: bool

Whether the cached CSV file exists on disk.

is_loaded `property` ¶

is_loaded: bool

Whether data has been parsed into memory.

inventory_count `property` ¶

inventory_count: int

Number of inventories loaded.

language_count `property` ¶

language_count: int

Number of distinct languages (by ISO 639-3).

segment_count `property` ¶

segment_count: int

Total number of segments across all inventories.

load ¶

load() -> None

Parse the cached PHOIBLE CSV into memory.

If already loaded, resets and reloads (idempotent on data).

Raises:

Type	Description
`FileNotFoundError`	If the CSV file does not exist. Call `download()` first or provide a valid cache_dir.

get_inventory ¶

get_inventory(identifier: str, source: str | None = None) -> Inventory

Get a single inventory for a language.

Parameters:

Name	Type	Description	Default
`identifier`	`str`	ISO 639-3 code or Glottocode.	required
`source`	`str \| None`	If specified, return the inventory from this source. If None, returns the inventory with the most segments.	`None`

Returns:

Type	Description
`Inventory`	An Inventory object.

Raises:

Type	Description
`KeyError`	If the identifier or source is not found.

get_all_inventories ¶

get_all_inventories(identifier: str) -> list[Inventory]

Get all inventories for a language.

Parameters:

Name	Type	Description	Default
`identifier`	`str`	ISO 639-3 code or Glottocode.	required

Returns:

Type	Description
`list[Inventory]`	List of Inventory objects.

Raises:

Type	Description
`KeyError`	If the identifier is not found.

get_union_inventory ¶

get_union_inventory(identifier: str) -> Inventory

Get the union of all inventories for a language.

Merges all inventories into a single maximally inclusive set. If a phoneme appears in multiple inventories, the Segment from the largest inventory is preferred (preserving its features, allophones, etc.).

Parameters:

Name	Type	Description	Default
`identifier`	`str`	ISO 639-3 code or Glottocode.	required

Returns:

Type	Description
`Inventory`	A synthetic Inventory with source="union".

Raises:

Type	Description
`KeyError`	If the identifier is not found.

search ¶

search(name: str) -> list[dict[str, Any]]

Search for languages by name (case-insensitive, partial match).

Parameters:

Name	Type	Description	Default
`name`	`str`	Search string to match against language names.	required

Returns:

Type	Description
`list[dict[str, Any]]`	List of dicts with language metadata for each match.

available_languages ¶

available_languages() -> list[dict[str, Any]]

List all languages in the dataset.

Returns:

Type	Description
`list[dict[str, Any]]`	Sorted list of dicts with language metadata.

sources_for ¶

sources_for(identifier: str) -> list[str]

List available PHOIBLE sources for a language.

Parameters:

Name	Type	Description	Default
`identifier`	`str`	ISO 639-3 code or Glottocode.	required

Returns:

Type	Description
`list[str]`	Sorted list of source identifiers.

Raises:

Type	Description
`KeyError`	If the identifier is not found.

get_inventory_for_espeak ¶

get_inventory_for_espeak(espeak_code: str, source: str | None = None) -> Inventory

Get an inventory using an espeak-ng voice code.

Maps the espeak code to ISO 639-3 via the bundled mapping, then looks up the PHOIBLE inventory.

Parameters:

Name	Type	Description	Default
`espeak_code`	`str`	espeak-ng voice identifier (e.g., 'en-us').	required
`source`	`str \| None`	Optional PHOIBLE source filter.	`None`

Returns:

Type	Description
`Inventory`	An Inventory object.

Raises:

Type	Description
`KeyError`	If the espeak code or resulting ISO is not found.

get_union_inventory_for_espeak ¶

get_union_inventory_for_espeak(espeak_code: str) -> Inventory

Get the union inventory using an espeak-ng voice code.

Parameters:

Name	Type	Description	Default
`espeak_code`	`str`	espeak-ng voice identifier (e.g., 'en-us').	required

Returns:

Type	Description
`Inventory`	A synthetic union Inventory.

Raises:

Type	Description
`KeyError`	If the espeak code or resulting ISO is not found.

download ¶

download() -> None

Download a pinned, checksum-verified PHOIBLE CSV release.

Creates the cache directory if it doesn't exist. The download is written atomically, so a failed or corrupt transfer cannot replace a previously valid cache.

Raises:

Type	Description
`RuntimeError`	If the downloaded file fails checksum verification.

Evaluation Results¶

EvaluationReport¶

EvaluationReport `dataclass` ¶

EvaluationReport(language: str, unit: str, target_phonemes: list[str], covered_phonemes: set[str], missing_phonemes: set[str], coverage: float, phoneme_counts: dict[str, int], total_sentences: int, sentence_details: list[SentenceDetail] = list(), phoneme_sources: dict[str, list[int]] = dict(), distribution: DistributionMetrics | None = None, text_quality: TextQualityMetrics | None = None)

Complete evaluation result with multi-level rendering and export.

This object always holds the full data internally. The verbosity level only controls what .render() outputs as human-readable text.

Attributes:

Name	Type	Description
`language`	`str`	Language code used for evaluation.
`unit`	`str`	Coverage unit type (phoneme, diphone, triphone).
`target_phonemes`	`list[str]`	Full list of target phonemes.
`covered_phonemes`	`set[str]`	Set of phonemes that were covered.
`missing_phonemes`	`set[str]`	Set of phonemes not yet covered.
`coverage`	`float`	Coverage fraction (0.0 to 1.0).
`phoneme_counts`	`dict[str, int]`	Per-phoneme occurrence counts.
`total_sentences`	`int`	Number of sentences evaluated.
`sentence_details`	`list[SentenceDetail]`	Per-sentence breakdown (for verbose output).
`phoneme_sources`	`dict[str, list[int]]`	Maps each phoneme to source sentence indices.

__post_init__ ¶

__post_init__() -> None

Normalize target units to a unique, stable-order inventory.

render ¶

render(verbosity: Verbosity = Verbosity.NORMAL) -> str

Render a human-readable report at the specified verbosity level.

Parameters:

Name	Type	Description	Default
`verbosity`	`Verbosity`	One of Verbosity.MINIMAL, NORMAL, or VERBOSE.	`NORMAL`

Returns:

Type	Description
`str`	Formatted string report.

to_dict ¶

to_dict() -> dict[str, Any]

Export as a plain Python dict.

Sets are converted to sorted lists for JSON compatibility.

to_json ¶

to_json(indent: int | None = None) -> str

Export as a JSON string.

Parameters:

Name	Type	Description	Default
`indent`	`int \| None`	JSON indentation level. None for compact output.	`None`

to_jsonld_ex ¶

to_jsonld_ex() -> dict[str, Any]

Export as a JSON-LD document compatible with jsonld-ex.

Returns a JSON-LD document with @context, @type, and all evaluation data mapped to linked data terms.

DistributionMetrics¶

DistributionMetrics `dataclass` ¶

DistributionMetrics(entropy: float, normalized_entropy: float, jsd_uniform: float, coefficient_of_variation: float, min_count: int, max_count: int, count_ratio: float, zero_count: int, pcd_uniform: float, jsd_reference: float | None, pearson_correlation: float | None)

Immutable container for corpus distribution quality metrics.

All metrics are computed over target phonetic units only; counts for units outside the target inventory are ignored.

Attributes:

Name	Type	Description
`entropy`	`float`	Shannon entropy H(X) in bits (base-2 log). 0.0 when all mass on one unit; log₂(N) when perfectly uniform.
`normalized_entropy`	`float`	H(X) / log₂(N). 1.0 = perfectly uniform. Defined as 1.0 when N ≤ 1 (trivially uniform by convention).
`jsd_uniform`	`float`	Jensen-Shannon Divergence vs. a uniform distribution over all target units. 0.0 = perfectly uniform; 1.0 = maximally divergent. Uses base-2 log so the range is exactly [0, 1].
`coefficient_of_variation`	`float`	Population standard deviation / mean of counts across all target units (including zeros). 0.0 = all counts equal.
`min_count`	`int`	Smallest count among target units (0 if any are missing).
`max_count`	`int`	Largest count among target units.
`count_ratio`	`float`	min_count / max_count. 0.0 if min is 0; 1.0 if all counts equal. Defined as 1.0 when max_count is 0.
`zero_count`	`int`	Number of target units with zero occurrences.
`pcd_uniform`	`float`	Phoneme Coverage Diversity (uniform reference): `coverage × (1 - jsd_uniform)` where coverage is the fraction of target units with count > 0.
`jsd_reference`	`float \| None`	JSD vs. a user-supplied reference distribution. None when no reference is provided.
`pearson_correlation`	`float \| None`	Pearson's r between corpus counts and a reference distribution. None when no reference is provided or when either distribution has zero variance (undefined).

to_dict ¶

to_dict() -> dict[str, Any]

Export as a plain Python dict (JSON-safe).

TextQualityMetrics¶

TextQualityMetrics `dataclass` ¶

TextQualityMetrics(sentence_length_words_mean: float, sentence_length_words_median: float, sentence_length_words_std: float, sentence_length_words_min: int, sentence_length_words_max: int, sentence_length_phonemes_mean: float, sentence_length_phonemes_median: float, sentence_length_phonemes_std: float, sentence_length_phonemes_min: int, sentence_length_phonemes_max: int, total_words: int, unique_words: int, type_token_ratio: float, hapax_ratio: float, flesch_reading_ease: float | None, flesch_kincaid_grade: float | None)

Text-level quality metrics for a corpus.

Attributes:

Name	Type	Description
`sentence_length_words_mean`	`float`	Mean sentence length in words.
`sentence_length_words_median`	`float`	Median sentence length in words.
`sentence_length_words_std`	`float`	Population std of sentence lengths (words).
`sentence_length_words_min`	`int`	Shortest sentence (words).
`sentence_length_words_max`	`int`	Longest sentence (words).
`sentence_length_phonemes_mean`	`float`	Mean sentence length in phonemes.
`sentence_length_phonemes_median`	`float`	Median in phonemes.
`sentence_length_phonemes_std`	`float`	Population std (phonemes).
`sentence_length_phonemes_min`	`int`	Shortest sentence (phonemes).
`sentence_length_phonemes_max`	`int`	Longest sentence (phonemes).
`total_words`	`int`	Total word tokens across all sentences.
`unique_words`	`int`	Number of distinct word types.
`type_token_ratio`	`float`	unique_words / total_words (0.0 if no words).
`hapax_ratio`	`float`	Words appearing exactly once / unique_words.
`flesch_reading_ease`	`float \| None`	Flesch Reading Ease score, or None if not computable (non-Latin script or empty corpus).
`flesch_kincaid_grade`	`float \| None`	Flesch-Kincaid Grade Level, or None.

to_dict ¶

to_dict() -> dict[str, Any]

Export as a plain Python dict (JSON-safe).

Verbosity¶

Verbosity ¶

Bases: Enum

Report verbosity levels.

compute_distribution_metrics¶

compute_distribution_metrics ¶

compute_distribution_metrics(phoneme_counts: dict[str, int], target_phonemes: list[str], reference_distribution: dict[str, float] | None = None) -> DistributionMetrics

Compute distribution quality metrics for a phoneme corpus.

Measures how well-balanced the corpus's phoneme distribution is across the target inventory, using information-theoretic and statistical metrics.

Parameters:

Name	Type	Description	Default
`phoneme_counts`	`dict[str, int]`	Mapping from phonetic unit to its occurrence count in the corpus. Units not in `target_phonemes` are ignored.	required
`target_phonemes`	`list[str]`	The target inventory of phonetic units. Only these units are considered. May contain duplicates (they are deduplicated internally while preserving order).	required
`reference_distribution`	`dict[str, float] \| None`	Optional mapping from phonetic unit to its expected relative frequency. Need not sum to 1.0 — values are normalized automatically. Units in `target_phonemes` but missing from the reference are treated as 0. When provided, `jsd_reference` and `pearson_correlation` are computed.	`None`

Returns:

Type	Description
`DistributionMetrics`	DistributionMetrics with all computed fields.

CoverageTrajectory¶

CoverageTrajectory `dataclass` ¶

CoverageTrajectory(snapshots: list[CoverageSnapshot], unit: str, target_size: int)

Complete coverage trajectory over an ordered sentence sequence.

Attributes:

Name	Type	Description
`snapshots`	`list[CoverageSnapshot]`	One snapshot per sentence, in input order.
`unit`	`str`	Coverage unit type ("phoneme", "diphone", "triphone").
`target_size`	`int`	Total number of target units.

coverages `property` ¶

coverages: list[float]

Coverage fractions for easy plotting — one value per sentence.

gains `property` ¶

gains: list[int]

Marginal gains (new unit counts) per sentence.

to_dict ¶

to_dict() -> dict[str, Any]

Export as a plain Python dict (JSON-safe).

compute_coverage_trajectory¶

compute_coverage_trajectory ¶

compute_coverage_trajectory(phoneme_sequences: list[list[str]], target_units: set[str], unit: str = 'phoneme') -> CoverageTrajectory

Compute a step-by-step coverage trajectory.

For each sentence in order, records which new target units it covers and the cumulative coverage fraction. This produces the data needed for the classic coverage saturation curve.

Parameters:

Name	Type	Description	Default
`phoneme_sequences`	`list[list[str]]`	Ordered list of phoneme lists. Each inner list is the phoneme sequence for one sentence, in the order sentences were selected or generated.	required
`target_units`	`set[str]`	The full set of target units to measure against.	required
`unit`	`str`	Coverage unit type — "phoneme", "diphone", or "triphone". Must match the format of `target_units`.	`'phoneme'`

Returns:

Type	Description
`CoverageTrajectory`	CoverageTrajectory with one snapshot per sentence.

ErrorRateResult¶

ErrorRateResult `dataclass` ¶

ErrorRateResult(wer: float, cer: float, per: float | None, ser: float, details: list[SentenceErrorDetail])

Corpus-level error rate metrics with per-sentence details.

Corpus-level WER, CER, and PER are computed as total edit distance divided by total reference length (micro-average), not as the mean of per-sentence rates. This is the standard convention in ASR evaluation.

Attributes:

Name	Type	Description
`wer`	`float`	Corpus-level Word Error Rate (micro-averaged).
`cer`	`float`	Corpus-level Character Error Rate (micro-averaged).
`per`	`float \| None`	Corpus-level Phoneme Error Rate (None if no phonemes).
`ser`	`float`	Sentence Error Rate.
`details`	`list[SentenceErrorDetail]`	Per-sentence breakdowns.

to_dict ¶

to_dict() -> dict[str, Any]

Export as a plain Python dict (JSON-safe).

compute_error_rates¶

compute_error_rates ¶

compute_error_rates(references: list[str], hypotheses: list[str], reference_phonemes: list[list[str]] | None = None, hypothesis_phonemes: list[list[str]] | None = None, case_sensitive: bool = False) -> ErrorRateResult

Compute corpus-level error rates with per-sentence details.

Corpus-level WER and CER are micro-averaged: total edit distance across all sentences divided by total reference tokens. This is the standard convention in ASR evaluation (not macro-average of per-sentence rates).

Parameters:

Name	Type	Description	Default
`references`	`list[str]`	List of reference text strings.	required
`hypotheses`	`list[str]`	List of hypothesis text strings (same length).	required
`reference_phonemes`	`list[list[str]] \| None`	Optional phoneme lists for each reference sentence. Required (along with hypothesis_phonemes) to compute PER.	`None`
`hypothesis_phonemes`	`list[list[str]] \| None`	Optional phoneme lists for each hypothesis sentence.	`None`
`case_sensitive`	`bool`	If False (default), word/sentence comparisons are case-insensitive.	`False`

Returns:

Type	Description
`ErrorRateResult`	ErrorRateResult with corpus-level and per-sentence metrics.

Raises:

Type	Description
`ValueError`	If references and hypotheses have different lengths, or if phoneme lists are provided with mismatched lengths.

CorpusPerplexityMetrics¶

CorpusPerplexityMetrics `dataclass` ¶

CorpusPerplexityMetrics(per_sentence: list[float], corpus_perplexity: float, mean_perplexity: float, median_perplexity: float, std_perplexity: float, min_perplexity: float, max_perplexity: float, num_sentences: int, num_tokens: int, total_nll: float)

Immutable container for corpus-level perplexity results.

Attributes:

Name	Type	Description
`per_sentence`	`list[float]`	Raw perplexity for each scored sentence (order matches the input order, excluding skipped sentences).
`corpus_perplexity`	`float`	`exp(total_nll / num_tokens)` — the standard language-modelling perplexity, weighted by token count.
`mean_perplexity`	`float`	Arithmetic mean of per-sentence perplexities.
`median_perplexity`	`float`	Median of per-sentence perplexities.
`std_perplexity`	`float`	Population standard deviation of per-sentence perplexities (0.0 when only one sentence).
`min_perplexity`	`float`	Lowest per-sentence perplexity.
`max_perplexity`	`float`	Highest per-sentence perplexity.
`num_sentences`	`int`	Number of sentences actually scored (excludes empty/whitespace and single-token sentences).
`num_tokens`	`int`	Total next-token predictions across all sentences.
`total_nll`	`float`	Sum of negative log-likelihood across all tokens, stored for reproducibility (`corpus_perplexity == exp(total_nll / num_tokens)`).

compute_corpus_perplexity¶

compute_corpus_perplexity ¶

compute_corpus_perplexity(sentences: list[str], model_name: str = 'gpt2', device: str | None = None, batch_size: int = 8, max_length: int = 512, model: Any = None, tokenizer: Any = None) -> CorpusPerplexityMetrics

Compute corpus-level perplexity metrics.

Parameters:

Name	Type	Description	Default
`sentences`	`list[str]`	List of text sentences to evaluate.	required
`model_name`	`str`	HuggingFace model ID, used only when model and tokenizer are not supplied. Defaults to `"gpt2"`.	`'gpt2'`
`device`	`str \| None`	Device string (`"cuda"`, `"cpu"`, `"auto"`). Auto-detected when None. Ignored when model is provided.	`None`
`batch_size`	`int`	Number of sentences per forward pass.	`8`
`max_length`	`int`	Maximum token length per sentence (truncated).	`512`
`model`	`Any`	An already-loaded HuggingFace causal LM. Pass together with tokenizer to avoid redundant model loading (e.g., share with :class:`PerplexityFluencyScorer`).	`None`
`tokenizer`	`Any`	The corresponding HuggingFace tokenizer. Must be provided together with model, or both omitted.	`None`

Returns:

Type	Description
`CorpusPerplexityMetrics`	class:`CorpusPerplexityMetrics` with per-sentence and
`CorpusPerplexityMetrics`	corpus-level statistics.

Raises:

Type	Description
`ValueError`	If both model and tokenizer are not provided together, or if no scoreable sentences remain.
`ImportError`	If `torch` or `transformers` is not installed and no model is injected.

Selection Results¶

SelectionResult¶

SelectionResult `dataclass` ¶

SelectionResult(selected_indices: list[int], selected_sentences: list[str], coverage: float, covered_units: set[str], missing_units: set[str], unit: str, algorithm: str, elapsed_seconds: float, iterations: int, metadata: dict = dict())

Immutable result of a sentence selection algorithm run.

Attributes:

Name	Type	Description
`selected_indices`	`list[int]`	Indices into the original candidate list.
`selected_sentences`	`list[str]`	The selected sentences (same order as indices).
`coverage`	`float`	Final coverage ratio (0.0–1.0) of target units.
`covered_units`	`set[str]`	Set of target units that were covered.
`missing_units`	`set[str]`	Set of target units that remain uncovered.
`unit`	`str`	Coverage unit type ("phoneme", "diphone", "triphone").
`algorithm`	`str`	Name of the algorithm that produced this result.
`elapsed_seconds`	`float`	Wall-clock time in seconds.
`iterations`	`int`	Number of algorithm iterations/steps taken.
`metadata`	`dict`	Algorithm-specific extras (e.g. solver_status for ILP, pareto_front for NSGA-II, sample_size for Stochastic Greedy).

num_selected `property` ¶

num_selected: int

Number of sentences selected.

Generation¶

GenerationLoop¶

GenerationLoop ¶

GenerationLoop(backend: GenerationBackend, targets: PhoneticTargetInventory, scorer: PhoneticScorer, stopping_criteria: StoppingCriteria | None = None, candidates_per_iteration: int = 5, candidate_filter: Callable[[dict], bool] | None = None, on_progress: Callable[[dict], None] | None = None)

Orchestrates the Phon-CTG generation process.

Connects a GenerationBackend, PhoneticTargetInventory, and PhoneticScorer into an iterative loop that generates sentences to maximize phonetic coverage.

Parameters:

Name	Type	Description	Default
`backend`	`GenerationBackend`	The generation backend to use.	required
`targets`	`PhoneticTargetInventory`	The phonetic target inventory to cover.	required
`scorer`	`PhoneticScorer`	The scorer for evaluating candidates.	required
`stopping_criteria`	`StoppingCriteria \| None`	When to stop generating. Defaults to full coverage with no other limits.	`None`
`candidates_per_iteration`	`int`	How many candidates to request from the backend each iteration.	`5`
`candidate_filter`	`Callable[[dict], bool] \| None`	Optional callable `(candidate_dict) -> bool`. Applied to each candidate before ranking. Candidates for which the filter returns `False` are discarded. Use :meth:`ReadabilityScorer.as_filter` for readability-based hard filtering.	`None`
`on_progress`	`Callable[[dict], None] \| None`	Optional callback invoked after each accepted sentence. Receives a dict with iteration info.	`None`

backend `property` ¶

backend: GenerationBackend

The generation backend.

targets `property` ¶

targets: PhoneticTargetInventory

The phonetic target inventory.

scorer `property` ¶

scorer: PhoneticScorer

The phonetic scorer.

stopping_criteria `property` ¶

stopping_criteria: StoppingCriteria

The stopping criteria.

candidates_per_iteration `property` ¶

candidates_per_iteration: int

Number of candidates requested per iteration.

run ¶

run() -> GenerationResult

Execute the generation loop.

Returns:

Type	Description
`GenerationResult`	GenerationResult with all generated sentences and metrics.

StoppingCriteria¶

StoppingCriteria `dataclass` ¶

StoppingCriteria(target_coverage: float = 1.0, max_sentences: int | None = None, max_iterations: int | None = None, timeout_seconds: float | None = None)

Configurable stopping conditions for the generation loop.

The loop terminates when ANY condition is met.

Attributes:

Name	Type	Description
`target_coverage`	`float`	Stop when this coverage fraction is reached.
`max_sentences`	`int \| None`	Maximum number of sentences to accept.
`max_iterations`	`int \| None`	Maximum loop iterations (backend calls).
`timeout_seconds`	`float \| None`	Wall-clock time limit in seconds.

Raises:

Type	Description
`ValueError`	If target coverage is outside [0.0, 1.0] or any optional limit is negative.

PhoneticTargetInventory¶

PhoneticTargetInventory ¶

PhoneticTargetInventory(target_phonemes: list[str] | None = None, unit: str = 'phoneme', tracker: CoverageTracker | None = None, weights: dict[str, float] | None = None, max_target_size: int | None = None)

Dynamic phonetic target inventory with weighted prioritization.

Wraps a CoverageTracker and adds priority-based target selection, enabling generation backends to query which phonetic units to pursue next.

Parameters:

Name	Type	Description	Default
`target_phonemes`	`list[str] \| None`	List of phonemes for the target inventory. Mutually exclusive with `tracker`.	`None`
`unit`	`str`	Coverage unit type — "phoneme", "diphone", or "triphone". Ignored when `tracker` is provided (uses tracker's unit).	`'phoneme'`
`tracker`	`CoverageTracker \| None`	An existing CoverageTracker to wrap. Mutually exclusive with `target_phonemes`.	`None`
`weights`	`dict[str, float] \| None`	Optional mapping from unit string to priority weight. Higher weight = higher priority. Units not in the dict default to 1.0.	`None`
`max_target_size`	`int \| None`	Forwarded to CoverageTracker in standalone mode.	`None`

tracker `property` ¶

tracker: CoverageTracker

The underlying CoverageTracker instance.

unit `property` ¶

unit: str

Coverage unit type.

target_size `property` ¶

target_size: int

Number of units in the target inventory.

target_units `property` ¶

target_units: set[str]

Full set of target units.

covered_count `property` ¶

covered_count: int

Number of target units covered so far.

covered_units `property` ¶

covered_units: set[str]

Set of target units covered so far.

coverage `property` ¶

coverage: float

Fraction of target units covered (0.0 to 1.0).

missing `property` ¶

missing: set[str]

Set of target units not yet covered.

next_targets ¶

next_targets(k: int) -> list[str]

Return the top-k highest-priority uncovered units.

Units are sorted by descending weight. Ties are broken by lexicographic order for determinism.

Parameters:

Name	Type	Description	Default
`k`	`int`	Maximum number of targets to return.	required

Returns:

Type	Description
`list[str]`	List of unit strings, ordered by priority (highest first).
`list[str]`	May be shorter than k if fewer uncovered units remain.

update ¶

update(phonemes: list[str], sentence_index: int) -> None

Update coverage with phonemes from a generated sentence.

Delegates to the underlying CoverageTracker.

Parameters:

Name	Type	Description	Default
`phonemes`	`list[str]`	List of phonemes extracted from the sentence.	required
`sentence_index`	`int`	Index of the sentence in the corpus.	required

reset ¶

reset() -> None

Reset coverage state, preserving targets and weights.

PhoneticScorer¶

PhoneticScorer ¶

PhoneticScorer(targets: PhoneticTargetInventory, phonotactic_scorer: Callable[[list[str]], float] | None = None, fluency_scorer: Callable[[str | None], float] | None = None, readability_scorer: Callable[[str | None], float] | None = None, coverage_weight: float = 1.0, phonotactic_weight: float = 0.0, fluency_weight: float = 0.0, readability_weight: float = 0.0)

Evaluates candidate text against a dynamic phonetic target inventory.

Parameters:

Name	Type	Description	Default
`targets`	`PhoneticTargetInventory`	The PhoneticTargetInventory to score against.	required
`phonotactic_scorer`	`Callable[[list[str]], float] \| None`	Optional callable (phonemes -> float) for phonotactic legality scoring.	`None`
`fluency_scorer`	`Callable[[str \| None], float] \| None`	Optional callable (text -> float) for fluency scoring.	`None`
`readability_scorer`	`Callable[[str \| None], float] \| None`	Optional callable (text -> float) for readability scoring. See :class:`ReadabilityScorer` for a ready-made implementation.	`None`
`coverage_weight`	`float`	Weight for the coverage component in the composite score.	`1.0`
`phonotactic_weight`	`float`	Weight for the phonotactic component.	`0.0`
`fluency_weight`	`float`	Weight for the fluency component.	`0.0`
`readability_weight`	`float`	Weight for the readability component.	`0.0`

targets `property` ¶

targets: PhoneticTargetInventory

The target inventory being scored against.

coverage_weight `property` ¶

coverage_weight: float

Weight for coverage component in composite score.

phonotactic_weight `property` ¶

phonotactic_weight: float

Weight for phonotactic component in composite score.

fluency_weight `property` ¶

fluency_weight: float

Weight for fluency component in composite score.

readability_weight `property` ¶

readability_weight: float

Weight for readability component in composite score.

score ¶

score(phonemes: list[str], text: str | None = None) -> ScoreResult

Score a candidate without modifying inventory state.

Parameters:

Name	Type	Description	Default
`phonemes`	`list[str]`	Phoneme list for the candidate sentence.	required
`text`	`str \| None`	Optional raw text (passed to fluency hook if provided).	`None`

Returns:

Type	Description
`ScoreResult`	ScoreResult with all score components.

score_batch ¶

score_batch(candidates: list[dict]) -> list[ScoreResult]

Score multiple candidates without modifying inventory state.

Each candidate is scored independently against the current state.

Parameters:

Name	Type	Description	Default
`candidates`	`list[dict]`	List of dicts, each with "phonemes" (required) and optionally "text".	required

Returns:

Type	Description
`list[ScoreResult]`	List of ScoreResult, one per candidate, in input order.

rank ¶

rank(candidates: list[dict], top_k: int | None = None) -> list[ScoreResult]

Score and rank candidates by composite score (descending).

Parameters:

Name	Type	Description	Default
`candidates`	`list[dict]`	List of dicts, each with "phonemes" (required) and optionally "text".	required
`top_k`	`int \| None`	If provided, return only the top-k results.	`None`

Returns:

Type	Description
`list[ScoreResult]`	List of ScoreResult sorted by composite_score descending.

score_and_commit ¶

score_and_commit(phonemes: list[str], sentence_index: int, text: str | None = None) -> ScoreResult

Score a candidate then update the inventory with its coverage.

Parameters:

Name	Type	Description	Default
`phonemes`	`list[str]`	Phoneme list for the candidate sentence.	required
`sentence_index`	`int`	Index of the sentence in the corpus.	required
`text`	`str \| None`	Optional raw text (passed to fluency hook if provided).	`None`

Returns:

Type	Description
`ScoreResult`	ScoreResult computed before the inventory update.

NgramPhonotacticScorer¶

NgramPhonotacticScorer ¶

NgramPhonotacticScorer(phonemes: list[str], n: int = 2)

N-gram phonotactic scorer with Laplace smoothing.

Scores phoneme sequences based on transition probabilities. Callable interface: scorer(phonemes) -> float in [0, 1].

Parameters:

Name	Type	Description	Default
`phonemes`	`list[str]`	Target phoneme inventory.	required
`n`	`int`	N-gram order (2 = bigram, 3 = trigram). Must be >= 2.	`2`

Raises:

Type	Description
`ValueError`	If phonemes is empty, has < 2 elements, or n < 2.

n `property` ¶

n: int

N-gram order.

phonemes `property` ¶

phonemes: list[str]

Target phoneme inventory.

from_corpus `classmethod` ¶

from_corpus(sequences: list[list[str]], n: int = 2) -> NgramPhonotacticScorer

Build an n-gram model from observed phoneme sequences.

Trains on actual transition frequencies from a corpus. This produces a scientifically stronger model than the inventory-derived default.

Parameters:

Name	Type	Description	Default
`sequences`	`list[list[str]]`	List of phoneme sequences (each a list of str).	required
`n`	`int`	N-gram order. Must be >= 2.	`2`

Returns:

Type	Description
`NgramPhonotacticScorer`	A trained NgramPhonotacticScorer.

Raises:

Type	Description
`ValueError`	If sequences is empty, n < 2, or no sequences are long enough to form n-grams.

save ¶

save(path: str | Path) -> None

Save the n-gram model to a JSON file for reproducibility.

Parameters:

Name	Type	Description	Default
`path`	`str \| Path`	File path to write. Created or overwritten.	required

load `classmethod` ¶

load(path: str | Path) -> NgramPhonotacticScorer

Load an n-gram model from a JSON file.

Parameters:

Name	Type	Description	Default
`path`	`str \| Path`	Path to a file previously created by :meth:`save`.	required

Returns:

Type	Description
`NgramPhonotacticScorer`	A restored NgramPhonotacticScorer.

Raises:

Type	Description
`FileNotFoundError`	If path does not exist.

call ¶

__call__(phonemes: list[str]) -> float

Score a phoneme sequence for phonotactic naturalness.

Computes the mean log-probability of n-gram transitions under the model with Laplace smoothing, then normalizes to [0, 1].

Parameters:

Name	Type	Description	Default
`phonemes`	`list[str]`	Phoneme sequence to score.	required

Returns:

Type	Description
`float`	Float in [0, 1]. Higher = more phonotactically natural.
`float`	Returns 0.0 for sequences shorter than n.

PerplexityFluencyScorer¶

PerplexityFluencyScorer ¶

PerplexityFluencyScorer(model_name: str = 'gpt2', device: str | None = None)

Fluency scorer based on causal LM perplexity.

Callable interface: scorer(text) -> float in [0, 1].

Higher scores indicate more fluent text (lower perplexity).

Parameters:

Name	Type	Description	Default
`model_name`	`str`	HuggingFace model ID (e.g., "gpt2").	`'gpt2'`
`device`	`str \| None`	Device string ("cuda", "cpu", "auto"). If None, auto-detects.	`None`

Raises:

Type	Description
`ImportError`	On first call if torch/transformers not installed.

model_name `property` ¶

model_name: str

HuggingFace model ID.

is_loaded `property` ¶

is_loaded: bool

Whether the model and tokenizer have been loaded.

from_model `classmethod` ¶

from_model(model: Any, tokenizer: Any) -> PerplexityFluencyScorer

Create a scorer from an already-loaded model and tokenizer.

Use this to share a model with the LocalBackend, avoiding loading the same model twice.

Parameters:

Name	Type	Description	Default
`model`	`Any`	A HuggingFace causal LM instance.	required
`tokenizer`	`Any`	The corresponding tokenizer.	required

Returns:

Type	Description
`PerplexityFluencyScorer`	A PerplexityFluencyScorer with the model pre-loaded.

call ¶

__call__(text: str | None) -> float

Score text for fluency via perplexity.

Parameters:

Name	Type	Description	Default
`text`	`str \| None`	Text to score. Returns 0.0 for None or empty/whitespace.	required

Returns:

Type	Description
`float`	Float in [0, 1]. Higher = more fluent (lower perplexity).

ReadabilityScorer¶

ReadabilityScorer ¶

ReadabilityScorer(target_range: tuple[float, float] | None = None)

Readability scorer based on Flesch Reading Ease.

Callable interface: scorer(text) -> float in [0, 1].

Parameters:

Name	Type	Description	Default
`target_range`	`tuple[float, float] \| None`	Optional `(lo, hi)` FRE band for trapezoidal scoring. If None, uses simple clamped mode.	`None`

Raises:

Type	Description
`ValueError`	If target_range has lo > hi or negative values.

target_range `property` ¶

target_range: tuple[float, float] | None

The target FRE range, or None for simple mode.

compute_fre ¶

compute_fre(text: str | None) -> float | None

Compute raw Flesch Reading Ease for a single sentence.

Parameters:

Name	Type	Description	Default
`text`	`str \| None`	Input text. Returns None for None, empty, whitespace-only, or non-Latin-script text.	required

Returns:

Type	Description
`float \| None`	Flesch Reading Ease score, or None if not computable.

call ¶

__call__(text: str | None) -> float

Score text for readability.

Parameters:

Name	Type	Description	Default
`text`	`str \| None`	Text to score. Returns 0.0 for None, empty, whitespace, or non-Latin text.	required

Returns:

Type	Description
`float`	Float in [0, 1].

as_filter ¶

as_filter(min_fre: float, max_fre: float) -> Callable[[dict], bool]

Create a hard accept/reject filter for candidate dicts.

Returns a callable suitable for the candidate_filter parameter of :class:GenerationLoop.

Parameters:

Name	Type	Description	Default
`min_fre`	`float`	Minimum Flesch Reading Ease to accept (inclusive).	required
`max_fre`	`float`	Maximum Flesch Reading Ease to accept (inclusive).	required

Returns:

Type	Description
`Callable[[dict], bool]`	A callable `(candidate_dict) -> bool`.

ReadabilityScorer is available as a Python scorer or candidate-filter hook; the CLI does not currently expose a readability flag.

PhoneticReward¶

PhoneticReward ¶

PhoneticReward(targets: PhoneticTargetInventory, phonotactic_scorer: Callable[[list[str]], float] | None = None, fluency_scorer: Callable[[str | None], float] | None = None, ref_log_probs_fn: Callable[[str], float] | None = None, coverage_weight: float = 1.0, phonotactic_weight: float = 0.0, fluency_weight: float = 0.0, language: str = 'en-us')

Composite reward function for Phon-RL training.

Evaluates generated text against a phonetic target inventory, combining coverage gain with optional phonotactic and fluency signals into a single scalar reward.

Parameters:

Name	Type	Description	Default
`targets`	`PhoneticTargetInventory`	PhoneticTargetInventory tracking coverage state.	required
`phonotactic_scorer`	`Callable[[list[str]], float] \| None`	Optional callable (phonemes -> float) for phonotactic legality scoring.	`None`
`fluency_scorer`	`Callable[[str \| None], float] \| None`	Optional callable (text -> float) for fluency scoring. Takes precedence over ref_log_probs_fn.	`None`
`ref_log_probs_fn`	`Callable[[str], float] \| None`	Optional callable (text -> float) returning the reference model's log-probability. Used as the fluency signal when fluency_scorer is None.	`None`
`coverage_weight`	`float`	Weight for the coverage component (must be >= 0).	`1.0`
`phonotactic_weight`	`float`	Weight for the phonotactic component (must be >= 0).	`0.0`
`fluency_weight`	`float`	Weight for the fluency component (must be >= 0).	`0.0`

targets `property` ¶

targets: PhoneticTargetInventory

The target inventory being scored against.

coverage_weight `property` ¶

coverage_weight: float

Weight for coverage component.

phonotactic_weight `property` ¶

phonotactic_weight: float

Weight for phonotactic component.

fluency_weight `property` ¶

fluency_weight: float

Weight for fluency component.

language `property` ¶

language: str

Language code for G2P phonemization.

phonotactic_scorer `property` ¶

phonotactic_scorer: Callable[[list[str]], float] | None

Optional phonotactic scoring callable.

fluency_scorer `property` ¶

fluency_scorer: Callable[[str | None], float] | None

Optional fluency scoring callable.

ref_log_probs_fn `property` ¶

ref_log_probs_fn: Callable[[str], float] | None

Optional reference model log-prob callable for KL fluency.

sentence_reward ¶

sentence_reward(phonemes: list[str], text: str | None = None) -> RewardBreakdown

Compute sentence-level composite reward without modifying inventory.

Coverage is normalized by target inventory size to yield values in [0, 1].

Parameters:

Name	Type	Description	Default
`phonemes`	`list[str]`	Phoneme list for the generated sentence.	required
`text`	`str \| None`	Raw text of the generated sentence (for fluency scoring).	`None`

Returns:

Type	Description
`RewardBreakdown`	RewardBreakdown with all score components.

commit_sentence_reward ¶

commit_sentence_reward(phonemes: list[str], text: str | None = None, sentence_index: int = 0) -> RewardBreakdown

Compute sentence-level reward then update the target inventory.

Scores first (peek), then commits the coverage update.

Parameters:

Name	Type	Description	Default
`phonemes`	`list[str]`	Phoneme list for the generated sentence.	required
`text`	`str \| None`	Raw text of the generated sentence.	`None`
`sentence_index`	`int`	Index for provenance tracking in the inventory.	`0`

Returns:

Type	Description
`RewardBreakdown`	RewardBreakdown computed before the inventory update.

token_rewards ¶

token_rewards(token_ids: list[int], tokenizer: Any) -> TokenRewardResult

Compute sparse per-token rewards at word boundaries.

Decodes tokens incrementally, detects word boundaries from trailing whitespace, leading whitespace/word markers, or the final token, phonemizes completed words, and assigns the coverage reward for each word to the boundary token that completed it. Non-boundary tokens receive 0.0.

This provides denser learning signal than pure sentence-level reward without fabricating sub-word phonetic information.

Parameters:

Name	Type	Description	Default
`token_ids`	`list[int]`	List of generated token IDs.	required
`tokenizer`	`Any`	HuggingFace-compatible tokenizer with `decode()` method.	required

Returns:

Type	Description
`TokenRewardResult`	TokenRewardResult with per-token rewards and boundary info.

hierarchical_reward ¶

hierarchical_reward(text: str, phonemes: list[str], token_ids: list[int], tokenizer: Any) -> tuple[RewardBreakdown, TokenRewardResult]

Compute both sentence-level and token-level rewards.

Neither component mutates the target inventory (both are peek).

Parameters:

Name	Type	Description	Default
`text`	`str`	Full generated text.	required
`phonemes`	`list[str]`	Full phoneme sequence for the text.	required
`token_ids`	`list[int]`	Token IDs of the generated sequence.	required
`tokenizer`	`Any`	HuggingFace-compatible tokenizer.	required

Returns:

Type	Description
`tuple[RewardBreakdown, TokenRewardResult]`	Tuple of (RewardBreakdown, TokenRewardResult).

TrainingConfig¶

TrainingConfig `dataclass` ¶

TrainingConfig(model_name: str, num_steps: int = 100, batch_size: int = 4, learning_rate: float = 1.41e-05, kl_coeff: float = 0.1, clip_epsilon: float = 0.2, gae_gamma: float = 1.0, gae_lambda: float = 0.95, value_loss_coeff: float = 0.5, output_dir: str | None = None, seed: int = 42, max_new_tokens: int = 64, temperature: float = 0.8, device: str | None = None, language: str = 'en-us', use_peft: bool = False, peft_r: int = 8, peft_alpha: int = 16)

Configuration for PhonRLTrainer.

Parameters:

Name	Type	Description	Default
`model_name`	`str`	HuggingFace model ID or local path.	required
`num_steps`	`int`	Number of PPO training steps.	`100`
`batch_size`	`int`	Number of sequences generated per step.	`4`
`learning_rate`	`float`	Learning rate for the optimizer.	`1.41e-05`
`kl_coeff`	`float`	KL penalty coefficient (constrains policy drift).	`0.1`
`clip_epsilon`	`float`	PPO clipping parameter (Schulman et al., 2017).	`0.2`
`gae_gamma`	`float`	Discount factor for GAE.	`1.0`
`gae_lambda`	`float`	GAE lambda for bias-variance tradeoff.	`0.95`
`value_loss_coeff`	`float`	Coefficient for value function loss.	`0.5`
`output_dir`	`str \| None`	Directory for checkpoints and logs.	`None`
`seed`	`int`	Random seed for reproducibility.	`42`
`max_new_tokens`	`int`	Maximum new tokens per generated sequence.	`64`
`temperature`	`float`	Sampling temperature.	`0.8`
`use_peft`	`bool`	Whether to use PEFT/LoRA for parameter-efficient training.	`False`
`peft_r`	`int`	LoRA rank (when use_peft is True).	`8`
`peft_alpha`	`int`	LoRA alpha scaling (when use_peft is True).	`16`

TrainingResult¶

TrainingResult `dataclass` ¶

TrainingResult(mean_rewards: list[float], total_steps: int, final_coverage: float, checkpoint_path: str | None)

Result from a completed PPO training run.

Attributes:

Name	Type	Description
`mean_rewards`	`list[float]`	Mean composite reward per training step.
`total_steps`	`int`	Total number of PPO steps completed.
`final_coverage`	`float`	Coverage fraction at end of training (0.0–1.0).
`checkpoint_path`	`str \| None`	Path where the model was saved, or None.

PhonRLTrainer¶

PhonRLTrainer ¶

PhonRLTrainer(reward: PhoneticReward, config: TrainingConfig)

Orchestrates PPO training with a phonetic composite reward.

Implements the full PPO loop from Schulman et al. (2017): generate → reward → GAE → clipped policy update. No dependency on trl — the entire training loop is self-contained.

Parameters:

Name	Type	Description	Default
`reward`	`PhoneticReward`	PhoneticReward instance providing the reward signal.	required
`config`	`TrainingConfig`	TrainingConfig with model, training, and PEFT settings.	required

reward `property` ¶

reward: PhoneticReward

The phonetic reward function.

config `property` ¶

config: TrainingConfig

Training configuration.

is_initialized `property` ¶

is_initialized: bool

Whether train() has been called and completed.

train ¶

train(prompts: list[str] | None = None, prompt_fn: Callable[[PhoneticTargetInventory], str] | None = None, step_callback: Callable[..., None] | None = None) -> TrainingResult

Run PPO training with phonetic reward.

Exactly one of prompts or prompt_fn must be provided.

Parameters:

Name	Type	Description	Default
`prompts`	`list[str] \| None`	Static list of prompt strings, cycled through.	`None`
`prompt_fn`	`Callable[[PhoneticTargetInventory], str] \| None`	Dynamic callable receiving the current PhoneticTargetInventory, returning a prompt string.	`None`
`step_callback`	`Callable[..., None] \| None`	Optional callback invoked after each step with keyword arguments: `step` (int), `mean_reward` (float), `policy_loss` (float).	`None`

Returns:

Type	Description
`TrainingResult`	TrainingResult with per-step rewards and final coverage.

save_checkpoint ¶

save_checkpoint(path: str) -> None

Save the trained model and tokenizer to disk.

Parameters:

Name	Type	Description	Default
`path`	`str`	Directory path for the checkpoint.	required

Raises:

Type	Description
`RuntimeError`	If train() has not been called yet.

AttributeWordIndex¶

AttributeWordIndex ¶

AttributeWordIndex(language: str = 'en-us', batch_size: int = 512)

Bidirectional index mapping phonetic units ↔ vocabulary token IDs.

Built lazily from a tokenizer's vocabulary via G2P phonemization. Once built, supports fast lookups for attribute and anti-attribute token sets used by Phon-DATG logit modulation.

Parameters:

Name	Type	Description	Default
`language`	`str`	Language code for G2P conversion.	`'en-us'`
`batch_size`	`int`	Number of tokens to phonemize per G2P batch call.	`512`

Raises:

Type	Description
`ValueError`	If `batch_size` is not positive.

language `property` ¶

language: str

Language code.

is_built `property` ¶

is_built: bool

Whether the index has been built.

unit_to_tokens `property` ¶

unit_to_tokens: dict[str, set[int]]

Mapping of phonetic unit → set of token IDs (read-only copy).

Raises:

Type	Description
`RuntimeError`	If the index has not been built yet.

token_units `property` ¶

token_units: dict[int, set[str]]

Mapping of token ID → set of phonetic units (read-only copy).

Raises:

Type	Description
`RuntimeError`	If the index has not been built yet.

build ¶

build(tokenizer: Any) -> None

Build the index by phonemizing the tokenizer's full vocabulary.

This is an expensive one-time operation. Subsequent calls are no-ops (idempotent).

Parameters:

Name	Type	Description	Default
`tokenizer`	`Any`	A HuggingFace tokenizer (or any object with `get_vocab()` returning `dict[str, int]` and `decode(token_id)` returning `str`).	required

get_attribute_tokens ¶

get_attribute_tokens(target_units: list[str]) -> set[int]

Get token IDs for words containing ANY of the target units.

Parameters:

Name	Type	Description	Default
`target_units`	`list[str]`	Phonetic units to target.	required

Returns:

Type	Description
`set[int]`	Set of token IDs whose decoded text contains at least one
`set[int]`	of the target units.

Raises:

Type	Description
`RuntimeError`	If the index has not been built yet.

get_anti_attribute_tokens ¶

get_anti_attribute_tokens(covered_units: set[str], unit_level: str | None = None) -> set[int]

Get token IDs for words whose units are ALL already covered.

These tokens contribute nothing new to coverage and can be penalized during generation to steer towards uncovered units.

Parameters:

Name	Type	Description	Default
`covered_units`	`set[str]`	Set of phonetic units already covered.	required
`unit_level`	`str \| None`	If provided, only check units at this level: `"phoneme"`, `"diphone"`, or `"triphone"`. If None, checks all unit levels.	`None`

Returns:

Type	Description
`set[int]`	Set of token IDs whose phonetic units (at the specified
`set[int]`	level) are all in `covered_units`.

Raises:

Type	Description
`RuntimeError`	If the index has not been built yet.

get_anti_attribute_tokens_by_frequency ¶

get_anti_attribute_tokens_by_frequency(unit_counts: dict[str, int], threshold: int, unit_level: str | None = None) -> set[int]

Get token IDs for words whose units all exceed a frequency threshold.

A fine-grained alternative to get_anti_attribute_tokens that uses actual coverage counts rather than a binary covered/uncovered distinction.

Parameters:

Name	Type	Description	Default
`unit_counts`	`dict[str, int]`	Mapping of phonetic unit → occurrence count.	required
`threshold`	`int`	Minimum count. Tokens are included only if ALL their units have counts strictly greater than this value.	required
`unit_level`	`str \| None`	If provided, only check units at this level: `"phoneme"`, `"diphone"`, or `"triphone"`. If None, checks all unit levels.	`None`

Returns:

Type	Description
`set[int]`	Set of token IDs whose phonetic units (at the specified
`set[int]`	level) all have counts above `threshold`.

Raises:

Type	Description
`RuntimeError`	If the index has not been built yet.

DATGStrategy¶

DATGStrategy ¶

DATGStrategy(targets: PhoneticTargetInventory, language: str = 'en-us', boost_strength: float = 5.0, penalty_strength: float = -5.0, anti_attribute_mode: str = 'covered', frequency_threshold: int = 10, attribute_word_index: AttributeWordIndex | None = None, batch_size: int = 512)

Bases: GuidanceStrategy

Phon-DATG: inference-time logit steering via dynamic attribute graphs.

Uses an AttributeWordIndex to identify vocabulary tokens that contain target phonetic units (attribute words) or only already-covered units (anti-attribute words), then applies additive logit adjustments via a LogitModulator during autoregressive generation.

Parameters:

Name	Type	Description	Default
`targets`	`PhoneticTargetInventory`	PhoneticTargetInventory for tracking covered/missing units.	required
`language`	`str`	Language code for G2P conversion.	`'en-us'`
`boost_strength`	`float`	Additive boost for attribute token logits.	`5.0`
`penalty_strength`	`float`	Additive penalty for anti-attribute token logits.	`-5.0`
`anti_attribute_mode`	`str`	How to determine anti-attribute tokens: `"covered"` — tokens whose units are all already covered. `"frequency"` — tokens whose units all exceed a count threshold.	`'covered'`
`frequency_threshold`	`int`	Minimum count for frequency mode. Tokens are anti-attribute only if ALL their units have counts above this.	`10`
`attribute_word_index`	`AttributeWordIndex \| None`	Optional pre-built index. If None, one is created and built lazily on first `prepare()` call.	`None`
`batch_size`	`int`	Batch size for vocabulary phonemization during index build.	`512`

name `property` ¶

name: str

Strategy identifier.

language `property` ¶

language: str

Language code.

boost_strength `property` ¶

boost_strength: float

Additive boost for attribute tokens.

penalty_strength `property` ¶

penalty_strength: float

Additive penalty for anti-attribute tokens.

anti_attribute_mode `property` ¶

anti_attribute_mode: str

Anti-attribute determination mode.

frequency_threshold `property` ¶

frequency_threshold: int

Frequency threshold for frequency mode.

attribute_word_index `property` ¶

attribute_word_index: AttributeWordIndex

The underlying AttributeWordIndex.

current_attribute_tokens `property` ¶

current_attribute_tokens: set[int]

Current attribute token IDs (after prepare()).

Raises:

Type	Description
`RuntimeError`	If prepare() has not been called.

current_anti_attribute_tokens `property` ¶

current_anti_attribute_tokens: set[int]

Current anti-attribute token IDs (after prepare()).

Raises:

Type	Description
`RuntimeError`	If prepare() has not been called.

prepare ¶

prepare(target_units: list[str], model: Any, tokenizer: Any) -> None

Build index lazily and compute attribute/anti-attribute sets.

Parameters:

Name	Type	Description	Default
`target_units`	`list[str]`	Phonetic units to target in this generation.	required
`model`	`Any`	The HuggingFace model instance (unused here, part of ABC).	required
`tokenizer`	`Any`	The HuggingFace tokenizer instance.	required

modify_logits ¶

modify_logits(input_ids: Any, logits: Any) -> Any

Apply logit modulation via the LogitModulator.

Parameters:

Name	Type	Description	Default
`input_ids`	`Any`	Current token sequence (unused, part of ABC).	required
`logits`	`Any`	Raw logits tensor [batch, vocab_size].	required

Returns:

Type	Description
`Any`	Modified logits tensor (same shape).

Raises:

Type	Description
`RuntimeError`	If prepare() has not been called.

Backends¶

RepositoryBackend¶

RepositoryBackend ¶

RepositoryBackend(pool: list[dict])

Bases: GenerationBackend

Generation backend that selects from a sentence pool.

Candidates are ranked by how many of the requested target units they contain. Used sentences can be removed via mark_used().

Parameters:

Name	Type	Description	Default
`pool`	`list[dict]`	List of dicts, each with `"phonemes"` (list[str]) and optionally `"text"` (str).	required

name `property` ¶

name: str

Backend identifier.

pool_size `property` ¶

pool_size: int

Number of sentences remaining in the pool.

pool `property` ¶

pool: list[dict]

Copy of the current pool.

from_texts `classmethod` ¶

from_texts(texts: list[str], language: str = 'en-us') -> RepositoryBackend

Create a backend from raw text by running G2P.

Parameters:

Name	Type	Description	Default
`texts`	`list[str]`	List of sentences.	required
`language`	`str`	Language code for G2P conversion.	`'en-us'`

Returns:

Type	Description
`RepositoryBackend`	A RepositoryBackend with phonemized pool.

Raises:

Type	Description
`ValueError`	If texts is empty.

from_huggingface `classmethod` ¶

from_huggingface(dataset_name: str, text_column: str = 'text', split: str | None = None, language: str = 'en-us', max_samples: int | None = None, **dataset_kwargs: Any) -> RepositoryBackend

Create a backend from a HuggingFace dataset.

Parameters:

Name	Type	Description	Default
`dataset_name`	`str`	HuggingFace dataset identifier (e.g., "wikitext").	required
`text_column`	`str`	Column name containing text data.	`'text'`
`split`	`str \| None`	Dataset split (e.g., "train", "test").	`None`
`language`	`str`	Language code for G2P conversion.	`'en-us'`
`max_samples`	`int \| None`	Maximum number of samples to load.	`None`
`**dataset_kwargs`	`Any`	Forwarded to `datasets.load_dataset()`.	`{}`

Returns:

Type	Description
`RepositoryBackend`	A RepositoryBackend with phonemized pool.

Raises:

Type	Description
`ImportError`	If `datasets` package is not installed.
`ValueError`	If max_samples is not positive, or if no split can be selected unambiguously from a multi-split dataset, or if text_column is absent or does not contain strings.

generate ¶

generate(target_units: list[str], k: int = 5, **kwargs: Any) -> list[dict]

Return top-k pool sentences ranked by target unit overlap.

Parameters:

Name	Type	Description	Default
`target_units`	`list[str]`	Phonetic units to target.	required
`k`	`int`	Maximum candidates to return.	`5`

Returns:

Type	Description
`list[dict]`	List of candidate dicts with "text", "phonemes", and
`list[dict]`	"_pool_index" (for `mark_used`).

mark_used ¶

mark_used(pool_index: int) -> None

Remove a sentence from the pool by its index.

Parameters:

Name	Type	Description	Default
`pool_index`	`int`	Index into the current pool.	required

Raises:

Type	Description
`IndexError`	If index is out of range.

LLMBackend¶

LLMBackend ¶

LLMBackend(model: str, language: str = 'en-us', api_key: str | None = None, prompt_template: str | None = None, temperature: float = 0.8, max_tokens: int = 1024, max_retries: int = 3, retry_delay: float = 1.0, request_delay: float = 0.0)

Bases: GenerationBackend

Generation backend using LLM APIs via litellm.

Prompts an LLM to generate sentences targeting specific phonetic units, then phonemizes the output via G2P.

Parameters:

Name	Type	Description	Default
`model`	`str`	litellm model string (e.g., "openai/gpt-4o-mini", "anthropic/claude-3-5-sonnet-latest").	required
`language`	`str`	Language code for G2P and prompt context.	`'en-us'`
`api_key`	`str \| None`	API key for the provider. If None, litellm falls back to environment variables.	`None`
`prompt_template`	`str \| None`	Custom prompt template. Must contain `{target_units}` placeholder. May also contain `{language}` and `{k}`.	`None`
`temperature`	`float`	Sampling temperature.	`0.8`
`max_tokens`	`int`	Maximum tokens in LLM response.	`1024`
`max_retries`	`int`	Maximum retry attempts on transient failures.	`3`
`retry_delay`	`float`	Seconds to wait between retries.	`1.0`
`request_delay`	`float`	Seconds to wait before each API call (basic rate limiting).	`0.0`

name `property` ¶

name: str

Backend identifier.

model `property` ¶

model: str

litellm model string.

language `property` ¶

language: str

Language code.

api_key `property` ¶

api_key: str | None

API key (if provided).

prompt_template `property` ¶

prompt_template: str

Prompt template string.

max_retries `property` ¶

max_retries: int

Maximum retry attempts.

retry_delay `property` ¶

retry_delay: float

Seconds between retries.

request_delay `property` ¶

request_delay: float

Seconds before each API call.

format_prompt ¶

format_prompt(target_units: list[str], k: int = 5) -> str

Format the prompt template with target units.

Parameters:

Name	Type	Description	Default
`target_units`	`list[str]`	Phonetic units to target.	required
`k`	`int`	Number of sentences to request.	`5`

Returns:

Type	Description
`str`	Formatted prompt string.

generate ¶

generate(target_units: list[str], k: int = 5, **kwargs: Any) -> list[dict]

Generate candidate sentences via LLM.

Parameters:

Name	Type	Description	Default
`target_units`	`list[str]`	Phonetic units to target.	required
`k`	`int`	Number of candidates to generate.	`5`

Returns:

Type	Description
`list[dict]`	List of candidate dicts with "text" and "phonemes".

LocalBackend¶

LocalBackend ¶

LocalBackend(model_name: str, language: str = 'en-us', prompt_template: str | None = None, device: str | None = None, quantization: str | None = None, guidance_strategy: GuidanceStrategy | None = None, max_new_tokens: int = 256, temperature: float = 0.8, top_p: float = 0.95, do_sample: bool = True, model_kwargs: dict[str, Any] | None = None)

Bases: GenerationBackend

Generation backend using a local HuggingFace transformers model.

Loads a causal LM and generates text conditioned on phonetic targets via prompt formatting. Supports quantization for VRAM efficiency and an optional GuidanceStrategy for inference-time logit modification.

Model and tokenizer are loaded lazily on the first generate() call.

Parameters:

Name	Type	Description	Default
`model_name`	`str`	HuggingFace model ID or local path.	required
`language`	`str`	Language code for G2P and prompt context.	`'en-us'`
`prompt_template`	`str \| None`	Custom prompt template. Must contain `{target_units}` placeholder. May also contain `{language}` and `{k}`. Defaults to a short template suited for smaller models.	`None`
`device`	`str \| None`	Device string (`"cuda"`, `"cpu"`). If None, auto-detects CUDA with CPU fallback.	`None`
`quantization`	`str \| None`	Quantization mode: `None`, `"4bit"`, or `"8bit"`. Requires bitsandbytes when non-None.	`None`
`guidance_strategy`	`GuidanceStrategy \| None`	Optional GuidanceStrategy instance for inference-time logit steering (Phon-DATG or Phon-RL).	`None`
`max_new_tokens`	`int`	Maximum new tokens to generate per sequence.	`256`
`temperature`	`float`	Sampling temperature.	`0.8`
`top_p`	`float`	Nucleus sampling threshold.	`0.95`
`do_sample`	`bool`	Whether to sample (True) or use greedy decoding.	`True`
`model_kwargs`	`dict[str, Any] \| None`	Extra keyword arguments forwarded to `AutoModelForCausalLM.from_pretrained()`.	`None`

name `property` ¶

name: str

Backend identifier.

model_name `property` ¶

model_name: str

HuggingFace model ID or local path.

language `property` ¶

language: str

Language code.

prompt_template `property` ¶

prompt_template: str

Prompt template string.

device `property` ¶

device: str | None

Device string, or None if not yet resolved.

quantization `property` ¶

quantization: str | None

Quantization mode.

guidance_strategy `property` ¶

guidance_strategy: GuidanceStrategy | None

Optional guidance strategy.

max_new_tokens `property` ¶

max_new_tokens: int

Maximum new tokens per generated sequence.

temperature `property` ¶

temperature: float

Sampling temperature.

top_p `property` ¶

top_p: float

Nucleus sampling threshold.

do_sample `property` ¶

do_sample: bool

Whether to use sampling.

model_kwargs `property` ¶

model_kwargs: dict[str, Any]

Extra kwargs for model loading.

is_loaded `property` ¶

is_loaded: bool

Whether the model and tokenizer have been loaded.

format_prompt ¶

format_prompt(target_units: list[str], k: int = 5) -> str

Format the prompt template with target units.

Parameters:

Name	Type	Description	Default
`target_units`	`list[str]`	Phonetic units to target.	required
`k`	`int`	Number of sentences to request.	`5`

Returns:

Type	Description
`str`	Formatted prompt string.

generate ¶

generate(target_units: list[str], k: int = 5, **kwargs: Any) -> list[dict]

Generate candidate sentences using the local model.

Parameters:

Name	Type	Description	Default
`target_units`	`list[str]`	Phonetic units to target.	required
`k`	`int`	Number of candidates to generate.	`5`

Returns:

Type	Description
`list[dict]`	List of candidate dicts with `"text"` and `"phonemes"`.

G2P¶

G2PManager¶

G2PManager ¶

G2PManager(backend: str = 'espeak')

Manages grapheme-to-phoneme conversion with configurable backends.

Currently supports

"espeak": espeak-ng via the phonemizer library (100+ languages)

Future backends (Phase 5+): - "neural": ByT5/CharsiuG2P transformer model - "epitran": Rule-based Epitran backend

Parameters:

Name	Type	Description	Default
`backend`	`str`	Which G2P backend to use. Default: "espeak".	`'espeak'`

backend `property` ¶

backend: str

Currently active backend name.

phonemize ¶

phonemize(text: str, language: str = 'en-us') -> G2PResult

Convert text to phonemes.

Parameters:

Name	Type	Description	Default
`text`	`str`	Input text (word or sentence).	required
`language`	`str`	Language code (e.g., 'en-us', 'fr-fr', 'ar').	`'en-us'`

Returns:

Type	Description
`G2PResult`	G2PResult with IPA transcription and parsed phonemes.

phonemize_batch ¶

phonemize_batch(texts: list[str], language: str = 'en-us') -> list[G2PResult]

Phonemize multiple texts efficiently.

Pre-filters empty/whitespace-only strings to avoid misalignment, since phonemizer may drop empty utterances from its output.

Parameters:

Name	Type	Description	Default
`texts`	`list[str]`	List of input texts.	required
`language`	`str`	Language code.	`'en-us'`

Returns:

Type	Description
`list[G2PResult]`	List of G2PResult, one per input text (order preserved).

phonemize_variants ¶

phonemize_variants(text: str, language: str = 'en-us') -> list[G2PResult]

Get pronunciation variants for a word/text.

Uses espeak-ng's built-in variant support. For espeak, this typically returns one canonical pronunciation. Future backends (neural, dictionary) will provide richer variant support.

Parameters:

Name	Type	Description	Default
`text`	`str`	Input text.	required
`language`	`str`	Language code.	`'en-us'`

Returns:

Type	Description
`list[G2PResult]`	List of G2PResult, one per pronunciation variant.

supported_languages ¶

supported_languages() -> list[str]

List languages supported by the current backend.

Returns:

Type	Description
`list[str]`	Sorted list of language codes (e.g., 'en-us', 'fr-fr').

G2PResult¶

G2PResult `dataclass` ¶

G2PResult(text: str, ipa: str, phonemes: list[str], language: str)

Result of a grapheme-to-phoneme conversion.

Attributes:

Name	Type	Description
`text`	`str`	Original input text.
`ipa`	`str`	IPA transcription string.
`phonemes`	`list[str]`	List of individual phoneme segments.
`language`	`str`	Language code used for conversion.

diphones `property` ¶

diphones: list[str]

Adjacent phoneme pairs (bigrams).

triphones `property` ¶

triphones: list[str]

Adjacent phoneme triples (trigrams).

phoneme_count `property` ¶

phoneme_count: int

Total number of phonemes.

unique_phonemes `property` ¶

unique_phonemes: set[str]

Set of distinct phonemes.

Coverage Tracking¶

CoverageTracker¶

CoverageTracker ¶

CoverageTracker(target_phonemes: list[str], unit: str = 'phoneme', max_target_size: int | None = None)

Tracks coverage of phonetic units against a target inventory.

Supports phoneme, diphone, and triphone units. Maintains frequency counts and sentence-level provenance for each covered unit.

Parameters:

Name	Type	Description	Default
`target_phonemes`	`list[str]`	List of phonemes in the target inventory.	required
`unit`	`str`	Coverage unit type — "phoneme", "diphone", or "triphone".	`'phoneme'`

unit `property` ¶

unit: str

Coverage unit type.

target_units `property` ¶

target_units: set[str]

Full set of target units (phonemes, diphones, or triphones).

target_size `property` ¶

target_size: int

Number of units in the target inventory.

covered_count `property` ¶

covered_count: int

Number of target units covered so far.

coverage `property` ¶

coverage: float

Fraction of target units covered (0.0 to 1.0).

covered_units `property` ¶

covered_units: set[str]

Set of target units covered so far.

missing `property` ¶

missing: set[str]

Set of target units not yet covered.

phoneme_counts `property` ¶

phoneme_counts: dict[str, int]

Per-unit occurrence counts (all occurrences, not just first).

phoneme_sources `property` ¶

phoneme_sources: dict[str, list[int]]

Maps each covered unit to the sentence indices where it appeared.

update ¶

update(phonemes: list[str], sentence_index: int) -> None

Update coverage state with phonemes from a sentence.

Parameters:

Name	Type	Description	Default
`phonemes`	`list[str]`	List of phonemes extracted from the sentence.	required
`sentence_index`	`int`	Index of the sentence in the corpus.	required

reset ¶

reset() -> None

Reset all coverage state, keeping the target inventory.

API Reference¶

Top-Level Functions¶

evaluate¶

evaluate ¶

select_sentences¶

select_sentences ¶

get_inventory¶

get_inventory ¶

Data Models¶

Inventory¶

Inventory dataclass ¶

phonemes property ¶

consonants property ¶

vowels property ¶

tones property ¶

marginal_phonemes property ¶

non_marginal_phonemes property ¶

consonant_segments property ¶

vowel_segments property ¶

tone_segments property ¶

marginal_segments property ¶

non_marginal_segments property ¶

all_allophones property ¶

size property ¶

consonant_count property ¶

vowel_count property ¶

tone_count property ¶

has_tones property ¶

marginal_count property ¶

segments_with_feature ¶

segments_with_features ¶

to_dict ¶

Segment¶

Segment dataclass ¶

PhoibleDataset¶

PhoibleDataset ¶

cache_dir property ¶

csv_path property ¶

csv_exists property ¶

is_loaded property ¶

inventory_count property ¶

language_count property ¶

segment_count property ¶

load ¶

get_inventory ¶

get_all_inventories ¶

get_union_inventory ¶

search ¶

available_languages ¶

sources_for ¶

get_inventory_for_espeak ¶

get_union_inventory_for_espeak ¶

download ¶

Evaluation Results¶

EvaluationReport¶

EvaluationReport dataclass ¶

__post_init__ ¶

render ¶

to_dict ¶

to_json ¶

to_jsonld_ex ¶

DistributionMetrics¶

DistributionMetrics dataclass ¶

to_dict ¶

TextQualityMetrics¶

TextQualityMetrics dataclass ¶

to_dict ¶

Verbosity¶

Verbosity ¶

compute_distribution_metrics¶

compute_distribution_metrics ¶

CoverageTrajectory¶

CoverageTrajectory dataclass ¶

coverages property ¶

gains property ¶

to_dict ¶

compute_coverage_trajectory¶

compute_coverage_trajectory ¶

ErrorRateResult¶

ErrorRateResult dataclass ¶

Inventory `dataclass` ¶

phonemes `property` ¶

consonants `property` ¶

vowels `property` ¶

tones `property` ¶

marginal_phonemes `property` ¶

non_marginal_phonemes `property` ¶

consonant_segments `property` ¶

vowel_segments `property` ¶

tone_segments `property` ¶

marginal_segments `property` ¶

non_marginal_segments `property` ¶

all_allophones `property` ¶

size `property` ¶

consonant_count `property` ¶

vowel_count `property` ¶

tone_count `property` ¶

has_tones `property` ¶

marginal_count `property` ¶

Segment `dataclass` ¶

cache_dir `property` ¶

csv_path `property` ¶

csv_exists `property` ¶

is_loaded `property` ¶

inventory_count `property` ¶

language_count `property` ¶

segment_count `property` ¶

EvaluationReport `dataclass` ¶

DistributionMetrics `dataclass` ¶

TextQualityMetrics `dataclass` ¶

CoverageTrajectory `dataclass` ¶

coverages `property` ¶

gains `property` ¶

ErrorRateResult `dataclass` ¶

CorpusPerplexityMetrics `dataclass` ¶

SelectionResult `dataclass` ¶

num_selected `property` ¶

backend `property` ¶

targets `property` ¶

scorer `property` ¶

stopping_criteria `property` ¶

candidates_per_iteration `property` ¶

StoppingCriteria `dataclass` ¶

tracker `property` ¶

unit `property` ¶

target_size `property` ¶

target_units `property` ¶

covered_count `property` ¶

covered_units `property` ¶

coverage `property` ¶

missing `property` ¶

targets `property` ¶

coverage_weight `property` ¶

phonotactic_weight `property` ¶

fluency_weight `property` ¶

readability_weight `property` ¶

n `property` ¶

phonemes `property` ¶

from_corpus `classmethod` ¶

load `classmethod` ¶

call ¶

model_name `property` ¶

is_loaded `property` ¶

from_model `classmethod` ¶

call ¶

target_range `property` ¶

call ¶

targets `property` ¶

coverage_weight `property` ¶