Utils methods

Module containing utils methods than can be used outside of tokenization.

miditok.utils.compute_ticks_per_bar(time_sig: TimeSignatureFactory(), time_division: int) → int

Compute the number of ticks in a bar.

The number of ticks per bar depends on the time signature.

Parameters:

  • time_sig – time signature object.

  • time_division – symusic.Score time division (.tpq) in ticks/quarter.

Returns:

bar resolution, in ticks/bar

miditok.utils.compute_ticks_per_beat(time_sig_denominator: int, time_division: int) → int

Compute the number of ticks in a beat at a given time signature.

Parameters:

  • time_sig_denominator – time signature denominator.

  • time_division – symusic.Score time division (.tpq) in ticks/quarter.

Returns:

number of ticks per beat at the given time signature.

miditok.utils.concat_scores(scores: Sequence[Score], end_ticks: Sequence[int]) → Score

Concatenate a sequence of symusic.Scores.

Note: the tracks are concatenated in the same order as they are given. The scores must all have the same time division. (score.tpq) The concatenated score might have identical consecutive tempos, time signatures or key signatures values.

Parameters:

  • scores – symusic.Scores to concatenate.

  • end_ticks – the ticks indicating the end of each score. The end for the last score is not required.

Returns:

the concatenated symusic.Score.

miditok.utils.convert_ids_tensors_to_list(ids) → list[int] | list[list[int]]

Convert a PyTorch, Tensorflow Tensor or numpy array to a list of integers.

This method works with Jax too. It is recursive and will convert nested Tensors/arrays.

Parameters:

ids – ids sequence to convert.

Returns:

the input, as a list of integers

miditok.utils.detect_chords(notes: NoteTickList, ticks_per_beat: np.array, chord_maps: dict[str, Sequence[int]], program: int | None = None, specify_root_note: bool = True, beat_res: int = 4, onset_offset: int = 1, unknown_chords_num_notes_range: tuple[int, int] | None = None, simul_notes_limit: int = 10) → list[Event]

Detect chords in a list of notes.

Make sure to sort notes by start time then pitch before: notes.sort(key=lambda x: (x.start, x.pitch)). On very large tracks with high note density this method can be slow. If you plan to use it with the Maestro or GiantMIDI datasets, it can take up to hundreds of seconds per file depending on your cpu. This method works by iterating over each note, find if it played with other notes, and if it forms a chord from the chord maps. It does not consider chord inversion..

Parameters:

  • notes – notes to analyse (sorted by starting time, them pitch).

  • ticks_per_beat – array indicating the number of ticks per beat per time signature denominator section. The numbers of ticks per beat depend on the time signatures of the Score being parsed. The array has a shape (N,2), for N changes of ticks per beat, and the second dimension representing the end tick of each section and the number of ticks per beat respectively.

  • chord_maps – list of chord maps, to be given as a dictionary where keys are chord qualities (e.g. “maj”) and values pitch maps as tuples of integers (e.g. (0, 4, 7)). You can use miditok.constants.CHORD_MAPS as an example.

  • program – program of the track of the notes. Used to specify the program when creating the Event object. (default: None)

  • specify_root_note – the root note of each chord will be specified in Events/tokens. Tokens will look as “Chord_C:maj”. (default: True)

  • beat_res – beat resolution, i.e. number of samples per beat (default 4).

  • onset_offset – maximum offset (in samples) ∈ N separating notes starts to consider them starting at the same time / onset (default is 1).

  • unknown_chords_num_notes_range – range of number of notes to represent unknown chords. If you want to represent chords that does not match any combination in chord_maps, use this argument. Leave None to not represent unknown chords. (default: None)

  • simul_notes_limit – number of simultaneous notes being processed when looking for a chord this parameter allows to speed up the chord detection, and must be >= 5 (default 10).

Returns:

the detected chords as Event objects.

miditok.utils.filter_dataset(files_paths: Sequence[Path], valid_fn: Callable[[Score, Path], bool] | None = None, delete_invalid_files: bool = False) → list[Path]

Filter a list of paths to only retain valid files.

This method ise useful in order to filter corrupted files that cannot be loaded, or that do not satisfy user-specified conditions thanks to the validation_fn.

Parameters:

  • files_paths – paths to the files to filter.

  • valid_fn – an optional validation function. It must take a Score and a Path arguments and return a boolean. (default: None)

  • delete_invalid_files – if True, will delete inplace the files on the file system. (default: False)

Returns:

the list of paths to the files satisfying your conditions.

miditok.utils.fix_offsets_overlapping_notes(notes: NoteTickList) → None

Reduce the durations of overlapping notes.

By applying this method, when a note starts while it is already being played, the previous note will end. Before running this method make sure the notes are sorted by start then pitch then end values: notes.sort(key=lambda x: (x.start, x.pitch, x.end)).

Parameters:

notes – notes to fix.

miditok.utils.get_average_num_tokens_per_note(tokenizer: MusicTokenizer, files_paths: Sequence[Path]) → float

Return the average number of tokens per note (tpn) for a list of music files.

With a trained tokenizer, the average tpn is likely to be very low.

Parameters:

  • tokenizer – tokenizer.

  • files_paths – list of paths to music files.

Returns:

the average tokens per note.

miditok.utils.get_bars_ticks(score: ScoreFactory(), only_notes_onsets: bool = False) → list[int]

Compute the ticks of the bars of a symusic.Score.

Note: When encountering multiple time signature messages at a same tick, we this method will automatically consider the last one (coming in the list). Other software can proceed differently. Logic Pro, for example, uses the first one. I haven’t found documentation or recommendations for this specific situation. It might be better to use the first one and discard the others. Time signatures with non-positive numerator or denominator values (including 0) will be ignored.

Parameters:

  • score – symusic.Score to analyze.

  • only_notes_onsets – if enabled, bars at the end without any note onsets, i.e. only active notes, will be ignored. (default: False)

Returns:

list of ticks for each bar.

miditok.utils.get_beats_ticks(score: ScoreFactory(), only_notes_onsets: bool = False) → list[int]

Return the ticks of the beats of a symusic.Score.

Note: When encountering multiple time signature messages at a same tick, we this method will automatically consider the last one (coming in the list). Other software can proceed differently. Logic Pro, for example, uses the first one. I haven’t found documentation or recommendations for this specific situation. It might be better to use the first one and discard the others. Time signatures with non-positive numerator or denominator values (including 0) will be ignored.

Parameters:

  • score – symusic.Score to analyze.

  • only_notes_onsets – if enabled, bars at the end without any note onsets, i.e. only active notes, will be ignored. (default: False)

Returns:

list of ticks for each beat.

miditok.utils.get_num_notes_per_bar(score: ScoreFactory(), tracks_indep: bool = False) → list[int] | list[list[int]]

Return the number of notes within each bar of a symusic.Score.

Parameters:

  • score – symusic.Score object to analyze.

  • tracks_indep – whether to process each track independently or all together.

Returns:

the number of notes within each bar.

miditok.utils.get_score_programs(score: ScoreFactory()) → list[tuple[int, bool]]

Return the list of programs of the tracks of a symusic.Score.

Parameters:

score – the symusic.Score object to extract tracks programs

Returns:

the list of track programs, as a list of tuples (program, is_drum)

miditok.utils.get_score_ticks_per_beat(score: ScoreFactory()) → ndarray

Compute the portions of numbers of ticks in a beat in a symusic.Score.

The method returns a numpy array of shape (N,2), for N ticks-per-beat changes, and the second dimension corresponding to the ending tick and the number of ticks per beat of the portion.

Parameters:

score – symusic.Score to analyze.

Returns:

ticks per beat values as a numpy array.

miditok.utils.is_track_empty(track: TrackFactory(), check_controls: bool = False, check_pedals: bool = False, check_pitch_bend: bool = False) → bool

Return a boolean indicating if a symusic.Track is empty.

By default, only the notes are checked.

Parameters:

  • track – symusic.Track to check.

  • check_controls – whether to check the control changes. (default: False)

  • check_pedals – whether to check the control changes. (default: False)

  • check_pitch_bend – whether to check the pitch bends. (default: False)

Returns:

a boolean indicating if the track has at least one element.

miditok.utils.merge_same_program_tracks(tracks: list[Track] | TrackTickList, effects: bool = True) → None

Merge tracks having the same program number.

Parameters:

  • tracks – list of tracks.

  • effects – will also merge effects, i.e. control changes, sustain pedals and pitch bends. (default: True)

miditok.utils.merge_scores(scores: Sequence[Score]) → Score

Merge a list of symusic.Scores into a single one.

This method will combine all their tracks and global events such as tempo changes or time signature changes.

Parameters:

scores – symusic.Scores to merge.

Returns:

the merged symusic.Score.

miditok.utils.merge_tracks(tracks: list[Track] | TrackTickList | Score, effects: bool = True) → Track

Merge several symusic.Tracks.

The notes (and optionally effects) will be concatenated and sorted by time. All the tracks will be merged into the first Track of the list.

Parameters:

  • tracks – list of tracks to merge, or symusic.Score object.

  • effects – will also merge effects, i.e. control changes, sustain pedals and pitch bends. (default: True)

Returns:

the merged track.

miditok.utils.merge_tracks_per_class(score: ScoreFactory(), classes_to_merge: list[int] | None = None, new_program_per_class: dict[int, int] | None = None, max_num_of_tracks_per_inst_class: dict[int, int] | None = None, valid_programs: list[int] | None = None, filter_pitches: bool = True) → None

Merge symusic.Tracks per MIDI program class.

Example, a list of tracks / programs [0, 3, 8, 10, 11, 24, 25, 44, 47] will become [0, 8, 24, 25, 40] if classes_to_merge is [0, 1, 5]. The classes are in miditok.constants.INSTRUMENT_CLASSES.

Note: programs of drum tracks will be set to -1.

Parameters:

  • score – symusic.Score object to merge tracks

  • classes_to_merge – instrument classes to merge, to give as list of indexes (see miditok.constants.INSTRUMENT_CLASSES). Give None to merge nothing, the function will still remove non-valid programs/tracks if given. (default: None)

  • new_program_per_class – new program of the final merged tracks, to be given per instrument class as a dict {class_id: program}. (default: None)

  • max_num_of_tracks_per_inst_class – max number of tracks per instrument class, if the limit is exceeded for one class only the tracks with the maximum notes will be kept, give None for no limit. (default: None)

  • valid_programs – valid program ids to keep, others will be deleted, give None for keep all programs. (default None)

  • filter_pitches – after merging, will remove notes whose pitches are out the recommended range defined by the GM2 specs. (default: True)

miditok.utils.num_bar_pos(seq: Sequence[int], bar_token_id: int, position_tokens_ids: Sequence[int]) → tuple[int, int]

Return the number of bars and the last position of a sequence of tokens.

This method is compatible with tokenizations representing time with Bar and Position tokens, such as miditok.REMI.

Parameters:

  • seq – sequence of tokens

  • bar_token_id – the bar token value

  • position_tokens_ids – position tokens values

Returns:

the current bar, current position within the bar, current pitches played at this position, and if a chord token has been predicted at this position.

miditok.utils.remove_duplicated_notes(notes: NoteTickList | dict[str, ndarray], consider_duration: bool = False) → None

Remove (inplace) duplicated notes, i.e. with the same pitch and onset time.

This can be done either from a symusic.NoteTickList, or a note structure of arrays (symusic.NoteTickList.numpy()). The velocities are ignored in this method. The notes need to be sorted by time, then pitch, and duration if consider_duration is True: notes.sort(key=lambda x: (x.start, x.pitch, x.duration)).

Parameters:

  • notes – notes to analyse.

  • consider_duration – if given True, the method will also consider the durations of the notes when detecting identical ones. (default: False)

miditok.utils.split_files_for_training(files_paths: Sequence[Path], tokenizer: MusicTokenizer, save_dir: Path, max_seq_len: int, average_num_tokens_per_note: float | None = None, num_overlap_bars: int = 1, min_seq_len: int | None = None, preprocessing_method: callable[Score, Score] | None = None) → list[Path]

Split a list of music files into smaller chunks to use for training.

Splitting files allows to split them into chunks of lengths calculated in function of the note densities of its bars in order to reduce the padding of the batches, using the miditok.pytorch_data.split_score_per_note_density() method. The files are only split at bars, in order have chunks starting at relevant times.

File splitting can be performed on a dataset once. This method will save a hidden file, with a name corresponding to the hash of the list of file paths, in the save_dir directory. When called, it will first check that this file does not already exist, and if it is the case will return the paths to all the files within save_dir.

If your tokenizer does not tokenize all tracks in one sequence of tokens (tokenizer.one_token_stream), the music tracks will be split independently.

Parameters:

  • files_paths – paths to music files to split.

  • tokenizer – tokenizer.

  • save_dir – path to the directory to save the files splits.

  • max_seq_len – maximum token sequence length that the model will be trained with.

  • average_num_tokens_per_note – average number of tokens per note associated to this tokenizer. If given None, this value will automatically be calculated from the first 200 files with the miditok.pytorch_data.get_average_num_tokens_per_note() method.

  • num_overlap_bars – will create chunks with consecutive overlapping bars. For example, if this argument is given 1, two consecutive chunks might end at the bar n and start at the bar n-1 respectively, thus they will encompass the same bar. This allows to create a causality chain between chunks. This value should be determined based on the average_num_tokens_per_note value of the tokenizer and the max_seq_len value, so that it is neither too high nor too low. (default: 1).

  • min_seq_len – minimum sequence length, only used when splitting at the last bar of the file. (default: None, see default value of miditok.pytorch_data.split_score_per_note_density())

  • preprocessing_method – a custom preprocessing method to apply to each symusic.Score before splitting it. This method must take as input a symusic.Score and return a symusic.Score. (default: None)

Returns:

the paths to the files splits.

miditok.utils.split_score_per_beats(score: ScoreFactory(), max_num_beats: int, min_num_beats: int = 1) → list[ScoreFactory()]

Split a symusic.Score into several smaller chunks per number of beats.

This method splits a symusic.Score into smaller chunks that contains max_num_beats beats. The symusic.Score chunks will all start at tick 0.

Parameters:

  • score – symusic.Score object to split.

  • max_num_beats – maximum number of beats per segment.

  • min_num_beats – minimum number of beats per segment. This only applied to the last segment of the input score. (default: 1)

Returns:

a list of symusic.Score chunks.

miditok.utils.split_score_per_note_density(score: ScoreFactory(), max_seq_len: int, average_num_tokens_per_note: float, num_overlap_bars: int = 1, min_seq_len: int | None = None) → list[ScoreFactory()]

Split a symusic.Score (at bars) into chunks depending on their note densities.

This method aims to split music files at bars to reduce the amount of padding to apply to batches during training. It offers several parameters to control where to split depending on the desired outcome, e.g. reduce padding or keep the largest amount of data at the cost of padding.

This method will estimate the number of tokens for each bar depending on the tokenizer’s average number of tokens per note (tpn), will loop over the estimated number of tokens per bar to determine the bars at which the file will be “cut”.

It is recommended to use this method with a non-zero num_overlap_bars, as overlapping allows to keep a form of causality throughout a file from one chunk to another. It also reduces padding, but will slightly increase the overall training time.

Parameters:

  • score – symusic.Score to split.

  • max_seq_len – maximum number of tokens per sequence.

  • average_num_tokens_per_note – average number of tokens per note associated to this tokenizer.

  • num_overlap_bars – will create chunks with consecutive overlapping bars. For example, if this argument is given 1, two consecutive music chunks might end at the bar n and start at the bar n-1 respectively, thus they will encompass the same bar. This allows to create a causality chain between chunks. This value should be determined based on the average_num_tokens_per_note value of the tokenizer and the max_seq_len value, so that it is neither too high nor too low. (default: 1).

  • min_seq_len – minimum sequence length, only used when splitting at the last bar of the file. (default: max_seq_len // 4)

Returns:

the list of symusic.Score chunks.

miditok.utils.split_score_per_ticks(score: ScoreFactory(), ticks: list[int]) → list[ScoreFactory()]

Split a symusic.Score into several smaller symusic.Scores.

The symusic.Score chunks will all start at tick 0. Example: for a symusic.Score with an end tick at 1000, and a list of tick [2000, 5000, 7000], this method will return a list of four symusic.Score which correspond respectively to the portions of the original Score from tick 0 to 2000, 2000 to 5000, 5000 to 7000 and 10000 to 10000.

Parameters:

  • score – symusic.Score object to split.

  • ticks – list of ticks to which the score will be split.

Returns:

a list of segmented symusic.Score objects.

miditok.utils.split_score_per_tracks(score: ScoreFactory()) → list[ScoreFactory()]

Split a symusic.Score into several scores for each of its tracks.

The split scores will all start at tick 0. Example: for a score with an end tick at 1000, and a list of tick [2000, 5000, 7000], this method will return a list of four scores which correspond respectively to the portions of the original score from tick 0 to 2000, 2000 to 5000, 5000 to 7000 and 10000 to 10000.

Parameters:

score – symusic.Score object to split.

Returns:

a list of split symusic.Score objects.

miditok.utils.split_seq_in_subsequences(seq: Sequence[any], min_seq_len: int, max_seq_len: int) → list[Sequence[Any]]

Split a sequence of tokens into subsequences.

The subsequences will have lengths comprised between min_seq_len and max_seq_len: min_seq_len <= len(sub_seq) <= max_seq_len.

Parameters:

  • seq – sequence to split.

  • min_seq_len – minimum sequence length.

  • max_seq_len – maximum sequence length.

Returns:

list of subsequences.

miditok.utils.split_tokens_files_to_subsequences(files_paths: Sequence[Path], out_dir: Path, min_seq_len: int, max_seq_len: int, one_token_stream: bool = True) → None

Split JSON tokens files into subsequences of defined lengths.

This method is particularly useful if you plan to use a miditok.pytorch_data.DatasetJSON, as it would split token sequences into subsequences with the desired lengths before loading them for training.

Parameters:

  • files_paths – list of files of tokens to split.

  • out_dir – output directory to save the subsequences.

  • min_seq_len – minimum sequence length.

  • max_seq_len – maximum sequence length.

  • one_token_stream – provide False if the token files contains multiple token streams, i.e. the first dimension of the value of the “ids” entry corresponds to several tracks. Otherwise, leave True. (default: True)