Overview
A loop has to make sense twice. Its notes need to form a musical phrase, and the final beat needs to meet the first beat naturally when playback starts over. The researchers built a detector for loop-like passages and a generator that writes in compressed musical patterns. That lets the generator learn at the scale of recurring phrases and rhythms.
Key Findings
Can a bassline meet its own beginning?
Picture an eight-bar bass and drum pattern playing in a DJ's remix. When the last beat ends, bar one starts again, and any seam becomes audible. Moontae and his colleagues at LG AI Research built a system around that moment. Their framework has three parts. A detector finds loop-like passages in MIDI files, a compressed vocabulary of musical patterns represents them, and an evaluation method compares generated music with real loops. The authors argue that fixing the target to a short, repeatable phrase simplifies generation, because long-form models still struggle with error accumulation and rhythmic irregularity.
Teaching a detector what a loop sounds like
MIDI collections rarely label loops, but audio sites do. The authors collected 1,000 eight-bar loops from Looperman, a site where producers share free loops. They converted each loop into an 8×8 table of bar-to-bar similarity, built from mel-spectrograms (a frequency picture of sound). For MIDI, they computed normalized Hamming distance between bars and renormalized it into the same kind of similarity table. This shared structure lets a detector trained on audio judge symbolic music.
The Lakh MIDI Dataset supplied the symbolic material. The authors kept bass guitar and drums, quantized notes to sixteenth notes, and kept only 4/4 music. Bass pitches spanned C1 to B4 (48 pitches), drums used nine standard components, and the full grid has 57 pitch rows. Sliding an eight-bar window by one bar produced 5,687,274 phrases.
The detector is a One-Class Deep SVDD, which learns only from normal examples (here, loops). Its training objective is the following, labeled Equation 1 in the paper.
Here is a small neural network, is a center point, counts training samples, and regularizes the weights. Training pulls loops toward the center. A phrase's loop score (Equation 2) is its squared distance from that center, so lower means more loop-like.
The authors checked the detector with 100 hand-picked loops against four random groups, as shown in Figure 2. The loop group scored lowest at 0.153 (±0.282), and paired t-tests against each other group gave p < 0.001. A threshold at one standard deviation above the training scores, described as conservative, kept 751,935 phrases.
A vocabulary of 512 musical patterns
Generation runs in two stages. A vector-quantized variational autoencoder (VQ-VAE) squeezes each 128-step loop into 32 latent positions of 16 dimensions. Each position snaps to the nearest of 512 learned codebook entries. A four-layer LSTM then learns to predict the next code, reaching 76.651% validation accuracy under teacher forcing. Sampling a code sequence and passing it through the decoder yields a new loop. The authors compared temperature (0.7), top-k (30), and nucleus sampling (top-p 0.08).
Compression has a price. Table 1 reports reconstruction error of 6.643e-3 for the VQ-VAE and 4.412e-3 for a CNN-VAE baseline. The authors also report that heavy compression along the position dimension badly hurt reconstruction, which shaped their choice of 32 positions.
How close is a new loop to the collection?
Music lacks a standard pretrained feature extractor like the ones behind image metrics. The authors followed Naeem and colleagues and embedded samples with randomly initialized CNNs. Precision and recall ask whether generated samples land inside the real data's region and whether real samples land inside the generated region. Density and coverage correct for outliers. All four use five nearest neighbors, 10,000 samples per network, and 10 networks. They also report loop score, unique pitches per bar, and note density.
Table 2 gives selected values, with the VQ-VAE+LSTM row using top-k sampling.
| Model | Precision | Recall | Density | Coverage |
|---|---|---|---|---|
| CNN-VAE | 0.642 | 0.625 | 0.617 | 0.746 |
| Music Transformer | 0.546 | 0.359 | 0.687 | 0.408 |
| MuseGAN | 0.641 | 0.689 | 0.673 | 0.842 |
| VQ-VAE+LSTM (top-k) | 0.779 | 0.636 | 1.328 | 0.952 |
The VQ-VAE+LSTM leads on precision, density, and coverage. MuseGAN keeps the higher recall, which the authors suggest its outliers may inflate. Table 3 combines the pairs into F1 scores. For density and coverage the VQ-VAE+LSTM variants reach 1.084 to 1.111, against 0.748 for MuseGAN. On loop score, the top-k variant reached 0.1978 versus 0.2307 for MuseGAN, and the Music Transformer reached 0.720, which the authors say shows it fails to preserve loop properties. Note density was closest to the training set, but unique pitches per bar (5.04 to 5.08 versus 5.769 in training) fell short, which the authors tentatively link to reconstruction limits. They also saw a fidelity-diversity trade-off across sampling methods.
Stricter selection changes the loops
The detector also works after generation. Rejection sampling discards samples whose loop scores miss a cutoff, with five rates from no rejection to 0.001.
Figure 4 shows both models trending similarly in precision and recall but diverging elsewhere. Against the authors' expectation, MuseGAN's density and coverage rose under stricter filtering. They read this as evidence against the idea that MuseGAN produces many outliers, and as a sign that the detector can move samples toward the true distribution. Note density barely changed, and the two models disagreed on unique pitches.
What listeners heard
Twenty participants rated 10 samples from each of three groups for how well they sound like seamlessly repeatable loops. Training loops averaged 3.885 (±1.045), VQ-VAE+LSTM with top-k averaged 3.585 (±1.141), and MuseGAN averaged 3.060 (±1.172). A Kruskal-Wallis test gave H = 49.811 and p < 0.001, which shows a difference among the groups. The study is small, covers only bass and drums at eight bars, and the authors judge only their top-k variant by ear.
Questions & Answers
Q: What makes a MIDI loop different from an audio loop?
A: A MIDI loop contains symbolic instructions that can be edited and played with different instruments. Its timing and note patterns can still be tested for a clean repeat. The generator in this study produces symbolic music.
Q: Why generate codes instead of individual notes?
A: A code can summarize a pattern spanning several musical events. The generation model can work with a more compact sequence and leave the decoder to reconstruct the note-level detail. The quantization can also lose details, which is reflected in some reconstruction results.
Q: What does it mean when precision rises but recall falls?
A: Precision rewards generated samples that resemble the real distribution. Recall asks how much of the real distribution the generated set covers. A generator can produce plausible loops while exploring fewer musical patterns.
Q: Did the listening test isolate which change improved perceived repetition?
A: The tested systems differ in loop extraction, the representation and sampling. Listener ratings assess the resulting clips, so they do not identify the effect of each part. The experiment also covers bass and drum pianorolls. Applying it to other styles would require fresh clips and listeners.
Future Ideas
A loop browser that helps musicians find clean transitions
Build an editable MIDI tool that generates several candidate variations from an existing phrase, then ranks them using a transition check trained on pairs of adjoining phrases. The paper already filters candidates using its whole-loop detector. This tool would add generation conditioned on a musician’s phrase and a separate score for the join from bar eight to bar one. Preview each candidate in repeated playback and retain the user’s ability to edit notes and alter the ending independently.
Compare the tool with unranked samples and with a rule-based boundary check. Have musicians assess seamlessness and time to produce a usable variation on music they supplied. Include unusual rhythms and held notes to see whether the ranker favors easy patterns at the expense of useful variation.
Evaluate the seam score and conditional generation independently on matched versions of the same base phrases. Randomize presentation order and report uncertainty across both musicians and clips, with sample diversity measured separately.
Sources
- ISMIR 2022 poster and paper record
- Paper PDF, including Figures 1, 3, and 5 and Table 2
- Authors’ implementation and listening examples
Original abstract
Since most of music has repetitive structures from motifs to phrases, repeating musical ideas can be a basic operation for music composition. The basic block that we focus on is conceptualized as loops which are essential ingredients of music. Furthermore, meaningful note patterns can be formed in a finite space, so it is sufficient to represent them with combinations of discrete symbols as done in other domains. In this work, we propose symbolic music loop generation via learning discrete representations. We first extract loops from MIDI datasets using a loop detector and then learn an autoregressive model trained by discrete latent codes of the extracted loops. We show that our model outperforms well-known music generative models in terms of both fidelity and diversity, evaluating on random space. Our code and supplementary materials are available at https://github.com/sjhan91/Loop_VQVAE_Official.