Runtime latency control for streaming guitar tablature transcription through constant-Q buffer truncation

Overview

Our system transcribes live guitar audio into tablature, the string and fret of each note, as the guitarist plays. Its latency comes mostly from the constant-Q analysis buffer that the lowest notes need. We truncate this buffer at runtime, so one trained model runs with buffers from 216 ms down to 16 ms without retraining.

Reference

A. A. Nugraha, N. Iino, K. Yoshii, and M. Hamanaka, “Runtime latency control for streaming guitar tablature transcription through constant-Q buffer truncation,” APSIPA Transactions on Signal and Information Processing, under review.

Supplementary Video

The video plays the same performances through the live system twice:

• with the full 216 ms buffer (top band);
• with the 16 ms buffer (bottom band).

Both bands use the same model file, trained with the 16,384-sample window, on the same laptop GPU. Only the buffer setting differs. It was chosen at start-up, without retraining.

How to read it.

• Time runs from right to left, and the green line on the right is now.
• An outlined ring is a reference note, drawn when it sounds.
• A filled marker is the model’s prediction, drawn when the model outputs it.
• The line between them is that note’s delay.
• Colors mark the result:
  • green, right string and fret;
  • amber, right pitch on another string;
  • red, wrong or missed.

The two GuitarSet clips come from the held-out MT3 test split. The excerpts were chosen for a clear picture. The paper reports accuracy on all 120 test clips at every buffer.