Melody Extraction From Polyphonic Audio

A polyphonic recording contains several simultaneous notes and sound sources. Melody extraction asks for the predominant sequence of pitches over time, often with a voiced-or-unvoiced decision at each moment. Salamon and Gómez’s research on pitch-contour characteristics is one influential approach: candidate frequencies and their harmonics are analyzed, then plausible contours are selected as a melody line. Other systems use learned representations, but the output remains an estimate of a particular target definition, not a transcription of every instrument. The target itself needs care. In a song, the lead singer may carry the melody during a verse, but an instrument may take over during a solo. A model trained to follow one kind of lead can make a different choice than a musician. The strongest harmonic peak may belong to accompaniment, while the true fundamental is weak. Octave errors can make a contour sound superficially related but place notes at the wrong register. Noise and reverb blur pitch evidence; spoken or unpitched sounds may have no meaningful melody at all. Evaluate both pitch and voicing. Frame-level accuracy against an annotated reference can show whether estimated notes are close, but summarize octave errors and silence mistakes separately. Check timing around phrase boundaries. A model can produce a smooth line by inventing pitches through rests, which is not faithful. If the output will become sheet music, additional work is needed to convert continuous pitch into note events, rhythm, key and expression. A melody curve alone does not provide a full score or isolated singer audio. Use representative genres and instruments when choosing a system. A benchmark on clear vocal pop may not transfer to choral music or heavily distorted guitar. Preserve the source audio and let users correct sections. Melody extraction is most helpful when it offers a draft contour that accelerates human analysis without concealing uncertainty about source choice or pitch.