Score-Informed Music Interpretation Analysis and Transcription

This page serves as companion website for the MMSP 2022 paper:

F. Simonetta, F. Avanzini, and S. Ntalampiras “Score-Informed Music Interpretation Analysis and Transcription,” Signal Processing, Elsevier, 2022, submitted

Abstract

In this paper, a novel mathematical and conceptual framework for Automatic Music Transcription (AMT) is proposed with the purpose of integrating acoustical context factors in the analysis of the audio signal. The proposed framework is specifically designed to resynthesize the output of AMT models without affecting the artistic content of the performance. First, a mathematical formalization describing acoustics- specific transcription and resynthesis is proposed. Then, a methodological approach for the analysis of music performances while considering the contextual acoustical factors is shown. To prove that the acoustical context can be used to reduce the analysis error, we propose a novel score-informed AMT method able to predict note velocities and pedaling levels. We test different approaches to integrate the acoustical context information in the AMT process and show that the proposed methodology generally outperforms the usual AMT pipeline which does not consider specificities of the instrument and of the acoustic environment. Overall, the proposed mathematical formalization is shown to be a promising approach to tackle realistic automated music resynthesis.

The code is available at https://github.com/LIMUNIMI/ContextAwareAMT