Abstract: This paper addresses identification of nonlinear circuits for power-balanced virtual analog modeling and simulation. The proposed method combines a port-Hamiltonian system formulation with kernel-based methods to retrieve model laws from measurements. This combination allows for the estimated model to retain physical properties that are crucial for the accuracy of simulations, while representing a variety of nonlinear behaviors. As an illustration, the method is used to identify a nonlinear passive peaking EQ.

Abstract: Nonlinear digital circuits and waveshaping are active areas of study, specifically for what concerns numerical and aliasing issues. In the past, an effective method was proposed to discretize nonlinear static functions with reduced aliasing based on the antiderivative of the nonlinear function. Such a method is based on the continuoustime convolution with an FIR antialiasing filter kernel, such as a rectangular kernel. These kernels, however, are far from optimal for the reduction of aliasing. In this paper we introduce the use of arbitrary IIR rational transfer functions that allow a closer approximation of the ideal antialiasing filter, required in the fictitious continuous-time domain before sampling the nonlinear function output. These allow a higher degree of aliasing reduction and can be flexibly adjusted to balance performance and computational cost.

Abstract: The recently proposed antiderivative antialiasing (ADAA) technique for stateful systems involves two key features: 1) replacing a nonlinearity in a physical model or virtual analog simulation with an antialiased nonlinear system involving antiderivatives of the nonlinearity and time delays and 2) introducing a digital filter in cascade with each original delay in the system. Both of these features introduce the same delay, which is compensated by adjusting the sampling period. The result is a simulation with reduced aliasing distortion. In this paper, we study ADAA using equivalent circuits, answering the question: “Which electrical circuit, discretized using the bilinear transform, yields the ADAA system?” This gives us a new way of looking at the stability of ADAA and how introducing extra filtering distorts a system’s response. We focus on the Wave Digital Filter (WDF) version of this technique.

Abstract: In this work, a number of numerical schemes are presented in the context of virtual-analog simulation. The schemes are linearlyimplicit in character, and hence directly solvable without iterative methods. Schemes of increasing order of accuracy are constructed, and convergence and stability conditions are proven formally. The schemes are able to handle stiff problems very efficiently, because of their fast update, and can be run at higher sample rates to reduce aliasing. The cases of the diode clipper and ring modulator are investigated in detail, including several numerical examples.

Abstract: This paper presents an application of the port Hamiltonian formalism to the nonlinear simulation of the OTA-based Korg35 filter circuit and the Moog 4-pole ladder filter circuit. Lyapunov analysis is used with their state-space representations to guarantee zero-input stability over the range of parameters consistent with the actual circuits. A zero-input stable non-iterative discrete-time scheme based on a discrete gradient and a change of state variables is shown along with numerical simulations. Simulations show behavior consistent with the actual operation of the circuits, e.g., self-oscillation, and are found to be stable and have lower computational cost compared to iterative methods.

Abstract: Component-wise circuit modeling, also known as “white-box” modeling, is a well established and much discussed technique in virtual analog modeling. This approach is generally limited in accuracy by lack of access to the exact component values present in a real example of the circuit. In this paper we show how this problem can be addressed by implementing the white-box model in a differentiable form, and allowing approximate component values to be learned from raw input–output audio measured from a real device.

Abstract: Möbius transforms provide for the definition of a family of onestep discretization methods offering a framework for alleviating well-known limitations of common one-step methods, such as the trapezoidal method, at no cost in model compactness or complexity. In this paper, we extend the existing theory around these methods. Here, we show how it can be applied to common frameworks used to structure virtual analog models. Then, we propose practical strategies to tune the transform parameters for best simulation results. Finally, we show how such strategies enable us to formulate much improved non-oversampled virtual analog models for several historical audio circuits.

Abstract: We release Amp-Space, a large-scale dataset of paired audio samples: a source audio signal, and an output signal, the result of a timbre transformation. The types of transformations we study are from blackbox musical tools (amplifiers, stompboxes, studio effects) traditionally used to shape the sound of guitar, bass, or synthesizer sounds. For each sample of transformed audio, the set of parameters used to create it are given. Samples are from both real and simulated devices, the latter allowing for orders of magnitude greater data than found in comparable datasets. We demonstrate potential use cases of this data by (a) pre-training a conditional WaveNet model on synthetic data and show that it reduces the number of samples necessary to digitally reproduce a real musical device, and (b) training a variational autoencoder to shape a continuous space of timbre transformations for creating new sounds through interpolation.

Abstract: Estimating mixtures of damped chirp sinusoids in noise is a problem that affects audio analysis, coding, and synthesis applications. Phase-based non-stationary parameter estimators assume that sinusoids can be resolved in the Fourier transform domain, whereas high-resolution methods estimate superimposed components with accuracy close to the theoretical limits, but only for sinusoids with constant frequencies. We present a new method for estimating the parameters of superimposed damped chirps that has an accuracy competitive with existing non-stationary estimators but also has a high-resolution like subspace techniques. After providing the analytical expression for a Gaussian-windowed damped chirp signal’s Fourier transform, we propose an efficient variational EM algorithm for nonlinear Bayesian regression that jointly estimates the amplitudes, phases, frequencies, chirp rates, and decay rates of multiple non-stationary components that may be obfuscated under the same local maximum in the frequency spectrum. Quantitative results show that the new method not only has an estimation accuracy that is close to the Cramér-Rao bound, but also a high resolution that outperforms the state-of-the-art.

Abstract: Decomposition of sounds into their sinusoidal, transient, and noise components is an active research topic and a widely-used tool in audio processing. Multiple solutions have been proposed in recent years, using time–frequency representations to identify either horizontal and vertical structures or orientations and anisotropy in the spectrogram of the sound. In this paper, we present SiTraNo: an easy-to-use MATLAB application with a graphic user interface for audio decomposition that enables visualization and access to the sinusoidal, transient, and noise classes, individually. This application allows the user to choose between different well-known separation methods to analyze an input sound file, to instantaneously control and remix its spectral components, and to visually check the quality of the separation, before producing the desired output file. The visualization of common artifacts, such as birdies and dropouts, is demonstrated. This application promotes experimenting with the sound decomposition process by observing the effect of variations for each spectral component on the original sound and by comparing different methods against each other, evaluating the separation quality both audibly and visually. SiTraNo and its source code are available on a companion website and repository.

Abstract: Sinusoids are widely used to represent the oscillatory modes of music and speech. The estimation of the sinusoidal parameters directly affects the quality of the representation. A parabolic interpolation of the peaks of the log-magnitude spectrum is commonly used to get a more accurate estimation of the frequencies and the amplitudes of the sinusoids at a relatively low computational cost. Recently, Werner and Germain proposed an improved sinusoidal estimator that performs parabolic interpolation of the peaks of a power-scaled magnitude spectrum. For each analysis window type and size, a power-scaling factor p is pre-calculated via a computationally demanding heuristic. Consequently, the powerscaling estimation method is currently constrained to a few tabulated power-scaling factors for pre-selected window sizes, limiting its practical applications. In this article, we propose a method to obtain the power-scaling factor p for any window size from the tabulated values. Additionally, we investigate the impact of zeropadding on the estimation accuracy of the power-scaled sinusoidal parameter estimator.

Abstract: Fractional order filters have been studied since a long time, along with their applications to many areas of physics and engineering. In particular, several solutions have been proposed in order to approximate their frequency response with that of an ordinary filter. In this paper, we tackle this problem with a new approach: we solve analytically a simplified version of the problem and we find the optimal placement of poles and zeros, giving a mathematical proof and an error estimate. This solution shows improved performance compared to the current state of the art and is suitable for real-time parametric control.

Abstract: We propose a new analog filter discretization method that is useful for discretizing systems with features near or above the Nyquist limit. A conformal mapping approach is taken, and we introduce the peaking conformal map and shelving conformal map. The proposed method provides a close match to the original analog frequency response below half the sampling rate and is parameterizable, order preserving, and agnostic to the original filter’s order or type. The proposed method should have applications to discretizing filters that have time-varying parameters or need to be implemented across many different sampling rates.

Abstract: This paper introduces hexaphonic distortion as a way of achieving harmonically rich guitar distortion while minimizing intermodulation products regardless of playing style. The simulated hexaphonic distortion effect described in this paper attempts to reproduce the characteristics of hexaphonic distortion for use with ordinary electric guitars with mono pickups. The proposed approach uses a parallel comb filter structure that separates a mono guitar signal into its harmonic components. This simulates the six individual string signals obtained from a hexaphonic pickup. Each of the signals are then individually distorted with oversampling used to avoid aliasing artifacts. Starting with the baseline of the distorted mono signal, the simulated distortion produces fewer intermodulation products with a result approaching that of hexaphonic distortion.

Abstract: Live music performances and music production often involve the manipulation of several parameters during sound generation, processing, and mixing. In hardware layouts, those parameters are usually controlled using knobs, sliders and buttons. When these layouts are virtualized, the use of physical (e.g. MIDI) controllers can make interaction easier and reduce the cognitive load associated to sound manipulation. The addition of haptic feedback can further improve such interaction by facilitating the detection of the nature (continuous / discrete) and value of a parameter. To this end, we have realized an endless-knob controller prototype with programmable resistance to rotation, able to render various haptic effects. Ten subjects assessed the effectiveness of the provided haptic feedback in a target-matching task where either visual-only or visual-haptic feedback was provided; the experiment reported significantly lower errors in presence of haptic feedback. Finally, the knob was configured as a multi-parametric controller for a real-time audio effect software written in Python, simulating the voltage-controlled filter aboard the EMS VCS3. The integration of the sound algorithm and the haptic knob is discussed, together with various haptic feedback effects in response to control actions.

Abstract: A central challenge for a cappella singers is to adjust their intonation and to stay in tune relative to their fellow singers. During editing of a cappella recordings, one may want to adjust local intonation of individual singers or account for global intonation drifts over time. This requires applying a time-varying pitch-shift to the audio recording, which we refer to as adaptive pitch-shifting. In this context, existing (semi-)automatic approaches are either laborintensive or face technical and musical limitations. In this work, we present automatic methods and tools for adaptive pitch-shifting with applications to intonation adjustment in a cappella recordings. To this end, we show how to incorporate time-varying information into existing pitch-shifting algorithms that are based on resampling and time-scale modification (TSM). Furthermore, we release an open-source Python toolbox, which includes a variety of TSM algorithms and an implementation of our method. Finally, we show the potential of our tools by two case studies on global and local intonation adjustment in a cappella recordings using a publicly available multitrack dataset of amateur choral singing.

Abstract: A filtering algorithm for generating subtle random variations in sampled sounds is proposed. Using only one recording for impact sound effects or drum machine sounds results in unrealistic repetitiveness during consecutive playback. This paper studies spectral variations in repeated knocking sounds and in three drum sounds: a hihat, a snare, and a tomtom. The proposed method uses a short pseudo-random velvet-noise filter and a low-shelf filter to produce timbral variations targeted at appropriate spectral regions, yielding potentially an endless number of new realistic versions of a single percussive sampled sound. The realism of the resulting processed sounds is studied in a listening test. The results show that the sound quality obtained with the proposed algorithm is at least as good as that of a previous method while using 77% fewer computational operations. The algorithm is widely applicable to computer-generated music and game audio.

Abstract: Sustained contact interactions like scraping and rolling produce a wide variety of sounds. Previous studies have explored ways to synthesize these sounds efficiently and intuitively but could not fully mimic the rich structure of real instances of these sounds. We present a novel source-filter model for realistic synthesis of scraping and rolling sounds with physically and perceptually relevant controllable parameters constrained by principles of mechanics. Key features of our model include non-linearities to constrain the contact force, naturalistic normal force variation for different motions, and a method for morphing impulse responses within a material to achieve location-dependence. Perceptual experiments show that the presented model is able to synthesize realistic scraping and rolling sounds while conveying physical information similar to that in recorded sounds.

Abstract: For physical modelling sound synthesis, many techniques are available; time-stepping methods (e.g., finite-difference time-domain (FDTD) methods) have an advantage of flexibility and generality in terms of the type of systems they can model. These methods do, however, lack the capability of easily handling smooth parameter changes while retaining optimal simulation quality and stability, something other techniques are better suited for. In this paper, we propose an efficient method to smoothly add and remove grid points from a FDTD simulation under sub-audio rate parameter variations. This allows for dynamic parameter changes in physical models of musical instruments. An instrument such as the trombone can now be modelled using FDTD methods, as well as physically impossible instruments where parameters such as e.g. material density or its geometry can be made time-varying. Results show that the method does not produce (visible) artifacts and stability analysis is ongoing.

Abstract: In this paper, a complete simulation of a trombone using finitedifference time-domain (FDTD) methods is proposed. In particular, we propose the use of a novel method to dynamically vary the number of grid points associated to the FDTD method, to simulate the fact that the physical dimension of the trombone’s resonator dynamically varies over time. We describe the different elements of the model and present the results of a real-time simulation.

Abstract: Air absorption effects lead to significant attenuation in high frequencies over long distances and this is critical to model in wide-band virtual acoustic simulations. Air absorption is commonly modelled using filter banks applied to an impulse response or to individual impulse events (rays or image sources) arriving at a receiver. Such filter banks require non-trivial fitting to air absorption attenuation curves, as a function of time or distance, in the case of IIR approximations, or may suffer from overlap-add artefacts in the case of FIR approximations. In this study, a filter method is presented which avoids the aforementioned issues. The proposed approach relies on a time-varying diffusion kernel that is found in an approximate Green’s function solution to Stokes’ equation in free space. This kernel acts as a low-pass filter that is parametrised by physical constants, and can be applied to an impulse response using time-varying convolution. Numerical examples are presented demonstrating the utility of this approach for adding air absorption effects to room impulse responses simulated using geometrical acoustics or wave-based methods.

Abstract: Physical modelling sound synthesis is a powerful method for constructing virtual instruments aiming to mimic the sound of realworld counterparts, while allowing for the possibility of engaging with these instruments in ways which may be impossible in person. Such a case is explored in this paper: particularly the simulation of a friction drum inspired instrument. It is an instrument played by causing the membrane of a drum head to vibrate via friction. This involves rubbing the membrane via a stick or a cord attached to its center, with the induced vibrations being transferred to the air inside a sound box. This paper describes the development of a real-time audio application which models such an instrument as a bowed membrane connected to an acoustic tube. This is done by means of a numerical simulation using finite-difference time-domain (FDTD) methods in which the excitation, whose position is free to change in real-time, is modelled by a highly non-linear elasto-plastic friction model. Additionally, the virtual instrument allows for dynamically modifying physical parameters of the model, thereby allowing the user to generate new and interesting sounds that go beyond a realworld friction drum.

Abstract: The article performs a generic comparison of integrator- and differentiator based continuous-time systems as well as their discretetime models, aiming to answer the reoccurring question in the music DSP community of whether there are any benefits in using differentiators instead of conventionally employed integrators. It is found that both kinds of models are practically equivalent, but there are certain reservations about differentiator based models.

Abstract: This paper presents a discrete-time model of the spherical harmonics expansion describing a sound field. The so-called radial functions are realized as digital filters, which characterize the spatial impulse responses of the individual harmonic orders. The filter coefficients are derived from the analytical expressions of the timedomain radial functions, which have a finite extent in time. Due to the varying degrees of discontinuities occurring at their edges, a time-domain sampling of the radial functions gives rise to aliasing. In order to reduce the aliasing distortion, the discontinuities are replaced with the higher-order anti-derivatives of a band-limited step function. The improved spectral accuracy is demonstrated by numerical evaluation. The proposed discrete-time sound field model is applicable in broadband applications such as spatial sound reproduction and active noise control.

Abstract: The distributed nature of coupling in helical springs presents specific challenges in obtaining efficient computational structures for accurate spring reverb simulation. For direct simulation approaches, such as finite-difference methods, this is typically manifested in significant numerical dispersion within the hearing range. Building on a recent study of a simpler spring model, this paper presents an alternative discretisation approach that employs higher-order spatial approximations and applies centred stencils at the boundaries to address the underlying linear-system eigenvalue problem. Temporal discretisation is then applied to the resultant uncoupled mode system, rendering an efficient and flexible modal reverb structure. Through dispersion analysis it is shown that numerical dispersion errors can be kept extremely small across the hearing range for a relatively low number of system nodes. Analysis of an impulse response simulated using model parameters calculated from a measured spring geometry confirms that the model captures an enhanced set of spring characteristics.

Abstract: A method is proposed to encode the acoustic scattering of objects for virtual acoustic applications through a multiple-input and multiple-output framework. The scattering is encoded as a matrix in the spherical harmonic domain, and can be re-used and manipulated (rotated, scaled and translated) to synthesize various sound scenes. The proposed method is applied and validated using Boundary Element Method simulations which shows accurate results between references and synthesis. The method is compatible with existing frameworks such as Ambisonics and image source methods.

Abstract: A perceptual study revealing a novel connection between modal properties of feedback delay networks (FDNs) and colorless reverberation is presented. The coloration of the reverberation tail is quantified by the modal excitation distribution derived from the modal decomposition of the FDN. A homogeneously decaying allpass FDN is designed to be colorless such that the corresponding narrow modal excitation distribution leads to a high perceived modal density. Synthetic modal excitation distributions are generated to match modal excitations of FDNs. Three listening tests were conducted to demonstrate the correlation between the modal excitation distribution and the perceived degree of coloration. A fourth test shows a significant reduction of coloration by the colorless FDN compared to other FDN designs. The novel connection of modal excitation, allpass FDNs, and perceived coloration presents a beneficial design criterion for colorless artificial reverberation.

Abstract: Decomposing a sound-field into its individual components and respective parameters can represent a convenient first-step towards offering the user an intuitive means of controlling spatial audio effects and sound-field modification tools. The majority of such tools available today, however, are instead limited to linear combinations of signals or employ a basic single-source parametric model. Therefore, the purpose of this paper is to present a parametric framework, which seeks to overcome these limitations by first dividing the sound-field into its multi-source and ambient components based on estimated spatial parameters. It is then demonstrated that by manipulating the spatial parameters prior to reproducing the scene, a number of sound-field modification and spatial audio effects may be realised; including: directional warping, listener translation, sound source tracking, spatial editing workflows and spatial side-chaining. Many of the effects described have also been implemented as real-time audio plug-ins, in order to demonstrate how a user may interact with such tools in practice.

Abstract: A novel algorithm for fast convolution reverberation is proposed. The convolution is implemented as partitioned convolution in the frequency domain. We show that computational cost can be reduced when multiplying the spectra of the impulse response with the spectra of the input signal by using only a fraction of the bins of the original spectra and by discarding phase information. Reordering the bins of the spectra allows to avoid overhead incurred by randomly accessing bins in the spectrum. The proposed algorithm is considerably faster than conventional partitioned convolution and perceptual convolution, where bins with low amplitudes are discarded. Speed increases depend on the impulse response used. For an impulse response of around 3 s length at 48 kHz sampling rate execution took only about 40 % of the time necessary for conventional partitioned convolution and 61 % of the time needed for perceptual convolution. A listening test showed that there is only a very slight degradation in quality, which can probably be neglected for implementations where speed is crucial. Sound samples are provided.

Abstract: We release synth1B1, a multi-modal audio corpus consisting of 1 billion 4-second synthesized sounds, paired with the synthesis parameters used to generate them. The dataset is 100x larger than any audio dataset in the literature. We also introduce torchsynth, an open source modular synthesizer that generates the synth1B1 samples on-the-fly at 16200x faster than real-time (714MHz) on a single GPU. Finally, we release two new audio datasets: FM synth timbre and subtractive synth pitch. Using these datasets, we demonstrate new rank-based evaluation criteria for existing audio representations. Finally, we propose a novel approach to synthesizer hyperparameter optimization.

Abstract: This paper proposes a novel way of doing audio synthesis at the waveform level using Transformer architectures. We propose a deep neural network for generating waveforms, similar to wavenet . This is fully probabilistic, auto-regressive, and causal, i.e. each sample generated depends on only the previously observed samples. Our approach outperforms a widely used wavenet architecture by up to 9% on a similar dataset for predicting the next step. Using the attention mechanism, we enable the architecture to learn which audio samples are important for the prediction of the future sample. We show how causal transformer generative models can be used for raw waveform synthesis. We also show that this performance can be improved by another 2% by conditioning samples over a wider context. The flexibility of the current model to synthesize audio from latent representations suggests a large number of potential applications. The novel approach of using generative transformer architectures for raw audio synthesis is, however, still far away from generating any meaningful music similar to wavenet, without using latent codes/meta-data to aid the generation process.

Abstract: This paper presents a technique addressing signal discontinuity and concatenation artefacts in real-time granular processing with rectangular windowing. By combining zero-crossing synchronicity, first-order derivative analysis, and Lagrange polynomials, we can generate streams of uncorrelated and non-overlapping sonic fragments with minimal low-order derivatives discontinuities. The resulting open-source algorithm, implemented in the Faust language, provides a versatile real-time software for dynamical looping, wavetable oscillation, and granulation with reduced artefacts due to rectangular windowing and no artefacts from overlap-add-to-one techniques commonly deployed in granular processing.

Abstract: Contemporary musicians commonly face the challenge of finding new, characteristic sounds that can make their compositions more distinct. They often resort to computers and algorithms, which can significantly aid in creative processes by generating unexpected material in controlled probabilistic processes. In particular, algorithms that present emergent behaviors, like genetic algorithms and cellular automata, have fostered a broad diversity of musical explorations. This article proposes an original technique for the computer-assisted creation and manipulation of sound textures. The technique uses Probabilistic Cellular Automata, which are yet seldom explored in the music domain, to blend two audio tracks into a third, different one. The proposed blending process works by dividing the source tracks into frequency bands and then associating each of the automaton’s cell to a frequency band. Only one source, chosen by the cell’s state, is active within each band. The resulting track has a non-repeating textural pattern that follows the changes in the Cellular Automata. This blending process allows the musician to choose the original material and the blend granularity, significantly changing the resulting blends. We demonstrate how to use the proposed blending process in sound design and its application in experimental and popular music.

Abstract: In this paper we describe similarity graphs computed from timefrequency analysis as a guide for audio playback, with the aim of extending the content of fixed recordings in creative applications. We explain the creation of the graph from the distance between spectral frames, as well as several features computed from the graph, such as methods for onset detection, beat detection, and cluster analysis. Several playback algorithms can be devised based on conditional pruning of the graph using these methods. We describe examples for looping, granulation, and automatic montage.

Abstract: In recent years, a range of topological methods have emerged for processing digital signals. In this paper we show how the construction of topological filters via sheaves can be used to topologize existing sound synthesis methods. I illustrate this process on two classes of synthesis approaches: (1) based on linear-time invariant digital filters and (2) based on oscillators defined on a circle. We use the computationally-friendly approach to modeling topologies via a simplicial complex, and we attach our classical synthesis methods to them via sheaves. In particular, we explore examples of simplicial topologies that mimic sampled lines and loops. Over these spaces we realize concrete examples of simple discrete harmonic oscillators (resonant filters), and simple comb filter based algorithms (such as Karplus-Strong) as well as frequency modulation.

Abstract: Matching pursuit (MP) algorithms are widely used greedy methods to find K-sparse signal approximations in redundant dictionaries. We present an acceleration technique and an implementation of the matching pursuit algorithm acting on a multi-Gabor dictionary, i.e., a concatenation of several Gabor-type time-frequency dictionaries, consisting of translations and modulations of possibly different windows, time- and frequency-shift parameters. The proposed acceleration is based on pre-computing and thresholding inner products between atoms and on updating the residual directly in the coefficient domain, i.e., without the round-trip to the signal domain. Previously, coefficient-domain residual updates have been dismissed as having prohibitive memory requirements. By introducing an approximate update step, we can overcome this restriction and greatly improve the performance of matching pursuit at a modest cost in terms of approximation quality per selected atom. An implementation in C with Matlab and GNU Octave interfaces is available, outperforming the standard Matching Pursuit Toolkit (MPTK) by a factor of 3.5 to 70 in the tested conditions. Additionally, we provide experimental results illustrating the convergence of the implementation.

Abstract: Onset detectors are used to recognize the beginning of musical events in audio signals. Manual parameter tuning for onset detectors is a time consuming task, while existing automated approaches often maximize only a single performance metric. These automated approaches cannot be used to optimize detector algorithms for complex scenarios, such as real-time onset detection where an optimization process must consider both detection accuracy and latency. For this reason, a flexible optimization algorithm should account for more than one performance metric in a multiobjective manner. This paper presents a generalized procedure for automated optimization of parametric onset detectors. Our procedure employs a bio-inspired evolutionary computation algorithm to replace manual parameter tuning, followed by the computation of the Pareto frontier for multi-objective optimization. The proposed approach was evaluated on all the onset detection methods of the Aubio library, using a dataset of monophonic acoustic guitar recordings. Results show that the proposed solution is effective in reducing the human effort required in the optimization process: it replaced more than two days of manual parameter tuning with 13 hours and 34 minutes of automated computation. Moreover, the resulting performance was comparable to that obtained by manual optimization.

Abstract: Deep neural networks have been recently applied to the task of automatic synthesizer programming, i.e., finding optimal values of sound synthesis parameters in order to reproduce a given input sound. This paper focuses on generative models, which can infer parameters as well as generate new sets of parameters or perform smooth morphing effects between sounds. We introduce new models to ensure scalability and to increase performance by using heterogeneous representations of parameters as numerical and categorical random variables. Moreover, a spectral variational autoencoder architecture with multi-channel input is proposed in order to improve inference of parameters related to the pitch and intensity of input sounds. Model performance was evaluated according to several criteria such as parameters estimation error and audio reconstruction accuracy. Training and evaluation were performed using a 30k presets dataset which is published with this paper. They demonstrate significant improvements in terms of parameter inference and audio accuracy and show that presented models can be used with subsets or full sets of synthesizer parameters.

Abstract: Virtual analog (VA) modeling using neural networks (NNs) has great potential for rapidly producing high-fidelity models. Recurrent neural networks (RNNs) are especially appealing for VA due to their connection with discrete nodal analysis. Furthermore, VA models based on NNs can be trained efficiently by directly exposing them to the circuit states in a gray-box fashion. However, exposure to ground truth information during training can leave the models susceptible to error accumulation in a free-running mode, also known as “exposure bias” in machine learning literature. This paper presents a unified framework for treating the previously proposed state trajectory network (STN) and gated recurrent unit (GRU) networks as special cases of discrete nodal analysis. We propose a novel circuit state-matching mechanism for the GRU and experimentally compare the previously mentioned networks for their performance in state matching, during training, and in exposure bias, during inference. Experimental results from modeling a diode clipper show that all the tested models exhibit some exposure bias, which can be mitigated by truncated backpropagation through time. Furthermore, the proposed state matching mechanism improves the GRU modeling performance of an overdrive pedal and a phaser pedal, especially in the presence of external modulation, apparent in a phaser circuit.

Abstract: Piano transcription is a classic problem in music information retrieval. More and more transcription methods based on deep learning have been proposed in recent years. In 2019, Google Brain published a larger piano transcription dataset, MAESTRO. On this dataset, Onsets and Frames transcription approach proposed by Hawthorne achieved a stunning onset F1 score of 94.73%. Unlike the annotation method of Onsets and Frames, Transition-aware model presented in this paper annotates the attack process of piano signals called atack transition in multiple frames, instead of only marking the onset frame. In this way, the piano signals around onset time are taken into account, enabling the detection of piano onset more stable and robust. Transition-aware achieves a higher transcription F1 score than Onsets and Frames on MAESTRO dataset and MAPS dataset, reducing many extra note detection errors. This indicates that Transition-aware approach has better generalization ability on different datasets.

Abstract: It is difficult to adjust the parameters of a complex synthesizer to create the desired sound. As such, sound matching, the estimation of synthesis parameters that can replicate a certain sound, is a task that has often been researched, utilizing optimization methods such as genetic algorithm (GA). In this paper, we introduce a novelty-based objective for GA-based sound matching. Our contribution is two-fold. First, we show that the novelty objective is able to improve the quality of sound matching by maintaining phenotypic diversity in the population. Second, we introduce a quality diversity approach to the problem of sound matching, aiming to find a diverse set of matching sounds. We show that the novelty objective is effective in producing high-performing solutions that are diverse in terms of specified audio features. This approach allows for a new way of discovering sounds and exploring the capabilities of a synthesizer.

Abstract: Piano transcription is a fundamental problem in the field of music information retrieval. At present, a large number of transcriptional studies are mainly based on audio or video, yet there is a small number of discussion based on audio-visual fusion. In this paper, a piano transcription model based on strategy fusion is proposed, in which the transcription results of the video model are used to assist audio transcription. Due to the lack of datasets currently used for audio-visual fusion, the OMAPS data set is proposed in this paper. Meanwhile, our strategy fusion model achieves a 92.07% F1 score on OMAPS dataset. The transcription model based on feature fusion is also compared with the one based on strategy fusion. The experiment results show that the transcription model based on strategy fusion achieves better results than the one based on feature fusion.