Abstract: There are several algorithms and approaches to Room Impulse Response (RIR) estimation. To the best of the author’s knowledge, there is no documentation of accuracy, speed, or even the feasibility of using signed distance functions (SDFs) in combination with sphere tracing for this task. A proof of concept with a focus on real-time performance is presented here, which still lacks many features such as frequency-dependent absorption and scattering coefficients, arbitrary source and receiver directives etc. The results are then compared to real room impulse responses and to a different simulation algorithm. Also, the rather special merits of such an approach, such as 4D reverberation and simple rounding of geometry, are briefly discussed and presented.

Abstract: Virtual reality applications require all kinds of methods to create plausible virtual acoustics environments to enhance the user experience. Here, we present an acoustic paintbrush method that modifies the timbre of a simple room acoustics simulation with the timbre of a measured room response while aiming to preserve the spatial aspects of the simulated room. In other words, the method only applies the measured spectral coloration and alters the simulated and temporal distribution of early reflections as little as possible. Three variations of the acoustic paintbrush method are validated with a listening test. The results indicate that the method works reasonably well. The paintbrushed room acoustic simulations were perceived to become closer to the measured room acoustics than the source simulation. However, the limits of the perceived effect varied depending on the input signal and the simulated and recorded responses. This warrants for further perceptual testing.

Abstract: Reverberation is one of the most important effects used in audio production. Although nowadays numerous real-time implementations of artificial reverberation algorithms are available, many of them depend on a database of recorded or pre-synthesized room impulse responses, which are convolved with the input signal. Implementations that use an algorithmic approach are more flexible but do not let the users have full control over the produced sound, allowing only a few selected parameters to be altered. The realtime implementation of an artificial reverberation synthesizer presented in this study introduces an audio plugin based on a feedback delay network (FDN), which lets the user have full and detailed insight into the produced reverb. It allows for control of reverberation time in ten octave bands, simultaneously allowing adjusting the feedback matrix type and delay-line lengths. The proposed plugin explores various FDN setups, showing that the lowest useful order for high-quality sound is 16, and that in the case of a Householder matrix the implementation strongly affects the resulting reverberation. Experimenting with delay lengths and distribution demonstrates that choosing too wide or too narrow a length range is disadvantageous to the synthesized sound quality. The study also discusses CPU usage for different FDN orders and plugin states.

Abstract: Reverb provides psychoacoustic cues that convey information concerning relative locations within an acoustical space. The need arises often in audio production to impart an acoustic context on an audio track that resembles a reference track. One tool for making audio tracks appear to be recorded in the same space is by applying reverb to a dry track that is similar to the reverb in a wet one. This paper presents a model for the task of “reverb matching,” where we attempt to automatically add artificial reverb to a track, making it sound like it was recorded in the same space as a reference track. We propose a model architecture for performing reverb matching and provide subjective experimental results suggesting that the reverb matching model can perform as well as a human. We also provide open source software for generating training data using an arbitrary Virtual Studio Technology plug-in.

Abstract: Various audio signal processing applications, such as source separation and dereverberation, require an accurate mathematical modeling of the input audio data. In the literature, many works have focused on source signal modeling, while the reverberation model is often kept very simplistic. This paper aims to investigate a stochastic room impulse response model presented in a previous article: this model is first adapted to discrete time, then we propose a parametric estimation algorithm, that we evaluate experimentally. Our results show that this algorithm is able to efficiently estimate the model parameters, in various experimental settings (various signal-to-noise ratios and absorption coefficients of the room walls).

Abstract: In the design of real-time spring reverberation algorithms, a modal architecture offers several advantages, including computational efficiency and parametric control flexibility. Due to the complex, highly dispersive behavior of helical springs, computing physically accurate parameters for such a model presents specific challenges. In this paper these are addressed by applying an implicit higher-order-in-space finite difference scheme to a two-variable model of helical spring dynamics. A novel numerical boundary treatment is presented, which utilises multiple centered boundary expressions of different stencil width. The resulting scheme is unconditionally stable, and as such allows adjusting the numerical parameters independently of each other and of the physical parameters. The dispersion relation of the scheme is shown to be accurate in the audio range only for very high orders of accuracy in combination with a small temporal and spatial step. The frequency, amplitude, and decay rate of the system modes are extracted from a diagonalised form of this numerical model. After removing all modes with frequencies outside the audio range and applying a modal amplitude correction to compensate for omitting the magnetic transducers, a light-weight modal reverb algorithm is obtained. Comparison with a measured impulse response shows a reasonably good match across a wide frequency range in terms of echo density, decay characteristics, and diffusive nature.

Abstract: A novel algorithm for fast convolution reverberation is proposed. The convolution is implemented as partitioned convolution in the frequency domain. We show that computational cost can be reduced when multiplying the spectra of the impulse response with the spectra of the input signal by using only a fraction of the bins of the original spectra and by discarding phase information. Reordering the bins of the spectra allows to avoid overhead incurred by randomly accessing bins in the spectrum. The proposed algorithm is considerably faster than conventional partitioned convolution and perceptual convolution, where bins with low amplitudes are discarded. Speed increases depend on the impulse response used. For an impulse response of around 3 s length at 48 kHz sampling rate execution took only about 40 % of the time necessary for conventional partitioned convolution and 61 % of the time needed for perceptual convolution. A listening test showed that there is only a very slight degradation in quality, which can probably be neglected for implementations where speed is crucial. Sound samples are provided.

Abstract: The Red Llama guitar overdrive effect pedal differs from most other overdrive effects because it utilizes CMOS inverters, formed by two metal-oxide-semiconductor field-effect transistors (MOSFETs), instead of a combination of operational amplifiers and diodes to obtain nonlinear distortion. This makes it an interesting subject for virtual analog modeling, obviously requiring a suitable model for the CMOS inverters. Therefore, in this paper, we extend a well-known model for MOSFETs by a straight-forward heuristic approach to achieve a good match between the model and measurement data obtained for the individual MOSFETs. This allows a faithful digital simulation of the Red Llama.

Abstract: A major problem in the emulation of discrete-time nonlinear systems, such as those encountered in Virtual Analog modeling, is aliasing distortion. A trivial approach to reduce aliasing is oversampling. However, this solution may be too computationally demanding for real-time applications. More advanced techniques to suppress aliased components are arbitrary-order Antiderivative Antialiasing (ADAA) methods that approximate the reference nonlinear function using a combination of its antiderivatives of different orders. While in its original formulation it is applied only to memoryless systems, recently, the applicability of first-order ADAA has been extended to stateful systems employing their statespace description. This paper presents an alternative formulation that successfully applies arbitrary-order ADAA methods to Wave Digital Filter models of dynamic circuits with one nonlinear element. It is shown that the proposed approach allows us to design ADAA models of the nonlinear elements in a fully local and modular fashion, independently of the considered reference circuit. Further peculiar features of the proposed approach, along with two examples of applications, are discussed.

Abstract: We propose a new style of continuous-time filter design composed of a cascade of 2nd-order state variable filters (SVFs) and a global feedback path. This family of filters is parameterized by the SVF cutoff frequencies and resonances, as well as the global feedback amount. For the case of two identical SVFs in cascade and a specific value of the SVF resonance, the proposed design reduces to the well-known Moog ladder filter. For another resonance value, it approximates the Octave CAT filter. The resonance parameter can be used to create new filters as well. We study the pole loci and transfer functions of the SVF building block and entire filter. We focus in particular on the effect of the proposed parameterization on important aspects of the filter’s response, including the passband gain and cutoff frequency error. We also present the first in-depth study of the Octave CAT filter circuit.

Abstract: This paper is concerned with the conception of methods tailored for the numerical simulation of power-balanced systems that are well-posed but implicitly described. The motivation is threefold: some electronic components (such as the ideal diode) can only be implicitly described, arbitrary connection of components can lead to implicit topological constraints, finally stable discretization schemes also lead to implicit algebraic equations. In this paper we start from the representation of circuits using a power-balanced Kirchhoff-Dirac structure, electronic components are described by a local state that is observed through a pair of power-conjugated algebro-differential operators (V, I) to yield the branch voltages and currents, the arc length is used to parametrize switching and non-Lipschitz components, and a power balanced functional time-discretization is proposed. Finally, the method is illustrated on two simple but non-trivial examples.

Abstract: A virtual bass system creates an impression of bass perception in sound systems with weak low-frequency reproduction, which is typical of small loudspeakers. Virtual bass systems extend the bandwidth of the low-frequency audio content using either a nonlinear function or a phase vocoder, and add the processed signal to the reproduced sound. Hybrid systems separate transients and steady-state sounds, which are processed separately. It is still challenging to reach a good sound quality using a virtual bass system. This paper proposes a novel method, which separates the tonal, transient, and noisy parts of the audio signal in a fuzzy way, and then processes only the transients and tones. Those upper harmonics, which can be detected above the cutoff frequency, are boosted using timbre-matched weights, but missing upper harmonics are generated to assist the missing fundamental phenomenon. Listening test results show that the proposed algorithm outperforms selected previous methods in terms of perceived bass sound quality. The proposed method can enhance the bass sound perception of small loudspeakers, such as those used in laptop computers and mobile devices.

Abstract: Biquad filters are a common tool for filter design. In this writing, we develop two structures for creating biquad filters with nonlinear elements. We provide conditions for the guaranteed stability of the nonlinear filters, and derive expressions for instantaneous pole analysis. Finally, we examine example filters built with these nonlinear structures, and show how the first nonlinear structure can be used in the context of analog modelling.

Abstract: Peak and shelving filters are parametric infinite impulse response filters which are used for amplifying or attenuating a certain frequency band. Shelving filters are parametrized by their cut-off frequency and gain, and peak filters by center frequency, bandwidth and gain. Such filters can be cascaded in order to perform audio processing tasks like equalization, spectral shaping and modelling of complex transfer functions. Such a filter cascade allows independent optimization of the mentioned parameters of each filter. For this purpose, a novel approach is proposed for deriving the necessary local gradients with respect to the control parameters and for applying the instantaneous backpropagation algorithm to deduce the gradient flow through a cascaded structure. Additionally, the performance of such a filter cascade adapted with the proposed method, is exhibited for head-related transfer function modelling, as an example application.

Abstract: A mechanical system is said to be bistable when its moving parts can rest at two equilibrium positions. The aim of this work is to model the vibration behaviour of a bistable system and use it to create a sound effect, taking advantage of the nonlinearities that characterize such systems. The velocity signal of the bistable system excited by an audio signal is the output of the digital effect. The latter is coded in C++ language and compiled into VST3 format that can be run as an audio plugin within most of the commercial digital audio workstation software in the market and as a standalone application. A Web Audio API demonstration is also available online as a support material.

Abstract: The paper presents a unified, flexible framework for the tasks of audio inpainting, declipping, and dequantization. The concept is further extended to cover analogous degradation models in a transformed domain, e.g. quantization of the signal’s time-frequency coefficients. The task of reconstructing an audio signal from degraded observations in two different domains is formulated as an inverse problem, and several algorithmic solutions are developed. The viability of the presented concept is demonstrated on an example where audio reconstruction from partial and quantized observations of both the time-domain signal and its time-frequency coefficients is carried out.

Abstract: Modern commodity GPUs offer high numerical throughput per unit of cost, but often sit idle during audio workstation tasks. Various researches in the field have shown that GPUs excel at tasks such as Finite-Difference Time-Domain simulation and wavefield synthesis. Concrete implementations of several such projects are available for use. Benchmarks and use cases generally concentrate on running one project on a GPU. Running multiple such projects simultaneously is less common, and reduces throughput. In this work we list some concerns when running multiple heterogeneous tasks on the GPU. We apply optimization strategies detailed in developer documentation and commercial CUDA literature, and show results through the lens of real-time audio tasks. We benchmark the cases of (i) a homogeneous effect chain made of previously separate effects, and (ii) a synthesizer with distinct, parallelizable sound generators.

Abstract: This paper explores Frequency Modulation (FM) for use in music synthesis. We take an in-depth look at Linear FM, LinearThrough-Zero FM, Phase Modulation (PM) and Exponential FM, and discuss their pros and cons for sound synthesis in a digital system. In the process we derive some useful formulas and discuss their implementation details. In particular we derive analytic expressions for DC correcting Exponential FM, and make it match the modulation depth of Linear FM. Finally, we review practical antialiasing solutions.

Abstract: We present a method for accurately synthesizing the acoustic response of a water bottle using modal signal processing. We start with extensive measurements of two water bottles with considerations for how the level of water inside the bottles, the area covered by stickers attached to the exterior of the bottles, and the method of striking the bottles affect their sound. We perform modal analysis of these measurements and implement a real-time modal water bottle synthesizer.

Abstract: This paper presents a system for morphing between audio recordings in a continuous parameter space. The proposed approach combines matrix decompositions used for audio source separation with displacement interpolation enabled by 1D optimal transport. By interpolating the spectral components obtained using nonnegative matrix factorization of the source and target signals, the system allows varying the timbre of a sound in real time, while maintaining its temporal structure. Using harmonic / percussive source separation as a pre-processing step, the system affords more detailed control of the interpolation in perceptually meaningful dimensions.

Abstract: This paper presents a novel method of analysing drum sounds, demonstrating that this can form the basis of a highly realistic synthesis technique for real-time use. The synthesis method can be viewed as an extension of IFFT synthesis; here we exploit the fact that audio signals can be recovered from solely the real component of their discrete Fourier transform (RDFT). All characteristics of an entire drum sample can therefore be conveniently encoded in a single, real-valued, frequency domain signal. These signals are interpreted, incorporating the physics of the instrument, and modelled to investigate how the perceptual features are encoded. The model was able to synthesize drum sound components in such detail that they could not be distinguished in an ABX test. This method may therefore be capable of outperforming existing synthesis techniques, in terms of realism. Sound examples available here.

Abstract: This paper describes a computationally efficient synthesis model for snare drum sounds. Its parameters can be modulated at audio rate while being played. The input to the model is an acoustic excitation signal which carries spectral information to color the output sound. This makes it suitable for acoustic interfaces – devices which provide excitation signal and control data simultaneously. The presented synthesis model builds up on work done by Miller Puckette and processes audio input from a piezoelectric microphone into a nonlinear reverberator. This paper details a strikingly simple but novel approach on how to make use of the momentary DC offset generated by piezoelectric microphones when pressed to simulate the changes in drumhead tension. This technique is especially of interest for interfaces without pressure sensing capabilities. In the design process we pursued an experimental approach rather than a purely mathematical. Implementations of the synthesis model are provided for Pure Data and FAUST as open source.

Abstract: Recent advancements in generative audio synthesis have allowed for the development of creative tools for generation and manipulation of audio. In this paper, a strategy is proposed for the synthesis of drum sounds using generative adversarial networks (GANs). The system is based on a conditional Wasserstein GAN, which learns the underlying probability distribution of a dataset compiled of labeled drum sounds. Labels are used to condition the system on an integer value that can be used to generate audio with the desired characteristics. Synthesis is controlled by an input latent vector that enables continuous exploration and interpolation of generated waveforms. Additionally we experiment with a training method that progressively learns to generate audio at different temporal resolutions. We present our results and discuss the benefits of generating audio with GANs along with sound examples and demonstrations.

Abstract: This paper introduces complementary n-gon waves and the shuffled samples noise effect. N-gon waves retain angles of the regular polygons and star polygons of which they are derived from in the waveform itself. N-gon waves are researched by the author since 2000 and were introduced to the public at ICMC|SMC in 2014. Complementary n-gon waves consist of an n-gon wave and a complementary angular wave. The complementary angular wave introduced in this paper complements an n-gon wave so that the two waveforms can be used to reconstruct the polygon of which the waveforms were derived from. If it is derived from a star polygon, it is not an n-gon wave and has its own characteristics. Investigations into how geometry, audio, visual and perception are related led to experiments with complementary n-gon waves and a shuffle algorithm. It is possible to reconstruct a digitised geometric shape from its shuffled samples and visualise the geometric shape with shuffled samples noise signals on a digital display device or also, within some limitations, on an oscilloscope in X-Y mode. This paper focuses on the description of discrete complementary n-gon waves and how a Fisher-Yates shuffle algorithm was applied to these waveforms and used to create the shuffled samples noise effect. In addition, some of the timbral and spatial characteristics of complementary n-gon waves and shuffled samples noise are outlined and audiovisual applications of these waveforms briefly discussed.

Abstract: Matching pursuit (MP) algorithms are widely used greedy methods to find K-sparse signal approximations in redundant dictionaries. We present an acceleration technique and an implementation of the matching pursuit algorithm acting on a multi-Gabor dictionary, i.e., a concatenation of several Gabor-type time-frequency dictionaries, consisting of translations and modulations of possibly different windows, time- and frequency-shift parameters. The proposed acceleration is based on pre-computing and thresholding inner products between atoms and on updating the residual directly in the coefficient domain, i.e., without the round-trip to the signal domain. Previously, coefficient-domain residual updates have been dismissed as having prohibitive memory requirements. By introducing an approximate update step, we can overcome this restriction and greatly improve the performance of matching pursuit at a modest cost in terms of approximation quality per selected atom. An implementation in C with Matlab and GNU Octave interfaces is available, outperforming the standard Matching Pursuit Toolkit (MPTK) by a factor of 3.5 to 70 in the tested conditions. Additionally, we provide experimental results illustrating the convergence of the implementation.

Abstract: Physical accuracy of virtual acoustics receives increasing attention due to renewed interest in virtual and augmented reality applications. So far, the modeling of vibrating objects as point sources is a common simplification which neglects effects caused by their spatial extent. In this contribution, we propose a technique for the interconnection of a distributed source to a room model, based on a modal representation of source and room. In particular, we derive a connection matrix that describes the coupling between the modes of the source and the room modes in an analytical form. Therefore, we consider the example of a string that is oscillating in a room. Both, room and string rely on well established physical descriptions that are modeled in terms of transfer functions. The derived connection of string and room defines the coupling between the characteristic string and room modes. The proposed structure is analyzed by numerical evaluations and sound examples on the supplementary website.

Abstract: Slide-string instruments allow continuous control of pitch by articulation with a slide object whose position of contact with the string is time-varying. This paper presents a method for simulation of such articulation. Taking into account sensing and musical practice considerations, an appropriate physical model configuration is determined, which is then formulated in numerical form using a finite difference approach. The model simulates the attachment and detachment phases of slide articulation which generally involve rattling, while finger damping is modelled in a more phenomenological manner as a regionally induced time-varying damping. A stability bound for the numerical model is provided via energy analysis, which also reveals the driving power contributions of the separate articulatory sources. The approach is exemplified with simulations of slide articulatory gestures that involve glissando, vibrato and finger damping.

Abstract: This paper proposes a new macroscopic physical model of ferromagnetic coils used in audio circuits. To account for realistic saturation and hysteretic phenomena, this model combines statistical physics results, measurement-driven refinements and portHamiltonian formulations that guarantee passivity, thermodynamic consistency and composability according to both electric and thermal ports. As an illustration, the model is used to simulate a passive high-pass filter. Different types of audio inputs are considered and simulations are compared to measurements.

Abstract: Feedback delay networks (FDNs) are recursive filters, which are widely used for artificial reverberation and decorrelation. While there exists a vast literature on a wide variety of reverb topologies, this work aims to provide a unifying framework to design and analyze delay-based reverberators. To this end, we present the Feedback Delay Network Toolbox (FDNTB), a collection of the MATLAB functions and example scripts. The FDNTB includes various representations of FDNs and corresponding translation functions. Further, it provides a selection of special feedback matrices, topologies, and attenuation filters. In particular, more advanced algorithms such as modal decomposition, time-varying matrices, and filter feedback matrices are readily accessible. Furthermore, our toolbox contains several additional FDN designs. Providing MATLAB code under a GNU-GPL 3.0 license and including illustrative examples, we aim to foster research and education in the field of audio processing.

Abstract: Artificial reverberation is an audio effect used to simulate the acoustics of a space while controlling its aesthetics, particularly on sounds recorded in a dry studio environment. Delay-based methods are a family of artificial reverberators using recirculating delay lines to create this effect. The feedback delay network is a popular delay-based reverberator providing a comprehensive framework for parametric reverberation by formalizing the recirculation of a set of interconnected delay lines. However, one known limitation of this algorithm is the initial slow build-up of echoes, which can sound unrealistic, and overcoming this problem often requires adding more delay lines to the network. In this paper, we study the effect of adding velvet-noise filters, which have random sparse coefficients, at the input and output branches of the reverberator. The goal is to increase the echo density while minimizing the spectral coloration. We compare different variations of velvet-noise filtering and show their benefits. We demonstrate that with velvet noise, the echo density of a conventional feedback delay network can be exceeded using half the number of delay lines and saving over 50% of computing operations in a practical configuration using low-order attenuation filters.

Abstract: In virtual acoustics, it is common to simulate the early part of a Room Impulse Response using approaches from geometrical acoustics and the late part using Feedback Delay Networks (FDNs). In order to transition from the early to the late part, it is useful to slowly fade-in the FDN response. We propose two methods to control the fade-in, one based on double decays and the other based on modal beating. We use modal analysis to explain the two concepts for incorporating this fade-in behaviour entirely within the IIR structure of a multiple input multiple output FDN. We present design equations, which allow for placing the fade-in time at an arbitrary point within its derived limit.

Abstract: Feedback delay network reverberators have decay filters associated with each delay line to model the frequency dependent reverberation time (T60) of a space. The decay filters are typically designed such that all delay lines independently produce the same T60 frequency response. However, in real rooms, there are multiple, concurrent T60 responses that depend on the geometry and physical properties of the materials present in the rooms. In this paper, we propose the Grouped Feedback Delay Network (GFDN), where groups of delay lines share different target T60s. We use the GFDN to simulate coupled rooms, where one room is significantly larger than the other. We also simulate rooms with different materials, with unique decay filters associated with each delay line group, designed to represent the T60 characteristics of a particular material. The T60 filters are designed to emulate the materials’ absorption characteristics with minimal computation. We discuss the design of the mixing matrix to control inter- and intra-group mixing, and show how the amount of mixing affects behavior of the room modes. Finally, we discuss the inclusion of air absorption filters on each delay line and physically motivated room resizing techniques with the GFDN.

Abstract: In artificial reverb algorithms, gains are commonly varied over time to break up temporal patterns, improving quality. We propose a family of novel Schroeder-style allpass filters that are energypreserving under arbitrary, continuous changes of their gains over time. All of them are canonic in delays, and some are also canonic in multiplies. This yields several structures that are novel even in the time-invariant case. Special cases for cascading and nesting these structures with a reduced number of multipliers are shown as well. The proposed structures should be useful in artificial reverb applications and other time-varying audio effects based on allpass filters, especially where allpass filters are embedded in feedback loops and stability may be an issue.

Abstract: Employing a finite number of discrete microphones, instead of a continuous distribution according to theory, reduces the physical accuracy of sound field representations captured by a spherical microphone array. For a binaural reproduction of the sound field, a number of approaches have been proposed in the literature to mitigate the perceptual impairment when the captured sound fields are reproduced binaurally. We recently presented a perceptual evaluation of a representative set of approaches in conjunction with reverberant acoustic environments. This paper presents a similar study but with acoustically dry environments with reverberation times of less than 0.25 s. We examined the Magnitude Least-Squares algorithm, the Bandwidth Extraction Algorithm for Microphone Arrays, Spherical Head Filters, spherical harmonics Tapering, and Spatial Subsampling, all up to a spherical harmonics order of 7. Although dry environments violate some of the assumptions underlying some of the approaches, we can confirm the results of our previous study: Most approaches achieve an improvement whereby the magnitude of the improvement is comparable across approaches and acoustic environments.

Abstract: The Synthetic Transaural Audio Rendering (STAR) method, first introduced at DAFx-06 then enhanced at DAFx-19, is a perceptive approach for sound spatialization aiming at reproducing the acoustic cues at the ears of the listener, using loudspeakers. To validate the method, several comparisons with state-of-the-art spatialization methods (VBAP and HOA) were conducted. Previously, quality comparisons with human subjects have been made, providing meaningful subjective results in real conditions. In this article an objective comparison is proposed, using acoustic cues error maps. The cartography enables us to study the spatialization effect in a 2D space, for a listening position within an audience, and thus not necessarily located at the center. Two approaches are conducted: the first simulates the binaural signals for a virtual KEMAR manikin, in ideal conditions and with a fine resolution; the second records these binaural signals using a real KEMAR manikin, providing real data with reverberation, though with a coarser resolution. In both cases the acoustic cues were derived from the binaural signals (either simulated or measured), and compared to the reference value taken at the center of the octophonic loudspeakers configuration. The obtained error maps display comforting results, our STAR method producing the smallest error for both simulated and experimental conditions.

Abstract: This paper proposes a neural network for carrying out parametric equalizer (EQ) matching. The novelty of this neural network solution is that it can be optimized directly in the frequency domain by means of differentiable biquads, rather than relying solely on a loss on parameter values which does not correlate directly with the system output. We compare the performance of the proposed neural network approach with that of a baseline algorithm based on a convex relaxation of the problem. It is observed that the neural network can provide better matching than the baseline approach because it directly attempts to solve the non-convex problem. Moreover, we show that the same network trained with only a parameter loss is insufficient for the task, despite the fact that it matches underlying EQ parameters better than one trained with a combination of spectral and parameter losses.

Abstract: Relative music loudness estimation is a MIR task that consists in dividing audio in segments of three classes: Foreground Music, Background Music and No Music. Given the temporal correlation of music, in this work we approach the task using a type of network with the ability to model temporal context: the Temporal Convolutional Network (TCN). We propose two architectures: a TCN, and a novel architecture resulting from the combination of a TCN with a Convolutional Neural Network (CNN) front-end. We name this new architecture CNN-TCN. We expect the CNN front-end to work as a feature extraction strategy to achieve a more efficient usage of the network’s parameters. We use the OpenBMAT dataset to train and test 40 TCN and 80 CNN-TCN models with two grid searches over a set of hyper-parameters. We compare our models with the two best algorithms submitted to the tasks of music detection and relative music loudness estimation in MIREX 2019. All our models outperform the MIREX algorithms even when using a lower number of parameters. The CNN-TCN emerges as the best architecture as all its models outperform all TCN models. We show that adding a CNN front-end to a TCN can actually reduce the number of parameters of the network while improving performance. The CNN front-end effectively works as a feature extractor producing consistent patterns that identify different combinations of music and non-music sounds and also helps in producing a smoother output in comparison to the TCN models.

Abstract: This paper proposes a grey-box neural network based approach to modelling LFO modulated time-varying effects. The neural network model receives both the unprocessed audio, as well as the LFO signal, as input. This allows complete control over the model’s LFO frequency and shape. The neural networks are trained using guitar audio, which has to be processed by the target effect and also annotated with the predicted LFO signal before training. A measurement signal based on regularly spaced chirps was used to accurately predict the LFO signal. The model architecture has been previously shown to be capable of running in real-time on a modern desktop computer, whilst using relatively little processing power. We validate our approach creating models of both a phaser and a flanger effects pedal, and theoretically it can be applied to any LFO modulated time-varying effect. In the best case, an errorto-signal ratio of 1.3% is achieved when modelling a flanger pedal, and previous work has shown that this corresponds to the model being nearly indistinguishable from the target device.

Abstract: This paper describes a new onset-informed source separation method based on non-negative matrix factorization (NMF) with binary masks. Many previous approaches to separate a target instrument sound from polyphonic music have used side-information of the target that is time-consuming to prepare. The proposed method leverages the onsets of the target instrument sound to facilitate separation. Onsets are useful information that users can easily generate by tapping while listening to the target in music. To utilize onsets in NMF-based sound source separation, we introduce binary masks that represent on/off states of the target sound. Binary masks are formulated as Markov chains based on continuity of musical instrument sound. Owing to the binary masks, onsets can be handled as a time frame in which the binary masks change from off to on state. The proposed model is inferred by Gibbs sampling, in which the target sound source can be sampled efficiently by using its onsets. We conducted experiments to separate the target melody instrument from recorded polyphonic music. Separation results showed about 2 to 10 dB improvement in target source to residual noise ratio compared to the polyphonic sound. When some onsets were missed or deviated, the method is still effective for target sound source separation.

Abstract: In this paper we present an approach to using traditional digital IIR filter structures inside deep-learning networks trained using backpropagation. We establish the link between such structures and recurrent neural networks. Three different differentiable IIR filter topologies are presented and compared against each other and an established baseline. Additionally, a simple Wiener-Hammerstein model using differentiable IIRs as its filtering component is presented and trained on a guitar signal played through a Boss DS-1 guitar pedal.

Abstract: In this paper, we present TIV.lib, an open-source library for the content-based tonal description of musical audio signals. Its main novelty relies on the perceptually-inspired Tonal Interval Vector space based on the Discrete Fourier transform, from which multiple instantaneous and global representations, descriptors and metrics are computed—e.g., harmonic change, dissonance, diatonicity, and musical key. The library is cross-platform, implemented in Python and the graphical programming language Pure Data, and can be used in both online and offline scenarios. Of note is its potential for enhanced Music Information Retrieval, where tonal descriptors sit at the core of numerous methods and applications.

Abstract: Guitar effects are commonly used in popular music to shape the guitar sound to fit specific genres or to create more variety within musical compositions. The sound is not only determined by the choice of the guitar effect, but also heavily depends on the parameter settings of the effect. This paper introduces a method to estimate the parameter settings of guitar effects, which makes it possible to reconstruct the effect and its settings from an audio recording of a guitar. The method utilizes audio feature extraction and shallow neural networks, which are trained on data created specifically for this task. The results show that the method is generally suited for this task with average estimation errors of ±5% − ±16% of different parameter scales and could potentially perform near the level of a human expert.

Abstract: Deep learning models have provided extremely successful solutions in most audio application fields. However, the high accuracy of these models comes at the expense of a tremendous computation cost. This aspect is almost always overlooked in evaluating the quality of proposed models. However, models should not be evaluated without taking into account their complexity. This aspect is especially critical in audio applications, which heavily relies on specialized embedded hardware with real-time constraints. In this paper, we build on recent observations that deep models are highly overparameterized, by studying the lottery ticket hypothesis on deep generative audio models. This hypothesis states that extremely efficient small sub-networks exist in deep models and would provide higher accuracy than larger models if trained in isolation. However, lottery tickets are found by relying on unstructured masking, which means that resulting models do not provide any gain in either disk size or inference time. Instead, we develop here a method aimed at performing structured trimming. We show that this requires to rely on global selection and introduce a specific criterion based on mutual information. First, we confirm the surprising result that smaller models provide higher accuracy than their large counterparts. We further show that we can remove up to 95% of the model weights without significant degradation in accuracy. Hence, we can obtain very light models for generative audio across popular methods such as Wavenet, SING or DDSP, that are up to 100 times smaller with commensurate accuracy. We study the theoretical bounds for embedding these models on Raspberry Pi and Arduino, and show that we can obtain generative models on CPU with equivalent quality as large GPU models. Finally, we discuss the possibility of implementing deep generative audio models on embedded platforms.