Sobolev Norms in Neural Embeddings Measure Audio Morphing Regularity

Théo Chasle Cauchy, Modan Tailleur, Barbara Pascal, Fanny Roche, Mathieu Lagrange

Contact: theo.chasle-cauchy@ls2n.fr

Abstract

Morphing has recently gained renewed interest with the emergence of generative models, particularly in audio and image generation. In musical sound synthesis, morphing can generate intermediate sounds between two targets, helping musicians and sound engineers explore new sounds with interesting perceptual properties. As morphing is inherently defined in perceptual terms, evaluating this task is challenging. In this work, we introduce Sobolev Distances to Ideal Morphing (SDIM), a novel objective metric to quantify the regularity of audio morphing trajectories in perceptually relevant audio embedding spaces. Leveraging a physics-based sound synthesizer, we evaluate the discriminative power of SDIM on controlled morphing trajectories with varying degrees of regularity and compare it with that of existing audio morphing metrics. Results show that, contrary to state-of-the-art metrics, the proposed metric reliably discriminates desirable trajectories from adversarial ones.

This is the companion page for the paper: Sobolev Norms in Neural Embeddings Measure Audio Morphing Regularity

Experiment code is available in this GitHub repository.

Reference Timbre Space

Parameters

Listening to the extrema of the hypercube

In the defined hypercube, the synthesizer produces a metal-like sound for log(p) at its minimum value and a wooden-like sound for log(p) at its maximum value.

By choosing the minimum or maximum values for each parameter, you can listen to the extrema of the hypercube:

log(ω)

𝜏

log(p)

log(D)

α

Trajectories Examples

Parameter Values of the Source s and Target t:

s:

log(ω) = 3.40, τ = 0.34, log(p) = -0.82, log(D) = -1.43, α = 0.32

t:

log(ω) = 3.40, τ = 0.34, log(p) = 0.30, log(D) = -1.43, α = 0.32

LUM and Null

Listening to a Linear and Uniform Morphing (LUM) trajectory

LUM trajectories are synthesized with a regularly sampled linear interpolation between anchors' log(p) values (other parameters have the same values between s and t).

To listen to the anchors s and t and the sounds of the morphing trajectory, click on the circles below:

-0.82 -0.73 -0.64 -0.54 -0.45 -0.36 -0.26 -0.17 -0.07 0.02 0.11 0.21 0.30
log(p)

Listening to a Null trajectory

Null trajectories are synthesized with tuples of parameters randomly sampled in the hypercube.

To listen to the anchors s and t and the sounds of the morphing trajectory, click on the circles below:

Adversarial Configurations

Listening to a Non-Uniform Configuration trajectory

NUC trajectories are synthesized with a non-uniform linear interpolation between anchors' log(p) values.

To listen to the anchors s and t and the sounds of the morphing trajectory, click on the circles below:

Listening to an Equi-distant Configuration trajectory

EQC trajectories are synthesized with parameters such that the distance to the source is the same as in the ideal trajectory but the angle is randomly sampled.

To listen to the anchors s and t and the sounds of the morphing trajectory, click on the circles below: