How to evaluate the performance of Voice Conversion systems

[mahdeslami11]

To evaluate the performance of Voice Conversion systems, different aspects like: speaker similarity, quality, and intelligibility could be evaluated. In your opinion, which objective or subjective tests necessarily correspond to human judgments? In your repository, what method have you used to evaluate the Voice conversion system, if any?

[sanagolmarzi]

In voice conversion (VC), it is highly desirable to obtain transformed speech signals that are perceptually close to a target speaker’s voice. To this end, a perceptually meaningful criterion where the human auditory system was taken into consideration in measuring the distances between the converted and the target voices was adopted in the proposed VC scheme. The conversion rules for the features associated with the spectral envelope and the pitch modification factor were jointly constructed so that perceptual distance measurement was minimized. This minimization problem was solved using a deep neural network (DNN) framework where input features and target features were derived from source speech signals and time-aligned version of target speech signals, respectively. The validation tests were carried out for the CMU ARCTIC database to evaluate the effectiveness of the proposed method, especially in terms of perceptual quality. The experimental results showed that the proposed method yielded perceptually preferred results compared with independent conversion using conventional mean-square error (MSE) criterion. The maximum improvement in perceptual evaluation of speech quality (PESQ) was 0.312, compared with the conventional VC method.

[alirezaahmadiii]

a new performance evaluation measure for assessing the capacity of voice conversion systems to modify the speech of one speaker (source) so that it sounds as if it was uttered by another speaker (target). This measure relies on a GMM-UBM-based likelihood estimator that estimates the degree of proximity between an utterance of the converted voice and the predefined models of the source and target voices. Existing objective evaluation metrics for voice conversion (VC) are not always correlated well with human perception. Therefore, training VC models with such criteria may not effectively improve naturalness and similarity of converted speech. we propose deep learning based assessment models to predict human ratings of converted speech. We adopt the convolutional and recurrent neural network models to build a mean opinion score (MOS) predictor, termed MOSNet. And also, in my project, due to the change of the power spectrum and the change of genes, it has reached its desired level, and due to random CNN, it has caused a change in the sound style.

[maissa10336]

This measure relies on a GMM-UBM-based likelihood estimator that estimates the degree of proximity between an utterance of the converted voice and the predefined models of the source and target voices. Existing objective evaluation metrics for voice conversion (VC) are not always correlated well with human perception. Therefore, training VC models with such criteria may not effectively improve naturalness and similarity of converted speech. we propose deep learning based assessment models to predict human ratings of converted speech. We adopt the convolutional and recurrent neural network models to build a mean opinion score (MOS) predictor, termed MOSNet. And also, in my project, due to the change of the power spectrum and the change of genes, it has reached its desired level, and due to random CNN, it has caused a change in the sound style.

[Shabnamlatifian]

Emotional Voice Conversion, or emotional VC, is a technique of
converting speech from one emotion state into another one, keep�ing the basic linguistic information and speaker identity. Previous
approaches for emotional VC need parallel data and use dynamic
time warping (DTW) method to temporally align the source-target
speech parameters. These approaches often define a minimum gen�eration loss as the objective function, such as L1 or L2 loss, to learn
model parameters. Recently, cycle-consistent generative adversarial
networks (CycleGAN) have been used successfully for non-parallel
VC. This paper investigates the efficacy of using CycleGAN for
emotional VC tasks. Rather than attempting to learn a mapping be�tween parallel training data using a frame-to-frame minimum gener�ation loss, the CycleGAN uses two discriminators and one classifier
to guide the learning process, where the discriminators aim to differ�entiate between the natural and converted speech and the classifier
aims to classify the underlying emotion from the natural and con�verted speech. The training process of the CycleGAN models ran�domly pairs source-target speech parameters, without any temporal
alignment operation. The objective and subjective evaluation results
confirm the effectiveness of using CycleGAN models for emotional
VC. The non-parallel training for a CycleGAN indicates its potential
for non-parallel emotional VC

[mahdigodarzi]

i have used it
To realize any-to-any (A2A) voice conversion (VC), most methods are to perform symmetric self-supervised reconstruction tasks (Xi to Xi), which usually results in inefficient performances due to inadequate feature decoupling, especially for unseen speakers. We propose a two-stage reconstruction task (Xi to Yi to Xi) using synthetic specific-speaker speeches as intermedium features, where A2A VC is divided into two stages: any-to-one (A2O) and one-to-Any (O2A). In the A2O stage, we propose a new A2O method: SingleVC, by employing a noval data augment strategy(pitch-shifted and duration-remained, PSDR) to accomplish Xi to Yi. In the O2A stage, MediumVC is proposed based on pre-trained SingleVC to conduct Yi to Xi. Through such asymmetrical reconstruction tasks (Xi to Yi in SingleVC and Yi to Xi in MediumVC), the models are to capture robust disentangled features purposefully. Experiments indicate MediumVC can enhance the similarity of converted speeches while maintaining a high degree of naturalness.

[SepidehKianasab]

They introduced a new performance evaluation measure for assessing the capacity of voice conversion systems to modify the speech of one speaker (source) so that it sounds as if it was uttered by another speaker (target). This measure relies on a GMM-UBM-based likelihood estimator that estimates the degree of proximity between an utterance of the converted voice and the predefined models of the source and target voices. The proposed approach allows the formulation of an objective criterion, which is applicable for both evaluation of the virtue of a single system and for direct comparison (benchmarking) among different voice conversion systems. To illustrate the functionality and the practical usefulness of the proposed measure, we contrast it with four well-known objective evaluation criteria.

[Zahrakarbalaeimohammadi]

In the completed project, a new one-shot VC approach is proposed, which is able to perform VC with only one example utterance of the source and target speakers respectively, and the source and target speakers do not even need to be seen during training. In this algorithm, by separating speaker and content representations by sample normalization (IN). Objective and subjective evaluation shows that the proposed model is capable of producing a voice similar to the target speaker. In addition to the performance measurement, we have also shown that the model can learn the speaker's meaningful representations without any supervision.
https://github.com/mahdeslami11/AdaIN-VC

[Fatemehmojaver]

We use recurrent convolutional neural network models to develop a mean opinion (MOS) predictor, called MOSNet.

[github-actions[bot]]

🕒 Discussion Activity Reminder 🕒

This Discussion has been labeled as dormant by an automated system for having no activity in the last 60 days. Please consider one the following actions:

1️⃣ Close as Out of Date: If the topic is no longer relevant, close the Discussion as out of date at the bottom of the page.

2️⃣ Provide More Information: Share additional details or context — or let the community know if you've found a solution on your own.

3️⃣ Mark a Reply as Answer: If your question has been answered by a reply, mark the most helpful reply as the solution.

Note: This dormant notification will only apply to Discussions with the Question label. To learn more, see our recent announcement.

Thank you for helping bring this Discussion to a resolution! 💬