Researchers have advanced voice tuning technology, making significant improvements to traditional tuning software’s previous capabilities.
More than just Auto-Tune on steroids, this innovation—called Diff-Pitcher—seamlessly corrects out-of-tune singing while maintaining the original vocal timbre and naturalness, expanding its possible applications beyond entertainment and the music industry and into the health care setting.
“Diff-Pitcher is a generative deep neural network that takes pitch-correction technology to a new level. Its precision and control can not only help musical artists and producers but also open new possibilities in areas such as voice rehabilitation and assistive technologies,” says team member Jiarui Hai, a PhD student in the Whiting School of Engineering’s electrical and computer engineering department.
Hai and lead researcher Mounya Elhilali, a professor in electrical and computer engineering, presented their new technology at the 2023 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics last fall.
In music, the ability of pitch-correcting software to adjust the sound of a voice or instrument to the desired pitch is crucial. Diff-Pitcher employs advanced algorithms to enhance the naturalness and quality of pitch correction, surpassing previous tools.
Unlike older software that trained on pairs of corrected and original vocals, Diff-Pitcher analyzes the visual representation (spectrogram) of original vocals requiring correction. It identifies target notes, predicts needed adjustments, and transforms the corrected spectrogram into audio, achieving pitch correction in two easy steps.
According to Hai, Diff-Pitcher is notable not only for its ability to create authentic and controlled voices but also for the method used to achieve it.
“[The results sound] really natural, and unlike in older ways of fixing pitch, we can still regulate how high or low the voice goes,” Hai says.
The researchers plan to continue to refine Diff-Pitcher to make voices sound even more natural and to extend the tool’s capabilities beyond enhancing music production. They believe Dif-Pitcher has a wide range of potential applications, including aiding those undergoing voice rehabilitation—particularly those with vocal impediments.
“The technology could revolutionize treatment for a spectrum of speech-related disorders, offering valuable support for post-laryngectomy patients and contributing to the voice rehabilitation of stroke victims,” Hai says.
Source: Johns Hopkins University
