iZotope’s award-winning RX 6 Audio Editor is the industry standard for audio repair, restoration, and enhancement. It offers a comprehensive suite of tools focused on alleviating common to complex audio issues. Post production professionals, audio engineers, and video editors alike use RX to transform problematic recordings into production-ready audio.
Design Philosophy
At Sweetwater, one of the things we've come to appreciate about iZotope products is that no matter how much they can do, they're still easy to navigate and use. For example, RX 6's unique graphical spectrogram display isn't just easy to read; it makes isolating humming and buzzing positively intuitive. RX features an advanced Spectrogram display that is capable of showing greater time and frequency resolution than other Spectrograms, uncovering an unprecedented level of detail when working with audio. Without access to audio recorded by an on-set location sound engineer, Mouthon has relied on iZotope RX over the years to clean up audio yet still retain the natural sound of whatever environment Bourdain is in. In this interview, Mouthon shares how he uses RX 6 Advanced on the show.
The RX 6 Audio Editor is a visual, selection-based editing environment. Most of its user interface is devoted to the Spectrogram/Waveform display, an integral part of the RX editing workflow. The display enables you to refine and visualize your audio, allowing for better recognition and selection of problem areas.
Using the spectrogram to identify audio problems
RX Audio Editor Feature Comparison Chart
RX Module Comparison Chart
RX Plug-in Comparison Chart
This help guide is shared by RX 6 Elements, RX 6 Standard and RX 6 Advanced. The following tags are used throughout the manual to differentiate the feature sets:
By inconspicuously attaching on clothing near a person’s mouth, the lavalier microphone (lav mic) provides multiple benefits when capturing dialogue. For video applications, there is no microphone distracting viewer attention, and the orator can move freely and naturally since they aren’t holding a microphone. Lav mics also benefit audio quality, since they are attached near the mouth they eliminate noise and reverberation from the recording environment.
Unfortunately, the freedom lav mics provide an orator to move around can also be a detriment to the audio engineer, as the mic can rub against clothing or bounce around creating disturbances often described as rustle. Here are some examples of lav-mic recordings where the person moved just a bit too much:
Numark mixtrack 2 mapping virtual dj download. https://izotopetech.files.wordpress.com/2017/03/de-rustle-3.wavhttps://izotopetech.files.wordpress.com/2017/03/de-rustle.wav
Rustle cannot be easily removed using the existing De-noise technology found in an audio repair program such as iZotope RX, because rustle changes over time in unpredictable ways based on how the person wearing the microphone moves their body. The material the clothing is made of also can have an impact on the rustle’s sonic quality, and if you have the choice attaching it to natural fibers such as cotton or wool is preferred to synthetics or silk in terms of rustling intensity. Attaching the lav mic with tape instead of using a clip can also change the amount and sound of rustle.
Because of all these variations, rustle presents itself sonically in many different ways from high frequency “crackling” sounds to low frequency “thuds” or bumps. Additionally, rustle often overlaps with speech and is not well localized in time like a click or in frequency like electrical hum. These difficulties made it nearly impossible to develop an effective deRustle algorithm using traditional signal processing approaches. Fortunately, with recent breakthroughs in source separation and deep learning removing lav rustle with minimal artifacts is now possible.
Audio Source Separation
Often referred to as “unmixing”, source separation algorithms attempt to recover the individual signals composing a mix, e.g., separating the vocals and acoustic guitar from your favorite folk track. While source separation has applications ranging from neuroscience to chemical analysis, its most popular application is in audio, where it drew inspiration from the cocktail party effect in the human brain, which is what allows you to hear a single voice in a crowded room, or focus on a single instrument in an ensemble.
We can view removing lav mic rustle from dialogue recordings as a source separation problem with two sources: rustle and dialogue. Audio source separation algorithms typically operate in the frequency domain, where we separate sources by assigning each frequency component to the source that generated it. This process of assigning frequency components to sources is called spectral masking, and the mask for each separated source is a number between zero and one at each frequency. When each frequency component can belong to only one source, we call this a binary mask since all masks contain only ones and zeros. Alternatively, a ratio mask represents the percentage of each source in each time-frequency bin. Ratio masks can give better results, but are more difficult to estimate.
For example, a ratio mask for a frame of speech in rustle noise will have values close to one near the fundamental frequency and its harmonics, but smaller values in low-frequencies not associated with harmonics and in high frequencies where rustle noise dominates.
To recover the separated speech from the mask, we multiply the mask in each frame by the noisy magnitude spectrum, and then do an inverse Fourier transform to obtain the separated speech waveform.
Mask Estimation with Deep Learning
The real challenge in mask-based source separation is estimating the spectral mask. Because of the wide variety and unpredictable nature of lav mic rustle, we cannot use pre-defined rules (e.g., filter low frequencies) to estimate the spectral masks needed to separate rustle from dialogue. Fortunately, recent breakthroughs in deep learning have led to great improvements in our ability to estimate spectral masks from noisy audio (e.g., this interesting article related to hearing aids). In our case, we use deep learning to estimate a neural network that maps speech corrupted with with rustle noise (input) to separated speech and rustle (output).
Since we are working with audio we use recurrent neural networks, which are better at modeling sequences than feed-forward neural networks (the models typically used for processing images), and store a hidden state between time steps that can remember previous inputs when making predictions. We can think of our input sequence as a spectrogram, obtained by taking the Fourier transform of short-overlapping windows of audio, and we input them to our neural network one column at a time. We learn to estimate a spectral mask for separating dialogue from lav mic rustle by starting with a spectrogram containing only clean speech.
https://izotopetech.files.wordpress.com/2017/04/clean_speech.wav
We can then mix in some isolated rustle noise, to create a nosiy spectrogram where the true separated sources are known.
https://izotopetech.files.wordpress.com/2017/04/noisy_speech.wav
We then feed this noisy spectrogram to the neural network which outputs a ratio mask. By multiplying the ratio mask with the noisy input spectrogram we have an estimate of our clean speech spectrogram. We can then compare this estimated clean speech spectrogram with the original clean speech, and obtain an error signal which can be backpropagated through the neural network to update the weights. We can then repeat this process over and over again with different clean speech and isolated rustle spectrograms. Once training is complete we can feed a noisy spectrogram to our network and obtain clean speech.
Drum Samples For Garageband Free Downloads - 2000 Shareware periodically updates software information and pricing of Drum Samples For Garageband from the publisher, so some information may be slightly out-of-date. You should confirm all information before relying on it. Garageband Samples. Browse our collection of Garageband samples, sample packs, loops, one shot drums and sound effects. All Garageband samples are available to download 100% royalty free for use in your music production or sound design project. Download drum samples for garageband. Spoil yourself with Garageband Drum Loops. Here's a list of pages within the Macloops site that contain free Garageband drum loops. We've got a varied selection of musical genres and drumming styles that range from live, acoustic samples to electronic beats that have been sequenced and quantized to. Load up on Apple Loops for Garageband. Macloops is a massive resource for free apple loops and samples. We have thousands of AIFF format apple loops ranging from drum loops, bass loops and synth loops to guitar loops, cinematic and lots more. Free Garageband Loops. Within these pages you will find hundreds of free demo drum loops. These can be easily located by using the drum loop menu to your right. Simply click on the Drum Loop Genre that you are interested in and you will be presented with a list of drum loop download packs for that genre.
Izotope Spectrogram Rx- 6 9Gathering Training Data
We ultimately want to use our trained network to generalize across any rustle corrupted dialogue an audio engineer may capture when working with a lav mic. To achieve this we need to make sure our network sees as many different rustle/dialogue mixtures as possible. Obtaining lots of clean speech samples is relatively easy; there are lots of datasets developed for speech recognition in addition to audio recorded for podcasts, video tutorials, etc. However, obtaining isolated rustle noises is much more difficult. Engineers go to great lengths to minimize rustle and recordings of rustle typically are heavily overlapped with speech. As a proof of concept, we used recordings of clothing or card shuffling from sound effects libraries as a substitute for isolated rustle.
https://izotopetech.files.wordpress.com/2017/04/cards_playing_cards_deal02_stereo.wav
These gave us promising initial results for rustle removal, but only worked well for rustle where the mic rubbed heavily over clothing. To build a general deRustle algorithm, we were going to have to record our own collection of isolated rustle.
We started by calling into the post production industry to obtain as many rustle corrupted dialogue samples as possible. This gave us an idea of the different qualities of rustle we would need to emulate in our dataset. Our sound design team then worked with different clothing materials, lav mounting techniques (taping and clipping), and motions from regular speech gestures to jumping and stretching to collect our isolated rustle dataset. Additionally, in machine learning any patterns can potentially be picked up by the algorithm, so we also varied things like microphone type and recording environment to make sure our algorithm didn’t specialize to a specific microphone frequency response for example. Here’s a greatest hits collection of some of the isolated rustle we used to train our algorithm:
https://izotopetech.files.wordpress.com/2017/04/rustle_training.wav
Izotope Rx ManualDebugging the Data
One challenge with machine learning is when things go wrong it’s often not clear what the root cause of the problem was. Your training algorithm can compile, converge, and appear to generalize well, but still behave strangely in the wild. For example, our first attempt at training a deRustle algorithm always output clean speech with almost no energy above 10 kHz, even though there was speech energy at those frequencies.
It turned out that a large percentage of our clean speech was recorded with a microphone that attenuated high frequencies. Here’s an example problematic clean speech spectrogram with almost no high-frequency energy:
Since all of our rustle recordings had high frequency energy the algorithm learned to assign no high frequency energy to speech. Adding more high quality clean speech to our training set corrected this problem.
Before and After Examples
Once we got the problems with our data straightened out and trained the network for a couple days on a NVIDIA K80 GPU, we were ready to try it out removing rustle from some pretty messy real-world examples:
Before
https://izotopetech.files.wordpress.com/2017/03/de-rustle.wav
Djay 2 pixel 3 cut out video. After
https://izotopetech.files.wordpress.com/2017/03/de-rustle_proc.wav
Before
https://izotopetech.files.wordpress.com/2017/03/de-rustle-3.wav
After
https://izotopetech.files.wordpress.com/2017/03/de-rustle-3_proc.wav
Izotope Spectrogram Rx- 6 0ConclusionIzotope Spectrogram Rx- 6 7
While lav mics are an extremely valuable tool, if they move a bit too much the rustle they produce can drive you crazy. Fortunately, by leveraging advances in deep learning we were able to develop a tool to accurately remove this disturbance. If you’re interested in trying this deRustle algorithm give the RX 6 Advanced demo a try.
Comments are closed.
|
AuthorWrite something about yourself. No need to be fancy, just an overview. Archives
December 2020
Categories |