Tutorial: Mixing Signals

Working with mixers

Audio Mixing

Audio mixing is a technique that allows us to combine multiple signals, sounds, or tracks together into a single object. Consider if we wanted to record a band composed of the vocals, guitar, bass, and drums. Instead of having just one microphone pick up all the musical notes from the set, we could use multiple microphones for each instrument. If we were to implement the latter approach, we can manually modulate each signal we are receiving. We have the ability to increase and decrease the volumes of each signal so we can have one instrument be emphasized more. If we were to use just a single input, we would have a single signal to work with. Mixers allow applications to tackle this issue.

Example Program

To fully comprehend how mixing works, let us take a look at the example program below:
use Libraries.Compute.Math
use Libraries.Containers.Array
use Libraries.Sound.Audio
use Libraries.Sound.AudioSamples

Array<AudioSamples> mix    // Array to hold the AudioSamples
AudioSamples mixedSamples  // The AudioSamples object that holds the mix
AudioSamples samples1  // AudioSamples 1
AudioSamples samples2  // AudioSamples 2
AudioSamples samples3  // AudioSamples 3
AudioSamples samples4  // AudioSamples 4

mixedSamples:SetChannels(1) // Set all samples to mono
samples1:SetChannels(1)
samples2:SetChannels(1)
samples3:SetChannels(1)
samples4:SetChannels(1)

mixedSamples:SetSizeInSeconds(4)   // 4 second playback
samples1:SetSizeInSeconds(4)   // 4 second playback (acts as background noise)
samples2:SetSizeInSeconds(3)   // 3 second playback
samples3:SetSizeInSeconds(2)   // 2 second playback
samples4:SetSizeInSeconds(1)   // 1 second playback

Math math
integer counter = 0
integer sampleSize = mixedSamples:GetSamplesPerSecond()
number twoPI = math:pi * 2
number frequency1 = 110    // frequency for samples1
number frequency2 = 220    // frequency for samples2
number frequency3 = 330    // frequency for samples3
number frequency4 = 440    // frequency for samples4

repeat while counter < mixedSamples:GetSize()
   if (counter < samples1:GetSize())
//        number s1 = math:Sine(frequency1 * twoPI * counter/sampleSize)  // Sine wave with frequency of 110
       number modulatorWave = 10 * math:Sine(10 * twoPI * counter/sampleSize)
       number s1 = 1 * math:Sine(frequency1 * twoPI * counter/sampleSize + modulatorWave) // FM synthesized wave
       samples1:Set(counter, s1, 0)
   end
   if (counter < samples2:GetSize())
       number s2 = math:Sine(frequency2 * twoPI * counter/sampleSize) // Sine wave with frequency of 220 Hz
       samples2:Set(counter, s2, 0)
   end
   if (counter < samples3:GetSize())
       number s3 = math:Sine(frequency3 * twoPI * counter/sampleSize) // Sine wave with frequency of 330 Hz
       samples2:Set(counter, s3, 0)
   end
   if (counter < samples4:GetSize())
       number s4 = math:Sine(frequency4 * twoPI * counter/sampleSize) // Sine wave with frequency of 440 Hz
       samples4:Set(counter, s4, 0)
   end
   counter = counter + 1
end

mix:Add(samples1)  // Add samples1 object to Array
mix:Add(samples2)  // Add samples2 object to Array
mix:Add(samples3)  // Add samples3 object to Array
mix:Add(samples4)  // Add samples4 object to Array

counter = 0
repeat while counter < mixedSamples:GetSize()
   number result = 0
   integer samplesMixed = 0    // Number of samples actually mixed
   integer i = 0

   repeat while i < mix:GetSize()  // i = 0, 1, 2, 3 since mix size = 4 (added 4 objects into mix array)
       AudioSamples mixable = mix:Get(i)   // Gets the current AudioSamples object (samples1 - samples4)
       if counter < mixable:GetSize()  // If the counter is withing the size of the AudioSamples object 
           result = result + mixable:Get(counter)  // result = result + current sample from current AudioSamples object
           samplesMixed = samplesMixed + 1 // Increment number of samples actually mixed
       end
       i = i + 1
   end

   result = result / 4    // result = result / number of samples actually mixed

  mixedSamples:Set(counter, result, 0)    // Set the combined samples into the mixedSamples object
   counter = counter + 1
end

Audio audio
audio:Load(mixedSamples)
audio:PlayUntilDone()

Creating the Signals

In the example above, notice that there are five different AudioSamples objects. The mixedSamples object is the “mixer” that holds the other four AudioSamples mixed signal.
Array<AudioSamples> mix    // Array to hold the AudioSamples
AudioSamples mixedSamples  // The AudioSamples object that holds the mix
AudioSamples samples1  // AudioSamples 1
AudioSamples samples2  // AudioSamples 2
AudioSamples samples3  // AudioSamples 3
AudioSamples samples4  // AudioSamples 4

Looking at this code, we see that we also have an array containing AudioSamples objects called mix. Mix holds the original signals (samples 1 through 4) which we will use later to generate the mixed signals. After creating the objects, we (as we did in previous tutorials) must set the channels and seconds of our samples. Next we move on to adding the values into our samples objects. 

Math math
integer counter = 0
integer sampleSize = mixedSamples:GetSamplesPerSecond()
number twoPI = math:pi * 2
number frequency1 = 110    // frequency for samples1
number frequency2 = 220    // frequency for samples2
number frequency3 = 330    // frequency for samples3
number frequency4 = 440    // frequency for samples4

repeat while counter < mixedSamples:GetSize()
   if (counter < samples1:GetSize())
//        number s1 = math:Sine(frequency1 * twoPI * counter/sampleSize)  // Sine wave with frequency of 110
       number modulatorWave = 10 * math:Sine(10 * twoPI * counter/sampleSize)
       number s1 = 1 * math:Sine(frequency1 * twoPI * counter/sampleSize + modulatorWave) // FM synthesized wave
       samples1:Set(counter, s1, 0)
   end
   if (counter < samples2:GetSize())
       number s2 = math:Sine(frequency2 * twoPI * counter/sampleSize) // Sine wave with frequency of 220 Hz
       samples2:Set(counter, s2, 0)
   end
   if (counter < samples3:GetSize())
       number s3 = math:Sine(frequency3 * twoPI * counter/sampleSize) // Sine wave with frequency of 330 Hz
       samples2:Set(counter, s3, 0)
   end
   if (counter < samples4:GetSize())
       number s4 = math:Sine(frequency4 * twoPI * counter/sampleSize) // Sine wave with frequency of 440 Hz
       samples4:Set(counter, s4, 0)
   end
   counter = counter + 1
end

To start off we set our frequencies for each of our samples. Next we run a loop that goes from 0 to mixedSamples:GetSize(). The reason why we use mixedSamples’ size is because samples1 through 4 have different sizes (due to their size we set in seconds earlier). So for example, if we had one sample object with a size of 5 and another at the size of 10, than in this context, we should use the size of 10 to limit the loop because we can access elements from both objects. If we were to use 5, then we are limited to accessing only half of what is in the second object. Alternatively, we can loop through each samples object individually but this method allows us to do so all at once. As explained, the samples have different sizes so we must account for them when accessing their elements. We do this through our conditional structures (an if statement) to check if counter is within the bounds of its size. Since we have four samples objects, we need four of these statements. As you can see, we used sine waves with different frequencies for all our samples not including the first one. The first sample uses an FM synthesized wave for reasons we will discuss later on but one could alternatively use another sine wave instead. After creating and setting the values within our samples objects, we add them to our array with:

mix:Add(samples1)  // Add samples1 object to Array
mix:Add(samples2)  // Add samples2 object to Array
mix:Add(samples3)  // Add samples3 object to Array
mix:Add(samples4)  // Add samples4 object to Array
and are now ready to mix the signals.

Mixing the Signals

Looking at the code below, we can see that we have two loops, one within another (a technique called “nesting”).
counter = 0
repeat while counter < mixedSamples:GetSize()
   number result = 0
   integer samplesMixed = 0    // Number of samples actually mixed
   integer i = 0

   repeat while i < mix:GetSize()  // i = 0, 1, 2, 3 since mix size = 4 (added 4 objects into mix array)
       AudioSamples mixable = mix:Get(i)   // Gets the current AudioSamples object (samples1 - samples4)
       if counter < mixable:GetSize()  // If the counter is withing the size of the AudioSamples object 
           result = result + mixable:Get(counter)  // result = result + current sample from current AudioSamples object
           samplesMixed = samplesMixed + 1 // Increment number of samples actually mixed
       end
       i = i + 1
   end

   result = result / samplesMixed    // result = result / number of samples actually mixed

  mixedSamples:Set(counter, result, 0)    // Set the combined samples into the mixedSamples object
   counter = counter + 1
end
The outside loop is the standard loop that goes from 0 to mixedSamples:Getsize(), and is nearly identical to the previous loops we’ve been working on with digital signal processing. The bottom of the loop contains code that sets the sample and also increments the counter.

The actual mixing happens inside the nested loop. Our samplesMixed variable is an integer that holds the amount of samples that we are actually mixing. This number will change as we go through the loop because each AudioSamples object we are going through have different sizes. This means that at certain points, we won’t be mixing all the signals, but only the ones within the boundaries we specified which in this case, would be its size.

Let's further examine this chunk of code: 
repeat while i < mix:GetSize()  // i = 0, 1, 2, 3 since mix size = 4 (added 4 objects into mix array)
   AudioSamples mixable = mix:Get(i)   // Gets the current AudioSamples object (samples1 - samples4)
   if counter < mixable:GetSize()  // If the counter is withing the size of the AudioSamples object 
      result = result + mixable:Get(counter)  // result = result + current sample from current AudioSamples object
       samplesMixed = samplesMixed + 1 // Increment number of samples actually mixed
   end
   i = i + 1
end

First we are going through and grabbing each samples object that we have in our AudioSamples array. Next, if the counter is within the size of the current AudioSamples object, then we are going to add the sample to our result number from that specific object. Result is the sum of all the samples which would result in the combination of all the signals. This approach is a sample-by-sample mix which elements the loss of signals. We repeat this process for every item in our array. We then divide our result by the amount of samples we actually mixed to separate each sound so we can hear which signal is playing. If we were not to do this, we would simply hear static.

Playing the Mixed Sounds

Lastly, we load the sound and play it until it’s done. Pay attention to the different sounds and frequencies being played here. We essentially have each sound playing at the same time but we hear one less signal each second. Why is this? This is because we set the samples to different sizes (in seconds) and what we hear is samples4 playing for one second, samples3 playing for two seconds, samples2 playing for two seconds, and samples1 playing for four seconds so what results is a sound getting deeper and deeper. The reason why we included an FM synthesized wave is to hear it in the background so we can actually hear what is going on. We can see how each wave is played simultaneously which proves that we actually “mixed” our signals.

End of Lesson

You have reached the end of the lesson for Audio. To view more tutorials, press the button below or you can go back to the previous lesson pages.

Go BackBack to Tutorials