This paper presents an algorithm enhanced mixing of audio streams for computation on GPUs, which combines multiple stages of mixing by using two-pass rendering, which significantly reduces the switching time between buffers. Methods of computer experimental comparative studies were carried out evaluating the performance of the developed algorithm. The purpose of the present paper is development of an efficient algorithm for mixing audio streams for processing on GPUs. The method of transfer operations computing on graphics processors with the use of Shader programs was developed. Novel features presented solutions is using a two-pass rendering. The results showed that the application of the developed algorithm leads to an increase in computational performance up to 6 times. Presented solution can be implemented as software in the telecommunications multiprocessor systems.
,
M i,t M t ui,t , где u k,t – the sample vector; i.e. the vector of samples obtained by microphone k in time t;
M t – resulting mixing vector.
The second stage is echo cancellation, which in its basic form consists in excluding the sample of the i-th device from the final vector [8]. This stage is represented in the form где M i,t – resulting mixing vector for the i-th device;
ui,t – sample of the i-th device.
For the correctness of the resulting vector M i,t , it is necessary that its dimension be equal to the dimension of the incoming vectors. However, after stages 1 and 2, the vector M i,t may be overcrowded, leading to unwanted noise [9]. In order to use the original vector of large dimensions M i,t , it is necessary to compress it for further use. This is done in the third stage by the formula
M i,t M i,t fraci,t ,
(
A block diagram of the basic mixing algorithm is shown in fig. 1.
(
n M t uk,t k0
M i,t M t ui,t M i,t M i,t fraci,t i = [0…n]
n1
mi u j ui fraci . (
Since the fourth step uses a different measurement than the first three, it cannot be combined within the framework of formula
(
Begin
Processing of source
samples n1 mi u j ui fraci j0
Max(M i,t )
End
fraci,t
Usually, calculations can be combined according to general criteria when their projections do not coincide and they do not have intersections. Therefore, if these calculations are denoted as A1=(Xl,P1,f1) and A2=(X2, P2, f2), then they can be combined as shown below.
f1 ( X 1, p), p P ,
X X1, X 2 , P P1 P2 , f ( X 1 , X 2 , p) f 2 ( X 2 , p), p P1 . (
As seen from (
The developed algorithm presents an alternative method for executing a large number of functions, which outputs data of different lengths to a single render buffer using multithreaded rendering. Here the main rule is to move the projection by modifying the projection matrix, so that the computational area of each function is limited to the required area, rather than the entire buffer.
Depending on the pixel, it can have a different amount of information without loss of versatility. w denote the total number of
pixels needed to store L bytes. Thus, the render buffer can be represented as (w+1)*n. 3D model of the developed algorithm is a
rectangle, which lies in the plane Z. It has the dimensions: 2w/(w+1) units in width and 2 units in height. Coordinates of vertices
– (-1,-1,0), (1-2/(w+1),-1,0), (
At the first rendering pass, a unit matrix is chosen as the projection matrix. This ensures that the projection matches the 3D
model. After converting the visible area, in this pass the formula (
X X 1 , X
2
P P1 P2 f ( X1, X 2 , p) f1( X1, p), p P1
f2 ( X 2 , p), p P2
In the second rendering pass, the projection is shifted to the left by L pixels. At the same time, the shader program switches to the search mode of the maximum sample. In this pass, only one column can be written, since most parts of the projection lie outside the render buffer and will be automatically ignored by the GPU. Since the clipping of the projection was performed at the beginning of the render, this method calls the function only for the correct pixels, and not for the entire buffer. The actions performed on the second pass of the algorithm are shown in fig. 4.
As noted above, both pass algorithms requires as input data samples every n sequences. Each sequence must have its own unique texture. Total needed n texture dimension L. This multi-textural technology is not suitable for audio mixing. First, each sound source requires its own texture. Therefore, additional passes may be required. Secondly, loading a lot of small textures is much slower than loading a single large one.
In the developed algorithm, it is proposed to load a single texture. As an example, the first pass of the algorithm is presented. The input contains n samples of sequences of size L bytes and n attenuation coefficients. Two textured RGBA format buffers are used. Texture T1 has the dimension [L/4]*n and stores all sample sequences in a line. Texture T2 has a dimension of 1*n and stores the attenuation coefficients. The coordinates of all the textures are given in Table 1.
Audio mixing is performed independently for each pixel. The pixel's texture coordinates (x, y) are calculated by interpolating the vertex texture coordinates. Based on Table 1, the texture coordinate for T1 or pt.t0 should be (X, Y), and for T2 or pt.t1 – (0.5, Y). Pt.t0 refers to the sample for which the current mixing transformations are made, this sample is called the "aiming point". To exclude the appearance of an echo, the other texture coordinate of ptCur is restricted by accessing T1 instead of pt.t0. The x component of the ptCur texture is identical to the pt.t0 texture, and the y component is calculated from a cyclic variable that designates each sound source. In the loop, the position register v is used to skip the "aiming point". Finally, the attenuation coefficients are read from the texture pt.t1. Since the coefficient is stored in the first byte, the sample is produced only by the blue component.
Test input data for mixing are sequences of 320 samples. All samples are generated randomly. Test bench configuration: CPU Intel Core i3-4130, 4 GB RAM, graphics card NVIDIA GeForce GT730. The number of M sequences was varied. The results for the basic and developed algorithms at the output are identical. The results of an experimental study of the dependence of the mixing time t on the number of sequences M are shown in fig. 5.
As can be seen from the test results shown in Fig. 5, application of the advanced algorithm of audio mixing allows to increase productivity of computer system in 5-6 times [9].
Table 2 shows the results of measuring the average running time of the algorithm t for 8 random sequences of 320 samples. During the measurement, 1000 processing cycles were carried out, for each of which new random sequences of samples were generated. For the results obtained, the dispersion of the output sequences is calculated, the values of which are also given in Table 2.
As can be seen from the test results presented in Table 2, the variance of the output sequences for the basic algorithm is substantially higher than for the developed one [12,13]. This is due to the fact that 32-bit numbers are used for the calculations in the GPU, whereas for the calculations on the central processor – 64-bit. Application of the developed algorithm in a heterogeneous computer system reduces the processing time to 0.22226 x 10-3 s instead of 1.351 x 10-3 s – the time of data processing by the basic algorithm.
This paper presents an algorithm for the extended downmix audio streams for computing on GPU. Its main advantage is the combination of multiple stages of mixing by using a two-pass rendering, which significantly reduces the switching time between buffers. The use of one texture for calculations increases the efficiency of I/O operations. Although I/O operations take approximately half the computation time, experimental studies of the developed algorithm showed an increase in performance up to 6 times.