Autoencoders (AE) are emerging artificial neural networks that learn efficient embedding of unlabeled data and have been considered in the design of end-to-end transceivers. However, AE-based end-to-end transceivers face with a major challenge of poor generalization ability due to the communication channel dynamics. In this paper, a message-passing algorithm (MPA) layer is incorporated into an AE to simultaneously enable coarse learning in training phase and adaptive reasoning in inference phase. Theoretical analysis is also conducted to demonstrate the effectiveness of the MPA layer, recommending that the proposed model is applicable in more general systems.
Furthermore, the channel variation will penetrate forward to the entire receiver neural
network part. In case of a fast varying channel, a well-trained receiver might as well become
obsolete quickly and receiving performance degrades soon.
● Loss of Important Connectivity: According to [
These three issues result in the poor generalization ability of the AE-based transceiver over the
dynamic and time varying wireless environment. Some prior works have developed some solutions. In
[
Although the aforementioned works can overcome the three issues to some extent, all of them demand high energy consumption, large controlling overhead, and none of them can meet the real-time latency requirement in future wireless communications. These issues motivated us to incorporate a MPA layer into an AE. Its existence can simultaneously solve the out-of-distribution (OOD) and outlier problems usually in the inference stage and reduce the communication and computation costs.
In this section, we first propose a new AE-based transceiver by inserting an MPA layer with the transmitter, i.e., MPA-AE. Then, we introduce the detailed training process, including the forward subiteration and backward sub-iteration.
2.1.
As illustrated in Fig. 1, we consider an AE-based transceiver that consists of a DNN-based transmitter and a DNN-based receiver. The transmitter and the receiver are connected by wireless channels. To adapt to the dynamic channel conditions, we insert an MPA layer between the transmitter and the communication channel. Without loss of generality, we assume that the channel state information is available at the transmitter, which can be realized by concurrent channel feedback or uplink/downlink channel reciprocity.
The MPA layer is responsible for conducting a linear dimension reduction transformation, whose coefficients are fine-tuned using an iterative algorithm with two sub-iterations. The first one is the forward sub-iteration that passes messages from the DNN transmitter to the DNN receiver through the channel. The other is the backward sub-iteration that passes message from channel layer to the output layer of the DNN transmitter. To better describe the working mechanism of the introduced transceiver, we first provide the key parameters throughout the paper, as summarized in Table I. F H N R T
Input feature matrix [ 1, … , ] Channel vector [ 1, … , ] Noise vector [ 1, … , ] Received signal vector[ 1, … , ]
Output feature matrix [ 1, … , ] Based on the notations, we elaborate the detailed training process in the sequel. 2.2.
Support vector machine (SVM) is a supervised machine learning model used for data classification, regression, and outlier detection. In general, an SVM model is composed of a non-linear dimension extension function (∙), a linear combination function ( ) = ∙ ( ) + , and a binary classification function sign(∙), where is the input data, is the weight coefficient vector and is the bias vector. The objective of SVM is to divide the data samples into classes to find a maximum marginal hyperplane. shows the detailed forward iteration with SVM. Specifically, the input of the MPA-layer is an Lvia communication channels, the received signal is given by dimension. The output of the MPA-layer is an N-dimensional feature matrix dimensional feature matrix = [ 1, 2, … , ], where is the i-th input feature vector with K= [ , , … , ] where is the i-th output feature vector with K-dimension. When the output feature vectors are transmitted =1 = ∑ , ∙ ∙ + , = 1, … , , where , is the coefficient of the connection between neuron l and neuron i.
Based on the above description, we can conclude that the forward sub-iteration is to keep fine-tuning the hyper-plane of the SVM model in both training and inference phases for given transmitter’s feature matrix , channel state information , noise vector , and received signal .
Note that the MPA layer is mathematically differentiable. Once the coefficients , are fixed, it can pass the BP gradients from the receiver side to the transmitter side during the training stage. 2.3.
As we discussed earlier, the MPA layer needs to be trained by a standalone mode rather than a connection mode with back-propagation from the receiver. In this regards, we consider to use an attention-DNN in the backward sub-iteration. dimensions. Fig. 3 depicts the structure of the attention-DNN. The input is the received signal . The attention operation is conducted by computing the inner product of each with an attention coefficient , i.e., ⟨ , ⟩. This inner product implies the similarity of the signal and the attention coefficient , which is normalized by a softmax layer as We shall note that the number of attentions are less than the number of received signals, i.e., < .
Attention-DNN can be employed in the MPA layer for back-propagation. Specifically, each extracted feature vector can be used as an attention coefficient. Then, in the backward sub-iteration, the coefficient , can be given by
the inference phase. 3.1.
In this section, we propose two algorithms for the AE-based transceiver in the training phase and The training of the AE-based transceiver includes two parts. One is for the MPA layer in a standalone mode and the other is for the DNNs in the transmitters and receivers by BP. The detailed procedure is summarized in Algorithm 1. 2. Initialize the batchsize . 3. For step from 1: T do
In tandem stage 1 Algorithm 1. The training algorithm 1. Initialize the coefficients of the MPA layer , = .
1 Compute Sample a batch of training messages = [ 1, … , ].
DNN-based transmitter computes = [ 1, … , ] based on the training messages . =1 = ∑ , ∙ , = 1, … , computes the decoded message.
Update the transmitter and the receiver by back-propagation.
= ∙ + , = 1, … , .
Send = [ 1, 2, … , ]to the DNN-based receiver via communication channels, as DNN-based receiver inputs the received signals = [ 1, 2, … , ] into the DNN and || || = √ 12 + 22 + ⋯ + 2 .
In tandem stage 2
For iteration from 1: M do
Compute = ∑ , ∙ .
Compute Update 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. Output , , = 1, … , ; = 1, … , .
3.2.
Once the training is finished, the AE-based transceiver is used for inference. In particular, the MPAlayer can help the transceiver adapt to the channel dynamics: when the neurons on the encoder neural network and decoder neural network are fixed, the coefficient , on the MAC layer could continue to adapt themselves in terms of the current physical channel condition. The detailed procedure is summarized in Algorithm 2.
Algorithm 2. The inference algorithm
Input：New messages . DNN-based transmitter computes = [ 1, … , ] based on the new message .
=
∑
=1 , ∙ , , = 1, … . , = softmax( , ), = 1, … , , = || || = √ 12 + 22 + ⋯ + 2 .
=1 = ∑ , ∙ , = 1, … , Compute Compute
Update 1, … , .
Endfor Compute
= ∙ + , = 1, … , . 10. DNN-based receiver inputs the received signals = [ 1, … , ] into the DNN and computes the
We consider the following simulation settings. The transmitter sends a block with 256 bits in each time slot through 16QAM modulation scheme without channel coding. The channel gain changes every 200 time slots by a random distortion, which follows ℎ +1 = ℎ + ∆ℎ , where the random distortion follows ∆ℎ ~
(0, 0.3).
The proposed MPA-AE is first pre-trained at the channel condition ℎ0 with a fixed SNR 10dB to learn all the neurons on the encoding and decoding neural network and the coefficients on the MAP layer. Each time that the channel varies, only the coefficients , at the MPA layer are fine-tuned, while the rest neurons are fixed.
For comparison, we simulate a pre-trained AE without the MPA layer as a baseline, whose neurons are fixed even when the channel has changed, and a retrained AE without MPA layer as another baseline, that is completely retrained whenever the channel varies.
The three frameworks use the same AE structure in which a fully connected neural network with one hidden layer of 16 neurons is used for both the encoder and the decoder and ReLU is used as the activated function for the hidden layer and Adam optimizer is used to train the AE. The only difference of MPA-AE to the rest AE is the inserted MPA layer.
In this paper, we proposed a MPA-AE structure and its corresponding algorithms to train the endto-end transceiver in the scenarios with time-varying channels. The MPA layer inserted between the encoder and decoder of traditional AE can be fine-tuned when the under-going channel changes from the one the transceiver is trained with. Simulations show the superior performance of the proposed method.
The MPA layer could be flexibly incorporated with the AE-based transceiver in many use cases, including single user scenarios and multiuser scenarios. Specifically, in the single user scenarios, the MPA layer can be used to design source coding scheme, high-order modulation scheme, massive MIMO scheme, and pre-distortion scheme, which can well adapt to the time-varying channel conditions. In principle, the DNN layers and MPA layer of a transmitter is to distort the input distribution to match the current channel distortion distribution. The non-linear DNN layers provide a quick and powerful non-linear distortion, while the linear MPA layer provides a quick and adaptive linear matching.
Moreover, the MPA layer can also be applied in multiuser scenarios for both uplink and downlink MIMO design and Multiple-Access design. More than one MPA-AE could share the same channel. Then, the MPA layer of each becomes autonomous coding design. These research aspects will be considered in future.
Another research direction is to use more complex channel models in the training phase. In particular, the channel models are generated by a DNN with the input of surrounding topological information. In this case, the MPA layer is still effective since the inference DNN of the channel is a non-linear function. However, the DNN-based channel model is often very large. Therefore, how to reduce the DNN model size is an interesting research topic for future investigation.