=Paper= {{Paper |id=None |storemode=property |title=Mathematical Optimization Based Channel Coding: Current Achievements and Future Challenges |pdfUrl=https://ceur-ws.org/Vol-750/yrs02.pdf |volume=Vol-750 }} ==Mathematical Optimization Based Channel Coding: Current Achievements and Future Challenges== https://ceur-ws.org/Vol-750/yrs02.pdf
 Mathematical Optimization Based Channel Coding:
   Current Achievements and Future Challenges
                                   Michael Helmling# , Stefan Scholl∗ , Akın Tanatmis#
                                     ∗
                                     Department of Electrical and Computer Engineering
                                                 #
                                                   Department of Mathematics
                                 University of Kaiserslautern, 67653 Kaiserslautern, Germany
                                      {helmling, tanatmis}@mathematik.uni-kl.de
                                                scholl@eit.uni-kl.de


   Abstract— Channel coding and mathematical optimization—
two omnipresent branches of science which heavily influence
our everyday life, which is certainly unimaginable without the
epochal achievements of each of the two disciplines since they
affect nearly every communication system as well as every
transportation, manufacturing, and organization process. The
following report is dedicated to some of the achievements of a
research project decisively influenced by a cooperative interplay
of these two disciplines.

                    I. C HANNEL C ODING                                           Fig. 1.     TV transmitter and two receivers
   Channel coding is an important technique for the correction
of transmission errors that is used in nearly every commu-
nication system today. It is used in mobile phones as well
as in satellite communication, navigation systems, storage
devices like hard disks, CDs and also in internet or broadcast
applications. Let us first review what channel coding is and
why we need it. To understand the problems channel coding
                                                                                     Fig. 2.    TV pictures from A and B
solves, we will have a look at the old days. As an example
we take a communication system without channel coding that
becomes more and more obsolete: analog TV.                          contrast to the presented analog system, there are additional
   Television broadcasting is a form of one way communi-            possibilities for techniques to improve the link from the
cation. On the one hand we have a transmitter, which is             transmitter to the TV set at home. One of them is channel
usually placed on top of a mountain and transmits radio waves       coding. Channel coding is a technique to detect and correct
with high power to reach an area as big as possible. On the         occuring errors. But how is that possible?
other hand we have the receivers, ordinary TV sets, in our             Whenever a signal is transmitted in modern digital commu-
households, that pick up the radio waves.                           nication systems, the data is transmitted in form of bits or bit-
   If the connection between transmitting antenna and receiv-       streams, respectively. To allow error correction on the receiver
ing antenna is of good quality, we a see clear picture and          side, we have to make some preprocessing before transmission,
hear a comfortable sound. But what happens if the quality of        called encoding. To encode the data, the bitstream is first cut
the connection is not so good? Let us assume our receiver           into small blocks. At the end of every block some extra bits,
is placed far away from the transmitter, behind a mountain,         called parity bits, are added as shown in Figure 3. The k data
in the cellar or simply in bad weather, so that the reception       bits together with the r parity bits form a vector of n bits
quality drops. Then the picture is distorted or, even worse, the    which is called codeword the c.
whole picture bounces up and down. Also the sound might be
clicking and crackling, disturbing the viewers enjoyment. The
TV signal is noisy or distorted.
   Figure 1 shows such a situation where we have two TV
watchers, A and B. Whereas A is close to the transmitter and
receives a clear picture, B is not in such a good position. He
has worse receiving conditions due to serveral obstacles.                           Fig. 3.    A bitstream is being encoded
   The goal of new digital TV broadcasting systems is to make
picture and sound more resistant to distortion and noise. In          During encoding the additional parity bits are calculated
from the data bits. The “way” we calculate the parity bits                                                     0
                                                                                                              10

is called channel code and determines the performance of the                                                                                                   without
                                                                                                               −1                                           channel coding
channel coding system. Since data is processed block by block,                                                10




                                                                                     frame error rate (FER)
the channel codes described here are called linear block codes.                                                −2
                                                                                                              10
                                                                                                                                         less errors
A linear block code is defined by a binary r × n matrix H,
so that every bitvector c that fullfills                                                                       −3
                                                                                                              10

                                                                                                                                             less
                                H ·c=0                                    (1)                                  −4
                                                                                                              10
                                                                                                                                             transmit power


(where the calculation takes place modulo 2) is a codeword. H                                                  −5
                                                                                                              10

is called parity-check matrix. By C = {c ∈ {0, 1}n : H · c ≡                                                                   with
                                                                                                                          channel coding
                                                                                                               −6

0 mod 2} we denote the set of all codewords [1]. Finding a                                                    10
                                                                                                                                                                 uncoded

good channel code with its matrix H is non-trivial and subject                                                 −7
                                                                                                                                                                 with channel code

                                                                                                              10
to research since decades.                                                                                          0       2        4         6        8
                                                                                                                            signal−to−noise ratio (SNR)[db]
                                                                                                                                                                   10                12


   After encoding the data it is sent to the receiver. During
transmission some bits may be distorted and change their                        Fig. 5.                             FER for a system without channel coding and one using channel
                                                                                coding
values.
   The actual correction of corrupted data takes place in the
receiver by a device called channel decoder, that tries to
                                                                                   Using channel coding we can handle more weak signals
“repair” the incoming blocks. For this purpose the decoder
                                                                                without any loss of quality or provide much more reliable data
uses the previously added parity bits. In many cases this
                                                                                for a given signal strength. This advantage can be translated
technique works well and many errors can be successfully
                                                                                into equipment cost reduction, for example smaller antennas,
corrected.
                                                                                cheaper electronics or smaller batteries.
                                                                                   We now have seen which processing steps in channel coding
            transmitter        errors occur        receiver                     are necessary, namely encoding and decoding. The decoder is
                                                                                much more complex than the encoder and is the main part of
                                                                                a channel coding system. So now we will have a closer look
               encode                           decode
    data                   data +                             corrected         at the decoding process.
                           parity
                                                                                   In channel coding the data is processed in blocks of n
                                                                                bits consisting of k data bits and r = n − k parity bits.
           Fig. 4.   Encoding, transmission, and decoding of data               Because the parity bits are calculated from the data bits, they
                                                                                are fully dependent on the data bits. Therefore there are only
   However there is a probability that the decoder cannot                       2k possible codewords, although there are 2n possible bit
correct an erroneous block. This remaining error probability                    combinations. When a erroneous codeword is received, the
is called frame error rate (FER) and is dependent on the                        channel decoder checks which one of the 2k valid codewords
signal-to-noise ratio (SNR), which is the signal power divided                  is most similar to the received block. This process is called
by the noise power at the receiver. The FER is an indicator                     maximum likelihood (ML) decoding and is the optimal way of
for the quality of the channel coding system. It is measured                    decoding [1]. However, because all 2k codewords need to be
with Monte-Carlo simulation on a computer and visualized                        tested, this procedure is applicable only for small values of k.
with a special chart, shown in Figure 5. This type of chart                     With increasing k the complexity grows exponentially. Since,
has a typical shape. For weak signals, that is, low SNR,                        in practice, we deal with pretty large blocks (for example
the FER is very high (close to 1) and drops with increasing                     k = 100 or k = 10000), the ML decoding procedure of
signal strength. In Figure 5 there are two curves, the blue                     going through all 2100 or even 210000 codewords is simply
one represents a system without channel coding, the red one                     not possible to calculate in an imaginable amount of time.
corresponds to the same system with channel coding. One                            To overcome this problem engineers often use heuristics or
can easily see that the channel coding provides significantly                   suboptimal algorithms for the decoding procedure. These al-
less errors. The lower a curve lies in the performance chart,                   gorithms are very often capable of finding the correct solution,
the better the coding system works. In fact there are many                      but with much less complexity. In some cases, however, the
parameters that influence the performance of channel coding,                    suboptimal algorithms decide for a wrong solution. This leads
but the performance chart allows for a good comparison.                         to a small degradation in performance, which one has to accept
   In practical applications different error probabilities are                  if the problem should be feasible.
necessary. For audio signals, like in a mobile, a minimal                          Only if the complexity is low enough, the decoding al-
error probability 0.01 is required, otherwise crackling or                      gorithm can be implemented on a chip for use in practical
interruptions make a call uncomfortable. For communications                     applications. For many of modern codes, algorithms with
through optical fiber error rates of less than 10−12 may be                     a good tradeoff between performance and complexity are
necessary for flawless operation.                                               known. Moreover, for many used channel codes only the
performance with suboptimal decoding algorithms is known—           become a standard tool in business for a wide range of
not the one of optimal ML decoding.                                 applications such as planning, scheduling, routing or facility
   From a mathematicans point of view there exist other             location.
means of solving the ML decoding problem than comparing                Yet, this is not the end of the story—for many applications,
all 2k codewords to the received one. The decoding can be           linear programs lack some important capabilities, such as the
formulated as follows.                                              restriction to integral or binary decision variables.
   Assume that at the receiver a vector c̃ was received. It will       From this requests, the field of integer linear programming
check all possible sent codewords c out of C and decide for         evolved. An integer program (IP) is defined in the same way
the codeword c∗ which was most likely sent. Thus, the ML            as a linear program, but additionally requires that some (or
codeword c∗ is the codeword which maximizes the so called           all) of the variables take values in Zn instead of Rn . The
conditional probability p(c̃ received|c sent):                      increased power obtained by this extension vastly enlarges the
                                                                    number of practical problems that can be modelled by IPs—
            c∗ = arg maxc∈C p(c̃ received|c sent)            (2)
                                                                    at the cost, however, of efficiency, since it was shown that
   This problem is an optimization problem. In the following,       the general integer program is NP-hard to solve. Nevertheless,
we will present how decoding can be done with methods of            since this way of modelling is so powerful, a lot of research
integer optimization.                                               has been done to find and improve algorithms that tackle
                                                                    integer programs and make them solvable at least for moderate
     II. O PTIMIZATION M ETHODS FOR ML D ECODING                    problem sizes or specific well-understood classes of programs.
   Mathematical optimization is the task of finding an optimal         How can integer linear programming techniques be used to
solution for and the optimal value of an objective function         decode binary linear codes and to add some surplus to the
under the condition that some constraints are satisfied. Ob-        state-of-the-art knowledge in channel coding?
jective function and side constraints typically model some             Jon Feldman was one of the first researchers who addressed
real-world scenario, where the variables of the function are        this question intensively. To this end, he reformulated the ML
the abstraction of parameters like the per-day work time of         problem (2) and interpreted it as an integer programming
different machines or the path on which a truck drives from         problem: Under the assumption of a memoryless channel,
location A to location B. Depending on the task, “optimal”          i.e. the error distributions are independent for subsequent bits,
may be either minimal (if the objective function represents         it can be calculated as
some sort of cost or time) or maximal (e.g., if the function                                          n
                                                                                                                    !
                                                                                                     Y
                                                                                 ∗
value stands for the profit gained). The constraints are used to               c = arg maxc∈C           p(c̃i |ci )
model limitations of resources, working time, money, etc., or                                        i=1
                                                                                                                                    !
to avoid solutions that would be mathematically correct, but
                                                                                                                    
                                                                                                      X                 p(c̃i |0)
do not make sense in the real world—for example, a machine                         = arg minc∈C                ln                        .
                                                                                                     i:ci =1
                                                                                                                        p(c̃i |1)
cannot work −2 hours per day, and one cannot build 1.72
facilities.                                                         The quotient                                    
   Linear programming is the best-studied discipline of mathe-                                           p(c̃i |0)
                                                                                           yi = ln                                           (3)
matical optimization and it deals with optimization of a linear                                          p(c̃i |1)
objective function which is constrained by linear (in)equalities.   is called the log likelihood ratio (LLR) of the i-th input bit.
The domain of the variables is Rn for some n ∈ N and a linear       It only depends on the parameters of the channel, which are
program (LP) is then defined as                                     assumed to be known, and the signal c̃ that was received.
                                       n
                                       X                            So, yi is known to the decoder.P   From the above calculation it
                                                                                                        n
              min (or max) wT x =            wi · xi                follows that c∗ = arg minc∈C ( i=1 yi ci ).
                                       i=1                             ML decoding can be achieved by minimizing this linear
                  such that   Ax ≤ b                                objective function over all codewords c that satisfy (1); in
                              x ∈ Rn                                other words,
                                                                                             n
where w ∈ Rn is the cost (benefit) vector, A the m × n
                                                                                             X
                                                                                       min      y i ci
constraint matrix, b ∈ Rm the right hand side vector of                                     i=1
the constraints, and the variables are real-valued. Linear pro-                such that    H · c ≡ 0 mod 2
gramming has become famous since George Dantzig in 1947
                                                                                            c ∈ {0, 1}n .
proposed an algorithm to solve LPs as the above one, called the
Simplex algorithm. Despite the fact that the Simplex algorithm         Due to the modulo constraints, this is not exactly an integer
was later proven to have exponential worst-case complexity,         programming model of the ML decoding problem. However,
it is still the algorithm most used in practice. For the far        there exist several integer programming re-formulations of this
most problem instances, the Simplex method is very fast, and        optimization problem. For example, Jon Feldman introduced
outperforms polynomial-time algorithms that were developed          a formulation in which the number of both variables and
in the 1980’s. Since its invention, linear programming has          constraints grows exponentially in the number of ones per
row [2]. Among others, this exponential growth led us to                                                     +                   +      Out 1

the development and investigation of the following improved                                   +
                                                                               Input                    D         D         D
integer programming model with less constraints and variables
by the introduction of artificial variables z ∈ Zm (one for each                                                       +
row of H)                                                                                                                               Out 2

                               n
                               X
                        min          y i ci                                      Fig. 6.   A convolutional encoder with 3 memory registers.
                               i=1
                                                                    (4)
               such that      H · c − 2z = 0
                                                                          The constraints λT c ≤ λ0 , (λ, λ0 ) ∈ Λ, are derived from the
                              c ∈ {0, 1}n , z ∈ Zm
                                                     Pn                   current LP solution based on some combinatorial reasoning
   The j-th row of the constraint system reads          i=1 Hi,j ·        and the indicator variables z allow for a very fast generation of
ci − 2zj = 0. Since zj is required to be integral, this                   these constraints. Moreover, these constraints can be classified
equation ensures that an even number of entries of ci is set              with respect to optimization theory: although they are derived
to 1. So, indeed this is a simple, general, and sparse integer            in a combinatorial manner, they correspond to what is known
programming problem for the ML decoding problem [3]. For                  as Gomory cuts. Additionally, it turned out that the formulation
any given code specified by a parity check matrix H, the                  (4) offers another possibility which leads to an algorithm with
solution to this problem is the maximum likelihood codeword.              enhanced performance: The representation of the code in terms
   It should be emphasized that this problem is difficult to              of a parity check matrix H is not unique. In our separation
solve (it is NP-hard in general). Therefore, the integrality              algorithm, alternative parity check matrices of the code are
requirements on the variables c and z are dropped in order                generated during the execution as soon as the current one does
to get the so-called LP-relaxation of this IP which can be                not allow for further improvement of the solution.
considered an approximation of the original problem.                         To sum it up, this formulation enabled the development of
   Our IP formulation proved to be beneficial in several ways.            an algorithm which
   First, in [4], we solved this IP formulation by a commercial              • is applicable to any binary linear block code in contrast
all-purpose solver and compared its results with those of five                 to existing heuristics which are tailored to one specific
other IP formulations existing in the literature. To sum it up,                type of code,
our formulation had the fewest number of variables as well as                • yields a better error-correcting performance than other
constraints and the solution time was the least.                               algorithms, and
   Second, this formulation allows for an efficient procedure                • employs properties of binary linear codes to improve the
to approximately solve this problem (this is important if                      algorithmic treatment.
commercial solvers are not efficient enough or cannot be                  Besides this general purpose formulation, we have also in-
used due to some constraints). In [3], a separation algorithm             vented and investigated specialized approaches for an impor-
was proposed utilizing the structural properties of this IP               tant subclass of codes, as described below.
formulation. This algorithm works as follows: First, the LP
solution is computed and checked for integrality. If it is                III. L INEAR P ROGRAMMING FOR T RELLIS -BASED C ODES
integral, it is obviously also the optimal IP solution and the al-           A class of codes that have gained tremendeous interest since
gorithm terminates. Otherwise, the IP-formulation is improved             their invention in 1993 are the so-called turbo codes. A turbo
by adding additional linear constraints λT c ≤ λ0 , (λ, λ0 ) ∈ Λ          code is built by the parallel concatenation of two convolutional
where Λ ⊂ Rn+1 is a set depending on some polyhedral                      codes that are separated by an interleaver. Convolutional codes
properties of the code (which shall not be specified due to               have an inherent graph structure by their trellis representation
intended brevity of this article). However, these constraints             which faciliates the processes both of encoding and decod-
ensure that if the LP obtained by adding this constraint is               ing. This structure is partly inherited by the more complex
solved, then the previous LP solution is infeasible and the               turbo codes, which allowed us to invent a powerful integer
new LP solution is closer to the IP solution. Applying this               programming formulation, as well as several approximation
idea iteratively yields an algorithm which approximates the IP            algorithms, for this special set of codes.
solution. Consequently, the LP to be solved repeatedly is of                 The encoder of a convolutional code is built by a number
the form                                                                  of delay shift registers (D) and several XOR (binary addi-
                          Xn                                              tion) gates, as shown in Figure 6. The registers comprise
                     min      y i ci                                      the memory of the encoder, and the (binary) content of the
                         i=1                                              registers define its state. Thus, an encoder with k registers has
            such that    H · c − 2z = 0                                   2k possible states. By the XOR gates, the current state of the
                         λT c ≤ λ0            (λ, λ0 ) ∈ Λ                encoder, together with the current input bit, determines both
                                                                          the two current output bits at Out 1 and Out 2 and new state
                         0 ≤ ci ≤ 1,            i = 1, . . . , n.
                                                                          for the next step. This process can be visualized by the trellis
                                                                          graph of the code, where each vertex represents a state at a
                0                 0                                       cost path in the case of a convolutional code. However, we
        0              0              0        0          0           0
                                                                          can formulate the synchronized trellises as an integer linear




                              1
        1   1          1              1        1          1           1   program, where the interleaver constraints are of the form
                             0                                                                X           X
                                                                                                   xa =         xa .                 (5)
        2              2              2        2          2           2                          i             π(i)
                                                                                              a∈S1,1       a∈S2,1
                              1
        3              3              3        3          3           3            i
                                                                          Here, S1,1  are the one-arcs in the first trellis for step i, and
                                                                            π(i)
            Fig. 7.         Excerpt from a trellis graph with 4 states.   S2,1 are the one-arcs in the second trellis for the step to which
                                                                          the interleaver maps i.
                                                              Out 1          We have run numerical simulations, showing that this for-
                                                                          mulation leads to greatly reduced computation time compared
                    input                   Ca                            to the generic one given in (4). This is because only a rather
                                                              Out 2
                                      π                                   small part of the problem definition, namely the interleaver
                                                              Out 3
                                                                          constraints (5), separate the problem from the easy problem
                                            Cb                            of unconstrained shortest path computation. Lagrangian relax-
                                                                          ation can be used in such a situation of several “complicating
Fig. 8.     Turbo encoder with two convolutional encoders Ca , Cb and     constraints”. We have adapted this technique to the case of
interleaver π.
                                                                          turbo codes [5]. The idea of Lagrangian relaxation is to remove
                                                                          those constraints, but penalize their violation in the objective
specific step, and there are two types of arcs for zero and one           function. In the context of turbo coding, this means that we
inputs, respectively, that model the transition of one state to           allow the two paths to use different types of arcs in the steps
another (see Figure 7 for a trellis with 4 states). One usually           that are interconnected by the interleaver, but each of this
assumes that initially all registers contain a zero, therefore at         violations increases the total cost of the path—the constraint
the first step there is only one possible state. Arcs representing        given in (5) leads to the term
a zero input bit are drawn in a dashed style, where solid arcs
                                                                                                                      
correspond to a one at the input.                                                                X            X
                                                                                            ηi        xa −         xa 
                                                                                                                      
   In a trellis, a codeword corresponds to a path from the initial                                 i            π(i)
                                                                                                a∈S1,1      a∈S2,1
state to the left to the end state at the right (which is not
visible in the figure). It is possible to assign a cost attribute         in the objective function which becomes larger the more the
to each arc, based on the LLR values from (3), in a way that              two sums differ.
the ML codeword can be found by computing the path of                        A central result of Lagrangian relaxation is that, for an
minimum cost—a task for which very fast algorithms exist.                 appropriate choice of penalty factors ηi , the optimal paths with
Therefore, for a simple convolutional code, ML decoding can               respect to the Lagrangian relaxation will converge to the LP
be accomplished efficiently.                                              relaxation of the problem, where all additional constraints are
   The encoder of a turbo code uses two convolutional en-                 satisfied. However, in general no integral solution is obtained.
coders. The first one processes the input bits as described                  One of our approaches to finding integral solutions is based
above. A copy of the input is sent into the second encoder after          on the following idea: Given the shortest path on one of
passing an interleaver π. The interleaver permutes the bits in a          the trellises, it is rather unlikely that the corresponding path
pseudo-random way before they enter the second encoder. This              on the other trellis (determined by the interleaver) is also
way, the input (and thus also the output) is different for both           the shortest path for that trellis. However, the optimal global
decoders. By this additional complexity, the theoretical error-           solution corresponds to the k-th shortest path on one of the
correcting performance of a turbo code is vastly increased                trellises, and under certain assumtions k will be relatively
compared to single convolutional encoding, as vital research              small k. We have adapted an efficient algorithm to compute
over the past decades has revealed. The downside of this is               the k shortest paths on a graph to the trellis scenario, and
that also the decoding complexity, the effort to compute the              showed that, for low noise and not too large trellises, this can
ML codeword for a given received signal, increases. Speaking              be an efficient approach—especially in conjunction with the
in terms of trellis graphs, a turbo code can be modelled by               modified objective function by the Lagrangian relaxation.
two trellises which are synchronized by the constraint that in
a given step, the path in the first trellis must use the same type                    IV. R ESULTS , I MPACT, & O UTLOOK
of arc (zero or one) as the corresponding step in the second                Remember that in practical communication systems the use
trellis which is determined by the interleaver function. The              of an optimal decoder is very often not feasible, because
ML codeword is now represented by two paths (one for each                 of its large complexity. So when implementing decoders on
trellis) of minimum cost that match this additional constraints           a chip suboptimal algorithms are used. They are much less
defined by the interleaver. Unfortunatelly, it turns out that this        complex than the optimal decoder, but also perform worse.
problem is much harder than the single, unconstrained min-                The algorithms are often iterative heuristics, that usually run
                                                          WiMAX LDPC Code, block length n= 96
                     0
                    10                                                                                       results. The study of different formulations also led to a new
                                                                                                             stopping criterion for a iterative decoding algorithm. Stopping
                     −1
                    10
                                                                                                             criterions abort the iterative process early, if sufficient itera-
                                                                                                             tions were made. Here we used a cost function to indicate,
                     −2
                                                                                   performance of            when the iterations can be stopped. This leads to a faster
                    10                                                                hardware
                                                                                    implementation           computation and a more power efficient decoding algorithm
 frame error rate




                                                                                                             in hardware.
                     −3                                                 gap
                    10        optimal performance
                               measured with our
                                                                                                                 The study of linear block codes gives rise to a lot of
                                    decoder
                                                                                                             new types of mathematical problems and structures that have
                     −4
                    10                                                                                       not been considered before, as well as interesting links to
                                                                                                             different mathematical fields (like integer programming, graph
                     −5
                    10
                                                                                                             and network theory, finite algebra, matroid theory). If we use
                                   without channel code
                                   suboptimal decoding for WiMAX code
                                                                                                             integer programming for ML decoding, we are in the situation
                     −6
                                   optimal decoding for WiMAX code                                           that, for a given code, we solve the same program over and
                    10
                          0           1            2           3              4
                                                                   signal−to−noise ratio[db]
                                                                                             5   6   7   8   over again, where only the cost vector varies. So far, only
                                                                                                             little attention has been given to this special case, which might
                                Fig. 9.     Comparison of suboptimal and optimal decoding                    allow for more efficient ways to solve the problem by re-using
                                                                                                             information that was collected in previous problem instances.
                                                                                                             As another example, by studying certain types of cyclic codes
a fixed number of iterations or abort earlier according to                                                   we encountered a new type of network flow problem, where
some stopping criterion. For example the decoding of LDPC                                                    the supply or demand of vertices is variable and thus a part
codes in practical applications is usually done with a special                                               of the problem itself, but restricted to be a multiple of 2.
iterative algorithm called belief-propagation, that is also used                                             Also, the application of coding theory might arouse interest in
in artificial intelligence [6].                                                                              constraints of the form Ax ≡ b (mod q) in integer programs,
   Finding decoding algorithms with a good tradeoff between                                                  which have not yet been studied extensively.
hardware complexity and communications performance is an                                                         The recently developed 3-dimensional turbo codes, which
important research area for hardware designers. To judge an                                                  are an extension of turbo codes that use a third convolutional
algorithm’s performance it can be compared to the optimal                                                    encoder in conjunction with a second interleaver, show that
decoding. This comparison also helps to estimate if further                                                  the development of good Lagrangian relaxation techniques for
research on new algorithms is promising or not. However the                                                  trellis-like graphs remains an important task for the area of
problem is that for most linear block codes used in practice                                                 convolutional coding.
the optimal decoding performance is unknown.                                                                                    ACKNOWLEDGEMENT
   Our aim is to provide ML simulation results for practical
channel codes with larger blocklengths. Furthermore with                                                       We thank Horst W. Hamacher, Frank Kienle, Stefan Ruzika,
mathematical techniques we like to get further insight in                                                    and Norbert Wehn for their constructive comments and sug-
channel codes and their decoding procedure.                                                                  gestions. We gratefully acknowledge financial support by the
                                                                                                             Center of Mathematical and Computational Modelling of the
   In chapter I we presented the optimal decoding problem
                                                                                                             University of Kaiserslautern.
as integer optimization problem. With help of the presented
techniques we built an optimal decoder, that is able to perform                                                                           R EFERENCES
ML decoding efficiently for larger block lengths. With this                                                  [1] S. Lin and D. J. Costello, Error Control Coding, Second Edition. Upper
decoder we measured the optimal performance of some short                                                        Saddle River, NJ, USA: Prentice-Hall, Inc., 2004.
                                                                                                             [2] J. Feldman, “Decoding error-correcting codes via linear programming,”
channel codes used in the recent communication standard                                                          Ph.D. dissertation, Massachusetts Institute of Technology, 2003.
WiMAX [7], which were previously unknown.                                                                    [3] A. Tanatmis, S. Ruzika, H. Hamacher, M. Punekar, F. Kienle, and
   Figure 9 shows a LDPC code taken from WiMAX with                                                              N. Wehn, “A separation algorithm for improved LP-decoding of linear
                                                                                                                 block codes,” Information Theory, IEEE Transactions on, vol. 56, no. 7,
a block lengths of 96 bits. The suboptimal performance as                                                        pp. 3277–3289, 2010.
it may be implemented on a chip is shown. Additionally a                                                     [4] A. Tanatmis, S. Ruzika, M. Punekar, and F. Kienle, “Numerical Com-
second curve is shown, which was simulated with the use                                                          parison of IP Formulations as ML Decoders,” in Communications (ICC),
                                                                                                                 2010 IEEE International Conference on. IEEE, 2010, pp. 1–5.
of our integer optimization decoder and shows the optimal                                                    [5] A. Tanatmis, S. Ruzika, and F. Kienle, “A lagrangian relaxation based
performance.                                                                                                     decoding algorithm for LTE turbo codes,” in Turbo Codes and Iterative
   One can see that there is a gap of about 1 db between the                                                     Information Processing (ISTC), 2010 6th International Symposium on.
                                                                                                                 IEEE, 2010, pp. 369–373.
algorithm used in hardware and what is theoretically achiev-                                                 [6] T. Richardson and R. Urbanke, “The Renaissance of Gallager’s Low-
able. This gap is quite large, if you consider that sometimes                                                    Density Pariy-Check Codes,” IEEE Communications Magazine, vol. 41,
a few tenths of a db are a good sales argument.                                                                  pp. 126–131, Aug. 2003.
                                                                                                             [7] IEEE 802.16, Local and metropolitan area networks; Part 16: Air Interface
   The work on ML decoding as an optimization problem                                                            for Fixed and Mobile Broadband Wireless Access Systems; Amendment
did not only lead to a better understanding of the decoding                                                      2:Physical and Medium Access Control Layers for Combined Fixed and
performance of the suboptimal algorithms and new simulation                                                      Mobile Operation in Licensed Bands.