Researchers often have to deal with the problem of data availability and privacy restrictions. This is why synthetic data generation is usually a solution for the deficiency of research data and a way to provide the researchers with the data fitting specific situations. This paper proposes a system of synthetic Call Detail Records (CDR) data generation based on a real-life dataset. Our approach uses statistical analysis to extract the users' behaviour from the real-life dataset. The generator is tested by simulating several unique situations, which appear in the real world. Since this kind of situations can be hard to capture into the dataset, we believe that the proposed system could be useful in many telco research fields.
Telecommunications play a vital role in people’s lives and provide opportunities for global
communication. Due to the increased use of smart technologies, this industry has grown
globally in recent years, and this growth is not expected to reduce for some time to come [
Call detail records (CDRs) are the records, which usually store the caller’s and the receiver’s
numbers, the date and time when the call starts, the time the call connects, and the time the call
ends [
The generator rules are designed in such a way to fit real-world data with the possibility to alter specific parameters and create diferent situations for various testing purposes.
The rest of the paper is organised as follows. Related work is provided in Section 2. The generator system and generated results are discussed in Section 3. The generated simulations of real-world situations are provided in Section 4. The conclusions are given in Section 5.
In this paper, the analysed data covered two main problems: synthetic real-world data generation and the problems that arise for telecommunication companies and researchers when analysing or working with CDR data.
Any human-related data collecting can be very time consuming and restricted by various
security measures. To overcome these dificulties, several synthetic data generation approaches
were proposed. The system described in [3] presents a framework that generates web
applications data from various layers, including networking, database queries and security checks. The
authors generated nine synthetic datasets with normal and attack data using multiple
diferent parameters. The data was used to simulate standard and malicious web application usage,
therefore testing various security and reliability layers. Another field of synthetic data
generation was introduced in [
In order to better understand what problematic situations occur when working with CDR
data, we analysed several typical areas of telco research. A number of research papers [
The goal of presented generator is to generate CDR data that is realistic enough to be used by researchers and telco companies. Moreover, we wish to simulate certain real-world situations which could be hard to capture even in manually gathered datasets. To achieve this, we created Original CDR data
Distributions analysis
CDR generator
Parameters from real-world situations
Synthetic CDR data a synthetic CDR data generator system, which is depicted in figure 1. First, to capture realworld users’ behaviour, the real data had to be collected. After this, we performed statistical analysis of the selected call record features, which our system will generate. Then, we carried out the analysis of possible problematic situations in telecommunications and implemented some of them in the generator system. Finally, the results were simulated according to the selected simulations rules.
In order to capture typical users’ calling behaviour, the analysis of real-world CDR data is needed. For this, we analysed data provided by a telecommunications company. The data consisted of over 20 thousand customers and six months of their activities. The real-life dataset had over 300 thousand call records collected in the year 2020. Moreover, it also contained failed (missed, denied or suspended for other reasons) calls records, which meant that we could precisely capture the call’s likelihood of success. Each call record had its precise date, caller id, receiver id, destination name and call duration. Our generator implements most of these features; however, since the data will be simulated in one country locally, we decided to replace the call destination with the receiver’s telecommunications company name. Also, to simulate the relations between diferent telco companies, each caller was assigned a company. The final feature vector is: Caller’s id, Caller’s company, Receiver’s id, Receiver’s company, Timestamp, Duration s.
Statistical analysis was used to extract features from the original dataset. The analysis consisted of several distinct parts: call durations, count and calling hour distributions analysis, most active weekdays analysis, finding the likelihood of the user making an international call and the probability of the call success. We used the distribution fit tests library fitdistrplus [12], provided by the statistical programming language R to tackle the distribution detection task. The fitdistrplus function estimates distribution parameters by maximising the likelihood function. After that, the results are displayed using Cullen and Frey (figure 2) graph, which depicts each distribution’s skewness and kurtosis.
The blue dot on the graph represents skewness and kurtosis of the tested distribution. The closer the dot is to the symbol depicting particular distribution, the better this distribution fits original data. To measure each distribution goodness-of-fit to the data, we considered four methods provided by the R library fitdist : 1. density plot, which shows the histogram of empirical data with fitted distribution’s density function, 2. empirical and fitted distributions CFD plot, 3. quantile plot (Q-Q plot) depicting the empirical quantiles (y-axis) against the theoretical quantiles (x-axis) 4. probability plot (P-P plot) representing the empirical distribution evaluated at each data point (y-axis) against the fitted distribution function (x-axis).
The methods described above were used to extract features from the original dataset. Based on the statistical analysis, we extracted the main features of each call record feature. The call durations ranged from 1 s (all unanswered calls were left out of the sample) up to 7000 s, and the median of call durations was 247 s. Each user’s number of calls ranged from 1 to 233 calls, with a median of 5 calls per user. The most regular calling hours were 7 p.m. to 8 p.m.; moreover, the day’s busiest period was from 10 a.m. to 9 p.m. Finally, the number of calls made during each weekday was similar; therefore, the weekdays’ distribution is uniform. The estimated theoretical distributions of each feature are given in the table 1.
The percentage of successful calls was 70%; thus, the estimated probability of the call success was set to be (callSuccess) = 0.7.
The generator was implemented using Python programming language. There are five distinct classes: local operator, international operator, local customer, international customer and CDR record. The parameters for each of these classes are provided in a .json format configuration ifle. That file contains basic information about the generation: total customer number, total operator number, generation time interval and max number of local and international friends that one user can acquire.
After the parameters are adjusted, the generator starts creating objects accordingly to the given five classes. First, it creates operators with the market share parameter. The more market the operator holds, the more customers it has. Market shares allocation is implemented in such a way that one company could not have more than 70% of all the customers. Next, the international and local operators and customers are created. Each customer is assigned a local operator. Since we decided to generate only local companies’ CDR data, a customer cannot be assigned an international operator. After all the customers had been assigned an operator, the generator starts building a network of people. Each customer can have a number of friends and acquaintances. Both numbers are chosen randomly from the interval from 1 to max number of friends/acquaintances. After that, a customer can make a call to a friend, acquaintance or the person not in his network with the probabilities: (callFriend) = 0.5, (callAcquaintance) = 0.3 and (callOther) = 0.2. Finally, the rest of the call record features are generated randomly according to the estimated distributions.
The data generation experiment was performed with selected parameters: 10 000 customers, three operators, max friends number: 5, max acquaintances number: 10, start date: 2020-12-04. A sample of generated CDR results is given in the table 2.
When analysing generated data, we mainly focused on the features’ samples from estimated distributions 4. Each of the four chosen call record features followed the distributions of realworld data almost precisely. However, there were some inadequacies noticed when analysing calling hour and weekdays. Even though the busiest calling hours in the collected data were 7 p.m. and 8 p.m., the model almost always generated most calls during the mid-day period (from 2 p.m. to 7 p.m.). The generated weekdays sample is close to being distributed uniformly; however, most of the time, the end of the week is busier than other weekdays.
To better understand what problems might occur in telecommunications, we asked the telco experts to consult us on the topic. According to them, there are four most common situations which are considered problematic in the telco field: 1. Increased load in the whole area. The best example of a situation like that would be new years eve. The call number in the CDR data is increased for a short time period. 2. Increased load at one certain place. These situations arise in the places of crowd events, for instance, a concert. In this case, one telecommunications station has to allocate more calls then it normally should. Therefore, it starts crashing, and the calls fail. 3. A natural disaster, for example, an earthquake. In such a case, there will be bigger trafic to that location; moreover, due to possible high numbers of casualties, many calls might also fail. 4. Cell tower failure. The consequences of this type of situation are similar to those caused by natural disasters.
Only two situations were chosen to test the generator: increased load in the whole area and the cell tower failure. The other situations were not included since they required the knowledge of an accurate user location, which the CDR data does not provide.
For this experiment, we chose to simulate the CDR workflow during New Year’s Eve. The main change in this situation is the increased call amount made by every user. The increment for each user was chosen randomly using the normal distribution. The parameters for the normal distribution were chosen according to the equations (1, 2). Let friends and acquaintances be the number of friends and acquaintances respectively for every user. Then the parameters of event calls number distribution would be: (1) (2) (3) (4) mean = min = 0 friends + acquaintances
2 max = friends + acquaintances std = 1,
The call record timestamp was also generated using the normal distribution with the parameters shown in the equations (3, 4), where event is the event duration. The day and calling minute was chosen uniformly during the event period. The distribution parameters and the rest of the variables were determined by a series of experiments. However, the actual generator has the possibility to adjust the model for it to fit certain other situations by fixing the parameters in the input files.
mean = event , std = event
, min = 0, max = event 2 4
The simulated event started at 8 p.m. and ended at 4 a.m. Figure 5 shows the resulting distributions of the newly generated features for 1 000 users. The majority of the users made less than 25 calls during the whole event, and most calls were made at around midnight.
The failed station simulation was created in the following steps. First, the company responsible for the failed station is chosen at random. Since we do not generate user locations, we have to assume that a certain number of users will be in the failed station range. To simulate that, we introduce a parameter representing the customers’ percentage, one or more failed stations has to service. This parameter will be the probability that the customer is in the station range. After that, we start simulating the failed calls. At the given time period, with the specified probability, we select calls from the CDR list and change their duration to 0s.
The simulated situation had the parameters: station_size = 0.3, start_date = 2020-12-06, end_date = 2020-12-10. The size of the failed area was chosen so large for a better demonstration. We calculated the percentage of failed calls during normal days and the event period to analyse the results. During the four days that event lasted, the number of failed calls increased by 70-90%. Figure 6 shows the failed calls’ histogram during normal days and the event period.
Even though only one company’s tower failed, this failure afected every company in the country. Whilst the companies with broken tower failed calls increased by 60%, failed calls in other companies had also increased by 30%. This phenomenon occurs because the customers from other companies might also call to the failed tower area. Whereas the people in the failed cell tower’s range has no connection, the call can not reach its destination and fails.
This paper proposes an idea of abstract synthetic data generation tool for call records generation. The system can be based on any telco operator data and generate simulations according to the prefered parameters. The concrete system described in this paper was based on the real-life telecommunications dataset, which was analysed using statistical analysis. We used distribution fit tests to extract monthly call number, call duration, calling hour and most popular weekdays from the dataset. The parameters were then implemented into the generator, and the CDR data was simulated according to three distinct situations: 0 - normal data flow, 1 - increased load in the whole area, 2 - cell tower failure. The motivation for this kind of tool arose from the need for distinct situations when analysing telecommunications data. Our generator focuses on simulating situations, which could be detected only from CDR data without additional information about users or telecommunication services providers. The generator’s source code and the instructions for others to use and expand the system are provided in the GitHub repository: https://github.com/CARD-AI/CDR-Generator.