This paper addresses the problem of teaching statistics to future programmers. It argues that the theoretical content of teaching statistics needs to be updated and oriented towards the practical field, even at the higher education level. It suggests that the teaching of statistics to students should move from theoretical methods to practical solutions of applied problems and emphasize the analysis and interpretation of results rather than the statistical calculations. The paper proposes a system of tasks based on real data sets obtained from statistical research as a way of improving the learning of statistics for future programmers. It shows that such tasks can increase the students' motivation compared to synthetic examples, which are commonly used in statistics courses. The paper also reviews the software tools for statistical data analysis and identifies their features and advantages for the learning process. It recommends using R, a specialized programming language, as the main tool for teaching statistics.
The rapid growth of information in the modern world poses a challenge for the statistical education of
society. Statistics is an essential component of the educational programs for training specialists in the
ifeld of IT [
The discipline of statistics has been undergoing significant changes and developments in recent
years. Cox [
Many researchers have investigated the issues and challenges of teaching statistics. They have
provided recommendations for teaching statistics in diferent types of educational institutions [
Nicholl [
Education is seen as an investment in human capital and production. The World Economic Forum predicts that the global demand for statistical data analysts will increase by almost six times in the next ifve years.
According to the Modis survey [
Every day, large amounts of various data are generated and accumulated in the world [
Therefore, improving the education of students in statistics requires moving from the theoretical methods to the practical solutions of applied problems and shifting the emphasis from the statistical calculations to the analysis and interpretation of results.
To prepare an intellectually active, knowledgeable and skillful specialist, education needs to move
from the reproductive to the innovative learning. Innovative learning is a creative combination of the
traditional and new teaching methods, tailored to each discipline, based on its theoretical content and
practical orientation [
To organize innovative teaching of statistics in accordance with modern requirements, it is advisable to use special software tools for statistical data analysis. However, there are also specialized programming languages and environments that can be used to analyze data, interpret results and prepare conclusions and reports in various formats quickly and eficiently.
Therefore, there is a contradiction between the traditional approaches to teaching statistics and the society’s expectations for the level of training of modern IT specialists in the field of statistical data analysis, as well as between the theoretical orientation of the content of statistics teaching and the need to train a specialist with applied tools and methods of statistical data analysis.
The aim of this paper is to justify the use of R programming language as a teaching method for learning statistics.
The following main methods were used in the research process: content analysis of scientific and methodical literature, generalization and systematization to clarify the state of the problem development; questionnaire of those getting higher education and initial statistical processing of the obtained results to clarify the current state of the researched problem; generalization of theoretical and practical data to justify the introduction of innovative approaches to the study of statistics by students based on the use of programming language R.
The process of teaching statistics to the students is associated with certain dificulties: the study material in this course contains a large number of definitions and formulas. At the same time, students need not only to reproduce them, but also to understand the meaning and be able to apply in practice. However, with the traditional organization of the educational process, practical tasks are far from the real economic, social and other processes that occur in real life. The analyzed data are generalized and do not allow to fully form students’ understanding of the need and expediency of studying this discipline and the opportunity to implement the acquired competencies in their further professional activities.
Therefore, most students learn statistics in fragments, and do not form systemic knowledge as a
result. In addition, mainly verbal presentation of information increases fatigue, resulting reducing
productivity of the learning process [
The number of statistically educated people is decreasing. It is dificult for potential employers to find a specialist who will be able to perform statistical calculations without prior training and explanation. Therefore, there is a need to improve the content of teaching this discipline through the introduction of practical tasks.
Improving the content of the statistics course requires the introduction of changes in the methods and means of its teaching using innovative technologies.
Scientific innovations that promote scientific progress cover all areas of knowledge. There are socioeconomic, organizational and managerial, technical and technological innovations. One of the types of social innovations is pedagogical innovation.
Pedagogical innovation is an innovation in the field of pedagogy, purposeful progressive changes that
make stable elements (innovations) in the educational environment that improve the characteristics of
both – its individual components and the educational system on the whole [
Pedagogical innovations can be carried out both with the application of the educational system’s own resources (intensive way of development) and with the involvement of additional capacities (investments) – new means, equipment, technologies, capital investments, etc. (extensive way of development).
Kazakov [
The main ways and objects of innovative transformations in the teaching of statistics are:
• making concepts and strategies for the development of statistical education [
Innovation can take place at diferent levels. The highest level includes innovations that afect the entire pedagogical system.
Kulinenko [
The current problems of teaching statistics in modern higher educational institutions include the
review of experience associated with the intensification of learning. One of the main teacher’s tasks
is to teach students to obtain the necessary information independently, to teach them to consciously
process the obtained information [
Possibilities of “Statistics” discipline for experts in the field of IT consists first of all of that knowing mathematical language and modeling that will allow the student to be better guided in forecasting of economic, social, technical and other processes; secondly, that statistics by its internal nature has rich opportunities for the formation of students algorithmic thinking.
Future IT professionals must not only know the theoretical foundations, but also be able to apply the means of automating statistical analysis. Such tools include specialized statistical software packages and programming languages.
Statistical packages on the basis of functionality can be divided into 3 main groups. 1. Universal statistical packages Statistica, SPSS, Statgraphics, STATA, Stadia, SYSTAT, S-PLUS and MS Excel. These packages are not targeted at a specific subject area and can be used to analyze data from diferent industries. Typically, they ofer a wide range of statistical methods and have a relatively simple interface.
It is recommended to work with such packages for starter users who have only basic knowledge in the field of statistics, as well as experienced users in the initial stages of working with data, when statistical methods that will be used to address a particular issue are not clearly defined yet. The versatility of the universal package allows holding a pilot analysis of diferent data types using a wide range of statistical methods. The vast majority of existing universal packages has much common functionality and is similar in the composition of the built-in statistical procedures. 2. Professional statistical packages such as SAS or BMDP. Professional packages, in the contrast to the universal ones, allow you to work with extremely large amounts of data, apply highly specialized methods of analysis and create your own data processing system. As a rule, such packages are complex and should not be used in the educational process. 3. Specialized statistical packages BioStat, Datastream, Datascope, etc. were designed for statistical analysis in specific areas of activity, which use special methods of statistical analysis, usually not presented in the universal packages.
Specialized packages allow analysis using a limited number of specialized statistical methods or are used in a specialized subject area. As a rule, such statistical packages are handled by specialists who are well acquainted with data analysis methods in the field to which the package is focused. For example, the BioStat statistical package was created to analyze data in the field of biology and medicine.
Most of the existing statistical packages have a flexible modular structure that can be supplemented and expanded owing to the custom modules that are optionally purchased or freely available on the Internet. Such flexibility allows you to adapt packages to the needs of a particular user.
Statistical packages are just the tools for an experienced professional. If the specialist does not have suficient knowledge and competencies, then, even the most advanced software product will not allow holding quality data analysis. However, the wrong software, which does not contain the required set of statistical procedures, can make the work of even an experienced specialist more dificult.
Therefore, during the training of IT specialists it is necessary to acquaint those who get higher education with the available statistical packages and their characteristics, but the application of specialized programming languages is closer and more understandable for the students while conducting statistical data analysis.
For statistical data analysis it is possible and appropriate to use R and Python programming languages.
We will consider the features of the programming language R. The language R is a powerful high-level object-oriented programming language and environment for statistical calculations and visualization of source and calculation data, which allows you to solve many problems in the field of data processing. It’s a free open source program under GNU GPL designed to run common operating systems (Windows, macOS, Linux).
Tens of thousands of specialized modules and utilities have been developed for this language. One of the most important features of the programming language R is the eficient implementation of vector operations, which allows the application of compact notation while processing large amounts of data. All this makes R an efective tool for obtaining useful information from large amounts of various statistics, including Big Data.
The R language is a convenient and efective tool for teaching statistical analysis, data processing and visualization.
It is also possible to use the Python programming language in the field of data analysis and interactive research calculations with results visualization. Python is an open source object-oriented programming language. The relatively recent advent of improved libraries for Python (primarily pandas) has made it a serious competitor to the R language for statistical data analysis. Combining with the benefits of Python as a universal programming language makes it an excellent choice for creating data processing applications.
So, the use of a specialized programming language as a learning tool contributes to the development of statistical data analysis skills as well as the development of algorithmic thinking of future IT professionals.
In order to study the relevance of the problem of scientific research, a ascertaining experiment was conducted among students of IT specialties. The issues that allow finding out the opinion of higher education students on the problem of improving the methods of teaching statistics to future IT professionals were studied.
The results of the ascertaining experiment are presented in percentages and indicate the number of positive answers to the questions. The survey was organized using Google Forms. 83 students majoring in 015 Professional Education (Digital Technology) and 015 Professional Education (Computer Technology) took part in ascertaining experiment.
In this block students were asked two questions. You can see the results of the answers to the first question of the survey in figure 1. The analysis of answers allows establishing the level of awareness of students in the demand for specialists in the labor market who know how to analyze data.
Analysis of students’ answers allows us to conclude that the majority of respondents, 42.17% believe that a data analysis specialist is in demand in the labor market. This confirms the relevance and need to study the course of statistics for IT professionals.
The second question clarified which specialties in data analysis students consider the most relevant today. The results of the student survey are shown in figure 2.
The most famous profession among the future students programmers is the profession of data analysts (65.06%), in second place is the profession of data scientists (51.81%). These professions are known to more than 50% of students, which indicates their awareness and interest in this field.
So, based on the results of studying the answers to the questions of this block, we can draw the following conclusion. Training statistics of future IT professionals is relevant, because students are aware of the existence of professions in the field of data analysis and believe that they will need statistics in future professional activities.
Students were asked to answer open-ended questions: “Which subject area data analysis you are interested conducting in?” The students’ answers showed that the most popular data for processing are data from sociology, medicine, engineering, economics and biology.
Also, the idea of what data students are interested in working with in practice was studied. The results of answers to the questions are shown in figure 3.
Among the surveyed respondents, 71.08% believe that data obtained as a result of practical research and having an applied nature are most attractive for them. This indicates the need to develop practical and laboratory work based on real data obtained from statistical studies.
The purpose of the third block of questions was to study the opinion of respondents about the need and feasibility of using software and programming languages for statistical data analysis.
Students were asked the following questions: “Do you know programming languages with which it is possible to perform statistical data analysis (enter)?”, “Which software product interface is more user friendly for you?”, “Are you more interested in data analysis using special software or using a programming language?”
According to the first question, the opinions of the respondents were divided as follows: 55.42% indicated the programming language R, 28.92% indicated the Python programming language. Programming languages such as C++ (9.64%) and Java (6.02%) were also indicated (figure 4).
The obtained results allow us to state that the R language is the best known as a mean of statistical data analysis. So, we will use this programming language to solve application problems.
In choosing the convenience of the software package interface, respondents preferred MS Excel (56.63%), followed by Statistica software package (28.92%), followed by SPSS (14.46%) (figure 5).
So, the students will be asked to use MS Excel and Statistica for practical calculations.
According to the results of students’ answers to the third question of this block, the programming language (57.83%) was chosen by the students as the main tool for organizing the training of statistical data analysis (figure 6).
So, students in the class will be asked to use the programming language R as the main tool for practical calculations. MS Excel and Statistica will be used as aids in statistical analysis.
Taking into account and summarizing the results of the study, in our opinion, it is advisable to build the content and structure of the course considering the wishes of students. In practical classes, tasks that are of a real applied nature and based on real statistics should be considered. One of the main teaching methods should be a practical method of learning based on programming. The means of statistical data analysis in practical classes can be both software tools for data analysis (MS Excel and Statistica) and the language and programming environment R.
A system of tasks has been developed for the course. Let’s consider an example for training of the statistical analysis in the R environment. For carrying out the analysis we will take data from the website https://abit-poisk.org.ua, namely data concerning entrants for 2017. This site contains large amounts of data, for our example we will take only entrants who entered the Faculty of Physical and Mathematical Computer and Technological Education of Berdyansk State Pedagogical University in the specialty “Professional Education (Computer Technology)” and “Professional Education (Digital Technology)”, the level of “bachelor”.
A total of 31 applications were submitted for these specialties. We will analyze these data, using descriptive statistics in R and present the results using the most common graphs in R when analyzing this data.
Step 1. We set the name, specialty, id, total score of the external evaluation, status (budget / contract), then enter the data into the table. We will set the value in the form of vectors with the command <- c (’’vector_value1’’, ‘‘vector_value2’’,. . . ). We build the table from the received vectors by means of the command > studentdata. Commands for a table creation with the information about applicants: > last_name <-c(’’Shvachko’’, ‘‘Dybiaga’’, ‘‘Kartashov’’, ‘‘Sytosenko’’, ‘‘Filipenko’’, ‘‘Klimenko’’, ‘‘Veretelnik’’, ‘‘Diakov’’,’’Salionov’’, ‘‘Bagnuk’’, ‘‘Kombarov’’, ‘‘Baranovsky’’, ‘‘Kiseliov’’, ‘‘Sakun’’, ‘‘Bova’’, ‘‘Potapova’’, ‘‘Kobzar’’, ‘‘Sementsov’’, ‘‘Cybulka’’, ‘‘Teplov’’, ‘‘Mitushkin’’, ‘‘Kartinik’’, ‘‘Gavrylenko’’, ‘‘Trotsenko’’, ‘‘Panchukov’’, ‘‘Kyslynsky’’, ‘‘Sagirov’’, ‘‘Korobov’’,
Step 2. We will calculate the main statistical values: average, median, standard deviation, minimum
and maximum value. The results of the main statistical values calculation:
> y <- mean(rating)
> y
[
According to the results of the calculations, the following data were obtained: the average score of entrants with external evaluation is 153, the average diference between the scores of diferent entrants is 22 points, the lowest result (min) – 123 points, the best result (max) – 186 points.
Step 3. Let’s construct a histogram of frequencies for external evaluation points using the command > barplot (figure 7): > counts <- table(studentdata$rating) > barplot(counts, main=’’Frequency diagram’’, xlab=’’Rating’’, ylab=’’Frequency’’)
The histogram of frequencies shows that the largest number of entrants has a score from 139 to 142 points, as well as the fact that the vast majority has a unique score with EIT, which is no longer repeated.
Step 4. We construct histograms of points / frequencies with a normal distribution curve. With this purpose we use the command > box. We will build: on the -axis – the parameter rating, and on the -axis – the frequency of the score in the table (figure 8): > box() > library(plotrix) > x <-studentdata$rating > h <-hist(x, breaks=12, col=’’darkblue’’, xlab=’’ZNO score’’, main=’’Frequencies histogram with the curve of distribution ‘‘) > xfit <-seq(min(x), max(x), length=40) > yfit <-dnorm(xfit, mean=mean(x), sd=sd(x)) > yfit <-yfit * diff(h$mids[1:2] * length(x)) > lines(xfit, yfit, col=’’red’’, lwd=3)
The distribution histogram shows that the data on the scores of applicants are not the subject to the normal law of distribution. We have a lot of “average” entrants, i.e. those who passed the external examination from 135 to 145 points. There are also those who passed 165 points, i.e. entrants with a “suficient” level. There are very few who scored more than 180 points.
Step 5. We construct a diagram of the nuclear estimation of the density of values for external evaluation points using the command > box (figure 9): > box() > par(mfrow=c(2,1)) > d <- density(studentdata$rating) > plot(d)
The nuclear density estimation diagram shows that the highest density is observed in the range from 130 to 155 points. That is, in this interval, based on the graph, the values difer by 25 points, then, if you take the full table, they difer by 22 (see standard deviation).
As a result of solving applied problems using theoretical knowledge from diferent sections of statistics, students will not only master the skills of using statistical methods, but also develop the ability to interpret the results and predict the studied processes. It should be emphasized that the use of programming as a practical teaching method will allow students to improve their knowledge and skills in the field of programming as well as the use of algorithms and design patterns.
Using real data for statistical analysis, students will be able to understand the need and feasibility of statistical research in future professional activities.
One of the problems of using application tasks with real data is the selection and use of data sets. Much of the datasets are closed and inaccessible for free research and use. However, there are organizations that provide free access to data: • World Bank Open Data (https://data.worldbank.org/) provides more than 3,000 sets of economic and social data on various indicators. Data can be downloaded in csv and xml formats. The service supports API access, which allows you to automate data downloads using the programming language R. • The unified state web portal of open data (https://data.gov.ua/) contains 15 categories of data sets that are constantly updated. Datasets are available for download in Excel, csv, json and xml formats. All data are available from Creative Commons Attribution 4.0 International license. • The oficial page of the All-Ukrainian Population Census (http://database.ukrcensus.gov.ua) provides access to information on the population living in the country, socio-economic characteristics, and demographic indicators, level of education, national composition and language characteristics.
Datasets can be downloaded in txt, csv, html formats. • Open World Health Organization data repository (https://www.who.int/data/gho/). The site provides datasets on the health status of citizens of World Health Organization member states. Datasets are divided into over 100 categories. Data can be downloaded in Excel format or use the API for direct access to data. • UNICEF Dataset (https://data.unicef.org/) collected relevant data on education, child labor, child disability, infant mortality, maternal mortality, water and sanitation, pneumonia, malaria and more. Datasets are available in Excel and csv formats. • Registry of Open Data on AWS (RODA) (https://registry.opendata.aws/) contains data located on AWS servers. The service ofers access to over 200 datasets. There is a page with additional information, usage examples, license information, and more for each data set. Using the wide range of computing products ofered by AWS (Amazon EC2, Amazon Athena, AWS Lambda and Amazon EMR), it is possible to share data in the cloud. This allows users to spend more time analyzing data rather than collecting data. When using data sets hosted on AWS, it is necessary to consider the type of license of each specific data set, as they belong to diferent agencies, government organizations, researchers, businesses and individuals. • Data.gov (https://www.data.gov/) provides open data sets of the US government. The resource contains more than 200,000 data sets from various sources: federal agencies, states, counties, cities, etc. Data can be obtained in various formats, including Excel, csv, json, xml. • The GroupLens Research (https://grouplens.org/) provides several sets of movie ratings data provided by MovieLens users. The kits contain movie ratings, movie metadata (genre and release year), and user demographics (age, gender, and occupation). Such data can be used to develop a recommendation system based on regression analysis. • Open data sets Yelp (https://www.yelp.com/dataset) is a subset of our businesses, reviews, and user data for application in personal, educational, and academic purposes. Available as JSON files, use it to teach students about databases, to learn NLP, or for sample production data while you learn statistics. • Kaggle (https://www.kaggle.com/datasets) a social network for researchers, which provides access to various data sets for analysis and research. The convenience of Kaggle is that it is not just a data warehouse. Each data set brings together a community of researchers in which data are discussed and approaches to data processing are elucidated. • Google Public Data Explorer (https://www.google.com/publicdata/directory) provides access to more than 130 datasets submitted by World Bank, U. S. Bureau of Labor Statistics, OECD, IMF and other organizations.
All considered services provide access to open data sets. This allows you to fill the content of teaching statistics for future programmers with the tasks of applied direction.
Using programming language R and tasks of applied direction while training statistics with future IT specialists.
The main purpose of the pedagogical experiment is to test the hypothesis that the use of programming language R and applied problems in teaching statistics to the future IT professionals will help increase the educational motivation of students.
According to the hypothesis of the study, the experiment involved checking the level of motivation of students of IT specialties in the field of statistics based on the results of implementation of applied problems and programming language R. The experiment was conducted on the basis of Berdyansk State Pedagogical University. Students majoring in 015 Professional Education (Digital Technology) and 015 Professional Education (Computer Technology) took part in it.
Control and experimental groups were organized. In the control group, the educational process was carried out according to the traditional methods. This technique involved the use of specialized software (Microsoft Excel, Statistica, etc.) and synthetic tasks, the content of which did not take into account the specifics of future professional activities of students of IT specialties. The control group (CG) consisted of 42 students. The experimental group used application problems and the programming language R to solve them. The experimental group (EG) included 32 students.
During the formation of control and experimental groups, their alignment was carried out taking into account the initial level of educational motivation of students.
The success of the pedagogical research was ensured by the application of the standardized methods. This guaranteed the reliability of the results.
Experimental methods of teaching statistics of future programmers using professional tasks and programming language R was based on their application at all stages of learning: in learning new material as a motivating task, at the stage of consolidation, in independent work of students as a professionally oriented project.
An electronic learning tool has been developed for students programmers to provide information and methodological support for the statistics course. The development of an electronic tool takes into account students age and preparation level. The developed learning tool contains theoretical materials, tasks for practical implementation, visual materials with examples of the application of the programming language R, a guide to the commands of the R language and a list of recommended reading. The e-learning tool is available on the Internet at the link https://r.ktuni.bdpu.org/.
In order to test the efectiveness of the implemented experimental training, the level of educational
motivation was chosen as a criterion. To assess the dynamics of changes in motivation to study statistics,
future IT specialists used the method of Rean and Yakunin [
Positive motivation for learning ensures the successful formation of knowledge and skills. High positive motivation can compensate for insuficiently high abilities of students. With the right choice of means of motivation for learning, there is a positive pedagogical influence. Focusing only on “negative” motives (avoidance, fear of failure, fear) is always less efective than “positive” ones. In our study, we will determine the impact of the developed system of tasks on the level of educational motivation of students.
Table 1 presents the results of calculating the average scores for each type of educational motives on
the scale of Rean and Yakunin [
Statistical analysis allows us to conclude that at the level of significance = 0.05 the initial states of the experimental and control groups (before the experiment) coincide. At the end of the experiment, the levels of educational motivation difer.
So, the results of the study indicate that the hypothesis of the study was confirmed, namely the introduction of statistics of the R programming language and applied problems in the learning process helps to increase the level of educational motivation of future IT professionals.
This paper has provided a theoretical justification for the introduction of innovative approaches to teaching statistics. It has shown that the teaching of statistics to future programmers should be based on the use of applied tasks developed with real data sets obtained from statistical research. Such tasks can increase the students’ motivation and interest compared to synthetic examples, which are often used in statistics courses.
Real data sets for statistical analysis are a rich source of applied tasks. They are freely available on the Internet and cover various subject areas, such as sociology, medicine, engineering, economics and biology. Therefore, the development of practical and laboratory work for future IT professionals should include tasks that involve real data from these domains.
Using the R programming language to teach statistics to future programmers allows the use of a practical training method based on programming. This method engages students in familiar and relevant activities and develops their programming skills. Therefore, we propose to use R as the main tool for teaching statistics. MS Excel and Statistica software packages can be used as supplementary tools.
In further research, we plan to develop a methodology for implementing and applying R and Python programming languages for statistical data analysis.