Crowd Intelligence System in Project Training of Students Galina Kurcheevaa, Georgy Klochkovb and Maria Lapinac a Novosibirsk State Technical University, K. Marksa 20, Novosibirsk, 630073, Russia b Novosibirsk State University of Economics and Management, Kamenskaya Str. 52/1, Novosibirsk, 630099, Russia c North-Caucasus Federal University, Prospect Kulakova, 2, 355000, Stavropol, Russia Abstract The authors propose modern developments in the field of organizing project training, setting goals and objectives. A new form of teaching students is considered. The novelty lies in attracting students of different specialties to the development of large projects and combining groups of performers on the topic of the project from students of the second, third and fourth years of study with the possibility of attracting graduate students. Moreover, project training can continue for several semesters. Students have the opportunity to acquire the skills of a programmer, website designer, developer, work coordinator, manager and consultant. The stages of organizing work for a project that has not only theoretical, but also practical importance for the Novosibirsk region are considered. The project participants have elaborated new streamlines of digital technologies that will be in demand with the development of information technologies. The students analyzed websites in all districts of the region, which made it possible to assess the availability of information for the population. One of the areas of project training allows one to identify the relationship between the visual and informational complexity of web pages and their accessibility and convenience for users. The organization of the crowd intelligence system allows uniting students for the implementation of collective intelligence technologies. Thus, the project participants assessed the indicators of the quality of life of the population in the digital environment, while various groups of the population were involved in the discussion on social networks. We can also use such a methodology to analyze regulatory documents and court decisions in order to search for contradictions contained in them. The crowd intelligence system is useful for digitalizing the anti-corruption expertise process in order to reduce the "manual" processing of materials. The proposed form of training allows you to apply the theoretical knowledge of students to solving practical problems, teaches how to distribute working time, manage work areas, and search for non-standard solutions. Moreover, project-based learning is an ongoing process and it helps in the selection of directions for graduate qualification works and topics for master's theses. Keywords 1 Project training, information technology, smart settlement, websites, crowd intelligence. 1. Introduction The process of digitalization and development of information technologies has caused the need to supplement classical forms of education with new active forms, including project-based teaching methods. Project-based training is training in the conditions of a joint, independent, meaningful activity of students organized by a pedagogical worker, culminating in the creation of a creative product, based on the use of problem, search, project, research, active teaching methods [1, 2]. SLET-2020: International Scientific Conference on Innovative Approaches to the Application of Digital Technologies in Education, November 12-13, 2020, Stavropol, Russia EMAIL: kurcheeva@yandex.ru (Galina Kurcheeva); klgeorge@yandex.ru (Georgy Klochkov); mlapina@ncfu.ru (Maria Lapina) ORCID: 0000-0002-0931-1775 (Galina Kurcheeva); 0000-0003-4911-2276 (Georgy Klochkov); 0000-0001-8117-9142 (Maria Lapina) ©️ 2020 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) 318 Due to this, for project-based learning, one should choose research topics that are important for the development of the digital economy at the level of the country, region, city and settlement. For project training, we selected topics related to the development of a “smart” or “digital” city and “smart” settlement, digitalization of law, artificial intelligence and the development of interfaces for applied tasks [3, 4]. The relevance of the selected topics is due to their focus on solving problems of improving the quality of life of the population based on the development of information technologies, the possibility of remote access to information services. At the same time, problems arise associated with the collection, processing and interpretation of large amounts of data, the selection of non- standard solutions for applicability to population groups and communities [5]. The research is innovative in nature and aimed at identifying new indicators of the quality of life and their assessment by different groups of the urban and rural population, at searching for contradictions in the interpretation of regulatory documents from the standpoint of digital law. Students of several directions at the Department of Automated Control Systems (ACS) of the Novosibirsk State Technical University (NSTU) participate in project training. 2. Purpose and objectives of the research The purpose of the study is to test new approaches to project-based teaching of students, the formation of independence in decision-making, the formation of a team spirit, mutual assistance in solving tasks, non-standard solutions in the process of collecting and processing information. For this purpose, we asked students to study the theoretical basis on the development and application of the crowd intelligence system and to choose a topic for research. The first research topic "Digital City" involves organizing work in such areas as the collection of official and unofficial information on the quality of life indicators, the distribution of tasks between project executors, collection, processing and interpretation of the results. The second research topic "Digital Law" requires the development of a database of regulatory documents to identify the propensity of a person to corruption. The third research topic "Development of interfaces for applied tasks" related to the organization of work on annotating images for machine learning; collection and systematization of results, verification of the correctness of work performance, assessment of labor costs and efficiency of executors and assessment of labor costs and efficiency of the group as a whole. Methods of synthesis, system analysis, calculation and analysis of quality of life indicators and other instruments used in the process of collecting and processing data served as a methodological basis of our research. Publications posted on Internet resources, final reports of the ministries of the Novosibirsk region, data from newspapers and magazines devoted to the infrastructure of rural settlements in the Novosibirsk region, as well as websites of the Supreme Court of Russia acted as an information base. 3. Literature review A review of the literature shows the importance of developing a project for “smart city” and “smart settlement”. There is a wide discussion on the terminology associated with the concepts of “smart city”, “digital city”, “developing city”, “smart settlement” and “digital law” in the scientific literature, [6, 7, 8]. While exploring the possibilities for improving the quality of life of rural settlements, which significantly lags behind the urban ones, we proceed from the assumption that both urban and rural populations can turn to Internet resources of national importance, designed to inform citizens about various spheres of life in Russia, such as the portal of the State services and others. In addition, a new direction is successfully developing, namely, the websites of rural settlements, which accumulate their own local characteristics and traditions. They contribute to improving the quality of life of the population of rural settlements. Therefore, to solve the highlighted problems, it is to choose crowdsourcing technologies, which also need to develop taking into account the possibilities of the Internet and social functions online. Crowdsourcing, in turn, makes it possible to involve a large number of specialists (students) of 319 various professional backgrounds with a wide geographical coverage in solving the problems and tasks of the project initiator at a relatively low cost. In general, crowdsourcing is an innovative way to get things done at the lowest cost using Internet technologies and the synergy effect of bringing together the intellectual potential of as many people as possible. The essence of crowdsourcing is to get the desired innovation from one idea with the help of a group of people united by a common goal. In any case, executors generate ideas on a voluntary basis. In project-based teaching, students independently distribute, perform and discuss all types of work in groups responsible for each task. The student himself evaluates the labor intensity and determines how much time he can spend on participating in a particular project without prejudice to learning. The crowdsourcing phenomenon is a worldwide trend. Finding new ways to develop innovation in any social area is very important. It is crowdsourcing that in the future may become a means of implementing economic, social and other civic initiatives, which is why it is so important in training [9, 10]. 4. Results The student team assigns tasks according to the following skills, namely Web Designer, Work Coordinator, Developer, Business Analyst, Web Programmer, and Project Consultant. 4.1. Smart city, smart settlement: digitalizing the quality of life This is the first streamline of the project, it required:  to develop and evaluate the indicators of the quality of life most demanded by the population  to assess the possibility of improving the quality of life of the population through the development of websites for local settlements, including rural ones The first stage of work included the collection of information on such indicators as the presence of preschool and school institutions, the presence of medical institutions, the presence of sports complexes and the presence of cultural institutions. An important indicator of the quality of life is also the availability of a water supply network (km per 100 inhabitants of the district) and the presence of sewage treatment plants (km of sewer pipes per 100 inhabitants of the district). At the same stage of work, the project coordinators got access to the Trello board. Trello is one of the most popular online project management systems and is in great demand among small companies and startups [11]. On the board "Development and application of a crowd intelligence system for a smart city" there were already the groundworks of the project participants last year. The project coordinators added a new assignment. The advantage of the Trello board is that it allows you to track the process of work in the project, and all the results (in the form of reports) are publicly available to the board members (project). Therefore, at the first stage, to highlight the quality of life indicators that are relevant for the transition to a smart city, the project participants initially conducted a questionnaire design of the population. The sources for data collection were also:  website of the Federal State Statistics Service  website of the Ministry of Agriculture of the Novosibirsk Region  sites of the districts of the Novosibirsk region  publications of newspapers and magazines for 2017-2019 Since one cannot find all the indicators necessary to assess the living standards of the population of these areas in the public domain on the Internet, and many of them are simply not relevant now, the project executors expanded the list of sources. This fact had a great positive impact on the quality of the work performed. The results obtained served as the basis for further monitoring and assessment of living standards indicators, as well as determining the relationships between regional indicators and components of the “smart city”, which have a significant impact on life quality indicators [12, 13]. 320 At the second stage, we worked mainly with the official websites of districts and municipalities of the Novosibirsk region. To do this, we analyzed the occupancy, the content of the district sites in the following areas: medical services, cultural events, educational services, interaction between business, authorities and the population, payments, subsidies, news coverage of the district [14, 15]. The role of the coordinators, business analyst and designer at this stage was to develop a single format for the convenience of working with information. The assignment at the second stage included: 1. To analyze the filling and content of the district sites according to the following points:  medical services  cultural events  educational services  interaction between business, government authorities and the population,  payments, subsidies  coverage of district news 2. Determine whether the population can through the sites of municipalities:  get local government services  sign up for a meeting of retirees  learn about a meeting with a deputy  make an appointment with a doctor  pay utility bills, etc. The coordinators added this assignment to the Trello board. 4.2. Development of a database for digitalization of law The next area of the project is digitalization of law. The task of the students here was to develop a database of normative legal documents to identify an individual's propensity for corruption. The lawyers acting as project consultants identified a number of corruption-generating factors, which distributed them among the students in the course of their work and they used these factors as keywords for searching judicial acts in the judicial reference of the Supreme Court of the Russian Federation and on the Internet portal "Justice" [16, 17]. These factors were: "anti-corruption expertise"; “corruption-generating factor”, “legal and linguistic uncertainty”, “collisions”, “determination of competence according to the formula“ have the right ”,“ paragraph 3 of the methodology for conducting anti-corruption expertise of regulatory legal acts and draft regulatory legal acts”, “paragraph 4 of the methodology for conducting anti-corruption expertise of regulatory legal acts and draft regulatory legal acts" The work algorithm was as follows:  enter keywords (corruption factors) into the search line on the websites of the Supreme Court of the Russian Federation and on the Internet portal "Justice"  download the judicial act and highlight the text that deals with the corruption-generating factor used for the search  distribute documents into folders, where each folder is a corruption-generating factor As a result, the students processed more than 400 judicial acts, each of which took an average of 5- 7 minutes. The time costs estimated based on the timestamp of edited documents. Thus, in total, it took about two days of continuous work by one student to process 400 judicial acts on the Digital Law block (excluding the time spent on the preparation of reports, website malfunctions and other troubles). 4.3. Crowd assessment of the quality of human-machine interaction As the second stage of educational practice, we identified the work on the block "Crowd assessment of the quality of human-machine interaction". The task of the students was to annotate images for machine learning. 321 We distributed among the students a database of 500 screenshots of the main pages of the websites of various educational institutions. On screenshots of web pages, it was necessary to select all visually distinguishable interface elements and sign their type. The work algorithm was as follows: 1. Download and run the annotation program "Labeling" 2. Open the required screenshot 3. For each of the visible interface elements:  emphasize an interface element with a rectangular area  specify the type of the interface element by choosing from the offered ones or by entering your own element type 4. After finishing work, save in Pascal VOC format as XML file As a result, only one student did not complete his task on time, which is permissible given the fact that the team is large and a huge number of factors affect the efficiency of work. The project managers got assistance in getting the job done, as the team as a whole is responsible for everything it does and does not do. Thus, they managed to reduce a delay by only a few hours from the deadline. Summing up, annotating 45 images for machine learning by one student took in total, about 13 hours of continuous work (excluding the time spent on installation and familiarity with the annotation program interface, reporting, program malfunctions, and other problems). 5. Conclusion In the course of project training, students acquired primary skills in project management, data collection and analysis. Modern technical capabilities allow the participants of the Internet space to develop innovative ideas independently. They are able to involve various groups of the population, communities of social networks in discussing and developing new indicators of the quality of life, as well as developing new services for the population that improve the quality of life and thus contribute to the construction of their own "smart settlements" both in the city and in the countryside. An in-depth analysis showed that the website's capabilities used only in part, especially for the development of areas that do not require large capital investments. We have identified the direction of further development of project training, and this is how we can develop a system for monitoring quality of life indicators. Thus, within the framework of project training our students performed several types of useful works. They collected information on the indicators of the quality of life of the population of all districts of the Novosibirsk region, analyzed district websites and created a database of judicial acts structured according to criminogenic factors provided by lawyers. In addition, they checked annotations of web interfaces and wrote articles for student conferences. Moreover, the students got the opportunity to apply the knowledge gained in two years in practice. They gained experience of working in groups in a remote format, collecting data on the Internet, writing reports on the work done. The student in the role of the coordinator carried out the distribution of the amount of work between the project participants, their consultation and interaction with the project leaders. 6. References [1] Regulations on the procedure for organizing project training. (2020). Retrieved June 15, 2020, from https://swsu.ru/omk/normative_documents_cm/П%2002.168-2019_1.0_.pdf. [2] Evstratova, L. A. (2020). Project training. Implementation practices at universities. Retrieved June 15, 2020, from https://uni.hse.ru/data/2018/07/02/1153130829.pdf. [3] Namiot, D. E.,  Schneps-schnepe M. A. (2016). About domestic standards for smart cities. International Journal of open Information Technologies, 7. 32–37. 322 [4] Cohen, B. 3 generations of smart cities: inside the development of a technology-driven city. (2020). Retrieved June 15, 2020, from https://www.fastcoexist.com/3047795/the-3-generations- of-smart-cities. [5] Parfenov D., Zaporozhko V. ,"Developing SMART educational cloud environment on the basis of adaptive massive open online courses", Proceedings of the Conference Internationalization of Education in Applied Mathematics and Informatics for HighTech Applications, Vol. 2093, pp. 35-41. [6] Ganin, O. B.,  Ganin, I. O., "Smart city". (2020). Retrieved June 15, 2020, from https://cyberleninka.ru/article/n/umnyy-gorod-perspektivy-i-tendentsii-razvitiya. [7] Wakhyuni I. I.,  Salmiyah, D. (2019). Digital marketing of rebranding Bandung became a smart city, in: Proceedings of the 1st International Conference on Economics, Business, Entrepreneurship and Finance (ICEBEF 2018), Atlantis Press. [8] Kurcheeva, G. I. (2018). Complex approach to intelligent urban planning based on the use of big data, in; Physical journal: conference series "Information Technologies in Business and Industry", Tomsk, 1015. [9] N. N. Maksimov. (2012). Crowdsourcing as a way to activate staff development. Young scientist, 12. 233–236. [10] Kurcheeva, G. I.,  Aletdinova, A. A. (2017). Improving business processes based on the information model "smart city "in: Digital economy and industry 4.0: problems and prospects, Saint Petersburg. 69-73. [11] Trello. (2020). Retrieved June 5, 2020, from https://quokka-да.media / review / Trello-project management. [12] E-government e-resource. (2020). Retrieved June 2, 2020, http://government.ru/rugovclassifier/719/events. [13] Russian public initiative e-resource. (2020). Retrieved June 18, 2020, http://www.ciin.ru. [14] Website of the Novosibirsk region e-resource. (2020). Retrieved June 5, 2020, https://www.nso.ru. [15] Federal state statistics service e-resource. (2020). Retrieved June 18, 2020, http://www.gks.ru. [16] Website Electronic reference. Supreme Court of the Russian Federation. (2019). Retrieved June 18, 2020, https://vsrf.ru/lk/practice/acts?&numberExact=off&actDateExact=off. [17] Search in cases and judicial acts. State Automated System of the Russian Federation "Justice". (2019). Retrieved April 1, 2020, https://bsr.sudrf.ru/bigs/portal.html. 323