<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>November</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Crowd Intelligence System in Project Training of Students</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Galina Kurcheeva</string-name>
          <email>kurcheeva@yandex.ru</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Georgy Klochkov</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Maria Lapina</string-name>
          <email>mlapina@ncfu.ru</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>North-Caucasus Federal University</institution>
          ,
          <addr-line>Prospect Kulakova, 2, 355000, Stavropol</addr-line>
          ,
          <country country="RU">Russia</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Novosibirsk State Technical University</institution>
          ,
          <addr-line>K. Marksa 20, Novosibirsk, 630073</addr-line>
          ,
          <country country="RU">Russia</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Novosibirsk State University of Economics and Management</institution>
          ,
          <addr-line>Kamenskaya Str. 52/1, Novosibirsk, 630099</addr-line>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2020</year>
      </pub-date>
      <volume>1</volume>
      <fpage>2</fpage>
      <lpage>13</lpage>
      <abstract>
        <p>The authors propose modern developments in the field of organizing project training, setting goals and objectives. A new form of teaching students is considered. The novelty lies in attracting students of different specialties to the development of large projects and combining groups of performers on the topic of the project from students of the second, third and fourth years of study with the possibility of attracting graduate students. Moreover, project training can continue for several semesters. Students have the opportunity to acquire the skills of a programmer, website designer, developer, work coordinator, manager and consultant. The stages of organizing work for a project that has not only theoretical, but also practical importance for the Novosibirsk region are considered. The project participants have elaborated new streamlines of digital technologies that will be in demand with the development of information technologies. The students analyzed websites in all districts of the region, which made it possible to assess the availability of information for the population. One of the areas of project training allows one to identify the relationship between the visual and informational complexity of web pages and their accessibility and convenience for users. The organization of the crowd intelligence system allows uniting students for the implementation of collective intelligence technologies. Thus, the project participants assessed the indicators of the quality of life of the population in the digital environment, while various groups of the population were involved in the discussion on social networks. We can also use such a methodology to analyze regulatory documents and court decisions in order to search for contradictions contained in them. The crowd intelligence system is useful for digitalizing the anti-corruption expertise process in order to reduce the "manual" processing of materials. The proposed form of training allows you to apply the theoretical knowledge of students to solving practical problems, teaches how to distribute working time, manage work areas, and search for non-standard solutions. Moreover, project-based learning is an ongoing process and it helps in the selection of directions for graduate qualification works and topics for master's theses. Project training, information technology, smart settlement, websites, crowd intelligence.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        The process of digitalization and development of information technologies has caused the need to
supplement classical forms of education with new active forms, including project-based teaching
methods. Project-based training is training in the conditions of a joint, independent, meaningful
activity of students organized by a pedagogical worker, culminating in the creation of a creative
product, based on the use of problem, search, project, research, active teaching methods [
        <xref ref-type="bibr" rid="ref1 ref2">1, 2</xref>
        ].
      </p>
      <p>2020 Copyright for this paper by its authors.</p>
      <p>
        Due to this, for project-based learning, one should choose research topics that are important for the
development of the digital economy at the level of the country, region, city and settlement. For project
training, we selected topics related to the development of a “smart” or “digital” city and “smart”
settlement, digitalization of law, artificial intelligence and the development of interfaces for applied
tasks [
        <xref ref-type="bibr" rid="ref3">3, 4</xref>
        ]. The relevance of the selected topics is due to their focus on solving problems of
improving the quality of life of the population based on the development of information technologies,
the possibility of remote access to information services. At the same time, problems arise associated
with the collection, processing and interpretation of large amounts of data, the selection of
nonstandard solutions for applicability to population groups and communities [5]. The research is
innovative in nature and aimed at identifying new indicators of the quality of life and their assessment
by different groups of the urban and rural population, at searching for contradictions in the
interpretation of regulatory documents from the standpoint of digital law.
      </p>
      <p>Students of several directions at the Department of Automated Control Systems (ACS) of the
Novosibirsk State Technical University (NSTU) participate in project training.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Purpose and objectives of the research</title>
      <p>The purpose of the study is to test new approaches to project-based teaching of students, the
formation of independence in decision-making, the formation of a team spirit, mutual assistance in
solving tasks, non-standard solutions in the process of collecting and processing information. For this
purpose, we asked students to study the theoretical basis on the development and application of the
crowd intelligence system and to choose a topic for research.</p>
      <p>The first research topic "Digital City" involves organizing work in such areas as the collection of
official and unofficial information on the quality of life indicators, the distribution of tasks between
project executors, collection, processing and interpretation of the results.</p>
      <p>The second research topic "Digital Law" requires the development of a database of regulatory
documents to identify the propensity of a person to corruption.</p>
      <p>The third research topic "Development of interfaces for applied tasks" related to the organization
of work on annotating images for machine learning; collection and systematization of results,
verification of the correctness of work performance, assessment of labor costs and efficiency of
executors and assessment of labor costs and efficiency of the group as a whole.</p>
      <p>Methods of synthesis, system analysis, calculation and analysis of quality of life indicators and
other instruments used in the process of collecting and processing data served as a methodological
basis of our research. Publications posted on Internet resources, final reports of the ministries of the
Novosibirsk region, data from newspapers and magazines devoted to the infrastructure of rural
settlements in the Novosibirsk region, as well as websites of the Supreme Court of Russia acted as an
information base.</p>
    </sec>
    <sec id="sec-3">
      <title>3. Literature review</title>
      <p>A review of the literature shows the importance of developing a project for “smart city” and “smart
settlement”. There is a wide discussion on the terminology associated with the concepts of “smart
city”, “digital city”, “developing city”, “smart settlement” and “digital law” in the scientific literature,
[6, 7, 8].</p>
      <p>While exploring the possibilities for improving the quality of life of rural settlements, which
significantly lags behind the urban ones, we proceed from the assumption that both urban and rural
populations can turn to Internet resources of national importance, designed to inform citizens about
various spheres of life in Russia, such as the portal of the State services and others. In addition, a new
direction is successfully developing, namely, the websites of rural settlements, which accumulate their
own local characteristics and traditions. They contribute to improving the quality of life of the
population of rural settlements.</p>
      <p>Therefore, to solve the highlighted problems, it is to choose crowdsourcing technologies, which
also need to develop taking into account the possibilities of the Internet and social functions online.
Crowdsourcing, in turn, makes it possible to involve a large number of specialists (students) of
various professional backgrounds with a wide geographical coverage in solving the problems and
tasks of the project initiator at a relatively low cost.</p>
      <p>In general, crowdsourcing is an innovative way to get things done at the lowest cost using Internet
technologies and the synergy effect of bringing together the intellectual potential of as many people as
possible. The essence of crowdsourcing is to get the desired innovation from one idea with the help of
a group of people united by a common goal. In any case, executors generate ideas on a voluntary
basis. In project-based teaching, students independently distribute, perform and discuss all types of
work in groups responsible for each task.</p>
      <p>The student himself evaluates the labor intensity and determines how much time he can spend on
participating in a particular project without prejudice to learning. The crowdsourcing phenomenon is a
worldwide trend. Finding new ways to develop innovation in any social area is very important. It is
crowdsourcing that in the future may become a means of implementing economic, social and other
civic initiatives, which is why it is so important in training [9, 10].</p>
    </sec>
    <sec id="sec-4">
      <title>4. Results</title>
      <p>The student team assigns tasks according to the following skills, namely Web Designer, Work
Coordinator, Developer, Business Analyst, Web Programmer, and Project Consultant.</p>
    </sec>
    <sec id="sec-5">
      <title>4.1. Smart city, smart settlement: digitalizing the quality of life</title>
      <p>This is the first streamline of the project, it required:
 to develop and evaluate the indicators of the quality of life most demanded by the
population
 to assess the possibility of improving the quality of life of the population through the
development of websites for local settlements, including rural ones</p>
      <p>The first stage of work included the collection of information on such indicators as the presence of
preschool and school institutions, the presence of medical institutions, the presence of sports
complexes and the presence of cultural institutions. An important indicator of the quality of life is also
the availability of a water supply network (km per 100 inhabitants of the district) and the presence of
sewage treatment plants (km of sewer pipes per 100 inhabitants of the district).</p>
      <p>At the same stage of work, the project coordinators got access to the Trello board. Trello is one of
the most popular online project management systems and is in great demand among small companies
and startups [11]. On the board "Development and application of a crowd intelligence system for a
smart city" there were already the groundworks of the project participants last year. The project
coordinators added a new assignment. The advantage of the Trello board is that it allows you to track
the process of work in the project, and all the results (in the form of reports) are publicly available to
the board members (project).</p>
      <p>Therefore, at the first stage, to highlight the quality of life indicators that are relevant for the
transition to a smart city, the project participants initially conducted a questionnaire design of the
population.</p>
      <p>The sources for data collection were also:
 website of the Federal State Statistics Service
 website of the Ministry of Agriculture of the Novosibirsk Region
 sites of the districts of the Novosibirsk region
 publications of newspapers and magazines for 2017-2019</p>
      <p>Since one cannot find all the indicators necessary to assess the living standards of the population
of these areas in the public domain on the Internet, and many of them are simply not relevant now, the
project executors expanded the list of sources. This fact had a great positive impact on the quality of
the work performed.</p>
      <p>The results obtained served as the basis for further monitoring and assessment of living standards
indicators, as well as determining the relationships between regional indicators and components of the
“smart city”, which have a significant impact on life quality indicators [12, 13].</p>
      <p>At the second stage, we worked mainly with the official websites of districts and municipalities of
the Novosibirsk region. To do this, we analyzed the occupancy, the content of the district sites in the
following areas: medical services, cultural events, educational services, interaction between business,
authorities and the population, payments, subsidies, news coverage of the district [14, 15].</p>
      <p>The role of the coordinators, business analyst and designer at this stage was to develop a single
format for the convenience of working with information.</p>
      <p>The assignment at the second stage included:
1. To analyze the filling and content of the district sites according to the following points:
 medical services
 cultural events
 educational services
 interaction between business, government authorities and the population,
 payments, subsidies
 coverage of district news
2. Determine whether the population can through the sites of municipalities:
 get local government services
 sign up for a meeting of retirees
 learn about a meeting with a deputy
 make an appointment with a doctor
 pay utility bills, etc.</p>
      <p>The coordinators added this assignment to the Trello board.</p>
    </sec>
    <sec id="sec-6">
      <title>4.2. Development of a database for digitalization of law</title>
      <p>The next area of the project is digitalization of law. The task of the students here was to develop a
database of normative legal documents to identify an individual's propensity for corruption.</p>
      <p>The lawyers acting as project consultants identified a number of corruption-generating factors,
which distributed them among the students in the course of their work and they used these factors as
keywords for searching judicial acts in the judicial reference of the Supreme Court of the Russian
Federation and on the Internet portal "Justice" [16, 17].</p>
      <p>These factors were: "anti-corruption expertise"; “corruption-generating factor”, “legal and
linguistic uncertainty”, “collisions”, “determination of competence according to the formula“ have the
right ”,“ paragraph 3 of the methodology for conducting anti-corruption expertise of regulatory legal
acts and draft regulatory legal acts”, “paragraph 4 of the methodology for conducting anti-corruption
expertise of regulatory legal acts and draft regulatory legal acts"</p>
      <p>The work algorithm was as follows:
 enter keywords (corruption factors) into the search line on the websites of the Supreme</p>
      <p>Court of the Russian Federation and on the Internet portal "Justice"
 download the judicial act and highlight the text that deals with the corruption-generating
factor used for the search
 distribute documents into folders, where each folder is a corruption-generating factor
As a result, the students processed more than 400 judicial acts, each of which took an average of
57 minutes. The time costs estimated based on the timestamp of edited documents.</p>
      <p>Thus, in total, it took about two days of continuous work by one student to process 400 judicial
acts on the Digital Law block (excluding the time spent on the preparation of reports, website
malfunctions and other troubles).</p>
    </sec>
    <sec id="sec-7">
      <title>4.3. Crowd assessment of the quality of human-machine interaction</title>
      <p>As the second stage of educational practice, we identified the work on the block "Crowd
assessment of the quality of human-machine interaction". The task of the students was to annotate
images for machine learning.</p>
      <p>We distributed among the students a database of 500 screenshots of the main pages of the websites
of various educational institutions.</p>
      <p>On screenshots of web pages, it was necessary to select all visually distinguishable interface
elements and sign their type.</p>
      <p>The work algorithm was as follows:
1. Download and run the annotation program "Labeling"
2. Open the required screenshot
3. For each of the visible interface elements:
 emphasize an interface element with a rectangular area
 specify the type of the interface element by choosing from the offered ones or by
entering your own element type
4. After finishing work, save in Pascal VOC format as XML file</p>
      <p>As a result, only one student did not complete his task on time, which is permissible given the fact
that the team is large and a huge number of factors affect the efficiency of work. The project
managers got assistance in getting the job done, as the team as a whole is responsible for everything it
does and does not do. Thus, they managed to reduce a delay by only a few hours from the deadline.</p>
      <p>Summing up, annotating 45 images for machine learning by one student took in total, about 13
hours of continuous work (excluding the time spent on installation and familiarity with the annotation
program interface, reporting, program malfunctions, and other problems).</p>
    </sec>
    <sec id="sec-8">
      <title>5. Conclusion</title>
      <p>In the course of project training, students acquired primary skills in project management, data
collection and analysis.</p>
      <p>Modern technical capabilities allow the participants of the Internet space to develop innovative
ideas independently. They are able to involve various groups of the population, communities of social
networks in discussing and developing new indicators of the quality of life, as well as developing new
services for the population that improve the quality of life and thus contribute to the construction of
their own "smart settlements" both in the city and in the countryside.</p>
      <p>An in-depth analysis showed that the website's capabilities used only in part, especially for the
development of areas that do not require large capital investments. We have identified the direction of
further development of project training, and this is how we can develop a system for monitoring
quality of life indicators.</p>
      <p>Thus, within the framework of project training our students performed several types of useful
works. They collected information on the indicators of the quality of life of the population of all
districts of the Novosibirsk region, analyzed district websites and created a database of judicial acts
structured according to criminogenic factors provided by lawyers. In addition, they checked
annotations of web interfaces and wrote articles for student conferences.</p>
      <p>Moreover, the students got the opportunity to apply the knowledge gained in two years in practice.
They gained experience of working in groups in a remote format, collecting data on the Internet,
writing reports on the work done.</p>
      <p>The student in the role of the coordinator carried out the distribution of the amount of work
between the project participants, their consultation and interaction with the project leaders.</p>
    </sec>
    <sec id="sec-9">
      <title>6. References</title>
      <p>[4] Cohen, B. 3 generations of smart cities: inside the development of a technology-driven city.
(2020). Retrieved June 15, 2020, from
https://www.fastcoexist.com/3047795/the-3-generationsof-smart-cities.
[5] Parfenov D., Zaporozhko V. ,"Developing SMART educational cloud environment on the basis
of adaptive massive open online courses", Proceedings of the Conference Internationalization of
Education in Applied Mathematics and Informatics for HighTech Applications, Vol. 2093, pp.
35-41.
[6] Ganin, O. B.,  Ganin, I. O., "Smart city". (2020). Retrieved June 15, 2020, from
https://cyberleninka.ru/article/n/umnyy-gorod-perspektivy-i-tendentsii-razvitiya.
[7] Wakhyuni I. I.,  Salmiyah, D. (2019). Digital marketing of rebranding Bandung became a
smart city, in: Proceedings of the 1st International Conference on Economics, Business,
Entrepreneurship and Finance (ICEBEF 2018), Atlantis Press.
[8] Kurcheeva, G. I. (2018). Complex approach to intelligent urban planning based on the use of big
data, in; Physical journal: conference series "Information Technologies in Business and
Industry", Tomsk, 1015.
[9] N. N. Maksimov. (2012). Crowdsourcing as a way to activate staff development. Young scientist,
12. 233–236.
[10] Kurcheeva, G. I.,  Aletdinova, A. A. (2017). Improving business processes based on the
information model "smart city "in: Digital economy and industry 4.0: problems and prospects,
Saint Petersburg. 69-73.
[11] Trello. (2020). Retrieved June 5, 2020, from https://quokka-да.media / review / Trello-project
management.
[12] E-government e-resource. (2020). Retrieved June 2, 2020,
http://government.ru/rugovclassifier/719/events.
[13] Russian public initiative e-resource. (2020). Retrieved June 18, 2020, http://www.ciin.ru.
[14] Website of the Novosibirsk region e-resource. (2020). Retrieved June 5, 2020,
https://www.nso.ru.
[15] Federal state statistics service e-resource. (2020). Retrieved June 18, 2020, http://www.gks.ru.
[16] Website Electronic reference. Supreme Court of the Russian Federation. (2019). Retrieved June
18, 2020, https://vsrf.ru/lk/practice/acts?&amp;amp;numberExact=off&amp;amp;actDateExact=off.
[17] Search in cases and judicial acts. State Automated System of the Russian Federation "Justice".
(2019). Retrieved April 1, 2020, https://bsr.sudrf.ru/bigs/portal.html.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <article-title>[1] Regulations on the procedure for organizing project training</article-title>
          . (
          <year>2020</year>
          ).
          <source>Retrieved June 15</source>
          ,
          <year>2020</year>
          , from https://swsu.ru/omk/normative_documents_cm/П%
          <year>2002</year>
          .
          <fpage>168</fpage>
          -
          <issue>2019</issue>
          _
          <fpage>1</fpage>
          .0_.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Evstratova</surname>
            ,
            <given-names>L. A.</given-names>
          </string-name>
          (
          <year>2020</year>
          ).
          <article-title>Project training. Implementation practices at universities</article-title>
          .
          <source>Retrieved June 15</source>
          ,
          <year>2020</year>
          , from https://uni.hse.ru/data/2018/07/02/1153130829.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>Namiot</surname>
            ,
            <given-names>D. E.</given-names>
          </string-name>
          , 
          <string-name>
            <surname>Schneps-schnepe</surname>
            <given-names>M. A.</given-names>
          </string-name>
          (
          <year>2016</year>
          ).
          <article-title>About domestic standards for smart cities</article-title>
          .
          <source>International Journal of open Information Technologies</source>
          ,
          <volume>7</volume>
          .
          <fpage>32</fpage>
          -
          <lpage>37</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>