The paper summarizes the participation of the 6 FabSpaces to the population estimation (remote) pilot task at ImageCLEF 2017 Lab. FabSpace 2.0 is an open-innovation network for geodata-driven innovation that aims at improving universities contribution to the socioeconomic and environmental performance of societies. In the framework of the ImageCLEF Lab, the 6 FabSpaces participated although only four of them succeeded in submitting a run. This paper summarizes their participations. For each FabSpace, we present the local organization to participate to the CLEF Lab, the participants and their work. We conclude this paper with some lessons we learned from this participation.
FabSpace 2.0 is the open-innovation network for geodata-driven innovation by
leveraging Space data in particular, in Universities 2.0. FabSpace 2.0 received
funding from the European Unions Horizon 2020 Research and Innovation
programme under the Grant Agreement no693210. The FabSpace 2.0 project aims
at making universities open innovation centres for their region and at
improving their contribution to the socio-economic and environmental performance of
societies [
Within this project, we have set up 6 local FabSpace laboratories which are one-stop shops with access to a range of data (including space data), free software and data processing tools, to develop new applications. FabSpace also provides a new free-access service and place dedicated to collaborative data-driven innovation in 6 European universities. Each local FabSpace provides students, researchers, entrepreneurs-to-be and citizens with access to geodata, applications, software and challenges in order to design and test their own applications.
Within CLEF 2007, each local FabSpace organised locally their teams in
order to try to answer the population estimation (remote) task as de ned in
[
The Greek FabSpace is operated by ICCS and Corallia, which are the academic and business partners. The ICCS (Institute of Communications and Computer Systems) is a non-pro t Academic Research Foundation, founded in 1989 by the Ministry of Education, with a view to conducting research and development activities in collaboration with the School of Electrical and Computer Engineering of the National Technical University of Athens. Corallia works in the eld of cohesive and productive innovative ecosystems within which actors operate in a coordinated manner, in speci c sectors and regions of the country, and where a competitive advantage and export orientation exists. In those clusters, Corallia acts as a cluster facilitator. Corallia has already developed and currently supports the growth of three highly-specialised cluster initiatives in Greece, in knowledge-intensive thematic sectors. Both entities are also the leading partners in the Copernicus Uptake Networks, Copernicus Academy Network in Greece (GR-CAN) and Copernicus Relay Network in Greece (GR-CRN) respectively.
The Greek FabSpace Lab is located in the Microprocessors and Digital Systems Lab (microlab) in the National Technical University of Athens. 2.2
Corallia, with the support of ICCS and a number of companies, ESA and Copernicus Relay and Academy in Greece, have organized the FabSpace HackOnEarth, in order to promote the FabSpace 2.0 project, the FabSpace challenges and the CLEF challenge. This event was an open innovation contest, with main objective the creation of new applications, services, platforms, technologies and innovative solutions for non-space markets like precision agriculture and smart cities. This objective was ful lled by using geo-data and geo-information, and by exploiting the infrastructure of the FabSpace lab. The applicants could select freely one or more challenges set by HackOnEarth, and they could have the assistance of experts to create their own product, solution, application and business concept, during a 24-hour contest.
Local Teams Prior to the HackOnEarth event dates, the interest was already high. About 12 teams were registered initially for all the challenges, 2 of them speci cally for the CLEF contest. Each team consisted in 2 to 5 persons, and it has been recommended to include people with di erent background (i.e. earth observation specialists, electrical or mechanical engineers, business oriented people, etc.). Indeed, this interdisciplinarity is an essential part of the FabSpace project. During the contest, eight teams have nally started, while the two teams originally selected to compete for the CLEF challenge, have merged as they considered it important for the better elaboration of their ideas. It is considered that a larger number of participants could have been registered for the CLEF challenge, if some of the requirements were more relaxed or the time availability was larger. Additionally a good knowledge of the Earth Observation techniques, although not necessary to participate in the contest, was deemed recommended, therefore limiting the number of potential candidates. Local Criteria For all the proposed concepts in the HackOnEarth similar business/ innovation criteria have been used to select the winners. This was true also for the ideas related with the CLEF contest, since one of the main objectives of the FabSpace 2.0 is to promote the use of earth observation (EO) for the creation of new start-ups.
The overall criteria were: 1. Innovation of the proposed concept, 2. Technological approach of the proposed solution, 3. Potential for commercialization, 4. Team quality, 5. Maturity of the concept in order to become faster a nal product.
The accuracy and technological excellence of the proposed solution for the
CLEF challenge have been also taken into account. To this end the criteria of
the CLEF call have been additionally used (see [
Regarding the Greek local winner, their solution is brie y summarized here, however a more detailed analysis can be found in [7]. More speci cally, in order to achieve a population estimation of the areas of West Uganda and Zambia, the Greek team "Grapes" used Sentinel 2 satellite images coupled with a statistical analysis procedure based on historical census data. A supervised classi cation of images was used where the living areas of an image are extracted onto a high resolution pixelated output. The population of the classi ed study areas have been estimated by specifying a weight variable representing the density multiplier for each pixel that represents living areas. Its value varies for di erent areas and has been estimated using a statistical study on historical data including a regression and a forecasting model. Table 1 shows the results of the aforementioned estimation. 3 3.1
The FabSpace 2.0 project is managed in Italy by the University of Rome Tor Vergata and the Business Innovation Centre of Lazio Region (BIC Lazio). A dedicated infrastructure has been already set-up to support on a daily basis new users to understand and analyse EO data, with a particular emphasis on Copernicus data. Various animation events, such round tables, open-days, hackatons, bootcamps are organized within the project. Among them, speci c attention is devoted to the launch of challenges involving the use of satellite images and of open geographic data. 3.2
The CLEF challenge has been organized jointly by University of Rome \Tor Vergata" and BIC Lazio. It has been promoted through the FabSpace network of registered users, through the institutional website of the Regione Lazio. The announcement has been also included in the newsletter of the BICs Holding company Lazio Innova. Moreover various oral announcements to students have been made during lectures held at Tor Vergata University. A continuous support has been provided to all possible participants or interested people by the Tor Vergata FabSpace team.
Local teams The local organizing team consists primarily of the FabSpace Managers, but also other Tor Vergata personnel involved in the FabSpace project (so far 2 professors, 2 technicians, 3 researchers) and of some additional professors and researchers who provided their feedback, either in terms of methodology or accuracy, about the delivered results. In particular an evaluation board of three experts has been formed for the selection of the winner. Participating teams are presented later in this section.
Local criteria The evaluation was partially based on the methodology used by the participants (50% weighting factor). The board examined the procedure and evaluated it according to the following criteria: { level of exploitation of Earth observation data, { level of automation, and { generalization capability in other regions.
The assessment was also based on the comparison between estimates and ground
truth (50% weighting factor). Either for the city of Lusaka or for West Uganda
auxiliary data could be used. They are based on estimates that uses a
combination of voluntary geographical information (VGIs) working on BING (2012)
images with further earthwork. Both were provided by NGOs. In addition,
accuracy was evaluated using CLEF pilot task criteria [
Although various groups of people expressed their interest to participate to the
challenge, only two teams worked e ectively on it. The rst one derives from
one of the companies incubated in the BIC Lazio o ces, in the other one, two
University of Tor Vergata students, one from Management Engineering, the other
from the Computer Science Engineering combined their di erent skills to face
with the challenge. Only this second team delivered a nal result in the end.
Participants and Approaches The work submitted by the participating team
was based on the two areas centred on the Lusaka city in Zambia and on a
rural area in Uganda. The participants chose to use the satellite imagery from
Sentinel-2 mission, in particular the 10 meters resolution RGB and Near-infrared
bands were used. This choice was made because of the highest resolution respect
to other bands. Available open data providing unprecedented ne scale of 250
m maps quantifying population starting from detection and density of built-up
structures have been considered[
South) of Sentinel-2 imagery 2. The image is resampled to t the same pixel size dimension of Sentinel-2 imagery 3. Each pixel has 10x10 meters of resolution, but the value is still related to an area of 250x250 meters. To correct this problem, an initial redistribution operation is applied following this equation: pi1;0jx10 = p250x250 n
; i = 1; : : : 25; j = 1; : : : 25; n = 25 25
where pi1;0jx10 is the pixel value in the new resampled image, obtained in the
2.) step, and p250x250 is the pixel value in original LDS imagery, with:
25 25
X X pi1;0jx10 = p250x250
The equation (1) is performed for each pixel in original LDS imagery.
(1)
(2)
The method described assumes that the relationship between pixels 4 re ectances
and the population is non-linear. To approximate this relationship a CNN is
dened, it takes in input the pixels values (RGB and near infrared) and returns the
population [
The output of this process is a new image representing the population for each pixel, also the geographic reference from the input is reused for the output. Each area of interest is represented by a shape le, to extract the population of each area, a QGIS procedure importing the shape le and the output image from the CNN has been used.
Results The model was tested using the ImageCLEF 2017 validation system. The following result are obtained: Looking at the Pearson coe cient in the table above, our model seems to perform better in rural area (UGD) than Lusaka city (ZMB). A possible reason why we have obtained this di erence could be found in the better initial estimation in LDS ground truth for UGD respect the ZMB area. However, in general, the method proposed by the team is characterized by a very high level of automation and makes a smart usage of open geographic data, which are considered for training the CNN, showing an interesting way to enhance the information contained in the EO data. Our conclusion is that the method is de nitely promising because it e ectively combines the potential higher spatial resolution of Sentinel satellite imagery with already world-wide available open data at coarser resolution.
The German FabSpace is running by the strong collaboration between the academic partner Institute of Geodesy, Technical University of Darmstadt and the business partner ESA Business Incubation Center (ESA-BIC) managed by cesah GmbH. The physical lab is located at the Institute of Geodesy, Technical University of Darmstadt and dealing all the technological aspects of practicing earth observation science in the laboratory. The business section and all the entrepreneurial activity is dealing by the Business Incubation Center cesah GmbH.
The Institute of Geodesy represents the teaching and research in the area of Geodesy and Geo-information at the Technical University of Darmstadt. In the course of a re-organization in the year 2012 the following sections have joined to form the institute : Remote Sensing and Image Analysis (with a focus on remote sensing, photogrammetry and image analysis), Geodetic Measuring Systems and Sensor Technology (with a focus on measuring systems, sensor technology, surveying engineering), Land Management (with a focus on land management, real estate regulations and economy, and urban planning), Physical and Satellite Geodesy (with a focus on physical geodesy, reference systems, satellite geodesy and navigation).
The Center for Satellite Navigation Hessen (cesah) is a competence, information, and start-up center for satellite navigation and is supported by the State of Hesse, Darmstadt as well as renowned industrial and research facilities. On behalf of ESA, cesah runs the ESA Business Incubation Center (ESA BIC) Darmstadt and supports young companies and start-ups in the technical development, implementation and launch of new products and services related to satellite navigation. Moreover, the organization is promoting satellite navigation and earth observation in a digital world. The ESA BIC precisely supporting on how satellite navigation, earth observation, geo-information, telecommunication and more can be used for a variety of new applications and products development. In close co-operation and with technical and nancial support from ESA, cesah is giving the necessary assistance for start-up creation. 4.2
Regarding the ImageCLEF population estimation (remote) 2017 local contest, both the academic and business partners spread the word in the news media. A poster has been created to attract the participants and also several news has been published in di erent electronic media (i.e. Facebook, Twitter, LinkedIn etc.). Most importantly a website 1 has been created for the subscription from the participants for the ImageCLEF 2017 population estimation (remote) pilot task. The local task was organized at the FabSpace Laboratory at Technical University of Darmstadt as an open competition where all the FabSpace core member were 1 https://www.fabspace-germany.de/clef/index.html strongly involved to guide the methodology and draw the technology driven results. The best team of three members has been selected and one of them has been awarded as a trip to Dublin for the ImageCLEF 2017 conference. The winner has been received the prize sponsored by a promising company named Telespazio Vega at Darmstadt, Germany.
At the beginning, the team was selected according to 1) the ability of image processing, 2) the experience of working with Earth Observation data, 3) the software knowledge of objective based image processing. After selecting the best team, the next stage was to select the winner according to 1) the accuracy of image processing and population estimation (based on the initial ground truth set from the secondary sources), 2) the diverse methodology used to solve the challenge, 3) the implication of software and tools. 4.3
Participants and Approaches According to the registration process of
ImageCLEF population estimation (remote) 2017 [
To estimate the population, participants used the provided bands of
Sentinel2 images as visual bands 2,3,4 (VIS) and 8 (NIR) to create a stacked image [
Results The calculation process includes several runs which was validated
according to the ImageCLEF population estimation (remote) 2017 evaluation
criteria [
The Polish FabSpace is operated by OPEGIEKA and the Warsaw University of Technology. An academic institution of higher learning the Warsaw University of Technology, set up in Warsaw in 1826 provides 36 elds of study on 19 faculties and one college. It has about 37 thousand students and Ph.D. students. The many generations of engineers it turned out and its signi cant contributions to the development of technical sciences earned the Warsaw University of Technology an acclaimed position in the country as well as international renown. The Warsaw University of Technology, recognized in Europe and around the world, is steadily increasing its contribution to international educational and research projects, providing a mutually complementary educational and research package.
The Warsaw University of Technology cooperates closely with OPEGIEKA, which is a business partner in FabSpace 2.0 project. OPEGIEKA is a leading geomatics company in Poland. Holding a status of research and development centre, it is uniquely positioned to serve as an innovation hub, bringing together students, researchers and businesses. OPEGIEKA hosts FabSpace services using high-end Data Center opened in 2012. This assures that the service is widely available. With over 25 years of experience on the geospatial market and being a founding member of the only geospatial cluster in Poland (Geopoli), OPEGIEKA takes advantage of its business contacts to network research, students and entrepreneurs with public authorities and business. Additionally, the experience of OPEGIEKAs sta in creating new business services, using EO data, IT competences, and searching for funds, helps new entrepreneurs to set up new business and search for funding.
The Polish FabSpace Lab acts as a free-access place and service, where students, researchers and external users can make use of a data platform, as well as design and test their own applications which have been set up in the Centre for Innovation and Technology Transfer Management of the Warsaw University of Technology (CZIiTT WUT). The Centre supports technology transfer and innovation management, as well as conducts innovative research projects in these areas. 5.2 The CLEF challenge was an additional challenge of the Hackathon "Miasto przyszlosci, city of dreams", which was organised by the Warsaw University of Technology and OPEGIEKA. Participation in the competition was open to students, researchers and other people, who, by means of using methods of gami cation and open data, tried to develop a prototype game that solves one of the urban problems: - Development of the Port of Prague - Communication in Warsaw 'Mordor' - Capital of the 22nd century - Revitalisation of Plac De lad (one of the main squares in Warsaw). 5.3
The CLEF challenge was presented locally after test data release. A group of 5
people, who came to our FabSpace Lab, started working on the CLEF challenge
with the support of FabSpace managers. They worked according to the
assumptions and requirements set by the CLEF pilot task [
The assessment criteria for each project focused on the following aspects: - use of data (Earth Observation Data, geospatial data, etc.) - an innovative and original nature of the project (technological approach, product, service, technological and organisational, business model, social innovation)
- the expected bene ts of the project (relevance of the project to major social issues, etc.)
After reviewing the competition purpose and available data, participants chose data from the optical Sentinel-2 imagery and radar Sentinel-1. Prior to working with satellite imagery and in order to benchmark their results and select the appropriate parameters for classi cation and masking, students counted the buildings by dots earlier dividing areas for several di erent types of area development using the QGIS Open Layers plugin. The areas were divided into 3 for the Zambia area: industrial, slums, houses and 1 for Uganda: rural buildings.
Working with optical images began with the implementation of atmospheric correction using the sen2cor center tool, which allowed the removal of the atmospheric e ects from pixels value. They then used 10 m RGB channels to create a stacked image. In order to determine the number of buildings in optical images, they performed the unsupervised classi cation using the SNAP software. However, with the selection of di erent number of clusters the results were largely di erent from the number of buildings counted.
Railways approach to the number of buildings was done using Sentinel-1 GRD product. For urban area the backscattering is high, allowing only the pixels with the roofs to be separated from the image. Firstly, students pre-processing radar data, all operations were performed for VH polarization. Students analysing the histogram and using the Band Maths tool in SNAP set the mask for buildings only, the value chosen was one of the higher for the buildings to separate other elements. For areas with industrial buildings and homes, the pixel value in Band Maths turned out to be good, but in the case of slums not all buildings were designated, so students for slums performed the operations again by decreasing the pixel value and setting their masks separately. Then, for industrial and home buildings and for slums, the exported QGIS masks were subjected by mean shiftsegmentation process. The assumption of students was to obtain the number of polygons in the attributes table equal to the number of buildings. In the case of industrial buildings and houses, a comparison of the number of objects to the previous one yielded satisfactory results. However in the case of slums, many of the polygons after the segmentation contained in average one slums 4 houses. Therefore, the number of buildings designated after the segmentation was multiplied by 4.
In the case of results obtained by students to estimate the number of buildings, radar imaging was more applicable. Optical data has been used to divide areas by type of building. The results obtained and their comparison to the number of buildings was very promising.
The representative of the CLEF team presented the concept of the solution concerning estimation of the population and he was quali ed to participation in ImageCLEF 2017. He delivered the nal result to CLEF challenge, but unfortunately resigned from submission of the working note papers. 6 6.1
The French FabSpace is operated by Aerospace Valley (AV) and Universite Paul Sabatier (UPS) -project leader, which are the business and academic partners for France in FabSpace 2.0.
Aerospace Valley is a competitiveness cluster dedicated to aeronautics, space and embedded systems. Aerospace Valley quickly became a regional, national and international recognised cluster which has the ambition of creating, through its member's activities, up to 40,000 new jobs by 2025. Today Aerospace Valley gathers over 800 members from industry and research including amongst others more than 400 SMEs as well as large corporate groups and SMEs, major aerospace research establishments, engineering schools and local authorities, all working together to develop synergies propelling the sector into the future. The di erent positioning of Aerospace Valleys members is a key asset to promote technology transfer from research to industry thus bridging the gap between research and the marketplace.
The purpose of AV is to leverage the competitiveness and visibility of all its members, both on the national and international scene and to foster the development of collaborative initiatives. It provides a speci c support to SMEs helping them to take part in collaborative projects with industry leaders, investors, and research organisations.
Since its creation in 1969, UPS has been expanding its o er of multidisciplinary education in the elds of science, health, engineering, technology and sports, developing one of the most important scienti c research clusters in France. Centred on Toulouse, European space and aeronautics capital, UPS is a renowned European university with a global outlook. With almost 29 000 registered students, UPS is one of the leading French universities, in the quality of its teaching, the breadth of its scienti c research and the number of students it attracts.
As one of the top research establishments in France, counting 2784 teachers and teacher-researchers along with 1510 researchers from national and research bodies divided up into 72 laboratories, UPS is active in public research at the highest level. The Institut de Recherche e Informatique de Toulouse is one of the pillars of research in Computer Science in Midi-Pyrenees region, with 270 researchers and research professors, on a global workforce of 700 people. Two teams have been implicated in the project: The "Generalized Information Systems" team (SIG) (Josiane Mothe) is specialized on structured, semi-structured and unstructured information processing (mainly textual information). The team develops methods, models and tools for e cient access to quali ed and relevant information. The \Visual Objects from Reality To Expression" team (VORTEX) (Veronique Gaildrat) is organized in four thematic groups whose overall research topic is the acquisition, treatment, behaviour and visualization of 3D visual objects.
The French FabSpace is located in the Universite Paul Sabatier Catalyseur which aims at fostering innovation. 6.2
The challenge was presented locally to people interested in the domain of Remote Sensing after having spread the information locally both at the university and through Aerospace Valley network. A group of 4 persons with di erent background was constituted. The local group met twice but it was not possible to continue the work due to their lack of time availability. During the meetings, the local organizers provided aid with the use of remote sensing tools such as QGIS and Orfeo ToolBox. The members of the team were beginners in Remote Sensing analysis and lacked the expertise to elaborate a sophisticated methodology. The local organizers o ered training in some of the available tools (QGIS, Sentinel Tool Box and Orfeo Tool Box). Unfortunately, the participants were not able to submit a solution for the challenge.
The local organizers looked for alternative teams and managed to capture the attention of two potential teams. The rst team was based in France and included a researcher on the population estimation using remote sensing. This person seemed interested, he indicated that he successfully register in the challenge website. However, local organizers have not received any submission from him. A second team that showed interest was based on Spain. However, only an initial contact was made. The participants and the French FabSpace had a nice collaboration and at the beginning the participants had a great motivation. However, the participants could not reach to the nal point. All the participants are willing to connected with the French FabSpace for their future activity and also looking forward to the next year ImageCLEF event. 7
In this paper, we present the FabSpaces' participation to ImageCLEF 2017, estimating population pilot task. Within this network of Universities and companies, we organized this participation as an open contest like hackathon. While FabSpace 2.0 H2020 project promotes highly interdisciplinary teams, it was clear that in this challenge skills in satellite imagery processing and knowledge on earth observation and geo-spatial technologies was mandatory. Several teams were constituted but only 4 of them completed the task from the six FabSpaces in Europe. The results obtained show that although some solutions can be used, there is still room for improvement using freely accessible but low resolution satellite images for estimating population. Moreover, this task approached some new and sophisticated methodologies as fusion between supervised and unsupervised classi cation, fusion between radar and optical imagery, and Convolutional Neural Network (CNN) which need more precise screening to draw better results. 8
This project received funding from the European Unions Horizon 2020 Research and Innovation programme under the Grant Agreement n693210. https://www.fabspace.eu/ ImageCLEF 2017: Information extraction from images. In: CLEF 2017 Proceedings. Lecture Notes in Computer Science, vol. 10456. Springer, Dublin, Ireland (September 11-14 2017) 6. Jones, G.J.F., Lawless, S., Gonzalo, J., Kelly, L., Goeuriot, L., Mandl, T., Cappellato, L., Ferro, N.: Experimental ir meets multilinguality, multimodality, and interaction 8th international conference of the clef association, clef 2017, dublin, ireland, september 11-14, 2017, proceedings. vol. 10456. LNCS, Springer (2017) 7. Koutsouri, A., Skepetari, I., Anastasakis, K., Lappas, S.: Population estimation using satellite imagery. CLEF (2017) 8. Matan, O., Kiang, R.K., Stenard, C.E., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D., LeCun, Y.: Handwritten character recognition using neural network architectures. In: Proc. of the 4th US Postal Service Advanced Technology Conference. Washington D.C. (November 1990)