=Paper=
{{Paper
|id=Vol-3183/131
|storemode=property
|title=Agriculture Professional GNSS Receivers: Performance Comparison in Controlled and Operational Scenarios
|pdfUrl=https://ceur-ws.org/Vol-3183/paper2.pdf
|volume=Vol-3183
|authors=Jacopo Capolicchio,Daniele Mennuti,Ileana Milani,Marco Fortunato,Riccardo Petix,Joaquin Reyes Gonzalez,Martin Sunkevic
|dblpUrl=https://dblp.org/rec/conf/icl-gnss/CapolicchioMMFP22
}}
==Agriculture Professional GNSS Receivers: Performance Comparison in Controlled and Operational Scenarios==
Agriculture Professional GNSS Receivers: Performance Comparison in Controlled and Operational Scenarios Jacopo Capolicchioa, Daniele Mennutia, Ileana Milania, Marco Fortunatob, Riccardo Petixc, Joaquin Reyes Gonzalezd and Martin Sunkevicd. a Thales Alenia Space Italia, Via Saccomuro 24, 00131 Rome, Italy b Randstad Italia, Via Tiburtina 1072, 00156 Rome, Italy c Business Integration Partners, Via Sicilia 43, 00187 Rome, Italy d European Union Agency for the Space Programme (EUSPA), Janovského 438/2, 170 00 Prague 7, Holesovice Czech Republic Abstract GNSS (Global Navigation Satellite Systems) plays nowadays a major role in different civilian activities and is a key technology enabling innovation in a wide variety of market sectors. For instance, GNSS-enabled solutions are widespread within the Precision Agriculture and, among them, applications in the field of machinery guidance are commonly employed to optimize typical agriculture practices. The scope of this paper is to present and compare the results obtained during two different agriculture test campaigns conducted between 2020 and 2021 by Thales Alenia Space Italia, in the implementation of a contract signed with the European Agency for the Space Program (EUSPA) and financed by the European Union under the Galileo Programme budget. The test campaigns were oriented to the performance evaluation of a set of GNSS receivers, pointing out the benefits on the use of the Galileo constellation both in single and multi-constellation configurations. Moreover, the focus was on the added value of using the Galileo system in the GNSS market segment of agriculture Machine Guidance, from the end user point of view. In particular, Single Point Positioning (with both Single Frequency and Multi-Frequency approach), Satellite Based Augmentation System (SBAS), Precise Point Positioning (PPP), Real Time Kinematic (RTK) and PPP-RTK (only in 2021) configurations have been tested. During the first campaign receivers were tested on a rail-carriage configuration, with all the additional features disabled for a fair comparison of the results between the receivers. On the other hand, in the second testing campaign, all the available features of the receivers were enabled, and the receivers were tested while operating on real machineries, to assess their performance under typical working conditions, such as terrain asperities and tractor vibrations. Furthermore, the RTK baseline was increased from a few hundreds of metres of the 2020 campaign to several kilometres for the tests executed in July 2021. The analysed performance was oriented to the assessment of the KPIs (Key Performance Indicators), involving cross-track accuracy, pass-to-pass accuracy, and horizontal accuracy. The comparison between the results of the two test campaigns shows that, despite the more challenging scenario selected for the second one, comparable performances are achieved by the same receivers, thanks to the exploitation of receiver’s embedded features. The main differences were observed for the RTK solutions where the longer baseline and the use of a real tractor could have an impact when the accuracy is of a few centimetres. Furthermore, Galileo signals, used in combination with other GNSS systems, are proven to enhance the performance of the evaluated GNSS receivers in most of the test cases. Keywords 1 GNSS, Galileo, Agriculture Market Segment. ICL-GNSS 2022 WiP, June 07–09, 2022, Tampere, Finland EMAIL: Jacopo.Capolicchio@thalesaleniaspace.com (A. 1); Daniele.Mennuti@thalesaleniaspace.com (A. 2); ileana.milani@thalesaleniaspace.com (A. 3); Marco.Fortunato-Somministrato@thalesaleniaspace.com (A. 4); riccardo.petix@mail-bip.com (A. 5); joaquin.reyes@euspa.europa.eu (A. 6); martin.sunkevic@euspa.europa.eu (A.7). ORCID: 0000-0002-0398-5111 (A. 1); 0000-0002-2212-1546 (A. 2); 0000-0002-7835-5085 (A. 3); 0000-0002-7925-3187 (A. 4); 0000-0003- 2341-4882 (A. 5); 0000-0002-0530-8763 (A. 6); 0000-0002-3874-5054 (A. 7). © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Wor Pr ks hop oceedi ngs ht I tp: // ceur - SSN1613- ws .or 0073 g CEUR Workshop Proceedings (CEUR-WS.org) Proceedings 1. Introduction Nowadays, GNSS (Global Navigation Satellite System) represents a key technology for location- based applications, providing cost-effective solutions to users. The European navigation system, Galileo, was declared operationally ready in December 2016, starting to offer its services worldwide. The performance and limitations of the Galileo Initial Services, together with the system configuration, are described in [1]. Galileo offers a highly accurate service despite, for the time being, the system is not yet in Full Operational Capability (FOC). More specifically, starting from December 2016, with the Early Operational Capability, the constellation evolved and reached the number of 22 usable satellites in July 2021. Further information on the Galileo satellites available for Position, Velocity and Time (PVT) computation can be found in [2]. Manufacturers are gradually enabling Galileo services into their products, and in order to check the status of the implementation in professional receivers, EUSPA (European Union Agency for the Space Programme) commissioned two testing campaigns, between 2020 and 2021. The focus of the testing campaigns was on the precision agriculture market segment, in particular in the field of machine guidance. The first testing campaign, executed in July 2020 and detailed in [3], exploited a rail-carriage configuration in an open sky scenario, to observe the behavior of the receivers in a controlled environment. During the second one, performed in July 2021, the receivers were fixed on a real tractor in two different scenarios, open sky and close to forest, to appreciate the achievable performance in an operational environment. The main aim of the testing campaigns was to evaluate the performance of a set of GNSS professional receivers, analysing the added value of the Galileo system in the GNSS market segment of agriculture machine guidance, from the end user point of view. Moreover, the testing campaigns were also oriented to support the manufacturers, pointing out the benefits on the use of Galileo and providing feedbacks on their products. In this article, the major results obtained in 2020 and 2021 are showed and compared with each other, to highlight differences and commonalities raised from the performance evaluation in two different testing environments. The tested positioning modes for both 2020 and 2021 campaigns are reported in Table 1, together with the selected frequencies. SPP (Single Point Positioning), SBAS (Satellite Based Augmentation System), PPP (Precise Point Positioning) and RTK (Real Time Kinematic) positioning modes were tested. It should be noticed that, in 2021, PPP-RTK mode was tested in some of the RUTs (Receivers Under Test). PPP-RTK extends the concept of PPP, also including the corrections for atmospheric errors (caused by the GNSS signal travelling through the ionosphere and troposphere) which are calculated using a CORS (Constantly Operating Reference Station) network [4]. Table 1 Positioning modes and selected frequency in 2020 and 2021 in Open Sky scenario Test ID Constellations Freq Pos. mode 2020 2021 REF-01 GPS SF SBAS ü ü REF-02 GPS + GLO MF RTK ü ü RTK-01 GPS+GAL+GLO MF RTK ü ü RTK-02 GPS+GAL MF RTK ü ü PPP-01 GPS+GAL+GLO MF PPP ü ü PPP-02 GPS + GAL MF PPP ü ü PPP-03 GPS+GLONASS MF PPP ü ü GPS+GLO + GAL + PPP-04 MF PPP û ü BDS YYY-01 GPS+GAL+GLO MF PPP-RTK û ü YYY-02 GPS + GLO MF PPP-RTK û ü YYY-03 GPS+GAL MF PPP-RTK û ü NOC-01 GPS+GAL+GLO MF SPP ü ü NOC-02 GAL MF SPP ü ü NOC-03 GPS MF SPP ü ü NOC-04 GAL SF SPP ü ü NOC-05 GPS SF SPP ü ü The RUTs were tested in single and multi-GNSS configurations, with single and multi-frequency modes. In particular, the SPP mode was tested with GPS-only and Galileo-only in single (L1 and E1) and multi-frequency modes. On the other side, the triple constellation in SPP and the augmentation modes were all tested in multi-frequency configuration. For the multi-frequency test-cases all the available frequencies supported by the receivers were enabled for the PVT estimation, as shown in Table 2. It should be underlined that the BeiDou constellation was exploited only in PPP mode in the 2021 testing campaign to investigate the possible benefits of a quadruple constellation configuration, mainly in challenging scenarios, in order to ensure high availability of the PVT. Table 2 Selected frequencies for different positioning modes Constellation Standalone SBAS PPP/PPP-RTK RTK GPS L1/L2/L5 L1 L1/L2/L5 L1/L2/L5 E1/E5a/E5b/E E1/E5a/E5b/E E1/E5a/E5b/ Galileo - 5AltBOC 5AltBOC E5AltBOC GLONASS L1/L2/L3 - L1/L2/L3 L1/L2/L3 BeiDou - - B1/B2 - The paper is organized as follows. In Section 2, the setup of the testing campaign has been described together with the main KPIs relevant for the agriculture market segment. The main results of the testing campaign have been presented in Section 3, through the comparison of the performance of the tested receiver. Finally, the conclusions of this work are shown in Section 4. 2. Agriculture test campaigns setup and KPIs In this section differences and commonalities regarding the two testing campaigns are analysed, focusing on the test configuration setup and the KPIs (Key Performance Indicators). 2.1. Test setup During the first testing campaign receivers were tested on a carriage moving on rails (Figure 1) with a speed of around 7 km/h, with all the additional algorithms, like RAIM (Receiver Autonomous Integrity Monitoring) and smoothing, disabled for a fair comparison of the results between the receivers. However, the use of different devices, hence potentially different proprietary algorithms, could lead to different outcomes in analogous positioning modes. The analysed performance was oriented to the assessment of KPIs, involving cross-track accuracy, pass-to-pass accuracy, and repeatability. During the second testing campaign, on the contrary, all the available functionalities of the receivers were enabled, and the RUTs were tested while operating on real machineries (Figure 1), to assess their performance under typical working conditions, such as terrain asperities and tractor vibrations. The comparison between the results obtained during the agriculture testing campaigns is oriented towards the assessment of receivers’ performance in controlled and operational scenario (which involves vibrations, tractor guidance, ground irregularities, typical working lines etc.). Figure 1 Testing Campaign Working Lines in 2020 (left) and 2021 (right). In order to compare the results of the two testing campaigns, various points must be clarified. The test cases had a different duration over the two testing campaigns: 3 hours for the 1st phase of the testing campaign and 1 hour for the testing campaign with real machineries. This could lead in different statistics, especially for test-cases with greater variance (i.e. Standalone Positioning Mode). Moreover, the satellites geometry, and thus the DOP (Dilution Of Precision), was different between the testing campaigns, as the tests were not simultaneous. The use of a carriage and a rail track enhances the relative positioning performance and, consequently, the pass-to-pass accuracy. On the other hand, the use of a real tractor operating typical working lines (at around 6 km/h), with U-turns, could significantly influence the obtained performance with deviations from the nominal trajectory. The RTK test-cases during the first testing campaign were performed with a base station located inside the testing field, a few hundred metres from the rover. While, in 2021, the corrections for the RTK test cases were provided by a base station located a few kilometres away from the testing field. Moreover, for the 2021 testing campaign all the additional features (i.e., RAIM, Hatch filters based smoothing, integration with additional sensors) implemented by the receivers were enabled, in order to evaluate the performance that a real user would achieve during agriculture operations. For both the testing campaigns, the RUTs were tested in parallel to compare their performance under the same conditions. As an additional information, five of the six tested receivers were smart antennas, while one of the RUTs was tested using an external antenna provided by the manufacturer. 2.2. Key Performance Indicators For each test case, in the 2021 testing campaign, the agriculture related KPIs showed in Table 3 have been assessed. In the 2020 testing campaign, the cross-track accuracy and the repeatability were mainly considered. In Figure 2, cross-track and pass-to-pass accuracy are showed in detail. In particular, for the computation of the accuracy performance, the 95 percentiles of the CDFs (Cumulative Distribution Functions) of the parameters of interest were considered. Table 3 KPIs Definition and Computation KPI Definition Computation The cross-track error (CTE) is calculated from a subset of the measurements, excluding the U- Variation between the actual turns performed by the tractor. Trajectory Error tilling trajectory w.r.t. the On the other hand, the (95%ile) reference one horizontal accuracy also considers the U-turns, with all the measurements obtained during the test interval Amount of skip or overlap field Difference between cross-track Pass to Pass (P2P) area that occurs during errors over a 15 minutes Accuracy (95%ile) agriculture operations interval Number of epochs with PVT Percentage of time over which available in the desired mode PVT availability the PVT has been computed by divided by the number of using the chosen configuration epochs for the specific test case Indicates the change from one solution type to another during Losses of lock vs time - the data collection (i.e. RTK fixed to RTK float). Number of Number of used or tracked tracked/used satellites satellites by the receiver for - vs. time PVT estimation during the data collection. HDOP (Dilution Of Dilution of precision due to the Precision) not ideal satellites geometry !𝜎! " + 𝜎# " Probability that the operational performance (PVT estimation This parameter has been Continuity (over 15 s with defined positioning mode) calculated by using a 15s time window) is kept over a fifteen seconds moving window period Time that the position estimates The convergence time was need to reach steadily a specific calculated either from NMEA Convergence time accuracy level, without leaving GPGGA message (Quality this level of accuracy Indicator) or from raw data files For the assessment of the positioning errors, the measured positions, extracted from the NMEA files generated by the receivers, were compared with a reference true trajectory generated using a post- processing double differences carrier-phase algorithm included in a commercial third-party software. It should be noted that in the computation of the HDOP, the first two elements of the diagonal of the covariance matrix were used (in East and North coordinates). Figure 1 Agriculture-related KPIs: CTE (on the left) and P2P accuracy (on the right). 3. Agriculture test campaign results The two parameters selected to compare the results obtained during the different testing campaigns are cross-track and pass-to-pass accuracy. In fact, these two parameters represent the most important accuracy indicators concerning precision agriculture requirements and can thus be used to make a qualitative comparison between the two testing campaigns. However, it is important to highlight, from a statistical point of view, as already mentioned, that the comparison between the 2020 and 2021 results is influenced by the differences described in section 2. 3.1. Overall comparison The number of receivers exploited for each test case is not the same between the two testing campaigns, as not all the RUTs had the same configurations available for both testing campaigns, but it is still possible to make several considerations on the outcomes. Figure 2 SPP Cross-track and P2P Accuracy Histograms [m] From Figure 3, it is clear that the GPS single-frequency (and multi-frequency, to some extent) configuration provided better results, with an average 57% fall of the cross-track error in 2021, probably due to the proprietary algorithms, that enhanced solution’s accuracy even if in a harsher environment. Moreover, the pass-to-pass accuracy also had a small (around 23%) improvement, that is probably less evident due to the test setup. As a matter of fact, the presence of a real tractor instead of a carriage favored the 2020 results over the 2021 ones. Galileo-only configurations provided comparable results between the two testing campaigns and highlighted, both in 2020 and in 2021, to be able to provide, in similar conditions, better or equivalent performance w.r.t. GPS-only modes. More specifically, the Galileo single frequency configuration provided, on average, a cross track error diminishing of around the 42% w.r.t. the GPS L1 in 2021. In 2020, as it can be seen in Figure 3, the difference is present (around 30% average smaller cross-track), but it is mainly related to the worse behavior of GPS in the first testing campaign. However, it should be noted again that tests have been performed in different years (e.g., different satellites’ geometries and general constellations conditions) and different testing environments (rail-carriage or real field/tractor), thus slight discrepancies between the results could occur. The triple constellation configuration provided better performance in the first testing campaign, especially in terms of cross-track error, with a 50% improved accuracy performance. However, it is interesting to notice that the GPS + GLONASS + Galileo standalone positioning mode obtained better performance w.r.t. GPS-only configurations in 2020, while this did not happen in 2021. As a matter of fact, the single constellation results obtained in 2021 are better or comparable with respect to the triple constellation configuration results from the 2020 testing campaign, highlighting once again the influence of the proprietary algorithms on accuracy performance. For what concerns the pass-to-pass, however, the results did not highlight any significant difference, except for RX3 and RX6, that showed results not in line with the other RUTs in the agriculture real machineries testing. In Figure 4 cross-track and pass-to-pass errors for SBAS and GPS single-frequency mode are compared with each other. As observed in 2020, an improvement is expected in the accuracy performance of the EGNOS + GPS L1 configuration over the GPS-only one. This trend was evident in the first testing campaign, especially for cross-track accuracy, while in 2021 some of the RUTs had accuracy improvements, and others performed in a slightly worse way, probably due to the significant enhancements in the GPS L1 results. As a matter of fact, the SBAS configuration provided comparable or better performance with respect to 2020 in the second campaign, despite the 10 cm increase in the HNSE (Horizontal Navigation System Error) of the EGNOS open service registered in July 2021 w.r.t. July 2020 [6]. These results confirm that the SBAS configuration, in open sky scenario, is not significantly influenced by the employment of real machineries in agriculture operations and suits perfectly the type B/C requirements reported in Figure 8. In fact, for both the testing campaigns, EGNOS provided, on average, cross track errors of 50 cm and pass to pass accuracies between 10 and 50 cm. As it can be seen from Figure 4, however, RX2 and RX6 had an unexpected behavior in 2021, that brought to results’ degradation. Figure 3 SBAS Cross-track and P2P Accuracy Histograms [m] In Figure 5, the results obtained in PPP mode in the two testing campaigns are shown. It can be noted that the Precise Point Positioning configurations were always analysed in multi-constellation multi-frequency mode, as it represents the most interesting setup for a real user. The obtained cross- track and pass-to-pass accuracies do not show any major discrepancy between the receivers in the two different environments, aside from the unexpected results observed for RX5 in 2021 with the triple constellation configuration. Once again it can be noticed that the triple constellation, as well as the quadruple constellation (also exploiting BeiDou), does not provide significant improvements in open sky scenario, as the number of satellites used for PVT is already sufficient with the GPS + Galileo configuration to obtain a certain accuracy level. For RX4, however, a systematic increase in the cross- track error can be highlighted in 2021 w.r.t. the 2020 testing campaign, with accuracy levels passing from around 2 cm to an average of 6 cm. Overall, it can be stated that the use of real machineries does not significantly increase the error in the main agriculture-related KPIs, but it could cause minor differences in the cross-track error for the PPP mode. Figure 4 PPP Cross-track and P2P Accuracy Histograms [m] For what concerns the RTK configurations, from Figure 6 it is evident that the results obtained in 2021 are degraded w.r.t. the 2020 ones. In particular, the majority of the RUTs obtained results at mm- level in 2020, for all the test cases, and with no significant difference between the tested configurations. On the other hand, in 2021, the receivers showed accuracy performance around 2 cm, with a slight worsening with the triple-constellation configuration. This behavior is partially related to the increase in the baseline of the reference station used for the differential corrections, but it cannot be totally justified by this factor. In fact, when working with cm level positioning modes, as RTK, the influence of the harsh environment faced by the receivers in the 2021 testing campaign can be noticed. Figure 5 RTK Cross-track and P2P Accuracy Histograms [m] As it can be seen in the histograms, RX6 performed under the expectations in all the RTK test cases and RX5 also had issues with the GPS + GLONASS configuration in 2021. Overall, RTK remains the most reliable and accurate positioning mode tested among the two campaigns for precision agriculture applications. 3.2. PPP-RTK added value In this section a comparison between the PPP and PPP-RTK results from the 2021 testing campaign is shown. The results are investigated to highlight any significant difference between the two positioning modes, as the PPP-RTK was exploited only in 2021, thus it could be interesting to look for a possible added value of such a configuration over the PPP mode, that performed in a similar way across the two campaigns. Figure 7 PPP-RTK vs PPP Cross-track and P2P Accuracy Histograms [m] in 2021 It is clear that, on average, PPP-RTK did not show any accuracy performance improvements w.r.t. PPP, as it managed to achieve accuracies between 4 and 10 cm for both cross track and pass-to-pass errors. The added value of such a positioning mode, however, can be seen for what concerns convergence time. In fact, PPP-RTK convergence times, on average, resulted to be around the 1 to 5 minutes range, while in PPP, a real user, would have to wait for 20 to 40 minutes to obtain a converged solution. In conclusion it can be stated that the PPP-RTK mode, even if it is not implemented in the majority of the tested receivers, could be able to provide comparable accuracy results with respect to PPP, but with shorter convergence times. 3.3. Performance benchmarked with typical agriculture requirements In this section the average results are listed and compared with the agriculture specific requirements for all the tested configurations of the two campaigns. As it can be seen in Figure 8, GPS-only configurations are required to obtain m-level results. These accuracies were provided both in 2020 and in 2021, with a slight enhancement in the second testing campaign. For what concerns the triple constellation, the SBAS and the Galileo-only configurations, the target is set at m/sub m-level. In 2021 all the mentioned configurations obtained the required results, being suitable for all the type B/C applications, like soil sampling and precision viticulture. On the other hand, in 2020, the need for some improvements in the Galileo-only configurations was noticed. PPP and PPP-RTK (tested only in 2021) modes are selected as suitable for applications requiring accuracies between 2.5 cm and 10 cm, perfectly in line with the obtained results in both the testing campaigns. It is clear that the RTK mode performed perfectly in line with the very stringent (down to 2.5 cm) requirements in the 2020 testing campaign. On the other hand, the more realistic operational environment together with the increased distance of the base station from the rover, brought to less accurate results in 2021, but still in line with the expectations. In particular, the GPS + GLONASS configuration highlighted some unexpected performance, not in line with the specified requirements for type D applications, like automatic steering and VRA-high (Variable Rate Applications). Figure 6 Average performance highlighted by the testing campaigns. Required cross-track accuracy is defined in [5]. 4. Conclusions This paper analysed, through two testing campaigns, the state of implementation and the added value of the Galileo constellation in the agriculture professional GNSS receivers, considering cross-track accuracy as the main KPI. The comparison was carried out considering different positioning modes: SPP, SBAS, PPP, RTK and PPP-RTK. The results demonstrated that the RUTs provided comparable or better results (under the same testing conditions and with a comparable number of satellites) than GPS-only modes with Galileo-only configurations. Moreover, the Galileo constellation showed to be able to significantly contribute to multi-constellation configurations’ availability and accuracy performance. From the comparison of the testing campaigns, it is evident that the professional receivers were able to provide results in line with the required accuracy performance both in controlled and in operational scenario. In particular, in 2021, in standalone positioning mode, the RUTs obtained sub-m accuracies (with an average of around 60 cm) with the single constellation configurations and m-level accuracies with the triple constellation. These results were slightly worse in 2020 only for the single constellation single frequency modes. On the other hand, SBAS and PPP, for both the testing campaigns, kept the cross-track error inside the required bounds, with results respectively of under 50 and 10 cm. To conclude, RTK showed cm-level results in 2021 and sub-cm results in 2020, mostly in line with the very stringent type D applications’ accuracies. However, it has been highlighted that, the typical working trajectories, together with the tractor vibrations and terrain asperities, contributed to partially degrade the RTK results in 2021, but still remaining in line with the most stringent agriculture requirements. More specifically, the sub-cm level results obtained in 2020 were not repeated in 2021, probably also due to the increase in the distance of the base station sending differential corrections to the rover. 5. References [1] OS Service Definition Document, Issue 1.1 May 2019. [2] https://www.gsc-europa.eu/system-status/Constellation-Information. [3] J. Capolicchio, D. Mennuti, I. Milani, L. Villa, J.R. Gonzalez, M. Sunkevic, “State of the art of the agriculture professional GNSS receivers”, CEUR Workshop Proceedings, June 2021. [4] European GNSS Agency, PPP-RTK market and technology report, 2019. [5] European GNSS Agency, Report on Agriculture User Needs and Requirements, Issue 2.0, July 2019. [6] ESSP, EGNOS Monthly Performance Report July 2021, Issue 01, August 2021.