The accessibility of positioning information derived from Global Navigation Satellite System (GNSS) is arguably one of the main drivers that has transformed the way spatial data is currently generated and consumed. As the form factor and power consumption of GNSS enabled devices starts to scale down, the technology will further become pervasive and ubiquitous. This is applicable to both safety and convenience related applications. Depending on what receiver is used, the correctness and reliability of the positioning information may greatly vary and end users may not necessarily be aware of the capabilities and limitations of the receiver being used. In this respect, the blind reliance of end users toward this technology has raised concerns within the GNSS community. As a part of a broader study on end user needs for GNSS testing, a survey was conducted to investigate the level of awareness amongst end users towards GNSS performance. This paper elaborates on the results of the survey. The findings of this study, together with this needs analysis are used as a basis for principles and recommendations of end user GNSS testing standards and certification guidelines in the context of Australia.
Global Navigation Satellite System (GNSS) derived coordinates are used in most spatial related applications. From data collection and analysis to real time position tracking, users rely on correctness and reliability of locational data to be presented within the desired tolerances. As locational data become increasingly ubiquitous, it is often difficult to diagnose the weakness of an operational system.
However, users can make a conscious decision in choosing and maintaining a suitable GNSS receiver which complies with the required positioning performance. Positioning performance is measured according to a system’s accuracy, integrity, continuity, availability, interoperability and timeliness [1,2].
The term positioning performance is perceived differently according to application, and research is being conducted to examine the needs and principles for end user GNSS testing. The scope of end users in this paper is taken from a broad perspective, and is not limited to handheld device users, but also GNSS derived data and non-manufacturer aligned GNSS data correctional service providers.
This paper presents the outcomes from an online survey that was conducted as part of this investigation. The focus of analysis is centred on the level of awareness of end users towards the technology and in-depth discussion on the overall results will be presented in a separate publication. This paper will elaborate on the need to educate and inform end users on standardised receiver testing guidelines and certification.
An online questionnaire, focused on GNSS user testing requirements, sampled 70 individuals representing different GNSS dependent sectors: construction and surveying (CS) with 25 respondents; land management (LM) with 14 respondents; aviation (AV) with 2 respondents, maritime (MR) with 4 respondents; agriculture (AG) with 2 respondents; road and rail transportation (RRT) with 3 respondents; location based services (LBS) with 9 respondents; and academic research (AR) with 11 respondents. The plain language statement of the questionnaire can be accessed at [3]. The volunteer respondents were either approached individually or received an email via sector specific mailing lists and forums. Australian participants totalled 81% of the respondents, with overseas respondents from
Questions and Results
How confident are you with the positioning information given by a GNSS receiver?
Classify the impact of the following GNSS receiver weaknesses on your operations: Occasionally erroneous coordinates or unrealistic coordinate quality indicators.
Response Not Applicable (Low) 1 Coordinate quality indicators of the receiver are vital in providing the usability of the coordinates for any location. The results in Figure 2 indicate varied opinion on the impact of those coordinate quality indicators. A total of 33% of responses provided high impact on quality indicators, 37% provided low impact and 23% of respondents provided a neutral response. A possible explanation for 60% of respondents providing a neutral or below response is that users may be unable to quantify the impact. This 60% constitutes 16 out of 25 from construction and surveying, 8 out of 14 from land management, 7 out of 9 from location based services, 5 out of 11 from academic research, 1 of out 2 from agriculture, 2 out of 4 from maritime, and 3 out of 3 from rail and road transport. These results also indicate that GNSS receivers are mostly reliable for its intended purposes and unrealistic quality indicators are uncommon, even in sectors with trained users such as construction and surveying. Classify the impact of the following GNSS receiver weaknesses on your operations: Unpredictable receiver behaviour under non-optimal signal environments Number of Responses 5 11
This question is closely related to the previous as receivers have a tendency to behave erratically under difficult GNSS environments. Shown in Figure 3, 65% of the responses were 3 and above, with the remaining 34% being 2 and below.
The response indicates that users are aware of the direct relationship between a GNSS receiver’s operational environment and its impact on receiver weaknesses. With 38% of respondents indicating high impact, standards which address unpredictable receiver behaviour under non-optimal signal environments should be adopted. This 38% includes 9 out of 25 from construction and surveying, 9 out of 14 from land management, 2 out of 9 from location based services, 6 out of 11 from academic research, 1 of out 2 from agriculture, 2 out of 4 from maritime, 1 out of 2 from aviation and 1 out of 3 from rail and road transport. In this respect, the transportation and land management sectors appear to be more aware and affected by such environmental limitations.
Classify the impact of the following GNSS receiver weaknesses on your operations: Older hardware models do not perform as well as newer models
Response Number of Percentage Participant Breakdown (Count)
Responses (%) Not Applicable 14 20 As shown in Figure 4, users’ perception of the physical receiver indicates that the age of the hardware is not indicative of its performance. With half the respondents acknowledging that older models work as well as newer models, testing for this variable may be unnecessary for the user. It is important to note that 20% of respondents selected ‘Not Applicable’, implying that a significant portion of users have not used older hardware, given GNSS’ relative newness. Indirectly, the results may signify difficulty for users to compare hardware performance and positioning performance. This is also reflected in the wide distribution of responses throughout different sectors, particularly from the construction and surveying, and land management sectors which constitutes the majority of participants.
Apart from testing coordinates against a known position, are there any other GNSS related tests routinely conducted?
Number of Responses 22 48
Percentage (%) 31 69
Participant Breakdown (Count) CS (9), LM (4), LBS (1), AR (2), AG (1), MR (2), AV (1), RRT (2).
CS (16), LM (10), LBS (8), AR (9), AG (1), MR (2), AV (1), RRT (1).
Response
Yes
No Response
Yes No
Having considered the limitations of the receivers, users were then asked to identify whether any GNSS related tests were conducted to validate positioning performance. The results in Figure 5 show that 69% of users conduct no further testing, which validates the users’ perception of confidence in positioning performance. Qualitative investigation users who conduct further testing are mostly focused on data quality control and radio frequency interference detection. However, this is limited to more specialised users.
Would independent tests directly benefit the organisation?
Number of Responses 40 30
Percentage (%) 57 43
Participant Breakdown (Count) CS (17), LM (8), LBS (3), AR (6),
MR (1), AV (2), RRT (3).
CS (8), LM (6), LBS (6), AR (5),
AG (2), MR (3).
Whilst the previous response indicates that further testing is limited within the user base, the results from this final question shown in Figure 6, signify that users perceive the usefulness for further receiver testing. With 57% of respondents indicating that further testing would benefit their organisation and 69% having responded previously that no further testing was being conducted, the results indicate that guidelines for testing may not be sufficient. This is evident within the construction and surveying and land management sectors where data traceability is of importance.
The 43% of respondents who answered ‘No’ were asked to provide a reason why they saw no benefit. The qualitative analysis revealed that most specified that manufacturer tests are sufficient and coupled with the cheap price of the receiver, the cost of testing was unjustified. Users believe there are more critical aspects to the system that need to be examined. This is particularly true within the location based services sector where GNSS positioning is perceived as a secondary function. DISCUSSION AND LIMITATION OF STUDY Currently, users’ confidence in GNSS receivers is derived from the manufacturer’s testing. Whilst users indicate that manufacturer tests are mostly sufficient, there is evidence that users recognise the benefits of independent testing and certification for more specialised applications; particularly those involving navigation and safety-of-life purposes. In addition, typical operational end user environments could vastly differ from manufacturer testing environments, thus justifying independent end user testing. In this respect, the aviation [4] and maritime [5] industry requires all GNSS receivers used for commercial-based navigation to be independently certified. The research aims to introduce such guidelines and standards beyond this scope.
Despite being aware that performance is limited by environmental factors, respondents find it difficult to gauge the performance of one receiver model to another. Users would be aided by standardised testing procedures and principles underlying an independent test bed, allowing individual receivers to be reliably validated and certified. This issue will be addressed in future papers.
One of the main challenges of the survey was to populate a large sample of respondents. This is due to the relative high level of knowledge required to understand the principles of this technology and it appears that many end users may be intimidated by the field specific questions. The researchers also acknowledge that it was difficult to approach participants within all transportation and agriculture sectors. Due to the small number of responses and similarity of its applications, the analysis from these sectors were treated as one generalised sector.
Although a small overall population sample of responses were gathered, the significance of the results are justified from the quality of responses given and the moderate level of expertise of chosen participants. The filter questions aided the process to exclude erroneous and irrelevant entries. From this filtering process, participants who use GNSS for more generic and mobile phone-based applications were categorised as location based services users.
Despite having this questionnaire widely distributed, the small number of respondents gathered also validated that the level of GNSS ubiquity has made many end users disconnected from the expectations of a GNSS receiver. Consequently, specialised end users such as system developers, service providers and system integrators bear the onus to ensure the receivers are operational to their minimum standards.
Only a small portion of the original questionnaire results were included in this paper so as to highlight the level of awareness of end users towards performance and testing. Responses from GNSS equipment providers and manufacturers were also excluded due to the end user focus of this paper. A separate publication will be written to explain the detailed results along with the qualitative study conducted.
CONCLUSION Overall results of the questionnaire is discussed, and it is shown that most GNSS users are confident with its positioning performance. However, there is a wide distribution of user knowledge and expectation on positioning performance under difficult operational environments. From the results, it appears that users are not overly concerned about comparing relative performance between receivers.
The outcome of this investigation is used to validate the criteria needed to establish an independent end user test and certification, particularly in the context of Australia. The varying levels of responses suggest that a hierarchy of tests need to be defined, depending on the level of complexity required. With such guidelines on standards being established, the test bed envisages users to reliably test their GNSS receivers, and in turn provide more confidence and traceability on the receivers being used. ACKNOWLEDGMENTS This research is funded by the Cooperative Research Centre for Spatial Information and its industry partners; ThinkSpatial, Geoscience Australia, and DELWP Victoria. The first author thanks the supervisors; Allison Kealy, Simon Fuller, Mark Moreland and Phil Collier in providing insight and guidance with the research. A special mention is given to Victoria Petrevski for the many hours spent to proof read the questionnaire documents and papers. Finally, to all survey participants: the outcomes of the study would not have been possible without your contribution. [1] AUSROADS Evaluation of the Potential Safety Benefits of Collision Avoidance Technologies Through Vehicle to Vehicle Dedicated Short Range Communications in Australia, Austroads Research Report, 2011.
Proc. of the 3rd Annual Conference of Research@Locate 11 [2] ACIL ALLEN CONSULTING The Value of Augmented GNSS in Australia: An Overview of The Economic and Social Benefits of the Use of Augmented GNSS Services in Australia, Prepared for the Department of Industry, Innovation, Climate Change, Science, Research and Tertiary Education, 2013, 12. [3] TENGKU, R. Initiating the Development of a Test Facility for Global Navigation Satellite Systems (GNSS) Positioning Validation and Certification: Plain Language Statement, 2014. Available at https://www.dropbox.com/s/408bnw740yo08uc/PLS%20Online%20Survey.pdf?dl=0 [4] ICAO AERONAUTICAL COMMUNICATIONS, Annex 10 to the Convention on International Civil Aviation, International Standards and Recommended Practices, Radio Navigation Aids, International Civil Aviation Authority (ICAO), 2008 [5] IEC, IEC 61108:2003 Maritime Navigation And Radiocommunication Equipment And Systems Global Navigation Satellite Systems (GNSS) Geneva, Switzerland: International Electrical Commisson (IEC), 2003.