Autonomous driving is currently one of the main focuses of attention in the automotive industry. A requirement for eficient and safe driving of autonomous vehicles is the ability to precisely pinpoint the location of the vehicle, in the decimeter- to centimeter-level on a global scale. GNSS is expected to play a major role in providing accurate absolute and global positioning, yet many challenges arise in dense urban environments due to lack of line-of-sight to satellites and multi-path, decreasing availability and accuracy. Also, the position accuracy announced by GNSS receiver manufacturers is rather optimistic, typically obtained in best-case scenarios. However, this is rarely encountered in real-world driving conditions, especially in urban areas, leading to a mismatch between receiver specification and real world performance. This paper provides a systematic study regarding the requirements, methods, and solutions available for the characterization/evaluation of a GNSS positioning system in real world driving conditions. An architecture for a precise Automotive Global Reference System (centimeter-level), able to characterize a decimeter-level accuracy GNSS positioning system in dynamic conditions, is proposed. To the best of authors' knowledge, such a study is not available in the literature.
GNSS positioning systems have been providing a wide range of services to the population,
industry and governmental organizations for many years. The improvements in GNSS
technology have been significant in the past decade, with a faster time-to-first-fix accurate position
acquisition, improved receiver sensitivity, more constellations and functional satellites, as well
as improved signals [
In Autonomous Driving (AD), GNSS is expected to play a major role in providing accurate
absolute positioning, with other technologies (e.g. LiDAR, Cameras) providing relative
positioning [
GNSS manufacturers typically announce accuracies obtained in controlled conditions, with direct line-of-sight to a clear sky which is the best-case scenario and not realistic for applications with demanding requirements. Real-world driving conditions are far more challenging for GNSS signals than the best-case scenario. AD is expected to outperform a human-controlled vehicle in terms of reliability and security. Since GNSS positioning is of utmost importance in this context, a full characterization of the system accuracy in real-world is mandatory to guarantee that the system is capable of providing centimeter- or decimeter-level accuracy 24/7.
To evaluate a GNSS receiver with decimetre positioning accuracy (e.g. 20 cm, as previously
defined) a reference system should fulfill the following requirements: (1) provide reliable
absolute ground truth with one order of magnitude better accuracy (e.g. 2 cm, 95% confidence);
(2) maintain high performance (e.g. 99.9% availability [
The main contribution of this paper is a systematic study on the challenges and possible solutions for a suitable reference system able to meet the requirements for accurate characterization and evaluation of precise positioning GNSS systems in real world driving conditions, which the authors could not find in the current literature. Section 2 describes the main parameters for a GNSS receiver characterization. Section 3 and 4, discuss approaches to improve the performance of GNSS positioning, in order to obtain suitable ground-truth to evaluate high accuracy systems in dynamic conditions. Section 5 presents the architecture for the proposed Global Reference System.
To characterize a GNSS receiver it is necessary to obtain a set of parameters that provide
information on the performance of the device when capturing and processing GNSS signals.
There are three dimensions (Fig. 1) where the performance of the receiver is tested: time,
signal power and accuracy [
The time dimension includes the Time-To-First-Fix (TTFF) under diferent conditions (cold
start: no information about the satellite position and time; warm start: valid almanac
information, no ephemeris information, position is within 100 km of last fix and time is known; hot
start: all information is known and position is within 100 km of last fix) [
1
1
1 Initial conditions: Cold, Warm and Hot-start
In the power domain, the minimum power level of the signals is typically evaluated at diferent stages of the signal processing. The acquisition sensitivity parameter is the minimum power level such that the correlators are able to search and identify a signal, which is typically below noise level, until a first fix is obtained. This parameter is also dependent on the initial conditions (cold, warm and hot-start) of the receiver, since knowledge of which satellites to search will speed-up the process. Tracking sensitivity is the minimum power level that allows the receiver to maintain lock of the signal.
The accuracy is divided in two components: static and dynamic. The static parameter can be subdivided into three categories: predictable, repeatable and relative. Static predictable is the accuracy of a receiver’s position solution with respect to a known fixed point of a map. Static repeatable is the accuracy with which a user can return to a position whose coordinates have been measured previously with the same receiver under the same conditions (precision of the receiver). Static relative is the accuracy with which a user can measure position relative to another user with the same receiver in the same conditions. Dynamic accuracy measures the receiver ability to pinpoint the true position of the vehicle in a map, when the vehicle is undergoing motion in any of the axes.
Many of the characterization parameters described above are obtained in laboratorial environment using two types of devices: GNSS simulators (e.g. from Spirent and Rohde & Schwarz) and Record & Replay (R&R) systems (e.g. from Spirent and RaceLogic). The former simulates one or more constellations of satellites, by generating the signals that would be observed by a GNSS receiver in a specific location on earth. The latter records real GNSS data, which can then be reproduced for each receiver under test.
Simulation typically does not address highly complex scenarios. When multi-path simulation
is ofered, it is often a simplistic test for a very specific use case. The influence of moving
objects (e.g. cars, trucks) and the properties of the materials surrounding the receiver (e.g.
trees, buildings) are also absent. Despite allowing testing GNSS receivers under very limited
conditions, this type of devices are expensive (100-300Ke). With R&R systems, data must
ifrst be collected in diferent conditions (e.g. in open area, intermediate/light urban area and
urban area [
While timing and power characterization are well covered by these devices, the same cannot be said for the accuracy parameters. On one hand, the GNSS simulation generates a given coordinate precisely, allowing for direct accuracy characterization, yet presents a simplistic scenario when implementing interference and multi-path. On the other hand, a R&R system captures interferences and replays them, although the exact position of recording may not be well accounted for, especially in dynamic scenarios. In addition, another GNSS receiver is needed, one with higher accuracy than the device under test, in order to characterize accuracy using the R&R system.
However, there is a fundamental issue regarding receiver characterization using only GNSS signals. As mentioned before, the position being estimated is afected by multiple external factors, and ultimately it may contain significant errors, even for the high accuracy GNSS receiver used as reference. Therefore, comparing against another higher quality GNSS receiver, cannot guarantee that the real accuracy is being characterized. Ideally, the system being used to characterize accuracy should be immune to the error sources that afect the device under test. However, considering the requirements defined (e.g., 2 cm 95%, 99% availability and global coverage) for this type of reference system, this is extremely dificult to achieve.
Dynamic accuracy is definitely the most challenging evaluation parameter, but at the same time one of the most important in the automotive context. This problem can be addressed by using GNSS Augmentation and merging GNSS with information from other sensors (e.g., Inertial Measurement Units (IMUs), Odometers, etc), in order to obtain higher accuracy ground truth in dynamic conditions. The following sections introduce the approaches of GNSS Augmentation and GNSS fusion with other sensors.
The typical accuracy of a GNSS system (2-3 m in open sky conditions [
The augmentation can be based on a single reference station or on a network of reference
stations, providing corrections with diferent coverage and accuracy (up to centimetre-level).
With these approaches, satellite position, clock and atmospheric errors can be greatly
minimized, leading to higher navigation performance (improved accuracy, integrity, continuity,
availability). However, not all GNSS errors can be eliminated (e.g. multi-path errors caused
by skyscrapers). Depending on the source of external information used, the augmentation can
be classiefid as Satellite-Based Augmentation Systems (SBAS) or Ground-Based Augmentation
Systems (GBAS). In SBAS [
More recently [
Double differences of code observations from a reference station and a rover. Requires a communications
link between the 2 receivers.
Double differences of code and phase observations from a reference station and a rover. Requires a communications
link between the 2 receivers.
differences of code and carrier phase observations from a local network of reference stations. Corrections (e.g. FKP,
3.1. OSR-Based Standard Positioning Service (SPS) OSR-Based
TIMELINE SSR-Based
through L-Band or FTP, based on the code and carrier phase.
phase bias) through L-Band or FTP, based on the code and carrier phase.
stations. Correction services (orbits + clocks + phase bias + troposphere + ionosphere) through L-Band or FTP, based on the code and carrier phase.
Diferential GNSS (DGNSS) [
GNSS SATELLITE GPS, Galileo, Other
USER GNSS RECEIVER With differential correction
BASELINE < 50 Km TRUE POSITION
GNSS POSITION
GNSS REFERENCE STATION With accurately known position
CORRECTION PARAMETERS
For each satellite
With Real-Time Kinematic (RTK) [
GEO SATELLITE Geostationary GNSS SATELLITE CONSTELLATIONS
GPS, Galileo, Other
CARRIER PHASE AND PSEUDORANGES
For High Precision NETWORK CONTROL CENTER
Compute corrections
CORRECTION PARAMETERS For each satellite
INTERNET A. UPLINK STATION
To GEO Satellite REFERENCE STATIONS USER GNSS RECEIVER With PPP Support
In order to deal with local biases, such as atmospheric conditions, multi-path and satellite geometry, a convergence time is required to achieve decimetre level or better accuracy (typically up to 3 cm). To obtain a 10 cm horizontal error, a convergence time between 20 and 40 minutes is usually necessary. Convergence time depends on the number of satellites available, satellite geometry, quality of the correction products, receiver multi-path environment and atmospheric conditions.
When comparing PPP with diferential processing, the main disadvantage of PPP is that
usually it takes longer to converge [
PPP-RTK can be seen as an extension of NRTK with SSR, or also as PPP with fast ambiguity
resolution [
There are several providers of global PPP correction services, with products where the error and the convergence time vary (see Table 1) [19, 20]. Almost all of them charge a fee to access the corrections. The announced performance is usually measured in static conditions over long periods of time [19].
TerraStar-C 3.3 - 5.3 Static 30 min GPS/GLO [20] , novatel.com TerraStar-C PRO 2.5 Static < 18 min GPS/GLO/GAL/BDS novatel.com
TerraStar-D 4.1 - 5.9 Static * GPS/GLO [19] TerraStar-X2 2 * < 1 min GPS/GLO novatel.com OmniSTAR G2 4.4 Static < 45 min GPS/GLO [19] , omnistar.com IGS (Final)1 2.9 - 5.6 Static 12 - 18 days GPS/GLO [19] , igs.org VERIPOS Apex < 5 Static * GPS/GLO/GAL/BDS veripos.com
StarFire < 5 * * GPS/* navcomtech.com * - No information or unclear. 1 - Free. Remaining are commercial. 2 - Regional coverage. Remaining are global.
Post-Processing techniques can also be used to obtain the maximum accuracy for applications that do not require real-time positioning. In post-processing, data can be processed ofline using forwards and backwards smoothing, allowing to minimize errors that would be obtained in real time [21].
A reference system can fuse GNSS signals with information from other sensors [22], such as Inertial Navigation Systems (INS) and Distance Measuring Instruments (DMI). While an INS, due to integration drift (very significant in lower grade INS), provides an accurate relative measure of position only in the short term, GNSS provides an absolute position in the long term. The integration of INS and DMI technologies allow to complement irregularities in the GNSS with continuous inertial, speed and distance measurements, improving the quality of the ground truth data, even in GNSS signal outages or when the line of sight to satellites is blocked.
An INS uses an Inertial Measurement Unit (IMU) to obtain angular velocity and linear acceleration measurements. These are used to compute a relative position and orientation (roll, pitch and heading) of the system over time in relation to a starting point, by applying dead reckoning techniques. There are diferent IMU grades, usually divided in: marine, aviation (sometimes grouped as navigation grade), tactical and consumer. Each grade has diferent bias [23], with higher grades translating into lower accumulated errors.
IMUs can use MicroElectroMechanical System (MEMS) accelerometers and gyroscopes, or higher quality sensors such as Servo accelerometers and Fiber-Optic (FOG) or Ring Laser (RLG) gyroscopes. FOG and RLG do not contain moving parts, therefore they generally perform much better over vibration and shock. More information about FOG and MEMS gyroscopes can be found in [24, 25].
The gyroscope’s bias is a critical point, since an error in the orientation will translate to an integration of part of the acceleration from gravity (which is usually much greater than the linear accelerations from the vehicle itself) in a diferent direction, leading to drift that, if not compensated, increases exponentially with time [26].
The integration of a DMI into the reference system provides speed and distance information
that can be used to reduce the error accumulation of the double integration process. It can
also be used to detect when the vehicle is immobile, allowing some IMUs to self-calibrate, as
well as for integrity, complementing GNSS Receiver Autonomous Integrity Monitoring (RAIM)
techniques, which are based on a consistency check of satellite measurements [27]. The integrity
requirements for AD are very strict, due to the small safety distances that autonomous vehicles
are required to handle [
Wheel-mounted rotary shaft encoders and non-contact optical sensors are examples of DMIs that can be installed in a vehicle and used in a reference system. Wheel-mounted devices are afected by measurement errors ( ≈ 0.12 km/h [28]). Wheel slipping due to loss of traction, wheel lifting above the ground (e.g. during tight curves or inclined pavements) and tire wear also introduce errors. Non-contact optical sensors provide slip-free measurement of distance, speed or angle and some models can be used at high speeds (up to 400 km/h). They are widely used in vehicles to evaluate parameters such as braking systems, tyres and sideslip angles [29, 30] and in demanding efilds, (e.g. in Formula 1). The downside is the cost of this type of device (≈ 30Ke), when compared with the wheel-mounted option (≈ 5Ke).
There are devices that integrate a GNSS receiver and an INS device, some into a single enclosure. In addition, many of these devices allow the input from a DMI. These integrated devices are used in diferent applications (e.g. mapping and surveying) and the cost of a highend system is virtually unlimited. Many of them can be configured with the state of the art technologies, including high end IMUs, with Servo accelerometers and FOG or RLG.
Depending on the level of integration (GNSS+INS), the device architecture can use loose, tight or deep coupling (Fig. 6). The typical approach used for sensor fusion is Kalman filtering, with: loosely-coupled, the sensor fusion is performed at the solution level (high grade INS are required); tightly-coupled, the sensor fusion is performed at the measurement level (requires more processing power); deeply-coupled, the sensor fusion is performed at the signal processing level (requires feedback to the GNSS measurement engine).
G N I L P CUO INS E S O O L
GNSS
POSITION, VELOCITY, ORIENTATION POSITION, VELOCITY, TIME
KALMAN FILTER G N I L PUO IMU C P E E D
POSITION VELOCITY
TIME ANGULAR RATE AND ACCELERATION CODE, PHASE, TIME, DOPPLER
KALMAN FILTER G N I L PUO IMU C T H G I T
ANGULAR RATE AND ACCELERATION CODE, PHASE, TIME,
DOPPLER
GNSS POSITION, VELOCITY
The benefits of tightly coupled systems are presented in [26], using real-world aircraft and land vehicle datasets. The authors show that a tightly coupled system provides a distinct advantage in urban environments, maximizing the amount of GPS measurements available for aiding in real-time and post-processing. The deep coupling approach uses feedback to the IMU and GNSS receiver, which improves the cold start and the reacquisition time, however most GNSS+INS devices on the market are loose or tight coupling.
Some GNSS+INS devices support multi-constellation and dual antenna, that can be installed in the vehicle (e.g. two meters apart), providing heading estimations from GNSS signals [22, 31, 32], with an accuracy proportional to the distance between antennas. These estimations are a complementing source of heading information, since the use of magnetometers inside of a moving car is not possible due to the harsh magnetic environment. In addition, they can also be used for integrity and to reduce the drift, when the vehicle is immobile or moving at low speeds.
Considering the benefits and limitations of the technologies, services, and approaches discussed above, the architecture presented in Figure 7 was defined in order to meet the requirements presented at the beginning of this paper. The proposed reference system solution is based on a Tightly-Coupled GNSS+INS device, with dual antenna, and an Optical DMI or/and a Wheel DMI to provide velocity information. Another essential element of the reference system is the GNSS correction service. As stated before, reliable centimeter-level accuracy is only obtained with GNSS correction data and post-processing. Although a camera does not provide information that can enhance the performance of the reference system, it is essential, for example, to identify possible sources of perturbation on the data from the other sensors. d. PPP CORRECTION SERVICE
Correction Parameters c1. OPTICAL DMI Speed and Distance b. DUAL GNSS ANTENNA GNSS Signals/ Heading Estimation e. CAMERA
Route Conditions Recording a. GNSS+INS (Tightly-Coupled) Absolute and Relative Position
Orientation c2. WHEEL DMI
Speed and Distance
The performance of the reference system is directly linked to the selected GNSS receiver and IMU. However, there are hundreds of these devices on the market, and selecting the best combination is a challenging task. There are several GNSS+INS as well as DMI devices on the market with diferent characteristics and cost. The technologies in these devices are usually protected, therefore the information regarding the devices’ characteristics and operation is very limited. Gathering information such as the one presented in Table 2 is dificult, because data sheets do not fully specify the conditions under which the tests were performed, making the comparison impossible or unfair. As we can see in Table 2, GNSS+INS devices with very distinct characteristics and price range (≈ 20 - 100+Ke) announce similar positioning and orientation performance.
Considering the information available from the manufacturers and presented in Table 2, all these devices report horizontal accuracy of 2 cm after applying DMI information, correction data, and post-processing. However, as mentioned earlier, these performances are usually obtained for best-case scenarios (e.g., open sky conditions), leading to diferent performance in real-world conditions. Moreover, the high price tag of these devices, limits the access to compare multiples devices in fair experiments with the same conditions.
Without reliable information, we opt to base the selection criteria on the characteristics and limitations of the technologies, and not on announced performances. This is the reason why a systematic study, such as the one presented in this paper, is important. In this context, considering these aspects, a device with Servo accelerometer, RLG or FOG, dual antenna, and DMI support, is one of the best candidates to obtain a stable 2 cm accuracy in real-world conditions.
As mentioned before, the correction service is also one of most important parts to achieve high accuracy on a global scale, and from the announced performances (usually also for optimistic scenarios) (see Tab. 1), TerraStar is one of the services with higher performance. However, it is also important to consider that some of the GNSS+INS devices only work with a limited set of correction services or even with only a single proprietary one. Therefore, the correction service must be selected considering the GNSS+INS device.
Other practical aspects must also be considered when choosing a GNSS+INS device, such as the fact that some of these devices may not be ITAR free (US International Trafic in Arms Regulations), and these restrictions can lead to shipment delays or even in limitations of use in some locations.
In this paper, the challenges related to the characterization and evaluation of GNSS systems for precise automotive positioning, in real-world driving scenarios, were discussed, resulting in an architecture that is proposed as adequate for an Automotive Global Reference System. Several technologies must be combined to create a reference system able to obtain precise ground-truth. High-grade dual antenna, multi-constellation GNSS+INS devices (with Servo accelerometer and RLG), as well as an optical DMI device, ensure the best available technology for this type of reference system. However, the high cost is a limitation, when considering worldwide tests with multiple vehicles. Correction services were discussed since they play a major role in achieving centimetre-level accuracy on a global scale. The specifications of most of these services show that in post-processing, it is possible to obtain consistent and accurate positioning. Therefore, since real-time evaluation is not usually required in the discussed context, and RTK is not practical in urban environments and for worldwide testing, the use of post-processing techniques is the ideal approach.
One of the main challenges in designing a reference system solution is that the performance promoted by the manufacturers of GNSS+INS systems is very similar, despite very distinct technologies and cost. The lack of real-world experiments conducted by independent researchers makes it dificult to find the ideal cost/performance balance. Therefore, a future work goal is to test diferent GNSS+INS systems in the same real world driving conditions and compare the obtained results.
This work has been supported by: European Structural and Investment Funds in the FEDER component, through the Operational Competitiveness and Internationalization Programme (COMPETE 2020) [Project no 037902; Funding Reference: POCI-01-0247-FEDER-037902]. [17] G. Wu¨bbena, M. Schmitz, A. Bagge, PPP-RTK : Precise Point Positioning Using StateSpace Representation in RTK Networks, Proc. 18th Int. Tech. Meet. Satell. Div. Inst. Navig. (ION GNSS 2005) (2005) 2584—-2594. URL: http://ww.ion.org/publications/ abstract.cfm?articleID=6467. [18] B. Zhang, Y. Chen, Y. Yuan, PPP-RTK based on undiferenced and uncombined observations: theoretical and practical aspects, J. Geod. 93 (2019) 1011–1024. URL: https://doi.org/10.1007/s00190-018-1220-5. doi:10.1007/s00190-018-1220-5. [19] A. Jokinen, et al., Kinematic performance of NovAtel CORRECT with terrastar-D Precise Point Positioning (PPP) service, in: Proc. 27th Int. Tech. Meet. Satell. Div. Inst. Navig. (ION GNSS’14), 2014, pp. 1020–1034. [20] N. White, P. April, Precise Positioning with NovAtel CORRECT Including Performance
Analysis (2015) 1–12. [21] G. Roesler, H. Martell, Tightly Coupled Processing of Precise Point Positioning (PPP) and INS Data, Proc. 22nd Int. Tech. Meet. Satell. Div. Inst. Navig. (ION GNSS 2009) (2009) 1898–1905. URL: http://www.ion.org/search/view{ }abstract.cfm?jp=p{&}idno=8598. [22] P. D. Groves, Principles of GNSS , Inertial , and Multisensor Integrated, 2008. [23] P. D. Groves, Navigation using inertial sensors, IEEE Aerosp. Electron. Syst. Mag. (2015).
doi:10.1109/MAES.2014.130191. [24] T. Ford, et al., GPS/MEMS inertial integration methodology and results, in: Proc. 17th
Int. Tech. Meet. Satell. Div. Inst. Navig. ION GNSS 2004, 2004. [25] O. Deppe, et al., MEMS and FOG technologies for tactical and navigation grade inertial sensors—Recent improvements and comparison, Sensors (Switzerland) 17 (2017). doi:10. 3390/s17030567. [26] S. Kennedy, D. Cosandier, J. Hamilton, GPS / INS Integration in Real-time and Postprocessing with NovAtel ’ s SPAN System, Int. Glob. Navig. Satell. Syst. Soc. IGNSS Symp. (2007). [27] M. J. Psan´ikowski, et al., Challenges for integrity in navigation of high precision, in: 28th
Int. Tech. Meet. Satell. Div. Inst. Navig. ION GNSS 2015, 2015. [28] J.-h. Han, C.-h. Park, Performance evaluation on GNSS, wheel speed sensor, yaw rate sensor, and gravity sensor integrated positioning algorithm for automotive navigation system, E3S Web Conf. 94 (2019) 2003. doi:10.1051/e3sconf/20199402003. [29] J. Ondrus,ˇ E. Kolla, Practical use of the braking attributes measurements results, in:
MATEC Web Conf., 2017. doi:10.1051/matecconf/201713400044. [30] S. Joanny, C. Ali, M. Dominique, Evaluation of sliding mode observer for vehicle sideslip angle, in: IFAC Proc. Vol., 2005. doi:10.1016/j.conengprac.2006.04.002. [31] K. Gade, The Seven Ways to Find Heading, J. Navig. (2016). doi:10.1017/
S0373463316000096. [32] F. Zhu, et al., Dual-Antenna GNSS Integrated with MEMS for Reliable and Continuous Attitude Determination in Challenged Environments, IEEE Sens. J. (2019). doi:10.1109/ JSEN.2019.2891783.