This article describes the use of a monocular camera attached to a multirotor perpendicular to the horizon, to recognize visual cues or artefacts (AprilTag) and use it as an anchor for aerial alignment to finally land on it, thus attempting to make autonomous flights safer and usable in slightly hard-to-reach locations. A Hexacopter frame with the DJI N3 flight controller was used for prototyping and realizing the desired algorithm. Factors like wind speed and gusts were taken into account as well as the center of gravity of the multirotor and the position of the molecular camera attached to the copter facing downwards or at a 90-degree angle. The results of the experiments conducted were verified against existing methods like the GPS (Global Positioning System) waypoint mission provided by major commercial Unmanned Aerial Vehicle(UAV) or Flight Controller manufacturers and were also compared to experimental methods presented in related research articles, fairing excellent results *This research was supported by a grant from - “The Tomsk State University Competitiveness Improvement Programme”
Multirotor unmanned aerial vehicles have come a long way since its inception when
auto stabilization was a dream. Today most off-the-shelf consumer multirotors are
equipped with features like autonomous flight, GPS waypoint-mission, optical-flow
stabilization and a lot more. An important aspect of autonomous flight is GPS or Global
Positioning System. It uses GNSS to triangulate a position approximate to 5 meters [
These features enable multirotors to navigate through the skies with ease, and in recent years the industry of multirotors has grown due to the consumer interest in these devices. Manual control of these multirotors are as safe as the pilots but autonomous 2
Assisted landing has existed for a while in open-source flight controllers like the
pixhawk, using an infrared emitter beacon and receiver [
Although this system is fairly accurate most of the time, it's reasonable to find alternatives. One can suggest the use of specialized colors and landing markers to use for this process but it might also be mistaken by the camera in many situations. This method uses an AprilTag detector system developed by the team of APRIL Robotics Laboratory at the University of Michigan, to get the coordinates of a marker in 3d space.
AprilTag is a visual fiducial system, useful for a wide variety of tasks including
augmented reality, robotics, and camera calibration. Targets can be created from an
ordinary printer, and the AprilTag detection software computes the precise 3D position,
orientation, and identity of the tags relative to the camera. The AprilTag library is
implemented in C with no external dependencies. It is designed to be easily included in
other applications, as well as be portable to embedded devices. Real-time performance
can be achieved even on cell-phone grade processors [
It is theorized that using a monocular camera and the April tag library to recognize and localize an artefact (printed April tag) as a marker will provide ample guidance for a multirotor to land at a designated location 2.1
Platform
1. Ground Station - A Basic Computer, preferably with Linux.
2. NVIDIA Jetson TX2 - A Single Board Computer, compact enough to be mounted
on a drone. The NVIDIA Jetson TX2 is a full-featured development platform for
visual computing. It comes with a Linux environment, which includes support for
many common APIs, and is supported by NVIDIA's complete development
toolchain.
3. Monocular Camera - Camera pointed downwards.
4. Flight Controller – A flight controller that will follow a path set by the assisting field
computer and perform various switching tasks. The DJI N3 flight controller is made
by DJI for industrial applications. Its robust flight control algorithm is ideal for
controlling a broad range of industrial and DIY multirotor aircraft, providing
professional stability
Software:
1. ROS- ROS or Robot Operating System [
Please note- An additional software- ‘DJI Assistant’ was used since the aforementioned platform used the DJI N3 Controller. (For ease of operation, running simulations and using additional features provided by the controller).
Left Side(Drone). The DJI onboard SDK for Robot Operating System is the software heart of this system, which controls the Flight Controller: DJI – N3. It is the flight controller SDK which has a lot of services, subscribed and published topics which we can use as inputs for algorithms and actuator controls which be used for movement. The UAV should also have a Camera pointing downward at the ground to look for the artefact.
Right Side (Ground Station). The Right Side of figure 2, contains the Ground Station i.e. the computer we use to connect to the field computer or the computer mounted in the drone to view back box data in real-time. It is also used to start all the scripts for sending autonomous flight commands to the controller.
Camera Assisted Autonomous UAV Landing 5 3
Airports have used high-intensity runway lights(HIRL) on the landing strips or runways to guide a pilot to a safe landing. Airports also have Pilot Controlled Lighting, or PCL, where pilots can adjust the lighting themselves by keying a microphone button a certain number of times.
High-intensity lights would be very useful as guidance markers for an Autonomous
Unmanned Aerial Vehicle. The use of HIRL is an aid to the human pilots and thus need
to be on the visible spectrum of lights. Although, HIRL is not used for assistance here,
an attempt to realise the same concept with High-Intensity Lights can be made in future.
To realise the aforementioned algorithm [
/flight_control_setpoint_generic /tag_detections /apriltags_ros
According to the flowchart (fig. 3) in the previous section, the landing sequence initializer takes over control after the GPS/Visual waypoint mission is finished. At this stage the UAV is at Z altitude, hovering near the landing area. The hover point may have an error radius of 5-10 meters depending on the GPS signal or the visual odometry setup used.
When the landing sequence is initialized, the onboard camera on the UAV looks for pre-configured AprilTag artefact in the ground. The apriltag_ros publishes the detected tags, in the /tag_detections topic as approximate x,y and z coordinates in 3d space (in meters). The accuracy of the detections depends on the camera calibration, and even with the best calibration, it can have discrepancies.
The x,y and z coordinates are then transformed as required by the flight controller. The transformation is defined in equations (1), (2) and (3):
AT – AprilTag and FC – Flight Controller.
The transformation will vary depending on the coordinate system used by the flight controller. Using the new coordinates, we need to calculate the delta distances or the difference in distances:
AT x → FC x, AT y → FC y, AT z → FC z.
Δx = Px – FC x,
Δy = Py – FC y, Δz = (Pz – FC z) * –1. (5) (6) where, Px, Py and Pz are the alignment coordinates or the ideal position the UAV needs to reach. (Δz is negated, as negative values sent to the flight controller to reduces the altitude)
The ideal coordinates for the UAV used in this experiment are as follows: Px – 0, Py – 0.3 and Pz – 0. The adjusted delta coordinates are sent to the flight controller by publishing to the /flight_control_setpoint_generic topic after applying a PID controller, to for smoother movements. Once the ideal position is achieved, the motors can be turned off as the UAV has landed. 5
*The approximate wind data was taken live from windy [
**The precision landing used by Robotic Institute at Carnegie Melon University is better at landing in un-even landing environments, whereas, under perfect conditions, AprilTag based landing is superior.
*** The same copter was used for doing the tests.
Figure 5 shows the position of the UAV after the autonomous landing sequence is completed. Since the UAV landed on the marker accurately with a negligible error of 0.15 meters, this test can be considered a successful one. On the other hand, if only GPS was used, a 5-meter radius of error can be expected. 6
An attempt to make autonomous flights safer and more useable was made by using a monocular camera pointed downwards, in the UAV multirotor. This monocular camera, in turn, recognises artefacts or markers(AprilTag) and publishes an estimated location. The location of the tag is used as an anchor to align the UAV in flight. While in flight, the UAV lowers the altitude while maintaining position lock with the marker. Once the altitude reaches 0 the UAV has safely landed. This approach had an error of 0.15 meters and thus effectively a tight and small landing zone as small as 0.30 - 0.40 meters can be used.
Light-based markers are used in airports to guide pilots while taking off and landing.
Since a computer controls movements in an Autonomous UAV, any kind of markers can be used. If light-based markers are used, they need not be in the visible spectrum and thus Infrared Light Markers can be used to mark a landing area for a UAV. A control system for precision landing using infrared markers can be made for a future project