New Interaction Interface with Mobile Robots

De Ensiwiki
Aller à : navigation, rechercher
Project schedule.png
Titre du projet New Interaction Interface with Mobile Robots
Cadre Projets de spécialité
Page principale Projets de spécialité image

Encadrants Céline Coutrix




Drones are getting more accessible to the general public, however, they continue to be hard to use and keep being a niche item that only those who put the time into getting to truly enjoy. Professional controllers are too bulky and confusing, overall unpleasant to use. This can lead to interested people getting frustrated since the learning curve can be very steep. Our goal is to smooth the learning curve by giving people an alternative easier start with, but still with a satisfying degree of freedom. Inspired by the previous implementation of Wii remote to control the drone, we decided to follow this approach but instead using HTC Vive's wands.


Our purpose is to utilize an HTC Vive controller set to reduce the control complexity of the classic controller and take advantage of the motion properties of the HTC to make the control more intuitive.

Figure 1 - Classic drone controller


RealFlight 8

To test this approach we used the flight simulator RealFlight 8, which support VR so that we can make use of the Vive controllers. However this software doesn't recognize the HTC Vive in its settings. To solve this problem we resorted to another third-party driver called vJoy, it make the simulator recognize the HTC Vive as gamepad / joystick.


This third-party driver is capable of receiving data from one input, and after translating it to joystick input. Meaning it maps data from one virtual one to the application. For our purpose, it was used by receiving data from our HTC Vive's control handler and feed them to RealFlight 8. Like this, it's possible to use the HTC on the RealFlight 8.

Vjoy Monitor screen

HTC Vive Drone Controller

Our program will map the Vive wand's positions and movements into controller's value so that VJoy can feed them into the simulator.



Although button mapping can be modified, we use the grip button to enable the control. While the trigger is used to balance the control inputs. The trigger is found to be useful in the context of the drone spinning wildly. Pressing both triggers will make the drone stop moving but still hovering in the air

The rumble is added to notify the user that the control value reached its peak.

RC mode 2 on Vive

Based on the typical RC transmitter this implementations has the control of the throttle and yaw on the left hand, and the controls for roll and pitch on the right hand.

The left controller when moved along the Z-axis will manipulate the throttle of the drone. Intuitively, when the controller is pulled up, the throttle increases. And when pulled down the throttle decreases. For the yaw, it moves left and right.

The right controller is in charge of the pitch, the roll To control the pitch the controller needs to be moved up and down. And the roll is controlled by rotating the controller.

Figure 2 - RC controller

Airplane mode

At first we tested the HTC Vive with the behavior of a classic drone controller, making the yam and throttle controlled by one hand, and the pitch and roll in the other hand. However this configuration turned out to not be very intuitive, making us switch to a more conventional method of an airplane stick. This meant having the throttle in one hand, and the other three components in the other.

Yaw Axis Corrected.png

The left controller when moved along the Z-axis will manipulate the throttle of the drone. Intuitively, when the controller is pulled up, the throttle increases. And when pulled down the throttle decreases. The right controller is in charge of the pitch, the roll, and the yam. To control the pitch the controller needs to be moved up and down. For the yam it moves left and right. And the roll is controller by rotating the controller


Using Realflight 8 challenges

In order to evaluate the participant's familiarity with the controller, we use three basic RealFlight 8's drone challenge to record the time of finish and the number of trials before successfully pass the level. The maximum number of trial for each level is 15. If a participant cannot pass it, he/she won't take the higher challenge level.

The first one, participant's goal is to pass the gate under one minute

Drone Challenge 1

The second challenge, participant's goal is to land on the platform under one minute

Drone Challenge 2

Lastly, combining previous challenges, participant's goals is to pass the gate and land on the platform under one minute

Challenges result

With a small group of just 6 users we decided to test them all in the three control configurations we had. So two of them started by testing out the classic controller, then the RC-2 mode and only after the airplane. The other two groups test in the two other different permutations to keep the data unbiased.

We timed all three challenges and after calculated the gain for the RC-2 and the Airplane mode compared to the Classic mode. For exemple, a given subject took 5.47s to do the first challenge on the classic mode, and 4.17s to do the same challenge on the RC-2 mode, this gives the time to complete the challenge 76.23% of time, result in 23.77% performance boost. After calculating each individual gain we made a mean average for each challenge. We counted also the number of failed attempts until the subject was capable of completing the challenge. The maximum number of trial for each challenge is 15.

Challenge RC-2 Mode # Failed attempts RC Airplane Mode # Failed attempts Airplane # Failed attempts Classic
1 52.35% 1.5 63.42% 1.67 4.83
2 36.25% 6.17 38.67% 2.83 2.17
3 51.47% 4.75 53.01% 7.0 3.83
Average 46.69% 4.14 51.70% 3.83 3.61

Performance chart

We had an positive result on both of our new approaches with an improvement on the time to complete on both the implementations. However, the number of failed attempts with the classic controller was lower, all the while taking longer for participants to actually complete it.

Failed Trials chart

At the end of the tests we asked each participant on a scale of 1 to 10 the ease of use of each control mode, 1 being impossible to control, and 10 being having absolute control of the drone. Our results matched the gain percentage that showed up before, with the airplane control mode being the easiest, having only a slight improvement over the RC-2 mode.

RC-2 Mode Airplane Mode Classic Mode
Easiness level 4.8 5.6 3.6

This results are not the most reliable due to the small number of participants but given that all subjects improved in every challenge. We can then conclude that even though the gains would change if the pool of subjects was bigger, the results would still be a positive one.


The project experiment result

Firstly, according to the comments of the participants, the implementation of vive using airplane mode is the easiest to use with the highest average score both on time and easiness of use. The reason of it due to the intuitive feeling of controlling yaw, roll and pitch in one controller just like having a real airplane on the hand, moving it also make the posture same in the simulator.

The RC-Mode 2 on Vive arrive at the second with the lower score than the other Vive mode come from the fact that sometimes, participant forget that rudder control is on the left hand.

Easiness score of the modes


In fact, the result of controlling does not satisfy us at all. While the controlling behavior is simplified, using one wand to control 2 axis result in interference of an axis while moving other. Still, participants who have already have experience with RC still prefer using Airplane mode on Vive wand than the physic controller. Therefore their improvement on results is not affected by the order of experiments.

Before the pilot test, we implemented the way to use the touchpad's buttons to restrict the axis change while the wand moves but it increases the complexity of the controlling. Sometimes, the tester was frustrated with remembering which buttons to use. So to simplify, we decided to let the interference of value between the axis and for the experienced participants result suggest that this weakness can be overcome with user's practicing.

During the experiments, the users often complain about the camera's viewpoint. Sometimes, it's hard to visually recognize the position of the drone. Moreover, users find it hard to identify the front and back of the drone. However, these drawbacks can be overcome in the real environment as the future expansion of the project.


Our project is hosted in Github via the link: