Interactive building of graphical models using multi-touch interfaces

De Ensiwiki
Aller à : navigation, rechercher
Project schedule.png
Titre du projet Interactive building of graphical models using multi-touch interfaces
Cadre Ensimag

Encadrants Francois Berard

Students

Context

Graphical modelling has been a very ubiquitous and useful form of representing knowledge, processes and algorithms. Due to their function, graphical models can be used in many different settings such as brainstorming, teaching or general collaborative tasks where two or more people interact with a model at the same time. Application of these models can be observed in UML, Neural Networks, Finite State machines, concept maps and mental maps.

Currently, the main way of creating graphical models is either through text-based editors or using collaboration tools online. These may not be optimal to facilitate discussion when creating the models or even learning about these, since real-time multiple user manipulation is not possible on the same device.

Problem

When building graphical models in teams, there are usually limitations for how many people can interact on just one station in the same location. Where the more people collaborating could lead to the need for more stations. Also, intuition when building and/or displaying such graphical models can also be hard to understand by using some of the current description methods (such as code).

Using a multi-touch display for collaboration could lead to the production of high-quality results quickly and the reuse of tried and tested solutions (components) effectively. It may also help to lay out complex information in a concise manner, by providing a layer of abstraction to make them more understandable and intuitive. And mainly, to facilitate collaboration between parties by using just one device to work on simultaneously.

This is why the main question for this project is the following: Is it more efficient to create graphical models in group by using a multi-touch table rather than multiple stations?

Approach

We propose that the use of a multi-touch display for collaboration could lead to the production of high-quality results quickly and the reuse of tried and tested solutions (components) effectively. It may also help to lay out complex information in a concise manner, by providing a layer of abstraction to make them more understandable and intuitive. And mainly, to facilitate collaboration between parties by using just one device to simultaneously work on.

Our Prototype

We decided to create the graphical model builder for the Muti-touch Table (MTT) using Python and the Kivy graphical library that integrates the Multi-touch functionality. We focused on implementing simple graphics that would be functional enough to go along with the experiment plan. When holding down the finger over the workspace (MTT) a Menu appears where the user can choose from 3 options ; create a shape that will be displayed at the touch position (Square, Triangle or Circle) , create a link from the touch down position to the touch up position and clear operations (delete the last created widget (shape or link) or clear the whole workspace). Moreover, shapes can be deleted by being swapped to either the upper right corner or lower left corner of the workspace. Links can only be removed by deleting the last object. Also, an opened Menu can be removed by touching its red center. With those simple functionalities, the user can now fully reproduce a simple graphical model .

Experiment

Model to be reproduced by the participants

For the experiment, groups of three participants would reproduce a given sample model in three separate stations (PC) using Google Slides and the Muti-touch Table (MTT). Where number of interactions (touches for MTT and clicks for PC), time to task completion, number of utterances, total number of misplaced shapes and total number of misplaced links were recorded for each task.

Before beginning with the experiments, the participants were given a brief introduction on how to use the MTT and 5 minutes to interact with it. After this, one of the experiments was randomly assigned to be the first they would perform. Participants were not allowed to use the keyboard or copy and paste operations on the PC since these were not part of the MTT interaction. The model given to the participants consisted of 30 objects connected with each other using 28 links.


Experiment in process on the MTT
Experiment in process on the MTT


Once the participants finish both tests, they will end the experiment by filling up a two-part survey. The first part of the survey was composed by the NASA Task Load Index [1], which aims to assess subjective workload of the participants for both PC and MTT. The second part of the survey is focused on subjective evaluation of the communication of the group for both PC and MTT.

Results

Analysis of our proposed metrics

The following are the results for the proposed metrics in the five groups tested.

caption
caption

On these two graphs we can observe that the models got produced faster and with lower variance on the PC than on the MTT. This could be due to previous experience with the PC interface, and also due to some unpolished features that the MTT software was running. According to feedback from the participants, the main setback on the MTT interaction was the failure of registering touches when moving gestures were being performed.

caption
caption

With respect to communication between group members, we can see that on average, the MTT participants talked more between each other while performing the task than on the PC. We observed that communication during both experiments was mainly focused on division on tasks and expression of participants’ feelings towards the difficulty of the task.

caption

caption

Regarding number of interactions per user with the experiments’ interfaces, we observe that although on average they were similar, some groups used many more interactions on the PC than on the MTT. Taking into account the data of average time to completion, we can infer that although both interfaces had a similar number of interactions, participants took more time per interaction on the MTT than on the PC.

Accuracy analysis

This table shows the number of mistakes found when comparing the resultant diagram of our participants to the given diagram to reproduce. An error is defined as a missing / wrong shape or a missing / extra link .

Errors Made
Group 1 Group 2 Group 3 Group 4 Group 5
PC 2 0 0 0 0
MTT 3 1 2 0 0

Survey analysis

For the NASA TLX survey done after the experiments, each question for the survey was asked for the task done on the Computer and on the Multi-touch Table. According to those results, participants seem to perceive the task of building the given model in the MTT slightly more mentally demanding than in the PC, but overall as an easy task. On the physical demand aspect, the MTT scores higher, this can be related to the fact that the task on it was done standing up, compared to sitting down on the PC.

Participants seemed to experience more hurry while performing the task on the PC than on the MTT. Also, subjective task achievement and amount of work evaluation in the PC are higher than in the MTT. This difference could be linked to the fact that all participants more familiarity with using the PC than using the MTT. Finally, participants also reported to feel more discouraged, insecure, irritated, stressed and annoyed in the MTT than in the PC.

As for the results of the second part of the survey, where the questions are focused on subjective evaluation of the communication of the group for both PC and MTT,we can see that the participants believe the task is slightly faster to be achieved using the PC even though they believe that collaboration is easier and more interesting when the MCC was used. More generally, users seemed to enjoy the task given for the MTT while some of them felt frustrated when using our prototype, mainly due to its limitation when managing the links. It was also reported that a bigger amount of time to learn our system could improve its usability.

Conclusion

In this project we present a comparison for the task of collaborative creation of graphical models in two settings: using multiple computers (PC) and using a multi-touch table (MTT). Model creation using the MTT led to more communication between the participants and a similar amount of interactions with the interface compared to the PC (touches and clicks, respectively).

The participants regarded the interaction in the MTT to be less intuitive in general mainly because of the lack of knowledge about this technology, but they also think that results can be improved in the future when learning the system and improving its functionalities.

Based on the previous results, we can say that is less time efficient to create graphical models in group by using a multi-touch table rather than in multiple stations and similarly accurate. The multi-touch table also encourages the participants to communicate more with each other compared to the multiple stations.

Survey results

caption
caption
caption
caption
caption
caption
caption
caption
caption
caption
caption
caption
caption
caption
caption