PrediBoard: a smarter keyboard

De Ensiwiki
Aller à : navigation, rechercher
Project schedule.png
Titre du projet PrediBoard: a smarter keyboard
Cadre Projets de spécialité
Page principale Projets de spécialité image

Encadrants Laurence Nigay, Professor of Computer Science at Université Grenoble Alpes.

Students

Concept

Introduction

The PrediBoard is a physical keyboard with the added functionality of word prediction. The keyboard will allow the users to select one of three word choices while typing. The three possible choices will be shown in the bottom of the screen and the user will be able to select a suggestion by interacting with the machine. For this project we will test and analyze the performance of different interactions for word selection, compare the users' performance against typing on a plain keyboard without predictions and observe the usability of word predictions in a real scenario.

Objective

Our main objective is to study whether a prediction enabled keyboard can help users type faster and which gestures are more suitable for accomplishing this task. For this project we focus on comparing two different ways of interaction: Button pressing and Hand Gestures as means to select predicted words. We then analyze the interactions with respect to the Keystroke Level Model (KLM) [1]. Since we make use of an existing platform (Swiftkey), we will not focus on the implementation of the prediction mechanism.

In this project, we will be analyzing the following Metrics:

  • Real world Usage: The amount of times that the tester would use the keyboard predictions in a real world scenario.
  • Comparison between keyboards: Comparison of time between typing with a prediction enabled keyboard, and a plain keyboard with no predictions.
  • Homing time: Time needed to move between two interfaces. In our case the Total Homing Time (THT) for completing the interaction task and positioning the hands back to the keyboard can be expressed by the following model:


  \textbf{THT = H + IT (Interaction Time) + H} 

Where H represents the Homing time.

Addressed problem

Even though nowadays predictions are commonplace on our mobile devices, there are no existing physical keyboards using this feature to our knowledge. Our inspiration to address this comes from the previously mentioned on-screen (virtual) keyboards found on iOS and Android devices, and personal experiences such as an increased typing speed while using these features on a smartphone. Other situations such as typing in a foreign language have led us to prefer typing on a smartphone rather than on a keyboard are also a source of inspiration.

The keyboards offered by default on these operating systems, as well as some other popular 3rd party keyboard apps such as Swiftkey can increase the performance of the user [7]. While typing, these keyboards show the three most probable words that the user might be typing above the virtual keyboard. This allows the user to select one of those words instead of finish typing the whole word. These keyboards also offer the capability of trying to "guess" the next word the user might want to type. We can see this in Figure 1.


Figure 1: SwiftKey app predictions

The main task of our project consists on suggesting new ways of interacting with the computer, in order to select one word out of the three available. Since the predictions will be shown on the screen, the most intuitive interaction would probably be to interact with a touch screen. This interaction however, might be problematic when using a desktop computer. Thus we decided to test the smart keyboard with the following interactions:

  • A gesture based interaction. (Figure 2, 3, 4)
  • An interaction using physical buttons placed beneath the space bar. (Figure 5, 6, 7 and 9)
Figure 2: Interaction using gestures
Figure 3: Interaction using gestures
Figure 4: Interaction using gestures
Figure 5: Interaction using buttons
Figure 6: Interaction using buttons
Figure 7: Interaction using buttons

Target Audience

The intended target of this project is for the following groups of people, and with their corresponding objective:

  • Slow Typists: Users with an average typing speed of less than 40 words/min trying to improve their typing speed and accuracy.
  • Language Learners: Users trying to learn a new language, to improve their spelling in the foreign language.
  • Fast typists/Programmers: Users that use programming tools constantly, to simplify the selection of suggested completion words, and diminish their homing time.

Approach & Implementation

In order to test our idea, we decided to split the task into two different and smaller tasks. The first task consisted on comparing the different methods of interaction with the keyboard, and the selection mechanism. In this first part, we compare gestural and button interactions in order to find out which one is better aimed at selection. To decide this, we take into account the efficiency of use, more specifically, the total time taken by the user to finish the task at hand. We also take into account the user experience, evaluated by interviewing the user after testing both interactions.

After evaluating the first task and selecting the preferred mode of interaction, we proceeded to perform the second phase of our testing. During the second task, we aim to discover the real world potential that our proposal has. In order to evaluate this, we set up a task that would allow us to see whether or not the user is willing to adapt and use this new interactions system. We evaluate the success of the system by the percentage of use during testing.


Comparison of Interactions

Figure 8: Testing platform - Home screen
Figure 9: Buttons Setup

During the first phase we compare both of the proposed interactions (gestures & buttons) to find out which one of the two allows the user a faster input speed. How comfortable a user is with the proposed interaction is also important, since users must be comfortable with it in order to use it in the future. During testing, we will consider the prediction mechanism and the mental activity as being the same for both interactions, since this will be further tested at a later time along with the positioning of the predictions. Instead we will only focus on evaluating the efficiency and the ease of use.


We will consider the previously mentioned Total homing time as the metric for efficiency on both interactions. In order to efficiently and accurately measure this timing, we developed a game - like software using YOYOGAMES GameMaker Studio [2] which allowed us to perform the testing for both interactions using the same set of simple English words. This program collects data concerning the Total Homing time (THT) and the total time to complete the task. With this program, we force the user to complete an interaction in between sets of words. This allows us to accurately measure the THT of the gestures. Since our implementation is not a complete working prototype, the interactions were performed by users swiping their hands above the keyboard for the gestures, and by pressing on non-functional buttons under the keyboard. This is a slightly different approach to the Wizard of Oz technique where the wizard, without the users awareness, simulates the system by taking care of the input. In our case, the system is not fully functional but the wizards function in the experiment is limited to the supervision of the testing, to ensure that the user does not commit any errors during the test. In the event that the user commits any errors, the test is to be restarted. We only take into account tests done without any errors in the interactions.

In our testing platform the user is asked to fill in the words that appear on the screen. After every three completed words an indicator pops up in the bottom of the screen (left, right or center) asking the user to perform a gesture or a button press after completing the word. The time it takes for the user to perform the interaction is recorded by measuring the time between finishing a word, performing the interaction, and continuing with the following word (Figure 10 - 13). After having filled all the words the execution stops and the data is recorded in a CSV file. Finally, we collected qualitative data by asking the users about their preference concerning the interactions and by taking into consideration factors such as fatigue and confusion.

Things to evaluate:

  • Total Time
  • Total Homing Time
  • Preference
Figure 10: Indicator not visible - Interaction indicator appears when word is close to the end
Figure 11: Indicator visible & Homing time counter - Homing time starts counting when last letter of the word is typed
Figure 12: Indicator visible & Homing time - Homing time stops being counted when first letter of next word is typed
Figure 13: Indicator not visible - Homing time has been calculated, indicator has disappeared
Figure 14: Testing platform - Center Indicator
Figure 15: Testing platform - Right Indicator

Prediboard in a real setting

During the second phase we will compare the PrediBoard using the "best" interaction compared to the plain keyboard. Here we will evaluate the completion time of a the same task for both keyboards and discover the real world usage of prediction selection. For this phase since we did not have a fully working prototype we conducted the tests using the Wizard of Oz testing technique. We used the SwiftKey Keyboard app version 6.4.8.57 [3] and BlueStacks android emulator version 2.5.83.6332 [4] on a Windows 10 computer. SwiftKey is an keyboard app that offers word predictions, contains learning mechanisms and can provide us with useful statistics about the usage such as keystrokes saved using the predictions, words completed etc. Our set-up consisted of: a laptop with a touch screen where we installed the previously mentioned software, an external QWERTY keyboard and a monitor. The user used the monitor and the external keyboard for typing the given text and the Wizard was responsible for selecting the correct word after the user's selection (interaction) by pressing on the laptop's touch screen.

Things to evaluate:

  • Performance, evaluated as the Words per minute
  • Total Usage of System
    • Complete prediction of the next word
    • Prediction of currently typed word
  • Keystrokes saved compared to the total amount of keystrokes possible

Testing

Since we would be measuring very small periods of time during the first testing phase, we needed to have a way of accurately measuring them. For this reason, we decided to develop a program that would allow us to accurately obtain these timings. For the second test, we decided to use an emulated version of the Android operating system in order to use the SwiftKey keyboard application to provide the predictions for the users.


Testing two different interactions

As an initial step of this first test, we asked all users to perform a one minute typing test [8], this test allows us to categorize our testers as above average typers, or below average typers. We define an average typer as someone who is able to type at speed between 38 and 40 wpm [9]. The obtained results can be seen in the following table:

User Words per Minute
1 34
2 74
3 61
4 61
5 24

In order for the learning curve of the new interactions to have a minimum effect on the final result, we first trained the users on the interaction use until they felt confident for the proper test. To train them, we first showed them the interaction idea, and ran a few dummy tests, in order for them to understand the way the final test was to be done. During testing, we forced users to perform the interactions by restarting the test every time an interaction wasn't performed. This approach allowed us to get optimal results in every test. The training process and testing took in average around 6 minutes per user, with the exception of one user who took 15 minutes to finish, since a lot of mistakes were done during the final testing.

It is worth mentioning that training was mostly only done for the first interaction performed (gestures) since once the concept of the interaction was grasped, the second interaction (buttons) was easy for them to do. It is also worth mentioning that during testing, users often skipped interactions (at least once per user), which forced us to restart the testing.

Once tests were finished, we asked users to give their opinion on both interactions and to choose their preferred one.

For the text used during the tests we used a random sequence of 30 "easy" words (same for all users) as found on Ogdens's Basic English word list [5],[6]. This word list contains the most frequently used English words and is being used worldwide as the beginner's vocabulary for English language learners. The list of words can be found in the appendix.

Testing the PrediBoard against a plain keyboard

Similar to the first testing phase, we first gave the users 5 minutes to get reacquainted with the keyboard predictions, the new emulated environment and the buttons interactions, which was the preferred mode of interaction from the first phase of testing. For this second phase, we presented the users with six simple sentences shown at the top of their screen, and gave them the task of copying them below. We allowed them the free choice of using or not the buttons for word completion, this was done with the purpose of seeing if the buttons would be used in a "real" scenario. The text consisted of 43 words and 194 characters in total. The text used can be found on the appendix.

It is worth mentioning that during the practice sessions, some users refused to use the interaction, since it slowed their average typing speed. This was mostly seen for users 2 and 3, who had a typing speed of 74 and 61 WPM respectively.

Figure 16: PrediBoard - Percentage of Keystrokes Saved out of the total

Results

All the data gathered during our testing can be found on the appendix.

Comparing the two ways of interaction

For the first phase of testing, the total time, and the graphs showing the distribution our collected values showed that both interactions performed similarly. From the box plot we observe that the Gestures tend to produce more consistent results with the exception of Interaction 3.

Figure 18 compares the speeds of interactions per user. We found it interesting to see that those users that preferred using gestures to buttons (User 1 and 4) were slightly slower when typing using the button interactions. Although user 5 had the same result, she still preferred buttons due to it having a more "natural feeling". We assume this is due to her typing style of using one or maximum two fingers per hand to click on the keyboard.

Another interesting pair of results were those of users 3 and 4. Although both users had the same typing speed, user 3 preferred buttons, while user 4 preferred gestures. We assume this is due to their respective backgrounds, where user 3 is a software engineer and is used to fast typing with hands on the keyboard at all times. User 4 on the other hand is a designer and is used to moving her hands around the keyboard.

Figure 17: Gestures vs Buttons
Figure 18: Gestures vs Buttons - Total Time per User
Figure 19: Gestures vs Buttons - Distribution



Figure 20: Gestures vs Buttons - Total time per Method


Based on the total time and the user opinions, where 60% of the users believe that the Buttons are easier to use, we decided that the winner is the Button interaction.

Comparing the PrediBoard against a plain keyboard

During our tests, we found that 60% of our users were distracted by the word suggestions. We consider that this result is due to the fact that these users were also proficient with typing on the keyboard, and are not used to looking away from what they are typing. These results are consistent with those of [7], during their tests, they found that "One reason given for the failure of word prediction to accelerate typing is that the user must look away from any source document to scan the prediction list during typing. Looking away from the source document may slow the typist more than any acceleration offered by word prediction."


Performance

When looking at Figure 24 we observe that the system helped only the slow typists perform slightly better against the plain keyboard. Slow typists increased their WPM using the PrediBoard, while fast typists' performance decreased using our system. Fast typists rarely used the predictions (Figure 23) and from the qualitative results collected, we can see that they became confused and lost their pace while trying to use predictions instead of typing. They mentioned positioning of predictions to be awkward, so we believe this can be improved in future implementations of the system. These observations support our initial intuition that the system is better aimed at slow typists. Fast users are usually concentrated on the task at hand and are not benefited in time by saving a few keystrokes by using predictions.

Usage

A useful remark can be seen on Figure 23, where we can see that all testers used the predictions to introduce a new complete word and not to auto-complete the currently typed word. This gives us reason to believe that while typing, users are mostly concentrated on typing, and are not looking at the shown predictions. Predictions are mostly noted when a word is not being typed. Another reason for this could be that when typing a short word, it is faster to type all the word rather than completing it using the prediction.


Keystrokes

Regarding the keystrokes saved, as seen on Figure 21, once again the slow typists saved 20-25 % of the total keystrokes needed while the fast ones only 5-10 %.

Figure 21: PrediBoard - Percentage of Keystrokes Saved out of the total
Figure 22: PrediBoard - Total times predictions were used
Figure 23: PrediBoard - Words Predicted vs Words Completed



Figure 24: Keyboard vs PrediBoard - WPM


Given these results, we consider that a different test might have to be done in order to properly see the usefulness of word prediction. More specifically, the positioning of predicted words while typing.

Conclusion

During our testing we found out that homing time for both types of interactions are very similar, but, contrary to our hypothesis, button interactions were slightly favored both in time and preference. The typing style of the user, as well as their background might give us a clue as to what type of interaction the user prefers. We also analyzed the possible real world usage for the chosen interaction. Our results lead us to conclude that while some users are slightly benefited by the use of interactions when not typing, other users have a hard time committing to change their focus while typing in order to find the currently typed word and select it via interactions.

Given that our tests were performed in a short amount of time, we believe that a longer lasting test, where the user has the chance of really getting used to the interactions will provide different and more accurate results. Results show that slow typists were slightly benefited, which agrees with our initial intuition that this group of people were better suited for predictions. Nevertheless, we believe that a larger pool of testers will provide different results. Future work might include complicated words and foreign language typing. Based on observed traits during the tests, we believe that users might benefit from predictions when they stop typing to think about the word they want to type. We also include positioning of predictions as future work, since based on the results obtained, we believe the location of the predictions is a key part in the selection and usage process, predicted words should be shown closer to the area of focus of the user.

Appendix

Word list used for the first phase of testing Fichier:WordList.pdf

Text used for the second phase of testing Fichier:Text.pdf

Results from the first phase of testing Fichier:Results.pdf

Qualitative Results from the first phase of testing Fichier:QResults.pdf

Results from the second phase of testing Fichier:Results2.pdf

Qualitative Results from the second phase of testing Fichier:QResults2.pdf

Références

[1] Stuart K. Card, Thomas P. Moran, Allen Newell [1] The keystroke-level model for user performance time with interactive systems, Communications of the ACM 23(7):396-410 · July 1980

[2] YOYOGames - GameMaker Studio[2]

[3] SwiftKey Keyboard [3]

[4] Bluestacks Android Emulator [4] [5] Charles. K. Ogden [5], Basic English: A General Introduction with Rules and Grammar · 1932

[6] Ogden's Basic English Word list [6]

[7] Anson, D. and Moist, P. and Przywara, M. and Wells, H. and Saylor, H. and Maxime, H. [7], The Effects of Word Completion and Word Prediction on Typing Rates Using On-Screen Keyboards, 2010

[8] Typing test by 10 fast fingers [8]

[9] Teresia R. Ostrach [9], Typing Speed: How Fast is Average: 4,000 typing scores statistically analyzed and interpreted