Tangible Interaction for animation of deformable characters
- 1 Students
- 2 Project
- 3 Experiments
- 4 Evaluation
- 5 Conclusion
- 6 References
Most popular 3D animation techniques are not intuitive. They involve complex manipulations and require a steep learning curve on behalf of the user, even for a simple animation. We turn to tangibles since they could be ideal to beginners in animation due to ease of manipulation. The use of tangibles to drive an animation is not new. However, most existing systems target rigid transformations through the deformation of rigid body joints. It is therefore interesting to consider the manipulation of soft bodies to deform characters in animation.
The main objective of this project is to investigate which level of prototype thickness best suits a type of animation. In particular, we consider two types of animation: Squash and Stretch. These principles of animation are key to making animations more lively and flexible. The absence of Squash and Stretch makes motion rigid or stiff. The transition between a proper Squash pose to a Stretch pose, or the other way around, breaks the perfect solidity that Computer Graphics animation in particular inherently gives to everything.
The goal is to animate 3D digital characters in real time when users manipulate soft body prototypes of different thickness by stretching or squashing them. The characters should stretch or squash by the corresponding amounts with varying degree of ease.
State Of The Art
I. Rig Animation Using Tangible & Modular Input Device 
- Advantage: With this system, only a few joints are required to setup a physical rig for animation.
- Limitation: It has limited degrees of freedom & it does not allow animation of deformation
II. Local Physical Models for Interactive Character Animation 
- Advantage: User manipulates real-time input devices to interactively drive the motion of a computer graphics character, and is provided with immediate feedback displaying the animation as it is being created.
- Limitation: In their kinematic ground contact model,since exactly one point is always fixed to the ground, double support is difficult to control accurately, while zero-support is not currently possible. This limits jumping and running.
III. DefSense: Computational Design of Customized Deformable Input Devices 
- Advantage: In this implementation, user-specified deformations can be achieved using 3D printed devices with embedded sensors.
- Limitation: The system is not capable of reconstructing deformations not specified by the user.
Our hypothesis was that thinner prototypes would be better for the stretch animation as they are more flexible while thicker prototypes would be better for squash as they offer more resistance due to rigidity.
We decided to create the soft prototypes out of silicone. We started by experimenting with different materials to vary the flexibility of the typical silicone sealant available in the store. Materials like corn flour and mineral spirits (Essence de térébenthine) were used to soften the silicone.
The prototypes were still not soft enough for stretching and bending, so we decided to use a commercial silicone (Ecoflex 00-50). We decided to go with arduino and stretch sensors as an initial implementation. The initial setup can be seen below.
We then switched to Optitrack because the sensor values were fluctuating too much and the variations in subsequent sensor values made it difficult to get a smooth animation. We used Unity for realizing the animations.
Below is the final prototype with different levels of thickness in decreasing order of of thickness (1.5cm, 1 cm, 5 mm, 3 mm). Note the 6 reflective markers placed on the sides of the wood. The wood was initially added as a hand support for stretching and squashing so that people with both large ans small-sized hands can manipulate it without occlusion.It was eventually only used for squashing, since the wood was too fragile for stretching. Stretching a prototype is done by pulling on the ends of the silicone while squashing is done by bringing the pieces of wood towards each other by causing the silicone in between to bend gradually forming an arc.
The Unity platform was used for creating the animations. 4 IR cameras were used during experiments. The users were asked to face the main camera while looking at the animation on a screen close to the main camera.
Targeted users were beginner animators, all students on campus who did not have a significant experience in animation. We targeted beginners so that we could have unbiased responses for this new type of interaction.
The participants were asked to complete two tasks: a Stretch animation and a Squash animation using the prototypes.
The prototypes were given to them in decreasing order of thickness.
1. Participants were first told to stretch the prototypes in order to animate a character. They were asked to first train for a 2-3 minutes with each prototype to get used to the prototype in question.
They were then timed for 1 minute for each prototype while they attempted to achieve the pace of a reference animation playing automatically, next to theirs. They had to pull the prototype by both ends to stretch the animated object while facing the camera and release to make it go back.
2. Participants were asked to fill a questionnaire regarding the stretch tasks completed and rank the prototypes according to 4 criteria.
3. Participants were told to squash prototypes to animate a character. They were asked to first train for a 2-3 minutes with each prototype to get used to the prototype in question.
They were then timed for 1 minute for each prototype while they attempted to achieve the pace of a reference animation playing automatically, next to theirs. They had to bend/curve the silicone prototype by both ends to squash the animated object while facing the camera and release it to make it go back to its original shape.
4. Participants were asked to fill the questionnaire regarding the squash tasks completed and rank the prototypes according to 4 criteria. They also filled out some questions regarding the experiment in addition to the NASA TLX questionnaire with tells us about the overall task load for animation using tangibles made of silicone.
The thickness is varied here to understand the effects on Stretch and Squash animations.
We use the Raw NASA TLX questionnaires in order to try to get global metrics for our interaction (combined values for both squash and stretch animations). We also ask the users to rank each prototype according to effort, ease of keeping up with pace of animation, frustration and performance for both squash and stretch and hence try to conclude which thicknesses are more suited for squash and stretch respectively. We also measure errors by comparing the difference in scale along y-axis in the reference and participant's animation in order to find the precision.
The total workload is divided into six subjective subscales that are represented on a single page, serving as one part of the questionnaire. They are rated for each task within a 100-points range with 5-point steps.
The subscales can be described as follows:
- Mental Demand: How much mental and perceptual activity was required? Was the task easy or demanding, simple or complex?
- Physical Demand: How much physical activity was required? Was the task easy or demanding, slack or strenuous?
- Temporal Demand: How much time pressure did you feel due to the pace at which the tasks or task elements occurred? Was the pace slow or rapid?
- Overall Performance: How successful were you in performing the task? How satisfied were you with your performance?
- Effort: How hard did you have to work (mentally and physically) to accomplish your level of performance?
- Frustration Level: How irritated, stressed, and annoyed versus content, relaxed, and complacent did you feel during the task?
We wrote an XML script to dump the scale values of the reference animation and the controlled animation at each frame. The sum of errors for each participant is computed by summing the distance measure between the y-scale for the reference animation and the controlled animation at each frame. The results per participant, for each prototype are then averaged, yielding an average sum of errors per prototype which can then be compared.
Here, by plotting stacked column charts for the 4 prototypes namely P1, P2, P3, P4 of thicknesses 1.5 cm, 1cm, 0.5cm and 3cm respectively, we can visualize the global ranking of each prototype in terms of effort, ease of keeping up with the reference animation, frustration and perceived performance for the 2 animation tasks: (i) squash and (ii) stretch.
In the following stacked columns:-
- ME: Most Effort
- LE: Least Effort
- MF: Most Frustrating
- LF: Least Frustrating
Here, we can easily see that prototype P1 requires the most effort followed by P2 and P3. P4 being the thinnest requires the least amount of effort while stretching. As thickness increases, so does the effort. They are thus directly proportional.2.
P3 is ranked as both the easiest and 2nd easiest for keeping up with the pace of the reference animation followed by P2. By using P1, the participants had the most difficulty in keeping up with the pace of animation. We can see that the thinner the prototype, easier it is to keep up with the pace of animation with the exception of P4 as most of the users reported that sine it is too thin, it stretches too easily so much so that it is hard to control the pace of the animation and have reported shooting beyond the targeted sphere. This is also confirmed by the values recorded in the XML file. Some people however reported that they found it the easiest to control P1 as they did not mind applying extra force.
P1 being too thick is the most frustrating prototype since it requires too much force to achieve the desired result. P4 is too thin and stretches too easily so, it is hard to control. It is less frustrating than P1. P3 is ranked as either the 3rd most frustrating for the least frustrating prototype to work with. The participants said that it is since it is of medium thickness and allows them to have the right balance in order to have optimal control over the stretch animation.4.
For most participants, P3 has the highest and second highest performance since it is easier to control for them. Some others reported that the 1cm protype gave the highest performance. A minority did report the highest performance with the thinnest prototype as it required the least effort to stretch and they did not find it too thin or hard to control. Even though most people find P4 very difficult to control while stretching, the effort used to stretch the P1 prototype is too high. Thus, P4 gives the 3rd highest performance and P1 gives the lowest.
Effort required to squash P1>P2>P3>P4. The thinner the prototype, lesser the required effort
Clearly for ease of keeping up the pace of animation, P3>P4>P1. P3 is considered to be the easiest since it is the right balance of resistance and flexibility. P4 being really thin offers lots of flexibility. However, P1, P2 and P4 are reported by the same number of participants as the most difficult prototypes for keeping up with the pace of the animation.
P1 and P4 are equally ranked as the most frustrating and also the 2nd most frustrating. P2 is clearly the 3rd most frustrating and P3 having the right balance of resistance and flexibility is the least frustrating to work with.
P3 has the best performance followed by P2 and P1. P4 and P1 are reported to have lowest performance reasons for that being that P1 is too rigid and requires lots of effort to control especially for the participants with larger hands. P4 is to thin and doesn’t offer enough resistance to control the movement of the prototype for squash.
Raw NASA TLX Evaluation
We chose to eliminate the pairwise comparisons made by users in order to weight the subscales to make the process less time-consuming for the users.
- As we can see from the bar graph, on an average, the physical demand and effort are above 50 % and higher than all other evaluation metrics. The rest of the metrics are below 50%
- Among the 10 volunteers there is a large discrepancy between the values for mental demand ranging from 0 to 75%.
- For most of the participants, temporal demand was not too high.
- Some users complained about having to apply a lot of force of the 1.5 cm prototype and they found the experiment quite frustrating. Some of the others like the same prototype and said they did not mind applying more force since this prototype gives them better controlled.
- The large variation in values of the evaluation metrics can also be explained by the fact that people with larger hands had difficulty in holding the prototype in order to stretch it without occlusion. On the other hand, participants with smaller hands report extremely low levels of frustration.
In the following graphs, we plot the average of the sum of errors per prototype for a total of 10 participants. The errors are computed by using the distance formula to calculate the difference between a reference animation's y-scale transform and the actual y-scale transform of the controlled animation. We use the y-scale only even though the space is 3D because users directly manipulate this scale when stretching and squashing. In addition, the other 2 scales do not change the effective distance measure. The sum of errors for each participant are normalized by subtracting the minimum value from each sum of errors and dividing by the difference between maximum and minimum values for the errors. Then an average of these values is computed to get the average sum for each prototype.
To remove outliers occurring as a result of occlusion, we compute the 1st and 3rd quartiles of the distance measure data, evaluate the interquartile range and use the lower and upper bounds to identify the outliers. The number of outliers varied per user but on average, around 200 data points were removed for squash and around 100 for stretch.
The data is plotted over 200 frames for stretch animation and 1000 frames for squash.From the first graph, we can see that for the stretch animation, the average number of errors is highest with the 1st thickest prototype(1.5cm) and smallest with the 3rd thickest (5mm). The 1.5cm , 1cm and 5mm prototypes follow a trend showing that decreasing thickness is desirable in stretch but to a certain limit since the 3mm prototype gives the next highest amount of errors.
From the second graph, we can see that for the squash animation, the average number of errors is highest with the 3rd prototype(5mm) and smallest with the 1st (1.5cm). In the same way as for stretch, the 1.5cm prototype, 1cm and 5mm show a trend; that decreasing the thickness is undesirable for squash, although with the 3mm prototype, performance was slightly better than 1cm one.
- According to user rankings, optimal thickness level for stretch is 0.5 cm in terms of performance as shown by the 5mm prototype.
- The 5mm prototype gives the least number of errors
- Thus our qualitative and quantitative findings agree.
- The 1.5cm prototype was deemed the worst for performance by users which agrees with our findings for quantitative analysis.
- According to user rankings, optimal thickness level for squash is with the 0.5 cm prototype in terms of performance.
- Equal number of users ranked the 1.5 cm prototype and the 3 mm prototype as the worst performing protoptypes.
- According to quantitative analysis the 5 mm prototype gives the highest number of errors while the 1.5 cm prototype gives the least.
- This is contradictory but the users' perception can be explained by the fact that they found the thinner prototypes more responsive (smaller motions produce larger displacements on screen), in particular the 5 mm prototype (the 3 mm prototype was much too loose for handling).
Overall, users reported a poor performance with the tangibles. We could not find a direct correlation between performance and mental, physical or temporal demand but occlusion was a significant factor to consider since it occurred rather frequently in the experiments and was difficult to avoid in spite of warning participants. In addition to causing performance issues, occlusion was also a major source of frustration.
To conclude, we found that thin prototypes work best for stretch but up to a certain limit (5mm is ideal). Too thin prototypes (<=3mm) give a fragile aspect and leads to lower precision, possibly because they provide less control.
Thick prototypes work best for squash but once more, to a certain limit. A thickness of 1.5cm gave least errors but was generally not recognised as giving good performance by users. Some users who preferred the thickest prototypes for squash suggested that using an intermediate level of thickness between P1 and P2 could benefit the squash animation. This means that rigidity was desirable to a certain amount for squash but that a slightly thinner (~1.3cm) one could be better.
These findings also suggest the benefit of switching between soft and more rigid tangibles for an animation with both stretch and squash.
 Igl.ethz.ch. (2020). igl | Interactive Geometry Lab | ETH Zurich | Rig Animation with a Tangible and Modular Input Device. [online] Available at: https://igl.ethz.ch/projects/rig-animation-input-device/ [Accessed 13 Jan. 2020].
 Cs.toronto.edu. (2020). [online] Available at: https://www.cs.toronto.edu/~hinton/absps/oore-terzopoulos-hinton-2002b.pdf [Accessed 13 Jan. 2020].
 Hilliges, O. (2020). Advanced Interactive Technologies, ETH Zürich. [online] Ait.ethz.ch. Available at: https://ait.ethz.ch/projects/2016/deformables/ [Accessed 13 Jan. 2020].