Tactile Tennis Coach
Students: Karan Doni, Neeraj Jhawar, Chris Jones, and Yida Zhang
Term: Fall 2010
The goal of the project was to create a tennis training module for beginners using a virtual environment and a haptic interface to minimize the cost and increase learning for the beginners.
Tennis is a complex game. It requires many things working at the same time, from correct initial body position and correct racquet position to correct and timely swinging. The key to a good tennis shot is a combination of all of these components working correctly. Given the nature of the game, students who want to learn tennis usually go to a tennis coach for training. While this is helpful, in the long run this turns out to be very costly.
In this project of a "Tennis Trainer" we make an attempt to make a computerized coach for someone who has just started learning tennis using a virtual environment and haptic devices. Also, since it is composed of so many components, we, given the time constraints, focused our effort on trying to make someone correct their initial position for hitting a shot. A haptic interface can be seen as a very important tool in achieving such a goal. Getting a correct feedback from the user as to where his body position is, mapping it in the virtual environment, and giving intuitive correction indicators are all important parts of this project.
Though a lot of game simulations with interactive haptic interfaces exist and some training methods for games exist, the training simulation game for tennis has not been looked into in literature extensively. Some papers about position correction exist, but it hasn't been applied to tennis as a training module. Some work in position correction from “Spatially distributed tactile feedback for kinesthetic motion guidance” by Pulkit Kapur and Katherine Kuchenbecker has been investigated.
OVERALL SYSTEM: The system consists of five parts: tracking, actuation, the virtual environment, the tennis dynamics, and the code module to correct and integrate all the other modules. All of them are explained below:
THE SENSORS - MAGNETIC TRACKING SYSTEM: The sensor used for tracking the arms is a magnetic tracking device by Ascension Technology Corporation. It has four sensors attached which can be tracked using a magnetic transmitter which gives the position, orientation, rotation matrix and quaternion values for each of the sensors with respect to the world frame. The sensor systems come with built-in functions which can be used with the program to easily integrate it into the overall project.
POSITION OF SENSORS: The first sensor is positioned to track the orientation of the torso. In the scope of the project, the roll angle of the torso is the most important degree of freedom. The second sensor is placed on the upper part of the right arm. The third sensor is placed on the right wrist. The fourth sensor is placed on the racquet itself. These rotations are then mapped into the virtual world of OpenGL using various transformations. The human body in the virtual world tracks the actual motion of the learner. Hence he can see how he is moving, along with his racquet.
THE VIRTUAL ENVIRONMENT: The Virtual Environment is based on OpenGL with the OpenGL Utility Toolkit(glut). The size of everything on the tennis court is scaled to real dimensions and generally drawn with GL_QUADS and GL_LINES. The simplified tennis racquet model contains three basic geometries: two cylinders and one box. In order to keep connection between the human body and the racquet, three geometries are layered in sequence from the human palm. The most complicated object in the virtual environment is the human body, which is drawn with cylinders based on the size and position computed from the magnet tracking system. The height of the body is measured from the sensor of the torso. The rest of the body lengths are based on the ratio between the body parts to the height of the body. Since only the right arm and torso are being tracked, the other body parts in the virtual environment, without violating any physical constraints, remain stationary.
OPEN DYNAMICS ENGINE: Tennis Dynamics in our software are realized with Open dynamics Engine(ODE). Generally the ODE is Open C++ library, which consists mostly of useful APIs for physical simulations. In dynamics systems, we set the tennis ball, racquet, ball shooter and net as rigid bodies, whose collisions can be detected and simulated with ODE. Based on the complex geometry of the tennis racquet, the basic idea of ball-hitting is to detect the distance between the ball and the racquet face every simulation step and change the ball velocity according to the perpendicular vector of racquet face.
ACTUATOR CONTROL: Actuator control is largely based on the API provided by National Instruments. From a coding perspective, our software gives six channels of digital output and one channel of analog output.
MECHANICAL SYSTEM: The goal of the mechanical part of the project was to effectively secure the sensors and actuators to the various areas of the user and tennis racquet, while allowing the user to have a full range of motion typical with a tennis swing. The waist, the upper and lower arm, and the racquet would all need to have a motion sensor and actuators. For motion tracking, at first wireless IMU's were thought to be the best choice, but then we decided upon using a magnetic tracking system. Although this system included wires, and magnetic interference from actuators, it made data acquisition much easier because it gave the position, orientation, and rotation matrix for each sensor.
ACTUATORS: For actuators, 14 different types were purchased, found, or fabricated, and then tested to find which worked best for which purpose. For the two waist and four upper arm actuators, Tactaid analog vibration motors were chosen because they would allow us to change variables of the actuation including duration, intensity, and frequency. We used a constant sinusoidal signal at 280Hz, which was found to be a more noticeable frequency with these motors. For the lower arm, the vibration motors were found to be difficult to distinguish between one another, so a more localized actuator was found in Ledex low profile solenoids. For the last degree of freedom, the wrist rotation, in order to give haptic feedback that would be somewhat intuitive, we designed our own actuators. An oval crank with a rubber outline was mounted on a motor such that when the motor placed adjacent to the skin rotates, it would repeatedly drag the long end of the oval a certain direction across the skin. It was found that when multiple actuators of this type were placed on the wrist, they could create the sensation of something pushing the user's wrist to rotate.
Tactaid vibration motor shown in its yellow mount
Ledex low profile solenoid
WEARABLE PARTS: The wearable waist part had to fit a wide range of users given the kind of application. It was hence made from stretchable fabric and Velcro that would strap around the user. The part for the waist consists of two vibration motors and one sensor. The upper arm has an added constraint that each actuator has to be evenly spaced around the arm. In order to avoid having to glide something all the way up the length of someone's arm, each of the four actuators are attached to a separate piece of Velcro so that the distance between each actuator can be re-adjusted. An attached Velcro loop contains the motion sensor. The lower arm section contains a motion sensor, four solenoids, and two of the actuators that were designed for wrist rotation. This part was made from a sleeve of fabric that could stretch to fit most arms. The sensor is attached to the top of the wrist near the hand, and because of the possible magnetic interference with the sensors, the actuators are kept a sufficient distance away. This means placing the four solenoids further towards the elbow, and placing both wrist rotation actuators on the other side of the wrist from the sensor.The final piece, the racquet, has one solenoid. It was found that an impact from this solenoid near the center of the racquet face could generate enough vibration through the racquet to give the user a recognizable sensation, along with graphics, that there was a successful contact with the ball.
A vibration motor (left) and a motion tracker (right) mounted onto the wearable upper arm part.
Bottom view of the wearable wrist part showing the wrist rotation actuators and their "window" (left) and two mounted solenoids (right).
IMPLEMENTATION: All of the sensors and actuators were attached by sewing the mounts onto the wearable parts or the racquet. The wires were threaded through holes in the wearable parts, with special care being taken to thread the sensor wires such that their weight would not move and affect the sensor position. The wires ran up the arm, then into and out of the waist through holes in the wearable parts. These were combined when possible using electrical tape to make the equipment looked neater. In order to connect to the electrical system, the wires were soldered to a set of plugs. To run the complete system, these plugs had to be plugged into the circuit boards, the plug near the user's wrist had to be plugged into the plug coming out of the bottom of the racquet, and the sensor wire from the racquet had to be connected to the sensor equipment. Once the user, the racquet and the electrical systems are each connected to each other, the user is ready to run the program.
Electrical Equipment Used:
- 30V DC Power Supply -Frequency Generator(Square Mode) -NI-DAQ Board
-Bipolar Junction Transistors (BJT's) TIP 31C
Two different circuits were developed: one for the six Vibration motors and a second for the five solenoid actuators. A 3:8 Demultiplexer was used so as to be able to have select lines to select any one of the actuators to be switched on. The input to these select lines is given from the digital output of the NI Daq board.
Using a frequency generator with the Tactaid vibration motors allowed us to easily change the intensity, frequency, and duty cycle. Since the bipolar junction transistors can be used as a switching device only for a positive voltage supply, we tweaked the output of a frequency generator to make it run as a square wave with the lower voltage very close to zero. For the solenoids, the input voltage was set at about 20V, which was found to give a reasonable output "Jolt".
TENNIS TRAINING SYSTEM
The torso correction: This forms the first part of the tennis training method. The torso has to be oriented at a certain angle w.r.t the base line of the tennis court. The first sensor finds exactly this and the system gives the output according to where the user has to move. There are two ways he corrects his torso position, either by moving clockwise or counter-clockwise. These directions are given by the actuators on the torso. In all the cases, the user is supposed to move away from the actuation. So if the vibration is felt on the right end of the torso, user moves away from it, rotating in counter-clockwise direction.
The upper arm correction: This forms the second part of the tennis training method. The upper arm has to have a certain position w.r.t. the torso. The two rotational matrices from the sensor 1 and sensor 2 and the position vectors are used for finding the correct current position and desired position. The upper arm has two degree of freedom w.r.t the torso and both these degree of freedom are constrained in this method. With each degree of freedom, two actuators in opposite directions are actuated depending upon the direction in which one has to move. The arm vector which directs from the shoulder joint to the elbow is distributed along three planes. The component of the upper arm vector on these three planes and the angles these components make with the shoulder direction dictates the accuracy of the position of the upper arm w.r.t. the torso. All the vector calculations have been done using OGRE_MATH toolbox.
The lower arm correction: This forms the third part of the tennis training method. The elbow can be modeled as a rotational joint with single degree of freedom. Hence there is only two directions the lower arm can move w.r.t. the upper arm. This is modeled using the cosine of two vectors; first vector being the upper arm vector and the second vector being the one denoting the lower arm.
The wrist correction: This forms the fifth part of the tennis training method. The wrist is rigidly connected to the tennis racquet. Hence correcting the wrist also corrects the racquet. The wrist is also modeled using the wrist vector and finding its angles about three planes. The three planes in this case however, are much tougher to find. The first plane has its normal along the lower arm vector. The next two vectors are perpendicular to this vector and the direction comes from the vector which is along the tennis racquet. These two vectors form two perpendicular planes. The angle of the wrist vector on these planes give the right direction in which to correction of position.
On December 10th, 2010, we took part in the Penn Haptics Open House. This was a learning experience for the visitors who were able to use our system, but also for our team. This was in effect an extensive user study. The only major problem we noted was that, since it was a cold day and many people were wearing sweaters and thick clothing, they had trouble feeling the vibration motors. The simplest fix for this would be to inform people that they should be wearing light clothing, such as one would be wearing while playing real tennis. Another small issue was the window that was cut in the wrist sleeve. Although intended as a way to let the user feel the wrist rotation feedback, it was often mistaken as a thumb hole. Overall though, people seemed to be impressed by the complexity and integration of all of the parts of the project.
Many people tested our demonstration at the Open House, and we learned a great deal from these interactions. The main results were :
1. The pose correction works well for the given angles, and the angles, once changed, can be adapted well by the software.
2. It takes some getting used to before the user can use it effectively. The first few trials are a little confusing but once learned it becomes easy to check if the pose is correct.
3. Not everyone was comfortable with the vibration motors. On the other hand, solenoid tapping seemed to work well without making anyone uncomfortable.
Attach:Tennis_Video.avi A video showing tennis training method
1. One of the degrees of freedom (the twist of the arm) was not completed. This must be included to fully define the initial position of the player.
2. This project deals with the position correction for initial pose. The correct motion of the racquet also needs to be modeled for a complete training module.
3. There can be a specialized correction module for different types of balls, ex. For forehand, backhand, volley etc.
4. The training module for tennis serve needs to be completed.
1. Lucio T. De Paolis, Marco pulimeno and Giovanni Aloisio, “An Interactive and Immersive 3D Game Simulation Provided with Force Feedback”, First International conference on Advances in Computer –Humar Interaction, IEEE 2008 0-7695-30876-9/08
2. Dan Morris, Neel Joshi, Kenneth Salisbury, “Haptic Battle Pong: High-Degree-of-Freedom Haptics in Multiplayer gaming environment”
3. Mareike jacobs, “Design and Recognition of Tactile Feedback Patterns for Snowboarding” as his Diploma Thesis at Media Computing group, RWTH Aachen University.
4. Arnold Baca, Philipp Kornfeind, “Rapid Feedback Systems for Elite Sports Training,” IEEE CS and IEEE ComSoc, 2006, 1536-1268/06
5. T. Wada, N. Yoshii, Y. Yamaji, S. Tanaka, K. Tsukamoto, “Analysis of Upper Limb Motions in Tennis Swings Toward Rehabilitation Training,”, 2002, International Conference on Intelligent Robots and Systems.