Yoga Pose Estimation using POSENET vision model

Yoga must be one of the best choices for most of us when it comes to staying fit and healthy. But, the fact is that not all of us get good results and this might be because we are not following the postures in the right manner which is in turn because there is no proper guidance.

It is important to practice the right yoga poses when we don’t prefer an instructor to guide our postures. When at home, it is easier to watch youtube videos and practice yoga at home. But we cannot be sure if the poses that we practice is the correct one.

Pose estimation is one of the real-world problems, where developers try to develop some kind of machine instructors which can monitor the user’s postures and provides live results about the posture status if it is right or wrong.

An attempt to solve this problem has been made with the help of cutting edge technologies available. Most of the pose estimation techniques require users to have costly hardware and software components to run the program. In this article, I would be discussing how to achieve this goal without involving any heavy GPUs or other software/hardware components.

To achieve this, I tried to implement all the computations and ML model training on the web and not on the local machine. There is n number of Java script libraries that can make this happen. In this article, I will be discussing 2 key Java script libraries,

P5.js — A javascript library that makes java script code much easy by providing simpler functions as well as it integrates well with

Ml5.js — A java script library that makes Machine Learning on Web with so much ease.

PoseNet is a computer vision and Machine learning model which helps in tracking full-size human poses with 17 keypoints as the output.

Note: Keypoints are nothing but the basic human skeleton joints on the body which on connecting resembles the human skeleton structure.

The model was introduced by TensorFlow community and was trained using full-size human images(from eyes to ankles) and therefore the model works well when the user provides full-size human input.

The reference to how posenet works have been mentioned at the end of this article :)

Implementing transfer learning to PoseNet model helps in training a model for a particular pose. Transfer learning is a method by which a trained Machine Learning model is retrained again for specific conditions. let us say, there is a trained PoseNet model which on implementing provides skeletal outputs referencing the keypoints in it. Now, in transfer Learning, the output from the PoseNet model(i.e the keypoints) is introduced as the inputs to another model and this model is trained for required poses that the user wants to estimate. The features from the PoseNet model allows you to ‘retrain’ or ‘reuse’ the model for a new custom task.

With the help of Ml5.js, the following tasks are achieved

Data Collection- Where json files are created depending on how many types of data(pose) has to be classified.

Data collection

Training and classifying- Construct necessary neural network to train the data recorded in the json files.

Loading data to the neural network

Deploying the model- The neural network is trained and now, it receives input from the users and provides required output as per the data(pose) trained

Deploying the model loaded and trained

ANGLE CALCULATION FOR ACCURACY

For making the pose estimation little more accurate, for a particular pose, let us say, for a half squat particular angle measurements have to be taken care of (i.e. spinal cord being straight, angle between the hip, knee and ankle, the angle between the shoulder, elbow and wrist) and by formulating a method to calculate these angles when a pose is being identified (differs for each poses being trained), it becomes more efficient to estimate accurate poses.

While training the model during the transfer learning process, provide basic yoga poses are provided as inputs to achieve yoga pose estimation. Model files are generated while training the data and it is used for the deployment of the model. On identifying the correct pose, an audio alert is made to intimate the user that the desired pose has been achieved.

REFERENCES:

1. My Medium Article on PoseNet-https://medium.com/happiestneurons/extensive-thoughts-on-machine-learning-in-javascript-878802398459
2. My Github Link: https://github.com/Anjanapradeep/POSE_ESTIMATION-YOGA-POSES
3. p5.js(tutorial)-https://p5js.org/get-started/
4. Online P5.js editor(provides live results and easy to work with)-https://editor.p5js.org/
5. Ml5.js(tutorial)-https://learn.ml5js.org/#/
6. Ml5.js for PoseNet-https://ml5js.org/reference/api-PoseNet/
7. PoseNet Model working-https://blog.tensorflow.org/2018/05/real-time-human-pose-estimation-in.html