analyse image for human pose from ()

Script

The idea is simple, we’ll add one image of  each class in the “costume” column by making one new sprite which will we display on the stage according to input from user. we’ll also change name of the image according to pose.

1. Add testing images to the backdrop and delete the default backdrop.
   
2. Now, come back to the coding tab and select the Tobi sprite.
3. We’ll start by adding a when flag clicked block from the Events palette.
4. We made the new variable “count” by choosing the “Make a Variable” option from the Variables palette.
5. Add the “hide variable () block from the Variables palette. Select count.
6. Add the “turn () video on stage with () transparency” block from the Machine Learning palette. Select the off option at the first empty place, and for the second, write a 0 value.
7. Add an “ask () and wait” block from the Sensing palette. Write an appropriate statement in an empty place.
   
8. Add the “if () then” block from the control palette for checking the user’s input.
9. In the empty place of the “if () then” block, add a condition checking block from the operators palette block. At the first empty place, put the answer block from the sensing palette, and at the second place, write an appropriate statement.
10. Inside the “if () then” block, add a “broadcast ()” block from the Events palette block. Select the “New message” option and write an appropriate statement for broadcasting a message to another sprite.
11. Add the “turn () video on stage with () transparency” block from the Machine Learning palette. Select the option at the first empty place, and for the second, write a 0 value.
12. Add the “() key points” block from the Machine Learning palette. Select the show option.
    
13. Add the “Set () to ()” block from the Variables palette. Select the count option at the first empty place, and for the second, write a 30 value.
14. Add the Show variable () block from the Variables palette. Select count.
    
15. Add “forever” from the Control palette.
16. Inside the “forever” block, add an “analysis image from ()” block from the Machine Learning palette. Select the Web camera option.
    
17. Inside the “forever” block, add an “if () then” block from the Control palette.
18. In the empty place of the “if () then” block, add an “is identified class ()” block from the Machine Learning palette. Select the appropriate class from the options.
19. Inside the “if () then” block, add an “say ()” block from the Looks palette block. Write an appropriate statement in an empty place.
20. Add “change () by ()” from the Variables palette. Select the count option in the first empty place, and for the second, write a -1 value.
    
    
21. Add the “if () then” block from the control palette for checking the user’s input.
22. In the empty place of the “if () then” block, add a condition checking block from the operators palette block. In the first empty place, put the “count” block from the sensing palette, and in the second place, write 0.
23. Add the “Set () to ()” block from the Variables palette. Select the count option at the first empty place, and for the second, write a 30 value.
24. Add the “turn () video on stage with () transparency” block from the Machine Learning palette. Select the off option at the first empty place, and for the second, write a 0 value.
    
25. Inside the “if () then” block, add an “say ()” block from the Looks palette block. Write an appropriate statement in an empty place.
26. Add the “() key points” block from the Machine Learning palette. Select the hide option
27. Add the “stop ()” block to the control pallet. Select all options.
    
28. Repeat “if () then” block code for other classes, make appropriate changes in copying block code according to other classes, and add code just below it.
    
29. The final block code looks like
    
30. Now click on another sprite and write code.
31. We’ll start writing code for this sprite by adding a when flag is clicked block from the Events palette.
32. Add the “hide” block from the Looks pallet.
    
33. Write a new code in the same sprite according to class and add the “when I receive ()” block from the Events palette. Select the appropriate class from the options.
34. Add the “show” block from the Looks pallet.
35. Add the “switch costume to ()” block from the Looks palette. Select the appropriate class from the options.
    
36. Repeat the same code for other classes and make changes according to the class.
    

    Final Result

    

Introduction

In this example project, we are going to create a machine learning model that can count the number of jumping jack activities from the camera feed.

Pose Classifier in Machine Learning Environment

The pose Classifier is the extension of the ML Environment used for classifying different body poses into different classes.

The model works by analyzing your body position with the help of 17 data points.

Pose Classifier Workflow

1. Open PictoBlox and create a new file.
2. You can click on “Machine Learning Environment” to open it.
   
3. Click on “Create New Project“.
4. A window will open. Type in a project name of your choice and select the “Pose Classifier” extension. Click the “Create Project” button to open the Pose Classifier window.
   
5. You shall see the Pose Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.

Class in Pose Classifier

Class is the category in which the Machine Learning model classifies the poses. Similar posts are put in one class.

There are 2 things that you have to provide in a class:

1. Class Name: The name to which the class will be referred.
2. Pose Data: This data can be taken from the webcam or uploaded from local storage.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

1. Naming the Class: You can rename the class by clicking on the edit button.
2. Adding Data to the Class: You can add the data using the Webcam or by Uploading the files from the local folder.
   1. Webcam:

Training the Model

After data is added, it’s fit to be used in model training. To do this, we have to train the model. By training the model, we extract meaningful information from the hand pose, and that in turn updates the weights. Once these weights are saved, we can use our model to predict previously unseen data.

The accuracy of the model should increase over time. The x-axis of the graph shows the epochs, and the y-axis represents the accuracy at the corresponding epoch. Remember, the higher the reading in the accuracy graph, the better the model. The range of accuracy is 0 to 1.

Testing the Model

To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

The model will return the probability of the input belonging to the classes.

Export in Block Coding

Click on the “Export Model” button on the top right of the Testing box, and PictoBlox will load your model into the Block Coding Environment if you have opened the ML Environment in the Block Coding.

Script

The idea is simple, after running code we will do jumping jack activity in front of camera and tobi sprite will say counting of jumping jack.

1. Select the Tobi sprite.
2. We’ll start by adding a when flag clicked block from the Events palette.
3. We made the new variable “count” by choosing the “Make a Variable” option from the Variables palette.
4. Also we made the new variable “temp” by choosing the “Make a Variable” option from the Variables palette.
   
5. Add “forever” from the Control palette.
6. Inside the “forever” block, add an “analysis image from ()” block from the Machine Learning palette. Select the Web camera option.
   
7. Inside the “forever” block, add an “if () then” block from the Control palette.
8. In the empty place of the “if () then” block, add an “key () pressed?” block from the Sensing palette. Select the ‘q’ key from the options.
9. Inside the “if () then” block, add the “Set () to ()” block from the Variables palette. Select the count option at the first empty place, and for the second, write a 0 value.
10. Also add the “Set () to ()” block from the Variables palette. Select the temp option at the first empty place, and for the second, write a 0 value.
    
11. Inside the “forever” block, add an new “if () then” block from the Control palette.
12. In the empty place of the “if () then” block, add an “is identified class ()” block from the Machine Learning palette. Select the ‘Upper hand‘ option from the options.
13. Inside the “if () then” block, add the “Set () to ()” block from the Variables palette. Select the temp option at the first empty place, and for the second, write a 1 value.
    
14. Inside the “forever” block, add an new “if () then” block from the Control palette.
15. In the empty place of the “if () then” block, add an “is identified class ()” block from the Machine Learning palette. Select the ‘Down hand‘ option from the options.
16. Inside the “if () then” block, add the another “if () then” block from the Control palette.
17. In the empty place of the “if () then” block, add a condition checking block from the operators palette block. At the first empty place, put the temp variable from the variables palette, and at the second place, write a 1 value.
    
18. Inside the “if () then” block, add the “Set () to ()” block from the Variables palette. Select the count option at the first empty place, and for the second, write a 1 value.
19. Also add the “Set () to ()” block from the Variables palette. Select the temp option at the first empty place, and for the second, write a 0 value
20. Inside the “if () then” block, add an “say () for () seconds” block from the Looks palette block. At the first empty place, add the “join () ()” block from operator palette and at the second place, write a 2 value.
21. Inside “join () ()” block at the first empty place, write the appropriate statement and at the second place, add count variable from Variables palette.
    

    Final Output