Table of Contents
Recognition Cards
PictoBlox Block

analyse image from ()

Description

To execute object detection, we use analyse image from () blocks. You can input the image in the following ways:

  1. Camera feed
  2. Stage

This block analyses the image and saves the face information locally, which can be accessed using other blocks.

You have to run this block every time you want to analyze a new image from the camera or stage.

Example

AI Delivery Robot

The example demonstrates how to make a delivery robot that follows the line and stops when it reaches checkpoint 1.
Download File

Script

Alert: You need to calibrate the IR sensor to get the best line detection by the robot. Also, you need to calibrate the speeds to make the robot follow the line correctly.

Output

 

Read More

Sign Detection – Tobi

The example demonstrates how to implement sign detection in PictoBlox.
Download File

Script

Output

Read More

Face Tracking Controlled Quadruped

Learn about face-tracking, and how to code a face-tracking Quadruped robot using sensors and computer vision techniques.
Download File

Introduction

A face-tracking robot is a type of robot that uses sensors and algorithms to detect and track human faces in real-time. The robot’s sensors, such as cameras or infrared sensors, capture images or videos of the surrounding environment and use computer vision techniques to analyze the data and identify human faces.
Face-tracking robots have many potential applications, including in security systems, entertainment, and personal robotics. For example, a face-tracking robot could be used in a museum or amusement park to interact with visitors, or in a home as a companion robot that can recognize and follow the faces of family members.

One of the most fascinating activities is face tracking, in which the Quadruped can detect a face and move its head in the same direction as yours. How intriguing it sounds, so let’s get started with the coding for a face-tracking Quadruped robot.

Logic

  1. If the face is tracked at the center of the stage, the Quadruped should be straight.
  2. As the face moves to the left side, the Quadruped will also move to the left side.
  3. As the face moves to the right side, the Quadruped will also move to the right side.

Code Explain

  1. Drag and drop the when green flag clicked block from the Events palette.
  2. Then, add a turn () video on stage with () % transparency block from the Face Detection extension and select one from the drop-down. This will turn on the camera.
  3. Add the set pins FR Hip () FL Hip () FR Leg () FL Leg() BR Hip () BL Hip () BR Leg () BL Leg () block from the Humanoid extension.
  4. Click on the green flag and your camera should start. Make sure this part is working before moving further.
  5. Add the forever block below turn () video on stage with () % transparency from the Control palette.
  6. Inside the forever block, add an analyzed image from the () block. This block will analyze the face the camera detects. Select the camera from the dropdown.
  7. Create a variable called Angle that will track the angle of the face. Based on the angle, the robot will move to adjust its position.
  8. Here comes the logical part as in this, the position of the face on the stage matters a lot. Keeping that in mind, we will add the division () / () block from the Operator palette into the scripting area.
  9. Place get () of the face () at the first place of addition () + (), and 3 at the second place. From the dropdown select X position.
  10. If the angle value is greater than 90, the Humanoid will move left at a specific speed. If the angle is less than 90, the Humanoid will move right at a specific speed. If the angle is exactly 90, the Humanoid will return to its home position.
Block Explained

  1. Create a variable called Angle and assign it the value of the face’s position.
  2. At the center of the stage, we will get the X position value which is zero.
  3. As we move to the left side the X position value will give you the negative value and as we move to the right side the X position value will give you the positive value.
  4. The x position value is divided by 3 which gives precise positioning.
  5. To set the angle at 90 when the face is at the center of the stage we have added 90 to the X position value.
  6. As we move to the left side the angle value will get decreased as the X position value is going in negative.
  7. As we move to the right side the angle value will get increased as the X position value is going in positive.

Code

Output

Our next step is to check whether it is working right or not. Whenever your face will come in front of the camera, it should detect it and as you move to the right or left, the head of your  Quadruped robot should also move accordingly.

Read More

Gesture Controlled Gripper Mecanum

In this activity, learn how to create a new Machine Learning model that will be able to identify and detect different types of hand poses and that can help us to control the Mecanum Gripper Robot.
Download File

In this activity, we will try to create a new Machine Learning model that will be able to identify and detect different types of hand poses and that can help us to control the Mecanum Gripper Robot. This activity can be quite fun and by knowing the process, you can develop your own customized hand pose classifier model easily!

We will use the same model that we have created in the previous Hand Pose Controlled Mecanum model to avoid any misdirection and confusion.

Note: You can always create your own model and use it to perform any type of functions as per your choice. This example proves the same point and helps you understand well the concept of Machine Learning models and environment.

Hand Gesture Classifier Workflow

Follow the steps below:

  1. Open PictoBlox and create a new file.
  2. Select the Block coding environment as appropriate Coding Environment.
  3. Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
  4. Click on “Create New Project“.
  5. A window will open. Type in a project name of your choice and select the “Hand Gesture Classifier” extension. Click the “Create Project” button to open the Hand Pose Classifier window.
  6. You shall see the Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.

Class in Hand Gesture Classifier

There are 2 things that you have to provide in a class:

  1. Class Name: It’s the name to which the class will be referred as.
  2. Hand Pose Data: This data can either be taken from the webcam or by uploading from local storage.

Note: You can add more classes to the projects using the Add Class button.
Adding Data to Class

You can perform the following operations to manipulate the data into a class.

  1. Naming the Class: You can rename the class by clicking on the edit button.
  2. Adding Data to the Class: You can add the data using the Webcam or by Uploading the files from the local folder.
    1. Webcam:
Note: You must add at least 20 samples to each of your classes for your model to train. More samples will lead to better results.
Training the Model

After data is added, it’s fit to be used in model training. In order to do this, we have to train the model. By training the model, we extract meaningful information from the hand pose, and that in turn updates the weights. Once these weights are saved, we can use our model to make predictions on data previously unseen.

The accuracy of the model should increase over time. The x-axis of the graph shows the epochs, and the y-axis represents the accuracy at the corresponding epoch. Remember, the higher the reading in the accuracy graph, the better the model. The range of the accuracy is 0 to 1.

Testing the Model

To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

The model will return the probability of the input belonging to the classes.

Export in Block Coding

Click on the “Export Model” button on the top right of the Testing box, and PictoBlox will load your model into the Block Coding Environment if you have opened the ML Environment in the Block Coding.

Logic

The mecanum will move according to the following logic:

  1. If the detected class is “forward”, we will make the Mecanum move forward.
  2. When the backward gesture is detected – Mecanum will move backwards.
  3. When the Lateral Left gesture is detected – Mecanum will move towards the left direction laterally with the help of its omnidirectional wheels.
  4. When the Lateral Right gesture is detected – Mecanum will move towards the right direction laterally with the help of its omnidirectional wheels.
  5. When the Normal Right gesture is detected – Mecanum will rotate on a single point towards the right direction.
  6. When the Normal Left gesture is detected – Mecanum will rotate on a single point towards the left direction.
  7. When the “Stop” class gesture is detected, we will use the gripper functions of the Mecanum and Pick the object.
  8. When the “Circular Motion ” class is detected, we will use the gripper functions of the Mecanum and Drop the object by opening the arms of the gripper robot.

Code

Initialization

Main Code

 

Output

Forward-Backward Motion:

Circular Right-Left Motion:

Lateral Right-Left Motions:

Gripper Mechanism with Hand Gestures:

Read More
All articles loaded
No more articles to load
PictoBlox Extensions
  1. Scratch Based Default Extensions
    1. Motion
    2. Looks
    3. Sound
    4. Control
    5. Events
    6. Sensing
    7. Operators
    8. My Blocks
    9. Variables
  2. Artificial Intelligence
    1. Face Detection
    2. Object Detection
    3. Human Body Detection
    4. Computer Vision
    5. Text Recognition
    6. Speech Recognition
    7. ChatGPT
    8. Natural Language Processing
    9. Recognition Cards
    10. Text to Speech
    11. Translate
    12. Open CV
  3. Innovative Extensions
    1. QR Code Scanner
    2. Weather Data
    3. Physics Engine
    4. IFTTT Webhooks
    5. Internet of Things (IoT)
    6. Data Logger
    7. Music
    8. Video Sensing
    9. Pen
  4. Machine Learning Environment
    1. Image Classifier (ML)
    2. Object Detection (ML)
    3. Pose Classifier (ML)
    4. Hand Pose Classifier (ML)
    5. Audio Classifier (ML)
    6. Number Classifier and Regression (ML)
    7. Text Classifier (ML)
  5. Quarky
    1. Quarky (Main)
    2. Display (Quarky)
    3. Robot (Quarky)
    4. Sensors (Quarky)
    5. Speaker (Quarky)
    6. Quarky Ultimate Robots
    7. Quarky Expansion Board
    8. Mars Rover
    9. Humanoid (Quarky)
    10. Quadruped (Quarky)
    11. IoT House (Quarky)
    12. Quarky Mecanum
    13. Quarky Robotic Arm
    14. Dabble
    15. Quarky Advance Line Following

Company

Community

Products

School Programs

Impact Programs

Learning Resources

Product Documentation

Get in Touch

Follow Us

Facebook Twitter Youtube Instagram Linkedin

PictoBlox - Block & Python Coding for Kids

Dabble App - One App. Infinite Control.

Copyright 2024 – Agilo Research Pvt. Ltd. All rights reserved – Terms & Condition | Privacy Policy