Table of Contents
Object Detection
PictoBlox Block

turn () video on stage with () % transparency

Description

turn () video on stage with () % transparency block controls the control the camera feed on the stage.

You can change the following parameters of the block:

  1. Camera state:
    1. OFF – The video feed on the stage will stop.
    2. ON – The video feed on the stage will start.
    3. ON flipped – The video feed on the stage will start with the video mirrored. So, your right hand will show left and vice versa.
  2. Transparency: This parameter makes the video translucent. O is defined as the camera feed shown on the stage, and 100 is the video that will fade out entirely from the stage.

Example

Biodegradable vs Non Biodegradable Waste

A waste management system that will differentiate the waste based on its type. If it detects biodegradable waste, the LEDs Quarky’s matrix will turn green. If it’s non-biodegradable waste, the LEDs will turn blue.
Download File

Script

Output

Read More

Face Tracking Controlled Quadruped

Learn about face-tracking, and how to code a face-tracking Quadruped robot using sensors and computer vision techniques.
Download File

Activity Description

In this activity, students will program Quarky to detect a face’s position using the camera and respond with movements. Based on which direction the face is (left, right, or center), Quarky will display a pattern and move accordingly. This teaches camera-based input, angle calculations, and conditional movements.

Let’s Learn

  1. Open the PictoBlox application from the Start Menu.
  2. Select the inviting realm of Blocks as your coding environment.
  3. Connect “Quarky” to your computer using a USB cable. Then, click the Board button in the toolbar and Select Board as Quarky.
  4. Next, select the appropriate Serial port if the Quarky is connected via USB or the Bluetooth Port if you want to connect Quarky via Bluetooth and press Connect.
  5. Click on the Add Extension button and add the Quarky Quadruped extension.
  6. Add when flag clicked block from the Event Palette. This block helps you to start the script.
  7. To set up the quadruped, you can drag and drop pins for each leg and hip into the initialisation block using set pins FR Hip () FL Hip () FR Leg () FL Leg() BR Hip () BL Hip () BR Leg () BL Leg () blocks. This block sets which pins on the Quarky controller board control each servo motor for the front right (FR), front left (FL), back right (BR), and back left (BL) hips and legs. Drag this block and set each PIN as shown.  FR Hip: 4, FL Hip: 1, FR Leg: 8, FL Leg: 5, BR Hip: 3, BL Hip: 2, BR Leg: 7, BL Leg: 6.
  8. Turn on the camera video on the stage with 0% transparency so it remains visible.
  9. Begin a forever loop to keep checking the face’s position continuously.
  10. Use the analyse image from camera block to start facial recognition.
  11. Declare the Variable ‘Angle’ Place get () of the face () at the first place of addition () + (), and 3 at the second place. From the dropdown, select X position.
  12. Set the variable Angle by calculating 90 + (x position of face ÷ 3) to decide how far the face is from the center.
  13. Use if-else blocks to respond based on the face’s horizontal position: If Angle > 90: Face is on the right side, show a face on the LED matrix and move left using “lateral left” motion.
  14. Else if Angle < 90: Face is on the left side, show a face and move right using “lateral right” motion.
  15. Else (Angle = 90): Face is centered, Show a smiley face and move to the home (neutral) position.
Note: Check by changing the angle value and also try to change the icons in the display matrix as L for left and R for right sid directions.

Output

Our next step is to check whether it is working right or not. Whenever your face will come in front of the camera, it should detect it and as you move to the right or left, the head of your  Quadruped robot should also move accordingly.

Read More

Gesture Controlled Quadruped

Explore the power of machine learning in recognizing hand gestures and controlling the movements of a Quadruped robot.
Download File

Introduction

This project demonstrates how to use Machine Learning Environment to make a machine–learning model that identifies the hand gestures and makes the Quadruped move accordingly. learning model that identifies the hand gestures and makes the qudruped move accordingly.

We are going to use the Hand Classifier of the Machine Learning Environment. The model works by analyzing your hand position with the help of 21 data points.

Hand Gesture Classifier Workflow

Follow the steps below:

  1. Open PictoBlox and create a new file.
  2. You can click on “Machine Learning Environment” to open it.
  3. Click on “Create New Project“.
  4. A window will open. Type in a project name of your choice and select the “Hand Gesture Classifier” extension. Click the “Create Project” button to open the Hand Pose Classifier window.
  5. You shall see the Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.

Class in Hand Gesture Classifier

There are 2 things that you have to provide in a class:

  1. Class Name: It’s the name to which the class will be referred as.
  2. Hand Pose Data: This data can either be taken from the webcam or by uploading from local storage.

Note: You can add more classes to the projects using the Add Class button.
Adding Data to Class

You can perform the following operations to manipulate the data into a class.

  1. Naming the Class: You can rename the class by clicking on the edit button.
  2. Adding Data to the Class: You can add the data using the Webcam or by Uploading the files from the local folder.
    1. Webcam:
Note: You must add at least 20 samples to each of your classes for your model to train. More samples will lead to better results.

We are going to use the Hand Classifier of the Machine Learning Environment.

 

Training the Model

After data is added, it’s fit to be used in model training. In order to do this, we have to train the model. By training the model, we extract meaningful information from the hand pose, and that in turn updates the weights. Once these weights are saved, we can use our model to make predictions on data previously unseen.

The accuracy of the model should increase over time. The x-axis of the graph shows the epochs, and the y-axis represents the accuracy at the corresponding epoch. Remember, the higher the reading in the accuracy graph, the better the model. The range of the accuracy is 0 to 1.

Testing the Model

 

Hand Pose Classifier

The model will return the probability of the input belonging to the classes.You will have the following output coming from the model.

Logic

The Quadruped will move according to the following logic:

  1. When the forward gesture is detected – Quadruped will move forward.
  2. When the backward gesture is detected – Quadruped will move backward.
  3. When the left gesture is detected – Quadruped will turn left.
  4. When the right gesture is detected – Quadruped will turn right.

Code

Output

Read More

Hand Pose Controlled Mecanum Pick and Place Robot

In this activity, learn how to create a new Machine Learning model that will be able to identify and detect different types of hand poses and that can help us to control the Mecanum Pick and Place Robot.
Download File

In this activity, we will try to create a new Machine Learning model that will be able to identify and detect different types of hand poses and that can help us to control the Mecanum Pick and Place Robot. This activity can be quite fun and by knowing the process, you can develop your own customized hand pose classifier model easily!

We will use the same model that we have created in the previous Hand Pose Controlled Mecanum model to avoid any misdirection and confusion.

Note: You can always create your own model and use it to perform any type of functions as per your choice. This example proves the same point and helps you understand well the concept of Machine Learning models and environment.

Hand Gesture Classifier Workflow

Follow the steps below:

  1. Open PictoBlox and create a new file.
  2. Select the Block coding environment as appropriate Coding Environment.
  3. Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
  4. Click on “Create New Project“.
  5. A window will open. Type in a project name of your choice and select the “Hand Gesture Classifier” extension. Click the “Create Project” button to open the Hand Pose Classifier window.
  6. You shall see the Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.

Class in Hand Gesture Classifier

There are 2 things that you have to provide in a class:

  1. Class Name: It’s the name to which the class will be referred as.
  2. Hand Pose Data: This data can either be taken from the webcam or by uploading from local storage.

Note: You can add more classes to the projects using the Add Class button.
Adding Data to Class

You can perform the following operations to manipulate the data into a class.

  1. Naming the Class: You can rename the class by clicking on the edit button.
  2. Adding Data to the Class: You can add the data using the Webcam or by Uploading the files from the local folder.
    1. Webcam:
Note: You must add at least 20 samples to each of your classes for your model to train. More samples will lead to better results.
Training the Model

After data is added, it’s fit to be used in model training. In order to do this, we have to train the model. By training the model, we extract meaningful information from the hand pose, and that in turn updates the weights. Once these weights are saved, we can use our model to make predictions on data previously unseen.

The accuracy of the model should increase over time. The x-axis of the graph shows the epochs, and the y-axis represents the accuracy at the corresponding epoch. Remember, the higher the reading in the accuracy graph, the better the model. The range of the accuracy is 0 to 1.

Testing the Model

To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

The model will return the probability of the input belonging to the classes.

Export in Block Coding

Click on the “Export Model” button on the top right of the Testing box, and PictoBlox will load your model into the Block Coding Environment if you have opened the ML Environment in the Block Coding.

Logic

The mecanum will move according to the following logic:

  1. If the detected class is “forward”, we will make the Mecanum move forward.
  2. When the backward gesture is detected – Mecanum will move backwards.
  3. When the Lateral Left gesture is detected – Mecanum will move towards the left direction laterally with the help of its omnidirectional wheels.
  4. When the Lateral Right gesture is detected – Mecanum will move towards the right direction laterally with the help of its omnidirectional wheels.
  5. When the Normal Right gesture is detected – Mecanum will rotate on a single point towards the right direction.
  6. When the Normal Left gesture is detected – Mecanum will rotate on a single point towards the left direction.
  7. When the Stop gesture is detected – Mecanum will stop and initiate the Pick Mechanism using the Pick function.
  8. When the Circular Motion gesture is detected – Mecanum will initiate the Place Mechanism using the Place function.

Code

Initialization

Main Code

Output

Forward-Backward Motion:

Circular Right-Left Motion:

Lateral Right-Left Motion:

Pick and Place Mechanism with Hand Pose:

Read More
All articles loaded
No more articles to load
PictoBlox Extensions
  1. Scratch Based Default Extensions
    1. Motion
    2. Looks
    3. Sound
    4. Control
    5. Events
    6. Sensing
    7. Operators
    8. My Blocks
    9. Variables
  2. Artificial Intelligence
    1. Face Detection
    2. Object Detection
    3. Human Body Detection
    4. Computer Vision
    5. Text Recognition
    6. Speech Recognition
    7. ChatGPT
    8. Natural Language Processing
    9. Recognition Cards
    10. Text to Speech
    11. Translate
    12. Open CV
  3. Innovative Extensions
    1. QR Code Scanner
    2. Weather Data
    3. Physics Engine
    4. IFTTT Webhooks
    5. Internet of Things (IoT)
    6. Data Logger
    7. Music
    8. Video Sensing
    9. Pen
  4. Machine Learning Environment
    1. Image Classifier (ML)
    2. Object Detection (ML)
    3. Pose Classifier (ML)
    4. Hand Pose Classifier (ML)
    5. Audio Classifier (ML)
    6. Number Classifier and Regression (ML)
    7. Text Classifier (ML)
  5. Quarky
    1. Quarky (Main)
    2. Display (Quarky)
    3. Robot (Quarky)
    4. Sensors (Quarky)
    5. Speaker (Quarky)
    6. Quarky Ultimate Robots
    7. Quarky Expansion Board
    8. Mars Rover
    9. Humanoid (Quarky)
    10. Quadruped (Quarky)
    11. IoT House (Quarky)
    12. Quarky Mecanum
    13. Quarky Robotic Arm
    14. Dabble
    15. Quarky Advance Line Following

Company

Community

Products

School Programs

Impact Programs

Learning Resources

Product Documentation

Get in Touch

Follow Us

Facebook Twitter Youtube Instagram Linkedin

PictoBlox - Block & Python Coding for Kids

Dabble App - One App. Infinite Control.

Copyright 2025 – Agilo Research Pvt. Ltd. All rights reserved – Terms & Condition | Privacy Policy