open recognition window

Description

The block opens the recognition window and shows the machine learning analysis on the camera feed. Very good for visualization of the model in PictoBlox.

Example

Gesture Controlled Robotic Arm With Machine Learning

Discover how gesture-controlled robotic arms revolutionize robotics with intuitive control. Learn about their applications in manufacturing, healthcare, and virtual reality.

Introduction

A gesture-controlled robotic arm is a robotic arm that can be controlled using hand or body movements instead of traditional buttons or joysticks. It uses sensors and algorithms to interpret the gestures made by a user and translates them into commands for the robotic arm.

The user wears or holds a device with sensors, such as a glove or wristband, that captures their hand movements or body gestures. These movements are processed by a computer or microcontroller, which analyzes them and recognizes specific gestures using algorithms and machine learning techniques.

Once the gestures are recognized, the system generates commands for the robotic arm to move accordingly. The arm can have multiple joints and degrees of freedom to perform complex movements. The user’s gestures are mimicked by the robotic arm, allowing them to control its actions.

Gesture-controlled robotic arms are used in various fields, including manufacturing, healthcare, and virtual reality. They provide a more intuitive and natural way of controlling robotic systems, eliminating the need for complex input devices and extensive training.

Hand Gesture Classifier Workflow

Follow the steps below:

Open PictoBlox and create a new file.
Select the coding environment as appropriate Coding Environment. you can click on “Machine Learning Environment” to open it.
Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
Click on “Create New Project“.
A window will open. Type in a project name of your choice and select the “Hand Gesture Classifier” extension. Click the “Create Project” button to open the Hand Pose Classifier window.
You shall see the Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.

Class in Hand Gesture Classifier

There are 2 things that you have to provide in a class:

Class Name: The name to which the class will be referred.
Hand Pose Data: This data can be taken from the webcam or uploaded from local storage.

Note: You can add more classes to the projects using the Add Class button.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

Naming the Class: You can rename the class by clicking on the edit button.
Adding Data to the Class: You can add the data using the Webcam or by Uploading the files from the local folder.
1. Webcam:

Note: You must add at least 20 samples to each of your classes for your model to train. More samples will lead to better results.

Training the Model

After data is added, it’s fit to be used in model training. To do this, we have to train the model. By training the model, we extract meaningful information from the hand pose, and that in turn updates the weights. Once these weights are saved, we can use our model to predict previously unseen data.

The accuracy of the model should increase over time. The x-axis of the graph shows the epochs, and the y-axis represents the accuracy at the corresponding epoch. Remember, the higher the reading in the accuracy graph, the better the model. The range of accuracy is 0 to 1.

Testing the Model

To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

The model will return the probability of the input belonging to the classes.

Export in Block Coding

Click on the “Export Model” button on the top right of the Testing box, and PictoBlox will load your model into the Block Coding Environment if you have opened the ML Environment in the Block Coding.

Logic

The robotic arm will move according to the following logic:

When the left gesture is detected – the robotic arm will move in the anti-clockwise direction.
When the right gesture is detected – the robotic arm will move in a Clockwise direction.

Code

Logic

First we Initialize Robitc Arm extension.
Then, we open the recognition window, which will identify different poses, and turn on the camera with a certain level of transparency to identify images from the stage.
If the identified class is “left,” the Robotic arm will move in anti-clockwise direction using move in () circle of center X() Z(),radius() & along Y() in ()ms block.
If the identified class is “right,” the Robotic Arm will move right using move in () circle of center X() Z(),radius() & along Y() in ()ms block
Press Run to run the code.

Output

Hand Pose Controlled Mecanum

This project demonstrates how to use Machine Learning Environment to make a machine–learning model that identifies the hand gestures and makes the Mecanum robot move accordingly.

This project demonstrates how to use Machine Learning Environment to make a machine–learning model that identifies the hand gestures and makes the Mecanum move accordingly.

We are going to use the Hand Classifier of the Machine Learning Environment. The model works by analyzing your hand position with the help of 21 data points. We will add in total 8 different classes to operate the different motions of the Mecanum Robot with the help of the ML Environment of the Pictoblox Software.

Hand Gesture Classifier Workflow

Follow the steps below:

Open PictoBlox and create a new file.
Select the coding environment as appropriate Coding Environment.
Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
Click on “Create New Project“.
A window will open. Type in a project name of your choice and select the “Hand Gesture Classifier” extension. Click the “Create Project” button to open the Hand Pose Classifier window.
You shall see the Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.

Class in Hand Gesture Classifier

There are 2 things that you have to provide in a class:

Class Name: It’s the name to which the class will be referred as.
Hand Pose Data: This data can either be taken from the webcam or by uploading from local storage.

Note: You can add more classes to the projects using the Add Class button.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

Naming the Class: You can rename the class by clicking on the edit button.
Adding Data to the Class: You can add the data using the Webcam or by Uploading the files from the local folder.
1. Webcam:

Note: You must add at least 20 samples to each of your classes for your model to train. More samples will lead to better results.

Training the Model

After data is added, it’s fit to be used in model training. In order to do this, we have to train the model. By training the model, we extract meaningful information from the hand pose, and that in turn updates the weights. Once these weights are saved, we can use our model to make predictions on data previously unseen.

Testing the Model

To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

The model will return the probability of the input belonging to the classes.

Export in Block Coding

Logic

The mecanum will move according to the following logic:

When the forward gesture is detected – Mecanum will move forward.
When the backward gesture is detected – Mecanum will move backwards.
When the Lateral Left gesture is detected – Mecanum will move towards the left direction laterally with the help of its omnidirectional wheels.
When the Lateral Right gesture is detected – Mecanum will move towards the right direction laterally with the help of its omnidirectional wheels.
When the Stop gesture is detected – Mecanum will stop moving.
When the Normal Left gesture is detected – Mecanum will rotate in the left direction.
When the Normal Right gesture is detected – Mecanum will rotate in the right direction.
When the Circular Motion gesture is detected – Mecanum will move in a lateral arc.

Code

Initialization

Main Code

Output

Forward-Backward Motions:

Lateral Right-Left Motions:

Circular Right-Left Motions:

Lateral Arc Motion:

Sound-Based Mars Control

Learn how to create custom sounds to control Mars Rover with the Audio Classifier of the Machine Learning Environment in PictoBlox. Start building your Sound Based Controlled Mars Rover now!

In this activity, we will use the Machine Learning Environment of the Pictoblox Software. We will use the Audio Classifier of the Machine Learning Environment and create our custom sounds to control the Mars Rover.

Audio Classifier Workflow

Follow the steps below to create your own Audio Classifier Model:

Open PictoBlox and create a new file.
Select the Block coding environment as appropriate Coding Environment.
Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
Click on “Create New Project“.
A new window will open. Type in an appropriate project name of your choice and select the “Audio Classifier” extension. Click the “Create Project” button to open the Audio Classifier Window.
You shall see the Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.
As you can observe in the above image, we will add two classes for audio. We will be able to add audio samples with the help of the microphone. Rename the class1 as “Clap” and class2 as “Snap”.

Note: You can add more classes to the projects using the Add Class button.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

Naming the Class: You can rename the class by clicking on the edit button.
Adding Data to the Class: You can add the data using the Microphone.
You will be able to add the audio sample in each class and make sure you add atleast 20 samples for the model to run with good accuracy.
Add the first class as “clap” and record the audio for clap noises through the microphone.
Add the second class as “snap” and record the audio for snap noises through the microphone.

Note: You will only be able to change the class name in the starting before adding any audio samples. You will not be able to change the class name after adding the audio samples in the respective class.

Training the Model

Testing the Model

To test the model simply, use the microphone directly and check the classes as shown in the below image:

You will be able to test the difference in audio samples recorded from the microphone as shown below:

Export in Block Coding

Logic

The Mars Rover will move according to the following logic:

When the audio is identified as “clap”- Mars Rover will move forward.
When the “snap” sound is detected –Mars Rover will move backward.

Note: You can add even more classes with different types of differentiating sounds to customize your control. This is just a small example from which you can build your own Sound Based Controlled Mars Rover in a very easy stepwise procedure.

Code

Logic

First we will initialize different Audio classes.
Then, we will open the recognition window, which will identify different audio and turn on the microphone to identify and record the audio from the microphone.
If the identified class from the analyzed audio is “clap,” the Mars Rover will move forward at a specific speed.
If the identified class is “snap,” the Mars Rover will move backward.

Output

Gesture Controlled Mars Rover

This project demonstrates how to use Machine Learning Environment to make a machine–learning model that identifies the hand gestures and makes the Mars Rover move accordingly.

This project demonstrates how to use Machine Learning Environment to make a machine–learning model that identifies hand gestures and makes the Mars Rover move accordingly.

We are going to use the Hand Classifier of the Machine Learning Environment. The model works by analyzing your hand position with the help of 21 data points.

Hand Gesture Classifier Workflow

Follow the steps below:

Open PictoBlox and create a new file.
Select the coding environment as appropriate Coding Environment.
Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
Click on “Create New Project“.
A window will open. Type in a project name of your choice and select the “Hand Gesture Classifier” extension. Click the “Create Project” button to open the Hand Pose Classifier window.
You shall see the Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.

Class in Hand Gesture Classifier

There are 2 things that you have to provide in a class:

Class Name: It’s the name to which the class will be referred as.
Hand Pose Data: This data can either be taken from the webcam or by uploading from local storage.

Note: You can add more classes to the projects using the Add Class button.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

Naming the Class: You can rename the class by clicking on the edit button.
Adding Data to the Class: You can add the data using the Webcam or by Uploading the files from the local folder.
1. Webcam:

Note: You must add at least 20 samples to each of your classes for your model to train. More samples will lead to better results.

Training the Model

Testing the Model

To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

The model will return the probability of the input belonging to the classes.

Export in Block Coding

Logic

The Mars Roverwill move according to the following logic:

When the forward gesture is detected – Mars Rover will move forward.
When the backward gesture is detected –Mars Rover will move backward.
When the left gesture is detected –Mars Rover will turn left.
When the right gesture is detected – Mars Rover will turn right.

Code

Logic

First, we will initialize different Gesture classes.
Then, we will open the recognition window, which will identify different poses and turn on the camera with a certain level of transparency to identify images from the stage.
If the identified class from the analyzed image is “forward,” the Mars Rover will move forward at a specific speed.
If the identified class is “backward,” the Mars Rover will move backward.
If the identified class is “left,” the Mars Rover will move left.
If the identified class is “right,” the Mars Rover will move right.
Otherwise, the Mars Rover will be in the home position.

Output

Sound Based Quadruped with Machine Learning

Learn how to create custom sounds to control Quadruped with the Audio Classifier of the Machine Learning Environment in PictoBlox.

Introduction

A Sound-Based Quadruped with Machine Learning refers to a Quadruped robot that can perceive and interact with its environment through sound-based sensing and uses machine-learning techniques to process and analyze the auditory data it receives.
Quadruped robots with machine learning have the potential to greatly enhance the way we interact with machines and each other, making communication more natural and intuitive while also enabling new applications in fields such as healthcare, education, and entertainment.
In this activity, we will use the Machine Learning Environment of the Pictoblox Software. We will use the Audio Classifier of the Machine Learning Environment and create our custom sounds to control the Quadruped.

Audio Classifier Workflow

Follow the steps below to create your own Audio Classifier Model:

Open PictoBlox and create a new file.
Select the Block coding environment as the appropriate Coding Environment.
Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
A new window will open. Type in an appropriate project name of your choice and select the “Audio Classifier” extension. Click the “Create Project” button to open the Audio Classifier Window.
You shall see the Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.
As you can observe in the above image, we will add two classes for audio. We will be able to add audio samples with the help of the microphone. Rename class 1 as “Clap” and class 2 as “Snap”.

Note: You can add more classes to the projects using the Add Class button.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

Naming the Class: You can rename the class by clicking on the edit button.
Adding Data to the Class: You can add the data using the Microphone.
You will be able to add the audio sample in each class and make sure you add at least 20 samples for the model to run with good accuracy.
Add the first class as “clap” and record the audio for clap noises through the microphone.
Add the second class as “snap” and record the audio for snap noises through the microphone.

Note: You will only be able to change the class name in the starting before adding any audio samples. You will not be able to change the class name after adding the audio samples in the respective class.

Training the Model

Testing the Model

To test the model simply, use the microphone directly and check the classes as shown in the below image:

You will be able to test the difference in audio samples recorded from the microphone as shown below:

Export in Block Coding

The Quadruped will move according to the following logic:

When the audio is identified as “clap” sound– Quadruped will move forward.
When the “snap” sound is detected –Quadruped will move backward.

Note: You can add even more classes with different types of differentiating sounds to customize your control. This is just a small example from which you can build your own Sound Based Controlled Quadruped in a very easy stepwise procedure.

Code

Logic

First, initialize the Quadruped extension.
Then, initialize a forever loop to continuously loop and analyze the camera from the stage.
If the program detects a clap sound, the Quadruped will move forward at a specific speed.
Similarly, if it identifies a snap sound, the Quadruped will move backward at a specific speed.
Otherwise, the Quadruped will remain in its initial position (home position).

Output

Sound Based Humanoid with Machine Learning

Learn how to create custom sounds to control Humanoid with the Audio Classifier of the Machine Learning Environment in PictoBlox.

Introduction

A Sound-Based Humanoid with Machine Learning refers to a Humanoid robot that can perceive and interact with its environment through sound-based sensing and uses machine-learning techniques to process and analyze the auditory data it receives.

Humanoid robots with machine learning have the potential to greatly enhance the way we interact with machines and each other, making communication more natural and intuitive while also enabling new applications in fields such as healthcare, education, and entertainment.

Audio Classifier Workflow

Follow the steps below to create your own Audio Classifier Model:

Open PictoBlox and create a new file.
Select the Block coding environment as the appropriate Coding Environment.
Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
A new window will open. Type in an appropriate project name of your choice and select the “Audio Classifier” extension. Click the “Create Project” button to open the Audio Classifier Window.
You shall see the Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.
As you can observe in the above image, we will add two classes for audio. We will be able to add audio samples with the help of the microphone. Rename class 1 as “Clap” and class 2 as “Snap”.

Note: You can add more classes to the projects using the Add Class button.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

Naming the Class: You can rename the class by clicking on the edit button.
Adding Data to the Class: You can add the data using the Microphone.
You will be able to add the audio sample in each class and make sure you add at least 20 samples for the model to run with good accuracy.
Add the first class as “clap” and record the audio for clap noises through the microphone.
Add the second class as “snap” and record the audio for snap noises through the microphone.

Note: You will only be able to change the class name in the starting before adding any audio samples. You will not be able to change the class name after adding the audio samples in the respective class.

Training the Model

Testing the Model

To test the model simply, use the microphone directly and check the classes as shown in the below image:

You will be able to test the difference in audio samples recorded from the microphone as shown below:

Export in Block Coding

The Humanoid will move according to the following logic:

When the audio is identified as “clap”- Humanoid will move forward.
When the “snap” sound is detected –the Humanoid will move backward.

Code

Logic

First, initialize the Humanoid extension.
Then, initialize a forever loop to continuously loop and analyze the camera from the stage.
If the program detects a clap sound, the Humanoid will move forward at a specific speed.
Similarly, if it identifies a snap sound, the Humanoid will move backward at a specific speed.
Otherwise, the Humanoid will remain in its initial position (home position).

Output

All articles loaded

No more articles to load

PictoBlox Extensions

open recognition window

Description

Example

Introduction

Hand Gesture Classifier Workflow

Class in Hand Gesture Classifier

Adding Data to Class

Training the Model

Testing the Model

Export in Block Coding

Logic

Code

Logic

Output

Hand Gesture Classifier Workflow

Class in Hand Gesture Classifier

Adding Data to Class

Training the Model

Testing the Model

Export in Block Coding

Logic

Code

Main Code

Output

Audio Classifier Workflow

Adding Data to Class

Training the Model

Testing the Model

Export in Block Coding

Logic

Code

Logic

Output

Hand Gesture Classifier Workflow

Class in Hand Gesture Classifier

Adding Data to Class

Training the Model

Testing the Model

Export in Block Coding

Logic

Code

Logic

Output

Introduction

Audio Classifier Workflow

Adding Data to Class

Training the Model

Testing the Model

Export in Block Coding

Code

Logic

Output

Introduction

Audio Classifier Workflow

Adding Data to Class

Training the Model

Testing the Model

Export in Block Coding

Code

Logic

Output

Company

Community

Get in Touch

Follow Us

Dabble App - One App. Infinite Control.