identified class

Flower Classification Bot using Image Classification in ML Environment

Learn how to build a Machine Learning model which can identify the type of flower from the camera feed or images using PictoBlox.

Introduction

In this example project we are going to create a Machine Learning Model which can identify the type of flower from the camera feed or images.

Images Classifier in Machine Learning Environment

Image Classifier is an extension of the ML environment that allows users to classify images into different classes. This feature is available only in the desktop version of PictoBlox for Windows, macOS, or Linux. As part of the Image Classifier workflow, users can add classes, upload data, train the model, test the model, and export the model to the Block Coding Environment.

Let’s create the ML model.

Opening Image Classifier Workflow

Alert: The Machine Learning Environment for model creation is available in the only desktop version of PictoBlox for Windows, macOS, or Linux. It is not available in Web, Android, and iOS versions.

Follow the steps below:

Open PictoBlox and create a new file.
Select the coding environment as Block Coding Environment.
Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
You’ll be greeted with the following screen.
Click on “Create New Project“.
A window will open. Type in a project name of your choice and select the “Image Classifier” extension. Click the “Create Project” button to open the Image Classifier window.
You shall see the Image Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.

Class in Image Classifier

Class is the category in which the Machine Learning model classifies the images. Similar images are put in one class.

There are 2 things that you have to provide in a class:

Class Name: It’s the name to which the class will be referred as.
Image Data: This data can either be taken from the webcam or by uploading from local storage.

Note: You can add more classes to the projects using the Add Class button.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

Naming the Class: You can rename the class by clicking on the edit button.
Adding Data to the Class: You can add the data using by Uploading the files from the local folder or the Webcam.

Training the Model

After data is added, it’s fit to be used in model training. In order to do this, we have to train the model. By training the model, we extract meaningful information from the images, and that in turn updates the weights. Once these weights are saved, we can use our model to make predictions on data previously unseen.

However, before training the model, there are a few hyperparameters that you should be aware of. Click on the “Advanced” tab to view them.

It’s a good idea to train a numeric classification model for a high number of epochs. The model can be trained in both JavaScript and Python. In order to choose between the two, click on the switch on top of the Training panel.

Note: These hyperparameters can affect the accuracy of your model to a great extent. Experiment with them to find what works best for your data.

Alert: Dependencies must be downloaded to train the model in Python, JavaScript will be chosen by default.

The accuracy of the model should increase over time. The x-axis of the graph shows the epochs, and the y-axis represents the accuracy at the corresponding epoch. Remember, the higher the reading in the accuracy graph, the better the model. The x-axis of the graph shows the epochs, and the y-axis represents the corresponding accuracy. The range of the accuracy is 0 to 1.

Other evaluating parameter we can see by clicking on Train Report

Here we can see confusion matrix and training accuracy of individual classes after training.

Testing the Model

To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

The model will return the probability of the input belonging to the classes.

Export in Block Coding

Click on the “Export Model” button on the top right of the Testing box, and PictoBlox will load your model into the Block Coding Environment if you have opened the ML Environment in the Block Coding.

Code

The idea is simple, we’ll add image samples in the “Backdrops” column. We’ll keep cycling through the backdrops and keep classifying the image on the stage.

Add testing images in the backdrop and delete default backdrop.
Now, come back to the coding tab and select the Tobi sprite.
We’ll start by adding a when flag clicked block from the Events palette.
Add switch backdrop to () block from the Looks palette. Select any image.
Add a forever block from the Control palette.
Inside the forever block add an analyze image from () block from the Machine Learning palette.
Add two blocks of say () for () seconds from the Looks palette.
Inside the say block add join () () block from operator palette.
Inside the join block write statement at first empty place and at second empty place add identified class from the Machine Learning palette.
Finally, add the next backdrop block from the Looks palette below the () bounding box block.

Final Result

Sound Based Quadruped with Machine Learning

Learn how to create custom sounds to control Quadruped with the Audio Classifier of the Machine Learning Environment in PictoBlox.

Introduction

A Sound-Based Quadruped with Machine Learning refers to a Quadruped robot that can perceive and interact with its environment through sound-based sensing and uses machine-learning techniques to process and analyze the auditory data it receives.
Quadruped robots with machine learning have the potential to greatly enhance the way we interact with machines and each other, making communication more natural and intuitive while also enabling new applications in fields such as healthcare, education, and entertainment.
In this activity, we will use the Machine Learning Environment of the Pictoblox Software. We will use the Audio Classifier of the Machine Learning Environment and create our custom sounds to control the Quadruped.

Audio Classifier Workflow

Follow the steps below to create your own Audio Classifier Model:

Open PictoBlox and create a new file.
Select the Block coding environment as the appropriate Coding Environment.
Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
A new window will open. Type in an appropriate project name of your choice and select the “Audio Classifier” extension. Click the “Create Project” button to open the Audio Classifier Window.
You shall see the Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.
As you can observe in the above image, we will add two classes for audio. We will be able to add audio samples with the help of the microphone. Rename class 1 as “Clap” and class 2 as “Snap”.

Note: You can add more classes to the projects using the Add Class button.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

Naming the Class: You can rename the class by clicking on the edit button.
Adding Data to the Class: You can add the data using the Microphone.
You will be able to add the audio sample in each class and make sure you add at least 20 samples for the model to run with good accuracy.
Add the first class as “clap” and record the audio for clap noises through the microphone.
Add the second class as “snap” and record the audio for snap noises through the microphone.

Note: You will only be able to change the class name in the starting before adding any audio samples. You will not be able to change the class name after adding the audio samples in the respective class.

Training the Model

After data is added, it’s fit to be used in model training. To do this, we have to train the model. By training the model, we extract meaningful information from the hand pose, and that in turn updates the weights. Once these weights are saved, we can use our model to make predictions on data previously unseen.

Testing the Model

To test the model simply, use the microphone directly and check the classes as shown in the below image:

You will be able to test the difference in audio samples recorded from the microphone as shown below:

Export in Block Coding

Click on the “Export Model” button on the top right of the Testing box, and PictoBlox will load your model into the Block Coding Environment if you have opened the ML Environment in the Block Coding.

The Quadruped will move according to the following logic:

When the audio is identified as “clap” sound– Quadruped will move forward.
When the “snap” sound is detected –Quadruped will move backward.

Note: You can add even more classes with different types of differentiating sounds to customize your control. This is just a small example from which you can build your own Sound Based Controlled Quadruped in a very easy stepwise procedure.

Code

Logic

First, initialize the Quadruped extension.
Then, initialize a forever loop to continuously loop and analyze the camera from the stage.
If the program detects a clap sound, the Quadruped will move forward at a specific speed.
Similarly, if it identifies a snap sound, the Quadruped will move backward at a specific speed.
Otherwise, the Quadruped will remain in its initial position (home position).

identified class

Description

Example

Flower Classification Bot using Image Classification in ML Environment

Introduction

Images Classifier in Machine Learning Environment

Opening Image Classifier Workflow

Class in Image Classifier

Adding Data to Class

Training the Model

Testing the Model

Export in Block Coding

Code

Final Result

Sound Based Quadruped with Machine Learning

Introduction

Audio Classifier Workflow

Adding Data to Class

Training the Model

Testing the Model

Export in Block Coding

Code

Logic

Output

Company

Community

Products

School Programs

Impact Programs

Learning Resources

Product Documentation

Get in Touch

Follow Us

PictoBlox - Block & Python Coding for Kids

Dabble App - One App. Infinite Control.