when green flag clicked

Description

Scripts that wear this block will activate once the Green Flag has been clicked — these scripts can activate other scripts and enable the entire program. Without this block, the only way a project could run would be that it would sense the pressing of a key or clicking a sprite; the project would only last until all scripts depending on the starting scripts have ended.

Example

Stamping Tobi

The example demonstrates how to use stamping and the mouse location sensing in Block coding.

Script

Output

Walking Tobi

The example demonstrates the wall bouncing of the sprite and rotation style.

Script

Output

Follow the Mouse

The example demonstrates how to make the sprite follow the mouse.

Script

Output

Object Counting Bot in Block Coding using Object Detection

Learn how to use the Object Detection extension of PictoBlox's Machine Learning Environment to count specific targets in images by writing Block code.

Introduction

In this example project we are going to create a Machine Learning Model which can count number of nuts and bolts from the camera feed or images.

Object Detection in Machine Learning Environment

Object Detection is an extension of the ML environment that allows users to detect images and making bounding box into different classes. This feature is available only in the desktop version of PictoBlox for Windows, macOS, or Linux. As part of the Object Detection workflow, users can add classes, upload data, train the model, test the model, and export the model to the Block Coding Environment.

Opening Image Detection Workflow

Alert: The Machine Learning Environment for model creation is available in the only desktop version of PictoBlox for Windows, macOS, or Linux. It is not available in Web, Android, and iOS versions.

Follow the steps below:

Open PictoBlox and create a new file.
Select the Block Coding Environment.
Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
You’ll be greeted with the following screen.
Click on “Create New Project”.
A window will open. Type in a project name of your choice and select the “Object Detection” extension. Click the “Create Project” button to open the Object Detection window.

You shall see the Object Detection workflow. Your environment is all set.

Collecting and Uploading the Data

Uploading images from your device’s hard drive

Now it’s time to upload the images which you downloaded from another source or captured from your camera. Click on the “Select from device” option from the Import Images block.
Now click on the “Choose images from your computer” and go to the folder where you downloaded your images.
Select all images which you want to upload then click on “open” option.
Now page of PictoBlox looks like:

Making Bounding Box – Labelling Images

Labeling is essential for Object Detection. Click on the “Bbox” tab to make the labels.

Notes: Notice how the targets are marked with a bounding box. The labels appear in the “Label List” column on the right.
To create the bounding box in the images, click on the “Create Box” button, to create a bounding box. After the box is drawn, go to the “Label List” column and click on the edit button, and type in a name for the object under the bounding box. This name will become a class. Once you’ve entered the name, click on the tick mark to label the object.
File List: It shows the list of images available for labeling in the project.
Label List: It shows the list of Labels created for the selected image.
Class Info: It shows the summary of the classes with the total number of bounding boxes created for each class.
You can view all the images under the “Image” tab.

Training the Model

In Object Detection, the model must locate and identify all the targets in the given image. This makes Object Detection a complex task to execute. Hence, the hyperparameters work differently in the Object Detection Extension.

Go to the “Train” tab. You should see the following screen:
Click on the “Train New Model” button.
Select all the classes, and click on “Generate Dataset”.
Once the dataset is generated, click “Next”. You shall see the training configurations.
Specify your hyperparameters. If the numbers go out of range, PictoBlox will show a message.
Click “Create”, It creates new model according to inserting value of hyperparameter.
Click “Start Training”, If desired performance is reached, click on the “Stop”
1. Total Loss
2. Regularization Loss
3. Localization Loss
4. Classification Loss
After the training is completed, you’ll see four loss graphs:

Note: Training an Object Detection model is a time taking task. It might take a couple of hours to complete training
You’ll be able to see the graphs under the “Graphs” panel. Click on the buttons to view the graph.
1. Graph between “Total loss” and “Number of steps”.
2. Graph between “Regularization loss” and “Number of steps”.
3. Graph between “Localization” and “Number of steps”.
4. Graph between “Classification loss” and “Number of steps”.

Evaluating the Model

Now, let’s move to the “Evaluate” tab. You can view True Positives, False Negatives, and False Positives for each class here along with metrics like Precision and Recall.

Testing the Model

The model will be tested by uploading an Image from device:

Export in Block Coding

Click on the “PictoBlox” button, and PictoBlox will load your model into the Block Coding Environment if you have opened the ML Environment in the Block Coding.

Code

The idea is simple, we’ll add image samples in the “Backdrops” column. We’ll keep cycling through the backdrops and keep predicting the image on the stage.

Add testing images in the backdrop and delete default backdrop.
Now, come back to the coding tab and select the Tobi sprite.
We’ll start by adding a when flag clicked block from the Events palette.
Add () bounding box block from the Machine Learning palette. Select the “hide” option.
Follow it up with a set detection threshold to () block from the Machine Learning palette and set the drop-down to 0.5.
Add switch backdrop to () block from the Looks palette. Select any image.
Add a forever block from the Control palette
Add analyse image from() block from the Machine Learning palette. Select the “stage” option.
Add () bounding box block from the Machine Learning palette. Select the “show” option.
Add two blocks of say () for () seconds from the Looks palette.
Inside the say block add join () () block from operator palette.
Inside the join block write statement at first empty place and at second empty place add get number of () detected? from the Machine Learning palette.
Select the “Nut” option for first get number of () detected and for second choose “Bolt” option.
Add the () bounding box block from the Looks palette. Select the “hide” option.
Finally, add the next backdrop block from the Looks palette below the () bounding box block.

Final Result

Flower Classification Bot using Image Classification in ML Environment

Learn how to build a Machine Learning model which can identify the type of flower from the camera feed or images using PictoBlox.

Introduction

In this example project we are going to create a Machine Learning Model which can identify the type of flower from the camera feed or images.

Images Classifier in Machine Learning Environment

Image Classifier is an extension of the ML environment that allows users to classify images into different classes. This feature is available only in the desktop version of PictoBlox for Windows, macOS, or Linux. As part of the Image Classifier workflow, users can add classes, upload data, train the model, test the model, and export the model to the Block Coding Environment.

Let’s create the ML model.

Opening Image Classifier Workflow

Follow the steps below:

Open PictoBlox and create a new file.
Select the coding environment as Block Coding Environment.
Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
You’ll be greeted with the following screen.
Click on “Create New Project“.
A window will open. Type in a project name of your choice and select the “Image Classifier” extension. Click the “Create Project” button to open the Image Classifier window.
You shall see the Image Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.

Class in Image Classifier

Class is the category in which the Machine Learning model classifies the images. Similar images are put in one class.

There are 2 things that you have to provide in a class:

Class Name: It’s the name to which the class will be referred as.
Image Data: This data can either be taken from the webcam or by uploading from local storage.

Note: You can add more classes to the projects using the Add Class button.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

Naming the Class: You can rename the class by clicking on the edit button.
Adding Data to the Class: You can add the data using by Uploading the files from the local folder or the Webcam.

Training the Model

After data is added, it’s fit to be used in model training. In order to do this, we have to train the model. By training the model, we extract meaningful information from the images, and that in turn updates the weights. Once these weights are saved, we can use our model to make predictions on data previously unseen.

However, before training the model, there are a few hyperparameters that you should be aware of. Click on the “Advanced” tab to view them.

It’s a good idea to train a numeric classification model for a high number of epochs. The model can be trained in both JavaScript and Python. In order to choose between the two, click on the switch on top of the Training panel.

Note: These hyperparameters can affect the accuracy of your model to a great extent. Experiment with them to find what works best for your data.

Alert: Dependencies must be downloaded to train the model in Python, JavaScript will be chosen by default.

The accuracy of the model should increase over time. The x-axis of the graph shows the epochs, and the y-axis represents the accuracy at the corresponding epoch. Remember, the higher the reading in the accuracy graph, the better the model. The x-axis of the graph shows the epochs, and the y-axis represents the corresponding accuracy. The range of the accuracy is 0 to 1.

Other evaluating parameter we can see by clicking on Train Report

Here we can see confusion matrix and training accuracy of individual classes after training.

Testing the Model

To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

The model will return the probability of the input belonging to the classes.

Export in Block Coding

Click on the “Export Model” button on the top right of the Testing box, and PictoBlox will load your model into the Block Coding Environment if you have opened the ML Environment in the Block Coding.

Code

The idea is simple, we’ll add image samples in the “Backdrops” column. We’ll keep cycling through the backdrops and keep classifying the image on the stage.

Add testing images in the backdrop and delete default backdrop.
Now, come back to the coding tab and select the Tobi sprite.
We’ll start by adding a when flag clicked block from the Events palette.
Add switch backdrop to () block from the Looks palette. Select any image.
Add a forever block from the Control palette.
Inside the forever block add an analyze image from () block from the Machine Learning palette.
Add two blocks of say () for () seconds from the Looks palette.
Inside the say block add join () () block from operator palette.
Inside the join block write statement at first empty place and at second empty place add identified class from the Machine Learning palette.
Finally, add the next backdrop block from the Looks palette below the () bounding box block.

Final Result

Waste Classification Bot using Image Classification in ML Environment

Learn how to build a Machine Learning model which can identify the type of waste from the camera feed or images using PictoBlox.

Introduction

In this example project we are going to create a Machine Learning Model which can identify the type of waste from the camera feed or images.

Images Classifier in Machine Learning Environment

Let’s create the ML model.

Opening Image Classifier Workflow

Follow the steps below:

Open PictoBlox and create a new file.
Select the coding environment as Block Coding Environment.
Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
You’ll be greeted with the following screen.
Click on “Create New Project“.
A window will open. Type in a project name of your choice and select the “Image Classifier” extension. Click the “Create Project” button to open the Image Classifier window.
You shall see the Image Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.

Class in Image Classifier

Class is the category in which the Machine Learning model classifies the images. Similar images are put in one class.

There are 2 things that you have to provide in a class:

Class Name: It’s the name to which the class will be referred as.
Image Data: This data can either be taken from the webcam or by uploading from local storage.

Note: You can add more classes to the projects using the Add Class button.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

Naming the Class: You can rename the class by clicking on the edit button.
Adding Data to the Class: You can add the data using by Uploading the files from the local folder or the Webcam.

Training the Model

After data is added, it’s fit to be used in model training. In order to do this, we have to train the model. By training the model, we extract meaningful information from the images, which in turn updates the weights. Once these weights are saved, we can use our model to make predictions on data previously unseen.

However, before training the model, there are a few hyperparameters that you should be aware of. Click on the “Advanced” tab to view them.

It’s a good idea to train an image classification model for a high number of epochs. The model can be trained in both JavaScript and Python. In order to choose between the two, click on the switch on top of the Training panel.

Note: These hyperparameters can affect the accuracy of your model to a great extent. Experiment with them to find what works best for your data.

Alert: Dependencies must be downloaded to train the model in Python, JavaScript will be chosen by default.

Other evaluating parameters can see by clicking on Train Report

Here we can see the confusion matrix and training accuracy of individual classes after training.

Testing the Model

To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

The model will return the probability of the input belonging to the classes.

Export in Block Coding

Code

The idea is simple, we’ll add image samples in the “Backdrops” column. We’ll keep cycling through the backdrops and keep classifying the image on the stage.

Add testing images in the backdrop and delete the default backdrop.
Now, come back to the coding tab and select the Tobi sprite.
We’ll start by adding a when flag clicked block from the Events palette.
Add switch backdrop to () block from the Looks palette. Select any image.
Add a forever block from the Control palette.
Inside the forever block add an analyze image from () block from the Machine Learning palette.
Add two blocks of say () for () seconds from the Looks palette.
Inside the say block add join () () block from operator palette.
Inside the join block write statement at first empty place and at second empty place add identified class from the Machine Learning palette.
Finally, add the next backdrop block from the Looks palette below the () bounding box block.

Final Result

You can build more applications on top of this waste classifier.

Yoga Pose Timer Bot using Pose Classifier in ML Environment

The examples show how to use pose recognition in PictoBlox to maintain a yoga pose for a particular time interval.

Script

The idea is simple, we’ll add one image of each class in the “costume” column by making one new sprite which will we display on the stage according to input from user. we’ll also change name of the image according to pose.

Add testing images to the backdrop and delete the default backdrop.
Now, come back to the coding tab and select the Tobi sprite.
We’ll start by adding a when flag clicked block from the Events palette.
We made the new variable “count” by choosing the “Make a Variable” option from the Variables palette.
Add the “hide variable () block from the Variables palette. Select count.
Add the “turn () video on stage with () transparency” block from the Machine Learning palette. Select the off option at the first empty place, and for the second, write a 0 value.
Add an “ask () and wait” block from the Sensing palette. Write an appropriate statement in an empty place.
Add the “if () then” block from the control palette for checking the user’s input.
In the empty place of the “if () then” block, add a condition checking block from the operators palette block. At the first empty place, put the answer block from the sensing palette, and at the second place, write an appropriate statement.
Inside the “if () then” block, add a “broadcast ()” block from the Events palette block. Select the “New message” option and write an appropriate statement for broadcasting a message to another sprite.
Add the “turn () video on stage with () transparency” block from the Machine Learning palette. Select the option at the first empty place, and for the second, write a 0 value.
Add the “() key points” block from the Machine Learning palette. Select the show option.
Add the “Set () to ()” block from the Variables palette. Select the count option at the first empty place, and for the second, write a 30 value.
Add the Show variable () block from the Variables palette. Select count.
Add “forever” from the Control palette.
Inside the “forever” block, add an “analysis image from ()” block from the Machine Learning palette. Select the Web camera option.
Inside the “forever” block, add an “if () then” block from the Control palette.
In the empty place of the “if () then” block, add an “is identified class ()” block from the Machine Learning palette. Select the appropriate class from the options.
Inside the “if () then” block, add an “say ()” block from the Looks palette block. Write an appropriate statement in an empty place.
Add “change () by ()” from the Variables palette. Select the count option in the first empty place, and for the second, write a -1 value.
Add the “if () then” block from the control palette for checking the user’s input.
In the empty place of the “if () then” block, add a condition checking block from the operators palette block. In the first empty place, put the “count” block from the sensing palette, and in the second place, write 0.
Add the “Set () to ()” block from the Variables palette. Select the count option at the first empty place, and for the second, write a 30 value.
Add the “turn () video on stage with () transparency” block from the Machine Learning palette. Select the off option at the first empty place, and for the second, write a 0 value.
Inside the “if () then” block, add an “say ()” block from the Looks palette block. Write an appropriate statement in an empty place.
Add the “() key points” block from the Machine Learning palette. Select the hide option
Add the “stop ()” block to the control pallet. Select all options.
Repeat “if () then” block code for other classes, make appropriate changes in copying block code according to other classes, and add code just below it.
The final block code looks like
Now click on another sprite and write code.
We’ll start writing code for this sprite by adding a when flag is clicked block from the Events palette.
Add the “hide” block from the Looks pallet.
Write a new code in the same sprite according to class and add the “when I receive ()” block from the Events palette. Select the appropriate class from the options.
Add the “show” block from the Looks pallet.
Add the “switch costume to ()” block from the Looks palette. Select the appropriate class from the options.
Repeat the same code for other classes and make changes according to the class.

Final Result

Jumping Jack Counter Bot using Pose Classifier in ML Environment

The examples show how to use pose recognition in PictoBlox to make jumping jack counter.

Introduction

In this example project, we are going to create a machine learning model that can count the number of jumping jack activities from the camera feed.

Pose Classifier in Machine Learning Environment

The pose Classifier is the extension of the ML Environment used for classifying different body poses into different classes.

The model works by analyzing your body position with the help of 17 data points.

Pose Classifier Workflow

Open PictoBlox and create a new file.
You can click on “Machine Learning Environment” to open it.
Click on “Create New Project“.
A window will open. Type in a project name of your choice and select the “Pose Classifier” extension. Click the “Create Project” button to open the Pose Classifier window.
You shall see the Pose Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.

Class in Pose Classifier

Class is the category in which the Machine Learning model classifies the poses. Similar posts are put in one class.

There are 2 things that you have to provide in a class:

Class Name: The name to which the class will be referred.
Pose Data: This data can be taken from the webcam or uploaded from local storage.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

Naming the Class: You can rename the class by clicking on the edit button.
Adding Data to the Class: You can add the data using the Webcam or by Uploading the files from the local folder.
1. Webcam:

Training the Model

After data is added, it’s fit to be used in model training. To do this, we have to train the model. By training the model, we extract meaningful information from the hand pose, and that in turn updates the weights. Once these weights are saved, we can use our model to predict previously unseen data.

Testing the Model

To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

The model will return the probability of the input belonging to the classes.

Export in Block Coding

Click on the “Export Model” button on the top right of the Testing box, and PictoBlox will load your model into the Block Coding Environment if you have opened the ML Environment in the Block Coding.

Script

The idea is simple, after running code we will do jumping jack activity in front of camera and tobi sprite will say counting of jumping jack.

Select the Tobi sprite.
We’ll start by adding a when flag clicked block from the Events palette.
We made the new variable “count” by choosing the “Make a Variable” option from the Variables palette.
Also we made the new variable “temp” by choosing the “Make a Variable” option from the Variables palette.
Add “forever” from the Control palette.
Inside the “forever” block, add an “analysis image from ()” block from the Machine Learning palette. Select the Web camera option.
Inside the “forever” block, add an “if () then” block from the Control palette.
In the empty place of the “if () then” block, add an “key () pressed?” block from the Sensing palette. Select the ‘q’ key from the options.
Inside the “if () then” block, add the “Set () to ()” block from the Variables palette. Select the count option at the first empty place, and for the second, write a 0 value.
Also add the “Set () to ()” block from the Variables palette. Select the temp option at the first empty place, and for the second, write a 0 value.
Inside the “forever” block, add an new “if () then” block from the Control palette.
In the empty place of the “if () then” block, add an “is identified class ()” block from the Machine Learning palette. Select the ‘Upper hand‘ option from the options.
Inside the “if () then” block, add the “Set () to ()” block from the Variables palette. Select the temp option at the first empty place, and for the second, write a 1 value.
Inside the “forever” block, add an new “if () then” block from the Control palette.
In the empty place of the “if () then” block, add an “is identified class ()” block from the Machine Learning palette. Select the ‘Down hand‘ option from the options.
Inside the “if () then” block, add the another “if () then” block from the Control palette.
In the empty place of the “if () then” block, add a condition checking block from the operators palette block. At the first empty place, put the temp variable from the variables palette, and at the second place, write a 1 value.
Inside the “if () then” block, add the “Set () to ()” block from the Variables palette. Select the count option at the first empty place, and for the second, write a 1 value.
Also add the “Set () to ()” block from the Variables palette. Select the temp option at the first empty place, and for the second, write a 0 value
Inside the “if () then” block, add an “say () for () seconds” block from the Looks palette block. At the first empty place, add the “join () ()” block from operator palette and at the second place, write a 2 value.
Inside “join () ()” block at the first empty place, write the appropriate statement and at the second place, add count variable from Variables palette.

Final Output

Iris Classifier Bot in Block Coding using Number Classifier

The example shows how to use a number classifier in PictoBlox to make the Iris Classifier Bot.

Script

Add iris image as another sprite and upload one image of all iris classes on costume.
Now, come back to the coding tab and select the Tobi sprite.
We’ll start by adding a when flag clicked block from the Events palette.
Add an “ask () and wait” block from the Sensing palette. Write an appropriate statement in an empty place.
Add the “set () as ()” block from the Machine Learning palette. Select the SepalLengthCm option at the first empty place, and for the second select an “answer” block from the Sensing palette.
Add an “ask () and wait” block from the Sensing palette. Write an appropriate statement in an empty place.
Add the “set () as ()” block from the Machine Learning palette. Select the SepalWidthCm option at the first empty place, and for the second select an “answer” block from the Sensing palette.
Add an “ask () and wait” block from the Sensing palette. Write an appropriate statement in an empty place.
Add the “set () as ()” block from the Machine Learning palette. Select the PetalLengthCm option at the first empty place, and for the second select an “answer” block from the Sensing palette.
Add an “ask () and wait” block from the Sensing palette. Write an appropriate statement in an empty place.
Add the “set () as ()” block from the Machine Learning palette. Select the PetalWidthCm option at the first empty place, and for the second select an “answer” block from the Sensing palette.
Add an “analyse numbers” block from the Machine Learning palette.
Add the “if () then” block from the control palette for checking the user’s input.
In the empty place of the “if () then” block, add an “is identified class ()” block from the Machine Learning palette. Select the appropriate class from the options.
Inside the “if () then” block, add an “say ()” block from the Looks palette block. Write an appropriate statement in an empty place.
Inside the “if () then” block, add a “broadcast ()” block from the Events palette block. Select the “New message” option and write an appropriate statement for broadcasting a message to another sprite.
Repeat “if () then” block code for other iris classes, make appropriate changes in copying block code according to other classes, and add code just below it.
Final code of “Tobi” sprite is
Now click on another sprite and write code.
We’ll start writing code for this sprite by adding a when flag is clicked block from the Events palette.
Add the “hide” block from the Looks pallet.
Write a new code in the same sprite according to class and add the “when I receive ()” block from the Events palette. Select the appropriate class from the options.
Add the “show” block from the Looks pallet.
Add the “switch costume to ()” block from the Looks palette. Select the appropriate class from the options.
Repeat the same code for other classes and make changes according to the class.

Final Result

Hand Sign Classifier Bot in Block Coding

The example shows how to use a hand pose classifier in PictoBlox to make the Sign Classifier Bot.

Introduction

In this example project, we are going to create a machine learning model that can classify different sign messages from the camera feed or image.

Hand Gesture Classifier in Machine Learning Environment

The Hand Gesture Classifier is the extension of the ML Environment used for classifying different hand poses into different classes.

Hand Gesture Classifier Workflow

Open PictoBlox and create a new file.
You can click on “Machine Learning Environment” to open it.
Click on “Create New Project“.
A window will open. Type in a project name of your choice and select the Hand Gesture Classifier” extension. Click the “Create Project” button to open the Pose Classifier window.
You shall see the Gesture Pose Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.

Class in Pose Classifier

Class is the category in which the Machine Learning model classifies the poses. Similar posts are put in one class.

Class in Hand Pose Classifier

Class is the category in which the Machine Learning model classifies the hand poses. Similar hand poses are put in one class.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

Naming the Class: You can rename the class by clicking on the edit button.
Adding Data to the Class: You can add the data using the Webcam or by Uploading the files from the local folder.
1. Webcam:
  
  Note: You can edit the capture setting in the camera with the following. Hold to Record allows you to capture images with pose till the time button is pressed. Whereas when it is off you can set the start delay and duration of the sample collection.
  
  If you want to change your camera feed, you can do it from the webcam selector in the top right corner.

Training the Model

After data is added, it’s fit to be used in model training. In order to do this, we have to train the model. By training the model, we extract meaningful information from the hand pose, and that in turn updates the weights. Once these weights are saved, we can use our model to make predictions on data previously unseen.

However, before training the model, there are a few hyperparameters that you should be aware of. Click on the “Advanced” tab to view them.

Note: These hyperparameters can affect the accuracy of your model to a great extent. Experiment with them to find what works best for your data.

Note: Hover your mouse over the question mark next to the hyperparameters to see their description.

Alert: Dependencies must be downloaded to train the model in Python, JavaScript will be chosen by default.

Testing the Model

To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

The model will return the probability of the input belonging to the classes.

Export in Block Coding

Script

Add one sign image as another sprite and upload at-least one image of all sign classes on costume.
Now, come back to the coding tab and select the Tobi sprite.
We’ll start by adding a when flag clicked block from the Events palette.
Add the “open recognition window” block from the Machine Learning palette.
Add an “forever” block from the Control palette.
Add the “if () then” block from the control palette for checking the user’s input.
In the empty place of the “if () then” block, add an “is identified class ()” block from the Machine Learning palette. Select the appropriate class from the options.
Inside the “if () then” block, add an “say ()” block from the Looks palette block. Write an appropriate statement in an empty place.
Inside the “if () then” block, add a “broadcast ()” block from the Events palette block. Select the “New message” option and write an appropriate statement for broadcasting a message to another sprite.
Repeat “if () then” block code for other classes, make appropriate changes in copying block code according to other classes, and add code just below it.
Final code of “Tobi” sprite is
Now click on another sprite and write code.
We’ll start writing code for this sprite by adding a when flag is clicked block from the Events palette.
Add the “hide” block from the Looks pallet.
Write a new code in the same sprite according to class and add the “when I receive ()” block from the Events palette. Select the appropriate class from the options.
Add the “show” block from the Looks pallet.
Add the “switch costume to ()” block from the Looks palette. Select the appropriate class from the options.
Repeat the same code for other classes and make changes according to the class.

Final Output

Bird Audio Classifier Bot in Block Coding

The example shows how to use a audio classifier in PictoBlox to make the Bird Audio Classifier Bot.

Introduction

In this example project, we are going to create a machine learning model that can classify different audio messages of birds from the microphone feed of computer.

Audio Classifier in Machine Learning Environment

The Audio Classifier is the extension of the ML Environment used for classifying different birds voice.

Audio Classifier Workflow

Follow the steps below to create your own Audio Classifier Model:

Open PictoBlox and create a new file.
Select the Block coding environment as the appropriate Coding Environment.
Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
A new window will open. Type in an appropriate project name of your choice and select the “Audio Classifier” extension. Click the “Create Project” button to open the Audio Classifier Window.
You shall see the Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.
As you can observe in the above image, we will add many classes for audio. We will be able to add audio samples with the help of the microphone.

Note: You can add more classes to the projects using the Add Class button.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

Naming the Class: You can rename the class by clicking on the edit button.
Adding Data to the Class: You can add the data using the Microphone.
You will be able to add the audio sample in each class and make sure you add at least 20 samples for the model to run with good accuracy.

Training the Model

Testing the Model

To test the model simply, use the microphone directly and check the classes as shown in the below image:

You will be able to test the difference in audio samples recorded from the microphone as shown below:

Export in Block Coding

Script

Add one bird image as another sprite and upload at-least one image of all bird classes on costume.
Now, come back to the coding tab and select the Tobi sprite.
We’ll start by adding a when flag clicked block from the Events palette.
Add the “open recognition window” block from the Machine Learning palette.
Add an “when () is predicted” block from the Machine Learning palette. Select the appropriate class from the options.
Add an “say () for () seconds” block from the Looks palette block. Write an appropriate statement in an empty place.
Repeat the same code for other classes and make changes according to the class.
For “BackNoise” class, don’t add any statement at place of empty space of “say () for () seconds” block
Final code of “Tobi” sprite is
Now click on another sprite and write code.
We’ll start writing code for this sprite by adding a “when () is predicted” block from the Machine Learning palette.
Add the “switch costume to ()” block from the Looks palette. Select the appropriate class from the options.
Repeat the same code for other classes and make changes according to the class.
Final code of another sprite is

Final Output

Customer Spending Money Classifier Bot in Block Coding

The example shows how to use a number classifier in PictoBlox to make the customer spending money classifier bot.

Introduction

In this example project, we are going to create a Machine Learning Model that can predict the amount of money a customer will spend based on the input details added by the user.

Numbers(C/R) in Machine Learning Environment

Datasets on the internet are hardly ever fit to directly train on. Programmers often have to take care of unnecessary columns, text data, target columns, correlations, etc. Thankfully, PictoBlox’s ML Environment is packed with features to help us pre-process the data as per our liking.

Let’s create the ML model.

Opening Image Classifier Workflow

Follow the steps below:

Open PictoBlox and create a new file.
Select the coding environment as Block Coding Environment.
Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
You’ll be greeted with the following screen.
Click on “Create New Project“.
You shall see the Numbers C/R workflow with an option to either “Upload Dataset” or “Create Dataset”.

Uploading/Creating Dataset

Datasets can either be uploaded or created on the ML Environment. Lets see how it is done.

Uploading a dataset
1. To upload a dataset, click on the Upload Dataset button and the Choose CSV from your files button.
  
  Note: An uploaded dataset must be a “.csv” file.
2. Once uploaded the first 50 rows of the uploaded CSV document will show up in the window.
Creating a Dataset
1. To create a dataset, click on the Create Dataset button.
2. Select the number of rows and columns that are to be added and click on the Create button. More rows and columns can be added as and when needed.
Notes:
1. Each column represents a feature. These are the values used by the model to train itself.
2. The “Output” column contains the target values. These are the values that we expect the model to return when features are passed.
3. The window only shows the first 50 rows of the dataset.
4. Un-check the “Select All” checkbox to un-select all the columns.
Training the Model

After data is pre-processed and optimized, it’s fit to be used in model training. To train the model, simply click the “Train Model” button found in the “Training” panel.

By training the model, meaningful information is extracted from the numbers, and that in turn updates the weights. Once these weights are saved, the model can be used to make predictions on data previously unseen.

The model’s function is to use the input data and predict the output. The target column must always contain numbers.

However, before training the model, there are a few hyperparameters that need to be understood. Click on the “Advanced” tab to view them.

There are three hyperparameters that can be altered in the Numbers(C/R) Extension:
1. Epochs– The total number of times the data will be fed through the training model. Therefore, in 10 epochs, the dataset will be fed through the training model 10 times. Increasing the number of epochs can often lead to better performance.
2. Batch Size– The size of the set of samples that will be used in one step. For example, if there are 160 data samples in the dataset, and the batch size is set to 16, each epoch will be completed in 160/16=10 steps. This hyperparameter rarely needs any altering.
3. Learning Rate– It dictates the speed at which the model updates the weights after iterating through a step. Even small changes in this parameter can have a huge impact on the model performance. The usual range lies between 0.001 and 0.0001.
Note: Hover the mouse pointer over the question mark next to the hyperparameters to see their description.

It’s a good idea to train a numeric classification model for a high number of epochs. The model can be trained in both JavaScript and Python. In order to choose between the two, click on the switch on top of the Training panel.

Alert: Dependencies must be downloaded to train the model in Python, JavaScript will be chosen by default.

The accuracy of the model should increase over time. The x-axis of the graph shows the epochs, and the y-axis represents the accuracy at the corresponding epoch.

A window will open. Type in a project name of your choice and select the “Numbers(C/R)” extension. Click the “Create Project” button to open the Numbers(C/R) window.

Testing the Model

To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

The model will return the probability of the input belonging to the classes.

Export in Block Coding

Script

Select the “Tobi” sprite.
We’ll start by adding a “when flag clicked” block from the Events palette.
Add an “ask () and wait” block from the Sensing palette. Write an appropriate statement in an empty place.
Add the “if () then” block from the control palette for checking the user’s input.
In the empty place of the “if () then” block, add an “()=()” block from the Operator palette. At the first empty place select an “answer” block from the Sensing palette and for the second place write an appropriate statement in an empty place.
Add the “set gender as ()” block from the Machine Learning palette. Select the Male option at the empty place.
Repeat “if () then” block code for other variables, make appropriate changes in copying block code according to other variables, and add code just below it.
Add an “ask () and wait” block from the Sensing palette. Write an appropriate statement in an empty place.
Add the “if () then” block from the control palette for checking the user’s input.
In the empty place of the “if () then” block, add an “()=()” block from the Operator palette. At the first empty place select an “answer” block from the Sensing palette and for the second place write an appropriate statement in an empty place.
Add the “set education as ()” block from the Machine Learning palette. Select the High School option at the empty place.
Repeat “if () then” block code for other variables, make appropriate changes in copying block code according to other variables, and add code
Add an “ask () and wait” block from the Sensing palette. Write an appropriate statement in an empty place.
Add the “set () as ()” block from the Machine Learning palette. Select the age option at the first empty place, and for the second select an “answer” block from the Sensing palette.
Repeat “ask () and wait” block code for other variables, make appropriate changes in copying block code according to other variables, and add code.
Repeat “ask () and wait” block code for other variables, make appropriate changes in copying block code according to other variables, and add code.
Add the “say ()” block from the Looks palette.
Add an “join () ()” block from the Operator palette. Write an appropriate statement in an first empty place and at second empty place add “analyse numbers” block from the Machine Learning palette.

Final Result

All articles loaded

No more articles to load

PictoBlox Extensions

when green flag clicked

Description

Example

Script

Output

Script

Output

Script

Output

Introduction

Object Detection in Machine Learning Environment

Opening Image Detection Workflow

Collecting and Uploading the Data

Making Bounding Box – Labelling Images

Training the Model

Evaluating the Model

Testing the Model

Export in Block Coding

Code

Final Result

Introduction

Images Classifier in Machine Learning Environment

Opening Image Classifier Workflow

Class in Image Classifier

Adding Data to Class

Training the Model

Testing the Model

Export in Block Coding

Code

Final Result

Introduction

Images Classifier in Machine Learning Environment

Opening Image Classifier Workflow

Class in Image Classifier

Adding Data to Class

Training the Model

Other evaluating parameters can see by clicking on Train Report

Testing the Model

Export in Block Coding

Code

Final Result

Script

Final Result

Introduction

Pose Classifier in Machine Learning Environment

Pose Classifier Workflow

Class in Pose Classifier

Adding Data to Class

Training the Model

Testing the Model

Export in Block Coding

Script

Final Output

Script

Final Result

Introduction

Hand Gesture Classifier in Machine Learning Environment

Hand Gesture Classifier Workflow

Class in Pose Classifier

Class in Hand Pose Classifier

Adding Data to Class

Training the Model

Testing the Model

Export in Block Coding

Script

Final Output

Introduction

Audio Classifier in Machine Learning Environment

Audio Classifier Workflow

Adding Data to Class

Training the Model

Testing the Model

Export in Block Coding

Script

Introduction

Numbers(C/R) in Machine Learning Environment

Opening Image Classifier Workflow

Uploading/Creating Dataset

Uploading a dataset

Creating a Dataset