Table of Contents

set detection threshold to ()

Description

This block is used to set the threshold for the confidence (accuracy) of object detection, 0 being low confidence and 1 being high confidence. With the threshold value, you can set the level of confidence required for object detection.

Example

The example demonstrates how to detect persons on the stage with different confidence thresholds.

Script

Output

Read More
The example demonstrates how to run an object detection on the stage and show all the objects with confidence.

Script

Output

  1. Detection at 0.3
  2. Detection at 0.5
  3. Detection at 0.8
Read More
Learn how to use the Object Detection extension of PictoBlox's Machine Learning Environment to count specific targets in images by writing Block code.

Introduction

In this example project we are going to create a Machine Learning Model which can count number of nuts and bolts from the camera feed or images.

Object Detection in Machine Learning Environment

Object Detection is an extension of the ML environment that allows users to detect images and making bounding box into different classes. This feature is available only in the desktop version of PictoBlox for Windows, macOS, or Linux. As part of the Object Detection workflow, users can add classes, upload data, train the model, test the model, and export the model to the Block Coding Environment.

 Opening Image Detection Workflow

Alert: The Machine Learning Environment for model creation is available in the only desktop version of PictoBlox for Windows, macOS, or Linux. It is not available in Web, Android, and iOS versions.

 

Follow the steps below:

  1. Open PictoBlox and create a new file.
  2.  Select the Block Coding Environment.
  3. Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
  4. You’ll be greeted with the following screen.
    Click on “Create New Project”.
  5. A window will open. Type in a project name of your choice and select the “Object Detection” extension. Click the “Create Project” button to open the Object Detection window.

You shall see the Object Detection workflow. Your environment is all set.
   

Collecting and Uploading the Data

Uploading images from your device’s hard drive

  1. Now it’s time to upload the images which you downloaded from another source or captured from your camera. Click on the “Select from device” option from the Import Images block.
  2. Now click on the “Choose images from your computer” and go to the folder where you downloaded your images.
  3. Select all images which you want to upload then click on “open” option.
  4. Now page of PictoBlox looks like:

Making Bounding Box – Labelling Images

  1. Labeling is essential for Object Detection. Click on the “Bbox” tab to make the labels.

    Notes: Notice how the targets are marked with a bounding box. The labels appear in the “Label List” column on the right.

  2. To create the bounding box in the images, click on the “Create Box” button, to create a bounding box. After the box is drawn, go to the “Label List” column and click on the edit button, and type in a name for the object under the bounding box. This name will become a class. Once you’ve entered the name, click on the tick mark to label the object.
  3. File List: It shows the list of images available for labeling in the project.
  4. Label List: It shows the list of Labels created for the selected image.
  5. Class Info: It shows the summary of the classes with the total number of bounding boxes created for each class.
  6.   You can view all the images under the “Image” tab.

Training the Model

In Object Detection, the model must locate and identify all the targets in the given image. This makes Object Detection a complex task to execute. Hence, the hyperparameters work differently in the Object Detection Extension.

  1. Go to the “Train” tab. You should see the following screen:
  2. Click on the “Train New Model” button.
  3. Select all the classes, and click on “Generate Dataset”.
  4.  Once the dataset is generated, click “Next”. You shall see the training configurations.
  5. Specify your hyperparameters. If the numbers go out of range, PictoBlox will show a message.
  6. Click “Create”, It creates new model according to inserting value of hyperparameter.
  7. Click “Start Training”, If desired performance is reached, click on the “Stop” 
    1. Total Loss
    2. Regularization Loss
    3. Localization Loss
    4. Classification Loss
  8. After the training is completed, you’ll see four loss graphs: 
    Note: Training an Object Detection model is a time taking task. It might take a couple of hours to complete training

     

  9. You’ll be able to see the graphs under the “Graphs” panel. Click on the buttons to view the graph.

    1. Graph between “Total loss” and “Number of steps”
    2. Graph between “Regularization loss” and “Number of steps”.
    3. Graph between “Localization” and “Number of steps”.
    4. Graph between “Classification loss” and “Number of steps”.

Evaluating the Model

Now, let’s move to the “Evaluate” tab. You can view True Positives, False Negatives, and False Positives for each class here along with metrics like Precision and Recall.

Testing the Model

The model will be tested by uploading an Image from device:

 

Export in Block Coding

Click on the “PictoBlox” button, and PictoBlox will load your model into the Block Coding Environment if you have opened the ML Environment in the Block Coding.

 

 

Code

The idea is simple, we’ll add image samples in the “Backdrops” column. We’ll keep cycling through the backdrops and keep predicting the image on the stage.

  1. Add testing images in the backdrop and delete default backdrop.
  2. Now, come back to the coding tab and select the Tobi sprite.
  3. We’ll start by adding a when flag clicked block from the Events palette.
  4. Add () bounding box block from the Machine Learning palette. Select the “hide” option.
  5. Follow it up with a set detection threshold to () block from the Machine Learning palette and set the drop-down to 0.5.
  6. Add switch backdrop to () block from the Looks palette. Select any image.
  7. Add a forever block from the  Control palette
  8. Add analyse image from() block from the Machine Learning palette. Select the “stage” option.
  9. Add () bounding box block from the Machine Learning palette. Select the “show” option.
  10. Add two blocks of say () for () seconds from the Looks palette.
  11. Inside the say block add join () () block from operator palette.
  12. Inside the join block write statement at first empty place and at second empty place add get number of () detected? from the Machine Learning palette.
  13. Select the “Nut” option for first get number of () detected and for second choose “Bolt” option.
  14. Add the () bounding box block from the Looks palette. Select the “hide” option.
  15. Finally, add the next backdrop block from the Looks palette below the () bounding box block.

Final Result

 

 

 

Read More
Learn how to code logic for video input detection with this example block code. You will be able to direct your own Mars Rover easily by just showing signs through the camera input.

Introduction

A sign detector Mars Rover robot is a robot that can recognize and interpret certain signs or signals, such as hand gestures or verbal commands, given by a human. The robot uses sensors, cameras, and machine learning algorithms to detect and understand the sign, and then performs a corresponding action based on the signal detected.

These robots are often used in manufacturing, healthcare, and customer service industries to assist with tasks that require human-like interaction and decision making.

Code

Initializing the Functions:

Main Code

Logic

  1. Firstly, the code sets up the stage camera to look for signs and detects and recognizes the signs showed on the camera.
  2. Next, the code starts a loop where the stage camera continuously checks for the signs.
  3. Finally, if the robot sees certain signs (like ‘Go’, ‘Turn Left’, ‘Turn Right’, or ‘U Turn’), it moves in a certain direction (forward, backward, left, or backward) based on the respective signs.
  4. This can help the Mars Rover to manoeuvre through the terrain easily by just showing signs on the camera.

Output

Forward-Backward Motions:

Right-Left Motions:

Read More
Learn how to code logic for video input detection with this example block code. You will be able to direct your own Mecanum easily by just showing signs through the camera input.

Introduction

A sign detector Mecanum robot is a robot that can recognize and interpret certain signs or signals, such as hand gestures or verbal commands, given by a human. The robot uses sensors, cameras, and machine learning algorithms to detect and understand the sign, and then performs a corresponding action based on the signal detected.

These robots are often used in manufacturing, healthcare, and customer service industries to assist with tasks that require human-like interaction and decision making.

Code

Initialization:

Main Code

Logic

  1. Firstly, the code sets up the stage camera to look for signs and detects and recognizes the signs showed on the camera.
  2. Next, the code starts a loop where the stage camera continuously checks for the signs.
  3. Finally, if the robot sees certain signs (like ‘Go’, ‘Turn Left’, ‘Turn Right’, or ‘U Turn’), it moves in a certain direction (forward, backward, left, or backward) based on the respective signs.
  4. This can help the Mecanum to manoeuvre through the terrain easily by just showing signs on the camera.

Output

Forward Motion:

Right-Left Motions:

Read More
All articles loaded
No more articles to load