Table of Contents

Function Definition: tf.expand_dims(input, axis, name)

Parameters

NameTypeDescriptionExpected ValuesDefault Value
inputstringA tensorTensor
axisintInteger specifying the dimension index at which to expand the shape of input. Given an input of D dimensions, axis must be in range [-(D+1), D] (inclusive).Integer0
namestringOptional string. The name of the output Tensor.String

Description

The function returns a tensor with a length 1 axis inserted at the index axis.

Given a tensor input, this operation inserts a dimension of length 1 at the dimension index axis of input‘s shape. The dimension index follows Python indexing rules: It’s zero-based, a negative index is counted backward from the end.

This operation is useful to:

  • Add an outer “batch” dimension to a single element.
  • Align axes for broadcasting.
  • To add an inner vector length axis to a tensor of scalars.

Import Libraries

####################imports####################
#do not change

import cv2
import numpy as np
import tensorflow as tf

#do not change
####################imports####################

Example

Learn to demonstrate how to use ML Environment to make a model that identifies the hand gestures and makes the humanoid move accordingly.

Introduction

This project demonstrates how to use Machine Learning Environment to make a machinelearning model that identifies  hand gestures and makes the humanoid move accordingly.

We are going to use the Hand Classifier of the Machine Learning Environment.  The model works by analyzing your hand position with the help of 21 data points.

Hand Gesture Classifier Workflow

Follow the steps below:

  1. Open PictoBlox and create a new file.
  2. You can click on “Machine Learning Environment” to open it.
  3. Click on “Create New Project“.
  4. A window will open. Type in a project name of your choice and select the “Hand Gesture Classifier” extension. Click the “Create Project” button to open the Hand Pose Classifier window.
  5. You shall see the Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.

Class in Hand Gesture Classifier

There are 2 things that you have to provide in a class:

  1. Class Name: It’s the name to which the class will be referred as.
  2. Hand Pose Data: This data can either be taken from the webcam or by uploading from local storage.

Note: You can add more classes to the projects using the Add Class button.
Adding Data to Class

You can perform the following operations to manipulate the data into a class.

  1. Naming the Class: You can rename the class by clicking on the edit button.
  2. Adding Data to the Class: You can add the data using the Webcam or by Uploading the files from the local folder.
    1. Webcam:
Note: You must add at least 20 samples to each of your classes for your model to train. More samples will lead to better results.
Training the Model

After data is added, it’s fit to be used in model training. In order to do this, we have to train the model. By training the model, we extract meaningful information from the hand pose, and that in turn updates the weights. Once these weights are saved, we can use our model to make predictions on data previously unseen.

The accuracy of the model should increase over time. The x-axis of the graph shows the epochs, and the y-axis represents the accuracy at the corresponding epoch. Remember, the higher the reading in the accuracy graph, the better the model. The range of the accuracy is 0 to 1.

Testing the Model

To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

The model will return the probability of the input belonging to the classes.

Export in Python Coding

Click on the “Export Model” button on the top right of the Testing box, and PictoBlox will load your model into the Python Coding Environment if you have opened the ML Environment in Python Coding.

code

The following code appears in the Python Editor of the selected sprite.

####################imports####################
# Do not change

import numpy as np
import tensorflow as tf

sprite=Sprite('Tobi')
import time
quarky = Quarky()
import time

humanoid = Humanoid(7, 2, 6, 3, 8, 1)

# Do not change
####################imports####################

#Following are the model and video capture configurations
# Do not change

model=tf.keras.models.load_model(
    "num_model.h5",
    custom_objects=None,
    compile=True,
    options=None)
pose = Posenet()                                                    # Initializing Posenet
pose.enablebox()                                                    # Enabling video capture box
pose.video("on",0)                                                  # Taking video input
class_list=['forward','backward','left','right','stop']                  # List of all the classes
def runQuarky(predicted_class):
    if pose.ishanddetected():
      if predicted_class == "forward":
        humanoid.move("forward", 1000, 1)
        
      if predicted_class == "backward":
        humanoid.move("backward", 1000, 1)
        
      if predicted_class == "left":
        humanoid.move("left", 1000, 1)
        
      if predicted_class == "right":
        humanoid.move("right", 1000, 1)
        
      if predicted_class == "stop":
        humanoid.home()
    else:
      quarky.stoprobot()

# Do not change
###############################################

#This is the while loop block, computations happen here
# Do not change

while True:
  pose.analysehand()                                             # Using Posenet to analyse hand pose
  coordinate_xy=[]
    
    # for loop to iterate through 21 points of recognition
  for i in range(21):
    if(pose.gethandposition(1,i,0)!="NULL"  or pose.gethandposition(2,i,0)!="NULL"):
      coordinate_xy.append(int(240+float(pose.gethandposition(1,i,0))))
      coordinate_xy.append(int(180-float(pose.gethandposition(2,i,0))))
    else:
      coordinate_xy.append(0)
      coordinate_xy.append(0)
            
  coordinate_xy_tensor = tf.expand_dims(coordinate_xy, 0)        # Expanding the dimension of the coordinate list
  predict=model.predict(coordinate_xy_tensor)                    # Making an initial prediction using the model
  predict_index=np.argmax(predict[0], axis=0)                    # Generating index out of the prediction
  predicted_class=class_list[predict_index]                      # Tallying the index with class list
  print(predicted_class)
  runQuarky(predicted_class)

 

Note: You can edit the code to add custom code according to your requirement.

Logic

  1. If the identified class from the analyzed image is “forward,” the humanoid will move forward at a specific speed.
  2. If the identified class is “backward,” the humanoid will move backward.
  3. If the identified class is “left,” the humanoid will move left.
  4. If the identified class is “right,” the humanoid will move right.
  5. Otherwise, the humanoid will be in the home position.
if pose.ishanddetected():
      if predicted_class == "forward":
        humanoid.move("forward", 1000, 1)
        
      if predicted_class == "backward":
        humanoid.move("backward", 1000, 1)
        
      if predicted_class == "left":
        humanoid.move("left", 1000, 1)
        
      if predicted_class == "right":
        humanoid.move("right", 1000, 1)
        
      if predicted_class == "stop":
        humanoid.home()
    else:
      quarky.stoprobot()

Output

Read More
The example shows how to use a number classifier in PictoBlox to make the Iris Classifier Bot.

Code

####################imports####################

import numpy as np
import tensorflow as tf
sprite=Sprite('Tobi')
sprite1 = Sprite('Iris-versicolor')


sprite1.hide()

####################imports####################

model= tf.keras.models.load_model(
    "num_model.h5", 
    custom_objects=None, 
    compile=True, 
    options=None)
SepalLengthCm= int(sprite.input("Enter Sepal Length"))
SepalWidthCm=int(sprite.input("Enter Sepal Width"))
PetalLengthCm=int(sprite.input("Enter Petal Length"))
PetalWidthCm=int(sprite.input("Enter Petal Width"))

class_list = ['Iris-versicolor','Iris-virginica','Iris-setosa',]                                     # List of all the classes

inputValue=[SepalLengthCm,SepalWidthCm,PetalLengthCm,PetalWidthCm,]       # Input List
inputTensor = tf.expand_dims(inputValue, 0)                 # Input Tensor

predict = model.predict(inputTensor)                        # Making an initial prediction using the model
predict_index = np.argmax(predict[0], axis=0)               # Generating index out of the prediction
predicted_class = class_list[predict_index]                 # Tallying the index with class list
sprite.say(predicted_class)
sprite1.show()
sprite1.switchcostume(predicted_class)

Logic

The example demonstrates how to count nuts and bolts from an image of a stage. Following are the key steps happening:

  1. Creates a sprite object named “Tobi”. A sprite is typically a graphical element that can be animated or displayed on a screen.
  2. Creates another sprite by uploading an iris image from the computer; add another image of a different type of iris by clicking at costume place.
  3. Initialize a new sprite variable in the “Tobi” script and hide another sprite initially.
    sprite1 = Sprite('Iris-versicolor') 
    sprite1.hide()
  4. Write a code for taking input of sepal and petal length and width from the user and storing all of this in a new variable.
    SepalLengthCm= int(sprite.input("Enter Sepal Length")) 
    SepalWidthCm=int(sprite.input("Enter Sepal Width")) 
    PetalLengthCm=int(sprite.input("Enter Petal Length")) 
    PetalWidthCm=int(sprite.input("Enter Petal Width"))
  5. Write a predefined function (sprite.say()) by which ‘Tobi’ will say the name of the predicted class.
  6. Now show the hidden sprite by calling a predefined function.
    sprite1.show()
  7. Also write a predefined function by which images will switch according to predicted class.
    sprite1.switchcostume(predicted_class)

Final Result

Read More
Learn how to create a dataset and Machine Learning Model for an automated game from the user's input. See how to open the ML environment, upload data, label images, train the model, and export the Python script.

Introduction

In this example project, we are going to create a Machine Learning Model where fish automatically feed on randomly generated food.

Data Collection

  • Now, we are going to collect the data of  automated fish feast Game .
  • This data will contain the actions that we have taken to accomplish the game successfully.
  • We will use the data we collect here, to teach our device how to play the automated fish feast game , i.e. to perform machine learning.
  • The data that you collect will get saved in your device as a csv (comma separated values) file. If you open this file in Microsoft Excel, it will look as shown below:

Follow the steps below

  1. Open PictoBlox and create a new file.
  2. Select the coding environment as Python Coding Environment.
  3. Now write code in python.

Code for making dataset

  1. Creates a sprite object named “Fish”. A sprite is typically a graphical element that can be animated or displayed on a screen.
  2. Creates another sprite object named “Orange” and also upload backdrop of “Underwater2” .
  3. Click on the Fish.py file from the Project files section.
    sprite = Sprite('Fish')
  4. Similarly, declare new sprite on the Fish.py file.
    sprite1 = Sprite('Orange')
  5. Then we will import the time, random, os, math, TensorFlow as tf  and Pandas as pd modules using the import keyword for using delay in the program later.
    1. Time – For using delay in the program.
    2. Random – For using random position.
    3. Pandas as pd – For using Data Frame.
    4. Math– For using math functions in code.
    5. Os– For reading files from Program files.
      import random
      import time
      import tensorflow as tf
      import pandas as pd
      import os
      import math
  6. Now, make 3 variables curr_x, curr_y, ang_f, mov_f and score with initial values 4, 3, 50, and 0 respectively.
    1. curr_x – To store the initial x – position of fish.
    2. curr_y – To store the initial y – position of fish.
    3. ang_f – To store increment value in angle of fish on pressing specific key.
    4. mov_f – To store increment value in movement of fish on pressing specific key.
    5. angle – To store initial angle of fish.
    6. score – To store the score while playing the game.
      curr_x = 4
      curr_y = 3
      ang_f= 10
      mov_f= 5
      score = 0
      angle = 90
  7. Now set initial position and angle of fish.
    sprite.setx(curr_x)
    sprite.sety(curr_y)
    sprite.setdirection(DIRECTION=90)
  8. Now, make a function settarget() in which we are generating food at a random position. We pass one argument “t” in the function for generating target food in the greed position of the t gap.
    1. x and y – To generate the food at random position on stage.
    2. time.sleep – For giving the time delay.
    3. sprite1.set()– To Set the position of food at random position on stage.
      def settarget(t):
      x = random.randrange(-200, 200, t)
      y = random.randrange(-155, 155, t)
      time.sleep(0.1)
      sprite1.setx(x)
      sprite1.sety(y)
      return x, y 
  9. Now set the target (food). In this, fish are chasing the food, and target_x  and  target_y should be equal to the x and y positions of the food.
    target_x, target_y = settarget(40) 
  10. Now create a data frame of name “Chase_Data.csv” to collect the data for machine learning and if this name csv exist then directly add data in it.
    if(os.path.isfile('Chase_Data.csv')):
      data=pd.read_csv('Chase_Data.csv')
    else:
      data = pd.DataFrame({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "direction": angle, "Action": "RIGHT"}, index=[0])
    
  11. After that, we will use the while True loop to run the code indefinitely. Don’t forget to add a colon ‘:’ just after the loop to avoid errors.
    while True:
  12. Now write the script for moving the Fish in forward direction and change clockwise or anticlockwise direction by fix value with the help of a conditional statement.
    1. If the up arrow key is pressed then fish will move mov_f position in same direction.
    2. After pressing the up arrow key action taken should be stored in the Data frame with data.append command.
      if sprite.iskeypressed("up arrow"):
          data = data.append({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "direction": angle, "Action": "UP"}, ignore_index=True)
          sprite.move(mov_f)
  13. Repeat the process for the set direction in clockwise or anticlockwise.
    if sprite.iskeypressed("left arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "direction": angle, "Action": "LEFT"}, ignore_index=True)
        angle = angle - ang_f
        sprite.setdirection(DIRECTION=angle)
     if sprite.iskeypressed("right arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "direction": angle, "Action": "RIGHT"}, ignore_index=True)
        angle = angle + ang_f
        sprite.setdirection(DIRECTION=angle)
  14. Write the conditional statement for the storing data in csv file after few score.
    if(score>0 and score%2==0):
        data.to_csv('Chase_Data.csv',index=False)
  15. Again write the conditional statement for the score variable if the fish and food position difference is less then 20 then the score should be increased by one.
    if abs(curr_x-target_x)<20 and abs(curr_y-target_y)<20: 
        score = score + 1 
        sprite.say(("your score is: {}".format(score)))
  16. If the score is equal to or greater than 40 then data should be printed on Chase Data.csv file.
    if (score >= 40):
          print(data)
          data.to_csv('Chase Data.csv')
          break
        target_x, target_y = settargetLED()
  17. Now update the curr_x and curr_y variables by storing the current position of the fish and delaying movement by 0.02 seconds.
    curr_x=math.floor(sprite.x())
    curr_y=math.floor(sprite.y())
    time.sleep(0.02)
  18. The final code is as follows:
    sprite = Sprite('Fish')
    sprite1 = Sprite('Orange')
     
    import random
    import time
    import tensorflow as tf
    import pandas as pd
    import os
    import math
    		
    curr_x = -170
    curr_y = 138
    score=0
    ang_f=10
    mov_f=5
    angle=90
    sprite.say(("your score is: {}".format(score)))
    sprite.setx(curr_x)
    sprite.sety(curr_y)
    sprite.setdirection(DIRECTION=90)
    
    def settarget(t):
      x = random.randrange(-200, 200, t)
      y = random.randrange(-155, 155, t)
      time.sleep(0.1)
      sprite1.setx(x)
      sprite1.sety(y)
      return x, y
    
    target_x, target_y = settarget(40)
    
    if(os.path.isfile('Chase_Data.csv')):
      data=pd.read_csv('Chase_Data.csv')
    else:
      data = pd.DataFrame({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "direction": angle, "Action": "RIGHT"}, index=[0])
    
    while True:
      angle=sprite.direction()
      if sprite.iskeypressed("up arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "direction": angle, "Action": "UP"}, ignore_index=True)
        sprite.move(mov_f)
    
         
      if sprite.iskeypressed("left arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "direction": angle, "Action": "LEFT"}, ignore_index=True)
        angle = angle - ang_f
        sprite.setdirection(DIRECTION=angle)
        
         
      if sprite.iskeypressed("right arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "direction": angle, "Action": "RIGHT"}, ignore_index=True)
        angle = angle + ang_f
        sprite.setdirection(DIRECTION=angle)
        
      if(score>0 and score%2==0):
        data.to_csv('Chase_Data.csv',index=False)
      
      if abs(curr_x-target_x)<20 and abs(curr_y-target_y)<20:
        score = score + 1
        sprite.say(("your score is: {}".format(score)))
        if (score >= 40):
          data.to_csv('Chase_Data.csv',index=False)
          break
        target_x, target_y = settarget(40)
      curr_x=math.floor(sprite.x())
      curr_y=math.floor(sprite.y())
      time.sleep(0.02)
     
    
  19. Press the Run button and play fish feast game to collect data.
  20. Store this dataset on your local computer.

    Numbers(C/R) in Machine Learning Environment

    Datasets on the internet are hardly ever fit to directly train on. Programmers often have to take care of unnecessary columns, text data, target columns, correlations, etc. Thankfully, PictoBlox’s ML Environment is packed with features to help us pre-process the data as per our liking.

    Let’s create the ML model.

    Opening Image Classifier Workflow

    Alert: The Machine Learning Environment for model creation is available in the only desktop version of PictoBlox for Windows, macOS, or Linux. It is not available in Web, Android, and iOS versions.

    Follow the steps below:

    1. Open PictoBlox and create a new file.
    2. Select the coding environment as Block Coding Environment.
    3. Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
    4. You’ll be greeted with the following screen.
      Click on “Create New Project“.
    5. You shall see the Numbers C/R workflow with an option to either “Upload Dataset” or “Create Dataset”.

      Uploading/Creating Dataset

      Datasets can either be uploaded or created on the ML Environment. Lets see how it is done.

      Uploading a dataset
      1. To upload a dataset, click on the Upload Dataset button and the Choose CSV from your files button.
        Note: An uploaded dataset must be a “.csv” file.
      2. Once uploaded the first 50 rows of the uploaded CSV document will show up in the window.

      3. If you look at the output column, all the values are currently “0”. Hence, first we need to create an output column.
        1. In the Dataset table, click on the tick near Select All to de-select all the columns.
        2. click on the tick of Action column to select it. We will make this column the output.
        3. The output column must always be numerical. Hence click on the button Text to Number to convert the data within this column to numerical type.
        4. Now select it again and press the Set as Output button to set this column as Output.
        5. There is also many which is not useful in training our model and needs to be disable. So select it and click the Disable button in the Selected columns section.

          Creating a Dataset
          1. To create a dataset, click on the Create Dataset button.
          2. Select the number of rows and columns that are to be added and click on the Create button. More rows and columns can be added as and when needed.

          Notes:

          1. Each column represents a feature. These are the values used by the model to train itself.
          2. The “Output” column contains the target values. These are the values that we expect the model to return when features are passed.
          3. The window only shows the first 50 rows of the dataset.
          4. Un-check the “Select All” checkbox to un-select all the columns.

          Training the Model

          After data is pre-processed and optimized, it’s fit to be used in model training. To train the model, simply click the “Train Model” button found in the “Training” panel.

          By training the model, meaningful information is extracted from the numbers, and that in turn updates the weights. Once these weights are saved, the model can be used to make predictions on data previously unseen.

          The model’s function is to use the input data and predict the output. The target column must always contain numbers.

          However, before training the model, there are a few hyperparameters that need to be understood. Click on the “Advanced” tab to view them.

          There are three hyperparameters that can be altered in the Numbers(C/R) Extension:

          1. Epochs– The total number of times the data will be fed through the training model. Therefore, in 10 epochs, the dataset will be fed through the training model 10 times. Increasing the number of epochs can often lead to better performance.
          2. Batch Size– The size of the set of samples that will be used in one step. For example, if there are 160 data samples in the dataset, and the batch size is set to 16, each epoch will be completed in 160/16=10 steps. This hyperparameter rarely needs any altering.
          3. Learning Rate– It dictates the speed at which the model updates the weights after iterating through a step. Even small changes in this parameter can have a huge impact on the model performance. The usual range lies between 0.001 and 0.0001.
          Note: Hover the mouse pointer over the question mark next to the hyperparameters to see their description.

          It’s a good idea to train a numeric classification model for a high number of epochs. The model can be trained in both JavaScript and Python. In order to choose between the two, click on the switch on top of the Training panel.

          Alert: Dependencies must be downloaded to train the model in Python, JavaScript will be chosen by default.

          The accuracy of the model should increase over time. The x-axis of the graph shows the epochs, and the y-axis represents the accuracy at the corresponding epoch.

          A window will open. Type in a project name of your choice and select the “Numbers(C/R)” extension. Click the “Create Project” button to open the Numbers(C/R) window.

          Testing the Model

          To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

          The model will return the probability of the input belonging to the classes.

      Export in Python Coding

      Click on the “PictoBlox” button, and PictoBlox will load your model into the Python Coding Environment if you have opened the ML Environment in Python Coding.

    Code

    1. Creates a sprite object named “Fish”. A sprite is typically a graphical element that can be animated or displayed on a screen.
    2. Creates another sprite object named “Orange” and also upload backdrop of “Underwater2” .
    3. Click on the Fish.py file from the Project files section.
      sprite = Sprite('Fish')
    4. Similarly, declare new sprite on the Fish.py file.
      sprite1 = Sprite('Orange')
    5. Then we will import the time, random, os, math, TensorFlow as tf  and Pandas as pd modules using the import keyword for using delay in the program later.
      1. Time – For using delay in the program.
      2. Random – For using random position.
      3. Pandas as pd – For using Data Frame.
      4. Math– For using math functions in code.
      5. Os– For reading files from Program files.
        import random
        import time
        import tensorflow as tf
        import pandas as pd
        import os
        import math
    6. Now, make 3 variables curr_x, curr_y, ang_f, mov_f,angle and score with initial values -170, 138, 30, 15, 90 and 0 respectively.
      1. curr_x – To store the initial x – position of fish.
      2. curr_y – To store the initial y – position of fish.
      3. ang_f – To store increment value in angle of fish on pressing specific key.
      4. mov_f – To store increment value in movement of fish on pressing specific key.
      5. angle – To store initial angle of fish.
      6. score – To store the score while playing the game.
        curr_x = -170
        curr_y = 138
        ang_f= 30
        mov_f= 15
        score = 0
        angle = 90
    7. Now set initial position and angle of fish.
      sprite.setx(curr_x)
      sprite.sety(curr_y)
      sprite.setdirection(DIRECTION=90)
    8. Now, make a function settarget() in which we are generating food at a random position. We pass one argument “t” in the function for generating target food in the greed position of the t gap.
      1. x and y – To generate the food at random position on stage.
      2. time.sleep – For giving the time delay.
      3. sprite1.set()– To Set the position of food at random position on stage.
        def settarget(t):
        x = random.randrange(-200, 200, t)
        y = random.randrange(-155, 155, t)
        time.sleep(0.1)
        sprite1.setx(x)
        sprite1.sety(y)
        return x, y 
    9. Now set the position of food. In this, fish are chasing the food, and target_x  and  target_y should be equal to the x and y positions of the food.
      target_x, target_y = settarget(40) 
    10. Now, make a function runprediction() in which we are predicting class (Left, Up, right) by taking argument from user . We pass three arguments “diff_x”, “diff-y”, “ang” in the function.
      1. inputvalue – To store input parameters of function in array.
      2. model.predict() – For predicting output from trained model.
      3. np.argmax(,)– To find the most probable prediction output.
        def runprediction(diff_x, diff_y, ang):
          inputValue=[diff_x, diff_y, ang]
          #Input Tensor
          inputTensor = tf.expand_dims(inputValue, 0)
          #Predict
          predict = model.predict(inputTensor)
          predict_index = np.argmax(predict[0], axis=0)
          #Output
          predicted_class = class_list[predict_index]
          return predicted_class
    11. After that, we will use the while True loop to run the code indefinitely. Don’t forget to add a colon ‘:’ just after the loop to avoid errors.
      while True:
    12. In while loop find angle of sprite and call runprediction function by passing arguments in it.
       angle=sprite.direction()
        move = runprediction(curr_x- target_x, curr_y-target_y, angle)
    13. Now write the script for moving the Fish in forward direction and change clockwise or anticlockwise direction by fix value with the help of a conditional statement.
      1. If the predicted value is “UP” then fish will move mov_f position in same direction.
      2. If the predicted value is “LEFT” then fish will change direction by some constant value in anticlockwise direction.
      3. If the predicted value is “RIGHT” then fish will change direction by some constant value in clockwise direction.
        if move == "UP":
            sprite.move(mov_f)
            curr_x=sprite.x()
            curr_y=sprite.y()
          if move == "LEFT":
            angle = angle - ang_f
            sprite.setdirection(DIRECTION=angle)
          if move == "RIGHT":
            angle = angle + ang_f
            sprite.setdirection(DIRECTION=angle)
    14.  Again write the conditional statement for the score variable if the fish and food position difference is less then 20 then the score should be increased by one also set new position of target.
      if abs(curr_x-target_x)<20 and abs(curr_y-target_y)<20: 
          score = score + 1 
          sprite.say(("your score is: {}".format(score)))
      target_x, target_y = settarget()
    15.  Now add delay function for delaying movement by 0.02 seconds.
      time.sleep(0.02)
    16. The final code is as follows
      sprite = Sprite('Fish')
      sprite1 = Sprite('Orange')
       
      import random
      import time
      import numpy as np
      import tensorflow as tf
      import pandas as pd
      import os
      import math
      
      #Load Number Model
      model= tf.keras.models.load_model(
      		"num_model.h5", 
      		custom_objects=None, 
      		compile=True, 
      		options=None)
      		
      #List of classes
      class_list = ['UP','LEFT','RIGHT',]  
      		
      curr_x = -170
      curr_y = 138
      score=0
      ang_f=30
      mov_f=15
      angle=90
      
      sprite.say(("your score is: {}".format(score)))
      sprite.setx(-170)
      sprite.sety(138)
      sprite.setdirection(DIRECTION=90)
      
      def settarget():
        x = random.randrange(-200, 200, 1)
        y = random.randrange(-155, 155, 1)
        time.sleep(0.1)
        sprite1.setx(x)
        sprite1.sety(y)
        return x, y
      
      target_x, target_y = settarget()
      
      def runprediction(diff_x, diff_y, ang):
        inputValue=[diff_x, diff_y, ang]
        #Input Tensor
        inputTensor = tf.expand_dims(inputValue, 0)
        #Predict
        predict = model.predict(inputTensor)
        predict_index = np.argmax(predict[0], axis=0)
        #Output
        predicted_class = class_list[predict_index]
        return predicted_class
      
      while True:
        angle=sprite.direction()
        move = runprediction(curr_x- target_x, curr_y-target_y, angle)
      
        if move == "UP":
          sprite.move(mov_f)
          curr_x=sprite.x()
          curr_y=sprite.y()
      
        if move == "LEFT":
          angle = angle - ang_f
          sprite.setdirection(DIRECTION=angle)
      
        if move == "RIGHT":
          angle = angle + ang_f
          sprite.setdirection(DIRECTION=angle)
       
      
        if abs(curr_x-target_x)<20 and abs(curr_y-target_y)<20:
          score = score + 1
          sprite.say(("your score is: {}".format(score)))
          target_x, target_y = settarget()
      
        time.sleep(0.2) 
      

      Final Result

      Conclusion

      Creating a Machine Learning Model of automated fish feast game can be both complex and time-consuming. Through the steps demonstrated in this project, you can create your own Machine Learning Model of automated game. Once trained, you can export the model into the Python Coding Environment, where you can tweak it further to give you the desired output. Try creating a Machine Learning Model of your own today and explore the possibilities of Number Classifier in PictoBlox!

Read More
Learn how to create a dataset and Machine Learning Model for an automated shark attack game from the user's input. See how to open the ML environment, upload data, label images, train the model, and export the Python script.

Introduction

In this example project, we are going to create a Machine Learning Model where shark run by the user and fish automatically feed on randomly generated food while escaping from sharks.

Data Collection

  • Now, we are going to collect the data of “Shark Attack: Hungry for Fish” game .
  • This data will contain the actions that we have taken to accomplish the game successfully.
  • We will use the data we collect here, to teach our device how to play the “Shark Attack: Hungry for Fish” game , i.e. to perform machine learning.
  • The data that you collect will get saved in your device as a csv (comma separated values) file. If you open this file in Microsoft Excel, it will look as shown below:

Follow the steps below

  1. Open PictoBlox and create a new file.
  2. Select the coding environment as Python Coding Environment.
  3. Now write code in python.

Code for making dataset

  1. Creates a sprite object named “Fish”. A sprite is typically a graphical element that can be animated or displayed on a screen.
  2. Creates three sprites object named “Orange” , “Shark2” and “Button3” and also upload backdrop of “Underwater2” .
  3. Click on the Fish.py file from the Project files section.
    sprite = Sprite('Fish')
  4. Similarly, declare new sprites on the Fish.py file.
    sprite1 = Sprite('Orange')
    sprite2 = Sprite('Shark 2')
    sprite3 = Sprite('Button3')
  5. Then we will import the time, random, os, math, TensorFlow as tf  and Pandas as pd modules using the import keyword for using delay in the program later.
    1. Time – For using delay in the program.
    2. Random – For using random position.
    3. Pandas as pd – For using Data Frame.
    4. Math– For using math functions in code.
    5. Os– For reading files from Program files.
      import random
      import time
      import tensorflow as tf
      import pandas as pd
      import os
      import math
  6. Now, make 3 variables curr_x, curr_y, shark_x, shark_y, score, chance, fish_d, fish_m, shark_m, angle_f and angle_s with initial values 25, 108, -177, 116, 0, 5, 20, 35, 25, 90 and 90 respectively.
    1.  curr_x – To store the initial x – position of fish.
    2. curr_y – To store the initial y – position of fish.
    3. shark_x – To store the initial x – position of shark.
    4. shark_y – To store the initial y – position of shark.
    5. score – To store the score while playing the game.
    6. chance– To store the chance of fish while playing the game.
    7. fish_d– To store increment value in direction of fish on pressing specific key.
    8. fish_m – To store increment value in movement of fish on pressing specific key.
    9. shark_m – To store increment value in movement of shark on pressing specific key.
    10. shark_d – To store increment value in direction of shark on pressing specific key.
    11. angle_f – To store increment value in angle of fish on pressing specific key.
    12. angle_s – To store increment value in angle of shark on pressing specific key.
      curr_x = 25 
      curr_y = 108 
      shark_x=-177 
      shark_y=116 
      score=0 
      chance=5 
      fish_d=20 
      fish_m=25 
      shark_m=4 
      angle_f=90 
      angle_s=90
  7. Now set initial position and angle of fish and shark both.
    sprite.setx(curr_x)
    sprite.sety(curr_y)
    sprite2.setx(shark_x)
    sprite2.sety(shark_y)
    sprite.setdirection(DIRECTION=angle_f)
    sprite2.setdirection(DIRECTION=angle_s)
  8. Now, make a function settarget1() in which we are generating food at a random position. We pass one argument “m” in the function for generating target food in the greed position of the t gap.
    1. x and y – To generate the fish at random position on stage.
    2. x1 and y1 – To generate the food at random position on stage.
    3. x2 and y2 – To generate the shark at random position on stage.
    4. time.sleep – For giving the time delay.
    5. sprite.set()– To Set the position of fish at random position on stage.
    6. sprite1.set()– To Set the position of food at random position on stage.
    7. sprite2.set()– To Set the position of shark at random position on stage.
      def settarget(t):
        x = random.randrange(-200, 200, t)
        y = random.randrange(-155, 155, t)
        x1 = random.randrange(-200, 200, t)
        y1 = random.randrange(-155, 155, t)
        x2 = random.randrange(-200, 200, t)
        y2 = random.randrange(-155, 155, t)
        time.sleep(0.1)
        sprite1.setx(x1)
        sprite1.sety(y1)
        sprite.setx(x)
        sprite.sety(y)
        sprite2.setx(x2)
        sprite2.sety(y2)
        return x, y, x1, y1, x2, y2
  9. Now, make a function settarget1() in which we are generating food at a random position. We pass one argument “m” in the function for generating target food in the greed position of the t gap.
    1. x and y – To generate the food at random position on stage.
    2. time.sleep – For giving the time delay.
    3. sprite1.set()– To Set the position of food at random position on stage.
      def settarget1(m):
      x = random.randrange(-200, 200, m)
      y = random.randrange(-155, 155, m)
      time.sleep(0.1)
      sprite1.setx(x)
      sprite1.sety(y)
      return x, y 
  10. Now set the target (food). In this, fish are chasing the food, and target_x  and  target_y should be equal to the x and y positions of the food.
    target_x, target_y = settarget(40) 
  11. Now create a data frame of name “Chase_Data.csv” to collect the data for machine learning and if this name csv exist then directly add data in it.
    target_x, target_y = settarget1(40)
    if(os.path.isfile('Chase_Data.csv')):
      data=pd.read_csv('Chase_Data.csv')
    else:
      data = pd.DataFrame({"curr_X": curr_x, "curr_Y": curr_y,"shark_X": shark_x, "shark_Y": shark_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "diff_x1":shark_x-curr_x, "diff_y1":shark_y-curr_y, "direction_f": angle_f, "direction_s": angle_s, "Action": "RIGHT"}, index=[0])
  12. After that, we will use the while True loop to run the code indefinitely. Don’t forget to add a colon ‘:’ just after the loop to avoid errors.
    while True:
  13. In a while loop, write code by which sharks follow fish by ‘shark_m’ steps.
    sprite2.spriteRequest.requestCommand("motion_pointtowards", {"TOWARDS": "Fish"})
     sprite2.move(shark_m)
  14. Find the direction of shark and fish using the Python pictoblox function and take the floor value of angle.
    angle_f=sprite.direction()
    angle_s=sprite2.direction()
    anglef=math.floor(angle_f)
    angles=math.floor(angle_s)
  15. Now write the script for moving the Fish in forward direction and change clockwise or anticlockwise direction by fix value with the help of a conditional statement.
    1. If the up arrow key is pressed then fish will move fish_m position in same direction.
    2. After pressing the up arrow key action taken should be stored in the Data frame with data.append command.
      if sprite.iskeypressed("up arrow"):
          data = data.append({"curr_X": curr_x, "curr_Y": curr_y,"shark_X": shark_x, "shark_Y": shark_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "diff_x1":shark_x-curr_x, "diff_y1":shark_y-curr_y, "direction_f": anglef, "direction_s": angles, "Action": "UP"}, ignore_index=True)
          sprite.move(fish_m)
  16. Repeat the process for the set direction in clockwise or anticlockwise.
    if sprite.iskeypressed("left arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y,"shark_X": shark_x, "shark_Y": shark_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "diff_x1":shark_x-curr_x, "diff_y1":shark_y-curr_y, "direction_f": anglef, "direction_s": angles, "Action": "LEFT"}, ignore_index=True)
        angle = anglef - fish_d
        sprite.setdirection(DIRECTION=angle)
    if sprite.iskeypressed("right arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y,"shark_X": shark_x, "shark_Y": shark_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "diff_x1":shark_x-curr_x, "diff_y1":shark_y-curr_y, "direction_f": anglef, "direction_s": angles, "Action": "RIGHT"}, ignore_index=True)
        angle = anglef + fish_d
        sprite.setdirection(DIRECTION=angle)
  17. Write the conditional statement for the storing data in csv file after few score.
    if(score>0 and score%2==0):
        data.to_csv('Chase_Data.csv',index=False)
  18. Write the conditional statement for the chance variable. If the fish and shark position difference is less than 20, then the chance should be decreased by one.
     if abs(shark_x-curr_x)<20 and abs(shark_y-curr_y)<20:
        chance= chance-1
  19. Update the position of all three sprites, and if chance becomes 0, then data should be printed on Chase Data. csv file, and the positions of all three sprites change randomly by the functions settarget() and update chance value.
      if abs(shark_x-curr_x)<20 and abs(shark_y-curr_y)<20:
        chance= chance-1
        curr_x, curr_y, target_x, target_y, shark_x, shark_y = settarget(40)
        sprite3.say(("score: ",score ," and chance:  ",chance,""))
        if (chance == 0):
          data.to_csv('Chase_Data.csv',index=False)
          curr_x, curr_y, target_x, target_y, shark_x, shark_y = settarget(40)
          chance=5
  20. Again write the conditional statement for the score variable if the fish and food position difference is less then 20 then the score should be increased by one.
    if abs(curr_x-target_x)<20 and abs(curr_y-target_y)<20: 
        score = score + 1 
        sprite.say(("your score is: {}".format(score)))
  21. If the score is equal to or greater than 50 then data should be printed on Chase Data.csv file and food positions change randomly by the function settarget1().
    if (score >= 40):
          print(data)
          data.to_csv('Chase Data.csv')
          break
     target_x, target_y = settarget1()
  22. Now update the curr_x, curr_y, shark_x and shark_y variables by storing the current position of the fish and shark and delaying movement by 0.02 seconds.
      curr_x=math.floor(sprite.x())
      curr_y=math.floor(sprite.y())
      shark_x=math.floor(sprite2.x())
      shark_y=math.floor(sprite2.y())
      time.sleep(0.02)
  23. The final code is as follows:
    sprite = Sprite('Fish')
    sprite1 = Sprite('Orange')
    sprite2 = Sprite('Shark 2')
    sprite3 = Sprite('Button3')
    import random
    import time
    import numpy as np
    import tensorflow as tf
    import pandas as pd
    import os
    import math
    		
    curr_x = 25
    curr_y = 108
    shark_x=-177
    shark_y=116
    score=0
    chance=5
    fish_d=20
    fish_m=25
    shark_m=4
    angle_f=90
    angle_s=90
    sprite3.say(("score: ",score ," and chance:  ",chance,""))
    sprite.setx(curr_x)
    sprite.sety(curr_y)
    sprite2.setx(shark_x)
    sprite2.sety(shark_y)
    sprite.setdirection(DIRECTION=angle_f)
    sprite2.setdirection(DIRECTION=angle_s)
    def settarget(t):
      x = random.randrange(-200, 200, t)
      y = random.randrange(-155, 155, t)
      x1 = random.randrange(-200, 200, t)
      y1 = random.randrange(-155, 155, t)
      x2 = random.randrange(-200, 200, t)
      y2 = random.randrange(-155, 155, t)
      time.sleep(0.1)
      sprite1.setx(x1)
      sprite1.sety(y1)
      sprite.setx(x)
      sprite.sety(y)
      sprite2.setx(x2)
      sprite2.sety(y2)
      return x, y, x1, y1, x2, y2
    def settarget1(m):
      x = random.randrange(-200, 200, m)
      y = random.randrange(-155, 155, m)
      time.sleep(0.1)
      sprite1.setx(x)
      sprite1.sety(y)
      return x, y
    target_x, target_y = settarget1(40)
    if(os.path.isfile('Chase_Data.csv')):
      data=pd.read_csv('Chase_Data.csv')
    else:
      data = pd.DataFrame({"curr_X": curr_x, "curr_Y": curr_y,"shark_X": shark_x, "shark_Y": shark_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "diff_x1":shark_x-curr_x, "diff_y1":shark_y-curr_y, "direction_f": angle_f, "direction_s": angle_s, "Action": "RIGHT"}, index=[0])
    while True:
      # sprite2.pointto()
      sprite2.spriteRequest.requestCommand("motion_pointtowards", {"TOWARDS": "Fish"})
      sprite2.move(shark_m)
      angle_f=sprite.direction()
      angle_s=sprite2.direction()
      anglef=math.floor(angle_f)
      angles=math.floor(angle_s)
      if sprite.iskeypressed("up arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y,"shark_X": shark_x, "shark_Y": shark_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "diff_x1":shark_x-curr_x, "diff_y1":shark_y-curr_y, "direction_f": anglef, "direction_s": angles, "Action": "UP"}, ignore_index=True)
        sprite.move(fish_m)
      if sprite.iskeypressed("left arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y,"shark_X": shark_x, "shark_Y": shark_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "diff_x1":shark_x-curr_x, "diff_y1":shark_y-curr_y, "direction_f": anglef, "direction_s": angles, "Action": "LEFT"}, ignore_index=True)
        angle = anglef - fish_d
        sprite.setdirection(DIRECTION=angle)
        
         
      if sprite.iskeypressed("right arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y,"shark_X": shark_x, "shark_Y": shark_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "diff_x1":shark_x-curr_x, "diff_y1":shark_y-curr_y, "direction_f": anglef, "direction_s": angles, "Action": "RIGHT"}, ignore_index=True)
        angle = anglef + fish_d
        sprite.setdirection(DIRECTION=angle)
        
      if(score>0 and score%2==0):
        data.to_csv('Chase_Data.csv',index=False)
      
      if abs(shark_x-curr_x)<20 and abs(shark_y-curr_y)<20:
        chance= chance-1
        curr_x, curr_y, target_x, target_y, shark_x, shark_y = settarget(40)
        sprite3.say(("score: ",score ," and chance:  ",chance,""))
        if (chance == 0):
          data.to_csv('Chase_Data.csv',index=False)
          curr_x, curr_y, target_x, target_y, shark_x, shark_y = settarget(40)
          chance=5
      if abs(curr_x-target_x)<20 and abs(curr_y-target_y)<20:
        score = score + 1
        sprite3.say(("score: ",score ," and chance:  ",chance,""))
        if (score >= 50):
          data.to_csv('Chase_Data.csv',index=False)
          break
        target_x, target_y = settarget1(40)
      curr_x=math.floor(sprite.x())
      curr_y=math.floor(sprite.y())
      shark_x=math.floor(sprite2.x())
      shark_y=math.floor(sprite2.y())
      time.sleep(0.02)
    
  24. Press the Run button and play fish feast game to collect data.
  25. Store this dataset on your local computer.

    Numbers(C/R) in Machine Learning Environment

    Datasets on the internet are hardly ever fit to directly train on. Programmers often have to take care of unnecessary columns, text data, target columns, correlations, etc. Thankfully, PictoBlox’s ML Environment is packed with features to help us pre-process the data as per our liking.

    Let’s create the ML model.

    Opening Image Classifier Workflow

    Alert: The Machine Learning Environment for model creation is available in the only desktop version of PictoBlox for Windows, macOS, or Linux. It is not available in Web, Android, and iOS versions.

    Follow the steps below:

    1. Open PictoBlox and create a new file.
    2. Select the coding environment as Block Coding Environment.
    3. Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
    4. You’ll be greeted with the following screen.
      Click on “Create New Project“.
    5. You shall see the Numbers C/R workflow with an option to either “Upload Dataset” or “Create Dataset”.

      Uploading/Creating Dataset

      Datasets can either be uploaded or created on the ML Environment. Lets see how it is done.

      Uploading a dataset
      1. To upload a dataset, click on the Upload Dataset button and the Choose CSV from your files button.
        Note: An uploaded dataset must be a “.csv” file.
      2. Once uploaded the first 50 rows of the uploaded CSV document will show up in the window.

      3. If you look at the output column, all the values are currently “0”. Hence, first we need to create an output column.
        1. In the Dataset table, click on the tick near Select All to de-select all the columns.
        2. click on the tick of Action column to select it. We will make this column the output.
        3. The output column must always be numerical. Hence click on the button Text to Number to convert the data within this column to numerical type.
        4. Now select it again and press the Set as Output button to set this column as Output.
        5. There is also many which is not useful in training our model and needs to be disable. So select it and click the Disable button in the Selected columns section.

          Creating a Dataset
          1. To create a dataset, click on the Create Dataset button.
          2. Select the number of rows and columns that are to be added and click on the Create button. More rows and columns can be added as and when needed.

          Notes:

          1. Each column represents a feature. These are the values used by the model to train itself.
          2. The “Output” column contains the target values. These are the values that we expect the model to return when features are passed.
          3. The window only shows the first 50 rows of the dataset.
          4. Un-check the “Select All” checkbox to un-select all the columns.

          Training the Model

          After data is pre-processed and optimized, it’s fit to be used in model training. To train the model, simply click the “Train Model” button found in the “Training” panel.

          By training the model, meaningful information is extracted from the numbers, and that in turn updates the weights. Once these weights are saved, the model can be used to make predictions on data previously unseen.

          The model’s function is to use the input data and predict the output. The target column must always contain numbers.

          However, before training the model, there are a few hyperparameters that need to be understood. Click on the “Advanced” tab to view them.

          There are three hyperparameters that can be altered in the Numbers(C/R) Extension:

          1. Epochs– The total number of times the data will be fed through the training model. Therefore, in 10 epochs, the dataset will be fed through the training model 10 times. Increasing the number of epochs can often lead to better performance.
          2. Batch Size– The size of the set of samples that will be used in one step. For example, if there are 160 data samples in the dataset, and the batch size is set to 16, each epoch will be completed in 160/16=10 steps. This hyperparameter rarely needs any altering.
          3. Learning Rate– It dictates the speed at which the model updates the weights after iterating through a step. Even small changes in this parameter can have a huge impact on the model performance. The usual range lies between 0.001 and 0.0001.
          Note: Hover the mouse pointer over the question mark next to the hyperparameters to see their description.

          It’s a good idea to train a numeric classification model for a high number of epochs. The model can be trained in both JavaScript and Python. In order to choose between the two, click on the switch on top of the Training panel.

          Alert: Dependencies must be downloaded to train the model in Python, JavaScript will be chosen by default.

          The accuracy of the model should increase over time. The x-axis of the graph shows the epochs, and the y-axis represents the accuracy at the corresponding epoch.

          A window will open. Type in a project name of your choice and select the “Numbers(C/R)” extension. Click the “Create Project” button to open the Numbers(C/R) window.

          Testing the Model

          To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

          The model will return the probability of the input belonging to the classes.

      Export in Python Coding

      Click on the “PictoBlox” button, and PictoBlox will load your model into the Python Coding Environment if you have opened the ML Environment in Python Coding.

    Code

    1. Creates a sprite object named “Fish”. A sprite is typically a graphical element that can be animated or displayed on a screen.
    2. Creates three sprites object named “Orange” , “Shark2” and “Button3” and also upload backdrop of “Underwater2” .
    3. Click on the Fish.py file from the Project files section.
      sprite = Sprite('Fish')
    4. Similarly, declare new sprites on the Fish.py file.
      sprite1 = Sprite('Orange')
      sprite2 = Sprite('Shark 2')
      sprite3 = Sprite('Button3')
    5. Then we will import the time, random, os, math, TensorFlow as tf  and Pandas as pd modules using the import keyword for using delay in the program later.
      1. Time – For using delay in the program.
      2. Random – For using random position.
      3. Pandas as pd – For using Data Frame.
      4. Math– For using math functions in code.
      5. Os– For reading files from Program files.
        import random
        import time
        import tensorflow as tf
        import pandas as pd
        import os
        import math
    6. Now, make 3 variables curr_x, curr_y, ang_f, mov_f and score with initial values 4, 3, 50, and 0 respectively.
      1. curr_x – To store the initial x – position of fish.
      2. curr_y – To store the initial y – position of fish.
      3. shark_x – To store the initial x – position of shark.
      4. shark_y – To store the initial y – position of shark.
      5. score – To store the score while playing the game.
      6. chance– To store the chance of fish while playing the game.
      7. fish_d– To store increment value in direction of fish on pressing specific key.
      8. fish_m – To store increment value in movement of fish on pressing specific key.
      9. shark_m – To store increment value in movement of shark on pressing specific key.
      10. shark_d – To store increment value in direction of shark on pressing specific key.
      11. angle_f – To store increment value in angle of fish on pressing specific key.
      12. angle_s – To store increment value in angle of shark on pressing specific key.
        curr_x = 25 
        curr_y = 108 
        shark_x=-177 
        shark_y=116 
        score=0 
        chance=5 
        fish_d=20 
        fish_m=35 
        shark_m=25 
        shark_d= 20 
        angle_f=90 
        angle_s=90
    7. Now set initial position and angle of fish and shark both.
      sprite.setx(curr_x) 
      sprite.sety(curr_y) 
      sprite2.setx(shark_x) 
      sprite2.sety(shark_y) 
      sprite.setdirection(DIRECTION=angle_f) sprite2.setdirection(DIRECTION=angle_s)
    8. Now, make a function settarget1() in which we are generating food at a random position. We pass one argument “t” in the function for generating target food in the greed position of the t gap.
      1. x and y – To generate the fish at random position on stage.
      2. x1 and y1 – To generate the food at random position on stage.
      3. x2 and y2 – To generate the shark at random position on stage.
      4. time.sleep – For giving the time delay.
      5. sprite.set()– To Set the position of fish at random position on stage.
      6. sprite1.set()– To Set the position of food at random position on stage.
      7. sprite2.set()– To Set the position of shark at random position on stage.
        def settarget1(t):
          x = random.randrange(-200, 200, t)
          y = random.randrange(-155, 155, t)
          x1 = random.randrange(-200, 200, t)
          y1 = random.randrange(-155, 155, t)
          x2 = random.randrange(-200, 200, t)
          y2 = random.randrange(-155, 155, t)
          time.sleep(0.1)
          sprite1.setx(x1)
          sprite1.sety(y1)
          sprite.setx(x)
          sprite.sety(y)
          sprite2.setx(x2)
          sprite2.sety(y2)
          return x, y, x1, y1, x2, y2
    9. Now, make a function settarget() in which we are generating food at a random position. We pass one argument “m” in the function for generating target food in the greed position of the t gap.
      1. x and y – To generate the food at random position on stage.
      2. time.sleep – For giving the time delay.
      3. sprite1.set()– To Set the position of food at random position on stage.
        def settarget(m):
        x = random.randrange(-200, 200, m)
        y = random.randrange(-155, 155, m)
        time.sleep(0.1)
        sprite1.setx(x)
        sprite1.sety(y)
        return x, y 
    10. Now set the target (food). In this, fish are chasing the food, and target_x  and  target_y should be equal to the x and y positions of the food.
      target_x, target_y = settarget(40)
    11. Now, make a function runprediction() in which we are predicting class (Left, Up, right) by taking argument from user . We pass three arguments “diff_x”, “diff-y”, “diff_x1”, “diff_y1”, “ang1”, “ang2” in the function.
      1. inputvalue – To store input parameters of function in array.
      2. model.predict() – For predicting output from trained model.
      3. np.argmax(,)– To find the most probable prediction output.
        def runprediction(diff_x, diff_y, diff_x1, diff_y1, ang1, ang2):
          inputValue=[diff_x, diff_y, diff_x1, diff_y1, ang1, ang2]
          #Input Tensor
          inputTensor = tf.expand_dims(inputValue, 0)
          #Predict
          predict = model.predict(inputTensor)
          predict_index = np.argmax(predict[0], axis=0)
          #Output
          predicted_class = class_list[predict_index]
          return predicted_class
    12. After that, we will use the while True loop to run the code indefinitely. Don’t forget to add a colon ‘:’ just after the loop to avoid errors.
      While True :
    13. Now write the script for moving the Shark in forward direction and change clockwise or anticlockwise direction by fix value with the help of a conditional statement.
      1. If the up arrow key is pressed then fish will move fish_m position in same direction.
      2. After pressing the up arrow key, the shark_x and shark_y variables update by storing the current position of the shark.
        if sprite.iskeypressed("up arrow"):
            sprite2.move(shark_m)
            shark_x=sprite2.x()
            shark_y=sprite2.y() 
    14. Repeat the process for the set direction in clockwise or anticlockwise.
        if sprite.iskeypressed("left arrow"):
          angles = angle_s - shark_d
          sprite2.setdirection(DIRECTION=angles)
          
        if sprite.iskeypressed("right arrow"):
          angles = angle_s + shark_d
          sprite2.setdirection(DIRECTION=angles)
    15. Find the direction of shark and fish using the Python pictoblox function and store prediction value in ‘move’ variable.
      angle_f=sprite.direction()
      angle_s=sprite2.direction()
      move = runprediction(curr_x- target_x, curr_y-target_y, shark_x-curr_x, shark_y-curr_y, angle_f, angle_s)
    16. Now write the script for moving the Fish in forward direction and change clockwise or anticlockwise direction by fix value with the help of a conditional statement.
      1. If the predicted value is “UP” then fish will move fish_m position in same direction.
      2. If the predicted value is “LEFT” then fish will change direction by some constant value in anticlockwise direction.
      3. If the predicted value is “RIGHT” then fish will change direction by some constant value in clockwise direction.
        if move == "UP":
            sprite.move(fish_m)
            curr_x=sprite.x()
            curr_y=sprite.y()
        
        if move == "LEFT":
            angle = angle_f - fish_d
            sprite.setdirection(DIRECTION=angle)
        
        if move == "RIGHT":
            angle = angle_f + fish_d
            sprite.setdirection(DIRECTION=angle)
    17. Write the conditional statement for the chance variable. If the fish and shark position difference is less than 20, then the chance should be decreased by one.
       if abs(shark_x-curr_x)<20 and abs(shark_y-curr_y)<20:
          chance= chance-1
    18. Update the position of all three sprites, and if chance becomes 0, then the positions of all three sprites change randomly by the functions settarget1() and update chance value.
        if abs(shark_x-curr_x)<20 and abs(shark_y-curr_y)<20:
          chance= chance-1
          curr_x, curr_y, target_x, target_y, shark_x, shark_y = settarget1(40)
          sprite3.say(("score: ",score ," and chance:  ",chance,""))
          if (chance == 0):
            chance=5
    19. Again write the conditional statement for the score variable if the fish and food position difference is less then 20 then the score should be increased by one and food positions change randomly by the function settarget().
      if abs(curr_x-target_x)<20 and abs(curr_y-target_y)<20: 
          score = score + 1 
          sprite.say(("your score is: {}".format(score)))
          target_x, target_y = settarget(4)
    20. The final code is as follows:
      sprite = Sprite('Fish')
      sprite1 = Sprite('Orange')
      sprite2 = Sprite('Shark 2')
      sprite3 = Sprite('Button3')
       
      import random
      import time
      import numpy as np
      import tensorflow as tf
      import pandas as pd
      import os
      import math
      #Load Number Model
      model= tf.keras.models.load_model(
      		"num_model.h5", 
      		custom_objects=None, 
      		compile=True, 
      		options=None)
      		
      #List of classes
      class_list = ['UP','RIGHT','LEFT',]  
      		
      curr_x = 25
      curr_y = 108
      shark_x=-177
      shark_y=116
      score=0
      chance=5
      fish_d=20
      fish_m=35
      shark_m=25
      shark_d=20
      angle_f=90
      angle_s=90
      
      sprite3.say(("score: ",score ," and chance:  ",chance,""))
       
      sprite.setx(curr_x) 
      sprite.sety(curr_y) 
      sprite2.setx(shark_x) 
      sprite2.sety(shark_y) 
      sprite.setdirection(DIRECTION=angle_f) 
      sprite2.setdirection(DIRECTION=angle_s)
      
      def settarget1(t):
        x = random.randrange(-200, 200, t)
        y = random.randrange(-155, 155, t)
        x1 = random.randrange(-200, 200, t)
        y1 = random.randrange(-155, 155, t)
        x2 = random.randrange(-200, 200, t)
        y2 = random.randrange(-155, 155, t)
        time.sleep(0.1)
        sprite1.setx(x1)
        sprite1.sety(y1)
        sprite.setx(x)
        sprite.sety(y)
        sprite2.setx(x2)
        sprite2.sety(y2)
        return x, y, x1, y1, x2, y2
        
      def settarget(m):
        x = random.randrange(-200, 200, m)
        y = random.randrange(-155, 155, m)
        time.sleep(0.1)
        sprite1.setx(x)
        sprite1.sety(y)
        return x, y
        
      target_x, target_y = settarget(40)
      def runprediction(diff_x, diff_y, diff_x1, diff_y1, ang1, ang2):
        inputValue=[diff_x, diff_y, diff_x1, diff_y1, ang1, ang2]
        #Input Tensor
        inputTensor = tf.expand_dims(inputValue, 0)
        #Predict
        predict = model.predict(inputTensor)
        predict_index = np.argmax(predict[0], axis=0)
        #Output
        predicted_class = class_list[predict_index]
        return predicted_class
      while True:
        if sprite.iskeypressed("up arrow"):
          sprite2.move(shark_m)
          shark_x=sprite2.x()
          shark_y=sprite2.y()
          
        if sprite.iskeypressed("left arrow"):
          angles = angle_s - shark_d
          sprite2.setdirection(DIRECTION=angles)
        if sprite.iskeypressed("right arrow"):
          angles = angle_s + shark_d
          sprite2.setdirection(DIRECTION=angles)
       
        angle_f=sprite.direction()
        angle_s=sprite2.direction()
        move = runprediction(curr_x- target_x, curr_y-target_y, shark_x-curr_x, shark_y-curr_y, angle_f, angle_s)
        
        if move == "UP":
          sprite.move(fish_m)
          curr_x=sprite.x()
          curr_y=sprite.y()
      
        if move == "LEFT":
          angle = angle_f - fish_d
          sprite.setdirection(DIRECTION=angle)
      
        if move == "RIGHT":
          angle = angle_f + fish_d
          sprite.setdirection(DIRECTION=angle)
       
        if abs(shark_x-curr_x)<20 and abs(shark_y-curr_y)<20:
          chance= chance-1
          curr_x, curr_y, target_x, target_y, shark_x, shark_y = settarget1(40)
          sprite3.say(("score: ",score ," and chance:  ",chance,""))
          if (chance == 0):
            chance=5
        if abs(curr_x-target_x)<35 and abs(curr_y-target_y)<35:
          score = score + 1
          sprite3.say(("score: ",score ," and chance:  ",chance,""))
          target_x, target_y = settarget(4)
      
        time.sleep(0.2) 
      

      Final Result

      Conclusion

      Creating a Machine Learning Model of “Shark Attack: Hungry for Fish” game can be both complex and time-consuming. Through the steps demonstrated in this project, you can create your own Machine Learning Model of automated game. Once trained, you can export the model into the Python Coding Environment, where you can tweak it further to give you the desired output. Try creating a Machine Learning Model of your own today and explore the possibilities of Number Classifier in PictoBlox!

Read More
Learn how to create a dataset and Machine Learning Model for an automated game from the user's input. See how to open the ML environment, upload data, label images, train the model, and export the Python script.

Introduction

In this example project, we are going to create a Machine Learning Model where beetle automatically feed on randomly generated food in space.

Data Collection

  • Now, we are going to collect the data of  “BeetleBot: Autonomous Feeding in Space” game.
  • This data will contain the actions that we have taken to accomplish the game successfully.
  • We will use the data we collect here, to teach our device how to play the “BeetleBot: Autonomous Feeding in Space” game, i.e. to perform machine learning.
  • The data that you collect will get saved in your device as a csv (comma separated values) file. If you open this file in Microsoft Excel, it will look as shown below:

Follow the steps below

  1. Open PictoBlox and create a new file.
  2. Select the coding environment as Python Coding Environment.
  3. Now write code in python.

Code for making dataset

  1. Creates a sprite object named “Beetle”. A sprite is typically a graphical element that can be animated or displayed on a screen.
  2. Creates another sprite object named “Strawberry” and also upload backdrop of “Galaxy”.
  3. Click on the Beetle.py file from the Project files section.
    sprite = Sprite('Beetle')
  4. Similarly, declare new sprite on the Beetle.py file.
    sprite1 = Sprite('Strawberry')
  5. Then we will import the time, random, os, math, TensorFlow as tf  and Pandas as pd modules using the import keyword for using delay in the program later.
    1. Time – For using delay in the program.
    2. Random – For using random position.
    3. Pandas as pd – For using Data Frame.
    4. Os– For reading files from Program files.
      import random
      import time
      import tensorflow as tf
      import pandas as pd
      import os
  6. Now, make 3 variables curr_x, curr_y, ,beetle_m, angle and score with initial values -170, 138, 5,0 and 90 respectively.
    1. curr_x – To store the initial x – position of beetle.
    2. curr_y – To store the initial y – position of beetle.
    3. beetle_m– To store increment value in movement of beetle on pressing specific key.
    4. angle – To store initial angle of beetle.
    5. score – To store the score while playing the game.
      curr_x = -170
      curr_y = 138
      mov_f= 5
      score = 0
      angle = 90
  7. Now set initial position and angle of beetle.
    sprite.setx(curr_x)
    sprite.sety(curr_y)
    sprite.setdirection(DIRECTION=angle)
  8. Now, make a function settarget() in which we are generating food at a random position. We pass one argument “t” in the function for generating target food in the greed position of the t gap.
    1. x and y – To generate the food at random position on stage.
    2. time.sleep – For giving the time delay.
    3. sprite1.set()– To Set the position of food at random position on stage.
      def settarget(t):
      x = random.randrange(-200, 200, t)
      y = random.randrange(-155, 155, t)
      time.sleep(0.1)
      sprite1.setx(x)
      sprite1.sety(y)
      return x, y 
  9. Now set the target (Strawberry). In this, beetle are chasing the target, and target_x  and  target_y should be equal to the x and y positions of the food.
    target_x, target_y = settarget(40) 
  10. Now create a data frame of name “Chase_Data.csv” to collect the data for machine learning and if this name csv exist then directly add data in it.
    if(os.path.isfile('Chase_Data.csv')):
      data=pd.read_csv('Chase_Data.csv')
    else:
      data = pd.DataFrame({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "Action": "RIGHT"}, index=[0])
    
  11. After that, we will use the while True loop to run the code indefinitely. Don’t forget to add a colon ‘:’ just after the loop to avoid errors.
    while True:
  12. Now write the script for moving the Beetle in upward direction, downward direction, left direction and right direction by fix value with the help of a conditional statement.
    1. If the up arrow key is pressed then beetle will move beetle_m position in y direction by adding fix value in beetle’s y position.
    2. After pressing the up arrow key action taken should be stored in the Data frame with data.append command.
       if sprite.iskeypressed("up arrow"):
          data = data.append({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "Action": "UP"}, ignore_index=True)
          curr_y = curr_y + beetle_m
          sprite.setdirection(DIRECTION=0)
          sprite.setx(curr_x)
          sprite.sety(curr_y)
  13. Repeat the process for the set direction and position of beetle for down, left and right movement.
    if sprite.iskeypressed("down arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "Action": "DOWN"}, ignore_index=True)
        curr_y = curr_y - beetle_m
        sprite.setdirection(DIRECTION=-180)
        sprite.setx(curr_x)
        sprite.sety(curr_y) 
        
      if sprite.iskeypressed("left arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "Action": "LEFT"}, ignore_index=True)
        curr_x = curr_x - beetle_m
        sprite.setdirection(DIRECTION=-90)
        sprite.setx(curr_x)
        sprite.sety(curr_y)
         
      if sprite.iskeypressed("right arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "Action": "RIGHT"}, ignore_index=True)
        curr_x = curr_x + beetle_m
        sprite.setdirection(DIRECTION=90)
        sprite.setx(curr_x)
        sprite.sety(curr_y)
  14. Write the conditional statement for the storing data in csv file after few score.
    if(score>0 and score%10==0):
        data.to_csv('Chase_Data.csv',index=False)
  15. Again write the conditional statement for the score variable if the fish and food position difference is less then 20 then the score should be increased by one.
    if abs(curr_x-target_x)<20 and abs(curr_y-target_y)<20: 
        score = score + 1 
        sprite.say(("your score is: {}".format(score)))
  16. If the score is equal to or greater than 20 then data should be printed on Chase Data.csv file.
    if (score >= 20):
          print(data)
          data.to_csv('Chase Data.csv')
          break
    target_x, target_y = settarget()
  17. The final code is as follows:
    sprite = Sprite('Fish')
    sprite1 = Sprite('Orange')
     
    import random
    import time
    import numpy as np
    import tensorflow as tf
    import pandas as pd
    import os
    		
    curr_x = -170
    curr_y = 138
    score=0
    angle=90
    beetle_m=5
    
    sprite.say(("your score is: {}".format(score)))
    sprite.setx(-170)
    sprite.sety(138)
    sprite.setdirection(DIRECTION=angle)
    
    def settarget(t):
      x = random.randrange(-200, 200, t)
      y = random.randrange(-155, 155, t)
    
      time.sleep(0.1)
      sprite1.setx(x)
      sprite1.sety(y)
      return x, y
    
    target_x, target_y = settarget(40)
    if(os.path.isfile('Chase_Data.csv')):
      data=pd.read_csv('Chase_Data.csv')
    else:
      data = pd.DataFrame({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "Action": "RIGHT"}, index=[0])
    
    while True:
      if sprite.iskeypressed("up arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "Action": "UP"}, ignore_index=True)
        curr_y = curr_y + beetle_m
        sprite.setdirection(DIRECTION=0)
        sprite.setx(curr_x)
        sprite.sety(curr_y)
    
      if sprite.iskeypressed("down arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "Action": "DOWN"}, ignore_index=True)
        curr_y = curr_y - beetle_m
        sprite.setdirection(DIRECTION=-180)
        sprite.setx(curr_x)
        sprite.sety(curr_y) 
        
      if sprite.iskeypressed("left arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "Action": "LEFT"}, ignore_index=True)
        curr_x = curr_x - beetle_m
        sprite.setdirection(DIRECTION=-90)
        sprite.setx(curr_x)
        sprite.sety(curr_y)
         
      if sprite.iskeypressed("right arrow"):
        data = data.append({"curr_X": curr_x, "curr_Y": curr_y, "tar_x": target_x, "tar_y": target_y, "diff_x":curr_x-target_x, "diff_y":curr_y-target_y, "Action": "RIGHT"}, ignore_index=True)
        curr_x = curr_x + beetle_m
        sprite.setdirection(DIRECTION=90)
        sprite.setx(curr_x)
        sprite.sety(curr_y)
        
      if(score>0 and score%10==0):
        data.to_csv('Chase_Data.csv',index=False)
        
      if abs(curr_x-target_x)<20 and abs(curr_y-target_y)<20:
        score = score + 1
        sprite.say(("your score is: {}".format(score)))
        if (score >= 20):
          data.to_csv('Chase_Data.csv',index=False)
          break
        target_x, target_y = settarget(40)
  18. Press the Run button and play “BeetleBot: Autonomous Feeding in Space” game to collect data.
  19. Store this dataset on your local computer.

    Numbers(C/R) in Machine Learning Environment

    Datasets on the internet are hardly ever fit to directly train on. Programmers often have to take care of unnecessary columns, text data, target columns, correlations, etc. Thankfully, PictoBlox’s ML Environment is packed with features to help us pre-process the data as per our liking.

    Let’s create the ML model.

    Opening Image Classifier Workflow

    Alert: The Machine Learning Environment for model creation is available in the only desktop version of PictoBlox for Windows, macOS, or Linux. It is not available in Web, Android, and iOS versions.

    Follow the steps below:

    1. Open PictoBlox and create a new file.
    2. Select the coding environment as Block Coding Environment.
    3. Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
    4. You’ll be greeted with the following screen.
      Click on “Create New Project“.
    5. You shall see the Numbers C/R workflow with an option to either “Upload Dataset” or “Create Dataset”.

      Uploading/Creating Dataset

      Datasets can either be uploaded or created on the ML Environment. Lets see how it is done.

      Uploading a dataset
      1. To upload a dataset, click on the Upload Dataset button and the Choose CSV from your files button.
        Note: An uploaded dataset must be a “.csv” file.
      2. Once uploaded the first 50 rows of the uploaded CSV document will show up in the window.

      3. If you look at the output column, all the values are currently “0”. Hence, first we need to create an output column.
        1. In the Dataset table, click on the tick near Select All to de-select all the columns.
        2. click on the tick of Action column to select it. We will make this column the output.
        3. The output column must always be numerical. Hence click on the button Text to Number to convert the data within this column to numerical type.
        4. Now select it again and press the Set as Output button to set this column as Output.
        5. There is also many which is not useful in training our model and needs to be disable. So select it and click the Disable button in the Selected columns section.

          Creating a Dataset
          1. To create a dataset, click on the Create Dataset button.
          2. Select the number of rows and columns that are to be added and click on the Create button. More rows and columns can be added as and when needed.

          Notes:

          1. Each column represents a feature. These are the values used by the model to train itself.
          2. The “Output” column contains the target values. These are the values that we expect the model to return when features are passed.
          3. The window only shows the first 50 rows of the dataset.
          4. Un-check the “Select All” checkbox to un-select all the columns.

          Training the Model

          After data is pre-processed and optimized, it’s fit to be used in model training. To train the model, simply click the “Train Model” button found in the “Training” panel.

          By training the model, meaningful information is extracted from the numbers, and that in turn updates the weights. Once these weights are saved, the model can be used to make predictions on data previously unseen.

          The model’s function is to use the input data and predict the output. The target column must always contain numbers.

          However, before training the model, there are a few hyperparameters that need to be understood. Click on the “Advanced” tab to view them.

          There are three hyperparameters that can be altered in the Numbers(C/R) Extension:

          1. Epochs– The total number of times the data will be fed through the training model. Therefore, in 10 epochs, the dataset will be fed through the training model 10 times. Increasing the number of epochs can often lead to better performance.
          2. Batch Size– The size of the set of samples that will be used in one step. For example, if there are 160 data samples in the dataset, and the batch size is set to 16, each epoch will be completed in 160/16=10 steps. This hyperparameter rarely needs any altering.
          3. Learning Rate– It dictates the speed at which the model updates the weights after iterating through a step. Even small changes in this parameter can have a huge impact on the model performance. The usual range lies between 0.001 and 0.0001.
          Note: Hover the mouse pointer over the question mark next to the hyperparameters to see their description.

          It’s a good idea to train a numeric classification model for a high number of epochs. The model can be trained in both JavaScript and Python. In order to choose between the two, click on the switch on top of the Training panel.

          Alert: Dependencies must be downloaded to train the model in Python, JavaScript will be chosen by default.

          The accuracy of the model should increase over time. The x-axis of the graph shows the epochs, and the y-axis represents the accuracy at the corresponding epoch.

          A window will open. Type in a project name of your choice and select the “Numbers(C/R)” extension. Click the “Create Project” button to open the Numbers(C/R) window.

          Testing the Model

          To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

          The model will return the probability of the input belonging to the classes.

      Export in Python Coding

      Click on the “PictoBlox” button, and PictoBlox will load your model into the Python Coding Environment if you have opened the ML Environment in Python Coding.

    Code

    1. Creates a sprite object named “Beetle”. A sprite is typically a graphical element that can be animated or displayed on a screen.
    2. Creates another sprite object named “Strawberry” and also upload backdrop of “Space” .
    3. Click on the Beetle.py file from the Project files section.
      sprite = Sprite('Beetle')
    4. Similarly, declare new sprite on the Beetle.py file.
      sprite1 = Sprite('Strawberry')
    5. Then we will import the time, random, TensorFlow as tf  and Pandas as pd modules using the import keyword for using delay in the program later.
      1. Time – For using delay in the program.
      2. Random – For using random position.
      3. Pandas as pd – For using Data Frame.
      4. TensorFlow – For loading model.
        import random
        import time
        import tensorflow as tf
        import pandas as pd
    6. Now, make 3 variables curr_x, curr_y, beetle_m, angle and score with initial values -170, 138, 15, 90 and 0 respectively.
      1. curr_x – To store the initial x – position of beetle.
      2. curr_y – To store the initial y – position of beetle.
      3. beetle_m – To store increment value in movement of beetle on pressing specific key.
      4. angle – To store initial angle of beetle.
      5. score – To store the score while playing the game.
        curr_x = -170
        curr_y = 138
        beetle_m= 15
        angle = 90
        score = 0
    7. Now set initial position and angle of beetle.
      sprite.setx(curr_x)
      sprite.sety(curr_y)
      sprite.setdirection(DIRECTION=90)
    8. Now, make a function settarget() in which we are generating strawberry at a random position. We pass one argument “t” in the function for generating target strawberry in the greed position of the t gap.
      1. x and y – To generate the food(strawberry) at random position on stage.
      2. time.sleep – For giving the time delay.
      3. sprite1.set()– To Set the position of food at random position on stage.
        def settarget(t):
        x = random.randrange(-200, 200, t)
        y = random.randrange(-155, 155, t)
        time.sleep(0.1)
        sprite1.setx(x)
        sprite1.sety(y)
        return x, y 
    9. Now set the position of strawberry. In this, beetle are chasing the strawberry, and target_x  and  target_y should be equal to the x and y positions of the strawberry.
      target_x, target_y = settarget(40) 
    10. Now, make a function runprediction() in which we are predicting class (Left, Up, Right, Down) by taking argument from user . We pass three arguments “diff_x”, “diff_y” in the function.
      1. inputvalue – To store input parameters of function in array.
      2. model.predict() – For predicting output from trained model.
      3. np.argmax(,)– To find the most probable prediction output.
        def runprediction(diff_x, diff_y):
          inputValue=[diff_x, diff_y]
          #Input Tensor
          inputTensor = tf.expand_dims(inputValue, 0)
          #Predict
          predict = model.predict(inputTensor)
          predict_index = np.argmax(predict[0], axis=0)
          #Output
          predicted_class = class_list[predict_index]
          return predicted_class
        
    11. After that, we will use the while True loop to run the code indefinitely. Don’t forget to add a colon ‘:’ just after the loop to avoid errors.
      while True:
    12. In while loop find angle of sprite and call runprediction function by passing arguments in it.
      angle=sprite.direction()
       move = runprediction(curr_x- target_x, curr_y-target_y, angle)
    13. Now write the script for moving the Beetle in forward direction, Upward direction, Left direction and Right direction by fix value with the help of a conditional statement.
      1. If the predicted value is “UP” then beetle will move beetle_m position in direction=0.
      2. If the predicted value is “LEFT” then beetle will move beetle_m position in direction=-90.
      3. If the predicted value is “RIGHT” then beetle will move beetle_m position in direction=90.
      4. If the predicted value is “DOWN” then beetle will move beetle_m position in direction=-180.
        if move == "UP":
            curr_y = curr_y + beetle_m
            sprite.setdirection(DIRECTION=0)
            sprite.setx(curr_x)
            sprite.sety(curr_y)
        
          if move == "LEFT":
            curr_x = curr_x - beetle_m
            sprite.setdirection(DIRECTION=-90)
            sprite.setx(curr_x)
            sprite.sety(curr_y)
        
          if move == "RIGHT":
            curr_x = curr_x + beetle_m
            sprite.setdirection(DIRECTION=90)
            sprite.setx(curr_x)
            sprite.sety(curr_y)
        
         if move == "DOWN":
            curr_y = curr_y - beetle_m
            sprite.setdirection(DIRECTION=-180)
            sprite.setx(curr_x)
            sprite.sety(curr_y) 
    14.  Again write the conditional statement for the score variable if the beetle and strawberry position difference is less then 20 then the score should be increased by one also set new position of target.
      if abs(curr_x-target_x)<20 and abs(curr_y-target_y)<20: 
          score = score + 1 
          sprite.say(("your score is: {}".format(score)))
      target_x, target_y = settarget()
    15.  Now add delay function for delaying movement by 0.02 seconds.
      time.sleep(0.02)
    16. The final code is as follows:
      sprite = Sprite('Beetle')
      sprite1 = Sprite('Strawberry')
       
      import random
      import time
      import numpy as np
      import tensorflow as tf
      
      #Load Number Model
      model= tf.keras.models.load_model(
      		"num_model.h5", 
      		custom_objects=None, 
      		compile=True, 
      		options=None)
      		
      #List of classes
      class_list = ['RIGHT','LEFT','DOWN','UP',]  
      		
      curr_x = -170
      curr_y = 138
      score=0
      beetle_m=30
      angle=90
      
      sprite.say(("your score is: {}".format(score)))
      sprite.setx(-170)
      sprite.sety(138)
      sprite.setdirection(DIRECTION=angle)
      
      def settarget():
        x = random.randrange(-200, 200, 1)
        y = random.randrange(-155, 155, 1)
        time.sleep(0.1)
        sprite1.setx(x)
        sprite1.sety(y)
        return x, y
      
      target_x, target_y = settarget()
      
      def runprediction(diff_x, diff_y):
        inputValue=[diff_x, diff_y]
        #Input Tensor
        inputTensor = tf.expand_dims(inputValue, 0)
        #Predict
        predict = model.predict(inputTensor)
        predict_index = np.argmax(predict[0], axis=0)
        #Output
        predicted_class = class_list[predict_index]
        return predicted_class
      
      while True:
        
        move = runprediction(curr_x- target_x, curr_y-target_y)
      
        if move == "UP":
          curr_y = curr_y + beetle_m
          sprite.setdirection(DIRECTION=0)
          sprite.setx(curr_x)
          sprite.sety(curr_y)
      
        if move == "DOWN":
          curr_y = curr_y - beetle_m
          sprite.setdirection(DIRECTION=-180)
          sprite.setx(curr_x)
          sprite.sety(curr_y) 
      
        if move == "LEFT":
          curr_x = curr_x - beetle_m
          sprite.setdirection(DIRECTION=-90)
          sprite.setx(curr_x)
          sprite.sety(curr_y)
           
      
        if move == "RIGHT":
          curr_x = curr_x + beetle_m
          sprite.setdirection(DIRECTION=90)
          sprite.setx(curr_x)
          sprite.sety(curr_y)
       
      
        if abs(curr_x-target_x)<20 and abs(curr_y-target_y)<20:
          score = score + 1
          sprite.say(("your score is: {}".format(score)))
          target_x, target_y = settarget()
      
        time.sleep(0.2) 
      

      Final Result

      Conclusion

      Creating a Machine Learning Model of “Beetle’s Cosmic Feast: Strawberry Space Adventure” can be both complex and time-consuming. Through the steps demonstrated in this project, you can create your own Machine Learning Model of automated game. Once trained, you can export the model into the Python Coding Environment, where you can tweak it further to give you the desired output. Try creating a Machine Learning Model of your own today and explore the possibilities of Number Classifier in PictoBlox!

Read More
All articles loaded
No more articles to load