Sign Detector Robotic Arm

Introduction

A sign detector robotic arm is a smart robot that can recognize and understand signs or signals in its surroundings. It uses cameras and other sensors to capture visual information and computer algorithms to analyze the signs. The robot can learn different types of signs through machine learning techniques. Once a sign is identified, the robotic arm can perform specific actions based on what the sign means. These robotic arms have many uses, such as helping in healthcare, manufacturing, transportation, and assisting people with communication disabilities. They are an exciting advancement in human-robot interaction, allowing robots to understand and respond to signs, expanding their abilities and applications.

Code

sprite = Sprite('Tobi')

recocards = RecognitionCards()
recocards.video("on flipped")
recocards.enablebox()
recocards.setthreshold(0.6)
roboticArm = RoboticArm(1,2,3,4)

roboticArm.sethome()
while True:
  recocards.analysecamera()
  sign = recocards.classname()
  sprite.say(sign + ' detected')
  if recocards.count() > 0:
	  if 'Turn Left' in sign:
		  roboticArm.movebyinoneaxis(10,"X",1000)

	  if 'Turn Right' in sign:
		  roboticArm.movebyinoneaxis(-10,"X",1000)
	
	  if 'Go' in sign:
		  roboticArm.movebyinoneaxis(10,"Y",1000)
			
	  if 'U Turn' in sign:
		  roboticArm.movebyinoneaxis(-10,"Y",1000)

Logic

1. Open the Pictoblox application.
2. Select the block-based environment.
3. Click on the Recognition Cards and robotic arm extension available in the left corner.
4. Initialize the video on stage and set the transparency as 0%.
5. Drag and drop the forever loop to continue to initialize the image from the stage, and get the input from the camera.
6. Show the bounding box around the sign detected from the stage.
7. A RecognitionCards object is created and assigned to the variable ‘recocards‘. This object represents the functionality related to recognizing and analyzing signs or signals.
8. We configures the video functionality of the RecognitionCards object. It sets the video to be flipped horizontally, ensuring that the signs appear correctly when displayed on the stage.
9. Now the program activates the feature that displays a bounding box around the detected sign on the stage using the enablebox() function. This helps visually identify the location of the sign.
10. Further sets the threshold value to 0.6 for the RecognitionCards object. The threshold determines the minimum confidence level required for a sign to be recognized. Any sign with a confidence level below 0.6 will not be considered valid.
11. Then A RoboticArm object is created and assigned to the variable ‘roboticArm’. The numbers 1, 2, 3, and 4 represent the specific configuration or parameters of the robotic arm.The object allows control over the robotic arm’s movements.
12. The robotic arm to move to a predefined home position using home(). The home position is a reference point or starting point for the arm’s movements.
13. We create a while loop that will continuously execute the code block.
14. Then we capture and analyzes the video input from the camera. The RecognitionCards object processes the captured frame to detect and recognize signs.
15. We check if there is at least one sign detected by the RecognitionCards object. If the count of detected signs is greater than 0, the following code block will execute.
16. If the detected sign ‘Turn Left‘, the robotic arm will move by 10 units in the X-axis direction over a duration of 1000 milliseconds. This function call controls the movement of the robotic arm.
17. If the detected sign ‘Turn Right‘, the robotic arm will move by -10 units (indicating the opposite direction) in the X-axis direction over a duration of 1000 milliseconds.
18. If the detected sign ‘Go‘, the robotic arm will move by 10 units in the Y-axis direction over a duration of 1000 milliseconds.
19. If the detected sign ‘U Turn‘, the robotic arm will move by -10 units in the Y-axis direction over a duration of 1000 milliseconds.

Output