Raulperula-colab

From jderobot
Jump to: navigation, search

The goal is to explore the use of Scratch with robots, both simulated or real, that JdeRobot-ROS allows and simplifies. We will start with simulated robots in Gazebo, despite with real robots will be the same as they use the same interfaces. This project expects to develop a new tool prototype that reads Scratch programs and translate them into ROS-Python components.

More info: https://summerofcode.withgoogle.com/projects/#6344228931108864

People[edit]

Development[edit]

These are the repositories that contains the implementations made for this project.

  • Technologies: Python 2.7, Scratch 2.0, ROS Kinetic

Resume

Commits Issues Pull Requests
  • JdeRobot: colab-gsoc2017-RaulPerula (Link)
  • Kurt: raulperula/kurt (Link)
  • JdeRobot: [libs][jderobotcomm_py] bug in return? #897 (Link)
  • JdeRobot: [libs][jderobotcomm_py] Support for drone commands #896 (Link)
  • JdeRobot: [tools][uav_viewer_py] Confusion between config files #895 (Link)
  • Kurt: "Set" block not supported #42 (Link)
  • Kurt: How to use the extract_blocks_20 script #40 (Link)
  • Kurt: How to support custom Scratch 2.0 blocks #38 (Link)
  • Kurt: Upgrades to Scratch 2.0 (and ScratchX) #41 (Link)
  • JdeRobot: [issue #900] added new tool scratch2jderobot. ready to be integrated #901 (Link)

Documentation[edit]

Setup[edit]

Both, the development and the tests, are made in a machine with the OS Ubuntu 16.04 (LTS).

Installing ROS Kinetic[edit]

Follow the steps that ROS provide in the next link.

Installing JdeRobot[edit]

  • Add the latest Gazebo sources:
sudo sh -c 'echo "deb http://packages.osrfoundation.org/gazebo/ubuntu-stable `lsb_release -cs` main" > /etc/apt/sources.list.d/gazebo-stable.list'
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key 67170598AF249743
  • Add the lastest zeroc-ice sources:
sudo apt-add-repository "deb http://zeroc.com/download/apt/ubuntu$(lsb_release -rs) stable main"
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 5E6DA83306132997
  • Add JdeRobot repository (using dedicated file /etc/apt/sources.list.d/jderobot.list):
sudo sh -c 'cat<<EOF>/etc/apt/sources.list.d/jderobot.list
# for ubuntu 16.04 LTS (64 bit)

deb http://jderobot.org/apt xenial main
deb-src http://jderobot.org/apt xenial main
EOF'
  • Get and add the public key from the JdeRobot repository
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv B0E7F58E82C8091DF945A0453DA08892EE69A25C
  • Update the repositories
sudo apt update
  • Install JdeRobot:
sudo apt install jderobot

After installing the package, you can close the terminal and reopen it to source the environment variables, OR just type:

source ~/.bashrc
  • Install JdeRobot dependencies
sudo apt install jderobot-deps-dev libgoogle-glog-dev
sudo pip2 install --upgrade pip
sudo pip2 install zeroc-ice
  • Download the latest version of the packages.
sudo apt update && sudo apt upgrade
  • Download the JdeRobot source code:
git clone http://github.com/JdeRobot/JdeRobot.git
cd JdeRobot
mkdir build && cd build
  • Install the script that tunes the environment variables for ROS in your .bashrc configuration file, and run it for the current shell:
echo "source /opt/ros/kinetic/setup.bash" >> ~/.bashrc 
source ~/.bashrc 
  • Check system and dependencies
cmake ..
  • Compile
make
  • Install
sudo make install
  • Once everything is installed, install the script that tunes the environment variables for Gazebo and ROS in your .bashrc configuration file, and run it for the current shell:
source ~/.bashrc 

Installing Scratch 2.0[edit]

1. Install AdobeAir
  • Download the adobeair file (.bin) and execute it
wget -O adobe-air.sh http://drive.noobslab.com/data/apps/AdobeAir/adobe-air.sh
chmod +x adobe-air.sh; sudo ./adobe-air.sh
2. Install Scratch 2.0
  • Download the scratch file (link) and then double click to install it

Troubleshooting

If when installing Scratch there is a pop-up asking for a root passord

References

[1] Tutorial to installing

[2] Download page

[3] Scratch 2.0 Installation

Installing Kurt library[edit]

  • Installation from pip
pip install kurt

State of the Art[edit]

Scratch 2.0[edit]

References

  1. Info about Python and Scratch 2.0 (Link)
  2. Info about the Scratch File Format (2.0) (Link)
  3. Interesting presentation about the capacities of Scratch and Snap! (Link)
  4. Conversion between Scratch and Python v3 (Link)
  5. Scratch and Python Programming Construct (Link)

Kurt[edit]

Kurt is a Python library for reading/writing MIT's Scratch file formats (1.4 and 2.0).

Kurt provides an API that manages Sprites and Stages. It also manages Scripts through the Blocks in the Scratch original program.

References

  1. Kurt website (Link)
  2. API of the library (Link)

Skip[edit]

Skip is a Python Scratch Interpreter based on Kurt.

This project comes from a talk in the Barcelona Scratch Conference (more info).

References

  1. Skip website (Link)

Progress[edit]

Phase 1[edit]

Week 1 (2017/05/29)[edit]

Tasks:

Number Task Status Comments
1 Install Ubuntu 16.04 DONE On a virtualBox machine (original OS macos Sierra)
1.1 Install Git DONE
1.2 Install ROS Kinetic DONE
1.3 Install JdeRobot from repositories DONE
1.4 Install Scratch 2.0 DONE

Week 2 (2017/06/05)[edit]

Tasks:

Number Task Status Comments
1 State of the Art for translators from MIT Scratch to Python DONE Documented in the wiki.
2 Check JdeRobot repository in GitHub DONE Checked repository and examples for camera and gazebo.

Week 3 (2017/06/12)[edit]

Tasks:

Number Task Status Comments
1 Test translators from MIT Scratch to Python DONE
1.1 Test project Kurt DONE Kurt support Scratch 1.4 and 2.0 blocks. Has its own class structure.
1.2 Test project Skip DONE Skip does not have some of the blocks implemented. Seems to be discontinued.
2 Translate a Scratch program to Python DONE See notes.

Notes

  • Program to convert a Scratch 2.0 program to Python
#!/usr/bin/env python
# -*- coding: utf-8 -*-

import kurt
import os

# get current working directory
path = os.getcwd()
path = path[:path.rfind('src')] + 'data/'

# load the scratch project
p = kurt.Project.load(path + 'hello_world.sb2')

# show the blocks included
for scriptable in p.sprites + [p.stage]:
	for script in scriptable.scripts:
		print script

Example output:

kurt.Script([
    kurt.Block('whenGreenFlag'),
    kurt.Block('say:', u'Hello World')], pos=(53, 166))
  • Program to modify a Scratch 2.0 program in Python
#!/usr/bin/env python
# -*- coding: utf-8 -*-

import kurt
import os

# get current working directory
path = os.getcwd()
path = path[:path.rfind('src')] + 'data/'

# load the scratch project
p = kurt.Project.load(path + 'test_scratch2.sb2')

# show the blocks included
for scriptable in p.sprites + [p.stage]:
	for script in scriptable.scripts:
		s = script
		
# change number of steps for the move block
s.blocks[1].args = [20]

# save modifications
p.save(path + 'test_scratch2_modified.sb2')

Week 4 (2017/06/19)[edit]

Tasks:

Number Task Status Comments
1 Proof of concept using Kurt DONE See note 1.
2 Extract the requisites about the software to implement DONE See note 2.
3 Design of the system DONE See note 2.

Note 1

  • Scratch 2 program

  • Translator program (link)
  • Output:
for i in range(int(10)):
    print "Hello World"

Note 2

More info in the Google Drive document.

Week 5 (2017/06/26)[edit]

Tasks:

Number Task Status Comments
1 Create robotic blocks in Scratch (move, stop) DONE
2 Translate Scratch blocks to Python syntax DONE
3 Create ROS template to include the Python translated program DONE
4 Create an example program to move the Kobuki robot (gazebo) DONE Program (repeat 3, stop, move, sleep 5s, stop)

Part 1 -> import part2 -> execute()

Part 2 -> implements execute()

Results[edit]

Create the Scratch program

Execution steps

1. Translate the Scratch program to Python

roscd scratch2ros/script
python scratch2python example.sb2
 

2. Launch the kobuki robot in gazebo

roslaunch kobuki_gazebo kobuki_empty_world.launch --screen

3. Launch the translated program

roslaunch scratch2ros scratch2ros.launch

Video


Phase 2[edit]

Week 6 (2017/07/03)[edit]

Tasks:

Number Task Status Comments
1 Improve template to implement jderobot_comm interfaces DONE Part 1: Analysis and tests

Main Code

This code can be found in [4].

Week 7 (2017/07/10)[edit]

Tasks:

Number Task Status Comments
1 Improve template to implement jderobot_comm interfaces DONE Part 2: Implementation and tests

Robot class: generic interface for the motors

Robot Class Template

Implementation of the code can be found in [5].

Week 8 (2017/07/17)[edit]

Tasks:

Number Task Status Comments
1 Create standard functional blocks (while, if) DONE while(true). See note 1.
2 Create Scratch blocks to control the turns DONE Blocks: turn(angle), turn(vel). See note 1.
3 Create Scratch blocks to control the laser sensor (simple and parametrized) DONE Blocks: frontal_laser_sensor, laser_sensor(list_of_sensors)

It is impossible to create at this time blocks that return a value. Only by an extension.

Additional tasks:

  • I've been researching how to create an extension to include our own blocks by default. More info: [6]
  • I've created an extension with the blocks defined by us (See note 2).
  • Created more Scratch examples to complement the tests. More info: [7]
  • Kurt does not recognize (or load) blocks from extensions (e.g. Wedo, Makeblock, or ours). It is necessary to upgrade the code to add this functionality.

Note 1:

MAPPING = [
    ['end', ''],
    ['if', 'if %s:'],
    ['move', 'robot.move(%s)'],
    ['move', 'robot.move()'],
    ['repeat', 'for i in range(%s):'],
    ['say', 'print(%s)'],
    ['stop', 'robot.stop()'],
    ['turn', 'robot.turn(%s)'],
    ['turn', 'robot.turn()'],
    ['wait', 'time.sleep(int("%s".replace(" secs", "")))'],
    ['while', 'while %s:'],
    ['forever', 'while True:'],
]

Note 2:

{
    "extensionName": "Scratch2Robot",
    "extensionPort": 12345,
	"blockSpecs": [
		["", "stop robot", "stop/robot"],
		["", "move robot %m.direction", "move/robot", "forward"],
		["", "move robot %m.direction speed %n", "move/robot/speed", "forward", 1],
		["", "turn robot %m.turnDirection", "turn/robot", "left"],
		["", "turn robot %m.turnDirection speed %n", "turn/robot/speed", "left", 1]
	],
	"menus": {
		"direction": ["forward", "back"],
		"turnDirection": ["left", "right"]
	}
}

Week 9 (2017/07/24)[edit]

Tasks:

Number Task Status Comments
1 Include extensions in Kurt (part 1) DONE Analyze the source code.
  • Open Pull Request to fix a bug in docs generation. Ref: [8]
  • Open Issue to "support custom Scratch blocks". Ref: [9]
  • Open Issue to "load Scratch extensions". Ref: [10]
  • Commits in Kurt (fork) repository implementing the extension loading (pending of Pull Request). Ref: [11]
  • Now Scratch loads our own extension. See note 1.
  • Implemented the script to auto-generate a source code current blocks of Scratch 2.0 from original Source Code in GitHub. See note 2.

Note 1:

Note 2:

Implementation of the file can be found in [12].

Phase 3[edit]

Week 10 (2017/07/31)[edit]

Tasks:

Number Task Status Comments
1 Include extensions in Kurt DONE
  • Fixed bugs and implemented solution for the Kurt project in order to load extensions included in Scratch 2.0. Refs: [13] [14]
  • New version of the scratch translator using the extensions
2 Create general test programs in Scratch for: move and laser sensor (simple and parametric) DONE
  • Created repertoire of Scratch test programs for basic and parametrized blocks
3 Study solution from Erle robotics for blockly DONE Ref: [15]
  • Seems that the ROS pkg is discontinued in the GitHub repository (last commit Feb.16)
  • It is interesting the way one is able to create new blocks
  • The pkg is made to work with Python 3 (although they say that some parts are compatible with 2.7)
  • I have not found the versions of ROS compatible with the pkg (some comments speak about the hydro version)
  • I have tried to follow the steps for compiling and executing the code but it does not work

Additional tasks:

  • Due to an invalid load of the conditional sentences (functions inside the condition), I have had to think an alternative way to load this kind of sentences.
  • Opened a new issue for the Kurt project due to it is not loading the "variable set block".

Week 11 (2017/08/07)[edit]

Tasks:

Number Task Status Comments
1 Implement variable loading for Kurt project DONE
  • Currently there is a bug that does not allow to load local variables
  • Issue opened in the project repository. Ref: [16]
  • It was due to a bug when generating the commands from the source code. Once the previous pull request was done, I will send another with this solution. Meanwhile I am working with my local solution
2 Implement a solution (maybe recursive) to load blocks inside other blocks DONE
  • This is useful for general blocks such as conditionals that have a function call in the condition
3 Create in Scratch a program in order to implement an "obstacle avoider" robot behavior (using the extension) DONE
  • Created two kind of programs: one implementation in scratch to the simulation in ICE, and another implementation to the simulation in ROS
  • Recorded videos with the robots working in the simulated scenario
4 Propose other kind of demos in order to show the advances in the project DONE

For instance:

  • Perceptive blocks for cameras: color detection
  • Easy behaviors for a drone: move/turn in 3D, take off, land

Studying the possibilities:

  • Analyzed and tested the rest of the components in the jderobotcomm library: camera and pose3d (odom)

Additional tasks:

  • Created a testing bench of Scratch programs for general and robotics blocks

Week 12 (2017/08/14)[edit]

Tasks:

Number Task Status Comments
1 Record videos for showing results at the end of the project DONE
2 Test drone functionality in JdeRobot DONE
  • Studied the source code of the uav_viewer program in order to create a common program to control drones
  • Tested the functionality individually from the examples in simulation
  • Created program to test all drone functionalities in conjunction
  • Created configuration file (only for ICE at this time) for drones
3 Implement blocks in Scratch extension related to drones DONE
  • Proposed blocks: take-off, land, move-linear (X, Y), move-angular (W), move-spatial (Z)

Additional tasks:

  • Created a new class named "Drone" similar to the Robot class but with a different implementation due to there is not common interface yet
  • Renamed the extension and the ROS pkg with the "scratch2jderobot" name

Week 13 (2017/08/21)[edit]

Tasks:

Number Task Status Comments
1 Make demo and tutorial videos DONE
  • [Scratch2JdeRobot][Tutorial] How to install the repository as a ROS package (individually)
  • [Scratch2JdeRobot][Tutorial] How to load the extension in Scratch offline
  • [Scratch2JdeRobot][Tutorial] How to add a new block in the extension and the converter
  • [Scratch2JdeRobot][Tutorial] How to program in Scratch: mobile robots
  • [Scratch2JdeRobot][Tutorial] How to program in Scratch: drones
  • [Scratch2JdeRobot][Tutorial] How to convert a Scratch program to Python
  • [Scratch2JdeRobot][Demo] Obstacle avoider robot (ROS)
  • [Scratch2JdeRobot][Demo] Obstacle avoider robot (ICE)
  • [Scratch2JdeRobot][Demo] Drone controlling (ICE)

Final Results[edit]

Tutorials (Spanish)
Demos (English)

Future Work[edit]

  • Camera application such as object detector (note: this is almost done)
  • 2d and 3d position control (robot & drone)
  • Test in real robots (although this is justified because the JdeRobot project works perfect with both simulated and real robots)