- 1 Introduction
- 2 Software infrastructure
- 3 Programming the cat drone into cat.py
- 4 Sponsors
- 5 Previous Competitions
This year the third edition of this competition will be held as part of the International Conference on Intelligent Robots and Systems (IROS 2018). They will take place in Madrid 1-5 in October 2018.
Drones are aerial robots that have gained popularity in recent years. They were born in the military field and with the lowering of their costs have opened up possibilities for commercial use in several civil applications such as infrastructure monitoring, agriculture, surveillance, event recording, etc.. They are robots, so that, they are composed of sensors, actuators and processors on the hardware side. Its intelligence, however, lies in its software.
This championship challenges the programming of a quadricopter (cat) to search, chase and stay close to another aerial robot (mouse). In the Robotics Laboratory of the Rey Juan Carlos University we have prepared the championship infrastructure using the Gazebo simulator and the JdeRobot software platform, which simplifies access to sensors and actuators and allows the programming of the cat's behaviour in Python language.
The environment needed to participate and test is easy to use and install. It consists of (a) the Gazebo simulator, (b) the JdeRobot software platform and (c) JdeRobot-Academy. The first and the second are installed as packages on Ubuntu Linux (16.04), the third is a collection of Python programs that are downloaded from GitHub. They can also be installed on Windows and MacOS computers using Docker containers.
Robotics simulators are used to create virtual worlds and observe how an emulated robot acts in this world. With them, robotic applications can be programmed and tested not depending on a physical robot, making testing cheaper and less dangerous. If the robot crashes or behaves strangely in a way that was not anticipated, it is possible to restart the simulation without damaging the actual model (or people close to it). Some of these simulators represent the 3D worlds and recreate their physics (gravity, pushing, collisions, etc.), realistically emulating the movement of the robot in different scenarios.
Gazebo is a very complete simulator that distributes the OSRF (Open Source Robotics Foundation) as free software and has become an international reference in robotics. It has robot models that can be used directly, in addition to including the possibility for the users to create their own robot and environments (e.g. a RoboCup football field, a village or the interior of a building) including textures, lights and shadows. It incorporates several physics motors and has a wide range of sensors such as cameras, lasers, contact sensors, IMU, etc. This simulator was selected four years ago by the North American DARPA for its international competition DRC, contributing with several millions of dollars in its development.
We have simplified its installation and it is done automatically when the JdeRobot Debian package is installed.
JdeRobot is a free software platform for the development of robotic and machine vision applications. This project is being maintained by the Robotics Laboratory of the Rey Juan Carlos University since 2008. It has recently been funded by Google and is 100% compatible with ROS (specifically Kinetic ROS).
It offers a programming environment based on components in which the robotic application consists of several components. Each is executed as a process and interoperates with each other through ICE middleware communications (Internet Communications Engine, ZeroC open source software) or through ROS messages. They can be written in different languages (C+++, Python, Java, etc...) but they interoperate without problems.
This platform abstracts the programmer from access to the robot's hardware devices, such as reading the sensors or sending commands to the motors. It simplifies the access to them, obtaining the reading of a sensor (even if it is remote) through a function call.
The source code for JdeRobot is in GitHub and to simplify installation we have prepared Debian packages. To install it from the package follow these instructions,, specifically the sections Requirements: Linux package sources and Installation for running JdeRobot.
This software is a collection of robot programming practices and exercises, including one for cat and mouse play. It's basically a set of programs in Python.
To install this software, simply clone your GitHub repository:
cd mkdir championship cd campeonato git clone https://github.com/JdeRobot/Academy.git
Test: 'cat and mouse game' with drones
The test consists of programming in Python language the intelligence of a quadricopter drone we call 'cat'. Its aim is to search, chase and stay close to another similar quadricopter we call 'mouse'. The cat has two cameras (one front and one belly) and a 3D position sensor as sensory equipment. Its actuators are the engines of its propellers. The mouse is red, for an easier detection in the images.
A 3D world has been prepared for the test in the Gazebo simulator. In this new world we will have two robots: the quadricopter AR.Drone which acts as a mouse (which has a red colour) and the cat quadricopter. There are no other obstacles nearby in the world but the ground. The following video shows an example cat chasing a mouse:
To launch the test environment you have to:
- Run the Gazebo simulator with the appropriate world file, which starts the test scenario.
- Run the mouse component that governs the behavior of the mouse drone.
- start the cat.py that governs the behavior of the cat drone.
- Optionally, you can execute the referee component that measures the score.
Once the infrastructure is installed we can start Gazebo with the world file of the scenario. From a terminal runs:
If your computer is running low on resources, try starting the simulator without a graphical interface:
We have prepared a component in JdeRobot-Academy that allows to teleoperate the AR. Drone, insert into it and execute the code of each participant. This component is called cat.py, it obtains the sensory data of the drone (images, inclinometers, compass, etc.) and displays a graphical interface for the operation of the AR.Drone and the visualization of its sensory data.
Your test code will be embedded within this component, and it is your code that will make the appropriate movement decisions based on sensory information. This is where the quadricopter's intelligence lies and whether his behaviour is good or not. The component starts in teleoperation mode, so you can control the movement of the quadricopter manually. With the vertical slider you can raise or lower the height and with the crosshead you can rotate and/or advance it horizontally.
- Before running your algorithm click on the Play button, and cat.py enters automatic mode by periodically invoking your code inserted in MyAlgorithm.py, the execute method. It calls it about 10 times per second, allowing your software to be in control in every moment of the drone's movement.
- To end the execution of your algorithm press the Stop button, which will take you back to cat.py in teleoperation mode.
Now in another terminal we will run the cat_py component:
cd ~/campeonato/src/drone_cat_mouse/cat_py python2 cat.py cat_conf.cfg
Programming the cat drone into cat.py
The behavior of the quadrichopter cat will typically have a perceptual part and a control part. The perception part collects the sensory data (mainly the camera) and analyzes them, extracts information from them. The control part decides which movement is appropriate and issues commands to the robot's motors.
Cat.py is iterative in nature, it continuously executes iterations and is perceived and controlled in each of them. The Cat.py component relies on JdeRobot's ICE interfaces for dialogue with the Gazebo simulator. In particular with the Gazebo plugins that materialize the quadricopter sensors and actuators. All this dialogue is hidden from the programmer, who simply has a few methods in Python as the programming interface for reading data from the camera, GPS sensor and commanding the quadricopter motors.
To insert your code in cat.py and thus implement your algorithm we recommend the following steps:
Open the cat.py file with your programming environment and go to the MyAlgorithm class in the MyAlgorithm.py file.
Go to the definition of the execute() method.
First of all it is recommended to implement a method that allows us to detect in the image obtained from the mouse drone, obtaining its position in pixels.
Typically the center of mass of the pixels that pass the red filter can be a good approximation of the center of the mouse robot (there are better ones).
With the mouse robot detected in the image, the next step is to decide which motion is appropriate to command the cat motors with the sendCMDVel() method. Here are many ways to decide it: case based control, PID control, automata, etc. This is where your skill comes into play.
In addition to the following description of the drone programming API in Python, you have more details about it here
The images of the front camera of the cat are obtained from the MyAlgorithm class. The getImage instruction returns the image from the active drone camera and stores it in the droneImage variable. From this moment on, this variable contains an image that we can treat as we wish. For example using the OpenCV machine vision library.
droneImage = self.sensor.getImage()
You can change the active camera of the drone by invoking the following method:
For this test we will need to detect the mouse robot. The mouse is red. It is typically located very well within the images using a conveniently adjusted HSV filter.
Reading position and orientation in 3D (GPS and inclinometers)
Two reference systems are used on the championship stage: an absolute system and a solidary system with the quadricopter. These are shown in Figure:
Mouse tracking can be addressed without knowing the absolute position of the drone in the world. However, it can be useful if you want to optimize your mouse search when you don't see it in the image. To obtain the 3D position you have the getPose3D method. The following instructions deliver the X, Y and Z coordinates of the drone within the Gazebo world.
self.sensor.getPose3D().x self.sensor.getPose3D().y self.sensor.getPose3D().z
To obtain the orientation (quadrature) of the drone:
self.sensor.getPose3D().q0 self.sensor.getPose3D().q1 self.sensor.getPose3D().q2 self.sensor.getPose3D().q3
Commanding motion to the drone's engines
The sensor class within cat.py includes several methods to interact with the drone and command its movement. In particular the sendCMDVel() method sends commands for travel speed and rotation speeds. The travel speeds follow the reference system shown in the figure above, in conjunction with the drone itself: vx front/rear, vy left/right and vz up/down. The speed of yaw marks the rotation around a vertical axis, perpendicular to the plane of the drone. The sendCMDVel() method receives 6 parameters: vy, vx, vz, yaw, roll and pitch. Each of the values must be indicated between -1 and 1. The roll and pitch values have no effect on the simulated world of Gazebo.
For example, the following command commands the drone to immediately move forward at a speed of 0.5 (at half power). This order is active until you are instructed otherwise.
The sendCMDVel() method allows you to command different commands at the same time. The following command causes the drone to move forward at a speed of 0.5, move to the right at 0.4 and simultaneously rise on the Z-axis to 0.2 and rotate on the Z-axis at a speed of 0.1.
Finally, the following instruction can be used to stop the movement of the drone.
To test and tune your cat you can leave the mouse still, but you can also teleoperate the mouse with the JdeRobot uav_viewer tool connected to it, while at the same time cat.py connects to the cat and executes the code you have prepared.
We have also prepared a training mouse that implements most of the movements that scoring rounds mouse will have. This mouse has an incremental difficulty, starting with very simple movements at minimum speed, continuing with more complex movements at medium speed and ending with even more complex movements at maximum speed.
You can download it from here, which includes several mouse (for 64-bit machines) for you to practice with. Once downloaded, copy that compressed file to the appropriate directory of JdeRobot-Academy and unzip it:
cp ratones.tgz ~/campeonato/src/drone_cat_mouse/mouse cd ~/campeonato/src/drone_cat_mouse/mouse tar -zxvf ratones.tgz
Every time you want to run the mouse, already with the scenario in Gazebo started, run:
cd ~/campeonato/src/drone_cat_mouse/mouse ./trainning_mouse trainning.cfg
cd ~/campeonato/src/drone_cat_mouse/mouse ./q1_mouse q1.cfg
During the competition different mouse programmed by the organization will be launched, some simpler others more sophisticated, which will be the same for all participants.
A referee evaluates your algorithm
To determine the score of your algorithm we have created a referee who measures the actual instantaneous distance your cat is from the mouse for 2 minutes. In addition, it shows on the screen the time evolution along the duration of each round. When it is below the threshold considered close to it, it paints it green and when it is above it, red. As long as the cat is near the mouse the score will increase, if your cat is not close enough to the cat it will not score. The referee consists of a plugin and a visor.
To throw at the referee you have to do it (right after you start the stage, the cat and the mouse, not before). On a terminal it runs:
cd ~/campeonato/src/drone_cat_mouse/referee python2 referee.py referee.cfg
Organization, evaluation and award
In order to participate it is REQUIRED to fill out this REGISTRATION FORM . Each participant must be registered individually. For announcements and communication with participants we will use the mailing list firstname.lastname@example.org (which will be deleted after the championship). To send messages, you must fill in the registration form beforehand and the organization will add you to this mailing list.
Then you have to program, properly retouching the file MyAlgorithm.py (the execute method), and send it to the organization by email (josemaria.plaza AT urjc.es) before Sep 30, 2018 (at 12h) along with the personal data. After that day, the organization will publish the list of all participants on the website. The championship finals will be held on 5 Oct 2018, as part of the National Robotics Days. That day the final session (Q3) will be streamed on YouTube. If the volume of participants allows it, also the qualifying sessions Q1 and Q2.
The evaluation is to start your robot cat in different configurations (2 rounds) with the robot mouse that we have prepared and in a machine of the organization. Your MyAlgorithm.py file will be added to cat.py to govern the behavior of the quadricopter. The machine and mouse behaviour is exactly the same for all participants. In each 2-minute round, your robot starts in a different relative position relative to the mouse, it must search and chase it. The referee component measures how many seconds of these runs your cat has been close enough to the mouse and displays it on the screen. That is the score.
The championship is organised in knockout stages, where your cat will gradually face more difficult to follow mouse. After the first round (Q1), the best 20 cats move on to the next round, in the second round (Q2) the best 10 cats and in the last round (Q3) the final positions in the championship are decided. The championship is won by the cat that is close to the mouse for the longest time in the accumulation of the two rounds of Q3. The jury will be a commission of experts from the Rey Juan Carlos University, which may leave the prize empty if the solution is not of sufficient quality and disqualify the participants who repeatedly collide with the mouse. Their decisions are unquestionable.
The aim of the championship is to provide a nice challenge and an opportunity to measure your solution with other fellow competitors.
Frequently Asked Questions
Who can participate?
Anybody, it's an open championship. It is aimed at university engineering students from all over Spain (undergraduate, master, doctorate...) but is open to any participant.
Can I use other middleware to access the air robot?
Not for the championship, no. Beyond the championship, use whatever you want :-). To make the comparison fair all participants will use the same version of the Gazebo simulator and the same version of JdeRobot middleware. In fact they have to deliver their code in a MyAlgorithms.py file that is tested inside the cat.py component of JdeRobot-Academy.
In JdeRobot Foundation we have organized several robotics competitions proposing some challenges to foster robotics and computer vision.