Club-PushkalKatara

From jderobot
Jump to: navigation, search

VisualStates Tool

VisualStates is a tool for programming robot behaviors using automata. It combines a graphical language to specify the states and the transitions with a text language (Python or C++). Hierarchical Finite State Machines are used to program robot behaviors and each state is a reactive controller which can be active or not at a particular instant. It automatically generates a Robotics Operating System node as output, in C++ or Python, which connects to the configured drivers at runtime and implements the automata. This tool speeds up the development of robot applications, reducing the code that has to be manually created.

People Involved

  • Pushkal Katara (katarapushkal [at] gmail [dot] com) [GSoC Student]
  • Okan Aşık (asik [dot] okan [at] gmail [dot] com) [Mentor]
  • José María Cañas Plaza (jmplaza [at] gsyc [dot] es) [Mentor]

Important Links

Development Status

S. No. Sub-Project Status Timeline
Phase 1: Development of Library of Automata May 27, 2018 - June 13, 2018
1. Feature to Import an Automata into Developing Automata Done
2. Demo example for Import. Done
3. Development of States as Library of Automata Done
Phase 2: Automata Namespace June 16, 2018 - July 10, 2018
1. Namespace Strategy while developing automata from scratch Done
2. Namespace Strategy with New Automata. Done
3. Namespace Strategy with Importing Automata Done
4. Namespace Strategy Demonstration Done
Phase 3: Release For VisualStates Tool July 14, 2018 - August 12, 2018
1. Developing Robotics Operating System Package Done
2. Developing Debian Package
3. Developing AppImage
4. Developing Demo Examples
5. Functionality to easily Edit State Parameters

Weekly Documentation

Week 11: July 26 - August 1

Discussions

  • Local Namespace Functionality XML Tag - The current implementation of namespace parsing contains namespace tag on the same level of states and transitions. The namespace tag should be in the parent state.
  • Rospackage branch merge with import functionality.
  • GSoC Documentation
    • Wrap up with the import functionality and soon merge it with master branch.
    • User documentation: Some of the examples related to Toyota Prius car were discussed. As the ToyotaSpawn example was created using imported state, a more detailed user perspective documentation is required. Another example could be the "Dancing Car".

Week Task

  • Create pull request with import functionality with namespace strategy
  • Implement namespace strategy with the import automata option.
  • Merge rospackage branch with the import functionality branch
  • Look into user documnetation and examples

Week Progress

  • Pull Request
  • Merged Rospackage branch with the import functionality by Sir Okan.

Week 10: July 19 - July 25

Discussions

  • Implmenetation of Global Namepspace functionality
  • Implementation of Local Namespace functionality to OpenAutomata and ImportAutomata

Week task

  • Parsers, Generators for Local Namespace Functionality
  • Global Namespace functionality implementation

Week Progress

  • Global Namespace Implemented
  • Local Namespace Implemented
  • New automata development with Local Namespace Strategy
  • Open automata development with Local Namespace Strategy
  • Commit for above implementation - Commit

Week 9: July 12 - July 18

Discussions

  • Final strategy explanation for Namespace Functionality.
    • Global Namespace Concept - Functions and Variables available to all the states and transitions through the whole automata.
    • Local Namespace Concept - Functions and Variables available to all the states and transitions on the same level of automata.
  • Namespace Implementation Demo

Week Task

  • Implement Local Namespaces to New Automata Creation
  • Pull request and merge into VisualStates/master
  • Shift to VisualStates/pushkalkatara fork from GSoCColabRepository for development.
  • Documentation to demonstrate namespace functionality.

Week Progress

Phase 2[edit]

Week 8: July 4 - July 11, 2018

Discussions

  • Strategy to implement namespace functionality.
    1. XML tag for namespaces under which all the namespaces present with ids.
      • State and Transitions with new variable, namespaceid to tell which namespace to use.
    2. Under the state tag, like transitions, new tag namespace. Same recursive parsing as transitions.

Week Task

  • Implement local namespace functionality with automata levels. Two strategies -
    • Recursive parsing of namespace like transitions.
    • Global namespace as individual single namespace.

Week Progress

Week 7: June 27 - July 3, 2018

Discussions

  • Namespace Tag added to XML.
    • Separate namespace for every level of automata or based on user selection?
  • State parameters display while developing states.
    • Toyota Prius Car example. throttle, steer, and brake is the three important parameters which need to be tuned in order to develop examples.
  • Video development on yolo v3 with Gazebo 9 and toyota prius car to detect stop sign on road and stop the prius car autonomously.

Week Tasks

  • Implement new namespace functionality with every import.
  • More approach to solve the issue of conflicting methods and variables.
  • Develop example for Toyota Prius car and darknet ros package.

Week Progress

  • New branch name-mangling to update read and write namespaces through XML format.
  • Integrating darknet-ros package to the toyota prius car repository.

Week 6: June 21 - June 27, 2018

Discussions

  • Fixed all the small bugs in "Import Automata" pull request.
  • Prius package update to Gazebo 9, modified the plugin according to Gazebo 9 updates. Removed Prius's model from default launch world model as spawn is from the launch file.
  • Significant example states for Prius.
    • ros_yolov3 package integration to detect the stop sign and stop the car. Pretrained Yolo v3 weights trained on coco dataset contains only Stop sign. Example. the output is bounding box with the probability and x,y,z,w coordinates.
  • State Parameters Display
    • What parameters are required to be displayed? Ros Messages, Auxiliary variables.
    • Approach 1 - Runtime GUI to display all the ROSTopics with auxiliary variables.
    • Approach 2 - Rqt Integration with custom Rqt plugin customized for VisualStates.
    • RQT Example
  • State Parameters Update
    • Runtime update or development time update?
    • Approach 1 - Publishers to ROS Messages to update on runtime.[Runtime Update]
    • Approach 2 - Display all ROS Messages and Aux Variables at the time of development.[Development Update]

Week Task

  • Resolve Merge Conflicts and merge into the main VisualStates repository.
  • Easy example to resolve data mangling issue.
  • Read about Automata Compositions
  • State Parameters display examples.

Week Progress

  • Development repository merged into the main VisualStates repository.

Week 5: June 13 - June 20, 2018

Discussions

  • Add publishers and subscribers for packages which are not built in ROS Kinetic main workspace.
  • Current Status of Import -
    • State and Transitions are imported recursively.
    • Configurations are imported.
      • RosConfig
        • Topics (Publishers and Subscribers) filtered by TopicName are imported. If they already exist in the developing automata, they are not imported.
        • Build Deps and Run Deps are imported. If they already exist, they are not imported.
      • JdeRobotConfig
        • Interfaces are imported. If they already exist they are not imported.
    • Function and Variables are imported, if they exist they are not imported. (String compared)
  • Review and Merge pull request in official VisualStates repository.
  • Every Level of automata be a class or a namespace. One class for all publishers and subscribers named as RosNode. Every class extends the RosNode class to access its variables.
  • The methods and variables of the RosNode class would be generated according to the Python double underscore method and variable mangling scheme, which mangles the methods and variables by appending class name to it.
  • Gazebo 8 and Gazebo 9, simulate Toyota Prius Car through ROS.
  • Develop easy example to control Prius Car using VisualStates.

Week Task

  • Develop basic self-driving car states. Look upon sensor data, image topics. Develop Gazebo model with control plugin.
  • Add functionality to add external ros package to develop related publishers and subscribers.

Week Progress

Phase 1[edit]

Week 4: June 6 - June 12, 2018

Discussions

  • Importing Functions and Variables while importing states.
    • Conflict: Function and variable name may be similar and collide while generating code.
    • Solution Strategy 1: While importing, check if the function or variable name previously exist, if exists, parse it with a new name and further add with all other functions.
    • Solution Strategy 2: Encapsulate all functions and variables in a different namespace or class. Every level of automata would be a ROS node. Example.
    • Solution Strategy 3: Encapsulate all functions and variables in specified namespace or class, let user choose the namespace for the particular state. (Every namespace would be a rosNode).
  • Issue with 2nd Strategy:
    • When two different RosNode(of two different level states) would have publishers on a common topic.
    • Very deep levels of automata would create code ambiguity as many ROSNodes would be created.
  • Issue with 3rd Strategy:
    • When two different RosNode(of two different level states) would have publishers on a common topic.

Week Task

  • Implement import functions and variables functionality.
  • Learn about C++ Name Mangling
  • Implement function and variable import using C++ Name mangling strategy.
  • Implement function and variable import using namespace or class strategy.
  • Make pull request on the main VisualStates repository with the most suitable import approach.
  • Make a demo example to demonstrate overall import functionality.
  • Illustrate the demo example through a video.

Week Progress


Week 3: May 30 - June 5, 2018

Discussions

  • Import State A, B, C Case Scenarios -
Case Developing State Importing State Final State
Case 1: Simple Developing State,
Simple Importing State
<state id = 1> </state> <state id = 1> </state> <state id = 1> </state>
<state id = 2> </state>
Case 2: Complex Developing State,
Simple Importing State
<state id = 1>
<state id = 2></state>
</state>
<state id = 1> </state> <state id = 1>
<state id = 2></state>
<state id = 3></state>
</state>
Case 3: Simple Developing State,
Complex Importing State
<state id = 1> </state> <state id = 1>
<state id = 2></state>
</state>
<state id = 1>
<state id = 2>
<state id = 3></state>
</state>
</state>
  • Considering Transitions:
Before Importing After Importing
Developing State
<state id = 1>
<transition id = 1>
<originid>1</originid>
<destinationid>2</destinationid>
</transition>
</state>
<state id = 2></state>
<state id =1>
<state id = 3>
<transition id = 2>
<originid>3</originid>
<destinationid>4</destinationid>
</transition>
</state>
<state id = 4></state>
<transition id = 1>
<transition id =1>
<originid>1</originid>
<destinationid>2</destinationid>
</transitionid>
</state>
<state id =2></state>
Importing State
<state id =1>
<transition id = 1>
<originid>1</originid>
<destinationid>2</destinationid>
</transition>
</state>
<state id=2>
</state>
  • If the parent state is active, child behavior would be active.
  • Updating IDs would work like Assembler's Lookup Tables.

Week Task

  • Refining Import Task
  • Considering Transitions, updating origin and destination ids according to the imports.
  • Generating code and lookup on configurations, functions and aux variables.

Week Progress

  • Import Update
  • Issues
  • Import Feature Updates
  • Transition ids updates according to imported state ids.
  • Complex Import feature working.
  • Folder scripts chmod issue resolved.
  • Import checks if imported libraries exist, if not appends to the libraries list.
  • Import checks if imported configurations exist, if not appends to the config list.
    • Updates topics, run, build dependencies in rosconfig
  • Bug - Same Topic Name Bug
  • Todo -
    • Update Jderobot Config with import.
    • Update imported auxiliary variables.
    • Update imported functions.


Week 2: May 23 - May 29, 2018

Discussions

  1. Import State A, B, C into State D Strategies:
    • Import Option in the Figure Menu
    • Select XML using FileDialog
    • Select Area on the automata to import.
    • Recursively add states and transitions on respective active states and automata scene.
  2. Import State Implementation Details:
    • Iterate State and Transition IDs and assign new IDs to imported automata recursively.
    • Add new OperationType as IMPORTSTATE and OperationData to pass the parsed state items.
  3. TurtleBot - VisualStates Obstacle Avoidance Tutorial needs to be updated for the standalone version of VisualStates, currently it depends on JdeRobot repository. There would be release of all the Gazebo related data as a standalone debian repository that would resolve this issue.
  4. Toyota Car Simulation in Gazebo
    • Nvidia Docker -> Standalone ROS Package
    • 4 Basic Possible States
      1. MoveForward
      2. MoveBackward
      3. MoveLeft
      4. MoveRight
    • Control Message Definition
      • Header header
      • float64 throttle
      • float64 brake
      • float64 steer
    • Useful Sensor Data
      • Front, Back, Right, Left Camera Feed (/prius/*image-topics)
      • Laser Data
    • Gazebo Simulation and RViz Sensor Data Visualization
  5. PR2 Simulation
    • PR2_Gazebo Package with Teleop Control
    • PR2 Teleop and PR2 RViz Data
    • Basic States
      • MoveForward
      • MoveBackward
      • MoveLeft
      • MoveRight
      • YawRight
      • YawLeft
    • Sensor Data And Useful Topics
      • Left And Right Forearm Camera Image
      • Main Camera /prosilica/image_raw
      • Left and Right Gripper Controller Action (Was not working)
      • Head Controller Action
      • Tilt Scanner Service (Was Working)
      • Narrow and Wide Stereo Cam
      • Torso Controller Action
      • Tilt Hokuyo Laser (Was working)

Week Task

  1. Implement ImportState functionality.
  2. Develop different cases which may occur while importing different states.

Week Progress

  1. Implemented ImportState functionality
  2. TreeView Child state remove bug resolved.
  3. Recursively import complex state into developing automata.

---

Week 1: May 9 - May 22, 2018

Discussions

  1. Toyota Prius Simulation On Gazebo 8 through ROS Kinetic.
  2. International Conference on Intelligent Robots and Systems Competitions
  3. Technofest Istanbul

Week Task:

  • Finding and fixing issues related to TurtleBot tutorials on JdeRobot website.
  • Understanding VisualStates tool source code.
  • Creating simple automata with simple states A, B, C and figuring out an approach on how to import A, B, and C into newly created automata D.
  • Figuring out an example to demonstrate with VisualStates tool in long term. Learning about PR2 Robot, Toyota Car Simulations.

Week Progress: