From jderobot
Jump to: navigation, search

Proyect Card[edit]

Project Name:

Author: Francisco Pérez Salgado [f.perez475@gmail.com]

Academic Year: 2018/2019

Degree: Computer Vision Master

GitHub Repositories:

Tags: Deep Learning, JdeRobot

More reading[edit]

This week has been another "theoric" week, lots of reading and getting myself informed about the state of the art in machine learning technologies.

This entry is going to be slightly shorter than the previous one because I've spend a lot of time trying to install a test CARLA and Udacity sim car.



I've been trying to install CARLA in three different computers with the same result

One of them was a virtual machine running ubuntu 16.04 so it's understandable. The other one had a pretty old fashioned hardware (my office desktop computer), so it's understandable too. And the last one I was afraid to install stuff to because it's not my machine an several colleagues use it for training neural networks. Solution? Install Ubuntu in my machine natively (making a partition also!!).

About CARLA documentation, there is not much to read about besides the installation manual. The idea behind CARLA is to have a heavy realistic simulated environment of a little town (with traffic, pedestrians,etc) to train a car in autonomous driving on a "as much as possible" realistic environment. This means that the simulator can let you change different aspects that affect the driving of a car (as it was the real life) as the weather, pedestrians crossing the street, unpredictable cars maneuvers, etc. So you can be ready for whatever situation it turns up. The whole simulator and assets are made with Unreal Engine which it is an awesome game engine used in some pretty good games.

CARLA offers you a client-server application:

  • Server will launch the simulated environment along with all the assets.
  • Client will launch a python script that lets you control the car manually/autonomously. But I need this to work first

I'm going to install Ubuntu on my machine natively and try again with CARLA.


This has turned out a little bit better. The only headache was to install all the dependencies required for it to work, but once it has been domesticated it worked pretty well. I was able to launch the simulated environment with a car in it. I must say that trying to control the car with the keyboard is hell! but I was able to record my driving into a bunch of images corresponding to a 3-tuple for center, right and left car cameras; and a .csv file recording my keyboard pressing pattern. With this, you must have all the information necessary for train a neural network. Nevertheless I don't know how to program a CNN yet, so I am just playing around.

Here's a little video showing how Udacity simulator works:


Also, this week I had to try to execute one of my colleagues component based on neural networks. It was the digit classifier from David Pascual [[1] here is his repo] which based on the images received from a camera, his component is able to recognize and classify handwritten numbers. This component is based on Keras API for Python. I had to use an example trained net(MNIST_net.h5) because his pre-trained network was not available. Here's a video showing how it works.

IoT (Internet of Things)[edit]

This week I've read lot more about IoT, because past week I was obsessed with autonomous driving. This field of machine learning is quite interesting because it submerges completely on home automation. There've been lots of improvements on this filed the last two decades however in my country (Spain) is not widespread because it's quite expensive. Diving on the technical aspects of the IoT I've realized that it has a part of machine learning but it has a bigger part of Big Data, because the core idea that you can subtract from IoT is the user data analysis. For instance:

  • what time the user leaves house on Monday mornings -> the house can turn off all the lights
  • which TV shows the users like the most -> the TV can alert the user when his favourite show starts
  • how much hot water the user use when taking a shower -> the water heater can heat only an aprox. amount of water the user is going to use.
  • what are the shopping habits of the user -> user's phone/pc/tablet could search for the best offers on the products the user use to consume.
  • and so on.

All of this is data, tons of data that needs to be processed in order to generate statistics, behaviours and more data for the user, in brief: Big Data, and I've only spoken about domestic environments, there is also industrial, business, etc. Imagine! Nontheless artificial vision is also applied on IoT in the manner of security autonomous systems, autonomous car driving, robotic vacuum cleaners, etc. There is tons of applications that need to process images in order to make systems as powerful as possible.

Here's an interesting TED talk about IoT by Jordan Duffy:

I don't know yet how to face my thesis, but there is a lot of interesting content here to investigate!

I think the next step is to program my own Neural Network and play with some images, or to play with home automation stuff, my tutor will say.

Getting started[edit]

The first step to enter in the fascinating world of the Artificial Intelligence is to learn about it... boring, right? Not really. Keeping in mind that the last thing I've seen about AI was like 8 years ago, I can say that I am a total noob on this matter. Nonetheless, my tutor (Jose María) gave me an interesting starting point: other folks thesis.

I've started reading an interesting article about the basics of Deep Learning, followed by Nuria Oyaga's job "Análisis de aprendizaje profundo con la plataforma Caffe" and David Pascual's "Study of Deep Learning Neural Networks with Keras ". These two readings gave me a global vision of the state of the art in regards to Deep Learning so I have now an starting point to head my Master's thesis. With those two readings complete (I sure will return to them in the near future), Jose Maria emailed me two new links about self driving cars Udacity open sources its driving car simulator and Toyota Research Institute self driving car simulator - Carla which grabbed all my attention and interest, I think I will point my thesis in this new technology, it looks really, really cool.

Those last articles, made me to want more, so I myself started to dive the internet looking for some other videos, documentation, webs, etc. about self driving cars and I ended up looking at the following content:

Siraj Raval udacity simulator - Siraj Raval is an emminence in terms of Machine learning, so it's always a good idea to see him working, or explaining something. Sentdex neural network in GTA V - Sentdex is also a good reference when it comes to learn python programming, so this series of live streams are very interesting to watch. - Muddassir Ahmed udacity project I found an interesting repository about udacity self-driving simulator of a guy that trained the car with awesome results.

This is all the documentation I've read/saw in term of autonomous driving, but it's not the only open front in machine learning... at all. I'm also interested in IoT (Internet of Things) because domotics always captured my attention (what a lazy guy I am), so I really don't know what to choose... yet!