Vision is important part of robotics and frequently it is unavoidable component of control loop. E.g. many clothes/garment handling algorithms rely on visual cues in deciding how to proceed. The question is - does simulation environments (Gazebo or some others) allow one to design world with robot and garment and simulate not only garment dynamics but simulate also what robot sees, how robot perceives garment in each simulation step? If it is not possible to simulate vision then how to simulate algorithms with vision as component of control loop?

Maybe simulation of vision can be good research theme? Are here some trend or good articles about it? Some initial projects that could be expanded?

Actually - it can be stated as more general question - is it possible to simulate sensors in Gazebo? E.g. food handling (soft-body handling) can involve tactile sensors. In principle Gazebo can calculate deformation and forces of soft-body and format these data as the simulated values of sensor readings. Maybe similar mechanism can be used for simulation of vision as well?

  • $\begingroup$ yes. I have tried usb cameras and Kinect $\endgroup$ – lanyusea Jul 23 '15 at 16:46
  • 1
    $\begingroup$ Alternatively V-REP simulator does. $\endgroup$ – acs Jul 24 '15 at 21:00

While Gazebo - and most other simulators - will allow you to simulate sensors, you have to realize that this the "weak" part of realtime simulators. Making an accurate simulated camera is really difficult. The result would therefore be great to test your integration and basic control algorithm behaviour (for instance that you move right when you should), but very limited to test the vision-based algorithm itself, as well as test fine print of the control laws.

The strategy I follow in these cases is to (1) create a "simulated component simulation" to test the control system in the simulator and (2) use log replay to test the vision component:

  1. Create a "vision component simulator". This component would output what the vision part of your system should output, but within the simulated environment and WITHOUT processing any image. Add noise / outliers as necessary in the simulated output.

  2. Use a framework that allows you to log data and replay it offline (as Rock or ROS do). You would then have test cases for your vision component that would test its behaviour in way that the simulator could never dream to achieve.


Your Answer

