Virtual plant modelling for deep learned robotics

Problem

One of the remaining problems in a deep learning recognition pipeline is the laborious process of gathering data sets. Taking many good pictures and let those being labelled by an expert takes a lot of time. A representative data set would consist of at least 1000 picture per single lighting condition per clas.

Solution

The solution is to create a digital twin model of a 3D plant using L-Systems, a grammar based modelling technique developed by Lindenmayer, a Hungarian theoretical biologist. Once the model has been created, and checked by an expert then lots of 2D projections (“pictures”) with label information is being generated instantly.

Video

Watch the video below to play the inference of a real footage video using only “fake” pictures from the 3D model  as training material!