TY - GEN
T1 - A Robot to Shape your Natural Plant: The Machine Learning Approach to Model and Control Bio-Hybrid Systems
AU - Wahby, Mostafa
AU - Heinrich, Mary Katherine
AU - Hofstadler, Daniel Nicolas
AU - Zahadat, Payam
AU - Risi, Sebastian
AU - Ayres, Phil
AU - Schmickl, Thomas
AU - Hamann, Heiko
PY - 2018
Y1 - 2018
N2 - Bio-hybrid systems-close couplings of natural organisms with technology-are high potential and still underexplored. In existing work, robots have mostly influenced group behaviors of animals. We explore the possibilities of mixing robots with natural plants, merging useful attributes. Significant synergies arise by combining the plants' ability to efficiently produce shaped material and the robots' ability to extend sensing and decision-making behaviors. However, programming robots to control plant motion and shape requires good knowledge of complex plant behaviors. Therefore, we use machine learning to create a holistic plant model and evolve robot controllers. As a benchmark task we choose obstacle avoidance. We use computer vision to construct a model of plant stem stiffening and motion dynamics by training an LSTM network. The LSTM network acts as a forward model predicting change in the plant, driving the evolution of neural network robot controllers. The evolved controllers augment the plants' natural light-finding and tissue-stiffening behaviors to avoid obstacles and grow desired shapes. We successfully verify the robot controllers and bio-hybrid behavior in reality, with a physical setup and actual plants.
AB - Bio-hybrid systems-close couplings of natural organisms with technology-are high potential and still underexplored. In existing work, robots have mostly influenced group behaviors of animals. We explore the possibilities of mixing robots with natural plants, merging useful attributes. Significant synergies arise by combining the plants' ability to efficiently produce shaped material and the robots' ability to extend sensing and decision-making behaviors. However, programming robots to control plant motion and shape requires good knowledge of complex plant behaviors. Therefore, we use machine learning to create a holistic plant model and evolve robot controllers. As a benchmark task we choose obstacle avoidance. We use computer vision to construct a model of plant stem stiffening and motion dynamics by training an LSTM network. The LSTM network acts as a forward model predicting change in the plant, driving the evolution of neural network robot controllers. The evolved controllers augment the plants' natural light-finding and tissue-stiffening behaviors to avoid obstacles and grow desired shapes. We successfully verify the robot controllers and bio-hybrid behavior in reality, with a physical setup and actual plants.
U2 - 10.1145/3205455.3205516
DO - 10.1145/3205455.3205516
M3 - Article in proceedings
SN - 978-1-4503-5618-3
SP - 165
EP - 172
BT - Proceedings of the Conference on Genetic and Evolutionary Computation (GECCO 2018)
PB - Association for Computing Machinery
ER -