Directors : R. Marlet, E. Dokladalova
Autonomous robots operating on a construction site require high vision capabilities. They have specific requirements, that are substantially different from the extremely controlled case of robots in a factory, with well-identified and repetitive tasks, in a fixed environment where sensors can be placed both in relevant and secure places, and calibrated once and for all. How computer vision and machine learning techniques can be applied and developed in this context, in particular for the specific construction scenarios investigated in DiXite project?
Context : Industrial needs and image processing techniques
DiXite organized a workshop January 31st 2019 on Machine vision for construction sites. The main objective was to match up advanced technologies in image processing developed by DiXite researchers with actual needs of industrial partners of the project (Vinci, Artelia, Saint-Gobain). Challenges related to the digitalization of the construction sites where the computer vision would be able to take up technological challenges :
- Short and mid-term challenges deal with tracking construction site resources; construction site safety; innovative inspection systems; conformity supervision.
- Long-term challenges are mainly related to the robotization of building sites; automation of construction processes; robotic assembly of buildings; automated 3D printing of buildings; human – robot collaboration.
Concerning short and mid-term challenges, the state of the art already provides some answers, that however have to be put into practice on actual cases in construction sites to discover possible remaining issues. Practical actions in this direction include scientific support for experimenting with real data on real conditions as well as method improvements or adaptations, based on specific targeted projects of the industrial partners. On the other hand, the discussion confirms that robotization of construction sites remains a major challenge for future construction sites. It is well known that the construction industry is one of the least automated industries that feature manual-intensive labor but, at same time, the construction tasks are notoriously difficult to automate.
Research theme : Accurate and robust vision for complex assembly with robots on construction sites
In the context set out above, we propose a research theme concerning the task of assembling complex structures with robots under the difficult, uncontrolled conditions of a construction site. The focus is on accuracy and robustness, which are the bottleneck of current methods to address this construction task, especially in a calibration-free context. We consider scenarios of wall and structure constructions of increasing difficulty, first with identical cuboid bricks up to complex interlocking blocks that all differ with only slight variations.
The work can be inspired by the recent advances in AI and deep learning developed in the IMAGINE team [1] completed by some recent work done at ESIEE [2]. Possible extensions include issues related to visual servoing to grab and, more importantly, to place an object into a structure being assembled. This task may also need to resort to force sensors in addition to cameras.
References
1. Loing, V., Marlet, R. & Aubry, M. Int J Comput Vis (2018) 126(9):1045-1060. https://doi.org/10.1007/s11263-018-1102-6
2. R. Rodriguez Salas, E. Dokladalova, P. Dokládal. Rotation invariant CNN using scattering transform for image classification. submitted to ICIP 2019. hal-02008378
