April 2020

The first results were very positive: the system has been able to handle all the parts considered as test case in the project (LCD, PC Tower, Emergency lamps, Microwaves oven) with an easy orientation of the gripper and a smart adaptation to the different parts in random surface positions.

In disassembling phase similar problem needs to be faced but in this case the use of a collaborative platform and a small payload robot introduce some issues to take into account as the safe management of interaction with operators as well as the high variety of tool to be used for the dismounting of the parts: also in this case in preliminary test has been adopted the novel KENOS vacuum gripper by PIAB, certified as collaborative and the PLUTO automatic screwdriver by KOLVER (https://kolver.it/prodotti-elenco/4-Serie-PLUTO-MITO-NATO) integrated in a customized solution mounted directly on the robot flange.

The preliminary test conducted on the emergency lamp case showed very good result in terms of reliability and reconfigurability of the system related to the different application that can be executed. Next steps to be performed can take in consideration the use of an appropriate tool changer useful in order to enhance the potentiality of the system and cover also possible task as wire cutting or handling of small objects inside the parts.
The optimization of the current solutions, the adoption of other tools and related tool changer will be face during the rest of the project.

CERTH explains their work in AI-enabled Cell-level perception methods.

The objective of this task is to provide the envisaged robot with the ability to perceive the industrial environment so as to effectively collaborate with the humans and assist them with the Waste Electrical and Electronic (WEEE) device recycling process. Towards this direction, CERTH recorded multiple recycling procedures of four WEEE devices, namely PC Towers, Microwave Ovens, Flat Panel Displays and Emergency Lamps, focusing on the human-robot collaboration aspect during the WEEE device disassembly, thus creating a WEEE device component detection dataset by manually annotating a number of the recorded frames.

Given this dataset, CERTH deployed state of the art Computer Vision methods utilizing Deep Learning techniques in order to train a Convolutional Neural Network (CNN) capable of detecting the four WEEE devices as well as their respective components during the various stages of the disassembly process. Additionally, CERTH enhanced these methods by proposing an anchoring mechanism targeting specifically the detection of small objects.

GAIKER goes on with the definition of the real-world scenarios in which the novel WEEE recycling approach based on human-robot collaborations will be demonstrated and validated

GAIKER continues the progress in the task of outlining the scenarios of the pilot demonstrations, whose ultimate aims are to test and evaluate, in real operating environments, the performance of the solutions developed within the HR-Recycler Project to manage WEEE (Waste Electric and Electronic Equipment). This advance is supported by the regular interaction with the partners of the Consortium that, as evidenced after the 1st Project Review Meeting held in Brussels (Belgium) at the end of January 2020, are doing significant and consistent progresses in the development of hardware and software solutions.

In the definition of scenarios, GAIKER is considering how to integrate all the relevant technological advances generated during the implementation of the scheduled tasks, involving user-cases definition, factory-level modelling, cell-level perception, collaborative schemes definition and robotic equipment design and programming. GAIKER is also striving to keep in close contact with the other project partners to understand, update and gather any relevant information that can help in the definition of more precise, realistic and functional pilot scenarios.

Until now, the greatest contributions have been:
– The definition of the use-cases and current recycling scenarios by the industrial WEEE recyclers involved in the HR-Recycler.
– The initial modelling of the factory floor and human workers, including work cell conceptual configuration, virtual factory 3D simulations and process orchestration.
– The demonstrated capabilities of advanced artificial intelligence-based perception in terms of automatic object recognition and manipulation, and human motion prediction.
– The preliminary definition of the robotic equipment, as articulated arms and automated vehicles, and the software for controlling robotic motion and interactions.
– The establishment of foundations for human-robot safe collaboration and adaptative interaction.

GAIKER has been thoughtfully registering, analyzing and integrating all these advances, particularly focusing on those aspects most closely linked to the key performance indicators defined to evaluate the HR-Recycler system. Thus, GAIKER has identified the main elements of the scenarios, including work cells and plant floor layouts, robot and worker movements (sifting and manipulation), detailed procedure requirements, material flows, human-robot interactions, safety issues and process orchestration. Based on these data GAIKER has begun to build the conceptual model for the pilot scenarios. This is, naturally, a flexible and additive model and GAIKER will be aware of any further novelty and progress in the projected tasks.

Robotnik explain their contribution to HR-Recycler project.

HR-Recycler is a multidisciplinary project aiming to improve the recycling capabilities of european countries. More specifically, it is targeted at integrating tight Human-Robot collaboration in the recycling process of WEEE, Waste Electrical and Electronic Equipment. E-waste is considered the “fastest-growing waste stream in the world” [1].

WEEE recycling is a time consuming and effort process, due to the variety of the devices to be recycled and the different components and materials they are made of.

One key issue in recycling processes is the routing of raw and disassembled materials and components through the recycling plant. This is a resource consuming process as materials and components are processed in several parts of the factory. In a typical recycling process, unclassified devices arrive at the recycling plant in large trucks. After sorting, each device has to be transported to its recycling station where it is separated into its component parts. Components can be either further separated or be ready to move them to their final destination.

Here is where Robotnik enters into action. Routing materials inside a factory has to be done in an affordable, efficient and safe manner. Moreover, with the advent of the fourth industrial revolution robots have to be able to operate collaboratively with humans sharing the same space.

Affordability is achieved by providing a robot that is able to carry the same baskets as the factory already has, decreasing the number of changes that have to be made in the factory. Efficient is provided by state of the art navigation algorithms, as well as planning algorithms for the overall factory. Collaboration comes through the use of human-aware navigation, but also increasing the communication between the robot and the human about the intentions of the former. Safety is achieved through applying strict measures, sensors and actuators that comply with the latest safety standards

All of this effort is directed by Robotnik towards creating a new robot: RB-Ares.

Robotnik RB-Ares

RB-Ares mission will be to pick and place EURO-pallets at floor level and route them through the factory with the required features of affordability, efficiency, safety and collaboration with humans. To accomplish this mission, RB-Ares is equipped with state-of-the-art actuators and sensors.

RB-Ares is powered by ROS, as well as Robotnik’s proprietary technology for navigation, localization and Human Machine Interface, which allows an easy configuration, programming and integration of the robot in different applications and Fleet Management Systems, as 4.0 industry demands. This is the main feature of Collaborative Mobile Robots as RB-Ares, an intelligent mobile robot that assists humans in a shared workspace and support to the optimizacion of the processes inside industry.

[1] World Economic Forum. (2019). A New Circular Vision for Electronics: Time for a Global Reboot, (January), 24. Retrieved from

SADAKO visits INDUMETAL’s facilities for a second round of recordings.

Last year, on July and September, a first round of recording sessions was carried out at Bianatt’s facilities and conducted by CERTH.

Indumetal was selected for a second round and, this time, Sadako was the partner in charge of the coordination and the recordings. On 17^th and 18^th December, Sadako arrived at Indumetal’s facilities, loaded with all their cameras equipment and ready for two hard working days. On the other hand, Indumetal arranged a specific area for the classification and disassembly capturing and, following Bianatt’s experience and indications, a metal structure for camera installation and a correct illumination were provided as well. The use cases selected for this round were microwaves and PC towers.

Additionally, human motion analysis and prediction were recorded in order to detect human actions at cell-level and hand gestures for human robot interaction and to identify task on the human worker. For that, different communication gestures and some of the described actions on D3.1 were performed by Indumetal’s workers.

Do we recycle together? Human-robot collaboration for recycling

With the Information Society and the steadily growing consumption of electrical and electronic equipment (EEE) worldwide, society is facing the challenge of dealing with increasing amounts of waste electrical and electronic equipment (WEEE) in a sustainable and responsible way.

Nowadays manufacturing companies are going through an increasing public and government pressure to reduce the environmental impact of their operations. But when dealing with WEEE, some difficulties arise in classifying and dismantling electronic devices.

How to improve the current process that is mainly manual and time consuming? Can technology help in the improvement of collaboration between humans and robots? A novel solution is to promote cooperation between an operator and robots in this type of processes.

More specifically, the overall goal of HR-Recycler project is to create a hybrid collaborative environment, where humans and robots will harmoniously share and undertake, at the same time, different processing and manipulation tasks, targeting the industrial application case of WEEE recycling.

This goal will be achieved through Human Robot Collaboration (HRC) systems. In the words of Sara Sillaurren, TECNALIA´s project coordinator: “In this new environment in which human and robot have to work collaboratively, it is essential that process design take into account the state of the human for a better understanding between the two of them.”

Within the scope of this project, TECNALIA will bring Human Factors and User experience Lab, which combines physiological sensors and technologies for monitoring human behavior.

Through the usage of wearable devices and platform in this laboratory, and with the collection of psico-physiological data from users, human behavior can be better understood when collaborating with robots and thus the system can be adapted so that processes are more fluid and reliable.

According to this, all types of signals captured give us very interesting information about the user and referred to the activity they are performing. For example, they can give us cognitive information, that is, if the user is paying attention and if the user is able to retain the information we are showing. Another type of information that can be measured is the emotional one. For example, the affective valence or degree of attraction towards the activity they are completing, or the activation and emotional impact, that measure the level of calm or excitement of the user. Finally, there are other types of metrics such as visual attention (that is, what first calls the attention of the user) or the implicit association (between a concept and an attribute).

Furthermore, considering the multiple factors that affect Human Robot Collaboration, TECNALIA is going to define the experimental studies targeting end- user with a purpose of understanding trust in collaborative disassembly and the validation of trust factor model to improve it.

For doing so, a variation of the inspection game has been designed in order to expose participants to different trust stimuli towards machines. Inspection game is a mathematical model of a non-cooperative situation where an inspector verifies that another party, adheres to legal rules instead of shrinking work duties. This experiment will allow us to detect how (un) predictability and prior experiences affect trust.