Tactile and haptic exploration to overcome uncertainty!

By | Blog | No Comments

Exploring the Environment: Haptic regression

The HR-Recycler project aims at creating a hybrid collaboration environment for disassembly of Waste of Electrical and Electronic Equipment (WEEE) materials, where humans and robots collaborate with the goal of disassembling WEEE products. Even for a single device type (e.g., microwave), recycling plants are interested in many variations that come in different shapes, weights, and disassembly steps. The complexity is further exacerbated when the mechanical structures of these devices are damaged which is often the case for disposed WEEE materials. When disassembling devices without knowing their construction plans, fixtures are easily not detected by visual sensors – either because they are not visible, such as snap-in fixtures, or because they are not properly recognized, such as recessed screws. In these situations, a solution is required for understanding the object and identifying its components.  Hence, a major challenge in disassembling an unknown object is to overcome these various sources of uncertainty. The shape of an object can be approximated well from vision and depth sensors. However, the information may be incomplete as visually occluded structures and the material composition that affects mechanical characteristics remains out of reach for such sensors. Thus, we are working at providing a robot with the ability to exploit its force sensing capability to refine the knowledge about these objects by interacting with the object.

Technical Challenges

By observing how robot force applied to the object affects interaction dynamics between them, we can improve the knowledge about the object properties, particularly material properties and estimation of structural features. This approach known as tactile and haptic exploration can significantly improve the results of visual object identification methods for complex objects with hidden structures.

Robotic Solution: Haptic SLAM

Haptic SLAM is dedicated to modeling the environment using haptic sensory data without visual feedback, for which haptic regression is an important technology. The entire space of the environment is divided into grids on which the probabilistic environmental model is constructed. Specifically, the probabilistic density functions of the objects in the environment are updated based on collected haptic data using the Bayesian inference and haptic regression. To improve the efficiency of large state space, occupancy and inference grids are applied and adopted with the discretized representation of the workspace. The represented model is built from iteratively collected sensor data given an initial belief over the geometry of the object.  An imminent challenge lies in the refinement of the estimated geometric shape of the underlying object. The critical point is to infer the shape candidates represented via analytical parameterization of different shapes from the initial sensor data and add this prior knowledge to further measurements. As a result, the overall dataset allows not only to update the geometric decomposition but also the material decomposition of the object. An example of the modeling process of an object can be referred in Figure 1, and the modeling results of the object are illustrated in Figure 2.

Indumetal discusses Industry 4.0 integration with Circular Economy practices

By | Blog | No Comments

The Industry 4.0 concept refers to the so-called fourth industrial revolution, which involves digital transformation of the industry with the integration and digitization of all the industrial processes that make up the value chain. The proper integration of the technologies that industry 4.0 offers, enable significant benefits such as greater productivity and better resource management, more efficient decision making based on real information, optimal and integrated production processes, increased flexibility to achieve massive and personalized production in real time, reduction of operational time and reduction of defects percentage in factories. Therefore, Industry 4.0 is characterized by adaptability, flexibility and efficiency.

 The current ‘linear economy’ where everyday products are made, used and disposed should no longer work for businesses, society or environment, as earth’s raw materials are not limitless. Nowadays, there is a need for finding a new path of economic development that includes dealing with waste generation. Hence, there is a need for a transition into a new circular business model that recycles and reuses such waste with the objective of transforming it into higher value-added products to meet the current demands of society.

The integration of I4.0 technologies with circular economy practices have the capacity to significantly reduce  natural resources consumption. New and innovative solutions could make possible to extract even more raw materials, especially critical raw materials, from waste components in order to return them to the cycle. Thus, becoming circular in industrial settings not only involves saving a considerable amount of time, raw materials and consequently, money. It also opens all kinds of opportunities to innovation. To progress in this path, the recycling plant of the future should integrate all kind of I4.0 solutions, having as a result a smart plant, capable of easily adapting to changes in the composition and quantity of the materials to be treated and ensuring at the same time workers safety, avoiding human contact with waste as much as possible. HR-Recycler will help Indumetal in this approach, since it will create a hybrid collaboration environment, where humans and robots will harmoniously share and undertake at the same time different processing and manipulation tasks.

This is not only an improvement for the company’s competitiveness but it is also a step forward towards improving the WEEE collection rate in the European Union. For instance, in 2017, the collection rate of WEEE was 46 % in the EU and its recycling rate was 38,8 %. These rates are no longer acceptable for transitioning EU industry into a circular economy model.

In Indumetal we believe that circular economy strategies are the key for successfully implementing Industry 4.0, at the same time that Industry 4.0 provides technology to activates those strategies.

SADAKO showcase their developed technologies!

By | Blog | No Comments

SADAKO explains their role in HR-Recycler and illustrates how their developed technologies and models are applied to various stages of the project.

As an AI and computer vision company, the role of Sadako in the HR-Recycler project is to provide the crucial vision algorithms to the robotic systems, allowing the perception of the environment in which the machines evolve.

Because of the wide variety of vision-dependent systems being employed in this project (different types of robot arms and Automatically Guided Vehicles, central factory planner), many different challenges are encountered which need each their own specific solution. All of the designed solutions rely on the use of RGB-D cameras and computers capable of Neural Network inference, and will be a good illustration of the many capabilities and applications of computer vision.

Until now, Sadako has developed solutions for the Classification and Disassembly stages of the recycling process, as well as for the AGV navigation.

Classification: 3D perception and object recognition for bin picking
Disassembly: Video-inference for gesture recognition, needed for Human-Robot collaboration and task status management
AGV: Human location perception, needed for optimal AGV path planning and human safety

  • Classification:

This stage is the first of the recycling process, where WEEE arrives mixed in baskets and has to be sorted by type of object (Microwave, Flat Panel Display, PC Tower, Emergency Lamp)
The goal in this stage of the process is to identify the type and 3D orientation of each object lying in the basket, and provide the robotic arm with a picking decision. This is achieved through the use of Deep Learning vision algorithms and point cloud processing

  • Disassembly:

The following process step, Disassembly, is where the previously sorted objects are dismantled and their parts are sorted by material to be sent to their final recycling location. The dismantling is made by an operator, with the support of a collaborative robotic arm. In order to achieve collaboration with the robot, Sadako has developed software based on Neural Networks capable of detecting actions in a video, in contrast with standard computer vision that detects objects in an image. This software is able to interpret the gestures being made by the operator, allowing him or her for example to stop or start the robot at any time. The software also allows the monitoring of the dismantling task’s status.

  • AGV:

AGVs are used to transport WEEE objects between each stage of the dismantling process, reducing high-strain and repetitive physical work for the operators. To ensure that the AGV trajectory does not interfere with the movement of humans around the factory, the central system has to be aware of their position. To achieve this, Sadako has developed a solution based on Openpose, a software that extracts the position of the joints of humans on an image, crossed with depth information provided by the camera, thus allowing to perceive the position and orientation of operators in three dimensions. This software can be deployed at the Disassembly workstation as well as onboard the AGVs.

In its compromise of developing technology to improve the world and reduce health risks endured by humans in the industry, Sadako is staying strongly active and working in the progress of the HR-Recycler project, despite the hard times imposed by the pandemic situation

From theory to practice!

By | Blog | No Comments

In two sessions held at the BIANATT  Aspropyrgos installations on July 8-9 and September 23-25, the procedures of dismantling FPD screens, desktop towers, emergency lighting, and microwave ovens, as well as the procedure of classifying WEEE appliances, were recorded in a special high power desktop computer using high-performance cameras.  Both sessions were coordinated and recorded by CERTH researchers.

BIANATT provided all the equipment and installed a customized construction to allow image recording from many different angles. Below you may see photos of the installation that was used for this purpose.

The recordings will be utilized by CERTH to produce Deep Learning models capable of detecting the aforementioned appliances as well as their internal parts during the disassembly process.

COMAU and new robotics tools for WEEE scenarios

By | Blog | No Comments

Comau explains the approach used on tools selection for WEEE scenarios.

Comau is working on different aspect of the WEEE management related to two main scenarios of the recycling process: the first related to classification of the part to be processed and the second to the disassembling phase in which the part are dismantled and the different materials and components are collected and selected for the final sorting phase.

From the robotics point view this two scenarios present some similarity about the main challenges to be solved even if some different approach and peculiar solutions can be adopted. In general the wide range of parts that can be managed is be very different in terms of:
– weight (from less than 1 kg up to 20 kg) ,
– shape (sharp or smooth edges)
– dimensions (from few cm up to 1 meter o more)
– materials (plastics, metals, glasses, etc)
So one the main requirement is the versatility of the robotics system about the tools to be adopted.

For the classification case an appropriate sensor base platform will be able to detect the parts in the bin but anyway a flexible gripper for the end effector able to handle this wide variety of objects is needed. After a benchmarking analysis on the possible solutions that meet these requirements available in the market, the first tests carried out have been executed with a vacuum grippers, called Kenos and manufactured by Piab ( It has been designed and optimized to be used in various application and represents a flexible tool for the handling manipulation of several products with different shapes, dimensions and compactness.

The first results were very positive: the system has been able to handle all the parts considered as test case in the project (LCD, PC Tower, Emergency lamps, Microwaves oven) with an easy orientation of the gripper and a smart adaptation to the different parts in random surface positions.

In disassembling phase similar problem needs to be faced but in this case the use of a collaborative platform and a small payload robot introduce some issues to take into account as the safe management of interaction with operators as well as the high variety of tool to be used for the dismounting of the parts: also in this case in preliminary test has been adopted the novel KENOS vacuum gripper by PIAB, certified as collaborative and the PLUTO automatic screwdriver by KOLVER ( integrated in a customized solution mounted directly on the robot flange.

The preliminary test conducted on the emergency lamp case showed very good result in terms of reliability and reconfigurability of the system related to the different application that can be executed. Next steps to be performed can take in consideration the use of an appropriate tool changer useful in order to enhance the potentiality of the system and cover also possible task as wire cutting or handling of small objects inside the parts.
The optimization of the current solutions, the adoption of other tools and related tool changer will be face during the rest of the project.

CERTH explain their work in AI-enabled Cell-level perception methods

By | Blog | No Comments

CERTH explains their work in AI-enabled Cell-level perception methods.

The objective of this task is to provide the envisaged robot with the ability to perceive the industrial environment so as to effectively collaborate with the humans and assist them with the Waste Electrical and Electronic (WEEE) device recycling process. Towards this direction, CERTH recorded multiple recycling procedures of four WEEE devices, namely PC Towers, Microwave Ovens, Flat Panel Displays and Emergency Lamps, focusing on the human-robot collaboration aspect during the WEEE device disassembly, thus creating a WEEE device component detection dataset by manually annotating a number of the recorded frames.

Given this dataset, CERTH deployed state of the art Computer Vision methods utilizing Deep Learning techniques in order to train a Convolutional Neural Network (CNN) capable of detecting the four WEEE devices as well as their respective components during the various stages of the disassembly process. Additionally, CERTH enhanced these methods by proposing an anchoring mechanism targeting specifically the detection of small objects.

GAIKER and real world scenarios within HR-Recycler

By | Blog | No Comments

GAIKER goes on with the definition of the real-world scenarios in which the novel WEEE recycling approach based on human-robot collaborations will be demonstrated and validated

GAIKER continues the progress in the task of outlining the scenarios of the pilot demonstrations, whose ultimate aims are to test and evaluate, in real operating environments, the performance of the solutions developed within the HR-Recycler Project to manage WEEE (Waste Electric and Electronic Equipment). This advance is supported by the regular interaction with the partners of the Consortium that, as evidenced after the 1st Project Review Meeting held in Brussels (Belgium) at the end of January 2020, are doing significant and consistent progresses in the development of hardware and software solutions.

In the definition of scenarios, GAIKER is considering how to integrate all the relevant technological advances generated during the implementation of the scheduled tasks, involving user-cases definition, factory-level modelling, cell-level perception, collaborative schemes definition and robotic equipment design and programming. GAIKER is also striving to keep in close contact with the other project partners to understand, update and gather any relevant information that can help in the definition of more precise, realistic and functional pilot scenarios.

Until now, the greatest contributions have been:
– The definition of the use-cases and current recycling scenarios by the industrial WEEE recyclers involved in the HR-Recycler.
– The initial modelling of the factory floor and human workers, including work cell conceptual configuration, virtual factory 3D simulations and process orchestration.
– The demonstrated capabilities of advanced artificial intelligence-based perception in terms of automatic object recognition and manipulation, and human motion prediction.
– The preliminary definition of the robotic equipment, as articulated arms and automated vehicles, and the software for controlling robotic motion and interactions.
– The establishment of foundations for human-robot safe collaboration and adaptative interaction.

GAIKER has been thoughtfully registering, analyzing and integrating all these advances, particularly focusing on those aspects most closely linked to the key performance indicators defined to evaluate the HR-Recycler system. Thus, GAIKER has identified the main elements of the scenarios, including work cells and plant floor layouts, robot and worker movements (sifting and manipulation), detailed procedure requirements, material flows, human-robot interactions, safety issues and process orchestration. Based on these data GAIKER has begun to build the conceptual model for the pilot scenarios. This is, naturally, a flexible and additive model and GAIKER will be aware of any further novelty and progress in the projected tasks.

Robotnik explain their contribution to HR-Recycler project

By | Blog | No Comments

Robotnik explain their contribution to HR-Recycler project.

HR-Recycler is a multidisciplinary project aiming to improve the recycling capabilities of european countries. More specifically, it is targeted at integrating tight Human-Robot collaboration in the recycling process of WEEE, Waste Electrical and Electronic Equipment. E-waste is considered the “fastest-growing waste stream in the world” [1].

WEEE recycling is a time consuming and effort process, due to the variety of the devices to be recycled and the different components and materials they are made of.

One key issue in recycling processes is the routing of raw and disassembled materials and components through the recycling plant. This is a resource consuming process as materials and components are processed in several parts of the factory. In a typical recycling process, unclassified devices arrive at the recycling plant in large trucks. After sorting, each device has to be transported to its recycling station where it is separated into its component parts. Components can be either further separated or be ready to move them to their final destination.

Here is where Robotnik enters into action. Routing materials inside a factory has to be done in an affordable, efficient and safe manner. Moreover, with the advent of the fourth industrial revolution robots have to be able to operate collaboratively with humans sharing the same space.

Affordability is achieved by providing a robot that is able to carry the same baskets as the factory already has, decreasing the number of changes that have to be made in the factory. Efficient is provided by state of the art navigation algorithms, as well as planning algorithms for the overall factory. Collaboration comes through the use of human-aware navigation, but also increasing the communication between the robot and the human about the intentions of the former. Safety is achieved through applying strict measures, sensors and actuators that comply with the latest safety standards

All of this effort is directed by Robotnik towards creating a new robot: RB-Ares.

Robotnik RB-Ares

RB-Ares mission will be to pick and place EURO-pallets at floor level and route them through the factory with the required features of affordability, efficiency, safety and collaboration with humans. To accomplish this mission, RB-Ares is equipped with state-of-the-art actuators and sensors.

RB-Ares is powered by ROS, as well as Robotnik’s proprietary technology for navigation, localization and Human Machine Interface, which allows an easy configuration, programming and integration of the robot in different applications and Fleet Management Systems, as 4.0 industry demands. This is the main feature of Collaborative Mobile Robots as RB-Ares, an intelligent mobile robot that assists humans in a shared workspace and support to the optimizacion of the processes inside industry.

[1] World Economic Forum. (2019). A New Circular Vision for Electronics: Time for a Global Reboot, (January), 24. Retrieved from

SADAKO visits INDUMETAL’s facilities

By | Blog | No Comments

SADAKO visits INDUMETAL’s facilities for a second round of recordings.

Last year, on July and September, a first round of recording sessions was carried out at Bianatt’s facilities and conducted by CERTH.

Indumetal was selected for a second round and, this time, Sadako was the partner in charge of the coordination and the recordings. On 17th and 18th December, Sadako arrived at Indumetal’s facilities, loaded with all their cameras equipment and ready for two hard working days. On the other hand, Indumetal arranged a specific area for the classification and disassembly capturing and, following Bianatt’s experience and indications, a metal structure for camera installation and a correct illumination were provided as well. The use cases selected for this round were microwaves and PC towers.

Additionally, human motion analysis and prediction were recorded in order to detect human actions at cell-level and hand gestures for human robot interaction and to identify task on the human worker. For that, different communication gestures and some of the described actions on D3.1 were performed by Indumetal’s workers.

Do we recycle together? Human-robot collaboration for recycling

By | Blog | No Comments

Do we recycle together? Human-robot collaboration for recycling

With the Information Society and the steadily growing consumption of electrical and electronic equipment (EEE) worldwide, society is facing the challenge of dealing with increasing amounts of waste electrical and electronic equipment (WEEE) in a sustainable and responsible way.

Nowadays manufacturing companies are going through an increasing public and government pressure to reduce the environmental impact of their operations. But when dealing with WEEE, some difficulties arise in classifying and dismantling electronic devices.

How to improve the current process that is mainly manual and time consuming? Can technology help in the improvement of collaboration between humans and robots? A novel solution is to promote cooperation between an operator and robots in this type of processes.

More specifically, the overall goal of HR-Recycler project is to create a hybrid collaborative environment, where humans and robots will harmoniously share and undertake, at the same time, different processing and manipulation tasks, targeting the industrial application case of WEEE recycling.

This goal will be achieved through Human Robot Collaboration (HRC) systems. In the words of Sara Sillaurren, TECNALIA´s project coordinator: “In this new environment in which human and robot have to work collaboratively, it is essential that process design take into account the state of the human for a better understanding between the two of them.”

Within the scope of this project, TECNALIA will bring Human Factors and User experience Lab, which combines physiological sensors and technologies for monitoring human behavior.

Through the usage of wearable devices and platform in this laboratory, and with the collection of psico-physiological data from users, human behavior can be better understood when collaborating with robots and thus the system can be adapted so that processes are more fluid and reliable.

According to this, all types of signals captured give us very interesting information about the user and referred to the activity they are performing. For example, they can give us cognitive information, that is, if the user is paying attention and if the user is able to retain the information we are showing. Another type of information that can be measured is the emotional one. For example, the affective valence or degree of attraction towards the activity they are completing, or the activation and emotional impact, that measure the level of calm or excitement of the user. Finally, there are other types of metrics such as visual attention (that is, what first calls the attention of the user) or the implicit association (between a concept and an attribute).

Furthermore, considering the multiple factors that affect Human Robot Collaboration, TECNALIA is going to define the experimental studies targeting end- user with a purpose of understanding trust in collaborative disassembly and the validation of trust factor model to improve it.

For doing so, a variation of the inspection game has been designed in order to expose participants to different trust stimuli towards machines. Inspection game is a mathematical model of a non-cooperative situation where an inspector verifies that another party, adheres to legal rules instead of shrinking work duties. This experiment will allow us to detect how (un) predictability and prior experiences affect trust.