Blog

WEEE recycling is a time consuming and effort process, due to the variety of the devices to be recycled and the different components and materials they are made of.

One key issue in recycling processes is the routing of raw and disassembled materials and components through the recycling plant. This is a resource consuming process as materials and components are processed in several parts of the factory. In a typical recycling process, unclassified devices arrive at the recycling plant in large trucks. After sorting, each device has to be transported to its recycling station where it is separated into its component parts. Components can be either further separated or be ready to move them to their final destination.

Here is where Robotnik enters into action. Routing materials inside a factory has to be done in an affordable, efficient and safe manner. Moreover, with the advent of the fourth industrial revolution robots have to be able to operate collaboratively with humans sharing the same space.

Affordability is achieved by providing a robot that is able to carry the same baskets as the factory already has, decreasing the number of changes that have to be made in the factory. Efficient is provided by state of the art navigation algorithms, as well as planning algorithms for the overall factory. Collaboration comes through the use of human-aware navigation, but also increasing the communication between the robot and the human about the intentions of the former. Safety is achieved through applying strict measures, sensors and actuators that comply with the latest safety standards

All of this effort is directed by Robotnik towards creating a new robot: RB-Ares.

Robotnik RB-Ares

RB-Ares mission will be to pick and place EURO-pallets at floor level and route them through the factory with the required features of affordability, efficiency, safety and collaboration with humans. To accomplish this mission, RB-Ares is equipped with state-of-the-art actuators and sensors.

RB-Ares is powered by ROS, as well as Robotnik’s proprietary technology for navigation, localization and Human Machine Interface, which allows an easy configuration, programming and integration of the robot in different applications and Fleet Management Systems, as 4.0 industry demands. This is the main feature of Collaborative Mobile Robots as RB-Ares, an intelligent mobile robot that assists humans in a shared workspace and support to the optimizacion of the processes inside industry.

[1] World Economic Forum. (2019). A New Circular Vision for Electronics: Time for a Global Reboot, (January), 24. Retrieved from

SADAKO visits INDUMETAL’s facilities for a second round of recordings.

Last year, on July and September, a first round of recording sessions was carried out at Bianatt’s facilities and conducted by CERTH.

Indumetal was selected for a second round and, this time, Sadako was the partner in charge of the coordination and the recordings. On 17^th and 18^th December, Sadako arrived at Indumetal’s facilities, loaded with all their cameras equipment and ready for two hard working days. On the other hand, Indumetal arranged a specific area for the classification and disassembly capturing and, following Bianatt’s experience and indications, a metal structure for camera installation and a correct illumination were provided as well. The use cases selected for this round were microwaves and PC towers.

Additionally, human motion analysis and prediction were recorded in order to detect human actions at cell-level and hand gestures for human robot interaction and to identify task on the human worker. For that, different communication gestures and some of the described actions on D3.1 were performed by Indumetal’s workers.

Do we recycle together? Human-robot collaboration for recycling

With the Information Society and the steadily growing consumption of electrical and electronic equipment (EEE) worldwide, society is facing the challenge of dealing with increasing amounts of waste electrical and electronic equipment (WEEE) in a sustainable and responsible way.

Nowadays manufacturing companies are going through an increasing public and government pressure to reduce the environmental impact of their operations. But when dealing with WEEE, some difficulties arise in classifying and dismantling electronic devices.

How to improve the current process that is mainly manual and time consuming? Can technology help in the improvement of collaboration between humans and robots? A novel solution is to promote cooperation between an operator and robots in this type of processes.

More specifically, the overall goal of HR-Recycler project is to create a hybrid collaborative environment, where humans and robots will harmoniously share and undertake, at the same time, different processing and manipulation tasks, targeting the industrial application case of WEEE recycling.

This goal will be achieved through Human Robot Collaboration (HRC) systems. In the words of Sara Sillaurren, TECNALIA´s project coordinator: “In this new environment in which human and robot have to work collaboratively, it is essential that process design take into account the state of the human for a better understanding between the two of them.”

Within the scope of this project, TECNALIA will bring Human Factors and User experience Lab, which combines physiological sensors and technologies for monitoring human behavior.

Through the usage of wearable devices and platform in this laboratory, and with the collection of psico-physiological data from users, human behavior can be better understood when collaborating with robots and thus the system can be adapted so that processes are more fluid and reliable.

According to this, all types of signals captured give us very interesting information about the user and referred to the activity they are performing. For example, they can give us cognitive information, that is, if the user is paying attention and if the user is able to retain the information we are showing. Another type of information that can be measured is the emotional one. For example, the affective valence or degree of attraction towards the activity they are completing, or the activation and emotional impact, that measure the level of calm or excitement of the user. Finally, there are other types of metrics such as visual attention (that is, what first calls the attention of the user) or the implicit association (between a concept and an attribute).

Furthermore, considering the multiple factors that affect Human Robot Collaboration, TECNALIA is going to define the experimental studies targeting end- user with a purpose of understanding trust in collaborative disassembly and the validation of trust factor model to improve it.

For doing so, a variation of the inspection game has been designed in order to expose participants to different trust stimuli towards machines. Inspection game is a mathematical model of a non-cooperative situation where an inspector verifies that another party, adheres to legal rules instead of shrinking work duties. This experiment will allow us to detect how (un) predictability and prior experiences affect trust.

Name: Anna Mura, Vicky Vouloutsi

A cognitive architecture for social robots to exchange knowledge about the world and the self.

Society more than ever needs robots performing side by side with humans in different environments and levels of operation, especially in industry and for environmental sustainability.

In our collective imagination, we are prepared to accept “social” robots that can help us to supply those societal/industrial needs that are no longer undertaken by humans. However, we are still far from having cognizant (mindful) robots that understand our needs, and we can trust. Remarkedly, very little is done to transfer principles of perception of the self and the others to autonomous robots that are intended to work side by side with humans. Why is that?

The fundamental problem scientists and engineers struggle with is how much of our understanding of how the mind operates must be transferred to the machine, and how? [Verschure, 2016; Lallee, 2015]. According to the literature, there are two approaches grounded on cognitive architecture that focus on the making of socially interactive robots [T. Fong 2003]: “Functionally- driven robots” where design and functionality are based more on the robot’s “social intelligence appearance” rather than a science-based design. These robots may not require a deep understanding of how the mind operates to build competent robots, i.e., assistive anthropomorphic robots [J. Pineau 2003]; Biologically-grounded robots that are based on theories of natural and social sciences and thus more connected to humans, as they may function using similar principles of perception, decision making, and empathy.

Nevertheless, and in spite of great advances made in the last decade towards solving the worker–robot interaction endeavor in HRI, very few robots work in industry side by side with humans in a collaborative/social manner. And it is clear that developing a cognitive/social architecture to guides our interactions with robots is more critical than previously thought.

The HR-Recycler Project, funded by the H2020 Program of the EU under GA 820742 addresses this challenge by developing a hybrid collaboration environment, where humans and robots will share and undertake at the same time different processing and manipulation tasks, targeting the industrial application of Waste Electrical and Electronic Equipment (WEEE) recycling.

HR-Recycler will advance “Social robotics for safe Human-Robot Collaboration” by capitalizing on the Distributed Adaptive Control (DAC) cognitive architecture [Verschure 2012]. DAC includes a motivation system for social interaction that drives communication and is based on self-regulation and autonomy, and high-level cognitive functions [Lallee 2015, C. Moulin Frier 2017 ].

DAC is organized along four layers (soma, reactive, adaptive and contextual) and three columns (world, self, action). The ‘soma’ designates the body with its sensors, organs and actua- tors. It defines the needs, or self-essential functions (SEF) the organism must satisfy in order to survive. The reactive layer (RL) comprises dedicated behav- iour systems (BS) each implementing predefined sensorimotor mappings serving the SEFs. In order to allow for action selection, task switching and conflict resolution, all BSs are regulated via a, so-called, allostatic controller that sets the internal homeostatic dynamics of BSs relative to overall demands and opportunities [28]. The AL acquires a state space of the agent–environ- ment interaction combining perceptual and behavioural learning constrained by value functions defined by the allostatic control of the RL, minimizing perceptual and behavioural prediction error [29,30]. The contextual layer (CL) further expands the time horizon in which the agent can operate through the use of episodic and sequential short- and long-term memory systems (STM and LTM, respectively). STM acquires conjunctive sensorimotor representations assisted by episodic memory as the agent acts in the world. STM sequences are retained as goal-oriented sequences in LTM when positive value is encountered, as defined by the RL and/or AL. The contribution of stored LTM policies to decision-making depends on four factors: goals, per- ceptual evidence, memory chaining and valence while action selection is further biased by the expected cost of the actions that pertain to reaching a goal state. The content of working memory (WM) is defined by the memory dynamics that represent this four-factor decision-making model. (from P.Verschure 2016)

For collaboration, effective communication is essential for the successful completion of a task. Collaboration is defined as “the mutually beneficial and well-defined relationship of two or more entities to achieve a common goal” [Johal et al. 2014]. Two or more entities that have complementary skills perform common tasks and even share common goals form a team. In Human-Robot Collaboration (HRC) settings, the team is mixed and typically comprises humans and robots working together. For collaboration to be efficient, robots are required to robustly perform a task, be trustworthy, and effectively communicate with the human co-worker. Additionally, we highlight the importance of safe operations, as the robots will be required to function in proximity to humans. For the successful coordination of teamwork, effective communication is critical, as team members may have different mental states [Cohen & Levesque, 2014].

“The integration of social cognition and adaptive behaviours to robots in a centralised system will open the door to new HRI and HRC possibilities.”

In this context, the SPECS-lab at IBEC as a research group expert in Synthetic Perceptive, Emotive and Cognitive Systems, will contribute to the development of those social and cognitive components needed to build and control “safe Human-Robot Collaboration.” SPECS-lab will do this by expanding DAC’s decision-making model via incorporating into its layered architecture an ethics-based engine responsible for the safe and robust operation of all the components of HR-Recycler.

In addition, the integration of the HR-Recycler human user model together, with social cognition, adaptive behaviours, and learning from human input, will create an environment where the robot will share its current knowledge with the human co-worker. This will help regulate the robot state-space, by either confirming its current knowledge or “adding” into the contextual/cognitive layer of the DAC architecture the appropriate information.

Interestingly the HR-Recycler project will not be using humanoid robots, making the adaptation of the robots’ social and communicative skills to a human co-worker, and vice versa, more challenging but essential for efficient collaboration in industry and manufacturing.

In the field of HRI, we have not found extensive studies to deliver novel ways of interaction suitable for non- anthropomorphic robots. The HR-Recycler project seeks to address the gap in communication for non- anthropomorphic robots based on theories drawn from human communication [Levinson, 2006]. Both Human-Human Interaction (HHI) and Human-Robot Collaboration (HRC) domains require knowledge about the principles and characteristics of social interaction. On the one hand, HHI can benefit from HRI experiments as studies with social robots can act as the testing ground for theoretical explanations. On the other hand, HRC can benefit from studies of HHI, as information acquired from this domain will inform adaptive systems that allow robots to interact with humans fluently. HR-Recycler will advance the field of HRI and HRC by including mechanisms that underlie social competence in a broader range of non-human social behaviors for communication and collaboration.

References:

• Verschure, P. (2012). Distributed adaptive control: a theory of the mind, brain, body nexus. Biologically Inspired Cognitive Architectures, 1, 55-72.
• Verschure, P. (2016). “Synthetic consciousness: the distributed adaptive control perspective.” Philosophical Transactions of the Royal Society B: Biological Sciences1701,
• Lallée, S., Vouloutsi, et al. (2015). Towards the synthetic self: making others perceive me as an other. Paladyn, Journal of Behavioral Robotics, 6(1),
• Fong, T., et al. (2003). A survey of socially interactive robots. Robotics and autonomous systems, 42(3-4), 143-166.
• Pineau, J., et al. (2003). Towards robotic assistants in nursing homes: Challenges and results. Robotics and autonomous systems, 42(3-4), 271-281.
• C Moulin-Frier et al. (2017). DAC-h3: a proactive robot cognitive architecture to acquire and express knowledge about the world and the self. IEEE Transactions on Cognitive and Developmental Systems 10 (4), 1005-1022
• Johal, W., Calvary, G., & Pesty, S. (2014). Non-verbal Signals in HRI: Interference in Human Perception. 6th International Conference, ICSR 2014, Proceedings, 412.
• Cohen, P. R., & Levesque, H. J. (2014). Teamwork. Nous, 25(4), 487–512.
• Levinson, S. (2006). Cognition at the heart of human interaction. Discourse Studies, 8, 85–93.

Interecycling activities and plan within HR-Recycler

Interecycling is one of the oldest companies in Portugal in the WEEE recycling sector, which wants to grow up alongside the new methodologies that the market offers. Therefore, Interecycling (INT) is one of 13 partners of the European project HR-RECYCLER which wants to develop a ‘hybrid human-robot recycling plant for electrical and electronic equipment’ operating in an indoor environment.

Over the years, waste management and processing has abandoned the typical manual process and has been embraced by artificial intelligence (AI), allowing to reduce the necessary manpower (hazardous and expensive manual tasks) and consequently the costs involved in waste management and processing. AI is in constant development and it is used in the autonomous waste classification process. Therefore, machines are necessary to be able to see and deal with the complexity of the waste streams. So, it is necessary to develop algorithms that allow these machines to respond to the needs required in the waste management and processing, getting as close as possible to human performance and manpower.

Sadako (SDK) is the project partner responsible for developing the vision capabilities needed by the robots to see and manipulate WEEE (Waste Electric and Electronic Equipment) objects in the human-robot collaborative recycling process targeted. In February, INT will welcome SDK at our facilities to proceed with the collection of images during the process of dismantling the FPD screens and PC towers with high performance cameras from different angles and perspectives, similar to the sessions already done in facilities of the BIANATT, during the last plenary meeting in Greece.

Ana Catarina Antunes

Legal and Ethic requirements for HR-Recycler

The VUB is the legal and ethical partner of the project. As such, its role consists of identifying legal, ethical and societal impacts that the project technology might entail, providing recommendations and ensuring compliance to those elements by the Consortium throughout the entire project. The particularity of HR-Recycler is the introduction of a novel technology in the workplace and the risks that human-robot-collaboration and human-robot-interaction can create, for example if it could result in an evaluation of workers’ performance at work or could lead to a feeling of surveillance. At present, the VUB team is finishing the TARES impact assessment of the project (comprising an assessment of the impacts on data protection, ethics and privacy), the third deliverable of the sixth to be drafted by the VUB. The outcomes are positive, although stronger safeguards have to be thought through, since workers are considered to be ‘vulnerable subjects’ and consent to participate in the research project is not deemed to be freely given by data protection authorities. For example, the Consortium is well-aware of the need to ensure clear separation of sensitive personal data processed by research partners and end-users (e.g. health data), foresee an independent observer role to collect consent forms, look for workers’ feedback at several moments of the project development and take into account the impact on workers’ social and labour rights. One aspect that makes easier the implementation of these strong safeguards is that all Partners are very experienced and extremely active in executing the necessary identified recommendations.

The HR-Recycler project also contributed to a research output, helping to build up the second policy brief of the d.pia.lab ‘Towards a method for data protection impact assessment: Making sense of GDPR requirements’. Live discussions at each General Meeting are guaranteed, with themes ranging from ethics, artificial intelligence, data protection, privacy and data protection by design, and the General Data Protection Regulation in general.

If you need wish to receive further information, do not hesitate to subscribe to the HR-Recycler newsletter.

Sara Roda

27 January 2020

DXT describes how various components were 3D reconstructed using photogrammetry.

The HR-Recycler project will implement learning paradigms in virtual reality (VR) settings, for which a recycling plant will be modelled in 3D to be displayed in a VR environment. One objective is to capture accurately the interaction of the workers with the environment and especially the objects present in the factory. Another objective is to allow workers to practice disassembly procedures and to properly interact with the environment. It is necessary to deliver a VR environment as realistic as possible, i.e. as close as possible to an actual recycling factory. To model accurately complex shapes or objects, one may use a full manual technique with tools such as 3DSmax. However, this technique is quite complex and time consuming. Imagine how long it could take to model a PC motherboard entirely manually, given the number of small elements (diodes, capacitors, etc.)

Photogrammetry is a 3D reconstruction technique that allows to produce 3D models of complex shapes much more quickly. It consists in taking dozens of pictures of an object from as many viewing angle as possible, then using an algorithm to reconstruct the object from all those pictures. Although it generally gives good results for complex shapes, it often fails for simple ones. The main reason is that simple shapes have very few points of interests that de algorithm may use for 3D reconstruction. Also, simple surfaces are often too mate or too bright, leading to reconstruction errors. To illustrate this, a 3D reconstruction of the above motherboard is shown in the figure below.

To overcome limitations of both techniques, DIGINEXT has set-up a 3D modelling technique that combines manual modelling and photogrammetry to produce more rapidly accurate models of WEEE. In the example of a PC case, the simple parts (e.g. exterior of the case, flat surfaces) were manually modelled using 3DSmax, whereas internal complex shapes were modelled using photogrammetry. This allowed to produce a full 3D model of a PC case more rapidly than using a 3D CAD tool only and more precisely than by using photogrammetry alone. Several objects have been created using these techniques, as shown in the pictures below and will be used in the virtual factory for the next steps of the project.

GAIKER starts the definition of scenarios in which innovative pilot studies of the recycling of WEEE (Waste Electric and Electronic Equipment) based on human-robot collaboration will be demonstrated.

The continuous technological development has increased the types and amounts of electric and electronic equipment (EEE) that are used daily by both industries and citizens. Additionally, as result of their continuous improvement and evolution, the devices have shorter life cycles and rapidly reach the end of life stage and become waste electric and electronic equipment (WEEE) that needs to be properly managed.

The grouping of the waste equipment by homogeneous batches prior to its treatment, the depollution (safe removal of potentially hazardous components or substances), the reclamation of reusable parts or assemblies and the separation of recyclable materials in fractions generated after the treatment of depolluted equipment, are the main stages included in the management of those complex products. Important innovations have been implemented in these operations during recent years but still comprise many actions that require considerable amounts of manual labour and demand experienced and skilled workers that make physical efforts and execute repetitive movements.

In this context, the HR-Recycler Project, funded by the H2020 Program of the EU under GA 820742 and coordinated by the Greek CERTH (Centre for Research and Technology Hellas), was started in December 2018. Its objective is creating collaborative environments between human and robots, where the tasks associated to handling and processing of WEEE, as classification of units, dismantling and removal of parts or concentration of material fractions, can be shared. GAIKER, as a research organisation expert in the design, development, testing and assessment of new recycling processes, has initiated the definition of scenarios in which the management of WEEE, based on human-robot collaboration, will be demonstrated and assessed. That work will start with the detailed description of plants lay-outs, robots, movements, navigation routes and interactions with workers. It will continue with the technical validation to measure increases in efficiency and productivity and the social life cycle assessment (SLCA) to determine the benefit in terms of increase of job quality associated to the sharing of task between humans and robots.

Waste of electrical and electronic equipment (WEEE) is one the fastest growing waste streams in the EU, growing at 3-5% per year, with a generation above 12 million tonnes estimated for 2020. WEEE is a complex mixture of valuable materials that can cause major environmental and health problems if not properly managed due to their hazardous content. The improvement of WEEE prevention, collection and recovery is essential to boost circular economy and enhance resource efficiency, which require new approaches in the design, manufacturing, use and end of life (EoL) of electrical and electronic equipment (EEE).

Legal requirements

The first WEEE Directive (2002/96/EC) provided a legal framework in order to structure the WEEE management, promote the recycling and avoid its landfill. However, the recast WEEE Directive (2012/19/EU) entered into force in 2012, setting out ambitious targets for the following terms:

• Collection: gathering of waste, including the preliminary sorting and preliminary storage of waste for the purposes of transport to a waste treatment facility
• Recovery: any operation the principal result of which is waste serving a useful purpose by replacing other materials which would otherwise have been used to fulfil a particular function, or waste being prepared to fulfil that function, in the plant or in the wider economy
• Preparation for re-use: checking, cleaning or repairing recovery operations, by which products or components of products that have become waste are prepared so that they can be re-used without any other pre-processing
• Recycling: any recovery operation by which waste materials are reprocessed into products, materials or substances whether for the original or other purposes

What are we doing?

Facing this situation, specialized companies such as INDUMETAL, carry out the integral handling of WEEE and complex scraps. Firstly, the WEEE is classified and then depolluted removing the hazardous components by means of a specific procedures and processes. Once depolluted, the wastes are introduced in the recycling process where, following successive steps of grinding, size reduction, mechanic separation and concentration steps, several materials from electronic scraps are separated and concentrated.

However, despite the effort of the recycling companies, only one-third of WEEE in the EU is being reported by compliance schemes as separately collected and managed. The remaining two-thirds are either collected by unregistered companies and treated or even illegally exported, or disposed of as part of residual waste.

The total amount of WEEE properly collected in the EU was 3.9 million tonnes in 2015; 88% of this amount was recovered, whilst the amount recycled/re-used was 81%, with re-use only representing 1.4%. These rates have been sufficient in the past, but now the targets are more ambitious and it is critical to make stronger efforts.

What should we do in the future?

At present, the main driving forces for WEEE treatment are the removal of hazardous substances and the recycling of metals, since they have a high market price and have so far contributed mostly to meet the WEEE recovery/recycling targets. However, other alternative and complementary solutions are still needed to move the EEE sector towards a true circular economy, allowing to reach the regulatory targets and helping to reduce the illegal export of WEEE and the derived impacts.

In this framework, HR-Recycler project is focused on the development of a ‘hybrid human-robot recycling plant for electrical and electronic equipment’ operating in an indoor environment, replacing thus multiple currently manual, expensive, hazardous and time-consuming tasks of the WEEE materials pre-processing. Attending to these objectives, HR-RECYCLER project expects to improve current WEEE treatment practices, extracting, sorting and classifying different components and concentrated fractions with a higher economic and environmental value.

With this project, INDUMETAL gets close to these new concepts of collaboration and automatization, expecting that the achieved results will allow the company: (1) to stand out as a leading company in technologies applied to the treatment of WEEE and (2) to improve the current working conditions, the productivity and the process costs.

Recycling of Waste Electrical and Electronic Equipment (WEEE) is a process still heavily based on manual tasks performed by human workers. HR-Recycler is developing a hybrid human-robot setup to replace currently manual tasks that are largely hazardous, time consuming, and expensive. Through the project, researchers from the Chair of Automatic Control Engineering at the Technical University of Munich have been developing methods that allow the robot to safety and robustly disassemble WEEE into recyclable components. One of the unique approaches to their work is how a robot is used not only for disassembling objects, but also to feel and understand the object characteristics, like a human does.

Technical Challenge

One of the challenges in robot-based object disassembly is removal of an outer shell or casing – these are common in products such as PC towers, microwave ovens and emergency lamps. The outer shells of modern devices are usually held together with multiple fixtures (e.g., screws). The precise locations and types of fixtures vary greatly between objects. When disassembling devices without knowing their construction details, fixtures are easily missed – either because they are not visible, such as snap-in fixtures, or because they are not properly recognized, for example when screws are recessed.  Despite the impressive progress in computer vision techniques,  visual data can only provide very limited information about materials and their properties, inside structures and general mechanical design.

Robotic solution

For unknown object exploration, humans do not solely rely on visual information, but also take further sensory information into account. It is promising to analyze and reproduce human behavior. Mimicking the human way of exploring unknown objects from haptic information bears a high potential towards improving robotic perception for unknown and complex objects. Using haptic feedback as an alternative source of information further allows improving the knowledge about the surroundings of the robot in terms of actually feeling the properties of objects. This approach, known as “haptic exploration”, significantly improves object identification over purely vision-based methods.

Thus, the challenge of exploring and identifying unknown objects is tackled by augmenting uncertain prior knowledge from the vision data with haptic information. The location of missing fixtures is determined through probing the case structure by bending it from multiple locations in order to identify a model for the structure and hidden fixtures like a human would do.