Category

Blog

COMAU and new robotics tools for WEEE scenarios

By | Blog | No Comments

Comau explains the approach used on tools selection for WEEE scenarios.

Comau is working on different aspect of the WEEE management related to two main scenarios of the recycling process: the first related to classification of the part to be processed and the second to the disassembling phase in which the part are dismantled and the different materials and components are collected and selected for the final sorting phase.

From the robotics point view this two scenarios present some similarity about the main challenges to be solved even if some different approach and peculiar solutions can be adopted. In general the wide range of parts that can be managed is be very different in terms of:
– weight (from less than 1 kg up to 20 kg) ,
– shape (sharp or smooth edges)
– dimensions (from few cm up to 1 meter o more)
– materials (plastics, metals, glasses, etc)
So one the main requirement is the versatility of the robotics system about the tools to be adopted.

For the classification case an appropriate sensor base platform will be able to detect the parts in the bin but anyway a flexible gripper for the end effector able to handle this wide variety of objects is needed. After a benchmarking analysis on the possible solutions that meet these requirements available in the market, the first tests carried out have been executed with a vacuum grippers, called Kenos and manufactured by Piab (https://www.piab.com/en-us/products/kenos-vacuum-gripping-systems/). It has been designed and optimized to be used in various application and represents a flexible tool for the handling manipulation of several products with different shapes, dimensions and compactness.

The first results were very positive: the system has been able to handle all the parts considered as test case in the project (LCD, PC Tower, Emergency lamps, Microwaves oven) with an easy orientation of the gripper and a smart adaptation to the different parts in random surface positions.

In disassembling phase similar problem needs to be faced but in this case the use of a collaborative platform and a small payload robot introduce some issues to take into account as the safe management of interaction with operators as well as the high variety of tool to be used for the dismounting of the parts: also in this case in preliminary test has been adopted the novel KENOS vacuum gripper by PIAB, certified as collaborative and the PLUTO automatic screwdriver by KOLVER (https://kolver.it/prodotti-elenco/4-Serie-PLUTO-MITO-NATO) integrated in a customized solution mounted directly on the robot flange.

The preliminary test conducted on the emergency lamp case showed very good result in terms of reliability and reconfigurability of the system related to the different application that can be executed. Next steps to be performed can take in consideration the use of an appropriate tool changer useful in order to enhance the potentiality of the system and cover also possible task as wire cutting or handling of small objects inside the parts.
The optimization of the current solutions, the adoption of other tools and related tool changer will be face during the rest of the project.

CERTH explain their work in AI-enabled Cell-level perception methods

By | Blog | No Comments

CERTH explains their work in AI-enabled Cell-level perception methods.

The objective of this task is to provide the envisaged robot with the ability to perceive the industrial environment so as to effectively collaborate with the humans and assist them with the Waste Electrical and Electronic (WEEE) device recycling process. Towards this direction, CERTH recorded multiple recycling procedures of four WEEE devices, namely PC Towers, Microwave Ovens, Flat Panel Displays and Emergency Lamps, focusing on the human-robot collaboration aspect during the WEEE device disassembly, thus creating a WEEE device component detection dataset by manually annotating a number of the recorded frames.

Given this dataset, CERTH deployed state of the art Computer Vision methods utilizing Deep Learning techniques in order to train a Convolutional Neural Network (CNN) capable of detecting the four WEEE devices as well as their respective components during the various stages of the disassembly process. Additionally, CERTH enhanced these methods by proposing an anchoring mechanism targeting specifically the detection of small objects.

GAIKER and real world scenarios within HR-Recycler

By | Blog | No Comments

GAIKER goes on with the definition of the real-world scenarios in which the novel WEEE recycling approach based on human-robot collaborations will be demonstrated and validated

GAIKER continues the progress in the task of outlining the scenarios of the pilot demonstrations, whose ultimate aims are to test and evaluate, in real operating environments, the performance of the solutions developed within the HR-Recycler Project to manage WEEE (Waste Electric and Electronic Equipment). This advance is supported by the regular interaction with the partners of the Consortium that, as evidenced after the 1st Project Review Meeting held in Brussels (Belgium) at the end of January 2020, are doing significant and consistent progresses in the development of hardware and software solutions.

In the definition of scenarios, GAIKER is considering how to integrate all the relevant technological advances generated during the implementation of the scheduled tasks, involving user-cases definition, factory-level modelling, cell-level perception, collaborative schemes definition and robotic equipment design and programming. GAIKER is also striving to keep in close contact with the other project partners to understand, update and gather any relevant information that can help in the definition of more precise, realistic and functional pilot scenarios.

Until now, the greatest contributions have been:
– The definition of the use-cases and current recycling scenarios by the industrial WEEE recyclers involved in the HR-Recycler.
– The initial modelling of the factory floor and human workers, including work cell conceptual configuration, virtual factory 3D simulations and process orchestration.
– The demonstrated capabilities of advanced artificial intelligence-based perception in terms of automatic object recognition and manipulation, and human motion prediction.
– The preliminary definition of the robotic equipment, as articulated arms and automated vehicles, and the software for controlling robotic motion and interactions.
– The establishment of foundations for human-robot safe collaboration and adaptative interaction.

GAIKER has been thoughtfully registering, analyzing and integrating all these advances, particularly focusing on those aspects most closely linked to the key performance indicators defined to evaluate the HR-Recycler system. Thus, GAIKER has identified the main elements of the scenarios, including work cells and plant floor layouts, robot and worker movements (sifting and manipulation), detailed procedure requirements, material flows, human-robot interactions, safety issues and process orchestration. Based on these data GAIKER has begun to build the conceptual model for the pilot scenarios. This is, naturally, a flexible and additive model and GAIKER will be aware of any further novelty and progress in the projected tasks.

Robotnik explain their contribution to HR-Recycler project

By | Blog | No Comments

Robotnik explain their contribution to HR-Recycler project.

HR-Recycler is a multidisciplinary project aiming to improve the recycling capabilities of european countries. More specifically, it is targeted at integrating tight Human-Robot collaboration in the recycling process of WEEE, Waste Electrical and Electronic Equipment. E-waste is considered the “fastest-growing waste stream in the world” [1].

WEEE recycling is a time consuming and effort process, due to the variety of the devices to be recycled and the different components and materials they are made of.

One key issue in recycling processes is the routing of raw and disassembled materials and components through the recycling plant. This is a resource consuming process as materials and components are processed in several parts of the factory. In a typical recycling process, unclassified devices arrive at the recycling plant in large trucks. After sorting, each device has to be transported to its recycling station where it is separated into its component parts. Components can be either further separated or be ready to move them to their final destination.

Here is where Robotnik enters into action. Routing materials inside a factory has to be done in an affordable, efficient and safe manner. Moreover, with the advent of the fourth industrial revolution robots have to be able to operate collaboratively with humans sharing the same space.

Affordability is achieved by providing a robot that is able to carry the same baskets as the factory already has, decreasing the number of changes that have to be made in the factory. Efficient is provided by state of the art navigation algorithms, as well as planning algorithms for the overall factory. Collaboration comes through the use of human-aware navigation, but also increasing the communication between the robot and the human about the intentions of the former. Safety is achieved through applying strict measures, sensors and actuators that comply with the latest safety standards

All of this effort is directed by Robotnik towards creating a new robot: RB-Ares.

Robotnik RB-Ares

RB-Ares mission will be to pick and place EURO-pallets at floor level and route them through the factory with the required features of affordability, efficiency, safety and collaboration with humans. To accomplish this mission, RB-Ares is equipped with state-of-the-art actuators and sensors.

RB-Ares is powered by ROS, as well as Robotnik’s proprietary technology for navigation, localization and Human Machine Interface, which allows an easy configuration, programming and integration of the robot in different applications and Fleet Management Systems, as 4.0 industry demands. This is the main feature of Collaborative Mobile Robots as RB-Ares, an intelligent mobile robot that assists humans in a shared workspace and support to the optimizacion of the processes inside industry.

[1] World Economic Forum. (2019). A New Circular Vision for Electronics: Time for a Global Reboot, (January), 24. Retrieved from

SADAKO visits INDUMETAL’s facilities

By | Blog | No Comments

SADAKO visits INDUMETAL’s facilities for a second round of recordings.

Last year, on July and September, a first round of recording sessions was carried out at Bianatt’s facilities and conducted by CERTH.

Indumetal was selected for a second round and, this time, Sadako was the partner in charge of the coordination and the recordings. On 17th and 18th December, Sadako arrived at Indumetal’s facilities, loaded with all their cameras equipment and ready for two hard working days. On the other hand, Indumetal arranged a specific area for the classification and disassembly capturing and, following Bianatt’s experience and indications, a metal structure for camera installation and a correct illumination were provided as well. The use cases selected for this round were microwaves and PC towers.

Additionally, human motion analysis and prediction were recorded in order to detect human actions at cell-level and hand gestures for human robot interaction and to identify task on the human worker. For that, different communication gestures and some of the described actions on D3.1 were performed by Indumetal’s workers.

Do we recycle together? Human-robot collaboration for recycling

By | Blog | No Comments

Do we recycle together? Human-robot collaboration for recycling

With the Information Society and the steadily growing consumption of electrical and electronic equipment (EEE) worldwide, society is facing the challenge of dealing with increasing amounts of waste electrical and electronic equipment (WEEE) in a sustainable and responsible way.

Nowadays manufacturing companies are going through an increasing public and government pressure to reduce the environmental impact of their operations. But when dealing with WEEE, some difficulties arise in classifying and dismantling electronic devices.

How to improve the current process that is mainly manual and time consuming? Can technology help in the improvement of collaboration between humans and robots? A novel solution is to promote cooperation between an operator and robots in this type of processes.

More specifically, the overall goal of HR-Recycler project is to create a hybrid collaborative environment, where humans and robots will harmoniously share and undertake, at the same time, different processing and manipulation tasks, targeting the industrial application case of WEEE recycling.

This goal will be achieved through Human Robot Collaboration (HRC) systems. In the words of Sara Sillaurren, TECNALIA´s project coordinator: “In this new environment in which human and robot have to work collaboratively, it is essential that process design take into account the state of the human for a better understanding between the two of them.”

Within the scope of this project, TECNALIA will bring Human Factors and User experience Lab, which combines physiological sensors and technologies for monitoring human behavior.

Through the usage of wearable devices and platform in this laboratory, and with the collection of psico-physiological data from users, human behavior can be better understood when collaborating with robots and thus the system can be adapted so that processes are more fluid and reliable.

According to this, all types of signals captured give us very interesting information about the user and referred to the activity they are performing. For example, they can give us cognitive information, that is, if the user is paying attention and if the user is able to retain the information we are showing. Another type of information that can be measured is the emotional one. For example, the affective valence or degree of attraction towards the activity they are completing, or the activation and emotional impact, that measure the level of calm or excitement of the user. Finally, there are other types of metrics such as visual attention (that is, what first calls the attention of the user) or the implicit association (between a concept and an attribute).

Furthermore, considering the multiple factors that affect Human Robot Collaboration, TECNALIA is going to define the experimental studies targeting end- user with a purpose of understanding trust in collaborative disassembly and the validation of trust factor model to improve it.

For doing so, a variation of the inspection game has been designed in order to expose participants to different trust stimuli towards machines. Inspection game is a mathematical model of a non-cooperative situation where an inspector verifies that another party, adheres to legal rules instead of shrinking work duties. This experiment will allow us to detect how (un) predictability and prior experiences affect trust.

A cognitive architecture for social robots to exchange knowledge about the world and the self.

By | Blog | No Comments

Name: Anna Mura, Vicky Vouloutsi

A cognitive architecture for social robots to exchange knowledge about the world and the self.

Society more than ever needs robots performing side by side with humans in different environments and levels of operation, especially in industry and for environmental sustainability.

In our collective imagination, we are prepared to accept “social” robots that can help us to supply those societal/industrial needs that are no longer undertaken by humans. However, we are still far from having cognizant (mindful) robots that understand our needs, and we can trust. Remarkedly, very little is done to transfer principles of perception of the self and the others to autonomous robots that are intended to work side by side with humans. Why is that?

The fundamental problem scientists and engineers struggle with is how much of our understanding of how the mind operates must be transferred to the machine, and how? [Verschure, 2016; Lallee, 2015]. According to the literature, there are two approaches grounded on cognitive architecture that focus on the making of socially interactive robots [T. Fong 2003]: “Functionally- driven robots” where design and functionality are based more on the robot’s “social intelligence appearance” rather than a science-based design. These robots may not require a deep understanding of how the mind operates to build competent robots, i.e., assistive anthropomorphic robots [J. Pineau 2003]; Biologically-grounded robots that are based on theories of natural and social sciences and thus more connected to humans, as they may function using similar principles of perception, decision making, and empathy.

Nevertheless, and in spite of great advances made in the last decade towards solving the worker–robot interaction endeavor in HRI, very few robots work in industry side by side with humans in a collaborative/social manner. And it is clear that developing a cognitive/social architecture to guides our interactions with robots is more critical than previously thought.

The HR-Recycler Project, funded by the H2020 Program of the EU under GA 820742 addresses this challenge by developing a hybrid collaboration environment, where humans and robots will share and undertake at the same time different processing and manipulation tasks, targeting the industrial application of Waste Electrical and Electronic Equipment (WEEE) recycling.

HR-Recycler will advance “Social robotics for safe Human-Robot Collaboration” by capitalizing on the Distributed Adaptive Control (DAC) cognitive architecture [Verschure 2012]. DAC includes a motivation system for social interaction that drives communication and is based on self-regulation and autonomy, and high-level cognitive functions [Lallee 2015, C. Moulin Frier 2017 ].

DAC is organized along four layers (soma, reactive, adaptive and contextual) and three columns (world, self, action). The ‘soma’ designates the body with its sensors, organs and actua- tors. It defines the needs, or self-essential functions (SEF) the organism must satisfy in order to survive. The reactive layer (RL) comprises dedicated behav- iour systems (BS) each implementing predefined sensorimotor mappings serving the SEFs. In order to allow for action selection, task switching and conflict resolution, all BSs are regulated via a, so-called, allostatic controller that sets the internal homeostatic dynamics of BSs relative to overall demands and opportunities [28]. The AL acquires a state space of the agent–environ- ment interaction combining perceptual and behavioural learning constrained by value functions defined by the allostatic control of the RL, minimizing perceptual and behavioural prediction error [29,30]. The contextual layer (CL) further expands the time horizon in which the agent can operate through the use of episodic and sequential short- and long-term memory systems (STM and LTM, respectively). STM acquires conjunctive sensorimotor representations assisted by episodic memory as the agent acts in the world. STM sequences are retained as goal-oriented sequences in LTM when positive value is encountered, as defined by the RL and/or AL. The contribution of stored LTM policies to decision-making depends on four factors: goals, per- ceptual evidence, memory chaining and valence while action selection is further biased by the expected cost of the actions that pertain to reaching a goal state. The content of working memory (WM) is defined by the memory dynamics that represent this four-factor decision-making model. (from P.Verschure 2016)

For collaboration, effective communication is essential for the successful completion of a task. Collaboration is defined as “the mutually beneficial and well-defined relationship of two or more entities to achieve a common goal” [Johal et al. 2014]. Two or more entities that have complementary skills perform common tasks and even share common goals form a team. In Human-Robot Collaboration (HRC) settings, the team is mixed and typically comprises humans and robots working together. For collaboration to be efficient, robots are required to robustly perform a task, be trustworthy, and effectively communicate with the human co-worker. Additionally, we highlight the importance of safe operations, as the robots will be required to function in proximity to humans. For the successful coordination of teamwork, effective communication is critical, as team members may have different mental states [Cohen & Levesque, 2014].

 

“The integration of social cognition and adaptive behaviours to robots in a centralised system will open the door to new HRI and HRC possibilities.”

 

In this context, the SPECS-lab at IBEC as a research group expert in Synthetic Perceptive, Emotive and Cognitive Systems, will contribute to the development of those social and cognitive components needed to build and control “safe Human-Robot Collaboration.” SPECS-lab will do this by expanding DAC’s decision-making model via incorporating into its layered architecture an ethics-based engine responsible for the safe and robust operation of all the components of HR-Recycler.

In addition, the integration of the HR-Recycler human user model together, with social cognition, adaptive behaviours, and learning from human input, will create an environment where the robot will share its current knowledge with the human co-worker. This will help regulate the robot state-space, by either confirming its current knowledge or “adding” into the contextual/cognitive layer of the DAC architecture the appropriate information.

Interestingly the HR-Recycler project will not be using humanoid robots, making the adaptation of the robots’ social and communicative skills to a human co-worker, and vice versa, more challenging but essential for efficient collaboration in industry and manufacturing.

In the field of HRI, we have not found extensive studies to deliver novel ways of interaction suitable for non- anthropomorphic robots. The HR-Recycler project seeks to address the gap in communication for non- anthropomorphic robots based on theories drawn from human communication [Levinson, 2006]. Both Human-Human Interaction (HHI) and Human-Robot Collaboration (HRC) domains require knowledge about the principles and characteristics of social interaction. On the one hand, HHI can benefit from HRI experiments as studies with social robots can act as the testing ground for theoretical explanations. On the other hand, HRC can benefit from studies of HHI, as information acquired from this domain will inform adaptive systems that allow robots to interact with humans fluently. HR-Recycler will advance the field of HRI and HRC by including mechanisms that underlie social competence in a broader range of non-human social behaviors for communication and collaboration.

References:

Interecycling activities and plan within HR-Recycler

By | Blog | No Comments

Interecycling activities and plan within HR-Recycler

Interecycling is one of the oldest companies in Portugal in the WEEE recycling sector, which wants to grow up alongside the new methodologies that the market offers. Therefore, Interecycling (INT) is one of 13 partners of the European project HR-RECYCLER which wants to develop a ‘hybrid human-robot recycling plant for electrical and electronic equipment’ operating in an indoor environment.

Over the years, waste management and processing has abandoned the typical manual process and has been embraced by artificial intelligence (AI), allowing to reduce the necessary manpower (hazardous and expensive manual tasks) and consequently the costs involved in waste management and processing. AI is in constant development and it is used in the autonomous waste classification process. Therefore, machines are necessary to be able to see and deal with the complexity of the waste streams. So, it is necessary to develop algorithms that allow these machines to respond to the needs required in the waste management and processing, getting as close as possible to human performance and manpower.

Sadako (SDK) is the project partner responsible for developing the vision capabilities needed by the robots to see and manipulate WEEE (Waste Electric and Electronic Equipment) objects in the human-robot collaborative recycling process targeted. In February, INT will welcome SDK at our facilities to proceed with the collection of images during the process of dismantling the FPD screens and PC towers with high performance cameras from different angles and perspectives, similar to the sessions already done in facilities of the BIANATT, during the last plenary meeting in Greece.

Ana Catarina Antunes

Legal and Ethic requirements for HR-Recycler

By | Blog | No Comments

Legal and Ethic requirements for HR-Recycler

The VUB is the legal and ethical partner of the project. As such, its role consists of identifying legal, ethical and societal impacts that the project technology might entail, providing recommendations and ensuring compliance to those elements by the Consortium throughout the entire project. The particularity of HR-Recycler is the introduction of a novel technology in the workplace and the risks that human-robot-collaboration and human-robot-interaction can create, for example if it could result in an evaluation of workers’ performance at work or could lead to a feeling of surveillance. At present, the VUB team is finishing the TARES impact assessment of the project (comprising an assessment of the impacts on data protection, ethics and privacy), the third deliverable of the sixth to be drafted by the VUB. The outcomes are positive, although stronger safeguards have to be thought through, since workers are considered to be ‘vulnerable subjects’ and consent to participate in the research project is not deemed to be freely given by data protection authorities. For example, the Consortium is well-aware of the need to ensure clear separation of sensitive personal data processed by research partners and end-users (e.g. health data), foresee an independent observer role to collect consent forms, look for workers’ feedback at several moments of the project development and take into account the impact on workers’ social and labour rights. One aspect that makes easier the implementation of these strong safeguards is that all Partners are very experienced and extremely active in executing the necessary identified recommendations.

The HR-Recycler project also contributed to a research output, helping to build up the second policy brief of the d.pia.lab ‘Towards a method for data protection impact assessment: Making sense of GDPR requirements’. Live discussions at each General Meeting are guaranteed, with themes ranging from ethics, artificial intelligence, data protection, privacy and data protection by design, and the General Data Protection Regulation in general.

If you need wish to receive further information, do not hesitate to subscribe to the HR-Recycler newsletter.

Sara Roda

27 January 2020

Producing 3D models using photogrammetry

By | Blog | No Comments

DXT describes how various components were 3D reconstructed using photogrammetry.

The HR-Recycler project will implement learning paradigms in virtual reality (VR) settings, for which a recycling plant will be modelled in 3D to be displayed in a VR environment. One objective is to capture accurately the interaction of the workers with the environment and especially the objects present in the factory. Another objective is to allow workers to practice disassembly procedures and to properly interact with the environment. It is necessary to deliver a VR environment as realistic as possible, i.e. as close as possible to an actual recycling factory. To model accurately complex shapes or objects, one may use a full manual technique with tools such as 3DSmax. However, this technique is quite complex and time consuming. Imagine how long it could take to model a PC motherboard entirely manually, given the number of small elements (diodes, capacitors, etc.)

Photogrammetry is a 3D reconstruction technique that allows to produce 3D models of complex shapes much more quickly. It consists in taking dozens of pictures of an object from as many viewing angle as possible, then using an algorithm to reconstruct the object from all those pictures. Although it generally gives good results for complex shapes, it often fails for simple ones. The main reason is that simple shapes have very few points of interests that de algorithm may use for 3D reconstruction. Also, simple surfaces are often too mate or too bright, leading to reconstruction errors. To illustrate this, a 3D reconstruction of the above motherboard is shown in the figure below.

To overcome limitations of both techniques, DIGINEXT has set-up a 3D modelling technique that combines manual modelling and photogrammetry to produce more rapidly accurate models of WEEE. In the example of a PC case, the simple parts (e.g. exterior of the case, flat surfaces) were manually modelled using 3DSmax, whereas internal complex shapes were modelled using photogrammetry. This allowed to produce a full 3D model of a PC case more rapidly than using a 3D CAD tool only and more precisely than by using photogrammetry alone. Several objects have been created using these techniques, as shown in the pictures below and will be used in the virtual factory for the next steps of the project.