This article first appeared on Automation World on November 26th, 2019

‍

Using HPE Pointnext services to deploy machine learning at the edge, Foxconn is able to address quality assurance issues in its production of highly customizable IT servers.

‍

‍

Customized manufacturing is a major trend across all manufacturing verticals—from food and beverage to automotive. But the industry vertical that has likely been impacted most by this trend is electronics.

Think about your electronics devices—from your computer to your tablet to your smartphone—and the various configurability options offered by the supplier. From memory to processor speeds to the amount and type of connection ports, the number of commonly available options on a consumer electronics product typically translates into hundreds of possible assemblies for the manufacturer. Now consider IT electronics, such as servers, and the amount of configurability offered by suppliers of these technologies to address various business demands.

That’s the level of complexity faced by Foxconn on its production lines in the company’s Kutna Hora, Czech Republic, facility which produces HPE (Hewlett Packard Enterprise) IT systems.

To give you an idea of how complex assemblies can be at this Foxconn site, consider that one HPE server model can be equipped with between two and sixteen memory modules, each of which can be 16, 32, 64, or 128 gigabytes. “These memory options allow for several hundreds of product variants—and these, in turn, must be multiplied with the number of options available for the processors, fans, disk drives, etc.,” says Norbert Reil, director of global IoT services, HPE Pointnext (the company’s digital transformation consultancy).

Reil notes that quality assurance of these servers not only requires assessment of the number and position of system components, but also of their correct implementation. “For example, a cable must not only be positioned in front of the correct port, but also checked to make sure it was actually plugged in successfully during manufacturing and assembly,” he says. “In addition, other product defects, such as scratches on the surface of the server chassis, must be accounted for.”

Conducting all of these quality checks can take a human several minutes each. And at the Foxconn facility, which can produce thousands of IT devices every day, this translates into “a considerable amount of cost and time added to the manufacturing process,” Reil adds.

‍

To address this issue, HPE Pointnext, together with Relimetrics (which conducts smart quality audits for Industry 4.0), deployed a video analytics for quality assurance system to automate the inspection process. “The video cameras—10 for every conveyor belt—capture high-resolution images of products on a conveyor belt and stream them to an embedded or attached IT system where the images are analyzed by a video analytics application using machine-learning (ML) algorithms,” says Reil. “ML compares the image of the actual product with reference images that display accurate and defective implementations. Thus, the machine learns if a cable is properly plugged into a port, if a memory module is properly inserted into its socket, or if there is a scratch on the chassis.”

The video analytics/quality system can work with a variety of industrial camera systems, Reil says. At the Foxconn Kutna Hora plant, the Basler acA4024-8gc gigabit Ethernet camera, which delivers 8 frames per second at 12.2 MP resolution, is used.

As efficient as ML technology can be at performing these types of inspections, training the analytics system can be challenging due to product variety. One approach to training an ML-based analytics system is to teach it to detect defects in configuration, implementation or any other damage to the product. This approach can require thousands of reference images that the analytics application can compare with the product image from the conveyor belt. The downsides to this approach include the weeks it takes to train the ML algorithms and the need to do a new training cycle for every new configuration, product refresh or update.

To avoid these steps, HPE Pointnext and Relimetrics designed the system so that it does not require the storage of reference images of complete servers, but only of components—such as memory modules placed in the proper slots or the processor socket with its fan. In this setup, the Manufacturing Execution System (MES) provides a bill of materials to the analytics application for every product on the conveyor belt. According to Reil, this enables the system to create a complete reference picture for assembly inspection based on the relevant reference image components. “The data exchange with the video solution happens through the SCADA layer and attached MES—both of which are part of FoxConn’s custom production system called eFox,” he adds.

Two key benefits of this approach, Reil says, are: “first, the ML algorithms aggregate learning much faster and more efficiently because reference image components are frequently reused. At Foxconn’s Kutna Hora factory, HPE Pointnext was able to train the ML model for new server quality assurance with roughly 1,000 configuration variants in two days, completely automating the defect detection process. Second, this approach enables increased flexibility by combining image components according to the actual product configuration as provided by the bill of materials.”

Dealing with the large amounts of data created by all the cameras used in this system was another issue that had to be confronted. Considering that these cameras generate 3 GB of image data per hour, it would be impractical to transfer that data via internal or external networks to be processed on remote servers. “The latency would be too high, networks would be overloaded with these data volumes, and production systems would grind to a halt during network outages,” Reil says.

To overcome this problem, HPE Pointnext deployed the ML-based video analytics system on HPE Edgeline Converged Edge Systems. “These rugged, compact systems are designed with manufacturing environments in mind and also integrate operational technology (OT) like data acquisition systems, control systems, and industrial networks to enable seamless bi-directional and deterministic communication and control of OT systems like video cameras, production machines or conveyor belts,” says Reil.

Data from the video camera stream is first pre-processed on HPE Edgeline Converged Edge Systems running near the conveyor belt. “The solution extracts images of the actual product and then analyzes the data in real-time using ML algorithms to detect defects,” explains Reil. “Only a subset of the analyzed images are transferred via the network to be archived for traceability and compliance.”

‍