Digital Twins – Deep Learning’s Super Heroes

Angie Kellen, Director, Client Services, Open Sky Communications

There is a lot of buzz around ‘Artificial Intelligence,’ ‘Machine Learning’ and ‘Deep Learning.’ But recently, I have been hearing the term ‘Digital Twins’ used more and more. And its use has been included in many different industries – not just semiconductors, which is where I first heard the term. This past July, I attended the inaugural ES Design West expo, which was co-located with SEMICON West in San Francisco, California. With the growing interest and capabilities in electronic systems design, this event was created to connect the design community with the entire electronics supply chain. I attended the afternoon session and one presentation clearly caught my attention. It was “Everything Needs a Digital Twin in the Deep-Learning Era” by Aki Fujimura, CEO of D2S, Inc.

Before I go into more about Aki’s presentation, here are a few key things to keep in mind:  deep learning is a subset of machine learning, and machine learning is a subset of artificial intelligence, which is a high-level or blanket term for computer programs that do something smart. Generally, when someone uses the term deep learning, he/she is referring to deep artificial neural networks. Deep learning maps inputs to outputs. It finds correlations. It is known as a “universal approximator”, because it can learn to approximate an unknown function. Computational intensity is one of the trademarks of deep learning, and it is one reason why a new kind of chip called Graphics Processing Unit (GPU) is in demand to train deep-learning models. Aki’s company, D2S, makes GPU-accelerated solutions for semiconductor manufacturing. You can read more about GPU-accelerated simulation in this paper co-authored by Aki. Still with me? Maybe after I explain more about Aki’s presentation this paragraph will make more sense as it did for me.

Aki’s presentation really put some things into perspective for me with regard to what is required to use deep learning effectively. Whether you are using deep learning to create better photomasks to make amazingly small chips or you are using it to teach an autonomous car how to drive, it takes data—not just a lot of data, TONS of data, unimaginable amounts of data. Deep learning’s power is proportionate to the size of the data that is available.

In fact, Aki proposes that data is the new currency. To support this statement, Aki points out that companies like Google and Facebook look for new acquisitions by ranking the target company’s value based on access to more data. Deep learning monetizes data. 

Now let’s go back to the unimaginable amounts of data. Why do we need so much data? A great example is self-driving cars. Car companies are constantly gathering data as the vehicles are traveling along the roads. To learn, it needs lots of data, in fact after clocking in three-billion (that is a B) miles of driving data, it still is not enough! And the main reason is that the unusual events, such as a child entering the roadway in front of the car or someone cutting off the car, happen rarely. To learn, this activity needs to happen more frequently. Aki notes, “In semiconductor manufacturing, as with driving, the unusual events are very rare. But for deep learning, these neural networks need to be trained as much with the rare data because the worst-case makes a chip fail, so overall average performance isn’t good enough.”

How can we speed up the occasion of rare events? Enter the super heroes of deep learning, Digital Twins! It can be accomplished by using a simulation strategy. Digital twins technology enables deep learning by providing a digitized duplicate or digital asset of a physical asset, such as your data, process or equipment. These digital assets can be created even before an asset is physically built. As a result, the digital twin will act as a live model of the physical equipment, driverless car, etc. This allows you to create the unusual data or rare unknowns using the digital twin to speed up the learning curve and produce a better result. This data can now be shared from the digital twin to the physical asset and vice versa to keep both assets current.

As you can see, digital twins technology is driving the success of deep learning and together, they exponentially speed up the learning curve for the physical assets. Aki concluded his presentation with an update on the Center for Deep Learning in Electronics Manufacturing (CDLe), located in San Jose, Calif. After only eight months, the partnership between NuFlare Technology, Mycronic and D2S has made significant progress in the use of deep learning. For semiconductor photomask applications, the CDLe is making deep learning-based digital twins of scanning electron microscope (SEM) images. The result is a higher-resolution image that can now be combined with other deep learning training data for the production of more precise photomasks. As Aki finished, he was met with vigorous applause, including from me, as his presentation brought to light for me the basic requirements for deep learning and the volumes upon volumes of data needed for it. I now understood the valuable role that digital twins play in supporting deep learning.

Outside of semiconductors and self-driving cars, where else do you think a digital replica of a physical object would be helpful? It really seems like there is no limit to the many ways digital twins technology will positively impact the world. Today, NASA is using digital twins to explore next-generation aircraft and vehicles. Fun fact: NASA engineers and astronauts actually used an early concept of digital twins to figure out the best way to rescue the Apollo 13 mission in 1970. Chevron used digital twins technology on equipment in its oil fields and refineries and expects to save millions of dollars in maintenance costs by 2024. Digital twins can revolutionize healthcare operations as well as patient care. For example, a digital twin of a patient or organs enables surgeons and healthcare professionals to practice procedures in a virtual environment rather than on a real patient. Formula 1 car racing, where every second counts, uses this technology to help the driver and the team know what adjustments can improve performance.

There is actually a digital twin of Singapore! With all the components that go into managing a city, digital twins help city planners to understand and reduce energy consumption as well as many other applications that improve life for its citizens. I could go on, but you get the point.

Deep learning is an amazing tool and its power to improve our lives in countless ways is fueled by digital twins technology and its capability to generate mass amounts of data. If Aki is right and data is the new currency, will we see a new era of deep learning super heroes emerge in the future—digital triplets?