This chip could massively increase autonomous car computing power and save energy

There is no easy way around this, though.

Self-driving cars rely on Artificial Intelligence (AI) to work. Packed with computing power, autonomous vehicles (AV) are outfitted with cameras, sensors, and software systems to track other vehicles, pedestrians, and unexpected obstacles, make accurate decisions based on training data, and so on. The seemingly most straightforward things are highly complex.

Light drives this autonomous car

In what looks promising, startups have now begun to target the "brains" behind autonomous vehicles. A startup called Drive.ai, founded by a group of former lab mates out of Stanford University's Artificial Intelligence Lab intends to use deep learning software to train cars to become better drivers. Silicon Valley-based Nvidia, already a global leader in AI hardware and software, is ready to launch a powerful processor named Drive Thor in 2024. The processor could be a game-changer in the EV industry and ensure a smooth merge into fully autonomous self-driving vehicles.

The shortage in chip supply continues to affect the automotive sector. Carmakers are looking for more ways to use semiconductor chips to offer customers more options and lower production costs.

At a time like this, Lightmatter, a company that emerged from a research lab at the Massachusetts Institute of Technology, is busy building the brains of a self-driving experimental vehicle using photonic or light-based chips, unlike traditional computer chips that use transistors to control the flow of electrons through a semiconductor.

In collaboration with Harvard University and Boston University, Lightmatter has received $4.8 million in funding from Intelligence Advanced Research Projects Activity (IARPA), the research and development arm of the Office of the Director of National Intelligence.

Mind you, Lightmatter is not just another AV company; they focus on developing photonic-based AI processors, not vehicles. "Our project is not only to deliver a photonic computing solution designed for general-purpose AI and machine learning. Part of the program is to target AV applications and develop a suitable algorithm," Darius Bunandar, Lightmatter's Chief Scientist and co-founder, tells Interesting Engineering in an interview.

Great scalability and performance advantages

A photonic chip uses light to carry out computing and interconnect the physical or logical connection between two electronic devices or networks.

"When you're using electronics, you're still using light, but that is electronic (Radio Frequency) light which oscillates only at a few gigahertz. When you use photonics, light oscillates at a few 100 hertz - and the type of light that we use is infrared. What is different about computing and interconnecting using photonics is that it is essentially a faster and more efficient use of power," Bunandar says, adding, "Therefore, your chips are cooler than transistor-based products, meaning traditional Silicon CMOS (complementary metal–oxide–semiconductor) electronics."

Using light to do the same actions done by electrons in traditional computer chips is the biggest competitive advantage that Lightmatter has.

"It is the building block of our photonic processors. The way you're manipulating information in light is slightly different in terms of device, but the logic block is still the same - most AI and machine learning applications use linear algebra. You're building your algorithms through two operations, multiplication and additions. And so we're doing multiplication or additions in light, while traditional CMOS would do it using electrons," says Bunandar.

Reducing energy consumption and environmental impact

A fundamental challenge that Lightmatter set to accomplish revolves around the now non-existent Dennard scaling, also known as MOSFET scaling. This principle states that, as transistors get smaller, their power density stays constant so that the power use stays in proportion to the area.

"If you look at the trends of electronic CMOS, for the past 70-80 years, we've improved transistors from a very simple building block to a very sophisticated computer today. So the one challenge is trying to integrate more and more transistors within a single chip," says Bunandar.

According to Moore's law, the number of transistors in an integrated circuit (IC) tends to double every two years. However, as per Dennard's Law, power consumption reduces as the dimensions of a device go down. Smaller transistors use less power and cost less. So, basically, one could cram 1.5 to 2 times the transistors on a chip as before for the same power consumption. "However, since the 90s, the size of transistors has been shrinking way too small, and they weren't so efficient anymore," Bunandar points out.

Transistors shrank to such an extent that they began to leak, overcoming Dennard scaling in 2004 because current and voltage kept dropping despite maintaining the "dependability" of integrated circuits. Current leakage and threshold voltage became major factors in establishing a power baseline per transistor.

"So now, the moment you've packed twice the amount of computing, you're going to be burning a lot more energy. The transistor is not scaling, in terms of its power efficiency, as much as before," says Bunandar.

The company's mission is to reduce the amount of heat generated by chips "so that we can actually reduce significantly the energy consumption that is consumed by high-performance computing, such as AI and machine learning, and at the same time deliver that massive compute performance that is needed for these types of sophisticated algorithms," explains Bunandar. At the same time, reducing environmental impact.

The AV of the future will be a combination of electronics and photonics. Here's why

The idea of using energy-efficient photonic chips as the brains of an AV vehicle came from gauging the challenges in the AV industry.

"Think about an AV - you've got a car that's typically driven by some kind of battery or fuel tank, and it's got a limited amount of fuel. If you're burning too [much energy on] compute for your AV, you're going to be significantly reducing the range at which that vehicle can go. Also, AVs are sort of like moving data centers, they have a lot of computing because there are sensors, cameras, Lidars, a reader, and ADAS (Advanced driver assistance systems) systems. There is a lot of processing that must be done, and most of it uses machine learning. We wanted to reduce the amount of energy consumption for this process to maximize the range at which these vehicles are going," explains Bunandar.

The company points out that the entire vehicle will not be running on these light-based chips; instead, it will use standard electronics and photonics.

"Photonics is really good for processing multiplication and addition algorithms which is what 90-95 percent of AI and machine learning do today. But you still need electronics to do other things," says Bunandar.

For example, AI algorithms are used to help AVs perceive the world, and detect cars, bikes, traffic lights, and road signs. "It needs to do what we call trajectory, prediction, and planning. And some of those algorithms can be performed in photonics; but we still use electronics, we use an x86 CPU to help us do all the other algorithms. A lot of algorithms are working in tandem to ensure that the AVs are working safely and comfortably for the users," Bunander tells IE.

Simulations showed promising results

How far is the company from a working prototype? "It has only been about nine months. So far, we've completed all the software simulation tests. We simulated the AV running using traditional computing. When we simulated this AV on a road with other vehicles, we were able to show that we could operate it safely," says Bunandar.

The next step is to get hardware-in-the-loop. This means testing and validating software systems on test benches that receive data inputs from cameras and radars.

"We'll be integrating a compute server that has electro-photonic computing hardware into the AV. Before we put that in the AV, we will rerun all the software simulations with the server itself. So that we're 100 percent sure about safety. From the software simulations, I think the results were very, very promising - you can reduce power consumption while getting much more performance when running an AV. After we get those hardware-in-loop simulations, the next step is to integrate that with the buggy," explains Bunandar.

Lightmatter has created benchmarks to help deliver a safe AV system

Meanwhile, the company is conducting stress tests on the chips to validate their performance in various traffic situations.

According to Bunandar, there are several AV companies but no standard metrics or benchmarks to gauge the performance of an AV computing system. "And that performance-heavy computing system cannot be just around how many operations you can do or how much power you're consuming," he says.

Instead, it must revolve around the scenarios a driver sees while driving a car. "Like a scenario when you're driving on the road, and another vehicle cutting in in front of you, or when you're driving following another car, and suddenly that car moved out of the way because there's an obstacle in front of it. So, these sorts of safety situations are important. We identified all these safety situations and we created some sort of a benchmark, a metric that allows users to know, whether that computing system will deliver a safe autonomous vehicle system," says Bunandar.

To an outsider, there still seem to be many technological challenges in implementing this idea. Is it the manufacturing of the chip? Figuring out how to extract the proper computing performance to aid the self-driving AI? Or is it the AI that needs work?

"Manufacturing is definitely a challenge," says Bunandar. "That's something that we have to figure out as a company. In terms of figuring out how how to perform AI using these photonic processors, that was something that we did at a research lab at MIT. That is how Lightmatter started. But in terms of manufacturing and supply chain, a company has got to figure that out because that's a production challenge," he says.

How to tell when your car is powered by light

Is self-driving AI one of the chief computing challenges in the future? I ask Bunandar. "I think there are a lot of computing challenges out there. AV is one area where it's very exciting that you've got autonomous cars that can be safer than human drivers. But you can also look at different applications in computing," he says.

The hardware that Lightmatter delivers is targeted for general-purpose AI on machine learning. According to the chief scientist, the same kind of system can be used for any high-performance computing application.

"I'm also excited about things like N-body simulations or density functional theory calculations. Applications around AI, machine learning, generating images, and talking to humans are very exciting. You can also use this hardware to do high-frequency trading, for example. What I'm excited about is that we can deliver this high-performance computing that can change people's lives without the environmental impact that we would have to bring if we were to do that using traditional CMOS electronics," says Bunandar.

With photonic chips set to revolutionize the AV industry, will a driver even know whether their car uses light-based chips?

"From a driver's perspective in terms of safety or comfort, the person wouldn't know what chip is running the vehicle. What the driver would say is, 'well, this chip is power efficient, I can go further'. I think that's how the driver would know that the vehicle is running on a photonic AI chip," adds Bunandar.