An aerial view taken of the Palisades Fire on January 9, 2025. Alphabet’s moon factory, X, has … [+]
Update, January 13, 2025: This article, originally published on July 7, 2024, has been reprinted with information about how Alphabet’s Bellwether system is helping with the LA wildfires.
Fires broke out in the Pacific Palisades neighborhood last week, and the area is still far from clear as more strong winds are expected. There are now six fires burning across the LA region, and the death toll from the Palisades and Eaton fires alone has reached 24.
These fires provide a stark reminder that first responders like the National Guard need quick access to the most actionable information. To respond to fires, floods and other increasingly common disasters, rangers must be deployed rapidly exactly where they are needed.
This need presents one of the greatest challenges imaginable for information technology as a field: Beyond the many square miles affected, which school, bridge or neighborhood is most in need of immediate help? Often, lives are at stake.
Google’s parent company Alphabet has a factory called X that aims to solve some of the world’s most critical and vexing issues, and X has developed a breakthrough solution — one that uses predictive AI in the same way that many companies do. Here’s how X did it, how it’s useful with the LA fires, and what all business professionals need to learn about combating uncertainty and risk with machine learning.
Problem: Labeling aerial photographs
Both during and after a severe weather incident, drones and manned aircraft collect thousands of aerial photographs of affected areas. These images potentially reveal which buildings and other infrastructure have been affected – but only after each one has been tagged with exactly what location it shows. Unfortunately, images generally lack this metadata.
Tagging photos manually slows down the National Guard response tremendously. After an incident, it usually takes his team about 12 hours to complete the task. Unfortunately, this process has so far remained manual. It is a challenging task to automate as the photos are taken from different heights and at steep angles.
But that’s exactly the kind of problem X was designed for: The stakes couldn’t be higher, yet it requires a technological breakthrough. X’s initiative to take on this and related challenges is called Bellwether, described as “the first prediction engine for Earth and everything on it.”
Sarah Russell, who has run Bellwether since she founded it in 2020, makes the case. “We took on this challenge because we realized that if we could solve it, we would shorten the response time to climate disasters and multiply the number of lives saved.”
Solution: Photo matching with machine learning
The revelation? Matching real photos with artificial ones. Bellwether has synthesized a database of simulated reference photos to use as exemplars. When a real photo matches one from the database, it is tagged – then the system knows exactly where and what a photo is. To synthesize the reference imagery, X used Google’s wealth of unique geospatial resources, the fundamental basis for products such as Google Earth and Maps.
It works. Just a few years after Bellwether formed and began working on this solution, the National Guard and other organizations are already deploying it. The Bellwether team has started working on aerial images from the LA area, which are just starting to come in as flight restrictions ease
With this solution, National Guard team members can immediately review the most affected areas and know which countries they are looking at. They can tell which bridges are out. They can ask the monitored areas, “Show me all the hospitals”. They can provide informed answers immediately, eliminating the processing delays that have held them back for years.
Machine learning plays a central role in moonshots like this — just as it does in more mainstream enterprise systems. After all, photo matching is exactly the kind of imprecise process that ML handles well. No match is a sure thing, as aerial photos do not match exactly. Each originates from a unique distance, magnification and angle, is potentially blocked by weather conditions, and the landscape they capture is often affected, sometimes catastrophically.
ML eliminates much of the uncertainty by assigning a confidence level to each match. With many photos coming in, it turns out that quite a few match with high confidence, so the system can provide visuals to operatives covering almost all affected locations, even after discarding those that didn’t find a sure match. .
National Guardsmen assist residents affected by flooding caused by Hurricane Harvey in 2017.
This approach is expansive. “Extending beyond our deployment with the National Guard, our goal is to make this type of service fundamentally easier for a broader group of disaster responders,” says Russell. “It can be applied across rescue and reconstruction responses to various weather-related phenomena, including heat waves and tornadoes, for example.”
The universality of predictive AI
Whether you’re shooting for the moon or hunting for more typical enterprise goals, ML’s core capacity to generate levels of trust solves operational challenges universally, across industries. Which customers are likely to buy? Marketing targets them. Which transactions are possibly fraudulent? Banks block them. Which addresses are likely to receive a delivery tomorrow? UPS plans for them.
This well-known paradigm—running large-scale operations with ML predictions—has a name: predictive AI. It is the practice of systematically filtering out cases that show less confidence and taking action on the more certain cases that remain.
So how safe is safe enough? That depends. Each project must determine the best choice of decision threshold based on practical necessity. For example, the National Guard needs photos that match very high confidence. In contrast, marketing and fraud detection can afford to target many cases that don’t pan out—an inevitable part of the numbers games these types of operations inevitably play.
In other words, predictive AI reduces uncertainty by quantifying uncertainty. Bellwether is working to expand this extraordinary approach in other ways that will reduce the damage caused by climate disasters, such as predicting where the most lives can be saved – which affected areas should be the highest priority for aid of evacuation – and predicting environmental incidents before they occur.
“ML has become the new paradigm for the Earth sciences,” says Russell. “Until recently, for example, hydrology primarily predicted flooding with location-specific models. Now, with ML, the best models are developed using data from different locations—the flood behavior of the U.S. East Coast can be used to predict flooding on the West Coast”.
