“There are numerous AI techniques of which machine learning is a subset,” he said. “While deep learning has been highly successful, it is also currently limited because there is little visibility into its decision rationale. Until we truly reach a point where this technique becomes fully “explainable”, it cannot inform humans or other automation as to how it arrived at a solution, or why it failed. We have to realize that deep networks are just one tool in the AI tool box.”
And, humans have to stay in the loop.
“Imagine you have an automated threat detection system on the bridge of your ship, and it picks up a small object on the horizon,” he said. “The deep network classification may indicate that it is a fast attack craft coming at you, but you know that a very small set of uncertain pixels can mislead the algorithm. Do you believe it?
“A human will have to examine it further. There may always need to be a human in the loop for high risk situations. There could be a high degree of uncertainty and the challenge is to increase the classification accuracy while keeping the false alarm rate low – it is sometimes very difficult to strike the perfect balance. ”
The problem with machine learning
When it comes to machine learning, the key factor, simply put, is data.
Consider one of Mittu’s previous projects: an analysis of commercial shipping vessel movements around the world. The project’s goal was to use machine learning to discern patterns in vessel traffic to identify ships involved in illicit activities. It proved a difficult problem to model and understand using machine learning, Mittu said.
“We cannot have a global model because the behaviors will differ for vessel classes, owners, etc.” he explained. “It is even different seasonally, because of sea state and weather patterns.”
But the bigger problem, Mittu found, was the possibility of mistakenly using poor quality data.
“Ships transmit their location and other information, just like aircraft. But what they transmit can be spoofed,” Mittu said. “You don’t know if it is good or bad information. It is like changing those small numbers of pixels on the dog image that causes the system to fail.”
Missing data is another issue. Imagine a case in which you must move large numbers of people and materials on a regular basis to sustain military operations, and you’re relying on incomplete data to predict how you might act more efficiently.
“The difficulty comes when you start to train machine learning algorithms on data that is of poor quality,” Mittu said. “Machine learning becomes unreliable at some point, and operators will not trust the outcomes of the algorithms.”
Current work in AI
Today Mittu’s team continues to pursue AI innovations in multiple areas of the field. They advocate an interdisciplinary approach to employing AI systems to solve complex problems.
“There are many ways to improve predictive capabilities, but probably the best–of-breed will take a holistic approach and employ multiple AI techniques and strategically integrate the human decision-maker,” he said.
“Aggregating various techniques (similar to ‘boosting’), which may ‘weight’ algorithms differently, could provide a better answer, or learning combined with reasoning, etc. By employing combinations of AI techniques, the resulting system may be more robust to poor data quality.”