This is the third in a series of 3 blog posts that serialize the use of Artificial Intelligence and Mathematical Optimization in business and public sector organizations.
The blog posts follow the same structure to summarize:
- where Machine Learning (ML) as a generalization of AI has found its place in process automation and decision support, and reasons why many potential applications have been slower to see value from ML
- what Mathematical Optimization (MO) does in comparison with ML
- how MO and ML can work together in hybrid solutions that offer more value than the sum of their parts.
In the last blog post we described some fundamental aspects of Mathematical Optimization as needed to highlight differences and similarities with Machine Learning and the challenges still faced by MO.
In this blog post we reveal that some of the challenges facing ML and its acceptance for automated or augmented decision-making are addressed by MO, while MO itself benefits from relaxing the need for facts and embracing the predictive nature of ML. We present evidence that, while organizations will continue to find value in new use cases of ML and MO on their own, more value will come from hybrid solutions.
Hybrid Solutions from the MO perspective
In short, a hybrid solution uses ML and MO together to provide actionable decisions based on both facts and predictions. One way to look at a hybrid solution is from the perspective of the MO solver, whereby ML provides the solver with information that it lacks. TABLE 3 summarizes three ways in which ML can “help” mathematical optimization, which we will call “Helper Types”.
TABLE 3 – Helper Types for ways that ML supports MO in a hybrid solution.
In a less obvious example, the “Do what you’re told” Helper Type, ML is telling us what the real-world constraints are. This also enables an MO solver to apply more constraints than it would otherwise. Remember, despite the name, constraints are a good thing (ask your nearest business controller 😊)
FIGURE 5 shows a similar MO solver as we presented in the last blog post but adapted to make more valuable decisions, as they would maximize a more valuable objective: units sold, rather than units on sale.
FIGURE 5 - Components of an MO solver frustrated by missing data or relationships.
This objective function however is unknown – in need of Helper Type “Reach for the goal”. The objective function also depends on an unknown trait (number of customers per store) – in need of Helper Type “New traits for old”. Finally, we have reason to believe that one of the external constraints (maximum units per delivery, dictated by the delivery company) is too high for our own interests – Helper Type “Do what you’re told” can recommend a new maximum by learning from historical data.
Now let’s imagine two ML models for Helper Types “New Traits for old” and “Reach for the gold” (FIGURE 6), applicable to the sales-in-stores problem above.
FIGURE 6 – Machine Learning models set up to help an MO solver in two ways.
These ML models enable the hybrid solution in FIGURE 7, which optimizes for sales rather than units on sale:
FIGURE 7 – Components of a hybrid solution
The benefit of adding ML in this example is clear: a more valuable objective function is optimized. In the hybrid solution, the value is greater than the sum of its parts (ML and MO) whereas the increased complexity comes only from the need to build both ML model and MO solver, i.e. exactly the sum of its parts.
Similarly the need to maintain a hybrid solution is no more complex than the individual need to maintain each ML model and MO solver to keep up both the accuracy of the predictions and the relevance of the decision-maker to our business goals.
These observations are likely to accelerate the adoption of ML and MO together, and moreover, use cases that were previously just not feasible for MO alone, become possible by the introduction of ML.
Another thing that ML “adds” to MO in a hybrid solution, is the probabilistic property. The ML components give the most likely predictions based on a sample of data. No matter how clean a sample of data is, it is only a sample, and two samples that represent the same reality can paint different pictures of that reality.
If we build more than one ML, each learning the same thing but from different samples, they too will be different and give different predictions for the same observation. While this is considered a limitation of the hybrid solution, we can quantify this limitation by observing:
- How much do the decisions fluctuate when models are re-trained on different samples?
- How much does the final objective (e.g. maximized revenue or minimized cost) change?
Observing these fluctuations or “uncertainty” in a hybrid solution gives a sense of how close we expect to be to a truly optimal process and although a ‘sense’ is not an absolute measure, its quantification allows us to monitor and further improve the hybrid solution.
Summary
We identified a behind-the-scenes class of use cases for ML, where the technology is applied to back-room processes and decision automation rather than directly as a part of a product or service.
For these use cases there is a gap in ML success stories, and we have covered some conceptual and theoretical reasons why ML and MO do not reach all organizations today. Looking at hybrid solutions from the perspective of how ML can help MO, we categorized the role of ML into three “Helper Types” that should go some way to filling the gap in ML success stories.
For those familiar with challenges to AI, the take-home message is that MO solves the well-known bottleneck of “how do we realize the benefits?”. For those more familiar with MO, the take-home message is that ML opens the door to problems traditionally ruled out because “we don’t have the data”.
The synergy between ML and MO is now starting to receive the attention it deserves, and we are starting to see real success stories in the industry such as optimizing power allocation when faced with unknown (but predictable) demand and external factors. Further evidence that we are approaching the advent of hybrid solutions comes from trends in google searches for the two technologies over the last 7 years (FIGURE 8).
FIGURE 8 – trends in searches for Machine Learning and Mathematical Optimization (% change in search volume relative to Nov. 2015 and normalized to a common maximum in Nov. 2022).
Online interest in ML appears to slow down between 2019 and 2021 (consistent with Gartner’s Hype Cycle in the first blog post of this series but the interest is again on the increase throughout 2022.
While the interest in MO has been lower, more volatile and perhaps even slowly falling between 2015 and 2022, an increase in 2022 is clear and follows the same pattern as for ML.
Whether interest in ML is driving the discovery of MO or vice versa, the duality of recent interest would be easily explained by the realization that their combination offers more value to more use cases.
While increased interest runs the risk of inflating expectations (a new ‘hype’), we have presented reasons to believe that this time, the expectations will be met and we will be firmly placed on Gartner’s “Plateau of Productivity” as proposed in FIGURE 9.
FIGURE 9 – an edited copy of Gartner’s Hype Cycle, proposing that the use of MO in benefits realization is the next big thing pushing us onto the plateau of productivity.
Conclusions
- Without MO, Machine Learning creates value
- the impact on business objectives may only be inferred, and
- full automation is not intended to completely replace human control
- where applied directly in products/services
- behind-the-scenes, where decision-making can be streamlined but
- also taking a “bottom-up” approach to finding new business opportunities in our predictive capability, rather than addressing the known business problems.
- Without ML, Mathematical Optimization creates value behind-the-scenes by
- automating any number of decisions simultaneously, even if they affect each other
- affecting real-world business objectives directly,
- applying real-world constraints, and
- from a “top-down” perspective where we start with an explicit problem to solve
- By combining ML and MO, hybrid solutions
- are applicable in more scenarios than either ML or ML alone,
- offer more value in these scenarios than ML alone at the expense of higher complexity
- give decision makers more confidence and “dials to turn” when controlling certain aspects of business according to strategic goals
- What ML adds to MO is in making more use cases possible and enabling higher-value objectives & constraints.
- What MO adds to ML is in translating predictions into actions that optimize real-world objectives.
- Hybrid solutions are complex and ’living’ solutions, calling for
- close consultation between Data Scientists and stakeholders in IT and Business to form a Business Case before spending too much time on a PoC
- Sustained relationship between Data Scientists and IT/Business stakeholders to maintain the solution, update the model/solver as external factors change over time and evaluate the quality, expected benefits and actual benefits continuously against the original business case.
Didn't read the previous posts yet? Don't worry, they are here waiting for you.