Enabling data-driven results

evergreen · May 9, 2023

In today’s fast-paced business environment, integrating data and Artificial Intelligence (AI) solutions into business processes has become crucial for companies looking to stay ahead of the competition. By incorporating these technologies into their operations, businesses can gain valuable insights into customer behavior, streamline processes, and make more informed decisions. This article explores how this can be done and highlights some of the key benefits of this integration.

The concept of a system is central when analyzing organizations. A system is a collection of interconnected elements that exhibit a set of behaviors or functions.¹ With this definition, a business can be defined as a system that transforms resources into products and services. The business processes are interrelated events and transformations that produce those outputs. Finally, the operations within the process can be defined as the specific actions performed on the resources by machines and workers to accomplish those transformations.² By understanding these different components of the systemic analysis of businesses, we can gain a deeper understanding of how it works and identify opportunities for improvement.

Process inputs flow through a chain of four transformations (circles) to process outputs; above each transformation, a triangle marks the machine or person that carries it out. — How a business process turns inputs into outputs — a chain of transformations, each carried out by machines and/or people. Adapted from Falconi.

Decision support systems

To integrate data and AI solutions into business processes, companies often deploy computational systems that collect, store, and analyze large volumes of data and inform real-time decisions. These decision support systems leverage process data to improve business results, identifying patterns and opportunities for improvement. They can generate recommendations and alerts that help analysts and operators make more informed decisions, driving business results. The improvements come by leveraging data-driven insights to change operations: reducing costs, improving quality, and increasing efficiency.

A 'decision support system' box sits above the process. Dotted lines draw process data up into the box and return process insights down to each operation, while the process runs inputs through its transformations to outputs. — A decision support system sits alongside the process — drawing on process data and returning insights that inform the operations.

A decision support system can be characterized by its type of solution, including descriptive, predictive, and prescriptive systems. Descriptive solutions focus on describing what has happened in the past and what is happening in the present. In contrast, predictive solutions concentrate on identifying patterns and forecasting what is likely to occur in the future based on historical data. Prescriptive solutions use optimization and heuristics techniques to identify the best course of action given a set of constraints and objectives.

While prescriptive systems are currently in the spotlight due to recent advancements in AI, both descriptive and predictive solutions can provide valuable insights and add business value. Besides, descriptive and predictive solutions can serve as intermediate steps towards the more advanced prescriptive one, allowing businesses to build a strong foundation of data and insights before diving into more complex AI-based options.

The building blocks of decision support systems

Let’s examine the decision support system in more detail. It consists of five primary components:

Data ingestion component: collects data from different sources and loads it into the system for analysis.
Data transformation component: cleans, processes, and transforms data in preparation for analysis by removing invalid data and extracting meaningful features.
Predictive modeling component: generates predictions and identifies phenomena by building statistical and machine learning models using historical data.
Optimization and heuristics component: generates recommendations to optimize process results based on the data collected and analyzed by the system using optimization algorithms or heuristics.
Delivery and visualization component: presents insights and recommendations in real-time through data integration, interactive visualizations, alerts, and notifications, allowing users to gain actionable insights into business processes.

Five components chained left to right inside one box, from process data to process insights: data ingestion, data transformation, predictive modeling, heuristics and optimization, and delivery and visualization. — The five main components of a decision support system.

Let’s see how those components can be combined when building descriptive, predictive, and prescriptive systems.

Decision support system interfaces

Integrating decision support systems into business processes requires at least two interfaces: the data ingestion interface and the delivery and visualization interface. The data ingestion interface collects historical and real-time data from various sources, such as sensors, databases, and external APIs. It then processes it into a format that the system can analyze. The delivery and visualization interface delivers insights and recommendations the system generates for business operations, informing decisions and actions that improve process results.

The same component box, but only two components are present — data ingestion on the left and delivery and visualization on the right — marking the system's two interfaces with the process. — A decision support system's two interfaces: data ingestion and delivery and visualization.

Technical details about data sources are essential in designing the system’s interfaces, but business and domain knowledge is also central. This knowledge helps to collect and process relevant data or deliver actionable insights aligned with process goals. For example, when building the data ingestion component, some considerations include identifying available variables and their meanings, identifying where datasets are stored and how they can be accessed, and determining whether external data should be acquired to augment available datasets.

Similarly, in the delivery interface, besides the details of the target systems, it’s crucial to understand how business operations will use the insights generated by the system and how to present them in a timely and actionable way. When building the delivery and visualization component, some considerations include identifying the variables that drive value in the process, determining each variable’s possible/reasonable values, and deciding who/what should receive this information and at what frequency. By designing business-oriented interfaces, decision support systems can integrate better with business processes and provide valuable insights to drive results.

Descriptive systems

By adding the data transformation component, we have a descriptive decision support system. These systems focus on describing what has happened in the past and what is happening in the present. The data processing module collects and transforms data into a format that can be analyzed and presented effectively, allowing users to understand and make decisions about business processes.

The component box with data ingestion, a highlighted data transformation component, and delivery and visualization; the predictive modeling and optimization slots are left empty. — The components of a descriptive decision support system.

The data transformation component often features data quality features checking for incomplete and inconsistent data. A strong emphasis is given to getting a dataset that best represents the assessed phenomena. This curated data is then articulated with data storytelling techniques to generate visual insights about the business process and inform operations. This type of system is discussed extensively in the context of Business Intelligence.

Predictive systems

On the other hand, the predictive modeling component adds a layer of statistical and machine learning techniques trained on historical data. This training allows the models to identify patterns and relationships between variables. After learning those structures, it can detect anomalies, classify events, and predict/forecast what is likely to happen.

The component box with data ingestion, data transformation, a highlighted predictive modeling component, and delivery and visualization; the optimization slot is left empty. — The components of a predictive decision support system.

This component implementation can involve various techniques, frequently using supervised learning approaches. The methods range from classical linear models to deep neural networks, and there are many ways that these predictive models can be used to drive business value, among them:

Predicted values: classifying events and predicting future occurrences can generate business value by streamlining analysis and anticipating changes.
Value ranges: sometimes, a range of “possible values”, such as confidence or prediction intervals, can be equally or more valuable than point estimates.
Explanations: model explanations can give insight into the variables affecting a predicted output. This information can inform operations, generating better process results.
Simulations: value manipulations can be performed on process variables to gain insight into how the results would be impacted.

Prescriptive systems

Finally, prescriptive solutions use optimization and heuristics techniques to identify the best course of action given a set of constraints and objectives. The optimization and heuristics component combines the insights the predictive component provides with the possible actions of operators and machines, generating recommendations to solve problems and get better results.

The full component box with all five components present; the heuristics and optimization component is highlighted. — The components of a prescriptive decision support system.

The techniques associated with the heuristics and optimization component can range from simple “if-then” statements to complex reinforcement learning algorithms, passing through classic optimization techniques such as linear programming and simulated annealing.

While, in some cases, the results generated by these prescriptive systems could be directly integrated into machines performing the operations, it is advisable to blend human specialists into the decision loops whenever possible. This approach, called collaborative intelligence or human-in-the-loop, can improve overall system results while avoiding catastrophic errors. Decision support systems are copilots; they still need a pilot.

Concluding remarks

Business results happen in business processes, not in their decision support systems.

The public debate around data and AI advancements tends to focus on solutions, overlooking the problems that motivated those solutions in the first place. Nonetheless, the best starting point when considering buying or building descriptive, predictive, or prescriptive systems is a clear assessment of your current business processes and their problems.

While reaching for the latest prescriptive AI-powered system to solve your business problems might seem appealing, a more gradual approach often brings more business value while reducing the chances of failure. Investing time in data integration and quality is essential before diving into more complex solutions. Finally, we must remember that decision support systems are as valuable as they can influence business processes and operations. Business results happen in business processes, not in their decision support systems.

For those wanting to further explore combining predictive models and optimization engines to build prescriptive decision support systems, I recommend looking into the drivetrain approach by Jeremy Howard et al.³ and the regression optimization approach by Dzung Phan et al.⁴

Meadows, D. H. & Wright, D. Thinking in Systems: A Primer. (Chelsea Green Publishing, 2008). ↩
Falconi, V. Daily Work Routine Management. (Falconi Editora, 2006). ↩
Howard, J., Zwemer, M. & Loukides, M. Designing Great Data Products: The Drivetrain Approach. (O’Reilly Media, 2012). https://www.oreilly.com/radar/drivetrain-approach-data-products/ ↩
Phan, D., Nguyen, L., Murali, P., Kalagnanam, J. & Liu, H. Regression Optimization for System-level Production Control. INFORMS Annual Meeting (2020). ↩