Tom Bacon outlines some of the risks that arise from working with the ‘invisible’ systems of RM, big data and analytics systems

Airline revenue management (RM) systems, like many modern applications of big data or analytics, are invisible. The forecasting methodology is often highly complex, tapping into millions of calculations on millions of data points. 

Optimisation is even more obtuse as the system runs linear programming routines across fare levels and markets, resulting in allocations by fare by ultimate destination on each flight. It is virtually impossible to reverse engineer the process for a specific flight in a typical hub complex, let alone a market or days’ worth of flights.

Increasingly, in the new world of big data, companies are facing such ‘invisible’ systems – incapable of complete understanding by any manager or analyst.

So, our job as managers of such systems is to manage the invisible. As demonstrated in the example of airline RM systems, invisibility often comes with some unfortunate attributes. These include:

• Over-confidence
• Lack of accountability
• Focus on the wrong things
• Use of inappropriate short-cuts

Needless to say each requires special management attention. Let’s take a closer look.

Over-confidence: Analysts may sometimes defer too much to the ‘sophisticated’ statistical modeling. Since it is too complex to follow, it is by definition ‘better’ than what they can do – or even understand – themselves. This is not the way to manage the invisible! Analysts need to understand the overall system logic and regularly question results. Appropriate challenges to the system come in the form of disciplined tests of hypotheses: ‘Is demand better than expected? Would a 10% increase in forecast demand result in superior performance?’

Lack of accountability: Managers and analysts can become less accountable for results as they can point to ‘the system’ as the ultimate determinant for pricing and other business decisions. Analysts may argue that they are merely following what the system says. This is not acceptable!  Analysts must be measured on the results and need to intervene – adjusting the forecasts or the allocations – when appropriate. The system does not work optimally completely on ‘auto pilot’.

• There are many factors that simply cannot be foreseen by the model. A major competitive move on scheduling or pricing, a sudden economic downturn, a major event – analysts need to stay apprised of macro and micro changes in their markets and make appropriate adjustments to the model.

Use of inappropriate shortcuts: To make a complex system more manageable, analysts tend to apply simple heuristics. In fact, too-simplistic heuristics may tend to drive too much intervention into the RM system:

• Missing the finance department load factor forecast should not mean that we need to adjust the demand!
• Booking velocity higher than last year does not necessarily mean demand should be adjusted up; this could mean the booking curve has shifted outward
• High numbers denied boarding last month doesn’t mean we need to be more conservative; the model properly trades off the cost of overbooking versus the opportunity cost of underbooking

Airline RM needs to avoid all-too-simplistic justifications for overriding the model. Intervention is important – but needs to be measured and well thought out, in recognition of the many complexities and inter-relationships of various factors affecting pricing and demand. Convening regular meetings of the analysts across markets – potentially daily – helps everyone see both the common trends and the market-specific irregularities and taps into the experience and analytical ability of the whole RM/pricing team.

Focus on the wrong things: In general, the output of the RM models is not the appropriate focus. The output is the culmination of sophisticated linear programming across fare levels and O&D’s. More appropriately, RM needs to focus on the inputs - forecast accuracy, or lack thereof, will more generally effect whether the airline is properly optimising revenue.

To sum up then, ‘invisible’ systems are a growing piece of management as more and more we rely on big data analytics to make business decisions in real-time. There are benefits to working with sophisticated systems but they come with some unique management challenges. RM departments, specifically, need to manage the ‘invisible’ effectively, taking into account possible over-confidence, reduced accountability, overly simplistic heuristics, and a focus on the wrong things.