A quick refresher on the three kinds of analytics:
1. Descriptive: collects data which describes customers’ past actions and behaviours, and analyses it to try to find patterns which explain that behaviour;
2. Predictive: takes the historical descriptive data and patterns contained within it and uses that to try to predict what those customer might do in the future;
3. Prescriptive: takes a more strategic approach and uses data, trends and predictions about customer behaviours to help inform recommendations on actions (such as process changes, changes in marketing tactics) which might help change the customers’ behaviours and move the company closer to achieving it’s business goals.
Defining a problem – asking the right question:
There are many kinds of things happening in business which we might term ‘problems’. In order to be effective at solving those problems, we need to develop a framework for defining the problem, and framing it in such a way that we can show what the problem is, how we know it’s a problem, what action(s) we believe are necessary to solve the problem, and – all important – how we will know whether we have solved the problem or not.
Framing the problem and solution consists of several steps:
1. Problem: What constitutes the problem, what does the ‘problem state’ look like, and what are the consequences of the problem which we’re not happy with? (E.g. Poor product quality is leading to increased complaints and returns/refunds which in turn is causing a significant drop in sales revenues.)
2. Solution: What are we trying to achieve in order to solve the problem, what is our desired outcome, what does the ‘solution state’ look like, and how will that impact on our business? (E.g. Customers are 100% satisfied with the product and are no longer complaining or returning products for refunds.)
3. Action: What are the ideal set of actions we believe we need to take to, and are able to take (i.e. they are within our control) to solve the problem and reach the solution state? (E.g. Undertake production process changes to improve the product quality, ideally returning product quality to it’s previously high standards or better.)
4. Goal: What quantifiable and measurable goal or objective (or metric) we can set which will allow us to measure and monitor the effect our implemented action/process (3) is having in moving us away from the problem state (1), and moving us towards the solution state (2) and when we have achieved the optimal/desired solution state? (E.g Monitor customer complaints and refund returns – including their reasons – to show whether production process improvements are leading to the improved product quality, as evidenced by reduced complaints/returns.)
5. Model: We can then build a model which represents the way in which the actions being recommended influence the achievement of the required goal. By solving the model for the optimal solution, we can establish what actions need to be taken in order to achieve the desired goal.
Example 1 – maximising sales volume:
We can observe from descriptive data how, when we increase or decrease the price of a product, the sales volume is likely to decrease or increase in line with the price change. Here the problem might be how much sales volume is decreasing, even while customer levels remain constant, resulting from production pressures causing us to increase prices to maintain margin. Our solution would be a pricing scenario where maximum sales volume is achieved for the given size of customer base. The action we want to take then is to set our prices at a level which achieves the goal of optimising sales volumes. We could go further and say that the action is not simply to set the optimal price once, but to implement better processes which improve and optimise our price-setting over the long-term, e.g. throughout the whole budget year or for the lifetime of the product.
In this case, the model we build comes from descriptive data collected about customers’ historical behaviour (sales quantity purchased) at different price levels. From this we plot a simple linear, one-to-one relationship between sales and price. Optimisation of the model can be solved graphically or mathematically to establish the price which optimises sales quantity.
Example 2 – maximising sales revenue:
Of course, we know that we can set the price to zero (i.e. give away the product for free) and we will shift the maximum number of units. But our revenue will be zero, which is not much use to us as a business – unless our goal was to shift hard-to-sell stock our of the warehouse to minimise storage/warehousing costs!
So instead we can set our objective to maximise sales revenue, not just sales volumes. We can still use the same model, based on the descriptive sales/price data as before, but this time to solve the model/equation to find that price which will maximise sales revenue. Here we take account of an additional factor – how changes in the two variables (selling price and quantity sold) affect sales revenues.
To account for this, we need to modify the model to reflect the added relationship of price and quantity on revenue. This is a simple case of reworking the original data points to calculate sales revenue in each case. By plotting these new data points – selling price against sales revenue – we find we have a new graph. This is no longer a straight line but a parabolic curve: a hump which starts low, rises to a maximum, then falls down again. Intuitively this makes sense: a low selling price leads to lower revenues despite the increased quantity sold, and higher prices will also generate a lower revenue due to the decreased quantity sold despite the higher price.
Again we have two choices for solving this graph – adding a best-fit curve and solving graphically by eye . This is generally adequate in simple cases. Alternatively, we could solve the problem accurately using calculus to find the maximum (the point where the graph turns, or the vertex) of the equation. The equation (known as a quadratic equation) will take the form:
y = a.x^2 + b.x + c
where:
- y is the dependent variable (e.g. sales)
- x is the independent variable (e.g. price)
- a and b are co-efficients of x – they modify the shape and size of the parabola being plotted (note is it possible for either co-efficient to be a negative number; where a is negative, this results in the two arms of the curve opening out downwards, or – in real-world terms – sales revenue is dropping either as price is decreasing (causing revenue to drop despite demand rising) or as price is increasing (causing demand to drop))
- c is the y-intercept – that is, some constant through which the line would intercept the y axis if the independent variable were zero (note, in the example used we expect c to be zero, since sales revenue will be zero when price is zero (product given away free))
Adding parameters to a model:
The above simple model works well when we know that the business has been able to isolate the two variables in question – so that the action taken (e.g. changing selling price) is the only factor which affects the outcome (e.g. sales revenue). In reality, we know this is unlikely in a complex business scenario (unless we did strict A/B testing to collect descriptive data specifically for this modelling purpose.) Where we know there will be multiple factors causing sales revenue to change, we can add parameters to our model to deal with their impact.
Example 3 – maximising profit:
A common example is where a company wishes to maximise profit instead of sales revenues. To do so, it will need to account for the cost of goods sold (production costs, fulfilment costs and so on), which – even where the unit cost of sale remains level per unit sold – will have the effect of reducing profit and leading to a different optimal selling price. There may be some prices where price is so low, and sales volumes so high, that sales revenue is not sufficient to cover costs of production and selling, resulting in a loss.
One way to solve for optimal price which will maximise profits is to find where is the maximum difference between the two graphs (one for sales revenue and the other for cost of sales) – the price at which the difference between the two curves is at a maximum represents the price at which profit will be maximised. However, a more elegant way to solve is to calculate the profit directly and plot this as a parabolic curve and solve that curve for the maximum price in the same way as was done in example 2.
This profit model will also show at which price the product will need to be sold in order to ensure sufficient sales revenue to cover production costs – the break-even price for the product – which is the point at which the graph crosses the x-axis.
In reality, costs of production may not be a straight line and may in fact decrease as manufacturing volumes increase, e.g. due to economies of scale. However, the principle will still hold for maximising profit, albeit with a more complex graphical model.
Gradients and optimisation:
Mathematically, the optimal profit occurs where the parabolic curve reaches the top and then turns downwards, i.e. at the vertex of the curve. The curve has a gradient which moves as you move along the curve (as you increase the selling price). In the example, at first the gradient decreases as it moves towards the turning point. After the curve passes its turning point, the gradient starts to increase again. At the turning point itself, the gradient is zero (in effect the graph is horizontal at that point).
Economically, the optimal profit occurs at that point where the additional or marginal sales revenue generated by increasing the selling price is the same as the additional or marginal cost of producing that same quantity of units sold, i.e. the point at which any additional revenue gained is balanced out by the reduction in costs, or where marginal profit is zero
This can also be stated as the point where marginal revenue equals marginal cost, which is the economic equivalent of the mathematical vertex or turning point, being the point where the curve’s gradient equals zero.
Competitive environment and pricing:
Another complexity is that of the market or competitive environment in which the company operates. This may alter the impact of price changes on the demand for the firm’s product in a number of ways. The customer’s brand loyalty to our product will affect their demand for the product at different price points. A higher brand loyalty results in a demand curve further to the right of the standard price-demand curve. Where the product is perceived to be a premium or high quality product, this may cause demand to drop off less rapidly at increased prices than the standard demand curve (the consumer is less price sensitive as they perceive the product to be a luxury item so they are willing to pay a premium price for it).
The same effect may be seen in industries where there is a monopoly situation (there are no other competitors vying for the buyer’s purchase) which allows the company to set prices artificially high without demand being diminished. Likewise, where there is a significant cost to the customer to switch to a competitor’s product, demand will not drop off as quickly at higher prices than would normally be expected.
Alternatively, where there is a strong competitive market with many competitors, the company will find demand is very sensitive to small changes in price, with customers far more likely to switch to a competitor’s product if our prices increase even a small amount. This will also be the case where there is low brand loyalty, or the product is not perceived as a quality/luxury product worthy of a premium price, or where the customer can easily switch to a competitor without any financial or other penalty.
Market size can also have an impact on pricing decisions. In industries where there are many potential customers for the product, the company is more easily able to change prices and, even though it loses some customers, they may be replaced by other customers. Whereas in an industry with only a small number of customers, those customers will be far more price sensitive and they may be more likely to switch to a cheaper competitor which has a greater impact on the firm.
Optimising for the correct objective (asking the right question):
While we can use one kind of model and method for optimisation, we can see by running a few simple examples with numbers that, by changing our required objective, we change the recommended action. For example, our recommendation to optimise sales volume would be to set price at zero, but this does not optimise revenue. To optimise revenue we set price at a certain level, but this does not account for the costs of producing the goods sold. So to optimise profit, by taking costs into account, we can end up with a different recommended action (optimal price).
So we need to be sure at the start that we are asking the right question – which means we need to be sure that we are optimising for the right goal or objective from the start. For this reason, I have stressed that prescriptive analytics (i.e. that branch of analytics which we use to generate recommendations to somehow improve the business’s activities) should stress the long-term or strategic aspect of the firm. There is no value in making recommendations which maximise revenue if our strategic objective is to generate profits (as will be the case in most growing/mature businesses).
We should remember that the business does not exist in a vacuum but rather is part of a larger marketplace filled with competing firms, who are likely to engage in actions in direct response to our own actions. (This interplay which exists between competing firms in the marketplace is known as Strategic Interaction.) Maintaining a strategic approach when preparing recommendations through the use of prescriptive analytics (indeed, ensuring the competitive element is in-built into the prescriptive models), ensures the business will maximise it’s actions and results with respect to both it’s customers and it’s competitors.
Correlation is not causation:
In concluding, it’s worth mentioning the cautionary note about analytics. We may see a pattern or trend within our descriptive data, but we should be careful to ensure that pattern or trend really is a causal relationship between two variables and not just some correlation caused by some external factor. To overcome this, we should run various tests and models, such as split testing, to confirm or deny the existence of the perceived causal relationship.
If we don’t take head of this caution, if we mistake a correlation in our descriptive data for some causal relationship which really isn’t there, we can end up giving the wrong recommendations about the wrong actions. And we may even end up making the problem worse, not better.
An illustrative example – online advertising:
We want to be able to monitor attribution, that is which advertising method (or specific ad) generates which sales, so we can evaluate which ads and campaigns were more effective (for a given marketing spend). That way we can plan to repeat more of the effective campaigns and drop (or modify) the less effective campaigns from our marketing arsenal. Since we can’t know which ads a customer saw but did not click on but which ultimately helped push the customer to a purchase, the best way we have of monitoring ad effectiveness is by recording click-through rates (i.e. ratio of the number of customers who clicked on the ad to the number of people who saw that ad).
We aim to show the same ad to the same customer on a number of different channels (different websites, blogs, social media, search engines, etc.). Intuitively, as the customer sees our ad on more channels, we can expect them to be more likely to make a purchase from us, since the probability of them clicking through on an ad is increasing for each time they see one of our ads as – on each subsequent ad viewing – the ad becomes increasingly impactful. That is, Conversion is a function of Number of Channels & Click-Though-Rate (CTR).
We need to be careful with online advertising in that some networks (channel owners) can monitor their visitors and identify those who are more likely to click on ads – then chose only to show ads to those who are likely to click through (thus driving up their click-through-rates and so appearing to businesses as being more worthy of investing online advertising budgets with). A click-through does not necessarily mean a qualified lead or prospect.
One way to get around that problem is to use targeted advertising. Here the potential customer is only shown ads after they’ve already visited the company’s website (hence indicating they already have an interest in the company and its products). Tests have shown that there is a good increase in click-through rates (a 3+% likelihood of click-through) after showing the ad the first time after the prospect visited your website. But subsequent shows lead to a marked drop off in likelihood of click-through.