Optimization – The Goal and The Constraints

Optimization is neither an art or a science – it’s a constructive way to think.  It has a precise definition which is usually ignored.  But being disciplined in thinking about optimization is a highly practical way to be effective in technology, in business, in foreign policy, and even in charity.  Today’s post is just about technology and business, though.

The definition in wiktionary until just a few minutes OK was “the design and operation of a system or process to make them as good as possible in some defined sense.”  “them” is not only bad grammar here; it’s also wrong; so I changed “them” to “it” in wiktionary (nice to be able to do that).  A common fallacy which gets in the way of both clear thinking and effective execution to think that optimization can be measured by more than one output variable.  This iron rule applies to both business and social optimization but let’s start with business; it’s less emotional.

International VoIP wholesaler ITXC, the company Mary and I founded, wanted to optimize its handling of calls. Since use of the Internet gave us the ability to modify the routing of calls almost continuously, we wanted to build an automated routing engine to make these decisions in near real time and “optimize” our operation.  “We need to optimize quality and margin,” said I ignorantly.

“Which one do you want to optimize?” asked someone smarter.

“Both,” I insisted.  “It’s what our customers expect.”  Fortunately, we didn’t have a staff of yes men and women.  What I asked for is meaningless.  Suppose that margin is 10% and call completion rate (just one measure of quality) is 75% and that either can be made higher at the expense of making the other lower.  What does optimization mean given the requirements I’d first set out?  Nothing; it gives no guidance on whether to make call completion rate (quality) better at lower margins or to increase margins but complete less calls.

Thinking about optimization means thinking about the real goal of your business.  In most cases, that’s making money.  The first mistake in my thinking in this case was to talk about percentage margin rather than dollars of margin.  We could use cheaper, lower-quality suppliers to increase margin percentage on calls which did get completed but, in many cases, that would result in completing less calls and getting less margin dollars.  Also continually poor quality would result in losing business so we’d get even less margin dollars.

Note that you could says we should have been optimizing ebitda or EPS or even stock price.  The rule for optimization – at least the mathematical kind – is that you can only optimize what you can model (which doesn’t include stock price).  Another good rule is Occam’s razor – keep it simple.  For our purposes a model which maximized margin dollars was the right approach.

But what about quality?  We had two choices in making our model: we could make quality into a constraint or we could find a way to express quality in terms of margin dollars.  Constraints are as important to modeling as the goal you choose.  Although an optimization process can have only one goal, it can have an infinite number of constraints. 

Businesses are constrained by laws, for example.  You might be able to run your factory at a greater margin if you pollute illegally but you can’t do that. You can’t increase your margins by defrauding your customers or not paying your suppliers.  Obeying the law and honoring contracts are constraints.  (An important role of government, even from my libertarian point of view, is to create these constraints so that they do go into business models and business practices).

Business are also constrained by natural laws and technical laws.  A traditional phone “line” can only handle one call at a time. Each port on a traditional switch can only be connected to one other switch at any given time. These were constraints we had to respect where our network touched the legacy phone network.  Neither of these constraints existed within the Internet portion of our virtual network, though. Because some parts of our network did not have certain constraints, we could be more efficient (cheaper) than traditional networks that did have these constraints.

Customer expectation is also a constraint.  In the first iteration of our model we treated quality as a constraint.  We were saying we want to optimize margin dollars but will never make a routing decision which brings any one of the measures of quality below thresholds we have set for them on any route.  Implicitly, this says that we would reduce quality to these thresholds whenever we could gain margin dollars by doing so.  Treating quality measures as constraints was good enough to make the first version of the model effective.

As we gained experience, we learned to model the effect of quality on margin.  Higher call completion rates mean more calls to earn margin on.  Higher grades of service can be sold at higher margins.  Business is lost by letting quality drop below certain levels and there is a cost to get it back.  Note that our goal is still clearly margin – but, by modeling the effect of quality on margin rather than simply treating quality as a constraint, we were able to achieve higher margins than with the simpler, more constrained model.

Now that the value of quality could be expressed in terms of margin dollars, the routing engine could, for example, choose to drop quality below the old fixed thresholds for certain routes in order to gain even more margin on other routes where the two routes were competing for resources.  The model could also increase quality in order to get more margin where that was effective.

That version of the model was pretty ruthless about low-margin routes.  Sometimes a route had low margin because of a temporary aberration either in the marketplace or in supply.  Sometimes a route had low margin because we were in the early stages of developing it.  The model wasn’t sophisticated enough to think long-term (maybe it is by now), so we “subsidized” some routes by giving them artificial margin dollars.

Again, the discipline of optimization was making us think.  How much margin is business development costing?  How much current margin are we willing to forgo for this future business?  Are the subsidies perpetual or is the new business panning out?

This post has been all about using computers to optimize a process.  Even in the case of computerized optimization, much of the value of the process was in disciplined identification of the goal and the constraints and understanding how measures like quality affect the goal of margin.

Capitalism and Externalities is about government’s role of making sure that costs such as pollution ARE a part of business models.