Big Data - Big Opportunity or Big Headache?

Big Data is the breaking news story in the IT world.

Here’s the picture. Sometime soon company managers will have an enormous amount of information at their fingertips. They will be able to see everything and optimize everything.

What could be better?

Within the past month, two very senior, astute individuals contacted me about this – one a senior IT industry analyst, and the other a senior editor of a major business publication. They both had the same question: If Big Data actually becomes available, what will be the consequences?

My answer – Big Data offers big opportunities, but it carries the very strong likelihood of creating really big headaches in three areas: (1) paving the cowpaths, (2) managing at the right level, and (3) driving without a roadmap.

Let me explain.

Paving the cowpaths

Remember RFID?

A few years ago, RFID was all the rage. The idea was that it was becoming technically and economically possible to affix an RFID tag to every item moving through a supply chain. (An RFID tag is a small passive label that essentially emits a precise identifier when it is hit by an electromagnetic field.) Armed with this capability, managers could know the identification and location of every item in their company, and even those flowing into and out of their companies.

I remember discussing this with a former student, who is a senior operating executive of a Fortune 20 company. His reaction was similar to mine: What ever would you do with all that information?

In fact, several years ago I co-authored a column, Are You Aiming Too Low with RFID? in Harvard Business School’s Working Knowledge. In this piece, I joined with Sanjay Sarma, co-founder of MIT’s Auto-ID Lab, and John Wass, CEO of WaveMark, to argue that the biggest danger with the flood of RFID information was that almost-irresistible temptation to focus on “paving the cowpaths.”

Here’s how we put it. “One of the most exquisite challenges of living in Boston is navigating the labyrinthine maze of streets in the downtown area. This part of town is the oldest part, and the streets follow the original paths formed by settlers driving their cows to pasture. Traffic flows poorly because the city fathers simply paved the cowpaths, making the ineffective more efficient. It’s much easier to navigate Back Bay, a part of Boston with grid-like streets, built on landfill centuries later.”

In the article, we outlined a number of highly focused, high-value analytical uses for the “Big Data” that RFID could produce, and counseled avoiding the large-scale applications that simply automated routine activities.

In the absence of this disciplined, strategic approach, managers are in grave danger of utilizing Big Data to pave the cowpaths, further entrenching existing practices and rendering the possibility of developing sweeping paradigm-changing initiatives more and more difficult.

This will almost inevitably occur because in the capital budgeting process, the tactical payoffs from paving the cowpaths will be clear and easy to measure, while the payoffs from far-reaching strategic changes in the business will be hazy and unmeasurable.

Managing at the right level

This past weekend, I was reminded again of the Big Data question when I re-read Wired for War, a terrific book by P.W. Singer which traces the robotics revolution and the use of robots in  twenty-first century conflict.

In a particularly telling chapter, Singer describes how the real-time video feeds from drone aircraft – Big Data – led to the systematic leadership problems that I call “managing at the wrong level.” (See my HBS Working Knowledge column, Managing at the Right Level.)

Over many years, improved communications technology has enabled commanders to command increasingly at a distance from the actual battles. This has led to a very effective management structure in which top commanders focus on strategy and personnel, mid-level commanders on operational initiatives, and local officers on tactical issues. This parallels the leadership structure of most effective companies.

However, the widespread availability of drone aircraft information feeds has led to serious and systematic command and leadership problems. The ability of top commanders to see battlefield video feeds in real time has rapidly increased the centralization of command and led to an explosion of micromanagement.

Crack for Generals

Singer relates, “Too frequently, generals at a distance are now using information technology to interpose themselves into matters that used to be handled by those on the scene and at ranks far below them. One battalion commander in Iraq told how he had twelve stars worth of generals (a four-star general, two three-star lieutenant generals, and a two-star major general) tell him where to position his units during a battle.”

Singer continues, “An army special operations forces captain even had a brigadier general (four levels of command up) radio him while his team was in the midst of hunting down an Iraqi insurgent who had escaped during a raid. The general, watching a live Predator video back at the command center…ordered the captain where to deploy not merely his units, but his individual soldiers. ‘It’s like crack for generals,’ says Chuck Kamps, a professor at the Air Command and Staff College. ‘It gives them unprecedented ability to meddle in mission commanders’ jobs.’”

This direct meddling by military leaders has led to the rise of what Singer calls the “tactical general,” as the line between timely supervision and micromanagement has became blurred. Officers in the field lament what they call the “Mother may I?” syndrome which has come with these new technologies.

Moreover, power struggles are common when the feeds are available to multiple command groups.  Singer notes, “At its worst, this pattern can lead to the battlefield versions of too many cooks spoiling the meal. A marine officer recalls, for example, that during an operation in Afghanistan, he was sent wildly diverging orders by three different senior commanders. One told him to seize a town fifty miles away. Another told him to seize the roadway just outside of town. And the third told him, ‘Don’t do anything beyond patrol five miles around the base.’”

But, the biggest problem with top-level micromanagement in the military – just like in business – is the huge hidden opportunity cost of failing to manage at the right level: a leader ignoring the critical issues of high-level strategy and organizational capability because he or she is so caught up in real-time micromanagement. This causes two very big, related problems.

First, the top managers fail to plan for the future. For example, in business, vice presidents should primarily be focused on defining and developing the company as it should be in three to five years, since that is the time it takes to develop a new set of capabilities. Their other critical responsibility: coaching and developing the next generation of leaders.

In the absence of this hierarchical discipline, the company is in grave danger of getting mired in the present, and falling further and further behind.

Second, when top managers – or generals – take over tactical decisions, the lower-level leaders cannot develop their skills. Instead, they must be empowered to act with initiative, even if it means making a few mistakes along the way. No – especially since it means making a few mistakes along the way, since false starts and errors are a natural and necessary part of doing anything significant and new.

The answer? Singer calls it “enlightened control,” a concept he credits to the great Prussian generals of the nineteenth century, whose ideal was that the best general gave his officers the objective and left it to them to figure out how best to achieve it. He cites the commanding general who so trusted his officers that the only order he supposedly issued on the eve of the Prussian invasion of the Danish province of Schleswig was, “On February 1^st, I want to sleep in Schleswig.”

The action question for managers: Will Big Data be “crack for your vice presidents,” or will they have the insight and discipline to double down on “enlightened control.”

Driving without a roadmap

One might ask: But won’t Big Data let a company’s managers optimize everything? After all, every manager will have, theoretically, the information needed to get everything right. And if a company’s managers optimize everything, won’t the company be great?

This question embodies one of the biggest false assumptions in thinking about Big Data.

The glib answer is that if it weren’t for the humans, this premise might actually happen. Let me explain.

When I think of Big Data, an analogy comes to mind. Imagine that you were living decades ago, at the time of the invention of the automobile. Assume further that all of a sudden, all the dirt roads and tracks were paved. What would you do? Where would you go?

The obvious answer is that either you would stay local or you would be paralyzed in the face of the enormous number of opportunities. In fact, you would need a roadmap, so you could see how to get to different places. Beyond that, you would need to understand the nature of the destinations so you could decide where to go, since you couldn’t get everywhere in one lifetime.

Further, if you had a number of different drivers, it is likely that each would head in a different direction, since each would go after the goal that he or she thought best. If these drivers had to coordinate, like the managers in a company, what would happen? The result would be chaos.

The problem here is two-fold. First, a company can’t do everything, because it takes significant time and resources to manage the change required to harvest any IT-based initiative. And second, the initiatives have to be coordinated and focused on the right long-term strategic goals to be effective. If the availability of Big Data encourages a massive flock of independent tactical initiatives, it will do more harm than good.

The problem with low-hanging fruit

This raises an important related problem. Managers have an almost overwhelming tendency to focus first on opportunities that are near to hand, and have quick, visible payoff. Sometimes these are called “low-hanging fruit.”

The problem is that these relatively small, parochial projects will absorb the organization’s resources and capability to change, even while they give the illusion of progress. The huge opportunity cost is losing the opportunity and ability to focus on the really important initiatives with the really big long-term payoffs.

Think of it this way. The analogy breaks down because the big money is not in harvesting fruit more efficiently, but rather in changing the location of the orchard and the type of trees you plant.

This dissipation of effort, with its focus on a large number of small, incremental projects – rather than on the smallest number of game-changing, high-payoff initiatives – is the ultimate danger of Big Data.

Keys to Success

Big Data offers great opportunities – and huge dangers. How can you navigate toward the benefits while avoiding the hazards?

The key is management insight and discipline.

The true promise of Big Data is to make your company better, not just to make parts of it more efficient. To accomplish this, you need one part technology to nine parts vision and great management.

Steve Jobs and Customer Value Creation

During the holidays, I read Walter Isaacson’s biography of Steve Jobs – which I strongly recommend.

Steve Jobs was a very complex individual, to say the least. He was abrasive and insightful. He drove people who worked with him to achieve things they didn’t think possible.

Over his career, Steve Jobs weathered near-failures, and built one of the most valuable companies in the world. This book offers a unique and valuable perspective on business strategy and management. The chapters on how Jobs built Pixar and refocused Apple after he returned from being fired are both fascinating and instructive.

At the end of the book, Isaacson quotes Jobs as he reflects on his business career. I offer a few excerpts below, and after that, some things to think about.

 

My passion has been to build an enduring company where people were motivated to make great products. Everything else was secondary. Sure, it was great to make a profit, because that was what allowed you to make great products. But the products, not the profits, were the motivation. [Ex-CEO John] Sculley flipped these priorities to where the goal was to make money. It’s a subtle difference, but it ends up meaning everything: the people you hire, who gets promoted, what you discuss in meetings.

Some people say, “Give the customers what they want.” But that’s not my approach. Our job is to figure out what they’re going to want before they do. I think Henry Ford once said, “If I’d asked customers what they wanted, they would have told me, ‘A faster horse!’” People don’t know what they want until you show it to them. That’s why I never rely on market research. Our task is to read things that are not yet on the page.

Edwin Land of Polaroid talked about the intersection of the humanities and science. I like that intersection. There’s something magical about that place. There are a lot of people innovating, and that’s not the main distinction of my career. The reason Apple resonates with people is that there’s a deep current of humanity in our innovation. I think great artists and great engineers are similar, in that they both have a desire to express themselves. In fact some of the best people working on the original Mac were poets and musicians on the side. In the seventies computers became a way for people to express their creativity. Great artists like Leonardo da Vinci and Michelangelo were also great at science. Michelangelo knew a lot about how to quarry stone, not just how to be a sculptor….

At different times in the past, there were companies that exemplified Silicon Valley. It was Hewlett-Packard for a long time. Then, in the semiconductor era, it was Fairchild and Intel. I think that it was Apple for a while, and then that faded. And then today, I think it’s Apple and Google – and a little more so Apple. I think Apple has stood the test of time. It’s been around for a while, but it’s still at the cutting edge of what’s going on.

It’s easy to throw stones at Microsoft. They’ve clearly fallen from their dominance…. And yet I appreciate what they did and how hard it was. They were very good at the business side of things. They were never as ambitious product-wise as they should have been. Bill likes to portray himself as a man of the product, but he’s really not. He’s a businessperson. Winning business was more important than making great products. He ended up the wealthiest guy around, and if that was his goal, then he achieved it. But it’s never been my goal, and I wonder, in the end, if it was his goal. I admire him for the company he built – it’s impressive – and I enjoyed working with him. He’s bright and actually has a good sense of humor. But Microsoft never had the humanities and the liberal arts in its DNA….

I have my own theory about why decline happens at companies like IBM and Microsoft. The company does a great job, innovates and becomes a monopoly or close to it in some field, and then the quality of the product becomes less important. The company starts valuing the great salesmen, because they’re the ones who can move the needle on revenues, not the product engineers and designers. So the salespeople end up running the company. John Akers at IBM was a smart, eloquent, fantastic salesperson, but he didn’t know anything about product. The same thing happened at Xerox. When the sales guys run the company, the product guys don’t matter as much, and a lot of them just turn off. It happened at Apple when Sculley came in, which was my fault, and it happened when [Steve] Ballmer took over at Microsoft. Apple was lucky and it rebounded, but I don’t think anything will change at Microsoft as long as Ballmer is running it.

I hate it when people call themselves “entrepreneurs” when what they’re really trying to do is launch a startup and then sell or go public, so they can cash in and move on. They’re unwilling to do the work it takes to build a real company, which is the hardest work in business. That’s how you really make a contribution and add to the legacy of those who went before. You build a company that will stand for something a generation or two from now. That’s what Walt Disney did, and Hewlett and Packard, and the people who built Intel. They created a company to last, not just to make money. That’s what I want Apple to be….

You always have to keep pushing to innovate. Dylan could have sung protest songs forever and probably made a lot of money, but he didn’t. He had to move on.... The Beatles were the same way. They kept evolving, moving, refining their art. That’s what I’ve always tried to do – keep moving. Otherwise, as Dylan says, if you’re not busy being born, you’re busy dying.

What drove me? I think most creative people want to express appreciation for being able to take advantage of the work that’s been done by others before us. I didn’t invent the language or mathematics I use. I make little of my own food, none of my own clothes. Everything I do depends on other members of our species and the shoulders that we stand on. And a lot of us want to contribute something back to our species and add something to the flow. It’s about trying to express something in the only way that most of us know how – because we can’t write Bob Dylan songs or Tom Stoppard plays. We try to use the talents we do have to express our deep feelings, to show our appreciation of all the contributions that came before us, and to add something to that flow. That’s what has driven me.

(From Isaacson, Walter, Steve Jobs, Simon & Schuster, 2011, pp. 567 – 570.) 

 

Things to think about

Your product is the value you create for your customers – the totality of your customers’ experience. This is how Steve Jobs saw the world. He saw his essential job as always pushing the envelope on customer value creation.

Here’s a challenge – something to think about as we enter the new year:

Are you focused on pushing the envelope on creating customer value in new ways?

Do you always “try to read things that are not yet on the page” when you think about customer value creation?

Is there a “deep current of humanity in your innovation”?

Does your company “exemplify your industry”? Are you at the “cutting edge of what’s going on”?

Are you building a company that will “stand for something” a generation from now?

Years from now, will you be able to look back and say that you “added something to the flow”?

If your answers to these questions are positive, you will always make money. Real customer value always wins, and gives you sustained high profitability and growth in the process.

Best wishes for a successful and happy 2012.