The FINANCIAL -- Sooner or later, every technological revolution gives rise to an organizational revolution. To realize the potential of new technologies, companies devise new ways of working; those that fail to adapt end up losing in the marketplace. The steam engine was fully exploited only with the development of the early factory system, the process technologies of the late 19th and early 20th centuries with the development of scientific management.
Now, business is in the midst of a wholesale digital transformation. Companies across the economy are using digital technologies and advanced analytics to unlock new sources of economic value and achieve step-function improvements in customer focus, productivity, flexibility, and speed. Parallel to this digital transformation is an organizational revolution-in-the-making, transforming not just what companies do but how they do it.
Take, for example, the recent popularity and rapid spread of agile. The term is shorthand for a variety of approaches to organizing work that emphasize small, self-managed, multidisciplinary teams with end-to-end control of product development, service delivery, and other business tasks; rapid cycles of activity known as sprints; and a test-and-iterate approach to performing work.
Agile started in software development, but as software and digital applications become more and more central to a broad array of industries—finance, retail, even industrial sectors being transformed by the Internet of Things—the approach has spread far beyond the software industry. And companies are increasingly applying the agile model to nonsoftware activities such as marketing, customer service, and other traditional business functions.
Agile is only the most recent example of work innovations emphasizing autonomous, self-managed teams. Other approaches that companies have been experimenting with in recent years go by a bewildering variety of names: lean, holacracy, the polycratic organization, and the exponential organization, to list a few.
But in this organizational revolution-in-the-making, a critical piece is missing. Companies lack a compelling model for the role of management.
Some agile champions seem to assume that the approach makes management irrelevant or even obsolete. “Why Do Managers Hate Agile?” reads the title of a commentary in Forbes by an agile consultant. His answer: because agile inevitably (and rightly) undermines their status, power, and control. In a world of self-organizing, autonomous teams, a lot of what passes for traditional management is no longer necessary. Or as the title of a webinar on the subject puts it, “(In Agile) Where Do All the Managers Go?”
Such perspectives circle around the right question: how do managers create value in the new work environment? But I think they have the answer exactly backwards. They embrace a traditional concept of management only to declare it irrelevant to the new way of working.
The challenge of the organizational revolution represented by agile and other new approaches is not that they make management somehow irrelevant or obsolete. Quite the opposite: they make management more important than ever before. But they also transform what managers—from the very top of the organization to the frontline of the business—have to do and how they need to work. In some cases, they even redefine who needs to be a manager.
Until organizations develop a management model that is equal to the challenges of the organizational revolution taking place today, that revolution won’t be successful.
Companies may implement the “letter” of agile or other new models, but they risk missing the “spirit,” including the invisible supports that actually make these innovations work.
Developing the new managerial model will require a shift in how managers conceive of their role. Put simply, they need to stop thinking of themselves as the master designers of hardwired organizational structures, processes, rules, and procedures. Instead, they need to become the everyday orchestrators of a flexible and dynamic behavioral system, one that unleashes employees’ autonomy and initiative, and puts it in the service of more effective cooperation to achieve the organization’s goals.
I call this shift “bringing managers back to work.”
How managers got separated from work
“Wait a minute,” you may be thinking. “Aren’t managers already working—coming to work earlier and staying later, their days consumed with meetings, conference calls, emails, and reports, their weekends spent trying to catch up?” It’s true. Most managers are putting in more effort and more hours than ever before. But fewer and fewer are actually creating value.
To understand why, it pays to look back to the origins of professional management. Ever since Frederick Winslow Taylor introduced his theory of scientific management in the early 20th century, a key principle of modern management has been the radical separation of design and execution. Managers set strategy, plan, and define and allocate work tasks; they establish formal organizational structures, procedures, and incentive systems, and then monitor employees’ performance against them. Employees execute according to the strategy and the plan, their actions governed by the organization’s rules, procedures, and incentives.
This command-and-control model worked reasonably well in relatively stable environments. It also had the advantage of being easy to scale in the new era of mass markets served by mass production. It led to the specialization of functions and hierarchical management as we know it today.
THE CHALLENGE OF BUSINESS COMPLEXITY
Whatever its advantages, command-and-control management is singularly ineffective at addressing the distinguishing feature of today’s business environment: the exponential increase in business complexity. There are many dimensions of that growing complexity: the proliferation of (sometimes conflicting) performance requirements; the increase of customer segments, local markets, and competitors; the growth in the number of relevant stakeholders and business partners; the multiplication of categories of specialized knowledge and expertise; the faster pace of innovation and change; the higher levels of uncertainty and volatility.
Business complexity may sound like a problem. In fact, it is an enormous opportunity—if organizations can take advantage of it. The more complex the business, the more ways to create value by breaking compromises among heretofore conflicting objectives or goals and by combining diverse skills and capabilities in unprecedented ways.
However, this is precisely where the command-and-control model becomes an obstacle. At the foundation of the separation of design and execution is the idea of rules, in the sense of formal procedures. The assumption is that if managers design the formal procedures carefully and employees follow them obediently, then people’s work will predictably deliver the desired performance outcomes. In a relatively simple business environment, this assumption works well enough.
As complexity increases, however, the correspondence between the organization’s formal procedures and its business outcomes begins to fall apart. It is in the nature of business complexity to impose competing performance requirements on the organization. Products need to be affordable but also of high quality. Manufacturing plants have to be efficient but also safe. Business processes require speed but also reliability. The challenge is to reconcile these requirements, so that achieving any one of them doesn’t preclude achieving the others, and, ultimately, to discover solutions that exploit synergies across them all. But there is no “super-rule” that will tell people the best way to balance conflicting goals.
Even worse, in a futile attempt to control complexity, many organizations design more rules, processes, and guidelines for each new performance objective. The paradoxical result is an increase in organizational complicatedness—that is, the proliferation of contradictory rules and instructions—which causes people to lose their sense of direction and to escalate decisions to committees or to senior leaders, who have no direct knowledge of the issues at hand. The growing coordination burden means that more and more managers end up spending the lion’s share of their time managing the complicatedness, not the work itself. In the process, they become ever further removed from the genuinely value-adding activities that constitute the work of the organization.
COMPLEXITY AND THE DIGITAL REVOLUTION
The digital revolution transforming business these days is greatly accelerating the growth in business complexity, introducing new channels, new types of capabilities, new ways to create business value. It is also definitively exposing the dysfunctionality of the traditional separation of design and execution. Work innovations like agile are founded on the recognition that in a business environment characterized by competing performance requirements, more ways to create value, and continuous innovation, work tasks cannot really be “designed”—in the sense of programmed in advance according to a set of formal procedures. Nor, once designed, can they be “executed”—in the sense of performed according to an unchanging plan. Rather, work under conditions of complexity is all about discovery.
The key to effective performance in complex work environments is to unleash individual autonomy and initiative so as to maximize people’s freedom to exercise judgment in the completion of a task. But since no single individual or work group will have all the answers, it also requires creating an environment where people have an interest in deploying their autonomy in the service of cooperation with others for the greater good of the organization.
This combination of autonomy and cooperation doesn’t happen on its own. Rather, it requires a particular kind of management. Instead of focusing on formal procedures, managers must pay attention to the behavioral dynamics that shape organizational performance: why people do what they do; how they understand their individual goals, the resources available to them to achieve those goals, the constraints that stand in their way; and how individual behaviors combine (often in unanticipated ways) to produce the collective behavior underlying performance. What’s more, because managers themselves are actors in the behavioral system, they need to know how to intervene in that system in order to foster more effective cooperation.
To guide their intervention in the behavioral system, managers need to cultivate a new skill set, consisting of three high-level tasks.
The first I call framing through action. It is the general principle of management in the new work environment.
I call the second integrating around the task. This is how that general principle plays itself out at the frontline of the organization.
Finally, I call the third shaping the organizational context. This is the role of senior managers in the new work environment.
In the new work environment, the separation of design and execution is replaced by the combination of framing and acting. Rather than design tasks, managers “frame” objectives and goals. That framing sets the context that allows employees not so much to execute but to “act”—that is, exercise initiative guided by strategic goals, not rigid processes and rules; operate more autonomously, making decisions in the moment in response to changing circumstances and unanticipated obstacles or opportunities; and work together to make the tradeoffs that will create the most value over time.
But managers don’t just need to frame; they too must act. Put another way, in a more dynamic and fluid business environment, the all-important framing that managers do must take place through action—that is, through their ongoing intervention in the organization’s behavioral system. Framing doesn’t happen once; it happens continually, in close interaction with employees, and in response to the constantly changing circumstances thrown up by the work people do and the challenges they face in the ongoing effort to create value.
WHAT REALLY DRIVES BEHAVIOR
Why is framing through action so essential in the new work environment? Partly it is a function of what drives behavior in organizations. As the decisions people make and the actions they take become ever more critical to performance, it’s important to understand that, despite the assumptions of the command-and-control model, people’s behavior doesn’t follow automatically from the organization’s formal structures, processes, and rules. Rather, their behavior depends on how they use those features to achieve their personal objectives and goals. In some cases, they may see the organization’s formal structures, processes, and rules as resources to achieve ends that are very different from what the organization intends. In other situations, they may view them as constraints to be worked around in pursuit of their own objectives. If managers hope to influence these complex behavioral dynamics, they need to be present, close to where work actually happens.
Take, for example, the concept of the agile sprint. People usually focus on the time dimension of sprints—rapid cycles of work of relatively short duration. To be sure, the time dimension is important. But just because a company organizes work cycles in sprints doesn’t necessarily mean that people will run their fastest. What makes a sprint a good sprint is not just the time allocated to it but what people achieve and the quality of their effort. If they don’t give their best, even the shortest sprint will be little more than another iteration of routine work. In other words, it is the team members who determine whether a sprint is really a sprint. It’s not enough for managers simply to design the new work processes—including agile or any other system for autonomous, self-managed teams. Rather, they need to create a context in which team members are motivated to put their best effort into the work of team.
Framing through action is also necessary because, as mentioned earlier, the more complex the business, the more difficult it is to make rules that will apply in any and every situation. Put simply, complexity “rules out rules” as an effective means for managing organizational tasks. Therefore, for managers to frame in a way that’s realistic and useful to the organization, they need to be involved in the action. The less framing through rules is effective, the more managers need to frame through action.
THE PARADOX OF SPECIALIZATION
A third reason framing through action is essential is that increasingly complex tasks often require the integration of new kinds of specialized expertise and roles. My colleagues and I call this the paradox of specialization: the more complex work becomes, the greater the need for focused and deep content knowledge in a proliferating number of areas. The greater the number of highly specialized functions and units, however, the more the need for cooperation across those functions and units—and yet, the harder it is to get people to think, work, and interact beyond their own specialized functional mindsets.5 It is impossible for any manager to grasp in advance the full range of knowledge and capabilities that needs to be brought to bear to create value. Therefore, they must be present in the moment, interacting with the organization’s cross-functional teams as they apply their diverse capabilities to perform a complex task.
Finally, framing through action is a requirement when creating value depends on more intricate forms of cooperation, as it does in any environment of high complexity. But the more cooperation is necessary, the less possible it is to isolate the contribution of any particular individual and, therefore, the harder it becomes to measure people’s performance by means of individual KPIs. The fact is, cooperation often comes at the expense of individual performance—in which case, individual KPIs and the incentives associated with them end up functioning as disincentives to the more effective cooperation the organization needs.
Typically, organizations try to address this dilemma by evaluating people, in part, on a proxy for cooperation: the collective performance of their work group, team, or unit. But the problem of free-riding—when individuals don’t pull their weight—will always be present, and no metric will capture it. Thus, the only way to evaluate people’s performance is if managers exercise judgment based on their observations of the actual behavioral dynamics of the group—who cooperates effectively and who does not. To exercise that judgment effectively, they need to be close to the work.
All of the above explains why framing through action is the general principle of management in the new work environment. What that principle means concretely, however, depends on where a manager is located in the organization.
Integrating around the task
At the frontline of any organization, value gets created by teams. People with diverse perspectives and capabilities pool their expertise and their efforts to come up with the best solutions for meeting customers’ needs and achieving organizational goals. The role of management at the level of the frontline team is to integrate the work of various team members around the task at hand—that is, to make sure cooperation happens, people work together productively on the task, and they make the tradeoffs necessary to create value across multiple performance objectives.
Take, for example, the product owner of an agile team. Unlike traditional project managers, whose chief responsibility is to deliver a product on time and on budget, product owners orchestrate a complex process of discovery that eventually leads to a product or service that creates value for the customer and the company. Striking that balance requires navigating many tradeoffs. The head of product development at a leading internet company described this to me in terms of managing four types of risk:
The risk that the end user won’t value the product
The risk that the user won’t know how to use the product
The risk that the organization won’t be able to build the product
The risk that selling the product won’t help the organization meet its business goals
The various practices of agile—rapid prototyping, user testing, and the like—are designed to help teams manage these risks and balance the tradeoffs across competing performance requirements. In this respect, the agile product owner is on the frontline of management in the new work environment.
And yet, proponents of agile almost never talk about product owners as managers. After all, agile product owners don’t have a formal reporting relationship with their team members. They aren’t responsible for members’ performance evaluations (although they sometimes provide input), nor do they determine a team member’s career progression.
But to conclude that product owners aren’t managers is to view the new way of working from the perspective of the old command-and-control model. It assumes that being a manager is a function of having a defined position in the hierarchy or being in charge of the structures, processes, and systems that, in theory, determine people’s performance.
The problem with this perspective is that it completely misses the essential task of frontline management in the new work environment, which is less about “managing people” (in the sense of reporting relationships, career progression, performance evaluation, and the like) than it is about “managing behavior” (in the sense of creating an environment in which people find it desirable to devote their full effort to the task at hand, to exercise initiative, to cooperate constructively with their colleagues). And managing that behavioral context is precisely what an effective product owner must do.
It’s an extremely challenging job. For one thing, the product owner can’t simply order his team members to do whatever he thinks they ought to do. The product development head puts it this way:
“The job of the product owner is not to make the right decisions; it’s to make sure the right decisions get made by the team.”
The product owner is what I call an integrator. He or she is responsible for managing effective cooperation—that is, for integrating all the diverse perspectives and capabilities of team members around the task.
What allows the agile product owner to be an effective integrator? The answer is power, but not the kind of power that comes from being someone’s supervisor or controlling his or her career progression. Rather, it is the power that comes from the capacity to make a difference in the goals or “stakes” that matter to individual team members. When a product owner has control (or, at a minimum, influence) over those key uncertainties, he or she has power in the behavioral system of the group and, therefore, is able to function as an effective integrator.
The product development head had an evocative way of describing this dynamic. Product owners, he said, “need to be able to tell a compelling story about the work.” For example, they must be able to convince team members that they are solving an important problem for users or that they are working on interesting technology or that the product, if successful, will have a significant impact on the company’s business. If product owners are unable to make the case persuasively, they are unlikely to attract the best people to the team or motivate them to do their best work. Telling a good story, creating a strong vision of the future, and tying that vision to business returns are how product owners frame through action and integrate the work of the team around the task.
One important source of power for agile product owners, reinforcing their ability to tell a compelling story about the work, is their role as a proxy for the customer (either the actual end customer or an internal customer such as the business owner). Product owners lead the “story mapping” exercise that defines the desired functionality of the product and the business value it is meant to deliver to the user. This user story frames the all-important question: What are we trying to accomplish? Product owners also are responsible for maintaining the product backlog, which sets the priorities that guide the team’s work in each sprint (What do we need to do next?). Finally, product owners also decide whether a particular feature meets the product’s acceptance criteria, thus determining when the feature is declared “done,” so the team can move on to the next items on the product backlog list. The control of these key uncertainties gives product owners the power to orchestrate cooperation among team members.
Agile practices like story mapping, scrums, and retrospectives; artifacts like user stories and product backlogs; even concepts like minimum viable products and sprints are all mechanisms for framing through action. They function as continuous feedback loops that make it easier to integrate the various contributions of team members around the task and, thus, are resources available to product owners in their role as integrator.
But such mechanisms don’t function automatically. For instance, even the most exciting or important project is going to have to navigate a variety of constraints in which not all necessary tasks will be equally exciting. In such situations, managers need to have sufficient power to induce people to accept adjustments to their individual goals (for example, their desire to work on the most interesting tasks or challenging technology) for the greater good of the team.
The problem with not conceiving the product owner role as explicitly managerial is that product owners can become disempowered. When that happens, they cannot be effective integrators and the work of the agile teams can become dysfunctional.
Shaping the organizational context
Recently, I asked a senior manager at a company that was implementing agile teams how he saw his role in the process. “I set up the teams,” he told me, “and then just get out of the way.”
The statement contains an important element of truth. A key principle of agile is that those closest to the work are in the best position to make decisions about how to develop the product or service in question and how to prioritize tasks and objectives over time. Nothing is more certain to disrupt an agile implementation than senior managers who try to retain control over the process or otherwise direct how teams do their jobs.
And yet, the manager’s statement underestimates the complexity of the senior management role in the new work environment. Ceding control to self-managed teams doesn’t mean abandoning them. Senior executives, just like frontline managers, have a critical and ongoing “framing through action” role to play. When they get it wrong, the result can be the worst of both worlds—simultaneously too much control and too little enabling engagement and support—which can undermine the effectiveness of the new way of working.
The role of the senior manager in an agile organization is not to determine the content of people’s work. Rather, it is to provide the context for that work. That means helping employees understand how their immediate objectives relate to the organization’s strategic and business goals. Senior managers need to articulate a robust strategic context that teams can use as a “North Star” that aligns their autonomy to those goals, guiding them as they exercise their initiative. In the absence of such a strategic context, just setting up agile teams is unlikely to create business value.
Nurturing the behavioral system
Bringing managers back to work has profound organizational implications. It will likely transform how companies select, develop, and promote managers.
Take, for example, the matter of career paths. In the old command-and-control environment, the typical managerial career path consisted of a progressive increase in an individual’s span of control in a single domain or activity—R&D, say, or operations, or marketing. It was only at the very top of the organizational hierarchy that executives had to cope with the complexity that comes from managing across distinct silos.
In the new work environment, by contrast, career progression is likely to be characterized more by an expansion in responsibility for managing the organization as a behavioral system. An individual’s advancement will depend on his or her capacity to manage behavior in organizational contexts of ever increasing complexity, defined by the number of competing requirements for creating value and by the diversity of the specialized profiles of the employees involved.
It will require considerable attention and effort for any organization, given its unique organizational context, to figure out the most effective practices for bringing managers back to work: to plan the experiences that will allow managers to develop the skills they will need, to establish pathways for career progression, and so on. The temptation, as always, will be to want to design everything in advance and then leave the new structures and processes to function on their own.
But that would just repeat the mistakes of the old command-and-control model. As one CEO put it, “The perfect agile organization can’t be designed, but it can be developed.” Doing so requires the continuous fine-tuning of the organization as a behavioral system.
Elsewhere, my colleagues and I have written about six simple rules that managers can start following today to begin this fine-tuning.6 These are light-touch interventions to improve people’s capacity for effective cooperation without falling into the trap of organizational complicatedness. They are directly relevant to how managers should think about implementing agile or any other work innovation. They are also critical to how managers themselves need to work in the new environment.
UNDERSTAND WHAT YOUR PEOPLE REALLY DO
To nurture an organization’s behavioral system, managers must first understand the behavioral dynamics that drive company performance—why people do what they do. For any performance-related problem or goal, managers should ask:
Who are the people or organizational units critical to the key moments of truth in delivering on our strategy?
What do these people do and how do their behaviors contribute, either individually or in combination with the behaviors of others, to produce the organization’s performance?
Why do these people do what they do? What are the “good” (in the sense of individually rational) reasons for their behavior, even if that behavior is counterproductive to the goals of the organization?
What changes to the organizational context will change behaviors in a way that improves performance?
What kind of interventions or joint commitments are necessary—whether from colleagues, other functions, or senior management—to make these changes?
How can I, as a manager, help bring about these interventions and joint commitments?
Increasingly, managers need to be evaluated on the quality of their answers to these questions. Why? Because the more executives at all levels routinely ask them, the more clearly they will understand the behavioral dynamics of the organization. And the more clearly they understand those dynamics, the more they will be in a position to intervene in the behavioral system to encourage more effective cooperation.
One such intervention is to identify and reinforce the natural integrators in the organization. Integrators are individuals or work units that rely on cooperation to get work done. Their role is particularly critical in complex work environments. Reinforcing them means, first, recognizing the managerial nature of the role and, then, giving those who occupy it (for example, the product owners of agile teams) sufficient power to influence the behavior of others so they are willing to cooperate with one another.
INCREASE THE TOTAL QUANTITY OF POWER
Power is often a dirty word in the new work environment because in the traditional organization, power is associated with hierarchy. I have a different way of thinking about it. Power is really only the capacity of one person to make a difference on issues that matter to someone else, and it is an inevitable feature of organizational life whether managers recognize it or not. What’s more, the way power is distributed in an organization is an important enabler of (or constraint on) effective performance. Sometimes, increasing the total quantity of power in the behavioral system can lead to more effective cooperation. It’s akin to increasing the number of cards in a deck. The greater the number of cards, the greater the variety of moves that each player can make. In the same way, expanding people’s behavioral options mobilizes them to find the best ways to satisfy the multiple requirements of a complex task.
Reciprocity is the mutual conviction among people in a behavioral system that they have a shared interest in cooperation and that each actor’s success depends on the success of others. Another key intervention is to create mechanisms for making the necessary interdependencies among employees more visible. For example, setting rich performance objectives that go beyond purely individual objectives to include the impact of an individual’s behavior on the performance of others is one way to increase the sense of reciprocity among actors in the behavioral system.
EXTEND THE SHADOW OF THE FUTURE
Another way managers can nurture the behavioral system is to create feedback loops that expose people directly to the consequences in the future of the decisions they make and the actions they take today. For example, managers can lengthen the amount of time a particular group of people have to work together so that they are “all in the same boat.” This is the basic principle behind the cross-functional agile team as a work unit. Or organizations can speed up the frequency of deadlines or targets—the basic idea behind a sprint. Another way to extend the shadow of the future is to force people to “walk in another’s shoes”—for example, by designing career paths so that people know that they will experience in their next assignments the consequences of the decisions they make today.
The final rule is to factor cooperation more heavily into the rewards, both financial and nonfinancial, that the organization distributes to employees.
Earlier, I described how difficult it is to measure cooperation. At first glance, managers may view this difficulty as a limitation or constraint. In fact, it is a key resource for re-establishing the fundamental integrity of their role, because it means that in the new work environment, there is no alternative to the continual exercise of managerial judgment. No system, no metric, no rubric will be able to assess the effectiveness of cooperation among a group of people better than a human being exercising his or her judgment, and it is the responsibility of the manager to do so, rewarding those who cooperate well and ensuring that those who do not bear the consequences.
Of course, precisely because managerial judgment is so critical in the new, more fluid, more autonomous and self-managed work environment, it is also critical that that judgment be informed. It’s no longer good enough for managers to make judgments from afar, on the basis of second-hand information and reports (let alone their own biases). To evaluate the quality of cooperation, they must experience up close how the organization and its people are working.
The growing importance of managerial judgment in work environments of increasing complexity is yet another reason for bringing managers back to work.