Understanding Government Failure

Public discontent over many of the federal government’s domestic policy efforts has reached alarming proportions. In my recent book, Why Government Fails So Often: And How It Can Do Better, I entertain a number of possible causal explanations, but I focus on the most straightforward one: across many different policy domains, the public perceives poor governmental performance – and generally speaking, the public is correct in this view.

This explanation is buttressed by social scientists who have reviewed and assessed federal domestic programs in great detail, producing many data points that in a large number of cases indicate government failure, defined as cost-ineffectiveness. This failure is evident even though the government allocates a vanishingly small percentage of its budget to assessing how effective the vast majority of its spending is in achieving its putative purposes. These failures, I argue, are structural and recurrent in nature, and each chapter in my book analyzes in depth a different set of structural causes, which I outline here.

But let me first very briefly explain what impelled me to write this book and the challenges that I encountered when I did so. I have long been intrigued by the central questions surrounding federal domestic regulation, some of which include:

• What conditions justify regulation in the first place?
• How do the specific features of particular markets affect the regulation that seeks to re-shape them?
• Which of the many regulatory forms, techniques, and procedures should be employed under what circumstances?
• Which political economy and other factors determine regulation’s consequences?
• How should regulatory effectiveness be assessed?

But regulation is only one policy instrument among many, and Why Government Fails So Often is concerned with policies’ failure and success regardless of the forms that those policies take. The research literature that I found contains many excellent studies of one or another policy, but very few look across the numerous discrete policy areas to identify the recurrent, systemic, structural determinants and patterns of federal policy performance. Even fewer bring the discipline of peer-reviewed and other authoritative social science data to bear on the vital question of governmental performance.

My book attempts to fill this gap. Why Government Fails So Often begins with a chapter introducing some important themes: the striking success of American society along many (but not all) dimensions; the growing public dissatisfaction with the federal government and possible reasons for this trend; the challenge of defining policy failure and success; the high social stakes in improving government performance; an introductory thumbnail sketch of the social science evidence demonstrating widespread failure; the difference between optimists and realists about government performance; and a preface to markets that cast such long shadows across all policymaking that seeks to control them.

Chapter 2 defines success, failure, and in-between assessments and defends a criterion of cost-effectiveness, rightly understood and suitably qualified. Chapter 3 describes the functions, processes, missions, instruments, and institutions of policymaking, and Chapter 4 analyzes the main elements of the political culture that shape and constrain it: constitutionalism; decentralization; protection of individual rights; interest group pluralism; acceptance of social and economic inequality; religious and political moralism; social diversity; populist suspicion of technical expertise and official discretion; public opinion; and civil society.

The next six chapters constitute the heart of my causal analysis of government performance and of the social science evidence on which this analysis is based. These chapters focus on incentives and collective irrationality (Chapter 5); information, inflexibility, incredibility, and mismanagement (Chapter 6); markets (Chapter 7); implementation (Chapter 8); the inherent limits of law, the almost universal policy tool (Chapter 9); and the federal bureaucracy (Chapter 10). Chapter 11 considers a dozen policy success stories, attempting to distill why they worked. Chapter 12 presents my proposed remedies, which are limited to those that can cut across all policy domains and which are sub-constitutional and relatively incremental in nature. Chapter 13 presents a summary conclusion and exhortation that we come to accept the brutal “facts of public life” and think more realistically about what government can accomplish.

Participants in the The Regulatory Review series discussing my book this week, as well as readers of the book, will of course address whichever issues they find most interesting. But in the interest of provoking the discussion, here are some issues that I posed previously at a Notre Dame conference organized around the book. I have classified them as descriptive, normative, prescriptive, and political questions.

Descriptive: How accurate is the book descriptively? How representative are its examples, and how accurate are the data on which it relies? How applicable are its analysis and findings to state and local governments?

Normative: In the analysis of performance, does the analysis neglect relevant values other than cost-effectiveness, and if so which ones? Is my approach too technocratic?

Prescriptive: Are my proposed remedies politically feasible? Are they equal to the nature and magnitude of the policy problems to which they are addressed? Are they too incremental? Is the bureaucracy on which reforms depend equal to the challenge of implementing them?

Political: Are the constitutional and cultural impediments to more effective government discussed in Chapter 4 too great? Is the public so disgusted and apathetic to be aroused to support targeted reforms (as distinct from its unfocused indignation)? Is the book (as one reviewer put it) “a gateway drug to libertarianism”? Are my moderate proposals doomed to failure in an increasingly polarized polity? Will those on the left likely to take my critique seriously?

These are vital questions both for public leaders and members of the public, and we need more analysis of them. I am therefore most grateful to Professor Cary Coglianese for organizing this The Regulatory Review series and to the several other participants in the recent Notre Dame conference on the book whom Coglianese has urged to participate in this online discussion. Only by understanding better the reasons for government failure can policymakers have any chance of making it succeed, at least in those ever-decreasing times when the conditions are right for its success.

This essay is part of The Regulatory Review’s seven-part series, Is Government Prone to Fail?

Peter H. Schuck is the Simeon E. Baldwin Professor Emeritus of Law at Yale Law School and the author of Why Government Fails So Often: And How It Can Do Better (Princeton University Press), among many other books. His current project is a book tentatively entitled Five Hard Issues and How to Think About Them (Princeton University Press), analyzing the issues of immigration, affirmative action, poverty, campaign finance, and religious accommodation.