A decade ago, California, along with other states and federal policymakers, began to rethink its approach to the electricity market. Frustrated by the high costs of long-term contracts and investments in electric power, compared to low costs offered by new suppliers, California launched and federal regulators approved a major restructuring of its investor-owned, regulated electric utility industry. Although the reform appeared to work well in the beginning, with low prices and abundant supply in the new market, in the summer of 2000 something suddenly went wrong. A year later (almost overnight in the traditionally sleepy utility business), the state's electricity market had melted down. Prices reached previously unimagined levels, large utilities became insolvent, lights went out, and the state government took over major parts of the business. New long-term contracts signed by Governor Gray Davis featured prices judged to be above market by about the amount that had precipitated the restructuring a decade before. The resulting recriminations descended to the lowest levels of public discourse.
The title for the case study that will ultimately be written on this public-policy fiasco is already in popular use. It was a "perfect storm," as everything that could go wrong did, all at the same time. The California story is not yet finished, but it is already a sobering one. The crisis has consumed the state's multibillion-dollar budget surplus and forced a major redirection of public priorities. After years of effort and major investments in new institutions to run the market, the local electricity market is a shambles and will have to be rebuilt again.
That the problems were man-made and therefore avoidable requires us to understand what went wrong and how to do better. Some analysts claim that California's experience reveals the folly of market reforms for a product as crucial in our technological age as electricity. But elsewhere in the United States, and in countries as diverse as Norway and New Zealand, such reforms have succeeded, bringing about needed investment in new supply to accommodate growing demand. Given the problems that had arisen in the "good old days" of regulation--such as expensive nuclear power plants and top-heavy bureaucracies--much is at stake in learning from California.
The power crisis there laid bare the dangers of designing a market driven by a blind faith in a pseudo-market ideology, while ignoring the fundamentals of how power systems operate. In fact, it is now clear that a flawed wholesale market and a caricature of a retail electricity market arose in California as the products of a volatile combination of bad economic theory and worse political-economic practice. As the perfect storm developed, bad market-design decisions were compounded by bad luck. There had been little addition to electricity generating capacity for more than a decade. Then, low water levels in the reservoirs behind power dams in California and the Northwest combined with suddenly higher natural gas prices and tighter environmental controls, just as an unexpected surge in demand from economic growth hit the inefficient market. A power surplus was transformed into a sudden shortage, producing unprecedented price increases. Starting in June 2000, wholesale prices surged and stayed above $150 per megawatt-hour, five times the previous level, while retail prices for the same energy were limited to approximately $65. The California state regulator and the governor refused to raise the retail prices that utilities were allowed to charge. When you buy for $150 (or soon $200, $300, and up) and sell for $65, you can't make it up on volume. A credit crisis ensued as utilities stopped paying their bills and suppliers became wary of shipping power and of extending more IOUs. The lights literally began to go out, and "deregulation" was pronounced dead.
Some of the lessons are obvious. California unwisely deregulated wholesale--but not retail--prices, betting that prices would remain low and shielding consumers from any signal to conserve. Because the state also prohibited the utilities from signing long-term contracts to buy electricity, the deregulation scheme created one of the largest "uncovered shorts" in any market: the utilities were in effect selling long and buying short. When changed market conditions hit, and the government balked at raising retail prices to pay the bills and provide incentives to reduce demand, a manageable energy problem mushroomed into a political and financial crisis. Fortunately, no other electricity market reform has adopted California's peculiar error of combining full reliance on a short-term market with unregulated wholesale prices and fixed retail prices.
Less well understood are the roles of business and government in designing electricity markets. California's market was in trouble well before it spun out of control in the summer of 2000; even without that season's bad luck and exploding prices, the state's electricity "reforms" needed reforming almost from their inception.
In 1994, the California state regulator produced an initial recommendation for fundamental reform of a dysfunctional regulated market that had grown bloated and bureaucratic. This plan was a reasonable and well-reasoned approach for gradual introduction of greater reliance on market forces to support competition in power generation and supply, but not in delivery (where it makes sense to have a single, regulated vendor operating wires to customers' homes or businesses, rather than multiple competing lines).
As its centerpiece, the plan recognized the importance of having a central system operator that could be responsible for managing the unique requirements of electricity: the complex network of generation and transmission facilities, the limited storage capacity for electricity, and the physical requirement for instantaneous balancing of generation and use. Given these constraints, a fully decentralized electricity market is not technologically possible. Therefore, in designing electricity markets, the choice is not between central and decentralized coordination--the choice is only about the rules of central coordination. Good rules that reflect the reality of the system can support competition in generation and supply. Bad rules will raise costs, and may cause a market collapse.
But the political process took over in 1995, and California turned away from these basic principles. The resulting system, based on a boundless faith in the ability of markets to solve all problems, fatally overlooked the inescapable need for a market-friendly coordination function. Worse yet, California embraced the notion that what little the system operator would do should be done inefficiently, in order to leave even more coordination problems for the market to solve. Any dissent was suppressed as "Soviet-style central planning" or the work of mere "academics" unfamiliar with "real" markets. In effect, California undertook an unprecedented experiment with a seriously flawed market design that could not work in theory. We now know that it did not work in practice, either.
At the heart of California's system was a commitment to creating a market for electricity traders, no matter what the cost, to be effected through a complicated trading regime. Instead of a single coordinator, there would be two--the California Independent System Operator (CAISO) and a separate Power Exchange (PX) --whose relationship and market separation required an expanding collection of arcane rules. Eventually CAISO and PX were operating so many uncoordinated and inconsistent markets for energy and ancillary services that it was amazing the system worked at all.
Failures precipitated by the market design began compounding from the moment of the market's inception in 1998. Market participants responded to the resulting perverse incentives, prompting rampant accusations that many generators were able to manipulate the confusing rules, further compounding the crisis and increasing prices. Rather than correcting the market flaws, CAISO issued a series of amendments adding command-and-control mechanisms to counteract the perverse incentives. The fixes themselves created new problems that served as barriers to the entry of efficient new generators into the market (Amendment 19), or impeded efficient use of the available supplies (Amendment 23). Interestingly, the New England system operator had in 1998 abandoned a policy like California's Amendment 19 when it proved unworkable. In its place, New England embraced a market model that has worked well in New York and the Mid Atlantic region, successfully mitigating the incentives and the opportunities for market manipulation.
Rejecting the relevance of the New England experience, California continued with its unique approach. CAISO recognized that the pricing system had not drawn forth the necessary level of power generation and transmission, but--hemmed in by its basic design principles--it sought to remedy the inadequate returns and lack of investment within transmission-constrained regions by proposing a process that would have, in effect, turned reform upside down. It would have recreated a vertically integrated monopoly to make the major investment decisions--but now the monopoly would be CAISO, rather than the regulated utilities of old.
In December 1999, federal regulators rejected the ad hoc market adjustments as "fundamentally flawed." (By the standards of regulatory prose, this was strong language.) There then began an intense process to rethink the full market design from first principles.
But time ran out. When the bad luck arrived in the summer of 2000, California's "comprehensive market redesign" effort was blown apart as variable wholesale prices collided explosively with fixed retail prices in a suddenly tight market. Bad luck collided with bad policy. The state government intervened to impose retail price caps in San Diego--the one region that had moved to an open retail market. Federal regulators issued a series of orders reflecting the view that the problems must be solved largely in California. The state Department of Water Resources jumped in to buy power for the soon-to-be-bankrupt utilities, which had stopped paying their bills. Soon PX closed its doors, but its essential functions were not transferred to CAISO. The state then launched a long-term program to take over or at least play a prominent role in the electricity market. Even those who had originally predicted problems were surprised at the scope and speed of the policy disaster.
The tragic case of California illustrates the basic truth that the "magic" of the market is no sure thing. Conditions were so extreme in California that even a good market design might not have survived the summer of 2000 and its aftermath. Whatever the ultimate outcome, one thing is already clear, wherever electricity reform is on the agenda. Beyond avoiding obvious mistakes that risk major failure in any market, like buying short and selling long, regulators must pay more attention to the complex and unique issues of electricity-market coordination.
At great expense, California has tested and found wanting the proposition that market forces can overcome anything, even bad market design. Markets for electricity can work, delivering benefits such as lower costs, innovative products, and more flexibility, but only if the rules are designed explicitly to support competition under the special conditions governing electricity supply and distribution. We in fact know what to do, as evidenced by the successes in New York, the Mid Atlantic region, and elsewhere. Replicating the successes will require leadership from government. Electricity markets are made, they don't just happen.
William W. Hogan is Littauer professor of public policy and administration at the Kennedy School of Government and research director of the Harvard Electricity Policy Group. He is also a director of the international consulting firm LECG, LLC and has consulted to utilities and government agencies throughout the United States and abroad. His research papers can be found on-line at www.ksg.harvard.edu/whogan.
