Paul Romer has been engaged for some time in a worthy campaign against the travesty of modern macroeconomics. A little over a year ago I commented favorably about Romer’s takedown of Robert Lucas, but I also defended George Stigler against what I thought was an unfair attempt by Romer to identify George Stigler as an inspiration and role model for Lucas’s transgressions. Now just a week ago, a paper based on Romer’s Commons Memorial Lecture to the Omicron Delta Epsilon Society, has become just about the hottest item in the econ-blogosophere, even drawing the attention of Daniel Drezner in the Washington Post.
I have already written critically about modern macroeconomics in my five years of blogging, and here are some links to previous posts (link, link, link, link). It’s good to see that Romer is continuing to voice his criticisms, and that they are gaining a lot of attention. But the macroeconomic hierarchy is used to criticism, and has its standard responses to criticism, which are being dutifully deployed by defenders of the powers that be.
Romer’s most effective rhetorical strategy is to point out that the RBC core of modern DSGE models posit unobservable taste and technology shocks to account for fluctuations in the economic time series, but that these taste and technology shocks are themselves simply inferred from the fluctuations in the times-series data, so that the entire structure of modern macroeconometrics is little more than an elaborate and sophisticated exercise in question-begging.
In this post, I just want to highlight one of the favorite catch-phrases of modern macroeconomics which serves as a kind of default excuse and self-justification for the rampant empirical failures of modern macroeconomics (documented by Lipsey and Carlaw as I showed in this post). When confronted by evidence that the predictions of their models are wrong, the standard and almost comically self-confident response of the modern macroeconomists is: All models are false. By which the modern macroeconomists apparently mean something like: “And if they are all false anyway, you can’t hold us accountable, because any model can be proven wrong. What really matters is that our models, being microfounded, are not subject to the Lucas Critique, and since all other models than ours are not micro-founded, and, therefore, being subject to the Lucas Critique, they are simply unworthy of consideration. This is what I have called methodological arrogance. That response is simply not true, because the Lucas Critique applies even to micro-founded models, those models being strictly valid only in equilibrium settings and being unable to predict the adjustment of economies in the transition between equilibrium states. All models are subject to the Lucas Critique.
Here is Romer’s take:
In response to the observation that the shocks are imaginary, a standard defense invokes Milton Friedman’s (1953) methodological assertion from unnamed authority that “the more significant the theory, the more unrealistic the assumptions (p.14).” More recently, “all models are false” seems to have become the universal hand-wave for dismissing any fact that does not conform to the model that is the current favorite.
Friedman’s methodological assertion would have been correct had Friedman substituted “simple” for “unrealistic.” Sometimes simplifications are unrealistic, but they don’t have to be. A simplification is a generalization of something complicated. By simplifying, we can transform a problem that had been too complex to handle into a problem more easily analyzed. But such simplifications aren’t necessarily unrealistic. To say that all models are false is simply a dodge to avoid having to account for failure. The excuse of course is that all those other models are subject to the Lucas Critique, so my model wins. But your model is subject to the Lucas Critique even though you claim it’s not, so even according to the rules you have arbitrarily laid down, you don’t win.
So I was just curious about where the little phrase “all models are false” came from. I was expecting that Karl Popper might have said it, in which case to use the phrase as a defense mechanism against empirical refutation would have been a particularly fraudulent tactic, because it would have been a perversion of Popper’s methodological stance, which was to force our theoretical constructs to face up to, not to insulate it from, empirical testing. But when I googled “all theories are false” what I found was not Popper, but the British statistician, G. E. P. Box who wrote in his paper “Science and Statistics” based on his R. A. Fisher Memorial Lecture to the American Statistical Association: “All models are wrong.” Here’s the exact quote:
Since all models are wrong the scientist cannot obtain a “correct” one by excessive elaboration. On the contrary following William of Occam he should seek an economical description of natural phenomena. Just as the ability to devise simple but evocative models is the signature of the great scientist so overelaboration and overparameterization is often the mark of mediocrity.
Since all models are wrong the scientist must be alert to what is importantly wrong. It is inappropriate to be concerned about mice when there are tigers abroad. Pure mathematics is concerned with propositions like “given that A is true, does B necessarily follow?” Since the statement is a conditional one, it has nothing whatsoever to do with the truth of A nor of the consequences B in relation to real life. The pure mathematician, acting in that capacity, need not, and perhaps should not, have any contact with practical matters at all.
In applying mathematics to subjects such as physics or statistics we make tentative assumptions about the real world which we know are false but which we believe may be useful nonetheless. The physicist knows that particles have mass and yet certain results, approximating what really happens, may be derived from the assumption that they do not. Equally, the statistician knows, for example, that in nature there never was a normal distribution, there never was a straight line, yet with normal and linear assumptions, known to be false, he can often derive results which match, to a useful approximation, those found in the real world. It follows that, although rigorous derivation of logical consequences is of great importance to statistics, such derivations are necessarily encapsulated in the knowledge that premise, and hence consequence, do not describe natural truth.
It follows that we cannot know that any statistical technique we develop is useful unless we use it. Major advances in science and in the science of statistics in particular, usually occur, therefore, as the result of the theory-practice iteration.
One of the most annoying conceits of modern macroeconomists is the constant self-congratulatory references to themselves as scientists because of their ostentatious use of axiomatic reasoning, formal proofs, and higher mathematical techniques. The tiresome self-congratulation might get toned down ever so slightly if they bothered to read and take to heart Box’s lecture.
Uneasy Money 
I dug up George Box’s quote a year ago in response to a few “all models are false” statements. I couched it in terms of over-parameterization and over-fitting. DSGE models (for example) are far too complex (I use the word “tedious”) given the available data and (un-)available theoretical frameworks to even be attempted today, much less 20 years ago.
http://informationtransfereconomics.blogspot.com/2015/04/all-models-are-wrong-but-some-are.html
I think Romer’s take on the identification problem is the most serious and effective charge, and effectively the RBC approach is identification by question begging as you put it. But I think the identification problem is a consequence of a more serious issue: that macroeconomics doesn’t have a framework that encodes empirically successful models
http://informationtransfereconomics.blogspot.com/2016/09/of-phlogiston-and-frameworks.html
LikeLike
I think my comment was captured by the spam filter. Essentially, I agree about Box and quoted him a year ago. And I think your thesis here is that RBC is identification by question begging — which is a nice phrase!
LikeLike
When I read the title of your post I thought of Box; in particular, I thought of an expanded version of “All models are wrong.” – “All models are wrong but some are useful.” The latter half of that sentence is key. I don’t have the source but Wikiquotes cites “Empirical Model-Building and Response Surfaces” from 1987.
LikeLike
The tiresome self-congratulation might get toned down ever so slightly if they bothered to read and take to heart Box’s lecture.
Don’t you think this statement is at least as contrary to all currently available evidence as anything in the DSGE-New Classical Macro framework (even with the weasel word “might”)? I cannot imagine why you would make this assertion even rhetorically. 😉
LikeLike
@marcel proust and David Glasner,
Perhaps the best response to macroeconomists (mis) quoting Box would be to quote back:
“… overparameterization is often the mark of mediocrity” 🙂
LikeLike
Quite right, I was glad to read Box’s excerpt. As is often the case, this is pure common sense (love the mice and the tigers.). A model which is never expected to predict real world outcomes is no more than a fantasy treasure map. Treasure maps are fun, complete with boxes of gold and sea serpents, but to defend them as true maps which, incidentally, are never expected to yield even one ducat or one dragon, is a peculiar approach indeed.
Regarding “…the standard and almost comically self-confident response of the modern macroeconomists is: All models are false.” There is another standard reply to misbehaviour of economic models: “It’s complex.”
This seems to apply even to results which would ordinarily serve to DISprove the thesis, such as the infamous “trickle-down” concept. The idea, I guess, is that the real world is so complicated that economic results are by definition unpredictable, and no theory can be expected to compass them. However, saying this aloud could perhaps interfere with your lecture tour and policy consultations, so is left as an interpretive exercise to the listener.
LikeLike
From your post on the Lucas critique: “Stated more generally, the problem with Lucas’s example is that it seems to be designed to exclude a priori the possibility of every type of disequilibrium but one, a disequilibrium corresponding to a single type of informational imperfection.”
This is a problem not only with DSGE but with its parent, General Equilibrium theory. Its assumptions are mathematically designed to prevent any possibility of disequilibrium, especially in the labour market which is the first in reality to experience major disequilibrium. See http://www.philipji.com/holes-in-general-equilibrium/ (This is work in progress)
LikeLike
Some advice from the late genius particle physicist Richard P. Feynman. Think of macroeconomics as the science he refers to. ( easyopinions.blogspot.com/#trueScience )
=== ===
“When someone says, ‘Science teaches such and such,’ he is using the word incorrectly. Science doesn’t teach anything; experience teaches it. If he says to you, ‘Science has shown such and such,’ you should ask, ‘How does science show it?’
How did the scientists find out? How? What? Where?’ It should not be ‘science has shown.’ And, you have as much right as anyone else, upon hearing about the experiments and after hearing all the evidence, to judge whether a sensible conclusion has been arrived at.”
=== ===
Mathematics is important in physics because there are miracles such as the exact conservation of energy, momentum, angular momentum, and electric charge. Many results implied by a new proposal can be calculated and compared to repeated experiment.
This has given math a good name, much better than it deserves outside a precise area such as physics. And, much math was invented in physics which has been discarded because it didn’t lead to a description of reality.
Economic theorems go well beyond what can be calculated, observed, or repeated. Such proposals as Keynesian Spending Theory are the equivalent of crazy perpetual motion machines in physics. Keynesians leave out much of the monetary cycle they claim to explain, and then announce that $1 of government spending produces $2, or $5 of increased wealth when the money circulates. Unfortunately for dreamers, the conservation of energy in both physics and economics prevents that from happening overall.
LikeLike
Great post.
However, I don’t agree that there is a difference between being simple and unrealistic. The moment you take a complex idea and convert it into a simpler one, then that simpler one is immediately less realistic than the complex one. It captures less of reality. It is in the nature of simplification that you decrease the realism, but its worth it because you receive an increase in pragmatic use.
In any case, I think this is just an issue of semantics. I don’t think Friedman would disagree with you. If he were here right now, I’m sure he would immediately agree that “simple” would have been a better word choice than “unrealistic”. Wouldn’t it be funny if it turned out that all of the hullaballoo over his famous essay is simply because he made a poor word choice!
LikeLike
@Andrew_M_Garland
“Unfortunately for dreamers, the conservation of energy in both physics and economics prevents that from happening overall.”
If the Law of Conservation of Energy applies to economics how is that an economy grows?
Why are we not still living in the caves?
LikeLike
Jason, Thanks for the link to your post, which really gets to the heart of the problem. However, I would add that a theorist who has good theoretical reasons for adding parameters shouldn’t be prevented from doing so. But he should acknowledge that his theory is not testable in practice, thought it might be in principle, because there are insufficient data. The problem with RBC and DSGE models is not just that they add parameters, but that the theoretical concepts that the models use, technology shocks, taste shocks and the like are simply vacuous, which is what I meant by question-begging. See also my response to Chris below.
Chris, I am still enough of a Popperian to think that we not only want models to be useful but we want them to be true, even though we know that perfect and certain truth is unattainable. What the whole discussion ignores – and even Box didn’t make the point clearly – is that there is no point in arguing about whether a single model or theory is true or false in the abstract. For a discussion about whether a given model or theory is true or useful to be worth having, you really need at least two theories. And so the relevant question becomes not is theory A true or false, but does theory A come closer to being true than some alternative theory B. Modern macroeconomics has used the microfoundations and Lucas Critique to disqualify every competing theory on methodological grounds, so that they can avoid having to answer the question whether their theory does a better job of explaining the known data or predicting new data than alternative theories.
marcel proust, Good point. I am afraid that my statement was just another example of the triumph of hope over experience. But hope springs eternal.
Herman, If not the best, certainly very good.
Noni, I actually believe that the world is extremely complex, and highly simplified theories are likely to leave a lot to be desired. But we should be honest about the limitations of complex theories, and not be too presumptuous or arrogant in disqualifying theories we don’t like based on purely methodological criteria. That’s what I found intolerable about modern macroeconomics.
Phillip, The difference between general equilibrium theory and DSGE as deployed by modern macroeconomists is that general equilibrium theory does not make strong assertions about macroeconomic relationships and causality. It is a framework that can accommodate a lot of very different causal mechanisms. And most important, GE theory makes no statement about the likelihood that actual economies operate as if general equilibrium obtains, that general equilibrium is locally or globally stable, or that it is unique, while modern macroeoncomics presumes that economies are continuously operating in the neighborhood of general equilibrium and that the equilibrium is unique and stable.
Andrew, Thanks for the quotes from Feynman. I think that you have misstated what Keynesians assert about the possible effects of government spending and even if you had accurately stated what it is that Keynesians assert it would not necessarily be the case that the Keynesians were asserting a violation of any conservation law. You could just as well say that a conservation law was being violated by saying that total income is greater at a pareto-optimal allocation of resources than at a pareto-suboptimal allocation.
Ilya, Thanks. You may be right that Friedman really meant what I did, but I am not so sure that he did.
LikeLike
Stuart in the big Bang Theory : “It’s a little wrong to say a tomato is a vegetable, but it’s very wrong to say it’s a a suspension bridge”.
I like setting things straight with DSGE models. But I wonder how far we can go with some very different models (non-linear dynamical systems for example) with that point (wrongness).
LikeLike
All models are false because all economists are stupid
Comment on David Glasner on ‘Paul Romer on Modern Macroeconomics, Or, the “All Models Are False” Dodge’
Theoretical economics is science and science is digital=binary=true/false and NOTHING in between. Political economics is the very opposite of science and it resides in the bottomless morass between true/false where “… nothing is clear and everything is possible.” (Keynes, 1973, p. 292). One will invariably find that political economists adhere to the freak methodology of anything goes/nothing matters.*
What David Glasner does not seem to realize in his laudable refutation of the silly ‘All models are wrong’ excuse is that with regard to simplification and abstraction and the deductive method ALL has been said by the founding fathers:
“Any order of phenomena, however complicated, may be studied scientifically provided the rule of proceeding from the simple to the complex is always observed.” (Walras, 2010, p. 211)
“The conclusions of geometry are not strictly true of such lines, angles, and figures, as human hands can construct. But no one, therefore, contends that the conclusions of geometry are of no utility, or that it would be better to shut up Euclid’s Elements, and content ourselves with ‘practice’ and ‘experience’.” (Mill, 1874, V.48)
“The ground of confidence in any concrete deductive science is not the à priori reasoning itself, but the accordance between its results and those of observation à posteriori.” (Mill, 2006, p. 896-897)
In his recent paper ‘The Trouble With Macroeconomics’** Paul Romer puts DSGE and its main proponents ― Lucas, Sargent, Prescott ― to rest. In the final section ‘The Trouble Ahead For All of Economics’ he appeals to emotions: “It is sad to recognize that economists who made such important scientific contributions in the early stages of their careers followed a trajectory that took them away from science.”
While it is true that DSGE and its proponents are out of science it is NOT correct to give the impression that an accident happened at some point on the way which lead from science to non-science. The fact of the matter is that DSGE has ALWAYS been out of science because economics has ALWAYS been out of science. The leaders of the DSGE/RBC program have done exactly what they were supposed to do: “It is a touchstone of accepted economics that all explanations must run in terms of the actions and reactions of individuals. Our behavior in judging economic research, in peer review of papers and research, and in promotions, includes the criterion that in principle the behavior we explain and the policies we propose are explicable in terms of individuals, not of other social categories.” (Arrow, 1994, p. 1)
This definition of the subject matter translates into the following set of hard core propositions/axioms: “HC1 economic agents have preferences over outcomes; HC2 agents individually optimize subject to constraints; HC3 agent choice is manifest in interrelated markets; HC4 agents have full relevant knowledge; HC5 observable outcomes are coordinated, and must be discussed with reference to equilibrium states.” (Weintraub, 1985, p. 147)
The critical axioms are HC2 and HC5. Krugman put it nicely: “… most of what I and many others do is sorta-kinda neoclassical because it takes the maximization-and-equilibrium world as a starting point.”
It is obvious to anyone with a modicum of scientific instinct that the axiomatic starting point of orthodox economics is methodologically forever unacceptable. The axiom set consists of blatant nonentities but each student generation has swallowed it without turning an eyelid. Lucas, Sargent, Prescott certainly did. This is scientifically disqualifying.
In order to be applicable HC3, which translates formally into calculus, requires a lot of auxiliary assumptions, most prominently a well-behaved production function. Taken together, all axioms and auxiliary assumptions crystallize to SS-DD-equilibrium or what Leijonhufvud famously called the Totem of Micro/Macro.
Needless to stress that ALL THREE elements of the standard tool (SS-function, DD-function, equilibrium) are nonentities. The fact that neither SS/DD functions nor equilibrium exist leads with inescapable consequence to the identification problem in Econometrics (see Romer Sec. 4). Romer takes this insurmountable technical difficulty as methodological silver bullet in order to finish off DSGE/RBC. It should be noted, though, that the identification problem has its roots in the Walrasian axiom set HC1/HC5 which is the accepted common ground of orthodox economics.
To throw DSGE/RBC unceremoniously out of science is only the first step because textbook supply-demand-equilibrium, which is built upon the same maximization-and-equilibrium axioms, is rubbish since Jevons/Walras/Menger.
This is the current state of economics: Walrasian microfoundations are false since 140 years and Keynesian macrofoundations are false since 85 years. As a consequence, roughly 90 percent of the content of peer-reviewed economic quality journals and 100 percent of textbooks is false.
To rise above the proto-scientific level requires a paradigm shift from Walrasian microfoundations and Keynes’s flawed macrofoundations to entirely new macrofoundations (2015). In methodological terms, rethinking macroeconomics requires the replacement of false axioms by true axioms and the dishonorable discharge of the Walrasian, Keynesian, Marxian, Austrian crowd from science.
Egmont Kakarot-Handtke
References
Arrow, K. J. (1994). Methodological Individualism and Social Knowledge.
American Economic Review, Papers and Proceedings, 84(2): 1–9. URL http://www.jstor.org/stable/2117792
Kakarot-Handtke, E. (2015). Major Defects of the Market Economy. SSRN Working Paper Series, 2624350: 1–40. URL http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2624350
Keynes, J. M. (1973). The General Theory of Employment Interest and Money. London, Basingstoke: Macmillan.
Mill, J. S. (1874). Essays on Some Unsettled Questions of Political Economy. On the Definition of Political Economy; and on the Method of Investigation Proper To It. Library of Economics and Liberty. URL http://www.econlib.org/library/
Mill/mlUQP5.html#EssayV.OntheDefinitionofPoliticalEconomy
Mill, J. S. (2006). A System of Logic Ratiocinative and Inductive. Being a Connected View of the Principles of Evidence and the Methods of Scientific Investigation, volume 8 of Collected Works of John Stuart Mill. Indianapolis, IN: Liberty Fund.
Walras, L. (2010). Elements of Pure Economics. London, New York, NY: Routledge.
Weintraub, E. R. (1985). Joan Robinson’s Critique of Equilibrium: An Appraisal. American Economic Review, Papers and Proceedings, 75(2): 146–149. URL http://www.jstor.org/stable/1805586.
* See also post ‘Eclecticism, anything goes, and the pluralism of false theories’
http://axecorg.blogspot.de/2016/08/eclecticism-anything-goes-and-pluralism.html
** Paper of Sep 14, 2016
Click to access WP-Trouble.pdf
LikeLike
To Henry,
Yes, I didn’t state that well. Clearly, skill and knowledge transform less valuable inputs into more valuable outputs, so there is no conservation of value in an economy. Certainly a good thing.
My unclear comment was about the supposed Keynesian effect of creating value by printing money, buying just anything with it, and producing wealth by the mere fact of money circulation. Our government calls this “stimulus”.
Keynes wrote that this wealth creation was so strong that it didn’t matter what was bought initially. Workers could be paid to dig ditches and refill them. The new money would create a stream of transaction which would create multiples of wealth and would also help the formerly unemployed who were paid for the digging.
Such “injections” of new money almost never create additional wealth overall. If the ditches are useless, the new money merely reallocates existing wealth to the diggers. The new money reduces the value of all other dollars as more cash chases the same amount of goods for sale in the economy.
If the ditches have some market value, but less than the money paid, then the shortfall reduces wealth and adds to inflation.
It is extremely unlikely that some government project (a government ditch) could produce goods with a market value greater than the stimulus funds used. In that case only, the stimulus funds would produce more value for the society than the value of the goods used to pay the workers.
LikeLike
Andrew
What about the situation where there is unemployment and unused capacity?
That’s the Keynesian context.
LikeLike
So, it goes like this: Cite theory when theory buttresses your politico-economic position.
If theory does not, then cite structural impediments.
If both those methods fail, then cite a model.
LikeLike
Adrien, If I am understanding you, I would agree that just because DSGE is bad, not every alternative is better.
Benjamin, I am pretty sure that I am not understanding you.
LikeLike
To Henry,
The creation of wealth works as I stated above. Digging ditches or supplying the government with more filing cabinets (government ditches) increases wealth only if the government projects produce things which are more valuable to the public than the costs.
The claim of “excess capacity” is not relevant. A change in priorities or tastes makes some activities excess and other activities scarce. Most businesses have excess capacity most of the time; they could produce more if more people wanted to buy more.
It doesn’t help for the government to buy more stuff with created money to temporarily support the jobs of people in a disfavored industry. That created money dilutes the value of the money supply (inflation), buys goods by definition which are not as highly valued as the money used to buy those goods, and it must be paid back in higher taxes at a later time. Meanwhile, the different goods are not created which are now in demand.
For example, losses and turmoil from the housing crisis prompted many people to increase their savings to support their future rather than consume in the present. But, government regulation and monetary policy discouraged these increased investments. Using scarce resources to build even more housing to support construction jobs would be a waste, no matter how much construction capacity was available.
Investments were needed, not more housing. Coincidentally, government was forming policies which discouraged new business formation, including higher minimum wages, more wage and hour rules, and a generally more aggressive view of regulation and enforcement.
LikeLike
the Lucas Critique applies even to micro-founded models, those models being strictly valid only in equilibrium settings and being unable to predict the adjustment of economies in the transition between equilibrium states. All models are subject to the Lucas Critique.
Thank you! This has bugged me for years. You can’t have an intertemporal equilibrium and then talk about transition from one equilibrium to another. You can have rational expectations an then talk about “shocks”. To say that there is a shock in this period, is just to say that expectations in the previous period were wrong.
LikeLike
you can’t have rational expectations…, I mean.
LikeLike
JW, Thanks, this is a very nice way of putting it. But I would say it that the shock and the disappointment of expectations are one and the same. But the rational expectations principle just assumes that people “recalculate” after the shock and instantaneously transition to the new equilibrium path.
LikeLike