Last week, looking for some information about the mathematician Richard Bellman who, among other feats and achievements, developed dynamic programming, I came across a film called the Bellman Equation which you can watch on the internet. It was written produced and narrated by Bellman’s grandson, Gabriel Bellman and features among others, Gabriel’s father (Bellman’s son), Gabriel’s aunt (Bellman’s daughter), Bellman’s first and second wives, and numerous friends and colleagues. You learn how brilliant, driven, arrogant, charming, and difficult Bellman was, and how he cast a shadow over the lives of his children and grandchildren. Aside from the stories about his life, his work on the atomic bomb in World War II, his meeting with Einstein when he was a young assistant professor at Princeton, his run-in with the Julius and Ethel Rosenberg at Los Alamos, and, as a result, with Joe McCarthy. And on top of all the family history, family dynamics, and psychological theorizing, you also get an interesting little account of the intuitive logic underlying the theory of dynamic programming. You can watch it for free with commercials on snagfilms.
But I especially wanted to draw attention to the brief appearance in the video of Bellman’s colleague at Rand Corporation in the 1950s, Stuart Dreyfus, with whom Bellman collaborated in developing the theory of dynamic programming, and with whom Bellman co-wrote Applied Dynamic Programming. At 14:17 into the film, one hears the voice of Stuart Dreyfus saying just before he comes into view on the screen:
The world is full of problems where what is required of the person making eh decision is not to just face a static situation and make one single decision, but to make a sequence of decisions as the situation evolves. If you stop to think about it, almost everything in the world falls in that category. So that is the kind of situation hat dynamic programming addressed. The principle on which it is based is such an intuitively obvious principle that it drives some mathematicians crazy, because it’s really kind of impossible to prove that it’s an intuitive principle, and pure mathematicians don’t like intuition.
Then a few moments later, Dreyfus continues:
So this principle of optimality is: why would you ever make a decision now which puts you into a position one step from now where you couldn’t do as well as [if you were in] some other position? Obviously, you would never do that if you knew the value of these other positions.
And a few moments after that:
The place that [dynamic programming] is used the most upsets me greatly — and I don’t know how Dick would feel — but that’s in the so-called “quants” doing so-called “financial engineering” that designed derivatives that brought down the financial system. That’s all dynamic programming mathematics basically. I have a feeling Dick would have thought that’s immoral. The financial world doesn’t produce any useful thing. It’s just like poker; it’s just a game. You’re taking money away from other people and getting yourself things. And to encourage our graduate students to learn how to apply dynamic programming in that area, I think is a sin.
Allowing for some hyperbole on Dreyfus’s part, I think he is making an important point, a point I’ve made before in several posts about finance. A great deal of the income earned by the financial industry does not represent real output; it represents trading based on gaining information advantages over trading partners. So the more money the financial industry makes from financial engineering, the more money someone else is losing to the financial industry, because every trade has two sides.
Not all trading has this characteristic. A lot of trading involves exchanges that are mutually beneficial, and middlemen that facilitate such trading are contributing to the welfare of society by improving the allocation of goods, services and resources. But trading that takes place in order to exploit an information advantage over a counter-party, and devoting resources to the creation of the information advantages that makes such trading profitable is socially wasteful. That is the intuitive principle insightfully grasped and articulated by Dreyfus.
As I have also pointed out in previous posts (e.g., here, here and here) the principle, intuitively grasped on some level, but not properly articulated or applied by people like Thorstein Veblen, was first correctly explicated by Jack Hirshleifer, who like Bellman and Dreyfus, worked for the Rand Corporation in the 1950s and 1960s, in his classic article “The Private and Social Value of Information and the Reward to Inventive Activity.”
Uneasy Money 
“The financial world doesn’t produce any useful thing. It’s just like poker; it’s just a game. You’re taking money away from other people and getting yourself things.”
That’s a very naïve generalization – but it probably reflects the true motivation underlying the particular attitude in this case very accurately.
It’s a slippery slope. Do you bank the issuance of common stock? Do you ban liquidity by banning the trading of common stock? Do you ban zero sum games? Do you ban people thinking about the future in trading common stock? Do you ban thinking about the future via dynamic programming logic? Do you then ban all the same for bonds that pay interest? Do you then ban all the same for bank deposits that pay interest? Do you then ban derivative instruments? Do you then ban people thinking about mathematics more generally in constructing their wealth? Do you ban personal portfolio construction? Do you ban financial markets? Do you ban capital? Do you ban free will? Do you hand it all over to government?
The financial system has problems and requires well thought out constraints in terms of regulation of risk taking. The game needs rules. But let’s get real. Dynamic programming can no more be shunned or outlawed in finance than breathing can be in bingo.
LikeLike
If I agree on the general remarks about moral judgments upon Financial markets utility, I beg to disagree on the connection with dynamic programming.
Dynamic Programming applications are mostly pricing and replication of dérivatives (the Black Scholes model) so it precisely consists in the mutual benefit category, where one agent transfers his risk to another agent willing to do so.
It is also important to notice that dynamic programming has strong connection with Pontriagyn principle and can be seen as a first order condition (with time as an additional variable). Nowadays also, it has been linked to multiple selves models and sequential equilibrium approaches, so that the tool is in the end very related to “rational expectations” and alikes.
LikeLike
JKH, I understand your discomfort with what Dreyfus says, and I am not in favor of suppressing financial transactions. However, we should understand that not all transactions are mutually beneficial and socially productive. In my first post on this topic over four years ago, I wondered if the divergence between the private and social value of information might provide a rationale for high marginal tax rates on income, because a lot of the people earning the highest incomes are employed in the financial sector. The divergence between the private and social value of information or other activities that generate high incomes is not limited to finance either. But I make the suggestion not to advocate a particular tax policy, but just to point out that there are issues here that frequently don’t get discussed.
Adrien, Thanks for your comment. I don’t know enough about financial models and the math underlying those models to agree or disagree with your characterization. Certainly my limited familiarity with dynamic programming and optimal control theory is in the context of growth theory and macro not finance.
LikeLike
A great deal of the income earned by the financial industry does not represent real output; it represents trading based on gaining information advantages over trading partners. —DG
I would go even a couple more steps. Some trading of course generates transaction fees and is profitable therefore. We have fees for services.
But I suspect a lot of trading generates profits by sheer chance.
That is, I buy IBM and IBM goes up later and I make a lot of money. Indeed, if we subscribe to EMT this must describe most of the profits made.
It is annoying that so much money is made in the unproductive financial and legal services industries.
LikeLike
Isn’t a social function of finance to allocate capital correctly, and “winning” a financial transaction is a payment for allocating the capital correctly, not *necessarily* both parties to the trade benefiting? If these trades don’t take place, prices never move, and capital is never correctly allocated.
Investing in being the first to make the trade transforms this into something like a positional good, but doesn’t that in turn mean the rent is being dissipated? Maybe we are over-investing in making EMH hold within less than a second instead of within ten seconds, but the fact that this dissipates rent makes me question the policy conclusion of higher marginal income tax rates. It does sound more like a call for more regulation. Or did I misunderstand something?
LikeLike
If there is anything that is essentially a zero sum game it is the foreign exchange market. Yes, I know some people are legitimate currency traders and hedgers but it is such a small share of the overall market as to be insignificant. Yet, every multinational bank and brokerage house, etc., has establish a large currency trading desk and considers it a profit center. Something does not compute.
LikeLike
Ryan, The point is that the return to trading includes not only a return for improving the allocation of capital, but also a wealth transfer from the other party. Thus, at the margin, there expected return to trading exceeds the gain from improved resource allocation. More resources are used up in the trading business than are saved by enhanced efficiency. I think that you may be confusing total and marginal effects. I agree that banning all trading would be bad, but at the margin trading is excessive.
You suspect me of calling for more regulation. My goodness, what a terrible accusation to make against a person. I may have to contact Melania Trump’s attorneys to sue you for defamation.
Spencer, Currency trading is profitable for banks because they charge fees to customers, and because there are a lot amateurs that trade in the market and consistently lose money to better informed banks.
LikeLike
The remark about regulation was not sarcastic, though I am now sad if deregulation and populism are now being lumped together in polite society.
The “maybe we are over-investing in making EMH hold within less than a second instead of within ten seconds” was regarding the total versus marginal issue. Someone is going to get the rent regardless of who owns the asset, but the rent-seeking involved here is about getting to the head of the line with respect to correcting the price discrepancy. I recall stories of firms on Wall Street bidding up the placement of their computers so they could place trades more quickly; aren’t these issues analogous to that?
Ultimately, the rent is partially getting dissipated. Using the income tax to address the issue seems like a pretty blunt object.
Implicit in this is the assumption that making markets more efficient carries no positive externalities. I’m not sure if that positive externality is greater or less than the positional good aspect of finance.
LikeLike
Ryan, I’ll try to control my sarcasm, which in these trying times, is often very challenging.
I got that your remark about “over-investing in making EMH hold” was aimed at the total/marginal issue, but I don’t get why you think that rent dissipation somehow makes it a non-issue. Rents are dissipated at the margin, but the rents dissipated at the margin are not those of the billionaires; it’s those of the day traders.
Granted that making markets more efficient does create positive externalities, but if the markets are largely mechanisms by which to exploit information advantages, I have trouble believing that the positive externalities offset the negative ones.
My aim in talking about raising marginal tax rates to offset the negative externalities of various forms of totally or partially unproductive methods of earning income is really just as nihilistic as your point about the positive externalities of making markets more efficient. We are both raising theoretical possibilities with no solid empirical knowledge about the relevant magnitudes. One theoretical possibility can be used to counter the theoretical case for controlling externalities; another can be used to counter the theoretical case for low marginal tax rates. Pure theory doesn’t get us very far. I would never concede that pure theory is useless, but I would be modest in making claims for what it can actually prove.
LikeLike