Business and economics are social sciences since their dynamics depend very much on human behavior. In the natural sciences, it is relatively easy to determine which mass weighs more and which body in motion has higher velocity. In business and economics comparisons are much more difficult to make than they appear. In fact, many news stories and high-ranking individuals make comparisons that are essentially false, because they don’t compare “apples to apples.”
The gold standard for an “apples to apples” comparison is the randomized, controlled experiment. In this situation, every possible variable is held constant except for the one variable being tested – for example, the effectiveness of a particular pharmaceutical.
In business and economics, randomized, controlled experiments are almost never possible for a variety of reasons such as ethics (it isn’t right to give half of your managers training that you think will help them develop, and half not, to see which group has a higher promotion rate five years later) or the impossibility of a control group (we can’t, for example, see what America would be like with the Affordable Care Act and run a parallel America without it to see what the differences would be).
Still, if we are going to be intellectually honest, we have to do the best possible job that we can in trying to make “apples to apples” comparisons. For example, in the past 12 months North Carolina has created more jobs than every state except Texas, Florida, and California. We often make comparisons between states, on a per person basis, because we can assume they are all operating under the same set of federal laws and policies. We, therefore, assume, any differences between the states must be a result of laws, policies, and factors specific to a given state. The state that has had a very large drop in unemployment is North Dakota. They have created more jobs than North Carolina per person in the state. Is it intellectually honest, then, to say that North Dakota has better job-creating policies than North Carolina? No, because North Dakota has a unique factor at this time. They have been able to create jobs based on their relatively unique access to oil, natural gas, and shale formations. Still, in the grand scheme of things, making state-to-state comparisons is generally fairly accurate, as long as we are on the lookout for unique situations.
Another common way to make comparisons is to evaluate performance at one point in time with performance at another point in time. For example, we could analyze whether a salesperson sold more cars in 2014 than in 2010. If she did, we might say she improved as a salesperson. Would comparing these two years be intellectually honest? No, because car sales are most dramatically impacted by the overall state of the economy. Is it intellectually honest to compare the overall state of the economy between, say, 2010 and 2014? No, because we would be comparing the first year after a recession with the fifth year after a recession. Since World War II, the United States has had 11 recessions, lasting in duration from 8 months to 18 months. The chart below contains the information from the last five recessions. The current recession’s data is in bold and marked with an asterisk.
GDP growth Income growth Job creation
35.4% 23.0% 17.3%
27.7% 16.7% 16.4%
16.7% 11.6% 9.6%
15.8% 8.7% **5.7%
**10.8% **3.2% 4.5%
So, how has our country been doing, in an intellectually honest comparison? Not very good. Actually, quite poorly. Of the last five recoveries, the current recovery is fifth in GDP growth, fifth in income growth, and fourth in job creation.
So, if you own a car dealership, don’t fall for your salesperson telling you how much better she is at sales now. And, don’t let anyone tell you our economy has performed well over the past five years. It hasn’t. Changes in laws, policies, or other factors during the past five years have caused our economy to function much more poorly than it should have.