Tuesday, June 04, 2013

The missing data...  

I have long been suspicious of most microeconomic analysis done on farms when it is based on FBFM (or whatever the equivalent is in your state).  There are several reasons.

First, the data sample is relatively small and not random. Indeed it is self-selected, and I think there are reasons to think it represents a unique sub-sector of any state's farmers. I cannot find any comparisons between the participators and the general farming population, but I suspect FBFM is highly concentrated on smaller and older farmers. Since one reason for continuing I have heard from neighbors who do use FBFM is to "get their taxes done", I'm uncomfortable reaching broad conclusions about the state of farm economics from this (possibly) divergent sample.

Second, I am persuaded FBFM gets little data from larger farm operations. Probably it is because they have their own accountants and tax preparers, or the comfort and skills needed to manage Quickbooks, TurboTax, etc. You can kinda see this in the reports where the numbers from large farms are pretty thin. Without good sampling in this group, any results are certainly skewed.

Similarly, I think FBFM is dealing with older - mostly Boomer - producers. It was very popular when I was younger, but not so much today, and the ease of stating with what you know may be the biasing factor for membership. Again, not a representative or statistically solid sample, IMHO.

Finally, I am concerned about the survivorship bias. We're reaching conclusions about farm business decisions based on winners alone.
Simply put, survivorship bias is your tendency to focus on survivors instead of whatever you would call a non-survivor depending on the situation. Sometimes that means you tend to focus on the living instead of the dead, or on winners instead of losers, or on successes instead of failures. In Wald’s problem, the military focused on the planes that made it home and almost made a terrible decision because they ignored the ones that got shot down.
It is easy to do. After any process that leaves behind survivors, the non-survivors are often destroyed or muted or removed from your view. If failures becomes invisible, then naturally you will pay more attention to successes. Not only do you fail to recognize that what is missing might have held important information, you fail to recognize that there is missing information at all.
You must remind yourself that when you start to pick apart winners and losers, successes and failures, the living and dead, that by paying attention to one side of that equation you are always neglecting the other. If you are thinking about opening a restaurant because there are so many successful restaurants in your hometown, you are ignoring the fact the only successful restaurants survive to become examples. Maybe on average 90 percent of restaurants in your city fail in the first year. You can’t see all those failures because when they fail they also disappear from view. As Nassim Taleb writes in his book The Black Swan, “The cemetery of failed restaurants is very silent.” Of course the few that don’t fail in that deadly of an environment are wildly successful because only the very best and the very lucky can survive. All you are left with are super successes, and looking at them day after day you might think it’s a great business to get into when you are actually seeing evidence that you should avoid it. [More worth reading]
My hypothesis is the losers in our industry are not so different from the winners, except for luck. We don't like to think random chance has that much influence on success, but between, birth, location, marriage, and weather, an awful lot of what we call success is delivered to us free.

That said, I will readily admit we don't have any other sources we could use to generate any kind of decision aids. But the fact FBFM is the best we can do still doesn't make it good information or the analysis useful for many. In fact, it could be, I suspect, a major paralyzing force for farmers who are trying to stay in the middle of the herd and avoid risk. By not being able to see what the both big players do and what failed producers have done, we can reach some false conclusions, and the competitive level we now face leaves little room for blunders based on bad data.

2 comments:

Anonymous said...

Interesting thoughts John. Do you think the number of larger farms in the data set are much different on a percentage basis than what is the true percentage of large farms by number in the state? In other words, how many are there really when one considers the fragmented nature of larger farms being split across multiple family members or entities? Do you feel that the representation of FBFM in your area is true of the rest of the state?

In our area, many of the larger farms do in fact work with FBFM, as the importance to a lender of having financial documents created by a third party (an informal audit if you will) gives some sense of security, particularly as the numbers and risk involved with farming increase.

Do you feel there is valuable data we are missing out on with larger farm operations? I have seen studies that conclude that the operational efficiencies with being a larger farm don't always exist.

I certainly agree that you may be correct, just posing food for your thought.

John Phipps said...

Anon:

The big are different, although not always in the ways we think and tend to measure. (Cost per acre means less for them, for example. As someone pointed out, at some point it's not the rate fo profit, but the volume that drives the compettive advantage.) I also think they tend to be more aggressive, which would show up as lower margins. which could be interpreted as less efficient. In a zero-sum game of land this tendency alone accounts for much of the concentration.

Second, all the FBFM reports I have seen from IL, anyway usually make note of the thin sample of over-5000 acre farms.

My overriding concern is the same I have with the universal advice from academia and finance for farmers: be conservative. Don't pay/bid too much, control costs, don't take unnecessary risks, etc.

Experience tells me this is a path to extinction in our industry today. And if we get a higher "safety net" crop insurance deal such advice will could be how to become roadkill.

But as I note we don't have any other benchmarks, whcih leads me to suggest to farmers that comparison to FBFM colleagues should offer little comfort or unease.

Do we know the loss rate/reasons for those who drop out of FBFM? Are there "autopsy" data that can be accessed?

It doesn't help for somebody like me to point out something isn't perfect when it's the best we can do. But I think there should be some cautions about what conclusions you can and cannot draw from FBFM summaries.

Thanks for responding.