The missing data...  

I have long been suspicious of most microeconomic analysis done on farms when it is based on FBFM (or whatever the equivalent is in your state).  There are several reasons.

First, the data sample is relatively small and not random. Indeed it is self-selected, and I think there are reasons to think it represents a unique sub-sector of any state's farmers. I cannot find any comparisons between the participators and the general farming population, but I suspect FBFM is highly concentrated on smaller and older farmers. Since one reason for continuing I have heard from neighbors who do use FBFM is to "get their taxes done", I'm uncomfortable reaching broad conclusions about the state of farm economics from this (possibly) divergent sample.

Second, I am persuaded FBFM gets little data from larger farm operations. Probably it is because they have their own accountants and tax preparers, or the comfort and skills needed to manage Quickbooks, TurboTax, etc. You can kinda see this in the reports where the numbers from large farms are pretty thin. Without good sampling in this group, any results are certainly skewed.

Similarly, I think FBFM is dealing with older - mostly Boomer - producers. It was very popular when I was younger, but not so much today, and the ease of stating with what you know may be the biasing factor for membership. Again, not a representative or statistically solid sample, IMHO.

Finally, I am concerned about the survivorship bias. We're reaching conclusions about farm business decisions based on winners alone.

Simply put, survivorship bias is your tendency to focus on survivors instead of whatever you would call a non-survivor depending on the situation. Sometimes that means you tend to focus on the living instead of the dead, or on winners instead of losers, or on successes instead of failures. In Wald’s problem, the military focused on the planes that made it home and almost made a terrible decision because they ignored the ones that got shot down.

It is easy to do. After any process that leaves behind survivors, the non-survivors are often destroyed or muted or removed from your view. If failures becomes invisible, then naturally you will pay more attention to successes. Not only do you fail to recognize that what is missing might have held important information, you fail to recognize that there is missing information at all.

You must remind yourself that when you start to pick apart winners and losers, successes and failures, the living and dead, that by paying attention to one side of that equation you are always neglecting the other. If you are thinking about opening a restaurant because there are so many successful restaurants in your hometown, you are ignoring the fact the only successful restaurants survive to become examples. Maybe on average 90 percent of restaurants in your city fail in the first year. You can’t see all those failures because when they fail they also disappear from view. As Nassim Taleb writes in his book The Black Swan, “The cemetery of failed restaurants is very silent.” Of course the few that don’t fail in that deadly of an environment are wildly successful because only the very best and the very lucky can survive. All you are left with are super successes, and looking at them day after day you might think it’s a great business to get into when you are actually seeing evidence that you should avoid it. [More worth reading]

My hypothesis is the losers in our industry are not so different from the winners, except for luck. We don't like to think random chance has that much influence on success, but between, birth, location, marriage, and weather, an awful lot of what we call success is delivered to us free.

That said, I will readily admit we don't have any other sources we could use to generate any kind of decision aids. But the fact FBFM is the best we can do still doesn't make it good information or the analysis useful for many. In fact, it could be, I suspect, a major paralyzing force for farmers who are trying to stay in the middle of the herd and avoid risk. By not being able to see what the both big players do and what failed producers have done, we can reach some false conclusions, and the competitive level we now face leaves little room for blunders based on bad data.