Experimentation is a reasonable tax.
Experimentation helps us reach a socially efficient equilibrium in an understudied market.
Bullshit producers produce their output at near zero marginal cost, but the social costs of bullshit are far greater. Some folks will hear the bullshit and believe it is true — perhaps supposing that people must be saying it for a reason besides vibes and hunches (the primary inputs to the bullshit production process). It’s impossible to look into everything you hear, so an oversupply of bullshit increases the probability that we make the wrong decision by inadvertendly “buying” the bullshit.
The key market failure that leads to an oversupply of bullshit is that the costs of dealing with bullshit fall almost entirely on economic agents that did not produce the bullshit. So, what we need to do to make the market efficient is to lower the costs of countering/dealing with bullshit and increase the costs of producing it.
Experimentation does a little of both. It provides data to refute bullshit, subsidizing the countering process, and it increases the costs to saying bullshit because someone might just run an experiment and show that the claim is bullshit.
VERY IMPORTANT: THE EFFICIENT LEVEL OF BULLSHIT IS NOT ZERO.
Some bullshit is socially valuable. For example, you have a hunch that something might work. It’s not based on anything but vibes, but you just want to test it out. It’s bullshit, right? Well, sometimes these ideas end up working out because vibes are sort of data… The point is that there’s a good level of bullshit. We need to take risks and not just wait until we can “prove” everything. That means allowing for a little bullshit.
I’ve yet to find a more efficient tax/subsidy scheme than Experimentation for approximating the socially efficient equilibrium supply of bullshit. It doesn’t overtax it to the point that no one’s willing to just “try things” or speculate, but it has enough of a tax to control the flow…
Thanks for reading!
Zach
Connect at: https://linkedin.com/in/zlflynn
Check out my Udemy course on causal inference: https://www.udemy.com/course/identifying-causal-effects-for-data-scientists/?couponCode=CHEAPCAUSALINF3
If you want my help with any Experimentation, Analytics, etc. problem, click here.