Racing's Forever A/B Testing Problem

I was listening to a twitter spaces last week, and I think it was Keith Bush (@thoreval) who brought up the concept of properly testing takeout rates to receive actionable data. That's a good question. 

I was reminded of these testing questions just this past week when training for a revamp of a tech platform - you know, new stuff from the smart people. The sales team and engineers had been working on it for tens of thousands of man hours costing millions and millions of dollars, and when it was released, it was met with a thud. At the training session I thought there was going to be a riot. There was no feedback loop, no preliminary testing, nothing was nimble, it was just built as an end product.  

The more noise you have, the more insular you are, the more complex the decision making tree, the more cooks in the kitchen, is (and always has been) inversely correlated to accomplishing your testing goals. 

Meanwhile for a complete shock to the system at the other extreme, I log onto twitter last night, where I see people talking about how twitter is now broken. I don't know if that's true or not, but this loop is certainly the diametrical opposite. 

Musk buys twitter, seemingly thinks of a few things to test on the way to work, and an hour later the engineering team has it out to the world of 200 million or so users. There's real-time feedback, and they can reverse the change after a sandwich at lunch. The team keeps going in a game of whack-a-mole, or shuts it down completely and tries something new. All of this is done transparently. 

This type of testing has never been done in human history in this way. Early on, there's ample evidence it's stupidity, and maybe it's a certainty business books we geeks read for fun will conclude so. But since we have no test-case for comparison, who the hell knows.  

Racing, as we all know, doesn't have a single person owning the entity. And, with its built-in protectionism this sport is not even approaching my training session company in terms of testing and feedback of policy change.  Achieving an A/B test on lasix, or takeout, or track off-times, or a hundred or a thousand others things to do with wagering is where ITP responds with his laughing Mike Tyson gif - it's not happening. 

This is a fascinating world and change is happening in real-time right in front of our mugs. It'll be written and spoken about for generations. What a time to be alive. 

But racing's forever A/B testing problem ensures it won't be a part of it; the good parts, or the bad. It's simply stuck in its lane. It can't change from blinkers to an open even if it wanted to. 

Have a nice weekend everyone.