Reinventing the Bread Baker’s DDT Formula, Pt 3

Evaluating Accuracy

With a new DDT formula and statistical models for its constituents in hand, it’s time to give it a test drive and look at the accuracy.

My goal: get an average accuracy that’s within 1°C (1.8°F)

Thanks to all the earlier experiments I did, I had tons of data that I could use to evaluate the DDT formula. But before looking at the new formula I figured it would be helpful to look at the common formula accuracy and use that as a baseline.

For friction factor, first I tried the Cooks Illustrated values of 5° by hand and 20° with a mixer:

mean abs error: 5.806153846153844
RMS: 6.932617998108094
SD: 6.525763733670741

(All reported temps are °C.)

So the average error was about 11°F / 6°C. That’s pretty terrible!

But hey, I have all this data so I can actually retroactively calculate what the best FF should be. I wrote a small program to do just that and it yielded:

by hand data best: 12.0
mixer data best: 13.0

So at least for me and my trial runs, the ideal FF is 22°F (12°C) when mixing by hand and 23°F (13°C) when mixing by mixer. Surprisingly, they’re almost the same!

Plugging those FFs back into the validation function:

mean abs error: 4.145384615384615
RMS: 6.616596266498821
SD: 5.365681326231977

Better, but still 7°F/4°C off on average.

Then I tried my shiny new DDT formula:

mean abs error: 3.4390894232822617
RMS: 3.548735658681194
SD: 0.8753220629365653

An improvement — 6.1°F/3.4°C — but still disappointing considering my goal of a 1°C average error.

However, there’s something very interesting about the results— all of the errors were positive, and the standard deviation was small, meaning the errors didn’t vary by much. In other words, if I simply subtracted some constant from the result, the accuracy would improve dramatically. Further, the amounts of these errors were similarly grouped among the by-hand and by-mixer data.

Earlier I mentioned that I wasn’t 100% sure about the specific heat capacity (aka C) values of the flour and starter, and there’s some research indicating that C values will change over different temperatures. Additionally there are undoubtedly other factors that the formula might not capture like the heat transfer to or from the bowl. (Btw, metal bowls like ones used with a stand mixer transfer A LOT more heat than plastic bowls which I used for my by-hand tests. This, probably more than friction, explains the need for different values when using a mixer vs mixing by hand.)

Hypothesizing that these imperfections are related lineraly to the actual behavior, I did another set of linear regression models and after putting those into use:

mean abs error: 0.34919998578937383
RMS: 0.49050557775115333
SD: 0.48761524336474427

Eureka! 0.6°F / 0.35°C is well under my goal of 1°C! And it’s about 12x more accurate than the common formula, even with ideal FF values plugged in.

Picking a Formula

Does all of this mean that the common DDT formula is terrible and you should stop using it? Not necessarily. The common formula works well enough for some bakers and has some definite advantages: you can work it out on the back of a napkin, there are tons of online calculators to use, and it’s been in use for 100 years.

But I think it’s important that the common formula not be used with arbitrary friction factors. The only way it’s going to be remotely useful is if you determine the ideal friction factor for yourself, your methods, and your kitchen. You can do that with the instructions from Part 1 or you can continually adjust it as you go.

On the other hand, the newer DDT formula doesn’t require arbitrary fudge factors, but does require a computer. It would take a lot of napkins to work this out by hand. For better or worse, these days we seem to be surrounded by computers of all sorts so I don’t think that’s much of a downside.

In terms of accuracy, unless you’ve worked really hard to establish your friction factor in all possible conditions, I think you’re going to find the new formula to be much more reliable.

Will it always be <1° off for you like it is for me? Probably not. While I tried my best to make these experiments scientifically sound, I’m not a chemist and I didn’t perform them in a lab. My thermometer, mixer, and flour might be different than yours. How fast do you mix by hand? What speed is your mixer set to? How warm are your hands? These are all variables that aren’t accounted for. That said, it’s been available in our app for a few weeks now and the feedback has been 100% positive.

Well, I think that covers it 😅.

If you want to try out the new DDT formula, it’s available now online or from our app, Rise.

If you’re a web developer and want to incorporate this formula into your own website, you can get the open-sourced javascript here.

Sincere thanks to Christopher Arthur for his help with some math & physics concepts. This work wouldn’t have been possible without his guidance.