The Last Non-Neural Candidate, and It Did Not Clear the Bar

The question this whole series exists to answer is whether language-model-like behaviour can come from something other than a large number of trained parameters. An early set of experiments turned tha...