Text this: Embers of autoregression show how large language models are shaped by the problem they are trained to solve.