This reverts commit c4a9a374. This and the followup patch keep hitting an assert I wrote on the build bots in a way that isn't clear. Reverting so I can fix it without a rush.
mentioned in commit 8ef2011b