[NFC] Fix int-compared-with-unsigned warnings

This is a follow-up to 7d77bbef where I added some incompatible
comparisons, this should clear:

https://lab.llvm.org/buildbot/#/builders/57/builds/30951