Address post commit review feedback from D137046

It was pointed out the verifier rejects inttoptr and ptrtoint casts with inputs and outputs whose scalability doesn't match.  As such, checking the input type separately from the type of the cast itself is redundant.