[InstCombine] Fix removal from deferred instructions

Make sure we don't skip the Deferred.remove() call if the
instruction is not in the worklist. Both of those are separate.

We don't have any cases where deferred instructions get removed
right now, but may cause problems in the future.