Revert "[llvm] Fix assertion error where we didn't check fixed point types." (#82285)

These changes break the `LLVM::fixed-point.ll` test some targets what
fails the builds for those targets
(more details
https://github.com/llvm/llvm-project/pull/80757#issuecomment-1949382211)

The problem wasn't fixed for few days,

Reverts llvm/llvm-project#80757