This reverts commit 4b6c2cd6. The patch caused numerous ARM 32 bit build failures, since we added a 5th item to the PointerUnion, and went over the 2-bits available in the 32 bit pointers.