fix: Don't override global settings for VISIBILITY if set (#2793)

Sometimes programmers want to control this, and while it can be changed after the fact, it's commonly set via a CMAKE_ variable; if that variable is set, we should respect that (like the CMAKE_INTERPROCEDURAL_OPTIMIZATION setting).