Fix diverse ERR code conflicts

There was a number of potential range conflicts between reason codes
from different places.  Library specific reason codes are allowed to
start at 100, so it means that anything "global" is limited to the
range 1..99.
At the same time, we have the ERR_R_LIB_xxx reason codes, which have
the same numbers as ERR_LIB_xxx, potential range 1..255.
And then we have the common ERR_R_ reason codes, potential range in
OpenSSL 1.1.1 is 1..99, where fatal reasons occupy 64..99.  For
OpenSSL 3.0-dev, the range for the common reason codes was pushed up
to 64..99 in an attempt to reduce the conflicts with the ERR_R_LIB_xxx
reason codes.

Currently existing conflicts in OpenSSL 1.1.1:

ERR_R_BUF_LIB and ERR_R_PASSED_INVALID_ARGUMENT have the same code.

There are currently no existing conflicts in OpenSSL 3.0-dev, but
considering that ERR_LIB_HTTP is 61, a few more modules and associated
ERR_R_LIB_xxx are going to sniff awfully close to 64, where the
common ERR_R_ codes currently start.

To avoid these range conflicts, the strategy to recognise common
reason codes is change to depend on a reason flag, ERR_RFLAG_COMMON,
and the common error codes themselves have moved start at 256, giving
them the potential range 256..2^18-1, and thus allowing ERR_R_LIB_xxx
the full range of library codes, 1..255.

The dual purpose ERR_R_FATAL is also handled in this change, by
allowing the rflags and reason codes to overlap by 1 bit, and make
both ERR_R_FATAL and ERR_RFLAG_FATAL have the same value, 2^18.

With this change, there's no need to worry about reason code conflicts
any more, every library specific range as well as the common range is
1..2^18-1.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13093)

0 files changed, 0 insertions, 0 deletions