NetworkPkg/IScsiDxe: fix potential integer overflow in IScsiBinToHex()

Considering IScsiBinToHex():

>   if (((*HexLength) - 3) < BinLength * 2) {
>     *HexLength = BinLength * 2 + 3;
>   }

the following subexpressions are problematic:

  (*HexLength) - 3
  BinLength * 2
  BinLength * 2 + 3

The first one may wrap under zero, the latter two may wrap over
MAX_UINT32.

Rewrite the calculation using SafeIntLib.

While at it, change the type of the "Index" variable from UINTN to UINT32.
The largest "Index"-based value that we calculate is

  Index * 2 + 2                                (with (Index == BinLength))

Because the patch makes

  BinLength * 2 + 3

safe to calculate in UINT32, using UINT32 for

  Index * 2 + 2                                (with (Index == BinLength))

is safe too. Consistently using UINT32 improves readability.

This patch is best reviewed with "git show -W".

The integer overflows that this patch fixes are theoretical; a subsequent
patch in the series will audit the IScsiBinToHex() call sites, and show
that none of them can fail.

Cc: Jiaxin Wu <jiaxin.wu@intel.com>
Cc: Maciej Rabeda <maciej.rabeda@linux.intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Siyuan Fu <siyuan.fu@intel.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3356
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Maciej Rabeda <maciej.rabeda@linux.intel.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210608121259.32451-6-lersek@redhat.com>

0 files changed, 0 insertions, 0 deletions