diff options
author | Joel Brobecker <brobecker@adacore.com> | 2020-12-05 23:56:59 -0500 |
---|---|---|
committer | Joel Brobecker <brobecker@adacore.com> | 2020-12-05 23:56:59 -0500 |
commit | 63c457b911043aa6ebf3558e2d2050ee53d28a8e (patch) | |
tree | c398cb09dc5e8fa3c217f18bd907cfa08a9cff99 /gdb/gmp-utils.h | |
parent | 6b1dce3a3d0c853dc11db2f942038af792cb7b3f (diff) | |
download | fsf-binutils-gdb-63c457b911043aa6ebf3558e2d2050ee53d28a8e.zip fsf-binutils-gdb-63c457b911043aa6ebf3558e2d2050ee53d28a8e.tar.gz fsf-binutils-gdb-63c457b911043aa6ebf3558e2d2050ee53d28a8e.tar.bz2 |
gmp-utils: protect gdb_mpz exports against out-of-range values
The gdb_mpz class currently provides a couple of methods which
essentially export an mpz_t value into either a buffer, or an integral
type. The export is based on using the mpz_export function which
we discovered can be a bit treacherous if used without caution.
In particular, the initial motivation for this patch was to catch
situations where the mpz_t value was so large that it would not fit
in the destination area. mpz_export does not know the size of
the buffer, and therefore can happily write past the end of our buffer.
While designing a solution to the above problem, I also discovered
that we also needed to be careful when exporting signed numbers.
In particular, numbers which are larger than the maximum value
for a given signed type size, but no so large as to fit in the
*unsigned* version with the same size, would end up being exported
incorrectly. This is related to the fact that mpz_export ignores
the sign of the value being exportd, and assumes an unsigned export.
Thus, for such large values, the appears as if mpz_export is able
to fit our value into our buffer, but in fact, it does not.
Also, I noticed that gdb_mpz::write wasn't taking its unsigned_p
parameter, which was a hole.
For all these reasons, a new low-level private method called
"safe_export" has been added to class gdb_mpz, whose goal is
to perform all necessary checks and manipulations for a safe
and correct export. As a bonus, this method allows us to factorize
the handling of negative value exports.
The gdb_mpz::as_integer and gdb_mpz::write methods are then simplified
to take advantage of this new safe_export method.
gdb/ChangeLog:
* gmp-utils.h (gdb_mpz::safe_export): New private method.
(gdb_mpz::as_integer): Reimplement using gdb_mpz::safe_export.
* gmp-utils.c (gdb_mpz::write): Rewrite using gdb_mpz::safe_export.
(gdb_mpz::safe_export): New method.
* unittests/gmp-utils-selftests .c (gdb_mpz_as_integer):
Update function description.
(check_as_integer_raises_out_of_range_error): New function.
(gdb_mpz_as_integer_out_of_range): New function.
(_initialize_gmp_utils_selftests): Register
gdb_mpz_as_integer_out_of_range as a selftest.
Diffstat (limited to 'gdb/gmp-utils.h')
-rw-r--r-- | gdb/gmp-utils.h | 40 |
1 files changed, 22 insertions, 18 deletions
diff --git a/gdb/gmp-utils.h b/gdb/gmp-utils.h index 12e4f8e..dae62cf 100644 --- a/gdb/gmp-utils.h +++ b/gdb/gmp-utils.h @@ -121,6 +121,24 @@ private: /* Helper template for constructor and operator=. */ template<typename T> void set (T src); + + /* Low-level function to export VAL into BUF as a number whose byte size + is the size of BUF. + + If UNSIGNED_P is true, then export VAL into BUF as an unsigned value. + Otherwise, export it as a signed value. + + The API is inspired from GMP's mpz_export, hence the naming and types + of the following parameter: + - ENDIAN should be: + . 1 for most significant byte first; or + . -1 for least significant byte first; or + . 0 for native endianness. + + An error is raised if BUF is not large enough to contain the value + being exported. */ + void safe_export (gdb::array_view<gdb_byte> buf, + int endian, bool unsigned_p) const; }; /* A class to make it easier to use GMP's mpq_t values within GDB. */ @@ -258,26 +276,12 @@ template<typename T> T gdb_mpz::as_integer () const { - /* Initialize RESULT, because mpz_export only write the minimum - number of bytes, including none if our value is zero! */ - T result = 0; - - gdb_mpz exported_val (val); - if (std::is_signed<T>::value && mpz_cmp_ui (val, 0) < 0) - { - /* We want to use mpz_export to set the return value, but - this function does not handle the sign. So give exported_val - a value which is at the same time positive, and has the same - bit representation as our negative value. */ - gdb_mpz neg_offset; + T result; - mpz_ui_pow_ui (neg_offset.val, 2, sizeof (T) * HOST_CHAR_BIT); - mpz_add (exported_val.val, exported_val.val, neg_offset.val); - } + this->safe_export ({(gdb_byte *) &result, sizeof (result)}, + 0 /* endian (0 = native) */, + !std::is_signed<T>::value /* unsigned_p */); - mpz_export (&result, NULL /* count */, -1 /* order */, - sizeof (T) /* size */, 0 /* endian (0 = native) */, - 0 /* nails */, exported_val.val); return result; } |