diff options
| author | Saagar Jha <saagar@saagarjha.com> | 2019-02-27 18:48:35 +0000 |
|---|---|---|
| committer | Pedro Alves <palves@redhat.com> | 2019-02-27 18:48:35 +0000 |
| commit | 23031e319207140e76a9c18e308995fe28322b58 (patch) | |
| tree | e182f185d020c2572216a1b916ad8f647aeaa78d /gdb | |
| parent | 6c28e44a359e9f6cf455ddff0009ca99406f7224 (diff) | |
| download | gdb-23031e319207140e76a9c18e308995fe28322b58.zip gdb-23031e319207140e76a9c18e308995fe28322b58.tar.gz gdb-23031e319207140e76a9c18e308995fe28322b58.tar.bz2 | |
Prevent overflow in rl_set_screen_size
GDB calls rl_set_screen_size in readline with the current screen size,
measured in rows and columns. To represent "infinite" sizes, GDB
passes in INT_MAX; however, since rl_set_screen_size internally
multiplies the number of rows and columns, this causes a signed
integer overflow. To prevent this we can instead pass in the
approximate square root of INT_MAX (which is still reasonably large),
so that even when the number of rows and columns is "infinite" we
don't overflow.
gdb/ChangeLog:
2019-02-27 Saagar Jha <saagar@saagarjha.com>
Pedro Alves <palves@redhat.com>
* utils.c (set_screen_size): Reduce "infinite" rows and columns
before calling rl_set_screen_size.
Diffstat (limited to 'gdb')
| -rw-r--r-- | gdb/ChangeLog | 6 | ||||
| -rw-r--r-- | gdb/utils.c | 21 |
2 files changed, 23 insertions, 4 deletions
diff --git a/gdb/ChangeLog b/gdb/ChangeLog index de23d4b..02602b2 100644 --- a/gdb/ChangeLog +++ b/gdb/ChangeLog @@ -1,3 +1,9 @@ +2019-02-27 Saagar Jha <saagar@saagarjha.com> + Pedro Alves <palves@redhat.com> + + * utils.c (set_screen_size): Reduce "infinite" rows and columns + before calling rl_set_screen_size. + 2019-02-27 Tom Tromey <tromey@adacore.com> * configure.ac (HAVE_LIBPYTHON2_4, HAVE_LIBPYTHON2_5): Never diff --git a/gdb/utils.c b/gdb/utils.c index ec26196..069da23 100644 --- a/gdb/utils.c +++ b/gdb/utils.c @@ -1380,11 +1380,24 @@ set_screen_size (void) int rows = lines_per_page; int cols = chars_per_line; - if (rows <= 0) - rows = INT_MAX; + /* If we get 0 or negative ROWS or COLS, treat as "infinite" size. + A negative number can be seen here with the "set width/height" + commands and either: - if (cols <= 0) - cols = INT_MAX; + - the user specified "unlimited", which maps to UINT_MAX, or + - the user spedified some number between INT_MAX and UINT_MAX. + + Cap "infinity" to approximately sqrt(INT_MAX) so that we don't + overflow in rl_set_screen_size, which multiplies rows and columns + to compute the number of characters on the screen. */ + + const int sqrt_int_max = INT_MAX >> (sizeof (int) * 8 / 2); + + if (rows <= 0 || rows > sqrt_int_max) + rows = sqrt_int_max; + + if (cols <= 0 || cols > sqrt_int_max) + cols = sqrt_int_max; /* Update Readline's idea of the terminal size. */ rl_set_screen_size (rows, cols); |