From deb1ba4e38bf1427d4297b1e9b3e5e73cbc9e456 Mon Sep 17 00:00:00 2001
From: Tom de Vries <tdevries@suse.de>
Date: Sun, 30 Apr 2023 13:06:23 +0200
Subject: [gdb/tui] Fix TUI resizing for TERM=ansi

With TERM=ansi, when resizing a TUI window from LINES/COLUMNS 31/118
(maximized) to 20/78 (de-maximized), I get a garbled screen (that ^L doesn't
fix) and a message:
...
@@ resize done 0, size = 77x20
...
with the resulting width being 77 instead of the expected 78.

[ The discrepancy also manifests in CLI, filed as PR30346. ]

The discrepancy comes from tui_resize_all, where we ask readline for the
screen size:
...
   rl_get_screen_size (&screenheight, &screenwidth);
...

As it happens, when TERM is set to ansi, readline decides that the terminal
cannot auto-wrap lines, and reserves one column to deal with that, and as a
result reports back one less than the actual screen width:
...
$ echo $COLUMNS
78
$ TERM=xterm gdb -ex "show width" -ex q
Number of characters gdb thinks are in a line is 78.
$ TERM=ansi  gdb -ex "show width" -ex q
Number of characters gdb thinks are in a line is 77.
...

In tui_resize_all, we need the actual screen width, and using a screenwidth of
one less than the actual value garbles the screen.

This is currently not causing trouble in testing because we have a workaround
in place in proc Term::resize.  If we disable the workaround:
...
-       stty columns [expr {$_cols + 1}] < $::gdb_tty_name
+       stty columns $_cols < $::gdb_tty_name
...
and dump the screen we get the same type of screen garbling:
...
    0 +---------------------------------------+|
    1                                         ||
    2                                         ||
    3                                         ||
...

Another way to reproduce the problem is using command "maint info screen".
After starting gdb with TERM=ansi, entering TUI, and issuing the command, we
get:
...
Number of characters curses thinks are in a line is 78.
...
and after maximizing and demaximizing the window we get:
...
Number of characters curses thinks are in a line is 77.
...
If we use TERM=xterm, we do get the expected 78.

Fix this by:
- detecting when readline will report back less than the actual screen width,
- accordingly setting a new variable readline_hidden_cols,
- using readline_hidden_cols in tui_resize_all to fix the resize problem, and
- removing the workaround in Term::resize.

The test-case gdb.tui/empty.exp serves as regression test.

I've applied the same fix in tui_async_resize_screen, the new test-case
gdb.tui/resize-2.exp serves as a regression test for that change.  Without
that fix, we have:
...
FAIL: gdb.tui/resize-2.exp: again: gdb width 80
...

Tested on x86_64-linux.

PR tui/30337
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=30337
---
 gdb/utils.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

(limited to 'gdb/utils.c')

diff --git a/gdb/utils.c b/gdb/utils.c
index 002a588..e10198a 100644
--- a/gdb/utils.c
+++ b/gdb/utils.c
@@ -1116,6 +1116,14 @@ static bool filter_initialized = false;
 
 
 
+/* See readline's rlprivate.h.  */
+
+EXTERN_C int _rl_term_autowrap;
+
+/* See utils.h.  */
+
+int readline_hidden_cols = 0;
+
 /* Initialize the number of lines per page and chars per line.  */
 
 void
@@ -1144,6 +1152,19 @@ init_page_info (void)
 
       /* Get the screen size from Readline.  */
       rl_get_screen_size (&rows, &cols);
+
+      /* Readline:
+	 - ignores the COLUMNS variable when detecting screen width
+	   (because rl_prefer_env_winsize defaults to 0)
+	 - puts the detected screen width in the COLUMNS variable
+	   (because rl_change_environment defaults to 1)
+	 - may report one less than the detected screen width in
+	   rl_get_screen_size (when _rl_term_autowrap == 0).
+	 We could set readline_hidden_cols by comparing COLUMNS to cols as
+	 returned by rl_get_screen_size, but instead simply use
+	 _rl_term_autowrap.  */
+      readline_hidden_cols = _rl_term_autowrap ? 0 : 1;
+
       lines_per_page = rows;
       chars_per_line = cols;
 
-- 
cgit v1.1