PR python/17372 - Python hangs when displaying help()

This is more of a readline/terminal issue than a Python one.

PR17372 is a regression in 7.8 caused by the fix for PR17072:

 commit 0017922d0292d8c374584f6100874580659c9973
 Author: Pedro Alves <palves@redhat.com>
 Date:   Mon Jul 14 19:55:32 2014 +0100

    Background execution + pagination aborts readline/gdb

    gdb_readline_wrapper_line removes the handler after a line is
    processed.  Usually, we'll end up re-displaying the prompt, and that
    reinstalls the handler.  But if the output is coming out of handling
    a stop event, we don't re-display the prompt, and nothing restores the
    handler.  So the next input wakes up the event loop and calls into
    readline, which aborts.
...
    gdb/
    2014-07-14  Pedro Alves  <palves@redhat.com>

        PR gdb/17072
        * top.c (gdb_readline_wrapper_line): Tweak comment.
        (gdb_readline_wrapper_cleanup): If readline is enabled, reinstall
        the input handler callback.

The problem is that installing the input handler callback also preps
the terminal, putting it in raw mode and with echo disabled, which is
bad if we're going to call a command that assumes cooked/canonical
mode, and echo enabled, like in the case of the PR, Python's
interactive shell.  Another example I came up with that doesn't depend
on Python is starting a subshell with "(gdb) shell /bin/sh" from a
multi-line command.  Tests covering both these examples are added.

The fix is to revert the original fix for PR gdb/17072, and instead
restore the callback handler after processing an asynchronous target
event.

Furthermore, calling rl_callback_handler_install when we already have
some input in readline's line buffer discards that input, which is
obviously a bad thing to do while the user is typing.  No specific
test is added for that, because I first tried calling it even if the
callback handler was still installed and that resulted in hundreds of
failures in the testsuite.

gdb/
2014-10-29  Pedro Alves  <palves@redhat.com>

	PR python/17372
	* event-top.c (change_line_handler): Call
	gdb_rl_callback_handler_remove instead of
	rl_callback_handler_remove.
	(callback_handler_installed): New global.
	(gdb_rl_callback_handler_remove, gdb_rl_callback_handler_install)
	(gdb_rl_callback_handler_reinstall): New functions.
	(display_gdb_prompt): Call gdb_rl_callback_handler_remove and
	gdb_rl_callback_handler_install instead of
	rl_callback_handler_remove and rl_callback_handler_install.
	(gdb_disable_readline): Call gdb_rl_callback_handler_remove
	instead of rl_callback_handler_remove.
	* event-top.h (gdb_rl_callback_handler_remove)
	(gdb_rl_callback_handler_install)
	(gdb_rl_callback_handler_reinstall): New declarations.
	* infrun.c (reinstall_readline_callback_handler_cleanup): New
	cleanup function.
	(fetch_inferior_event): Install it.
	* top.c (gdb_readline_wrapper_line) Call
	gdb_rl_callback_handler_remove instead of
	rl_callback_handler_remove.
	(gdb_readline_wrapper_cleanup): Don't call
	rl_callback_handler_install.

gdb/testsuite/
2014-10-29  Pedro Alves  <palves@redhat.com>

	PR python/17372
	* gdb.python/python.exp: Test a multi-line command that spawns
	interactive Python.
	* gdb.base/multi-line-starts-subshell.exp: New file.

8 files changed, 204 insertions, 11 deletions

diff --git a/gdb/ChangeLog b/gdb/ChangeLog
index 58a7e98..0aa99c6 100644
--- a/gdb/ChangeLog
+++ b/gdb/ChangeLog
@@ -1,5 +1,31 @@
 2014-10-29  Pedro Alves  <palves@redhat.com>
 
+	PR python/17372
+	* event-top.c (change_line_handler): Call
+	gdb_rl_callback_handler_remove instead of
+	rl_callback_handler_remove.
+	(callback_handler_installed): New global.
+	(gdb_rl_callback_handler_remove, gdb_rl_callback_handler_install)
+	(gdb_rl_callback_handler_reinstall): New functions.
+	(display_gdb_prompt): Call gdb_rl_callback_handler_remove and
+	gdb_rl_callback_handler_install instead of
+	rl_callback_handler_remove and rl_callback_handler_install.
+	(gdb_disable_readline): Call gdb_rl_callback_handler_remove
+	instead of rl_callback_handler_remove.
+	* event-top.h (gdb_rl_callback_handler_remove)
+	(gdb_rl_callback_handler_install)
+	(gdb_rl_callback_handler_reinstall): New declarations.
+	* infrun.c (reinstall_readline_callback_handler_cleanup): New
+	cleanup function.
+	(fetch_inferior_event): Install it.
+	* top.c (gdb_readline_wrapper_line) Call
+	gdb_rl_callback_handler_remove instead of
+	rl_callback_handler_remove.
+	(gdb_readline_wrapper_cleanup): Don't call
+	rl_callback_handler_install.
+
+2014-10-29  Pedro Alves  <palves@redhat.com>
+
 	* event-top.c (command_line_handler): Clear the first byte of
 	linebuffer, when it is first allocated.
 
diff --git a/gdb/event-top.c b/gdb/event-top.c
index f539733..282c0fe 100644
--- a/gdb/event-top.c
+++ b/gdb/event-top.c
@@ -205,7 +205,7 @@ change_line_handler (void)
   else
     {
       /* Turn off editing by using gdb_readline2.  */
-      rl_callback_handler_remove ();
+      gdb_rl_callback_handler_remove ();
       call_readline = gdb_readline2;
 
       /* Set up the command handler as well, in case we are called as
@@ -214,6 +214,57 @@ change_line_handler (void)
     }
 }
 
+/* The functions below are wrappers for rl_callback_handler_remove and
+   rl_callback_handler_install that keep track of whether the callback
+   handler is installed in readline.  This is necessary because after
+   handling a target event of a background execution command, we may
+   need to reinstall the callback handler if it was removed due to a
+   secondary prompt.  See gdb_readline_wrapper_line.  We don't
+   unconditionally install the handler for every target event because
+   that also clears the line buffer, thus installing it while the user
+   is typing would lose input.  */
+
+/* Whether we've registered a callback handler with readline.  */
+static int callback_handler_installed;
+
+/* See event-top.h, and above.  */
+
+void
+gdb_rl_callback_handler_remove (void)
+{
+  rl_callback_handler_remove ();
+  callback_handler_installed = 0;
+}
+
+/* See event-top.h, and above.  Note this wrapper doesn't have an
+   actual callback parameter because we always install
+   INPUT_HANDLER.  */
+
+void
+gdb_rl_callback_handler_install (const char *prompt)
+{
+  /* Calling rl_callback_handler_install resets readline's input
+     buffer.  Calling this when we were already processing input
+     therefore loses input.  */
+  gdb_assert (!callback_handler_installed);
+
+  rl_callback_handler_install (prompt, input_handler);
+  callback_handler_installed = 1;
+}
+
+/* See event-top.h, and above.  */
+
+void
+gdb_rl_callback_handler_reinstall (void)
+{
+  if (!callback_handler_installed)
+    {
+      /* Passing NULL as prompt argument tells readline to not display
+	 a prompt.  */
+      gdb_rl_callback_handler_install (NULL);
+    }
+}
+
 /* Displays the prompt.  If the argument NEW_PROMPT is NULL, the
    prompt that is displayed is the current top level prompt.
    Otherwise, it displays whatever NEW_PROMPT is as a local/secondary
@@ -267,7 +318,7 @@ display_gdb_prompt (char *new_prompt)
 	     the above two functions.  Calling
 	     rl_callback_handler_remove(), does the job.  */
 
-	  rl_callback_handler_remove ();
+	  gdb_rl_callback_handler_remove ();
 	  do_cleanups (old_chain);
 	  return;
 	}
@@ -282,8 +333,8 @@ display_gdb_prompt (char *new_prompt)
 
   if (async_command_editing_p)
     {
-      rl_callback_handler_remove ();
-      rl_callback_handler_install (actual_gdb_prompt, input_handler);
+      gdb_rl_callback_handler_remove ();
+      gdb_rl_callback_handler_install (actual_gdb_prompt);
     }
   /* new_prompt at this point can be the top of the stack or the one
      passed in.  It can't be NULL.  */
@@ -1040,6 +1091,6 @@ gdb_disable_readline (void)
   gdb_stdtargerr = NULL;
 #endif
 
-  rl_callback_handler_remove ();
+  gdb_rl_callback_handler_remove ();
   delete_file_handler (input_fd);
 }
diff --git a/gdb/event-top.h b/gdb/event-top.h
index 2d05d45..ac0d47b 100644
--- a/gdb/event-top.h
+++ b/gdb/event-top.h
@@ -62,6 +62,17 @@ extern void (*input_handler) (char *);
 extern int input_fd;
 extern void (*after_char_processing_hook) (void);
 
+/* Wrappers for rl_callback_handler_remove and
+   rl_callback_handler_install that keep track of whether the callback
+   handler is installed in readline.  Do not call the readline
+   versions directly.  */
+extern void gdb_rl_callback_handler_remove (void);
+extern void gdb_rl_callback_handler_install (const char *prompt);
+
+/* Reinstall the readline callback handler (with no prompt), if not
+   currently installed.  */
+extern void gdb_rl_callback_handler_reinstall (void);
+
 extern void cli_command_loop (void *);
 
 #endif
diff --git a/gdb/infrun.c b/gdb/infrun.c
index 7af5e0f..e25f392 100644
--- a/gdb/infrun.c
+++ b/gdb/infrun.c
@@ -3107,6 +3107,23 @@ wait_for_inferior (void)
   do_cleanups (old_cleanups);
 }
 
+/* Cleanup that reinstalls the readline callback handler, if the
+   target is running in the background.  If while handling the target
+   event something triggered a secondary prompt, like e.g., a
+   pagination prompt, we'll have removed the callback handler (see
+   gdb_readline_wrapper_line).  Need to do this as we go back to the
+   event loop, ready to process further input.  Note this has no
+   effect if the handler hasn't actually been removed, because calling
+   rl_callback_handler_install resets the line buffer, thus losing
+   input.  */
+
+static void
+reinstall_readline_callback_handler_cleanup (void *arg)
+{
+  if (async_command_editing_p && !sync_execution)
+    gdb_rl_callback_handler_reinstall ();
+}
+
 /* Asynchronous version of wait_for_inferior.  It is called by the
    event loop whenever a change of state is detected on the file
    descriptor corresponding to the target.  It can be called more than
@@ -3129,6 +3146,9 @@ fetch_inferior_event (void *client_data)
 
   memset (ecs, 0, sizeof (*ecs));
 
+  /* End up with readline processing input, if necessary.  */
+  make_cleanup (reinstall_readline_callback_handler_cleanup, NULL);
+
   /* We're handling a live event, so make sure we're doing live
      debugging.  If we're looking at traceframes while the target is
      running, we're going to need to get back to that mode after
diff --git a/gdb/testsuite/ChangeLog b/gdb/testsuite/ChangeLog
index 3c51ba1..367187a 100644
--- a/gdb/testsuite/ChangeLog
+++ b/gdb/testsuite/ChangeLog
@@ -1,3 +1,10 @@
+2014-10-29  Pedro Alves  <palves@redhat.com>
+
+	PR python/17372
+	* gdb.python/python.exp: Test a multi-line command that spawns
+	interactive Python.
+	* gdb.base/multi-line-starts-subshell.exp: New file.
+
 2014-10-29  Yao Qi  <yao@codesourcery.com>
 
 	* gdb.base/fileio.exp: Make directories on host.
diff --git a/gdb/testsuite/gdb.base/multi-line-starts-subshell.exp b/gdb/testsuite/gdb.base/multi-line-starts-subshell.exp
new file mode 100644
index 0000000..031883f
--- /dev/null
+++ b/gdb/testsuite/gdb.base/multi-line-starts-subshell.exp
@@ -0,0 +1,55 @@
+# Copyright (C) 2014 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Test spawning a subshell from a multi-line command, where the last
+# prompt displayed is a secondary prompt.  GDB used to have a bug
+# where in this case the subshell would start with the wrong terminal
+# settings.  See PR python/17372.
+
+gdb_exit
+gdb_start
+
+# The subshell's prompt.
+set shell_prompt "gdb-subshell$ "
+set shell_prompt_re [string_to_regexp $shell_prompt]
+
+# Point HOME nowhere to avoid the shell sourcing the user's init
+# scripts.
+set shell_cmd "HOME=/dev/null PS1=\"$shell_prompt\" /bin/sh"
+
+set test "spawn subshell from multi-line"
+gdb_test_multiple "if 1\nshell ${shell_cmd}\nend" $test {
+    -re "$shell_prompt_re$" {
+	pass $test
+
+	# Now check that shell input works and that echo is enabled.
+	set test "shell input works"
+	send_gdb "echo foo\n"
+	gdb_expect {
+	    -re "^echo foo\r\nfoo\r\n$shell_prompt_re$" {
+		pass $test
+	    }
+	    timeout {
+		fail "$test (timeout)"
+	    }
+	}
+
+	# Exit the subshell, back to GDB
+	gdb_test "exit" ".*" "exit shell"
+    }
+}
+
+# Check that we restore input in GDB correctly.
+gdb_test "print 1" "^print 1\r\n.. = 1" "gdb input still works"
diff --git a/gdb/testsuite/gdb.python/python.exp b/gdb/testsuite/gdb.python/python.exp
index 3df9347..0e77256 100644
--- a/gdb/testsuite/gdb.python/python.exp
+++ b/gdb/testsuite/gdb.python/python.exp
@@ -61,6 +61,22 @@ gdb_py_test_multiple "multi-line python command" \
   "print (23)" "" \
   "end" "23"
 
+# Spawn interactive Python help from a multi-line command, thus, after
+# a secondary prompt.
+
+with_test_prefix "python interactive help" {
+    set test "python; help(); end"
+    gdb_test_multiple "python\nhelp()\nend" $test {
+	-re "online help utility.*help> $" {
+	    pass $test
+
+	    # The "quit" must be seen on the output.  A buggy GDB
+	    # would not display it.
+	    gdb_test "quit" "^quit.*leaving help.*" "quit help"
+	}
+    }
+}
+
 gdb_py_test_multiple "show python command" \
   "define zzq" "Type commands for definition of .* just \"end\"\\.*" \
   "python" "" \
diff --git a/gdb/top.c b/gdb/top.c
index 5a947d7..83d858a 100644
--- a/gdb/top.c
+++ b/gdb/top.c
@@ -763,9 +763,14 @@ gdb_readline_wrapper_line (char *line)
 
   /* Prevent parts of the prompt from being redisplayed if annotations
      are enabled, and readline's state getting out of sync.  We'll
-     restore it in gdb_readline_wrapper_cleanup.  */
+     reinstall the callback handler, which puts the terminal in raw
+     mode (or in readline lingo, in prepped state), when we're next
+     ready to process user input, either in display_gdb_prompt, or if
+     we're handling an asynchronous target event and running in the
+     background, just before returning to the event loop to process
+     further input (or more target events).  */
   if (async_command_editing_p)
-    rl_callback_handler_remove ();
+    gdb_rl_callback_handler_remove ();
 }
 
 struct gdb_readline_wrapper_cleanup
@@ -787,10 +792,12 @@ gdb_readline_wrapper_cleanup (void *arg)
   gdb_assert (input_handler == gdb_readline_wrapper_line);
   input_handler = cleanup->handler_orig;
 
-  /* Reinstall INPUT_HANDLER in readline, without displaying a
-     prompt.  */
-  if (async_command_editing_p)
-    rl_callback_handler_install (NULL, input_handler);
+  /* Don't restore our input handler in readline yet.  That would make
+     readline prep the terminal (putting it in raw mode), while the
+     line we just read may trigger execution of a command that expects
+     the terminal in the default cooked/canonical mode, such as e.g.,
+     running Python's interactive online help utility.  See
+     gdb_readline_wrapper_line for when we'll reinstall it.  */
 
   gdb_readline_wrapper_result = NULL;
   gdb_readline_wrapper_done = 0;