From 2b5b9d09ed3260ded86629aacc83d6b91bb7d4bf Mon Sep 17 00:00:00 2001 From: Joel Brobecker Date: Mon, 24 Oct 2011 16:51:36 +0000 Subject: [powerpc] crash trying to allocate memory in inferior Our testsuite noticed a crash when trying to call a function which requires GDB to allocate memory in the inferior. Typically, this happens when one of the parameters is a string. For instance, our testcase tries: (gdb) call debug.trace (me, "You") [1] 32737 segmentation fault /path/to/gdb What happens is that GDB sees the string, and thus tries to allocate memory for it in the inferior: > /* Allocate NBYTES of space in the inferior using the inferior's > malloc and return a value that is a pointer to the allocated > space. */ > > struct value * > value_allocate_space_in_inferior (int len) > { > struct objfile *objf; > struct value *val = find_function_in_inferior ("malloc", &objf); And find_function_in_inferior first searches the symtab in case we have debug info. But, in our case (bareboard powerpc), we don't, so it gets "malloc"'s address from the minimal symbols, and builds a value whose type is a TYPE_CODE_PTR, not a TYPE_CODE_FUNC. As a result, when we later try to make the call to malloc, we end up inside the powerpc tdep code that has: > do_ppc_sysv_return_value (struct gdbarch *gdbarch, struct type *func_type, [...] > if (func_type > && TYPE_CALLING_CONVENTION (func_type) == DW_CC_GDB_IBM_OpenCL) The problem is that func_type is not a TYPE_CODE_FUNC, and thus the type-specific kind is not TYPE_SPECIFIC_FUNC, and so we do TYPE_CALLING_CONVENTION is an invalid access. Interestingly, the other call to TYPE_CALLING_CONVENTION is correctly preceded by a check of the type's TYPE_CODE (making sure that it is TYPE_CODE_FUNC). gdb/ChangeLog: * ppc-sysv-tdep.c (do_ppc_sysv_return_value): Do not check FUNC_TYPE's calling convention if FUNC_TYPE is not a function. --- gdb/ChangeLog | 5 +++++ gdb/ppc-sysv-tdep.c | 1 + 2 files changed, 6 insertions(+) diff --git a/gdb/ChangeLog b/gdb/ChangeLog index 30cf144..f5cdd45 100644 --- a/gdb/ChangeLog +++ b/gdb/ChangeLog @@ -1,3 +1,8 @@ +2011-10-24 Joel Brobecker + + * ppc-sysv-tdep.c (do_ppc_sysv_return_value): Do not check + FUNC_TYPE's calling convention if FUNC_TYPE is not a function. + 2011-10-24 Pedro Alves * linux-nat.c (linux_handle_extended_wait): When handling a clone diff --git a/gdb/ppc-sysv-tdep.c b/gdb/ppc-sysv-tdep.c index e431363..bda4544 100644 --- a/gdb/ppc-sysv-tdep.c +++ b/gdb/ppc-sysv-tdep.c @@ -692,6 +692,7 @@ do_ppc_sysv_return_value (struct gdbarch *gdbarch, struct type *func_type, int opencl_abi = 0; if (func_type + && TYPE_CODE (func_type) == TYPE_CODE_FUNC && TYPE_CALLING_CONVENTION (func_type) == DW_CC_GDB_IBM_OpenCL) opencl_abi = 1; -- cgit v1.1