openmp: Fix up strtoul and strtoull uses in libgomp

Yesterday when working on numa_domains, I've noticed because of a bug
in my patch a hang on a large NUMA machine.  I've fixed the bug, but
also discovered that the hang was a result of making wrong assumptions
about strtoul/strtoull.  All the uses were for portability setting
errno = 0 before the calls and treating non-zero errno after the call
as invalid input, but for the case where there are no valid digits at
all strtoul may set errno to EINVAL, but doesn't have to and with
glibc doesn't do that.  So, this patch goes through all the strtoul calls
and next to errno != 0 checks adds also endptr == startptr check.
Haven't done it in places where we immediately reject strtoul returning 0
the same as we reject errno != 0, because strtoul must return 0 in the
case where it sets endptr to the start pointer.  In some spots the code
was using errno = 0; x = strtoul (p, &p, 10); if (errno) { /*invalid*/ }
and those spots had to be changed to
errno = 0; x = strtoul (p, &end, 10); if (errno || end == p) { /*invalid*/ }
p = end;

2021-10-15  Jakub Jelinek  <jakub@redhat.com>

	* env.c (parse_schedule): For strtoul or strtoull calls which don't
	clearly reject return value 0 as invalid handle the case where end
	pointer is the same as first argument as invalid.
	(parse_unsigned_long_1): Likewise.
	(parse_one_place): Likewise.
	(parse_places_var): Likewise.
	(parse_stacksize): Likewise.
	(parse_spincount): Likewise.
	(parse_affinity): Likewise.
	(parse_gomp_openacc_dim): Likewise.  Avoid strict aliasing violation.
	Make code valid C89.
	* config/linux/affinity.c (gomp_affinity_find_last_cache_level):
	For strtoul calls which don't clearly reject return value 0 as
	invalid handle the case where end pointer is the same as first
	argument as invalid.
	(gomp_affinity_init_level_1): Likewise.
	(gomp_affinity_init_numa_domains): Likewise.
	* config/rtems/proc.c (parse_thread_pools): Likewise.

1 files changed, 7 insertions, 4 deletions

diff --git a/libgomp/config/rtems/proc.c b/libgomp/config/rtems/proc.c
index 97b9e2f..ad40de4 100644
--- a/libgomp/config/rtems/proc.c
+++ b/libgomp/config/rtems/proc.c
@@ -78,22 +78,25 @@ parse_thread_pools (char *env, unsigned long *count, unsigned long *priority,
 {
   size_t len;
   int i;
+  char *end;
 
   if (*env == ':')
     ++env;
 
   errno = 0;
-  *count = strtoul (env, &env, 10);
-  if (errno != 0)
+  *count = strtoul (env, &end, 10);
+  if (errno != 0 || end == env)
     gomp_fatal ("Invalid thread pool count");
+  env = end;
 
   if (*env == '$')
     {
       ++env;
       errno = 0;
-      *priority = strtoul (env, &env, 10);
-      if (errno != 0)
+      *priority = strtoul (env, &end, 10);
+      if (errno != 0 || end == env)
 	gomp_fatal ("Invalid thread pool priority");
+      env = end;
     }
   else
     *priority = -1;