Update OpenACC data clause semantics to the 2.5 behavior

	gcc/c-family/
	* c-pragma.h (enum pragma_omp_clause): Add
	PRAGMA_OACC_CLAUSE_{FINALIZE,IF_PRESENT}. Remove
	PRAGMA_OACC_CLAUSE_PRESENT_OR_{COPY,COPYIN,COPYOUT,CREATE}.

	gcc/c/
	* c-parser.c (c_parser_omp_clause_name): Add support for finalize
	and if_present. Make present_or_{copy,copyin,copyout,create} aliases
	to their non-present_or_* counterparts. Make 'self' an alias to
	PRAGMA_OACC_CLAUSE_HOST.
	(c_parser_oacc_data_clause): Update GOMP mappings for
	PRAGMA_OACC_CLAUSE_{COPY,COPYIN,COPYOUT,CREATE,DELETE}. Remove
	PRAGMA_OACC_CLAUSE_{SELF,PRESENT_OR_*}.
	(c_parser_oacc_all_clauses): Handle finalize and if_present clauses.
	Remove support for present_or_* clauses.
	(OACC_KERNELS_CLAUSE_MASK): Remove PRESENT_OR_* clauses.
	(OACC_PARALLEL_CLAUSE_MASK): Likewise.
	(OACC_DECLARE_CLAUSE_MASK): Likewise.
	(OACC_DATA_CLAUSE_MASK): Likewise.
	(OACC_ENTER_DATA_CLAUSE_MASK): Remove PRESENT_OR_* clauses.
	(OACC_EXIT_DATA_CLAUSE_MASK): Add FINALIZE clause.
	(OACC_UPDATE_CLAUSE_MASK): Remove SELF, add IF_PRESENT.
	(c_parser_oacc_declare): Remove PRESENT_OR_* clauses.
	* c-typeck.c (c_finish_omp_clauses): Handle IF_PRESENT and FINALIZE.

	gcc/cp/
	* parser.c (cp_parser_omp_clause_name): Add support for finalize
	and if_present. Make present_or_{copy,copyin,copyout,create} aliases
	to their non-present_or_* counterparts. Make 'self' an alias to
	PRAGMA_OACC_CLAUSE_HOST.
	(cp_parser_oacc_data_clause): Update GOMP mappings for
	PRAGMA_OACC_CLAUSE_{COPY,COPYIN,COPYOUT,CREATE,DELETE}. Remove
	PRAGMA_OACC_CLAUSE_{SELF,PRESENT_OR_*}.
	(cp_parser_oacc_all_clauses): Handle finalize and if_present clauses.
	Remove support for present_or_* clauses.
	(OACC_KERNELS_CLAUSE_MASK): Remove PRESENT_OR_* clauses.
	(OACC_PARALLEL_CLAUSE_MASK): Likewise.
	(OACC_DECLARE_CLAUSE_MASK): Likewise.
	(OACC_DATA_CLAUSE_MASK): Likewise.
	(OACC_ENTER_DATA_CLAUSE_MASK): Remove PRESENT_OR_* clauses.
	(OACC_EXIT_DATA_CLAUSE_MASK): Add FINALIZE clause.
	(OACC_UPDATE_CLAUSE_MASK): Remove SELF, add IF_PRESENT.
	(cp_parser_oacc_declare): Remove PRESENT_OR_* clauses.
	* pt.c (tsubst_omp_clauses): Handle IF_PRESENT and FINALIZE.
	* semantics.c (finish_omp_clauses): Handle IF_PRESENT and FINALIZE.

	gcc/fortran/
	* gfortran.h (gfc_omp_clauses): Add unsigned if_present, finalize
	bitfields.
	* openmp.c (enum omp_mask2): Remove OMP_CLAUSE_PRESENT_OR_*. Add
	OMP_CLAUSE_{IF_PRESENT,FINALIZE}.
	(gfc_match_omp_clauses): Update handling of copy, copyin, copyout,
	create, deviceptr, present_of_*. Add support for finalize and
	if_present.
	(OACC_PARALLEL_CLAUSES): Remove PRESENT_OR_* clauses.
	(OACC_KERNELS_CLAUSES): Likewise.
	(OACC_DATA_CLAUSES): Likewise.
	(OACC_DECLARE_CLAUSES): Likewise.
	(OACC_UPDATE_CLAUSES): Add IF_PRESENT clause.
	(OACC_ENTER_DATA_CLAUSES): Remove PRESENT_OR_* clauses.
	(OACC_EXIT_DATA_CLAUSES): Add FINALIZE clause.
	(gfc_match_oacc_declare): Update to OpenACC 2.5 semantics.
	* trans-openmp.c (gfc_trans_omp_clauses): Add support for IF_PRESENT
	and FINALIZE.

	gcc/
	* gimplify.c (gimplify_scan_omp_clauses): Add support for
	OMP_CLAUSE_{IF_PRESENT,FINALIZE}.
	(gimplify_adjust_omp_clauses): Likewise.
	(gimplify_oacc_declare_1): Add support for GOMP_MAP_RELEASE, remove
	support for GOMP_MAP_FORCE_{ALLOC,TO,FROM,TOFROM}.
	(gimplify_omp_target_update): Update handling of acc update and
	enter/exit data.
	* omp-low.c (install_var_field): Remove unused parameter
	base_pointers_restrict.
	(scan_sharing_clauses): Remove base_pointers_restrict parameter.
	Update call to install_var_field. Handle OMP_CLAUSE_{IF_PRESENT,
	FINALIZE}
	(omp_target_base_pointers_restrict_p): Delete.
	(scan_omp_target): Update call to scan_sharing_clauses.
	* tree-core.h (enum omp_clause_code): Add OMP_CLAUSE_{IF_PRESENT,
	FINALIZE}.
	* tree-nested.c (convert_nonlocal_omp_clauses): Handle
	OMP_CLAUSE_{IF_PRESENT,FINALIZE}.
	(convert_local_omp_clauses): Likewise.
	* tree-pretty-print.c (dump_omp_clause): Likewise.
	* tree.c (omp_clause_num_ops): Add entries for 	OMP_CLAUSE_{IF_PRESENT,
	FINALIZE}.
	(omp_clause_code_name): Likewise.

	gcc/testsuite/
	* c-c++-common/goacc/declare-1.c: Update test case to utilize OpenACC
	2.5 data clause semantics.
	* c-c++-common/goacc/declare-2.c: Likewise.
	* c-c++-common/goacc/default-4.c: Likewise.
	* c-c++-common/goacc/finalize-1.c: New test.
	* c-c++-common/goacc/kernels-alias-2.c: Update test case to utilize
	OpenACC 2.5 data clause semantics.
	* c-c++-common/goacc/kernels-alias.c: Likewise.
	* c-c++-common/goacc/routine-5.c: Likewise.
	* c-c++-common/goacc/update-if_present-1.c: New test.
	* c-c++-common/goacc/update-if_present-2.c: New test.
	* g++.dg/goacc/template.C: Update test case to utilize OpenACC
	2.5 data clause semantics.
	* gfortran.dg/goacc/combined-directives.f90: Likewise.
	* gfortran.dg/goacc/data-tree.f95: Likewise.
	* gfortran.dg/goacc/declare-2.f95: Likewise.
	* gfortran.dg/goacc/default-4.f: Likewise.
	* gfortran.dg/goacc/enter-exit-data.f95: Likewise.
	* gfortran.dg/goacc/finalize-1.f: New test.
	* gfortran.dg/goacc/kernels-alias-2.f95: Update test case to utilize
	OpenACC 2.5 data clause semantics.
	* gfortran.dg/goacc/kernels-alias.f95: Likewise.
	* gfortran.dg/goacc/kernels-tree.f95: Likewise.
	* gfortran.dg/goacc/nested-function-1.f90: Likewise.
	* gfortran.dg/goacc/parallel-tree.f95: Likewise.
	* gfortran.dg/goacc/reduction-promotions.f90: Likewise.
	* gfortran.dg/goacc/update-if_present-1.f90: New test.
	* gfortran.dg/goacc/update-if_present-2.f90: New test.

	libgomp/
	* libgomp.h (struct splay_tree_key_s): Add dynamic_refcount member.
	(gomp_acc_remove_pointer): Update declaration.
	(gomp_acc_declare_allocate): Declare.
	(gomp_remove_var): Declare.
	* libgomp.map (OACC_2.5): Define.
	* oacc-mem.c (acc_map_data): Update refcount.
	(acc_unmap_data): Likewise.
	(present_create_copy): Likewise.
	(acc_create): Add FLAG_PRESENT when calling present_create_copy.
	(acc_copyin): Likewise.
	(FLAG_FINALIZE): Define.
	(delete_copyout): Update dynamic refcounts, add support for FINALIZE.
	(acc_delete_finalize): New function.
	(acc_delete_finalize_async): New function.
	(acc_copyout_finalize): New function.
	(acc_copyout_finalize_async): New function.
	(gomp_acc_insert_pointer): Update refcounts.
	(gomp_acc_remove_pointer): Return if data is not present on the
	accelerator.
	* oacc-parallel.c (find_pset): Rename to find_pointer.
	(find_pointer): Add support for GOMP_MAP_POINTER.
	(handle_ftn_pointers): New function.
	(GOACC_parallel_keyed): Update refcounts of variables.
	(GOACC_enter_exit_data): Add support for finalized data mappings.
	Add support for GOMP_MAP_{TO,ALLOC,RELESE,FROM}. Update handling
	of fortran arrays.
	(GOACC_update): Add support for GOMP_MAP_{ALWAYS_POINTER,TO,FROM}.
	(GOACC_declare): Add support for GOMP_MAP_RELEASE, remove support
	for GOMP_MAP_FORCE_FROM.
	* openacc.f90 (module openacc_internal): Add
	acc_copyout_finalize_{32_h,64_h,array_h,_l}, and
	acc_delete_finalize_{32_h,64_h,array_h,_l}. Add interfaces for
	acc_copyout_finalize and acc_delete_finalize.
	(acc_copyout_finalize_32_h): New subroutine.
	(acc_copyout_finalize_64_h): New subroutine.
	(acc_copyout_finalize_array_h): New subroutine.
	(acc_delete_finalize_32_h): New subroutine.
	(acc_delete_finalize_64_h): New subroutine.
	(acc_delete_finalize_array_h): New subroutine.
	* openacc.h (acc_copyout_finalize): Declare.
	(acc_copyout_finalize_async): Declare.
	(acc_delete_finalize): Declare.
	(acc_delete_finalize_async): Declare.
	* openacc_lib.h (acc_copyout_finalize): New interface.
	(acc_delete_finalize): New interface.
	* target.c (gomp_map_vars): Update dynamic_refcount.
	(gomp_remove_var): New function.
	(gomp_unmap_vars): Use it.
	(gomp_unload_image_from_device): Likewise.
	* testsuite/libgomp.oacc-c-c++-common/data-already-1.c: Update test
	case to utilize OpenACC 2.5 data clause semantics.
	* testsuite/libgomp.oacc-c-c++-common/data-already-2.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/data-already-3.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/data-already-4.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/data-already-5.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/data-already-6.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/data-already-7.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/data-already-8.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/lib-16.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/lib-25.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/lib-32.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/lib-83.c: Likewise.
	* testsuite/libgomp.oacc-fortran/data-5.f90: New test.
	* testsuite/libgomp.oacc-fortran/data-already-1.f: Update test case to
	utilize OpenACC 2.5 data clause semantics.
	* testsuite/libgomp.oacc-fortran/data-already-2.f: Likewise.
	* testsuite/libgomp.oacc-fortran/data-already-3.f: Likewise.
	* testsuite/libgomp.oacc-fortran/data-already-4.f: Likewise.
	* testsuite/libgomp.oacc-fortran/data-already-5.f: Likewise.
	* testsuite/libgomp.oacc-fortran/data-already-6.f: Likewise.
	* testsuite/libgomp.oacc-fortran/data-already-7.f: Likewise.
	* testsuite/libgomp.oacc-fortran/data-already-8.f: Likewise.
	* testsuite/libgomp.oacc-fortran/lib-32-1.f: Likewise.
	* testsuite/libgomp.oacc-fortran/lib-32-2.f: Likewise.

Co-Authored-By: Cesar Philippidis <cesar@codesourcery.com>
Co-Authored-By: Thomas Schwinge <thomas@codesourcery.com>

From-SVN: r261813

1 files changed, 162 insertions, 34 deletions

diff --git a/libgomp/oacc-mem.c b/libgomp/oacc-mem.c
index 158f086..3787ce4 100644
--- a/libgomp/oacc-mem.c
+++ b/libgomp/oacc-mem.c
@@ -347,6 +347,7 @@ acc_map_data (void *h, void *d, size_t s)
 
       tgt = gomp_map_vars (acc_dev, mapnum, &hostaddrs, &devaddrs, &sizes,
 			   &kinds, true, GOMP_MAP_VARS_OPENACC);
+      tgt->list[0].key->refcount = REFCOUNT_INFINITY;
     }
 
   gomp_mutex_lock (&acc_dev->lock);
@@ -389,6 +390,9 @@ acc_unmap_data (void *h)
 		  (void *) n->host_start, (int) host_size, (void *) h);
     }
 
+  /* Mark for removal.  */
+  n->refcount = 1;
+
   t = n->tgt;
 
   if (t->refcount == 2)
@@ -460,6 +464,11 @@ present_create_copy (unsigned f, void *h, size_t s)
 	  gomp_fatal ("[%p,+%d] not mapped", (void *)h, (int)s);
 	}
 
+      if (n->refcount != REFCOUNT_INFINITY)
+	{
+	  n->refcount++;
+	  n->dynamic_refcount++;
+	}
       gomp_mutex_unlock (&acc_dev->lock);
     }
   else if (!(f & FLAG_CREATE))
@@ -483,6 +492,8 @@ present_create_copy (unsigned f, void *h, size_t s)
 
       tgt = gomp_map_vars (acc_dev, mapnum, &hostaddrs, NULL, &s, &kinds, true,
 			   GOMP_MAP_VARS_OPENACC);
+      /* Initialize dynamic refcount.  */
+      tgt->list[0].key->dynamic_refcount = 1;
 
       gomp_mutex_lock (&acc_dev->lock);
 
@@ -499,13 +510,13 @@ present_create_copy (unsigned f, void *h, size_t s)
 void *
 acc_create (void *h, size_t s)
 {
-  return present_create_copy (FLAG_CREATE, h, s);
+  return present_create_copy (FLAG_PRESENT | FLAG_CREATE, h, s);
 }
 
 void *
 acc_copyin (void *h, size_t s)
 {
-  return present_create_copy (FLAG_CREATE | FLAG_COPY, h, s);
+  return present_create_copy (FLAG_PRESENT | FLAG_CREATE | FLAG_COPY, h, s);
 }
 
 void *
@@ -542,7 +553,8 @@ acc_pcopyin (void *h, size_t s)
 }
 #endif
 
-#define FLAG_COPYOUT (1 << 0)
+#define FLAG_COPYOUT  (1 << 0)
+#define FLAG_FINALIZE (1 << 1)
 
 static void
 delete_copyout (unsigned f, void *h, size_t s, const char *libfnname)
@@ -581,15 +593,52 @@ delete_copyout (unsigned f, void *h, size_t s, const char *libfnname)
 		  (void *) n->host_start, (int) host_size, (void *) h, (int) s);
     }
 
-  gomp_mutex_unlock (&acc_dev->lock);
+  if (n->refcount == REFCOUNT_INFINITY)
+    {
+      n->refcount = 0;
+      n->dynamic_refcount = 0;
+    }
+  if (n->refcount < n->dynamic_refcount)
+    {
+      gomp_mutex_unlock (&acc_dev->lock);
+      gomp_fatal ("Dynamic reference counting assert fail\n");
+    }
 
-  if (f & FLAG_COPYOUT)
-    acc_dev->dev2host_func (acc_dev->target_id, h, d, s);
+  if (f & FLAG_FINALIZE)
+    {
+      n->refcount -= n->dynamic_refcount;
+      n->dynamic_refcount = 0;
+    }
+  else if (n->dynamic_refcount)
+    {
+      n->dynamic_refcount--;
+      n->refcount--;
+    }
 
-  acc_unmap_data (h);
+  if (n->refcount == 0)
+    {
+      if (n->tgt->refcount == 2)
+	{
+	  struct target_mem_desc *tp, *t;
+	  for (tp = NULL, t = acc_dev->openacc.data_environ; t != NULL;
+	       tp = t, t = t->prev)
+	    if (n->tgt == t)
+	      {
+		if (tp)
+		  tp->prev = t->prev;
+		else
+		  acc_dev->openacc.data_environ = t->prev;
+		break;
+	      }
+	}
 
-  if (!acc_dev->free_func (acc_dev->target_id, d))
-    gomp_fatal ("error in freeing device memory in %s", libfnname);
+      if (f & FLAG_COPYOUT)
+	acc_dev->dev2host_func (acc_dev->target_id, h, d, s);
+
+      gomp_remove_var (acc_dev, n);
+    }
+
+  gomp_mutex_unlock (&acc_dev->lock);
 }
 
 void
@@ -599,11 +648,35 @@ acc_delete (void *h , size_t s)
 }
 
 void
+acc_delete_finalize (void *h , size_t s)
+{
+  delete_copyout (FLAG_FINALIZE, h, s, __FUNCTION__);
+}
+
+void
+acc_delete_finalize_async (void *h , size_t s, int async)
+{
+  delete_copyout (FLAG_FINALIZE, h, s, __FUNCTION__);
+}
+
+void
 acc_copyout (void *h, size_t s)
 {
   delete_copyout (FLAG_COPYOUT, h, s, __FUNCTION__);
 }
 
+void
+acc_copyout_finalize (void *h, size_t s)
+{
+  delete_copyout (FLAG_COPYOUT | FLAG_FINALIZE, h, s, __FUNCTION__);
+}
+
+void
+acc_copyout_finalize_async (void *h, size_t s, int async)
+{
+  delete_copyout (FLAG_COPYOUT | FLAG_FINALIZE, h, s, __FUNCTION__);
+}
+
 static void
 update_dev_host (int is_dev, void *h, size_t s)
 {
@@ -659,11 +732,37 @@ gomp_acc_insert_pointer (size_t mapnum, void **hostaddrs, size_t *sizes,
   struct goacc_thread *thr = goacc_thread ();
   struct gomp_device_descr *acc_dev = thr->dev;
 
+  if (acc_is_present (*hostaddrs, *sizes))
+    {
+      splay_tree_key n;
+      gomp_mutex_lock (&acc_dev->lock);
+      n = lookup_host (acc_dev, *hostaddrs, *sizes);
+      gomp_mutex_unlock (&acc_dev->lock);
+
+      tgt = n->tgt;
+      for (size_t i = 0; i < tgt->list_count; i++)
+	if (tgt->list[i].key == n)
+	  {
+	    for (size_t j = 0; j < mapnum; j++)
+	      if (i + j < tgt->list_count && tgt->list[i + j].key)
+		{
+		  tgt->list[i + j].key->refcount++;
+		  tgt->list[i + j].key->dynamic_refcount++;
+		}
+	    return;
+	  }
+      /* Should not reach here.  */
+      gomp_fatal ("Dynamic refcount incrementing failed for pointer/pset");
+    }
+
   gomp_debug (0, "  %s: prepare mappings\n", __FUNCTION__);
   tgt = gomp_map_vars (acc_dev, mapnum, hostaddrs,
 		       NULL, sizes, kinds, true, GOMP_MAP_VARS_OPENACC);
   gomp_debug (0, "  %s: mappings prepared\n", __FUNCTION__);
 
+  /* Initialize dynamic refcount.  */
+  tgt->list[0].key->dynamic_refcount = 1;
+
   gomp_mutex_lock (&acc_dev->lock);
   tgt->prev = acc_dev->openacc.data_environ;
   acc_dev->openacc.data_environ = tgt;
@@ -671,7 +770,8 @@ gomp_acc_insert_pointer (size_t mapnum, void **hostaddrs, size_t *sizes,
 }
 
 void
-gomp_acc_remove_pointer (void *h, bool force_copyfrom, int async, int mapnum)
+gomp_acc_remove_pointer (void *h, size_t s, bool force_copyfrom, int async,
+			 int finalize, int mapnum)
 {
   struct goacc_thread *thr = goacc_thread ();
   struct gomp_device_descr *acc_dev = thr->dev;
@@ -679,6 +779,9 @@ gomp_acc_remove_pointer (void *h, bool force_copyfrom, int async, int mapnum)
   struct target_mem_desc *t;
   int minrefs = (mapnum == 1) ? 2 : 3;
 
+  if (!acc_is_present (h, s))
+    return;
+
   gomp_mutex_lock (&acc_dev->lock);
 
   n = lookup_host (acc_dev, h, 1);
@@ -693,40 +796,65 @@ gomp_acc_remove_pointer (void *h, bool force_copyfrom, int async, int mapnum)
 
   t = n->tgt;
 
-  struct target_mem_desc *tp;
+  if (n->refcount < n->dynamic_refcount)
+    {
+      gomp_mutex_unlock (&acc_dev->lock);
+      gomp_fatal ("Dynamic reference counting assert fail\n");
+    }
 
-  if (t->refcount == minrefs)
+  if (finalize)
     {
-      /* This is the last reference, so pull the descriptor off the
-	 chain. This avoids gomp_unmap_vars via gomp_unmap_tgt from
-	 freeing the device memory. */
-      t->tgt_end = 0;
-      t->to_free = 0;
+      n->refcount -= n->dynamic_refcount;
+      n->dynamic_refcount = 0;
+    }
+  else if (n->dynamic_refcount)
+    {
+      n->dynamic_refcount--;
+      n->refcount--;
+    }
 
-      for (tp = NULL, t = acc_dev->openacc.data_environ; t != NULL;
-	   tp = t, t = t->prev)
+  gomp_mutex_unlock (&acc_dev->lock);
+
+  if (n->refcount == 0)
+    {
+      if (t->refcount == minrefs)
 	{
-	  if (n->tgt == t)
+	  /* This is the last reference, so pull the descriptor off the
+	     chain. This prevents gomp_unmap_vars via gomp_unmap_tgt from
+	     freeing the device memory. */
+	  struct target_mem_desc *tp;
+	  for (tp = NULL, t = acc_dev->openacc.data_environ; t != NULL;
+	       tp = t, t = t->prev)
 	    {
-	      if (tp)
-		tp->prev = t->prev;
-	      else
-		acc_dev->openacc.data_environ = t->prev;
-	      break;
+	      if (n->tgt == t)
+		{
+		  if (tp)
+		    tp->prev = t->prev;
+		  else
+		    acc_dev->openacc.data_environ = t->prev;
+		  break;
+		}
 	    }
 	}
-    }
 
-  if (force_copyfrom)
-    t->list[0].copy_from = 1;
+      /* Set refcount to 1 to allow gomp_unmap_vars to unmap it.  */
+      n->refcount = 1;
+      t->refcount = minrefs;
+      for (size_t i = 0; i < t->list_count; i++)
+	if (t->list[i].key == n)
+	  {
+	    t->list[i].copy_from = force_copyfrom ? 1 : 0;
+	    break;
+	  }
 
-  gomp_mutex_unlock (&acc_dev->lock);
+      /* If running synchronously, unmap immediately.  */
+      if (async < acc_async_noval)
+	gomp_unmap_vars (t, true);
+      else
+	t->device_descr->openacc.register_async_cleanup_func (t, async);
+    }
 
-  /* If running synchronously, unmap immediately.  */
-  if (async_synchronous_p (async))
-    gomp_unmap_vars (t, true);
-  else
-    t->device_descr->openacc.register_async_cleanup_func (t, async);
+  gomp_mutex_unlock (&acc_dev->lock);
 
   gomp_debug (0, "  %s: mappings restored\n", __FUNCTION__);
 }