options: add gcc/regenerate-opt-urls.py

In r14-5118-gc5db4d8ba5f3de I added a mechanism to automatically add
URLs to quoted strings in diagnostics.  This was based on a data table
mapping strings to URLs, with placeholder data covering various pragmas
and a couple of options.

The following patches add automatic URLification in our diagnostic
messages to mentions of *all* of our options in quoted strings, linking
to our HTML documentation.

For example, with these patches, given:

  ./xgcc -B. -S t.c -Wctad-maybe-unsupported
  cc1: warning: command-line option ‘-Wctad-maybe-unsupported’ is valid for C++/ObjC++ but not for C

the quoted string '-Wctad-maybe-unsupported' gets automatically URLified
in a sufficiently modern terminal to:
  https://gcc.gnu.org/onlinedocs/gcc/C_002b_002b-Dialect-Options.html#index-Wctad-maybe-unsupported

Objectives:
- integrate with DOCUMENTATION_ROOT_URL
- integrate with the existing .opt mechanisms
- automate keeping the URLs up-to-date
- work with target-specific options based on current configuration
- work with lang-specific options based on current configuration
- keep autogenerated material separate from the human-maintained .opt
  files
- no new build-time requirements (by using awk at build time)
- be maintainable

The approach is a new regenerate-opt-urls.py which:
- scrapes the generated HTML documentation finding anchors
  for options,
- reads all the .opt files in the source tree
- for each .opt file, generates a .opt.urls file; for each
  option in the .opt file it has either a UrlSuffix directives giving
  the final part of the URL of that option's documentation (relative
  to DOCUMENTATION_ROOT_URL), or a comment describing the problem.

regenerate-opt-urls.py is written in Python 3, and has unit tests.
I tested it with Python 3.8, and it probably works with earlier
releases of Python 3.
The .opt.urls files it generates become part of the source tree, and
would be regenerated by maintainers whenever new options are added.
Forgetting to update the files (or not having Python 3 handy) merely
means that URLs might be missing or out of date until someone else
regenerates them.

At build time, the .opt.urls are added to .opt files when regenerating
the optionslist file.  A new "options-urls-cc-gen.awk" is run at build
time on the optionslist to generate a "options-urls.cc" file, and this
is then used by the gcc_urlifier class when emitting diagnostics.

Changed in v5:
- removed commented-out code

Changed in v4:
- added PER_LANGUAGE_OPTION_INDEXES
- added info to sourcebuild.texi on adding a new front end
- removed TODOs and out-of-date comment

Changed in v3:
- Makefile.in: added OPT_URLS_HTML_DEPS and a comment

Changed in v2:
- added convenience targets to Makefile for regenerating the .opt.urls
  files, and for running unit tests for the generation code
- parse gdc and gfortran documentation, and create LangUrlSuffix_{lang}
directives for language-specific URLs.
- add documentation to sourcebuild.texi

gcc/ChangeLog:
	* Makefile.in (OPT_URLS_HTML_DEPS): New.
	(regenerate-opt-urls): New target.
	(regenerate-opt-urls-unit-test): New target.
	* doc/options.texi (Option properties): Add UrlSuffix and
	description of regenerate-opt-urls.py.  Add LangUrlSuffix_*.
	* doc/sourcebuild.texi (Anatomy of a Language Front End): Add
	reference to regenerate-opt-urls.py's PER_LANGUAGE_OPTION_INDEXES
	and Makefile.in's OPT_URLS_HTML_DEPS.
	(Anatomy of a Target Back End): Add
	reference to regenerate-opt-urls.py's TARGET_SPECIFIC_PAGES.
	* regenerate-opt-urls.py: New file.

Signed-off-by: David Malcolm <dmalcolm@redhat.com>

2 files changed, 35 insertions, 0 deletions

diff --git a/gcc/doc/options.texi b/gcc/doc/options.texi
index ca0e96c..01f3acf 100644
--- a/gcc/doc/options.texi
+++ b/gcc/doc/options.texi
@@ -597,4 +597,30 @@ This warning option corresponds to @code{cpplib.h} warning reason code
 @var{CPP_W_Enum}.  This should only be used for warning options of the
 C-family front-ends.
 
+@item UrlSuffix(@var{url_suffix})
+Adjacent to each human-written @code{.opt} file in the source tree is
+a corresponding file with a @code{.opt.urls} extension.  These files
+contain @code{UrlSuffix} directives giving the ending part of the URL
+for the documentation of the option, such as:
+
+@smallexample
+Wabi-tag
+UrlSuffix(gcc/C_002b_002b-Dialect-Options.html#index-Wabi-tag)
+@end smallexample
+
+These URL suffixes are relative to @code{DOCUMENTATION_ROOT_URL}.
+
+There files are generated from the @code{.opt} files and the generated
+HTML documentation by @code{regenerate-opt-urls.py}, and should be
+regenerated when adding new options, via manually invoking
+@code{make regenerate-opt-urls}.
+
+@item LangUrlSuffix_@var{lang}(@var{url_suffix})
+In addition to @code{UrlSuffix} directives, @code{regenerate-opt-urls.py}
+can generate language-specific URLs, such as:
+
+@smallexample
+LangUrlSuffix_D(gdc/Code-Generation.html#index-MMD)
+@end smallexample
+
 @end table
diff --git a/gcc/doc/sourcebuild.texi b/gcc/doc/sourcebuild.texi
index 152b0397..3a394e7 100644
--- a/gcc/doc/sourcebuild.texi
+++ b/gcc/doc/sourcebuild.texi
@@ -585,6 +585,11 @@ The front end's manuals should be mentioned in
 and the online manuals should be linked to from
 @file{onlinedocs/index.html}.
 @item
+If the front end has its own manual with its own index of options, the
+generated @file{Option-Index.html} should be added to
+@code{PER_LANGUAGE_OPTION_INDEXES} in @file{gcc/regenerate-opt-urls.py}
+and to @code{OPT_URLS_HTML_DEPS} in @file{gcc/Makefile.in}.
+@item
 Any old releases or CVS repositories of the front end, before its
 inclusion in GCC, should be made available on the GCC web site at
 @uref{https://gcc.gnu.org/pub/gcc/old-releases/}.
@@ -813,6 +818,10 @@ options supported by this target (@pxref{Run-time Target, , Run-time
 Target Specification}).  This means both entries in the summary table
 of options and details of the individual options.
 @item
+An entry in @file{gcc/regenerate-opt-urls.py}'s TARGET_SPECIFIC_PAGES
+dictionary mapping from target-specific HTML documentation pages
+to the target specific source directory.
+@item
 Documentation in @file{gcc/doc/extend.texi} for any target-specific
 attributes supported (@pxref{Target Attributes, , Defining
 target-specific uses of @code{__attribute__}}), including where the