|
This patch adds support for embeddding profiling information about the
compiler itself into the SARIF output.
Specifically, if SARIF diagnostic output is requested, via
-fdiagnostics-format=sarif-file or -fdiagnostics-format=sarif-stderr,
then any -ftime-report output is written in JSON form into the SARIF
output, rather than to stderr.
In earlier versions of this patch I extended -ftime-report so that
*as well* as writing to stderr, it would embed the information in any
SARIF output. This turned out to be awkward to use, in that I found
myself needing to get the data in JSON form without also having it
emitted on stderr (which was fouling my build scripts).
The timing information is written to the SARIF as a "gcc/timeReport"
property within a property bag of the "invocation" object.
Here's an example of the output:
"invocations": [
{
"executionSuccessful": true,
"toolExecutionNotifications": [],
"properties": {
"gcc/timeReport": {
"timevars": [
{
"name": "phase setup",
"elapsed": {
"user": 0.04,
"sys": 0,
"wall": 0.04,
"ggc_mem": 1863472
}
},
[...snip...]
{
"name": "analyzer: processing worklist",
"elapsed": {
"user": 0.06,
"sys": 0,
"wall": 0.06,
"ggc_mem": 48
}
},
{
"name": "analyzer: emitting diagnostics",
"elapsed": {
"user": 0.01,
"sys": 0,
"wall": 0.01,
"ggc_mem": 0
}
},
{
"name": "TOTAL",
"elapsed": {
"user": 0.21,
"sys": 0.03,
"wall": 0.24,
"ggc_mem": 3368736
}
}
],
"CHECKING_P": true,
"flag_checking": true
}
}
}
]
The documentation notes that the precise output format is subject
to change.
I have successfully used this in my analyzer integration tests to get
timing information about which source files get slowed down by the
analyzer. I've validated the generated .sarif files against the SARIF
schema.
gcc/ChangeLog:
PR analyzer/109361
* diagnostic-client-data-hooks.h (class sarif_object): New forward
decl.
(diagnostic_client_data_hooks::add_sarif_invocation_properties):
New vfunc.
* diagnostic-format-sarif.cc: Include "diagnostic-format-sarif.h".
(class sarif_invocation): Inherit from sarif_object rather than
json::object.
(class sarif_result): Likewise.
(class sarif_ice_notification): Likewise.
(sarif_object::get_or_create_properties): New.
(sarif_invocation::prepare_to_flush): Add "context" param. Use it
to call the context's add_sarif_invocation_properties hook.
(sarif_builder::flush_to_file): Pass m_context to
sarif_invocation::prepare_to_flush.
* diagnostic-format-sarif.h: New header.
* doc/invoke.texi (Developer Options): Clarify that -ftime-report
writes to stderr. Document that if SARIF diagnostic output is
requested then any timing information is written in JSON form as
part of the SARIF output, rather than to stderr.
* timevar.cc: Include "json.h".
(timer::named_items::m_hash_map): Split out type into...
(timer::named_items::hash_map_t): ...this new typedef.
(timer::named_items::make_json): New function.
(timevar_diff): New function.
(make_json_for_timevar_time_def): New function.
(timer::timevar_def::make_json): New function.
(timer::make_json): New function.
* timevar.h (class json::value): New forward decl.
(timer::make_json): New decl.
(timer::timevar_def::make_json): New decl.
* tree-diagnostic-client-data-hooks.cc: Include
"diagnostic-format-sarif.h" and "timevar.h".
(compiler_data_hooks::add_sarif_invocation_properties): New vfunc
implementation.
gcc/testsuite/ChangeLog:
PR analyzer/109361
* c-c++-common/diagnostic-format-sarif-file-timevars-1.c: New test.
* c-c++-common/diagnostic-format-sarif-file-timevars-2.c: New test.
Signed-off-by: David Malcolm <dmalcolm@redhat.com>
|
|
This patch adds support to gcc's diagnostic subsystem for emitting
diagnostics in SARIF, aka the Static Analysis Results Interchange Format:
https://sarifweb.azurewebsites.net/
by extending -fdiagnostics-format= to add two new options:
-fdiagnostics-format=sarif-stderr
and:
-fdiagnostics-format=sarif-file
The patch targets SARIF v2.1.0
This is a JSON-based format suited for capturing the results of static
analysis tools (like GCC's -fanalyzer), but it can also be used for plain
GCC warnings and errors.
SARIF supports per-event metadata in diagnostic paths such as
["acquire", "resource"] and ["release", "lock"] (specifically, the
threadFlowLocation "kinds" property: SARIF v2.1.0 section 3.38.8), so
the patch extends GCC"s diagnostic_event subclass with a "struct meaning"
with similar purpose. The patch implements this for -fanalyzer so that
the various state-machine-based warnings set these in the SARIF output.
The heart of the implementation is in the new file
diagnostic-format-sarif.cc. Much of the rest of the patch is interface
classes, isolating the diagnostic subsystem (which has no knowledge of
e.g. tree or langhook) from the "client" code in the compiler proper
cc1 etc).
The patch adds a langhook for specifying the SARIF v2.1.0
"artifact.sourceLanguage" property, based on the list in
SARIF v2.1.0 Appendix J.
The patch adds automated DejaGnu tests to our testsuite via new
scan-sarif-file and scan-sarif-file-not directives (although these
merely use regexps, rather than attempting to use a proper JSON parser).
I've tested the patch by hand using the validator at:
https://sarifweb.azurewebsites.net/Validation
and the react-based viewer at:
https://microsoft.github.io/sarif-web-component/
which successfully shows most of the information (although not paths,
and not CWE IDs), and I've fixed all validation errors I've seen (though
bugs no doubt remain).
I've also tested the generated SARIF using the VS Code extension linked
to from the SARIF website; I'm a novice with VS Code, but it seems to be
able to handle my generated SARIF files (e.g. showing the data in the
SARIF tab, and showing squiggly underlines under issues, and when I
click on them, it visualizes the events in the path inline within the
source window).
Has anyone written an Emacs mode for SARIF files? (pretty please)
gcc/ChangeLog:
* Makefile.in (OBJS): Add tree-diagnostic-client-data-hooks.o and
tree-logical-location.o.
(OBJS-libcommon): Add diagnostic-format-sarif.o; reorder.
(CFLAGS-tree-diagnostic-client-data-hooks.o): Add TARGET_NAME.
* common.opt (fdiagnostics-format=): Add sarif-stderr and sarif-file.
(sarif-stderr, sarif-file): New enum values.
* diagnostic-client-data-hooks.h: New file.
* diagnostic-format-sarif.cc: New file.
* diagnostic-path.h (enum diagnostic_event::verb): New enum.
(enum diagnostic_event::noun): New enum.
(enum diagnostic_event::property): New enum.
(struct diagnostic_event::meaning): New struct.
(diagnostic_event::get_logical_location): New vfunc.
(diagnostic_event::get_meaning): New vfunc.
(simple_diagnostic_event::get_logical_location): New vfunc impl.
(simple_diagnostic_event::get_meaning): New vfunc impl.
* diagnostic.cc: Include "diagnostic-client-data-hooks.h".
(diagnostic_initialize): Initialize m_client_data_hooks.
(diagnostic_finish): Clean up m_client_data_hooks.
(diagnostic_event::meaning::dump_to_pp): New.
(diagnostic_event::meaning::maybe_get_verb_str): New.
(diagnostic_event::meaning::maybe_get_noun_str): New.
(diagnostic_event::meaning::maybe_get_property_str): New.
(get_cwe_url): Make non-static.
(diagnostic_output_format_init): Handle
DIAGNOSTICS_OUTPUT_FORMAT_SARIF_STDERR and
DIAGNOSTICS_OUTPUT_FORMAT_SARIF_FILE.
* diagnostic.h (enum diagnostics_output_format): Add
DIAGNOSTICS_OUTPUT_FORMAT_SARIF_STDERR and
DIAGNOSTICS_OUTPUT_FORMAT_SARIF_FILE.
(class diagnostic_client_data_hooks): New forward decl.
(class logical_location): New forward decl.
(diagnostic_context::m_client_data_hooks): New field.
(diagnostic_output_format_init_sarif_stderr): New decl.
(diagnostic_output_format_init_sarif_file): New decl.
(get_cwe_url): New decl.
* doc/invoke.texi (-fdiagnostics-format=): Add sarif-stderr and
sarif-file.
* doc/sourcebuild.texi (Scan a particular file): Add
scan-sarif-file and scan-sarif-file-not.
* langhooks-def.h (lhd_get_sarif_source_language): New decl.
(LANG_HOOKS_GET_SARIF_SOURCE_LANGUAGE): New macro.
(LANG_HOOKS_INITIALIZER): Add
LANG_HOOKS_GET_SARIF_SOURCE_LANGUAGE.
* langhooks.cc (lhd_get_sarif_source_language): New.
* langhooks.h (lang_hooks::get_sarif_source_language): New field.
* logical-location.h: New file.
* plugin.cc (struct for_each_plugin_closure): New.
(for_each_plugin_cb): New.
(for_each_plugin): New.
* plugin.h (for_each_plugin): New decl.
* tree-diagnostic-client-data-hooks.cc: New file.
* tree-diagnostic.cc: Include "diagnostic-client-data-hooks.h".
(tree_diagnostics_defaults): Populate m_client_data_hooks.
* tree-logical-location.cc: New file.
* tree-logical-location.h: New file.
gcc/ada/ChangeLog:
* gcc-interface/misc.cc (gnat_get_sarif_source_language): New.
(LANG_HOOKS_GET_SARIF_SOURCE_LANGUAGE): Redefine.
gcc/analyzer/ChangeLog:
* checker-path.cc (checker_event::get_meaning): New.
(function_entry_event::get_meaning): New.
(state_change_event::get_desc): Add dump of meaning of the event
to the -fanalyzer-verbose-state-changes output.
(state_change_event::get_meaning): New.
(cfg_edge_event::get_meaning): New.
(call_event::get_meaning): New.
(return_event::get_meaning): New.
(start_consolidated_cfg_edges_event::get_meaning): New.
(warning_event::get_meaning): New.
* checker-path.h: Include "tree-logical-location.h".
(checker_event::checker_event): Construct m_logical_loc.
(checker_event::get_logical_location): New.
(checker_event::get_meaning): New decl.
(checker_event::m_logical_loc): New.
(function_entry_event::get_meaning): New decl.
(state_change_event::get_meaning): New decl.
(cfg_edge_event::get_meaning): New decl.
(call_event::get_meaning): New decl.
(return_event::get_meaning): New decl.
(start_consolidated_cfg_edges_event::get_meaning): New.
(warning_event::get_meaning): New decl.
* pending-diagnostic.h: Include "diagnostic-path.h".
(pending_diagnostic::get_meaning_for_state_change): New vfunc.
* sm-file.cc (file_diagnostic::get_meaning_for_state_change): New
vfunc impl.
* sm-malloc.cc (malloc_diagnostic::get_meaning_for_state_change):
Likewise.
* sm-sensitive.cc
(exposure_through_output_file::get_meaning_for_state_change):
Likewise.
* sm-taint.cc (taint_diagnostic::get_meaning_for_state_change):
Likewise.
* varargs.cc
(va_list_sm_diagnostic::get_meaning_for_state_change): Likewise.
gcc/c/ChangeLog:
* c-lang.cc (LANG_HOOKS_GET_SARIF_SOURCE_LANGUAGE): Redefine.
(c_get_sarif_source_language): New.
* c-tree.h (c_get_sarif_source_language): New decl.
gcc/cp/ChangeLog:
* cp-lang.cc (LANG_HOOKS_GET_SARIF_SOURCE_LANGUAGE): Redefine.
(cp_get_sarif_source_language): New.
gcc/d/ChangeLog:
* d-lang.cc (d_get_sarif_source_language): New.
(LANG_HOOKS_GET_SARIF_SOURCE_LANGUAGE): Redefine.
gcc/fortran/ChangeLog:
* f95-lang.cc (gfc_get_sarif_source_language): New.
(LANG_HOOKS_GET_SARIF_SOURCE_LANGUAGE): Redefine.
gcc/go/ChangeLog:
* go-lang.cc (go_get_sarif_source_language): New.
(LANG_HOOKS_GET_SARIF_SOURCE_LANGUAGE): Redefine.
gcc/objc/ChangeLog:
* objc-act.h (objc_get_sarif_source_language): New decl.
* objc-lang.cc (LANG_HOOKS_GET_SARIF_SOURCE_LANGUAGE): Redefine.
(objc_get_sarif_source_language): New.
gcc/testsuite/ChangeLog:
* c-c++-common/diagnostic-format-sarif-file-1.c: New test.
* c-c++-common/diagnostic-format-sarif-file-2.c: New test.
* c-c++-common/diagnostic-format-sarif-file-3.c: New test.
* c-c++-common/diagnostic-format-sarif-file-4.c: New test.
* gcc.dg/analyzer/file-meaning-1.c: New test.
* gcc.dg/analyzer/malloc-meaning-1.c: New test.
* gcc.dg/analyzer/malloc-sarif-1.c: New test.
* gcc.dg/plugin/analyzer_gil_plugin.c
(gil_diagnostic::get_meaning_for_state_change): New vfunc impl.
* gcc.dg/plugin/diagnostic-test-paths-5.c: New test.
* gcc.dg/plugin/plugin.exp (plugin_test_list): Add
diagnostic-test-paths-5.c to tests for
diagnostic_plugin_test_paths.c.
* lib/gcc-dg.exp: Load scansarif.exp.
* lib/scansarif.exp: New test.
libatomic/ChangeLog:
* testsuite/lib/libatomic.exp: Add load_gcc_lib of scansarif.exp.
libgomp/ChangeLog:
* testsuite/lib/libgomp.exp: Add load_gcc_lib of scansarif.exp.
libitm/ChangeLog:
* testsuite/lib/libitm.exp: Add load_gcc_lib of scansarif.exp.
libphobos/ChangeLog:
* testsuite/lib/libphobos-dg.exp: Add load_gcc_lib of scansarif.exp.
Signed-off-by: David Malcolm <dmalcolm@redhat.com>
|
