qapi-gen: New common driver for code and doc generators

Whenever qapi-schema.json changes, we run six programs eleven times to
update eleven files.  Similar for qga/qapi-schema.json.  This is
silly.  Replace the six programs by a single program that spits out
all eleven files.

The programs become modules in new Python package qapi, along with the
helper library.  This requires moving them to scripts/qapi/.  While
moving them, consistently drop executable mode bits.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-9-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
[eblake: move change to one-line 'blurb' earlier in series, mention mode
bit change as intentional, update qapi-code-gen.txt to match actual
generated events.c file]
Signed-off-by: Eric Blake <eblake@redhat.com>

1 files changed, 48 insertions, 54 deletions

diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index 5900b39..a525ef3 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -899,12 +899,13 @@ the names of built-in types.  Clients should examine member
 
 == Code generation ==
 
-Schemas are fed into five scripts to generate all the code/files that,
-paired with the core QAPI libraries, comprise everything required to
-take JSON commands read in by a Client JSON Protocol server, unmarshal
-the arguments into the underlying C types, call into the corresponding
-C function, map the response back to a Client JSON Protocol response
-to be returned to the user, and introspect the commands.
+The QAPI code generator qapi-gen.py generates code and documentation
+from the schema.  Together with the core QAPI libraries, this code
+provides everything required to take JSON commands read in by a Client
+JSON Protocol server, unmarshal the arguments into the underlying C
+types, call into the corresponding C function, map the response back
+to a Client JSON Protocol response to be returned to the user, and
+introspect the commands.
 
 As an example, we'll use the following schema, which describes a
 single complex user-defined type, along with command which takes a
@@ -922,18 +923,23 @@ qmp_my_command(); everything else is produced by the generator.
 
     { 'event': 'MY_EVENT' }
 
+We run qapi-gen.py like this:
+
+    $ python scripts/qapi-gen.py --output-dir="qapi-generated" \
+    --prefix="example-" example-schema.json
+
 For a more thorough look at generated code, the testsuite includes
 tests/qapi-schema/qapi-schema-tests.json that covers more examples of
 what the generator will accept, and compiles the resulting C code as
 part of 'make check-unit'.
 
-=== scripts/qapi-types.py ===
+=== Code generated for QAPI types ===
 
-Used to generate the C types defined by a schema, along with
-supporting code. The following files are created:
+The following files are created:
 
 $(prefix)qapi-types.h - C types corresponding to types defined in
-                        the schema you pass in
+                        the schema
+
 $(prefix)qapi-types.c - Cleanup functions for the above C types
 
 The $(prefix) is an optional parameter used as a namespace to keep the
@@ -943,8 +949,6 @@ created code.
 
 Example:
 
-    $ python scripts/qapi-types.py --output-dir="qapi-generated" \
-    --prefix="example-" example-schema.json
     $ cat qapi-generated/example-qapi-types.h
 [Uninteresting stuff omitted...]
 
@@ -1008,28 +1012,26 @@ Example:
         visit_free(v);
     }
 
-=== scripts/qapi-visit.py ===
+=== Code generated for visiting QAPI types ===
 
-Used to generate the visitor functions used to walk through and
-convert between a native QAPI C data structure and some other format
-(such as QObject); the generated functions are named visit_type_FOO()
-and visit_type_FOO_members().
+These are the visitor functions used to walk through and convert
+between a native QAPI C data structure and some other format (such as
+QObject); the generated functions are named visit_type_FOO() and
+visit_type_FOO_members().
 
 The following files are generated:
 
-$(prefix)qapi-visit.c: visitor function for a particular C type, used
+$(prefix)qapi-visit.c: Visitor function for a particular C type, used
                        to automagically convert QObjects into the
                        corresponding C type and vice-versa, as well
                        as for deallocating memory for an existing C
                        type
 
-$(prefix)qapi-visit.h: declarations for previously mentioned visitor
+$(prefix)qapi-visit.h: Declarations for previously mentioned visitor
                        functions
 
 Example:
 
-    $ python scripts/qapi-visit.py --output-dir="qapi-generated"
-    --prefix="example-" example-schema.json
     $ cat qapi-generated/example-qapi-visit.h
 [Uninteresting stuff omitted...]
 
@@ -1137,30 +1139,22 @@ Example:
         error_propagate(errp, err);
     }
 
-=== scripts/qapi-commands.py ===
+=== Code generated for commands ===
+
+These are the marshaling/dispatch functions for the commands defined
+in the schema.  The generated code provides qmp_marshal_COMMAND(), and
+declares qmp_COMMAND() that the user must implement.
 
-Used to generate the marshaling/dispatch functions for the commands
-defined in the schema. The generated code implements
-qmp_marshal_COMMAND() (registered automatically), and declares
-qmp_COMMAND() that the user must implement.  The following files are
-generated:
+The following files are generated:
 
-$(prefix)qmp-marshal.c: command marshal/dispatch functions for each
-                        QMP command defined in the schema. Functions
-                        generated by qapi-visit.py are used to
-                        convert QObjects received from the wire into
-                        function parameters, and uses the same
-                        visitor functions to convert native C return
-                        values to QObjects from transmission back
-                        over the wire.
+$(prefix)qmp-marshal.c: Command marshal/dispatch functions for each
+                        QMP command defined in the schema
 
 $(prefix)qmp-commands.h: Function prototypes for the QMP commands
-                         specified in the schema.
+                         specified in the schema
 
 Example:
 
-    $ python scripts/qapi-commands.py --output-dir="qapi-generated"
-    --prefix="example-" example-schema.json
     $ cat qapi-generated/example-qmp-commands.h
 [Uninteresting stuff omitted...]
 
@@ -1242,20 +1236,20 @@ Example:
                              qmp_marshal_my_command, QCO_NO_OPTIONS);
     }
 
-=== scripts/qapi-event.py ===
+=== Code generated for events ===
 
-Used to generate the event-related C code defined by a schema, with
-implementations for qapi_event_send_FOO(). The following files are
-created:
+This is the code related to events defined in the schema, providing
+qapi_event_send_EVENT().
+
+The following files are created:
 
 $(prefix)qapi-event.h - Function prototypes for each event type, plus an
                         enumeration of all event names
+
 $(prefix)qapi-event.c - Implementation of functions to send an event
 
 Example:
 
-    $ python scripts/qapi-event.py --output-dir="qapi-generated"
-    --prefix="example-" example-schema.json
     $ cat qapi-generated/example-qapi-event.h
 [Uninteresting stuff omitted...]
 
@@ -1301,24 +1295,24 @@ Example:
         QDECREF(qmp);
     }
 
-    const char *const example_QAPIEvent_lookup[] = {
-        [EXAMPLE_QAPI_EVENT_MY_EVENT] = "MY_EVENT",
-        [EXAMPLE_QAPI_EVENT__MAX] = NULL,
+    const QEnumLookup example_QAPIEvent_lookup = {
+        .array = (const char *const[]) {
+            [EXAMPLE_QAPI_EVENT_MY_EVENT] = "MY_EVENT",
+        },
+        .size = EXAMPLE_QAPI_EVENT__MAX
     };
 
-=== scripts/qapi-introspect.py ===
+=== Code generated for introspection ===
 
-Used to generate the introspection C code for a schema. The following
-files are created:
+The following files are created:
 
 $(prefix)qmp-introspect.c - Defines a string holding a JSON
-                            description of the schema.
-$(prefix)qmp-introspect.h - Declares the above string.
+                            description of the schema
+
+$(prefix)qmp-introspect.h - Declares the above string
 
 Example:
 
-    $ python scripts/qapi-introspect.py --output-dir="qapi-generated"
-    --prefix="example-" example-schema.json
     $ cat qapi-generated/example-qmp-introspect.h
 [Uninteresting stuff omitted...]