From c06982122036546fcbfe40f5b22ae7088d28c9a2 Mon Sep 17 00:00:00 2001 From: Markus Armbruster Date: Tue, 3 Jul 2018 10:53:27 +0200 Subject: qmp: Say "out-of-band" instead of "Out-Of-Band" Affects documentation and a few error messages. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Message-Id: <20180703085358.13941-2-armbru@redhat.com> --- docs/devel/qapi-code-gen.txt | 2 +- docs/interop/qmp-spec.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) (limited to 'docs') diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt index 94a7e8f..9625798 100644 --- a/docs/devel/qapi-code-gen.txt +++ b/docs/devel/qapi-code-gen.txt @@ -641,7 +641,7 @@ possible, the command expression should include the optional key 'success-response' with boolean value false. So far, only QGA makes use of this member. -A command can be declared to support Out-Of-Band (OOB) execution. By +A command can be declared to support out-of-band (OOB) execution. By default, commands do not support OOB. To declare a command that supports it, the schema includes an extra 'allow-oob' field. For example: diff --git a/docs/interop/qmp-spec.txt b/docs/interop/qmp-spec.txt index 6fa193a..2bb492d 100644 --- a/docs/interop/qmp-spec.txt +++ b/docs/interop/qmp-spec.txt @@ -85,7 +85,7 @@ The greeting message format is: Currently supported capabilities are: -- "oob": the QMP server supports "Out-Of-Band" (OOB) command +- "oob": the QMP server supports "out-of-band" (OOB) command execution. For more details, please see the "run-oob" parameter in the "Issuing Commands" section below. Not all commands allow this "oob" execution. The "query-qmp-schema" command can be used to -- cgit v1.1 From 71696cc66ce74a645e68844340e9fb5cfd598cc2 Mon Sep 17 00:00:00 2001 From: Markus Armbruster Date: Tue, 3 Jul 2018 10:53:29 +0200 Subject: docs/interop/qmp: Improve OOB documentation OOB documentation is spread over qmp-spec.txt sections 2.2.1 Capabilities and 2.3 Issuing Commands. The amount of detail is a bit distracting there. Move the meat of the matter to new section 2.3.1 Out of band execution. Throw in a few other improvements while there: * 2.2 Server Greeting: Drop advice to search entire capabilities array; should be obvious. * 3. QMP Examples - 3.1 Server Greeting: Update greeting to the one we expect for the release. Now shows capability "oob". Update qmp-intro.txt likewise. - 3.2 Capabilities negotiation: Show client accepting capability "oob". - 3.7 Out-of-band execution: New. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Message-Id: <20180703085358.13941-4-armbru@redhat.com> [Whitespace tidied up] --- docs/interop/qmp-intro.txt | 13 ++++---- docs/interop/qmp-spec.txt | 74 ++++++++++++++++++++++++++++++---------------- 2 files changed, 56 insertions(+), 31 deletions(-) (limited to 'docs') diff --git a/docs/interop/qmp-intro.txt b/docs/interop/qmp-intro.txt index 900d69d..9d54a71 100644 --- a/docs/interop/qmp-intro.txt +++ b/docs/interop/qmp-intro.txt @@ -52,13 +52,14 @@ Escape character is '^]'. "QMP": { "version": { "qemu": { - "micro": 50, - "minor": 6, - "major": 1 - }, - "package": "" - }, + "micro": 0, + "minor": 0, + "major": 3 + }, + "package": "v3.0.0" + }, "capabilities": [ + "oob" ] } } diff --git a/docs/interop/qmp-spec.txt b/docs/interop/qmp-spec.txt index 2bb492d..6b72b69 100644 --- a/docs/interop/qmp-spec.txt +++ b/docs/interop/qmp-spec.txt @@ -77,8 +77,7 @@ The greeting message format is: is the same of the query-version command) - The "capabilities" member specify the availability of features beyond the baseline specification; the order of elements in this array has no - particular significance, so a client must search the entire array - when looking for a particular capability + particular significance. 2.2.1 Capabilities ------------------ @@ -86,16 +85,7 @@ The greeting message format is: Currently supported capabilities are: - "oob": the QMP server supports "out-of-band" (OOB) command - execution. For more details, please see the "run-oob" parameter in - the "Issuing Commands" section below. Not all commands allow this - "oob" execution. The "query-qmp-schema" command can be used to - inspect which commands support "oob" execution. - -QMP clients can get a list of supported QMP capabilities of the QMP -server in the greeting message mentioned above. By default, all the -capabilities are off. To enable any QMP capabilities, the QMP client -needs to send the "qmp_capabilities" command with an extra parameter -for the requested capabilities. + execution, as described in section "2.3.1 Out-of-band execution". 2.3 Issuing Commands -------------------- @@ -115,14 +105,38 @@ The format for command execution is: - The "id" member is a transaction identification associated with the command execution. It is required for all commands if the OOB - capability was enabled at startup, and optional otherwise. The same - "id" field will be part of the response if provided. The "id" member - can be any json-value, although most clients merely use a - json-number incremented for each successive command -- The "control" member is optional, and currently only used for - out-of-band execution. The handling or response of an "oob" command - can overtake prior in-band commands. To enable "oob" handling of a - particular command, just provide a control field with: { "control": - { "run-oob": true } } + "id" field will be part of the response if provided. The "id" + member can be any json-value. A json-number incremented for each + successive command works fine. +- The optional "control" member further specifies how the command is + to be executed. Currently, its only member is optional "run-oob". + See section "2.3.1 Out-of-band execution" for details. + + +2.3.1 Out-of-band execution +--------------------------- + +The server normally reads, executes and responds to one command after +the other. The client therefore receives command responses in issue +order. + +With out-of-band execution enabled via capability negotiation (section +4.), the server reads and queues commands as they arrive. It executes +commands from the queue one after the other. Commands executed +out-of-band jump the queue: the command get executed right away, +possibly overtaking prior in-band commands. The client may therefore +receive such a command's response before responses from prior in-band +commands. + +To execute a command out-of-band, the client puts "run-oob": true into +execute's member "control". + +If the client sends in-band commands faster than the server can +execute them, the server will eventually drop commands to limit the +queue length. The sever sends event COMMAND_DROPPED then. + +Only a few commands support out-of-band execution. The ones that do +have "allow-oob": true in output of query-qmp-schema. 2.4 Commands Responses ---------------------- @@ -223,12 +237,13 @@ This section provides some examples of real QMP usage, in all of them 3.1 Server greeting ------------------- -S: { "QMP": { "version": { "qemu": { "micro": 50, "minor": 6, "major": 1 }, - "package": ""}, "capabilities": []}} +S: { "QMP": {"version": {"qemu": {"micro": 0, "minor": 0, "major": 3}, + "package": "v3.0.0"}, "capabilities": ["oob"] } } + +3.2 Capabilities negotiation +---------------------------- -3.2 Client QMP negotiation --------------------------- -C: { "execute": "qmp_capabilities" } +C: { "execute": "qmp_capabilities", "arguments": { "enable": ["oob"] } } S: { "return": {}} 3.3 Simple 'stop' execution @@ -255,6 +270,15 @@ S: { "error": { "class": "GenericError", "desc": "Invalid JSON syntax" } } S: { "timestamp": { "seconds": 1258551470, "microseconds": 802384 }, "event": "POWERDOWN" } +3.7 Out-of-band execution +------------------------- + +C: { "execute": "migrate-pause", "id": 42, "control": { "run-oob": true } } +S: { "id": 42, + "error": { "class": "GenericError", + "desc": "migrate-pause is currently only supported during postcopy-active state" } } + + 4. Capabilities Negotiation =========================== -- cgit v1.1 From 80cd93bd966bbb0907caa7f1d5676342f27f8f9e Mon Sep 17 00:00:00 2001 From: Markus Armbruster Date: Tue, 3 Jul 2018 10:53:33 +0200 Subject: qmp: Make "id" optional again even in "oob" monitors MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Commit cf869d53172 "qmp: support out-of-band (oob) execution" made "id" mandatory for all commands when the client accepted capability "oob". This is rather onerous when you play with QMP by hand, and unnecessarily so: only out-of-band commands need an ID for reliable matching of response to command. Revert that part of commit cf869d53172 for now, but have documentation advise on the need to use "id" with out-of-band commands. Signed-off-by: Markus Armbruster Message-Id: <20180703085358.13941-8-armbru@redhat.com> Reviewed-by: Daniel P. Berrangé Reviewed-by: Eric Blake --- docs/interop/qmp-spec.txt | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) (limited to 'docs') diff --git a/docs/interop/qmp-spec.txt b/docs/interop/qmp-spec.txt index 6b72b69..a1d6f9e 100644 --- a/docs/interop/qmp-spec.txt +++ b/docs/interop/qmp-spec.txt @@ -103,16 +103,13 @@ The format for command execution is: required. Each command documents what contents will be considered valid when handling the json-argument - The "id" member is a transaction identification associated with the - command execution. It is required for all commands if the OOB - - capability was enabled at startup, and optional otherwise. The same - "id" field will be part of the response if provided. The "id" - member can be any json-value. A json-number incremented for each - successive command works fine. + command execution, it is optional and will be part of the response + if provided. The "id" member can be any json-value. A json-number + incremented for each successive command works fine. - The optional "control" member further specifies how the command is to be executed. Currently, its only member is optional "run-oob". See section "2.3.1 Out-of-band execution" for details. - 2.3.1 Out-of-band execution --------------------------- @@ -128,6 +125,10 @@ possibly overtaking prior in-band commands. The client may therefore receive such a command's response before responses from prior in-band commands. +To be able to match responses back to their commands, the client needs +to pass "id" with out-of-band commands. Passing it with all commands +is recommended for clients that accept capability "oob". + To execute a command out-of-band, the client puts "run-oob": true into execute's member "control". -- cgit v1.1 From 00ecec151d2323e742af94cccf2de77025f3c0c1 Mon Sep 17 00:00:00 2001 From: Markus Armbruster Date: Tue, 3 Jul 2018 10:53:38 +0200 Subject: qmp: Redo how the client requests out-of-band execution Commit cf869d53172 "qmp: support out-of-band (oob) execution" added a general mechanism for command-independent arguments just for an out-of-band flag: The "control" key is introduced to store this extra flag. "control" field is used to store arguments that are shared by all the commands, rather than command specific arguments. Let "run-oob" be the first. However, it failed to reject unknown members of "control". For instance, in QMP command {"execute": "query-name", "id": 42, "control": {"crap": true}} "crap" gets silently ignored. Instead of fixing this, revert the general "control" mechanism (because YAGNI), and do it the way I initially proposed, with key "exec-oob". Simpler code, simpler interface. An out-of-band command {"execute": "migrate-pause", "id": 42, "control": {"run-oob": true}} becomes {"exec-oob": "migrate-pause", "id": 42} Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Message-Id: <20180703085358.13941-13-armbru@redhat.com> [Commit message typo fixed] --- docs/devel/qapi-code-gen.txt | 10 ++++------ docs/interop/qmp-spec.txt | 18 ++++++++---------- 2 files changed, 12 insertions(+), 16 deletions(-) (limited to 'docs') diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt index 9625798..f020f6b 100644 --- a/docs/devel/qapi-code-gen.txt +++ b/docs/devel/qapi-code-gen.txt @@ -649,13 +649,11 @@ example: { 'command': 'migrate_recover', 'data': { 'uri': 'str' }, 'allow-oob': true } -To execute a command with out-of-band priority, the client specifies -the "control" field in the request, with "run-oob" set to -true. Example: +To execute a command with out-of-band priority, the client uses key +"exec-oob" instead of "execute". Example: - => { "execute": "command-support-oob", - "arguments": { ... }, - "control": { "run-oob": true } } + => { "exec-oob": "migrate-recover", + "arguments": { "uri": "tcp:192.168.1.200:12345" } } <= { "return": { } } Without it, even the commands that support out-of-band execution will diff --git a/docs/interop/qmp-spec.txt b/docs/interop/qmp-spec.txt index a1d6f9e..1566b8a 100644 --- a/docs/interop/qmp-spec.txt +++ b/docs/interop/qmp-spec.txt @@ -92,12 +92,16 @@ Currently supported capabilities are: The format for command execution is: -{ "execute": json-string, "arguments": json-object, "id": json-value, - "control": json-object } +{ "execute": json-string, "arguments": json-object, "id": json-value } + +or + +{ "exec-oob": json-string, "arguments": json-object, "id": json-value } Where, -- The "execute" member identifies the command to be executed by the Server +- The "execute" or "exec-oob" member identifies the command to be + executed by the server. The latter requests out-of-band execution. - The "arguments" member is used to pass any arguments required for the execution of the command, it is optional when no arguments are required. Each command documents what contents will be considered @@ -106,9 +110,6 @@ The format for command execution is: command execution, it is optional and will be part of the response if provided. The "id" member can be any json-value. A json-number incremented for each successive command works fine. -- The optional "control" member further specifies how the command is - to be executed. Currently, its only member is optional "run-oob". - See section "2.3.1 Out-of-band execution" for details. 2.3.1 Out-of-band execution --------------------------- @@ -129,9 +130,6 @@ To be able to match responses back to their commands, the client needs to pass "id" with out-of-band commands. Passing it with all commands is recommended for clients that accept capability "oob". -To execute a command out-of-band, the client puts "run-oob": true into -execute's member "control". - If the client sends in-band commands faster than the server can execute them, the server will eventually drop commands to limit the queue length. The sever sends event COMMAND_DROPPED then. @@ -274,7 +272,7 @@ S: { "timestamp": { "seconds": 1258551470, "microseconds": 802384 }, 3.7 Out-of-band execution ------------------------- -C: { "execute": "migrate-pause", "id": 42, "control": { "run-oob": true } } +C: { "exec-oob": "migrate-pause", "id": 42 } S: { "id": 42, "error": { "class": "GenericError", "desc": "migrate-pause is currently only supported during postcopy-active state" } } -- cgit v1.1 From 153d73f320f422ecb5807ac3a93547b9f819599b Mon Sep 17 00:00:00 2001 From: Markus Armbruster Date: Tue, 3 Jul 2018 10:53:58 +0200 Subject: qapi: Polish command flags documentation in qapi-code-gen.txt Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Message-Id: <20180703085358.13941-33-armbru@redhat.com> --- docs/devel/qapi-code-gen.txt | 61 ++++++++++++++++++-------------------------- 1 file changed, 25 insertions(+), 36 deletions(-) (limited to 'docs') diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt index f020f6b..8decd6f 100644 --- a/docs/devel/qapi-code-gen.txt +++ b/docs/devel/qapi-code-gen.txt @@ -624,60 +624,48 @@ its return value. In rare cases, QAPI cannot express a type-safe representation of a corresponding Client JSON Protocol command. You then have to suppress generation of a marshalling function by including a key 'gen' with -boolean value false, and instead write your own function. Please try -to avoid adding new commands that rely on this, and instead use -type-safe unions. For an example of this usage: +boolean value false, and instead write your own function. For +example: { 'command': 'netdev_add', 'data': {'type': 'str', 'id': 'str'}, 'gen': false } +Please try to avoid adding new commands that rely on this, and instead +use type-safe unions. + Normally, the QAPI schema is used to describe synchronous exchanges, where a response is expected. But in some cases, the action of a command is expected to change state in a way that a successful response is not possible (although the command will still return a normal dictionary error on failure). When a successful reply is not -possible, the command expression should include the optional key +possible, the command expression includes the optional key 'success-response' with boolean value false. So far, only QGA makes use of this member. -A command can be declared to support out-of-band (OOB) execution. By -default, commands do not support OOB. To declare a command that -supports it, the schema includes an extra 'allow-oob' field. For -example: +Key 'allow-oob' declares whether the command supports out-of-band +(OOB) execution. It defaults to false. For example: { 'command': 'migrate_recover', 'data': { 'uri': 'str' }, 'allow-oob': true } -To execute a command with out-of-band priority, the client uses key -"exec-oob" instead of "execute". Example: - - => { "exec-oob": "migrate-recover", - "arguments": { "uri": "tcp:192.168.1.200:12345" } } - <= { "return": { } } - -Without it, even the commands that support out-of-band execution will -still be run in-band. +See qmp-spec.txt for out-of-band execution syntax and semantics. -Under normal QMP command execution, the following apply to each -command: +Commands supporting out-of-band execution can still be executed +in-band. -- They are executed in order, -- They run only in main thread of QEMU, -- They run with the BQL held. +When a command is executed in-band, its handler runs in the main +thread with the BQL held. -When a command is executed with OOB, the following changes occur: +When a command is executed out-of-band, its handler runs in a +dedicated monitor I/O thread with the BQL *not* held. -- They can be completed before a pending in-band command, -- They run in a dedicated monitor thread, -- They run with the BQL not held. +An OOB-capable command handler must satisfy the following conditions: -OOB command handlers must satisfy the following conditions: - -- It terminates quickly, -- It does not invoke system calls that may block, +- It terminates quickly. +- It does not invoke system calls that may block. - It does not access guest RAM that may block when userfaultfd is - enabled for postcopy live migration, + enabled for postcopy live migration. - It takes only "fast" locks, i.e. all critical sections protected by any lock it takes also satisfy the conditions for OOB command handler code. @@ -686,17 +674,18 @@ The restrictions on locking limit access to shared state. Such access requires synchronization, but OOB commands can't take the BQL or any other "slow" lock. -If in doubt, do not implement OOB execution support. +When in doubt, do not implement OOB execution support. -A command may use the optional 'allow-preconfig' key to permit its execution -at early runtime configuration stage (preconfig runstate). -If not specified then a command defaults to 'allow-preconfig': false. +Key 'allow-preconfig' declares whether the command is available before +the machine is built. It defaults to false. For example: -An example of declaring a command that is enabled during preconfig: { 'command': 'qmp_capabilities', 'data': { '*enable': [ 'QMPCapability' ] }, 'allow-preconfig': true } +QMP is available before the machine is built only when QEMU was +started with --preconfig. + === Events === Usage: { 'event': STRING, '*data': COMPLEX-TYPE-NAME-OR-DICT, -- cgit v1.1