From 3ae3fcfae533086d3e210589c10f317717c9d789 Mon Sep 17 00:00:00 2001 From: Vladimir Sementsov-Ogievskiy Date: Fri, 31 Jan 2020 17:22:18 +0300 Subject: docs: improve qcow2 spec about extending image header Make it more obvious how to add new fields to the version 3 header and how to interpret them. The specification is adjusted so that for new defined optional fields: 1. Software may support some of these optional fields and ignore the others, which means that features may be backported to downstream Qemu independently. 2. If we want to add incompatible field (or a field, for which some of its values would be incompatible), it must be accompanied by incompatible feature bit. Also the concept of "default is zero" is clarified, as it's strange to say that the value of the field is assumed to be zero for the software version which don't know about the field at all and don't know how to treat it be it zero or not. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Message-Id: <20200131142219.3264-2-vsementsov@virtuozzo.com> Reviewed-by: Alberto Garcia [mreitz: s/some its/some of its/] Signed-off-by: Max Reitz --- docs/interop/qcow2.txt | 45 ++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 42 insertions(+), 3 deletions(-) (limited to 'docs') diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index af5711e..823cc26 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -79,9 +79,9 @@ The first cluster of a qcow2 image contains the file header: Offset into the image file at which the snapshot table starts. Must be aligned to a cluster boundary. -If the version is 3 or higher, the header has the following additional fields. -For version 2, the values are assumed to be zero, unless specified otherwise -in the description of a field. +For version 2, the header is exactly 72 bytes in length, and finishes here. +For version 3 or higher, the header length is at least 104 bytes, including +the next fields through header_length. 72 - 79: incompatible_features Bitmask of incompatible features. An implementation must @@ -164,6 +164,45 @@ in the description of a field. 100 - 103: header_length Length of the header structure in bytes. For version 2 images, the length is always assumed to be 72 bytes. + For version 3 it's at least 104 bytes and must be a multiple + of 8. + + +=== Additional fields (version 3 and higher) === + +In general, these fields are optional and may be safely ignored by the software, +as well as filled by zeros (which is equal to field absence), if software needs +to set field B, but does not care about field A which precedes B. More +formally, additional fields have the following compatibility rules: + +1. If the value of the additional field must not be ignored for correct +handling of the file, it will be accompanied by a corresponding incompatible +feature bit. + +2. If there are no unrecognized incompatible feature bits set, an unknown +additional field may be safely ignored other than preserving its value when +rewriting the image header. + +3. An explicit value of 0 will have the same behavior as when the field is not +present*, if not altered by a specific incompatible bit. + +*. A field is considered not present when header_length is less than or equal +to the field's offset. Also, all additional fields are not present for +version 2. + + < ... No additional fields in the header currently ... > + + +=== Header padding === + +@header_length must be a multiple of 8, which means that if the end of the last +additional field is not aligned, some padding is needed. This padding must be +zeroed, so that if some existing (or future) additional field will fall into +the padding, it will be interpreted accordingly to point [3.] of the previous +paragraph, i.e. in the same manner as when this field is not present. + + +=== Header extensions === Directly after the image header, optional sections called header extensions can be stored. Each extension has a structure like the following: -- cgit v1.1 From 66fcbca5a7202d2ff2973b61a30f915204917eb7 Mon Sep 17 00:00:00 2001 From: Vladimir Sementsov-Ogievskiy Date: Fri, 31 Jan 2020 17:22:19 +0300 Subject: docs: qcow2: introduce compression type feature The patch adds a new additional field to the qcow2 header: compression_type, which specifies compression type. If field is absent or zero, default compression type is set: ZLIB, which corresponds to current behavior. New compression type (ZSTD) is to be added in further commit. Suggested-by: Denis Plotnikov Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20200131142219.3264-3-vsementsov@virtuozzo.com> [mreitz: s/Bits 3-63: Reserved/Bits 4-63: Reserved/] Signed-off-by: Max Reitz --- docs/interop/qcow2.txt | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) (limited to 'docs') diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index 823cc26..5597e24 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -109,7 +109,12 @@ the next fields through header_length. An External Data File Name header extension may be present if this bit is set. - Bits 3-63: Reserved (set to 0) + Bit 3: Compression type bit. If this bit is set, + a non-default compression is used for compressed + clusters. The compression_type field must be + present and not zero. + + Bits 4-63: Reserved (set to 0) 80 - 87: compatible_features Bitmask of compatible features. An implementation can @@ -190,7 +195,19 @@ present*, if not altered by a specific incompatible bit. to the field's offset. Also, all additional fields are not present for version 2. - < ... No additional fields in the header currently ... > + 104: compression_type + + Defines the compression method used for compressed clusters. + All compressed clusters in an image use the same compression + type. + + If the incompatible bit "Compression type" is set: the field + must be present and non-zero (which means non-zlib + compression type). Otherwise, this field must not be present + or must be zero (which means zlib). + + Available compression type values: + 0: zlib === Header padding === -- cgit v1.1 From 168468fe19c872d31d17581e6a6cf7b69bd105b1 Mon Sep 17 00:00:00 2001 From: David Edmondson Date: Wed, 5 Feb 2020 11:02:48 +0000 Subject: qemu-img: Add --target-is-zero to convert In many cases the target of a convert operation is a newly provisioned target that the user knows is blank (reads as zero). In this situation there is no requirement for qemu-img to wastefully zero out the entire device. Add a new option, --target-is-zero, allowing the user to indicate that an existing target device will return zeros for all reads. Signed-off-by: David Edmondson Message-Id: <20200205110248.2009589-2-david.edmondson@oracle.com> Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Signed-off-by: Max Reitz --- docs/interop/qemu-img.rst | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) (limited to 'docs') diff --git a/docs/interop/qemu-img.rst b/docs/interop/qemu-img.rst index 42e4451..5f40137 100644 --- a/docs/interop/qemu-img.rst +++ b/docs/interop/qemu-img.rst @@ -214,6 +214,13 @@ Parameters to convert subcommand: will still be printed. Areas that cannot be read from the source will be treated as containing only zeroes. +.. option:: --target-is-zero + + Assume that reading the destination image will always return + zeros. This parameter is mutually exclusive with a destination image + that has a backing file. It is required to also use the ``-n`` + parameter to skip image creation. + Parameters to dd subcommand: .. program:: qemu-img-dd @@ -366,7 +373,7 @@ Command description: 4 Error on reading data -.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME +.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME Convert the disk image *FILENAME* or a snapshot *SNAPSHOT_PARAM* to disk image *OUTPUT_FILENAME* using format *OUTPUT_FMT*. It can -- cgit v1.1