diff options
author | Tom Rini <trini@konsulko.com> | 2022-08-13 07:37:48 -0400 |
---|---|---|
committer | Tom Rini <trini@konsulko.com> | 2022-08-13 07:37:48 -0400 |
commit | 20d4c6052fe5826b3421e86b2f0e76a6c22581a7 (patch) | |
tree | b11d7fc88bcabb8aac89bda03e30c1af40eab613 | |
parent | 8f9eee8275cf475f6d9435e85aa2d04b61b3cd75 (diff) | |
parent | 046d7a0bb1b383fe14bfe950b91ce2e20f835bd4 (diff) | |
download | u-boot-20d4c6052fe5826b3421e86b2f0e76a6c22581a7.zip u-boot-20d4c6052fe5826b3421e86b2f0e76a6c22581a7.tar.gz u-boot-20d4c6052fe5826b3421e86b2f0e76a6c22581a7.tar.bz2 |
Merge tag 'efi-2022-10-rc3' of https://source.denx.de/u-boot/custodians/u-boot-efiWIP/13Aug2022
Pull request for efi-2022-10-rc3
Documentation:
* Add HTML documentation for patman
* Improve binman documentation
* Man-page for gpio
UEFI:
* move udevice pointer into struct efi_object
* fix efi_convert_device_path_to_text()
Other:
* fs/erofs: silence messages from erofs_probe()
-rw-r--r-- | cmd/efidebug.c | 2 | ||||
-rw-r--r-- | doc/Makefile | 1 | ||||
-rw-r--r-- | doc/develop/bootstd.rst | 6 | ||||
-rw-r--r-- | doc/develop/codingstyle.rst | 4 | ||||
-rw-r--r-- | doc/develop/crash_dumps.rst | 64 | ||||
-rw-r--r-- | doc/develop/index.rst | 1 | ||||
l--------- | doc/develop/patman.rst | 1 | ||||
-rw-r--r-- | doc/develop/process.rst | 4 | ||||
-rw-r--r-- | doc/develop/sending_patches.rst | 16 | ||||
-rw-r--r-- | doc/git-mailrc | 1 | ||||
-rw-r--r-- | doc/usage/cmd/gpio.rst | 90 | ||||
-rw-r--r-- | doc/usage/index.rst | 1 | ||||
-rw-r--r-- | fs/erofs/super.c | 4 | ||||
-rw-r--r-- | include/efi_loader.h | 4 | ||||
-rw-r--r-- | lib/efi_driver/efi_block_device.c | 3 | ||||
-rw-r--r-- | lib/efi_loader/efi_device_path_to_text.c | 1 | ||||
-rw-r--r-- | lib/efi_loader/efi_disk.c | 15 | ||||
-rw-r--r-- | lib/efi_loader/efi_helper.c | 13 | ||||
-rw-r--r-- | tools/binman/binman.rst | 200 | ||||
-rw-r--r-- | tools/binman/entries.rst | 146 | ||||
-rw-r--r-- | tools/binman/entry.py | 5 | ||||
l--------- | tools/patman/README.rst | 1 | ||||
-rwxr-xr-x | tools/patman/main.py | 3 | ||||
-rw-r--r-- | tools/patman/patman.rst (renamed from tools/patman/README) | 556 |
24 files changed, 859 insertions, 283 deletions
diff --git a/cmd/efidebug.c b/cmd/efidebug.c index df928ce..84e6ff5 100644 --- a/cmd/efidebug.c +++ b/cmd/efidebug.c @@ -77,7 +77,7 @@ static int do_efi_capsule_update(struct cmd_tbl *cmdtp, int flag, ret = EFI_CALL(RT->update_capsule(&capsule, 1, 0)); if (ret) { - printf("Cannot handle a capsule at %p", capsule); + printf("Cannot handle a capsule at %p\n", capsule); return CMD_RET_FAILURE; } diff --git a/doc/Makefile b/doc/Makefile index 050d9dd..6081858 100644 --- a/doc/Makefile +++ b/doc/Makefile @@ -56,6 +56,7 @@ quiet_cmd_sphinx = SPHINX $@ --> file://$(abspath $(BUILDDIR)/$3/$4) PYTHONDONTWRITEBYTECODE=1 \ BUILDDIR=$(abspath $(BUILDDIR)) SPHINX_CONF=$(abspath $(srctree)/$(src)/$5/$(SPHINX_CONF)) \ $(SPHINXBUILD) \ + -j$(shell nproc) \ -b $2 \ -c $(abspath $(srctree)/$(src)) \ -d $(abspath $(BUILDDIR)/.doctrees/$3) \ diff --git a/doc/develop/bootstd.rst b/doc/develop/bootstd.rst index b8773f8..f7fc725 100644 --- a/doc/develop/bootstd.rst +++ b/doc/develop/bootstd.rst @@ -21,7 +21,7 @@ feature is typically called `distro boot` (see :doc:`distro`) because it is a way for distributions to boot on any hardware. Traditionally U-Boot has relied on scripts to implement this feature. See -disto_boodcmd_ for details. This is done because U-Boot has no native support +distro_bootcmd_ for details. This is done because U-Boot has no native support for scanning devices. While the scripts work remarkably well, they can be hard to understand and extend, and the feature does not include tests. They are also making it difficult to move away from ad-hoc CONFIGs, since they are implemented @@ -52,7 +52,7 @@ BootLoaderSpec_ format. which looks something like this:: initrd /initramfs-5.3.7-301.fc31.armv7hl.img As you can see it specifies a kernel, a ramdisk (initrd) and a directory from -which to load devicetree files. The details are described in disto_boodcmd_. +which to load devicetree files. The details are described in distro_bootcmd_. The bootflow is provided by the distro. It is not part of U-Boot. U-Boot's job is simply to interpret the file and carry out the instructions. This allows @@ -669,7 +669,7 @@ Other ideas: not need to specify things like `pxefile_addr_r` -.. _disto_boodcmd: https://github.com/u-boot/u-boot/blob/master/include/config_distro_bootcmd.h +.. _distro_bootcmd: https://github.com/u-boot/u-boot/blob/master/include/config_distro_bootcmd.h .. _BootLoaderSpec: http://www.freedesktop.org/wiki/Specifications/BootLoaderSpec/ .. _distro_boot: https://github.com/u-boot/u-boot/blob/master/boot/distro.c .. _bootflow_h: https://github.com/u-boot/u-boot/blob/master/include/bootflow.h diff --git a/doc/develop/codingstyle.rst b/doc/develop/codingstyle.rst index a6bc37b..1d5d019 100644 --- a/doc/develop/codingstyle.rst +++ b/doc/develop/codingstyle.rst @@ -248,5 +248,5 @@ will be much faster (10-100x or more) if they can directly call run_command() and ut_check_console_line() instead of using Python to send commands over a pipe to U-Boot. -Tests run all supported CI systems (gitlab, travis, azure) using scripts in the -root of the U-Boot tree. +Tests run all supported CI systems (GitLab, Azure) using scripts in the root of +the U-Boot tree. diff --git a/doc/develop/crash_dumps.rst b/doc/develop/crash_dumps.rst index 85030b4..4237b07 100644 --- a/doc/develop/crash_dumps.rst +++ b/doc/develop/crash_dumps.rst @@ -12,8 +12,8 @@ Creating a crash dump voluntarily --------------------------------- For describing the analysis of a crash dump we need an example. U-Boot comes -with a command 'exception' that comes in handy here. The command is enabled -by:: +with a command :doc:`exception <../usage/cmd/exception>` that comes in handy +here. The command is enabled by:: CONFIG_CMD_EXCEPTION=y @@ -122,3 +122,63 @@ If we want to dive deeper, we can disassemble the U-Boot binary:: This example is based on the ARMv8 architecture but the same procedures can be used on other architectures as well. + +Crashs in UEFI binaries +----------------------- + +If UEFI images are loaded when a crash occurs, their load addresses are +displayed. If the process counter points to an address in a loaded UEFI +binary, the relative process counter position is indicated. Here is an +example executed on the U-Boot sandbox:: + + => load host 0:1 $kernel_addr_r buggy.efi + 5632 bytes read in 0 ms + => bootefi $kernel_addr_r + Booting /buggy.efi + Buggy world! + + Segmentation violation + pc = 0x19fc264c, pc_reloc = 0xffffaa4688b1664c + + UEFI image [0x0000000019fc0000:0x0000000019fc6137] pc=0x264c '/buggy.efi' + +The crash occured in UEFI binary buggy.efi at relative position 0x264c. +Disassembly may be used to find the actual source code location:: + + $ x86_64-linux-gnu-objdump -S -D buggy_efi.so + + 0000000000002640 <memset>: + 2640: f3 0f 1e fa endbr64 + 2644: 48 89 f8 mov %rdi,%rax + 2647: 48 89 f9 mov %rdi,%rcx + 264a: eb 0b jmp 2657 <memset+0x17> + 264c: 40 88 31 mov %sil,(%rcx) + +Architecture specific details +----------------------------- + +ARMv8 +~~~~~ + +On the ARM 64-bit architecture CONFIG_ARMV8_SPL_EXCEPTION_VECTORS controls +if the exception vector tables are set up in the Secondary Program Loader (SPL). +Without initialization of the tables crash dumps cannot be shown. The feature is +disabled by default on most boards to reduce the size of the SPL. + +RISC-V +~~~~~~ + +On the RISC-V architecture CONFIG_SHOW_REGS=y has to be specified to show +all registers in crash dumps. + +Sandbox +~~~~~~~ + +The sandbox U-Boot binary must be invoked with parameter *-S* to display crash +dumps: + +.. code-block:: bash + + ./u-boot -S -T + +Only with CONFIG_SANDBOX_CRASH_RESET=y the sandbox reboots after a crash. diff --git a/doc/develop/index.rst b/doc/develop/index.rst index c94c7fe..f7ee09d 100644 --- a/doc/develop/index.rst +++ b/doc/develop/index.rst @@ -14,6 +14,7 @@ General process release_cycle system_configuration + sending_patches Implementation -------------- diff --git a/doc/develop/patman.rst b/doc/develop/patman.rst new file mode 120000 index 0000000..0fcb7d6 --- /dev/null +++ b/doc/develop/patman.rst @@ -0,0 +1 @@ +../../tools/patman/patman.rst
\ No newline at end of file diff --git a/doc/develop/process.rst b/doc/develop/process.rst index 0ef24e8..388945c 100644 --- a/doc/develop/process.rst +++ b/doc/develop/process.rst @@ -139,6 +139,10 @@ like this: as the Linux kernel. Following this and adding a ``Signed-off-by:`` line that contains the developer's name and email address is required. + * Please note that when importing code from other projects you must say + where it comes from, and what revision you are importing. You must not + however copy ``Signed-off-by`` or other tags. + #. Everybody who can is invited to review and test the changes. Typically, we follow the same guidelines as the Linux kernel for `Acked-by <https://www.kernel.org/doc/html/latest/process/submitting-patches.html#when-to-use-acked-by-cc-and-co-developed-by>`_ diff --git a/doc/develop/sending_patches.rst b/doc/develop/sending_patches.rst new file mode 100644 index 0000000..0542ade --- /dev/null +++ b/doc/develop/sending_patches.rst @@ -0,0 +1,16 @@ +.. SPDX-License-Identifier: GPL-2.0+ + +Sending patches +=============== + +.. toctree:: + :maxdepth: 2 + + patman + + +You can use a tool called patman to prepare, check and sent patches. It creates +change logs, cover letters and patch notes. It also simplified the process of +sending multiple versions of a series. + +See more details at :doc:`patman`. diff --git a/doc/git-mailrc b/doc/git-mailrc index b00c278..96623c3 100644 --- a/doc/git-mailrc +++ b/doc/git-mailrc @@ -111,7 +111,6 @@ alias x86 uboot, sjg, bmeng alias dm uboot, sjg alias cfi uboot, stroese alias dfu uboot, lukma -alias efi uboot, agraf alias eth uboot, jhersh alias kerneldoc uboot, marex alias fdt uboot, sjg diff --git a/doc/usage/cmd/gpio.rst b/doc/usage/cmd/gpio.rst new file mode 100644 index 0000000..f6a5668 --- /dev/null +++ b/doc/usage/cmd/gpio.rst @@ -0,0 +1,90 @@ +.. SPDX-License-Identifier: GPL-2.0+: + +gpio command +============ + +Synopsis +-------- + +:: + + gpio <input|set|clear|toggle> <pin> + gpio read <name> <pin> + gpio status [-a] [<bank>|<pin>] + +The gpio command is used to access General Purpose Inputs/Outputs. + +gpio input +---------- + +Switch the GPIO *pin* to input mode. + +gpio set +-------- + +Switch the GPIO *pin* to output mode and set the signal to 1. + +gpio clear +---------- + +Switch the GPIO *pin* to output mode and set the signal to 0. + +gpio toggle +----------- + +Switch the GPIO *pin* to output mode and reverse the signal state. + +gpio read +--------- + +Read the signal state of the GPIO *pin* and save it in environment variable +*name*. + +gpio status +----------- + +Display the status of one or multiple GPIOs. By default only claimed GPIOs +are displayed. + +-a + Display GPIOs irrespective of being claimed. + +bank + Name of a bank of GPIOs to be displayed. + +pin + Name of a single GPIO to be displayed or manipulated. + +Examples +-------- + +Switch the status of a GPIO:: + + => gpio set a5 + gpio: pin a5 (gpio 133) value is 1 + => gpio clear a5 + gpio: pin a5 (gpio 133) value is 0 + => gpio toggle a5 + gpio: pin a5 (gpio 133) value is 1 + => gpio read myvar a5 + gpio: pin a5 (gpio 133) value is 1 + => echo $myvar + 1 + => gpio toggle a5 + gpio: pin a5 (gpio 133) value is 0 + => gpio read myvar a5 + gpio: pin a5 (gpio 133) value is 0 + => echo $myvar + 0 + +Configuration +------------- + +The *gpio* command is only available if CONFIG_CMD_GPIO=y. +The *gpio read* command is only available if CONFIG_CMD_GPIO_READ=y. + +Return value +------------ + +If the command succeds the return value $? is set to 0. If an error occurs, the +return value $? is set to 1. diff --git a/doc/usage/index.rst b/doc/usage/index.rst index 2ba8733..28f9683 100644 --- a/doc/usage/index.rst +++ b/doc/usage/index.rst @@ -45,6 +45,7 @@ Shell commands cmd/fatload cmd/fdt cmd/for + cmd/gpio cmd/load cmd/loadm cmd/loady diff --git a/fs/erofs/super.c b/fs/erofs/super.c index 4cca322..8277d9b 100644 --- a/fs/erofs/super.c +++ b/fs/erofs/super.c @@ -65,14 +65,14 @@ int erofs_read_superblock(void) ret = erofs_blk_read(data, 0, 1); if (ret < 0) { - erofs_err("cannot read erofs superblock: %d", ret); + erofs_dbg("cannot read erofs superblock: %d", ret); return -EIO; } dsb = (struct erofs_super_block *)(data + EROFS_SUPER_OFFSET); ret = -EINVAL; if (le32_to_cpu(dsb->magic) != EROFS_SUPER_MAGIC_V1) { - erofs_err("cannot find valid erofs superblock"); + erofs_dbg("cannot find valid erofs superblock"); return ret; } diff --git a/include/efi_loader.h b/include/efi_loader.h index 3a63a1f..b0d6fff 100644 --- a/include/efi_loader.h +++ b/include/efi_loader.h @@ -375,6 +375,7 @@ enum efi_object_type { * @protocols: linked list with the protocol interfaces installed on this * handle * @type: image type if the handle relates to an image + * @dev: pointer to the DM device which is associated with this EFI handle * * UEFI offers a flexible and expandable object model. The objects in the UEFI * API are devices, drivers, and loaded images. struct efi_object is our storage @@ -392,6 +393,7 @@ struct efi_object { /* The list of protocols */ struct list_head protocols; enum efi_object_type type; + struct udevice *dev; }; enum efi_image_auth_status { @@ -690,6 +692,8 @@ struct efi_device_path *efi_get_dp_from_boot(const efi_guid_t guid); const char *guid_to_sha_str(const efi_guid_t *guid); int algo_to_len(const char *algo); +int efi_link_dev(efi_handle_t handle, struct udevice *dev); + /** * efi_size_in_pages() - convert size in bytes to size in pages * diff --git a/lib/efi_driver/efi_block_device.c b/lib/efi_driver/efi_block_device.c index 5baa6f8..d57d281 100644 --- a/lib/efi_driver/efi_block_device.c +++ b/lib/efi_driver/efi_block_device.c @@ -158,8 +158,7 @@ static int efi_bl_bind(efi_handle_t handle, void *interface) * FIXME: necessary because we won't do almost nothing in * efi_disk_create() when called from device_probe(). */ - ret = dev_tag_set_ptr(bdev, DM_TAG_EFI, handle); - if (ret) + if (efi_link_dev(handle, bdev)) /* FIXME: cleanup for bdev */ return ret; diff --git a/lib/efi_loader/efi_device_path_to_text.c b/lib/efi_loader/efi_device_path_to_text.c index 4d73954..6c428ee 100644 --- a/lib/efi_loader/efi_device_path_to_text.c +++ b/lib/efi_loader/efi_device_path_to_text.c @@ -432,6 +432,7 @@ static uint16_t EFIAPI *efi_convert_device_path_to_text( *(u8 **)&device_path += device_path->length; } + *str = 0; text = efi_str_to_u16(buffer); out: diff --git a/lib/efi_loader/efi_disk.c b/lib/efi_loader/efi_disk.c index 1d700b2..16d14b0 100644 --- a/lib/efi_loader/efi_disk.c +++ b/lib/efi_loader/efi_disk.c @@ -46,7 +46,6 @@ struct efi_disk_obj { struct efi_device_path *dp; unsigned int part; struct efi_simple_file_system_protocol *volume; - struct udevice *dev; /* TODO: move it to efi_object */ }; /** @@ -124,16 +123,16 @@ static efi_status_t efi_disk_rw_blocks(struct efi_block_io *this, return EFI_BAD_BUFFER_SIZE; if (CONFIG_IS_ENABLED(PARTITIONS) && - device_get_uclass_id(diskobj->dev) == UCLASS_PARTITION) { + device_get_uclass_id(diskobj->header.dev) == UCLASS_PARTITION) { if (direction == EFI_DISK_READ) - n = dev_read(diskobj->dev, lba, blocks, buffer); + n = dev_read(diskobj->header.dev, lba, blocks, buffer); else - n = dev_write(diskobj->dev, lba, blocks, buffer); + n = dev_write(diskobj->header.dev, lba, blocks, buffer); } else { /* dev is a block device (UCLASS_BLK) */ struct blk_desc *desc; - desc = dev_get_uclass_plat(diskobj->dev); + desc = dev_get_uclass_plat(diskobj->header.dev); if (direction == EFI_DISK_READ) n = blk_dread(desc, lba, blocks, buffer); else @@ -552,8 +551,7 @@ static int efi_disk_create_raw(struct udevice *dev) return -1; } - disk->dev = dev; - if (dev_tag_set_ptr(dev, DM_TAG_EFI, &disk->header)) { + if (efi_link_dev(&disk->header, dev)) { efi_free_pool(disk->dp); efi_delete_handle(&disk->header); @@ -609,8 +607,7 @@ static int efi_disk_create_part(struct udevice *dev) log_err("Adding partition for %s failed\n", dev->name); return -1; } - disk->dev = dev; - if (dev_tag_set_ptr(dev, DM_TAG_EFI, &disk->header)) { + if (efi_link_dev(&disk->header, dev)) { efi_free_pool(disk->dp); efi_delete_handle(&disk->header); diff --git a/lib/efi_loader/efi_helper.c b/lib/efi_loader/efi_helper.c index c4499f6..8ed564e 100644 --- a/lib/efi_loader/efi_helper.c +++ b/lib/efi_loader/efi_helper.c @@ -158,3 +158,16 @@ int algo_to_len(const char *algo) return 0; } + +/** efi_link_dev - link the efi_handle_t and udevice + * + * @handle: efi handle to associate with udevice + * @dev: udevice to associate with efi handle + * + * Return: 0 on success, negative on failure + */ +int efi_link_dev(efi_handle_t handle, struct udevice *dev) +{ + handle->dev = dev; + return dev_tag_set_ptr(dev, DM_TAG_EFI, handle); +} diff --git a/tools/binman/binman.rst b/tools/binman/binman.rst index 935839c..6e1fd34 100644 --- a/tools/binman/binman.rst +++ b/tools/binman/binman.rst @@ -42,7 +42,7 @@ the devicetree description of the image. Binman is designed primarily for use with U-Boot and associated binaries such as ARM Trusted Firmware, but it is suitable for use with other projects, such as Zephyr. Binman also provides facilities useful in Chromium OS, such as CBFS, -vblocks and and the like. +vblocks and the like. Binman provides a way to process binaries before they are included, by adding a Python plug-in. @@ -118,6 +118,10 @@ flash. For U-Boot, binman should not be used to create ad-hoc images in place of FIT. +Note that binman can itself create a FIT. This helps to move mkimage +invocations out of the Makefile and into binman image descriptions. It also +helps by removing the need for ad-hoc tools like `make_fit_atf.py`. + Relationship to mkimage ----------------------- @@ -140,6 +144,9 @@ seems better to use the mkimage tool to generate binaries and avoid blurring the boundaries between building input files (mkimage) and packaging then into a final image (binman). +Note that binman can itself invoke mkimage. This helps to move mkimage +invocations out of the Makefile and into binman image descriptions. + Using binman ============ @@ -170,6 +177,164 @@ This simplifies the U-Boot Makefile somewhat, since various pieces of logic can be replaced by a call to binman. +Invoking binman within U-Boot +----------------------------- + +Within U-Boot, binman is invoked by the build system, i.e. when you type 'make' +or use buildman to build U-Boot. There is no need to run binman independently +during development. Everything happens automatically and is set up for your +SoC or board so that binman produced the right things. + +The general policy is that the Makefile builds all the binaries in INPUTS-y +(the 'inputs' rule), then binman is run to produce the final images (the 'all' +rule). + +There should be only one invocation of binman in Makefile, the very last step +that pulls everything together. At present there are some arch-specific +invocations as well, but these should be dropped when those architectures are +converted to use binman properly. + +As above, the term 'binary' is used for something in INPUTS-y and 'image' is +used for the things that binman creates. So the binaries are inputs to the +image(s) and it is the image that is actually loaded on the board. + +Again, at present, there are a number of things created in Makefile which should +be done by binman (when we get around to it), like `u-boot-ivt.img`, +`lpc32xx-spl.img`, `u-boot-with-nand-spl.imx`, `u-boot-spl-padx4.sfp` and +`u-boot-mtk.bin`, just to pick on a few. When completed this will remove about +400 lines from `Makefile`. + +Since binman is invoked only once, it must of course create all the images that +are needed, in that one invocation. It does this by working through the image +descriptions one by one, collecting the input binaries, processing them as +needed and producing the final images. + +The same binaries may be used by multiple images. For example binman may be used +to produce an SD-card image and a SPI-flash image. In this case the binaries +going into the process are the same, but binman produces slightly different +images in each case. + +For some SoCs, U-Boot is not the only project that produces the necessary +binaries. For example, ARM Trusted Firmware (ATF) is a project that produces +binaries which must be incorporate, such as `bl31.elf` or `bl31.bin`. For this +to work you must have built ATF before you build U-Boot and you must tell U-Boot +where to find the bl31 image, using the BL31 environment variable. + +How do you know how to incorporate ATF? It is handled by the atf-bl31 entry type +(etype). An etype is an implementation of reading a binary into binman, in this +case the `bl31.bin` file. When you build U-Boot but do not set the BL31 +environment variable, binman provides a help message, which comes from +`missing-blob-help`:: + + See the documentation for your board. You may need to build ARM Trusted + Firmware and build with BL31=/path/to/bl31.bin + +The mechanism by which binman is advised of this is also in the Makefile. See +the `-a atf-bl31-path=${BL31}` piece in `cmd_binman`. This tells binman to +set the EntryArg `atf-bl31-path` to the value of the `BL31` environment +variable. Within binman, this EntryArg is picked up by the `Entry_atf_bl31` +etype. An EntryArg is simply an argument to the entry. The `atf-bl31-path` +name is documented in :ref:`etype_atf_bl31`. + + +Invoking binman outside U-Boot +------------------------------ + +While binman is invoked from within the U-Boot build system, it is also possible +to invoke it separately. This is typically used in a production build system, +where signing is completed (with real keys) and any missing binaries are +provided. + +For example, for build testing there is no need to provide a real signature, +nor is there any need to provide a real ATF BL31 binary (for example). These can +be added later by invoking binman again, providing all the required inputs +from the first time, plus any that were missing or placeholders. + +So in practice binman is often used twice: + +- once within the U-Boot build system, for development and testing +- again outside U-Boot to assembly and final production images + +While the same input binaries are used in each case, you will of course you will +need to create your own binman command line, similar to that in `cmd_binman` in +the Makefile. You may find the -I and --toolpath options useful. The +device tree file is provided to binman in binary form, so there is no need to +have access to the original `.dts` sources. + + +Assembling the image description +-------------------------------- + +Since binman uses the device tree for its image description, you can use the +same files that describe your board's hardware to describe how the image is +assembled. Typically the images description is in a common file used by all +boards with a particular SoC (e.g. `imx8mp-u-boot.dtsi`). + +Where a particular boards needs to make changes, it can override properties in +the SoC file, just as it would for any other device tree property. It can also +add a image that is specific to the board. + +Another way to control the image description to make use of CONFIG options in +the description. For example, if the start offset of a particular entry varies +by board, you can add a Kconfig for that and reference it in the description:: + + u-boot-spl { + }; + + fit { + offset = <CONFIG_SPL_PAD_TO>; + ... + }; + +The SoC can provide a default value but boards can override that as needed and +binman will take care of it. + +It is even possible to control which entries appear in the image, by using the +C preprocessor:: + + #ifdef CONFIG_HAVE_MRC + intel-mrc { + offset = <CONFIG_X86_MRC_ADDR>; + }; + #endif + +Only boards which enable `HAVE_MRC` will include this entry. + +Obviously a similar approach can be used to control which images are produced, +with a Kconfig option to enable a SPI image, for example. However there is +generally no harm in producing an image that is not used. If a board uses MMC +but not SPI, but the SoC supports booting from both, then both images can be +produced, with only on or other being used by particular boards. This can help +reduce the need for having multiple defconfig targets for a board where the +only difference is the boot media, enabling / disabling secure boot, etc. + +Of course you can use the device tree itself to pass any board-specific +information that is needed by U-Boot at runtime (see binman_syms_ for how to +make binman insert these values directly into executables like SPL). + +There is one more way this can be done: with individual .dtsi files for each +image supported by the SoC. Then the board `.dts` file can include the ones it +wants. This is not recommended, since it is likely to be difficult to maintain +and harder to understand the relationship between the different boards. + + +Producing images for multiple boards +------------------------------------ + +When invoked within U-Boot, binman only builds a single set of images, for +the chosen board. This is set by the `CONFIG_DEFAULT_DEVICE_TREE` option. + +However, U-Boot generally builds all the device tree files associated with an +SoC. These are written to the (e.g. for ARM) `arch/arm/dts` directory. Each of +these contains the full binman description for that board. Often the best +approach is to build a single image that includes all these device tree binaries +and allow SPL to select the correct one on boot. + +However, it is also possible to build separate images for each board, simply by +invoking binman multiple times, once for each device tree file, using a +different output directory. This will produce one set of images for each board. + + Example use of binman for x86 ----------------------------- @@ -188,19 +353,25 @@ the configuration of the Intel-format descriptor. Installing binman ----------------- -First install prerequisites, e.g:: +First install prerequisites, e.g: + +.. code-block:: bash sudo apt-get install python-pyelftools python3-pyelftools lzma-alone \ liblz4-tool You can run binman directly if you put it on your PATH. But if you want to -install into your `~/.local` Python directory, use:: +install into your `~/.local` Python directory, use: + +.. code-block:: bash pip install tools/patman tools/dtoc tools/binman Note that binman makes use of libraries from patman and dtoc, which is why these need to be installed. Also you need `libfdt` and `pylibfdt` which can be -installed like this:: +installed like this: + +.. code-block:: bash git clone git://git.kernel.org/pub/scm/utils/dtc/dtc.git cd dtc @@ -209,7 +380,9 @@ installed like this:: This installs the `libfdt.so` library into `~/lib` so you can use `LD_LIBRARY_PATH=~/lib` when running binman. If you want to install it in the -system-library directory, replace the last line with:: +system-library directory, replace the last line with: + +.. code-block:: bash make NO_PYTHON=1 PREFIX=/ install @@ -218,14 +391,20 @@ Running binman Type:: +.. code-block: bash + + make NO_PYTHON=1 PREFIX=/ install binman build -b <board_name> to build an image for a board. The board name is the same name used when configuring U-Boot (e.g. for sandbox_defconfig the board name is 'sandbox'). Binman assumes that the input files for the build are in ../b/<board_name>. -Or you can specify this explicitly:: +Or you can specify this explicitly: +.. code-block:: bash + + make NO_PYTHON=1 PREFIX=/ install binman build -I <build_path> where <build_path> is the build directory containing the output of the U-Boot @@ -254,6 +433,7 @@ file, typically <soc>-u-boot.dtsi, where <soc> is your CONFIG_SYS_SOC value. You can use other, more specific CONFIG options - see 'Automatic .dtsi inclusion' below. +.. _binman_syms: Access to binman entry offsets at run time (symbols) ---------------------------------------------------- @@ -264,13 +444,17 @@ is useful to be able to find the location of U-Boot so that it can be executed when SPL is finished. Binman allows you to declare symbols in the SPL image which are filled in -with their correct values during the build. For example:: +with their correct values during the build. For example: + +.. code-block:: c binman_sym_declare(ulong, u_boot_any, image_pos); declares a ulong value which will be assigned to the image-pos of any U-Boot image (u-boot.bin, u-boot.img, u-boot-nodtb.bin) that is present in the image. -You can access this value with something like:: +You can access this value with something like: + +.. code-block:: c ulong u_boot_offset = binman_sym(ulong, u_boot_any, image_pos); diff --git a/tools/binman/entries.rst b/tools/binman/entries.rst index ae4305c..782bff1 100644 --- a/tools/binman/entries.rst +++ b/tools/binman/entries.rst @@ -11,6 +11,8 @@ features to produce new behaviours. +.. _etype_atf_bl31: + Entry: atf-bl31: ARM Trusted Firmware (ATF) BL31 blob ----------------------------------------------------- @@ -25,6 +27,8 @@ about ATF. +.. _etype_atf_fip: + Entry: atf-fip: ARM Trusted Firmware's Firmware Image Package (FIP) ------------------------------------------------------------------- @@ -179,6 +183,8 @@ FIPs so that binman and other tools can access the entire image correctly. +.. _etype_blob: + Entry: blob: Arbitrary binary blob ---------------------------------- @@ -201,6 +207,8 @@ data. +.. _etype_blob_dtb: + Entry: blob-dtb: A blob that holds a device tree ------------------------------------------------ @@ -210,6 +218,8 @@ obtained from the list of available device-tree files, managed by the +.. _etype_blob_ext: + Entry: blob-ext: Externally built binary blob --------------------------------------------- @@ -223,6 +233,8 @@ See 'blob' for Properties / Entry arguments. +.. _etype_blob_ext_list: + Entry: blob-ext-list: List of externally built binary blobs ----------------------------------------------------------- @@ -237,6 +249,8 @@ Args: +.. _etype_blob_named_by_arg: + Entry: blob-named-by-arg: A blob entry which gets its filename property from its subclass ----------------------------------------------------------------------------------------- @@ -255,6 +269,8 @@ See cros_ec_rw for an example of this. +.. _etype_blob_phase: + Entry: blob-phase: Section that holds a phase binary ---------------------------------------------------- @@ -264,6 +280,8 @@ entry; similarly for SPL. +.. _etype_cbfs: + Entry: cbfs: Coreboot Filesystem (CBFS) --------------------------------------- @@ -416,6 +434,8 @@ both of size 1MB. +.. _etype_collection: + Entry: collection: An entry which contains a collection of other entries ------------------------------------------------------------------------ @@ -429,6 +449,8 @@ the image, not necessarily child entries. +.. _etype_cros_ec_rw: + Entry: cros-ec-rw: A blob entry which contains a Chromium OS read-write EC image -------------------------------------------------------------------------------- @@ -440,6 +462,8 @@ updating the EC on startup via software sync. +.. _etype_fdtmap: + Entry: fdtmap: An entry which contains an FDT map ------------------------------------------------- @@ -488,6 +512,8 @@ without the fdtmap header, so it can be viewed with `fdtdump`. +.. _etype_files: + Entry: files: A set of files arranged in a section -------------------------------------------------- @@ -504,6 +530,8 @@ at run-time so you can obtain the file positions. +.. _etype_fill: + Entry: fill: An entry which is filled to a particular byte value ---------------------------------------------------------------- @@ -520,6 +548,8 @@ byte value of a region. +.. _etype_fit: + Entry: fit: Flat Image Tree (FIT) --------------------------------- @@ -803,6 +833,8 @@ U-Boot SPL can then load the firmware (U-Boot proper) and all the loadables +.. _etype_fmap: + Entry: fmap: An entry which contains an Fmap section ---------------------------------------------------- @@ -828,6 +860,8 @@ CBFS entries appear as a single entry, i.e. the sub-entries are ignored. +.. _etype_gbb: + Entry: gbb: An entry which contains a Chromium OS Google Binary Block --------------------------------------------------------------------- @@ -847,6 +881,8 @@ README.chromium for how to obtain the required keys and tools. +.. _etype_image_header: + Entry: image-header: An entry which contains a pointer to the FDT map --------------------------------------------------------------------- @@ -866,6 +902,8 @@ first/last in the entry list. +.. _etype_intel_cmc: + Entry: intel-cmc: Intel Chipset Micro Code (CMC) file ----------------------------------------------------- @@ -879,6 +917,8 @@ See README.x86 for information about x86 binary blobs. +.. _etype_intel_descriptor: + Entry: intel-descriptor: Intel flash descriptor block (4KB) ----------------------------------------------------------- @@ -900,6 +940,8 @@ See README.x86 for information about x86 binary blobs. +.. _etype_intel_fit: + Entry: intel-fit: Intel Firmware Image Table (FIT) -------------------------------------------------- @@ -911,6 +953,8 @@ At present binman only supports a basic FIT with no microcode. +.. _etype_intel_fit_ptr: + Entry: intel-fit-ptr: Intel Firmware Image Table (FIT) pointer -------------------------------------------------------------- @@ -919,6 +963,8 @@ This entry contains a pointer to the FIT. It is required to be at address +.. _etype_intel_fsp: + Entry: intel-fsp: Intel Firmware Support Package (FSP) file ----------------------------------------------------------- @@ -936,6 +982,8 @@ See README.x86 for information about x86 binary blobs. +.. _etype_intel_fsp_m: + Entry: intel-fsp-m: Intel Firmware Support Package (FSP) memory init -------------------------------------------------------------------- @@ -953,6 +1001,8 @@ See README.x86 for information about x86 binary blobs. +.. _etype_intel_fsp_s: + Entry: intel-fsp-s: Intel Firmware Support Package (FSP) silicon init --------------------------------------------------------------------- @@ -970,6 +1020,8 @@ See README.x86 for information about x86 binary blobs. +.. _etype_intel_fsp_t: + Entry: intel-fsp-t: Intel Firmware Support Package (FSP) temp ram init ---------------------------------------------------------------------- @@ -986,6 +1038,8 @@ See README.x86 for information about x86 binary blobs. +.. _etype_intel_ifwi: + Entry: intel-ifwi: Intel Integrated Firmware Image (IFWI) file -------------------------------------------------------------- @@ -1020,6 +1074,8 @@ See README.x86 for information about x86 binary blobs. +.. _etype_intel_me: + Entry: intel-me: Intel Management Engine (ME) file -------------------------------------------------- @@ -1040,6 +1096,8 @@ See README.x86 for information about x86 binary blobs. +.. _etype_intel_mrc: + Entry: intel-mrc: Intel Memory Reference Code (MRC) file -------------------------------------------------------- @@ -1054,6 +1112,8 @@ See README.x86 for information about x86 binary blobs. +.. _etype_intel_refcode: + Entry: intel-refcode: Intel Reference Code file ----------------------------------------------- @@ -1068,6 +1128,8 @@ See README.x86 for information about x86 binary blobs. +.. _etype_intel_vbt: + Entry: intel-vbt: Intel Video BIOS Table (VBT) file --------------------------------------------------- @@ -1081,6 +1143,8 @@ See README.x86 for information about Intel binary blobs. +.. _etype_intel_vga: + Entry: intel-vga: Intel Video Graphics Adaptor (VGA) file --------------------------------------------------------- @@ -1096,6 +1160,8 @@ See README.x86 for information about Intel binary blobs. +.. _etype_mkimage: + Entry: mkimage: Binary produced by mkimage ------------------------------------------ @@ -1130,6 +1196,8 @@ this example which also produces four arguments:: +.. _etype_opensbi: + Entry: opensbi: RISC-V OpenSBI fw_dynamic blob ---------------------------------------------- @@ -1143,6 +1211,8 @@ https://github.com/riscv/opensbi for more information about OpenSBI. +.. _etype_powerpc_mpc85xx_bootpg_resetvec: + Entry: powerpc-mpc85xx-bootpg-resetvec: PowerPC mpc85xx bootpg + resetvec code for U-Boot ----------------------------------------------------------------------------------------- @@ -1155,11 +1225,13 @@ placed at offset 'RESET_VECTOR_ADDRESS - 0xffc'. +.. _etype_pre_load: + Entry: pre-load: Pre load image header -------------------------------------- Properties / Entry arguments: - - key-path: Path of the directory that store key (provided by the environment variable KEY_PATH) + - pre-load-key-path: Path of the directory that store key (provided by the environment variable PRE_LOAD_KEY_PATH) - content: List of phandles to entries to sign - algo-name: Hash and signature algo to use for the signature - padding-name: Name of the padding (pkcs-1.5 or pss) @@ -1193,6 +1265,8 @@ For example, this creates an image with a pre-load header and a binary:: +.. _etype_scp: + Entry: scp: System Control Processor (SCP) firmware blob -------------------------------------------------------- @@ -1203,6 +1277,8 @@ This entry holds firmware for an external platform-specific coprocessor. +.. _etype_section: + Entry: section: Entry that contains other entries ------------------------------------------------- @@ -1338,6 +1414,8 @@ available. This is set by the `SetAllowMissing()` method, if +.. _etype_tee_os: + Entry: tee-os: Entry containing an OP-TEE Trusted OS (TEE) blob --------------------------------------------------------------- @@ -1351,6 +1429,8 @@ https://github.com/OP-TEE/optee_os for more information about OP-TEE. +.. _etype_text: + Entry: text: An entry which contains text ----------------------------------------- @@ -1399,6 +1479,8 @@ by setting the size of the entry to something larger than the text. +.. _etype_u_boot: + Entry: u-boot: U-Boot flat binary --------------------------------- @@ -1420,6 +1502,8 @@ Note that this entry is automatically replaced with u-boot-expanded unless +.. _etype_u_boot_dtb: + Entry: u-boot-dtb: U-Boot device tree ------------------------------------- @@ -1435,6 +1519,8 @@ binman to know which entries contain a device tree. +.. _etype_u_boot_dtb_with_ucode: + Entry: u-boot-dtb-with-ucode: A U-Boot device tree file, with the microcode removed ----------------------------------------------------------------------------------- @@ -1451,6 +1537,8 @@ it available to u-boot-ucode. +.. _etype_u_boot_elf: + Entry: u-boot-elf: U-Boot ELF image ----------------------------------- @@ -1462,6 +1550,8 @@ relocated to any address for execution. +.. _etype_u_boot_env: + Entry: u-boot-env: An entry which contains a U-Boot environment --------------------------------------------------------------- @@ -1471,6 +1561,8 @@ Properties / Entry arguments: +.. _etype_u_boot_expanded: + Entry: u-boot-expanded: U-Boot flat binary broken out into its component parts ------------------------------------------------------------------------------ @@ -1486,6 +1578,8 @@ image, so that the entries positions are provided to the running U-Boot. +.. _etype_u_boot_img: + Entry: u-boot-img: U-Boot legacy image -------------------------------------- @@ -1500,6 +1594,8 @@ applications. +.. _etype_u_boot_nodtb: + Entry: u-boot-nodtb: U-Boot flat binary without device tree appended -------------------------------------------------------------------- @@ -1514,6 +1610,8 @@ section containing u-boot and u-boot-dtb +.. _etype_u_boot_spl: + Entry: u-boot-spl: U-Boot SPL binary ------------------------------------ @@ -1541,6 +1639,8 @@ unless --no-expanded is used or the node has a 'no-expanded' property. +.. _etype_u_boot_spl_bss_pad: + Entry: u-boot-spl-bss-pad: U-Boot SPL binary padded with a BSS region --------------------------------------------------------------------- @@ -1563,6 +1663,8 @@ binman uses that to look up the BSS address. +.. _etype_u_boot_spl_dtb: + Entry: u-boot-spl-dtb: U-Boot SPL device tree --------------------------------------------- @@ -1575,6 +1677,8 @@ to activate. +.. _etype_u_boot_spl_elf: + Entry: u-boot-spl-elf: U-Boot SPL ELF image ------------------------------------------- @@ -1586,6 +1690,8 @@ be relocated to any address for execution. +.. _etype_u_boot_spl_expanded: + Entry: u-boot-spl-expanded: U-Boot SPL flat binary broken out into its component parts -------------------------------------------------------------------------------------- @@ -1609,6 +1715,8 @@ this is non-empty (and not 'n' or '0') then this expanded entry is selected. +.. _etype_u_boot_spl_nodtb: + Entry: u-boot-spl-nodtb: SPL binary without device tree appended ---------------------------------------------------------------- @@ -1633,6 +1741,8 @@ binman uses that to look up symbols to write into the SPL binary. +.. _etype_u_boot_spl_with_ucode_ptr: + Entry: u-boot-spl-with-ucode-ptr: U-Boot SPL with embedded microcode pointer ---------------------------------------------------------------------------- @@ -1643,6 +1753,8 @@ process. +.. _etype_u_boot_tpl: + Entry: u-boot-tpl: U-Boot TPL binary ------------------------------------ @@ -1670,6 +1782,8 @@ unless --no-expanded is used or the node has a 'no-expanded' property. +.. _etype_u_boot_tpl_bss_pad: + Entry: u-boot-tpl-bss-pad: U-Boot TPL binary padded with a BSS region --------------------------------------------------------------------- @@ -1692,6 +1806,8 @@ binman uses that to look up the BSS address. +.. _etype_u_boot_tpl_dtb: + Entry: u-boot-tpl-dtb: U-Boot TPL device tree --------------------------------------------- @@ -1704,6 +1820,8 @@ to activate. +.. _etype_u_boot_tpl_dtb_with_ucode: + Entry: u-boot-tpl-dtb-with-ucode: U-Boot TPL with embedded microcode pointer ---------------------------------------------------------------------------- @@ -1714,6 +1832,8 @@ process. +.. _etype_u_boot_tpl_elf: + Entry: u-boot-tpl-elf: U-Boot TPL ELF image ------------------------------------------- @@ -1725,6 +1845,8 @@ be relocated to any address for execution. +.. _etype_u_boot_tpl_expanded: + Entry: u-boot-tpl-expanded: U-Boot TPL flat binary broken out into its component parts -------------------------------------------------------------------------------------- @@ -1748,6 +1870,8 @@ this is non-empty (and not 'n' or '0') then this expanded entry is selected. +.. _etype_u_boot_tpl_nodtb: + Entry: u-boot-tpl-nodtb: TPL binary without device tree appended ---------------------------------------------------------------- @@ -1772,6 +1896,8 @@ binman uses that to look up symbols to write into the TPL binary. +.. _etype_u_boot_tpl_with_ucode_ptr: + Entry: u-boot-tpl-with-ucode-ptr: U-Boot TPL with embedded microcode pointer ---------------------------------------------------------------------------- @@ -1780,6 +1906,8 @@ process. +.. _etype_u_boot_ucode: + Entry: u-boot-ucode: U-Boot microcode block ------------------------------------------- @@ -1830,6 +1958,8 @@ Entry types that have a part to play in handling microcode: +.. _etype_u_boot_with_ucode_ptr: + Entry: u-boot-with-ucode-ptr: U-Boot with embedded microcode pointer -------------------------------------------------------------------- @@ -1846,6 +1976,8 @@ complicated. Otherwise it is the same as the u-boot entry. +.. _etype_vblock: + Entry: vblock: An entry which contains a Chromium OS verified boot block ------------------------------------------------------------------------ @@ -1869,6 +2001,8 @@ and kernel are genuine. +.. _etype_x86_reset16: + Entry: x86-reset16: x86 16-bit reset code for U-Boot ---------------------------------------------------- @@ -1885,6 +2019,8 @@ For 64-bit U-Boot, the 'x86_reset16_spl' entry type is used instead. +.. _etype_x86_reset16_spl: + Entry: x86-reset16-spl: x86 16-bit reset code for U-Boot -------------------------------------------------------- @@ -1901,6 +2037,8 @@ For 32-bit U-Boot, the 'x86_reset_spl' entry type is used instead. +.. _etype_x86_reset16_tpl: + Entry: x86-reset16-tpl: x86 16-bit reset code for U-Boot -------------------------------------------------------- @@ -1917,6 +2055,8 @@ For 32-bit U-Boot, the 'x86_reset_tpl' entry type is used instead. +.. _etype_x86_start16: + Entry: x86-start16: x86 16-bit start-up code for U-Boot ------------------------------------------------------- @@ -1935,6 +2075,8 @@ For 64-bit U-Boot, the 'x86_start16_spl' entry type is used instead. +.. _etype_x86_start16_spl: + Entry: x86-start16-spl: x86 16-bit start-up code for SPL -------------------------------------------------------- @@ -1953,6 +2095,8 @@ For 32-bit U-Boot, the 'x86-start16' entry type is used instead. +.. _etype_x86_start16_tpl: + Entry: x86-start16-tpl: x86 16-bit start-up code for TPL -------------------------------------------------------- diff --git a/tools/binman/entry.py b/tools/binman/entry.py index a07a588..e3767ae 100644 --- a/tools/binman/entry.py +++ b/tools/binman/entry.py @@ -750,6 +750,11 @@ features to produce new behaviours. first_line = lines[0] rest = [line[4:] for line in lines[1:]] hdr = 'Entry: %s: %s' % (name.replace('_', '-'), first_line) + + # Create a reference for use by rST docs + ref_name = f'etype_{module.__name__[6:]}'.lower() + print('.. _%s:' % ref_name) + print() print(hdr) print('-' * len(hdr)) print('\n'.join(rest)) diff --git a/tools/patman/README.rst b/tools/patman/README.rst new file mode 120000 index 0000000..76368b9 --- /dev/null +++ b/tools/patman/README.rst @@ -0,0 +1 @@ +patman.rst
\ No newline at end of file diff --git a/tools/patman/main.py b/tools/patman/main.py index 15e7af0..5a7756a 100755 --- a/tools/patman/main.py +++ b/tools/patman/main.py @@ -164,7 +164,8 @@ elif args.cmd == 'send': elif args.full_help: tools.print_full_help( - os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), 'README') + os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), + 'README.rst') ) else: diff --git a/tools/patman/README b/tools/patman/patman.rst index e3466e6..9226b66 100644 --- a/tools/patman/README +++ b/tools/patman/patman.rst @@ -1,19 +1,31 @@ -# SPDX-License-Identifier: GPL-2.0+ -# Copyright (c) 2011 The Chromium OS Authors. - -What is this? -============= +.. SPDX-License-Identifier: GPL-2.0+ +.. Copyright (c) 2011 The Chromium OS Authors +.. Simon Glass <sjg@chromium.org> +.. v1, v2, 19-Oct-11 +.. revised v3 24-Nov-11 +.. revised v4 04-Jul-2020, with Patchwork integration + +Patman patch manager +==================== This tool is a Python script which: + - Creates patch directly from your branch + - Cleans them up by removing unwanted tags + - Inserts a cover letter with change lists + - Runs the patches through checkpatch.pl and its own checks + - Optionally emails them out to selected people It also has some Patchwork features: + - shows review tags from Patchwork so you can update your local patches + - pulls these down into a new branch on request + - lists comments received on a series It is intended to automate patch creation and make it a less @@ -24,9 +36,9 @@ It is configured almost entirely by tags it finds in your commits. This means that you can work on a number of different branches at once, and keep the settings with each branch rather than having to git format-patch, git send-email, etc. with the correct parameters -each time. So for example if you put: +each time. So for example if you put:: -Series-to: fred.blogs@napier.co.nz + Series-to: fred.blogs@napier.co.nz in one of your commits, the series will be sent there. @@ -35,30 +47,33 @@ patches automatically (unless you use -m to disable this). How to use this tool -==================== +-------------------- This tool requires a certain way of working: - Maintain a number of branches, one for each patch series you are -working on + working on + - Add tags into the commits within each branch to indicate where the -series should be sent, cover letter, version, etc. Most of these are -normally in the top commit so it is easy to change them with 'git -commit --amend' + series should be sent, cover letter, version, etc. Most of these are + normally in the top commit so it is easy to change them with 'git + commit --amend' + - Each branch tracks the upstream branch, so that this script can -automatically determine the number of commits in it (optional) + automatically determine the number of commits in it (optional) + - Check out a branch, and run this script to create and send out your -patches. Weeks later, change the patches and repeat, knowing that you -will get a consistent result each time. + patches. Weeks later, change the patches and repeat, knowing that you + will get a consistent result each time. How to configure it -=================== +------------------- For most cases of using patman for U-Boot development, patman can use the file 'doc/git-mailrc' in your U-Boot directory to supply the email aliases you need. To make this work, tell git where to find the file by typing -this once: +this once:: git config sendemail.aliasesfile doc/git-mailrc @@ -68,19 +83,16 @@ out where to send patches pretty well. During the first run patman creates a config file for you by taking the default user name and email address from the global .gitconfig file. -To add your own, create a file ~/.patman like this: - ->>>> -# patman alias file +To add your own, create a file ~/.patman like this:: -[alias] -me: Simon Glass <sjg@chromium.org> + # patman alias file -u-boot: U-Boot Mailing List <u-boot@lists.denx.de> -wolfgang: Wolfgang Denk <wd@denx.de> -others: Mike Frysinger <vapier@gentoo.org>, Fred Bloggs <f.bloggs@napier.net> + [alias] + me: Simon Glass <sjg@chromium.org> -<<<< + u-boot: U-Boot Mailing List <u-boot@lists.denx.de> + wolfgang: Wolfgang Denk <wd@denx.de> + others: Mike Frysinger <vapier@gentoo.org>, Fred Bloggs <f.bloggs@napier.net> Aliases are recursive. @@ -90,263 +102,281 @@ used. Failing that you can put it into your path or ~/bin/checkpatch.pl If you want to avoid sending patches to email addresses that are picked up by patman but are known to bounce you can add a [bounces] section to your .patman file. Unlike the [alias] section these are simple key: value pairs -that are not recursive. - ->>> - -[bounces] -gonefishing: Fred Bloggs <f.bloggs@napier.net> +that are not recursive:: -<<< + [bounces] + gonefishing: Fred Bloggs <f.bloggs@napier.net> If you want to change the defaults for patman's command-line arguments, you can add a [settings] section to your .patman file. This can be used for any command line option by referring to the "dest" for the option in patman.py. For reference, the useful ones (at the moment) shown below -(all with the non-default setting): - ->>> - -[settings] -ignore_errors: True -process_tags: False -verbose: True -smtp_server: /path/to/sendmail -patchwork_server: https://patchwork.ozlabs.org - -<<< +(all with the non-default setting):: + [settings] + ignore_errors: True + process_tags: False + verbose: True + smtp_server: /path/to/sendmail + patchwork_server: https://patchwork.ozlabs.org If you want to adjust settings (or aliases) that affect just a single project you can add a section that looks like [project_settings] or -[project_alias]. If you want to use tags for your linux work, you could -do: - ->>> +[project_alias]. If you want to use tags for your linux work, you could do:: -[linux_settings] -process_tags: True - -<<< + [linux_settings] + process_tags: True How to run it -============= +------------- First do a dry run: -$ ./tools/patman/patman send -n +.. code-block:: bash + + ./tools/patman/patman send -n If it can't detect the upstream branch, try telling it how many patches -there are in your series: +there are in your series -$ ./tools/patman/patman -c5 send -n +.. code-block:: bash + + ./tools/patman/patman -c5 send -n This will create patch files in your current directory and tell you who -it is thinking of sending them to. Take a look at the patch files. +it is thinking of sending them to. Take a look at the patch files: + +.. code-block:: bash -$ ./tools/patman/patman -c5 -s1 send -n + ./tools/patman/patman -c5 -s1 send -n Similar to the above, but skip the first commit and take the next 5. This is useful if your top commit is for setting up testing. How to install it -================= +----------------- The most up to date version of patman can be found in the U-Boot sources. However to use it on other projects it may be more convenient to install it as a standalone application. A distutils installer is included, this can be used to install patman: -$ cd tools/patman && python setup.py install +.. code-block:: bash + + cd tools/patman && python setup.py install How to add tags -=============== +--------------- To make this script useful you must add tags like the following into any commit. Most can only appear once in the whole series. Series-to: email / alias - Email address / alias to send patch series to (you can add this - multiple times) + Email address / alias to send patch series to (you can add this + multiple times) Series-cc: email / alias, ... - Email address / alias to Cc patch series to (you can add this - multiple times) + Email address / alias to Cc patch series to (you can add this + multiple times) Series-version: n - Sets the version number of this patch series + Sets the version number of this patch series Series-prefix: prefix - Sets the subject prefix. Normally empty but it can be RFC for - RFC patches, or RESEND if you are being ignored. The patch subject - is like [RFC PATCH] or [RESEND PATCH]. - In the meantime, git format.subjectprefix option will be added as - well. If your format.subjectprefix is set to InternalProject, then - the patch shows like: [InternalProject][RFC/RESEND PATCH] + Sets the subject prefix. Normally empty but it can be RFC for + RFC patches, or RESEND if you are being ignored. The patch subject + is like [RFC PATCH] or [RESEND PATCH]. + In the meantime, git format.subjectprefix option will be added as + well. If your format.subjectprefix is set to InternalProject, then + the patch shows like: [InternalProject][RFC/RESEND PATCH] Series-postfix: postfix - Sets the subject "postfix". Normally empty, but can be the name of a - tree such as net or net-next if that needs to be specified. The patch - subject is like [PATCH net] or [PATCH net-next]. + Sets the subject "postfix". Normally empty, but can be the name of a + tree such as net or net-next if that needs to be specified. The patch + subject is like [PATCH net] or [PATCH net-next]. Series-name: name - Sets the name of the series. You don't need to have a name, and - patman does not yet use it, but it is convenient to put the branch - name here to help you keep track of multiple upstreaming efforts. + Sets the name of the series. You don't need to have a name, and + patman does not yet use it, but it is convenient to put the branch + name here to help you keep track of multiple upstreaming efforts. Series-links: [id | version:id]... - Set the ID of the series in patchwork. You can set this after you send - out the series and look in patchwork for the resulting series. The - URL you want is the one for the series itself, not any particular patch. - E.g. for http://patchwork.ozlabs.org/project/uboot/list/?series=187331 - the series ID is 187331. This property can have a list of series IDs, - one for each version of the series, e.g. + Set the ID of the series in patchwork. You can set this after you send + out the series and look in patchwork for the resulting series. The + URL you want is the one for the series itself, not any particular patch. + E.g. for http://patchwork.ozlabs.org/project/uboot/list/?series=187331 + the series ID is 187331. This property can have a list of series IDs, + one for each version of the series, e.g. + + :: - Series-links: 1:187331 2:188434 189372 + Series-links: 1:187331 2:188434 189372 - Patman always uses the one without a version, since it assumes this is - the latest one. When this tag is provided, patman can compare your local - branch against patchwork to see what new reviews your series has - collected ('patman status'). + Patman always uses the one without a version, since it assumes this is + the latest one. When this tag is provided, patman can compare your local + branch against patchwork to see what new reviews your series has + collected ('patman status'). Series-patchwork-url: url - This allows specifying the Patchwork URL for a branch. This overrides - both the setting files and the command-line argument. The URL should - include the protocol and web site, with no trailing slash, for example - 'https://patchwork.ozlabs.org/project' + This allows specifying the Patchwork URL for a branch. This overrides + both the setting files and the command-line argument. The URL should + include the protocol and web site, with no trailing slash, for example + 'https://patchwork.ozlabs.org/project' Cover-letter: -This is the patch set title -blah blah -more blah blah -END - Sets the cover letter contents for the series. The first line - will become the subject of the cover letter + Sets the cover letter contents for the series. The first line + will become the subject of the cover letter:: + + Cover-letter: + This is the patch set title + blah blah + more blah blah + END Cover-letter-cc: email / alias - Additional email addresses / aliases to send cover letter to (you - can add this multiple times) + Additional email addresses / aliases to send cover letter to (you + can add this multiple times) Series-notes: -blah blah -blah blah -more blah blah -END - Sets some notes for the patch series, which you don't want in - the commit messages, but do want to send, The notes are joined - together and put after the cover letter. Can appear multiple - times. + Sets some notes for the patch series, which you don't want in + the commit messages, but do want to send, The notes are joined + together and put after the cover letter. Can appear multiple + times:: + + Series-notes: + blah blah + blah blah + more blah blah + END Commit-notes: -blah blah -blah blah -more blah blah -END - Similar, but for a single commit (patch). These notes will appear - immediately below the --- cut in the patch file. - - Signed-off-by: Their Name <email> - A sign-off is added automatically to your patches (this is - probably a bug). If you put this tag in your patches, it will - override the default signoff that patman automatically adds. - Multiple duplicate signoffs will be removed. - - Tested-by: Their Name <email> - Reviewed-by: Their Name <email> - Acked-by: Their Name <email> - These indicate that someone has tested/reviewed/acked your patch. - When you get this reply on the mailing list, you can add this - tag to the relevant commit and the script will include it when - you send out the next version. If 'Tested-by:' is set to - yourself, it will be removed. No one will believe you. + Similar, but for a single commit (patch). These notes will appear + immediately below the --- cut in the patch file:: + + Commit-notes: + blah blah + blah blah + more blah blah + +Signed-off-by: Their Name <email> + A sign-off is added automatically to your patches (this is + probably a bug). If you put this tag in your patches, it will + override the default signoff that patman automatically adds. + Multiple duplicate signoffs will be removed. + +Tested-by / Reviewed-by / Acked-by + These indicate that someone has tested/reviewed/acked your patch. + When you get this reply on the mailing list, you can add this + tag to the relevant commit and the script will include it when + you send out the next version. If 'Tested-by:' is set to + yourself, it will be removed. No one will believe you. + + Example:: + + Tested-by: Their Name <fred@bloggs.com> + Reviewed-by: Their Name <email> + Acked-by: Their Name <email> Series-changes: n -- Guinea pig moved into its cage -- Other changes ending with a blank line -<blank line> - This can appear in any commit. It lists the changes for a - particular version n of that commit. The change list is - created based on this information. Each commit gets its own - change list and also the whole thing is repeated in the cover - letter (where duplicate change lines are merged). - - By adding your change lists into your commits it is easier to - keep track of what happened. When you amend a commit, remember - to update the log there and then, knowing that the script will - do the rest. + This can appear in any commit. It lists the changes for a + particular version n of that commit. The change list is + created based on this information. Each commit gets its own + change list and also the whole thing is repeated in the cover + letter (where duplicate change lines are merged). + + By adding your change lists into your commits it is easier to + keep track of what happened. When you amend a commit, remember + to update the log there and then, knowing that the script will + do the rest. + + Example:: + + Series-changes: n + - Guinea pig moved into its cage + - Other changes ending with a blank line + <blank line> Commit-changes: n -- This line will not appear in the cover-letter changelog -<blank line> - This tag is like Series-changes, except changes in this changelog will - only appear in the changelog of the commit this tag is in. This is - useful when you want to add notes which may not make sense in the cover - letter. For example, you can have short changes such as "New" or - "Lint". + This tag is like Series-changes, except changes in this changelog will + only appear in the changelog of the commit this tag is in. This is + useful when you want to add notes which may not make sense in the cover + letter. For example, you can have short changes such as "New" or + "Lint". + + Example:: + + Commit-changes: n + - This line will not appear in the cover-letter changelog + <blank line> Cover-changes: n -- This line will only appear in the cover letter -<blank line> - This tag is like Series-changes, except changes in this changelog will - only appear in the cover-letter changelog. This is useful to summarize - changes made with Commit-changes, or to add additional context to - changes. + This tag is like Series-changes, except changes in this changelog will + only appear in the cover-letter changelog. This is useful to summarize + changes made with Commit-changes, or to add additional context to + changes. + + Example:: + + Cover-changes: n + - This line will only appear in the cover letter + <blank line> Patch-cc: Their Name <email> - This copies a single patch to another email address. Note that the - Cc: used by git send-email is ignored by patman, but will be - interpreted by git send-email if you use it. + This copies a single patch to another email address. Note that the + Cc: used by git send-email is ignored by patman, but will be + interpreted by git send-email if you use it. Series-process-log: sort, uniq - This tells patman to sort and/or uniq the change logs. Changes may be - multiple lines long, as long as each subsequent line of a change begins - with a whitespace character. For example, + This tells patman to sort and/or uniq the change logs. Changes may be + multiple lines long, as long as each subsequent line of a change begins + with a whitespace character. For example, -- This change - continues onto the next line -- But this change is separate + Example:: - Use 'sort' to sort the entries, and 'uniq' to include only - unique entries. If omitted, no change log processing is done. - Separate each tag with a comma. + - This change + continues onto the next line + - But this change is separate + + Use 'sort' to sort the entries, and 'uniq' to include only + unique entries. If omitted, no change log processing is done. + Separate each tag with a comma. Change-Id: - This tag is stripped out but is used to generate the Message-Id - of the emails that will be sent. When you keep the Change-Id the - same you are asserting that this is a slightly different version - (but logically the same patch) as other patches that have been - sent out with the same Change-Id. + This tag is stripped out but is used to generate the Message-Id + of the emails that will be sent. When you keep the Change-Id the + same you are asserting that this is a slightly different version + (but logically the same patch) as other patches that have been + sent out with the same Change-Id. Various other tags are silently removed, like these Chrome OS and -Gerrit tags: +Gerrit tags:: -BUG=... -TEST=... -Review URL: -Reviewed-on: -Commit-xxxx: (except Commit-notes) + BUG=... + TEST=... + Review URL: + Reviewed-on: + Commit-xxxx: (except Commit-notes) Exercise for the reader: Try adding some tags to one of your current patch series and see how the patches turn out. Where Patches Are Sent -====================== +---------------------- Once the patches are created, patman sends them using git send-email. The whole series is sent to the recipients in Series-to: and Series-cc. You can Cc individual patches to other people with the Patch-cc: tag. Tags in the subject are also picked up to Cc patches. For example, a commit like -this: +this:: ->>>> -commit 10212537b85ff9b6e09c82045127522c0f0db981 -Author: Mike Frysinger <vapier@gentoo.org> -Date: Mon Nov 7 23:18:44 2011 -0500 + commit 10212537b85ff9b6e09c82045127522c0f0db981 + Author: Mike Frysinger <vapier@gentoo.org> + Date: Mon Nov 7 23:18:44 2011 -0500 x86: arm: add a git mailrc file for maintainers @@ -354,46 +384,47 @@ Date: Mon Nov 7 23:18:44 2011 -0500 Patch-cc: sandbox, mikef, ag Patch-cc: afleming -<<<< will create a patch which is copied to x86, arm, sandbox, mikef, ag and afleming. If you have a cover letter it will get sent to the union of the Patch-cc lists of all of the other patches. If you want to sent it to additional -people you can add a tag: +people you can add a tag:: -Cover-letter-cc: <list of addresses> + Cover-letter-cc: <list of addresses> These people will get the cover letter even if they are not on the To/Cc list for any of the patches. Patchwork Integration -===================== +--------------------- Patman has a very basic integration with Patchwork. If you point patman to -your series on patchwork it can show you what new reviews have appears since +your series on patchwork it can show you what new reviews have appeared since you sent your series. To set this up, add a Series-link tag to one of the commits in your series (see above). -Then you can type +Then you can type: + +.. code-block:: bash patman status and patman will show you each patch and what review tags have been collected, -for example: +for example:: -... - 21 x86: mtrr: Update the command to use the new mtrr - Reviewed-by: Wolfgang Wallner <wolfgang.wallner@br-automation.com> - + Reviewed-by: Bin Meng <bmeng.cn@gmail.com> - 22 x86: mtrr: Restructure so command execution is in - Reviewed-by: Wolfgang Wallner <wolfgang.wallner@br-automation.com> - + Reviewed-by: Bin Meng <bmeng.cn@gmail.com> -... + ... + 21 x86: mtrr: Update the command to use the new mtrr + Reviewed-by: Wolfgang Wallner <wolfgang.wallner@br-automation.com> + + Reviewed-by: Bin Meng <bmeng.cn@gmail.com> + 22 x86: mtrr: Restructure so command execution is in + Reviewed-by: Wolfgang Wallner <wolfgang.wallner@br-automation.com> + + Reviewed-by: Bin Meng <bmeng.cn@gmail.com> + ... This shows that patch 21 and 22 were sent out with one review but have since attracted another review each. If the series needs changes, you can update @@ -402,6 +433,8 @@ series. To automatically pull into these tags into a new branch, use the -d option: +.. code-block:: bash + patman status -d mtrr4 This will create a new 'mtrr4' branch which is the same as your current branch @@ -409,6 +442,8 @@ but has the new review tags in it. The tags are added in alphabetic order and are placed immediately after any existing ack/review/test/fixes tags, or at the end. You can check that this worked with: +.. code-block:: bash + patman -b mtrr4 status which should show that there are no new responses compared to this new branch. @@ -417,7 +452,7 @@ There is also a -C option to list the comments received for each patch. Example Work Flow -================= +----------------- The basic workflow is to create your commits, add some tags to the top commit, and type 'patman' to check and send them. @@ -425,7 +460,7 @@ commit, and type 'patman' to check and send them. Here is an example workflow for a series of 4 patches. Let's say you have these rather contrived patches in the following order in branch us-cmd in your tree where 'us' means your upstreaming activity (newest to oldest as -output by git log --oneline): +output by git log --oneline):: 7c7909c wip 89234f5 Don't include standard parser if hush is used @@ -438,36 +473,46 @@ but that you don't want to submit because there is an existing patch for it on the list. So you can tell patman to create and check some patches (skipping the first patch) with: +.. code-block:: bash + patman -s1 send -n If you want to do all of them including the work-in-progress one, then (if you are tracking an upstream branch): +.. code-block:: bash + patman send -n Let's say that patman reports an error in the second patch. Then: +.. code-block:: bash + git rebase -i HEAD~6 - <change 'pick' to 'edit' in 89234f5> - <use editor to make code changes> + # change 'pick' to 'edit' in 89234f5 + # use editor to make code changes git add -u git rebase --continue Now you have an updated patch series. To check it: +.. code-block:: bash + patman -s1 send -n Let's say it is now clean and you want to send it. Now you need to set up the destination. So amend the top commit with: +.. code-block:: bash + git commit --amend -Use your editor to add some tags, so that the whole commit message is: +Use your editor to add some tags, so that the whole commit message is:: The current run_command() is really only one of the options, with hush providing the other. It really shouldn't be called directly in case the hush parser is bring used, so rename this function to - better explain its purpose. + better explain its purpose:: Series-to: u-boot Series-cc: bfin, marex @@ -490,6 +535,8 @@ mmc and sparc, and the last one to sandbox. Now to send the patches, take off the -n flag: +.. code-block:: bash + patman -s1 send The patches will be created, shown in your editor, and then sent along with @@ -502,36 +549,42 @@ Also, the patch on the list that you were waiting for has been merged, so you can drop your wip commit. Take a look on patchwork and find out the URL of the series. This will be -something like http://patchwork.ozlabs.org/project/uboot/list/?series=187331 -Add this to a tag in your top commit: +something like `http://patchwork.ozlabs.org/project/uboot/list/?series=187331` +Add this to a tag in your top commit:: - Series-link: http://patchwork.ozlabs.org/project/uboot/list/?series=187331 + Series-links: 187331 You can use then patman to collect the Acked-by tag to the correct commit, creating a new 'version 2' branch for us-cmd: +.. code-block:: bash + patman status -d us-cmd2 git checkout us-cmd2 You can look at the comments in Patchwork or with: +.. code-block:: bash + patman status -C Then you can resync with upstream: - git fetch origin (or whatever upstream is called) +.. code-block:: bash + + git fetch origin # or whatever upstream is called git rebase origin/master and use git rebase -i to edit the commits, dropping the wip one. -Then update the Series-cc: in the top commit to add the person who reviewed -the v1 series: +Then update the `Series-cc:` in the top commit to add the person who reviewed +the v1 series:: Series-cc: bfin, marex, Heiko Schocher <hs@denx.de> and remove the Series-prefix: tag since it it isn't an RFC any more. The series is now version two, so the series info in the top commit looks like -this: +this:: Series-to: u-boot Series-cc: bfin, marex, Heiko Schocher <hs@denx.de> @@ -541,7 +594,7 @@ this: Finally, you need to add a change log to the two commits you changed. You add change logs to each individual commit where the changes happened, like -this: +this:: Series-changes: 2 - Updated the command decoder to reduce code size @@ -551,7 +604,7 @@ this: When you run patman it will collect all the change logs from the different commits and combine them into the cover letter, if you have one. So finally -you have a new series of commits: +you have a new series of commits:: faeb973 Don't include standard parser if hush is used 1b2f2fe mmc: sparc: Stop using builtin_run_command() @@ -560,59 +613,63 @@ you have a new series of commits: so to send them: +.. code-block:: bash + patman and it will create and send the version 2 series. General points -============== +-------------- 1. When you change back to the us-cmd branch days or weeks later all your -information is still there, safely stored in the commits. You don't need -to remember what version you are up to, who you sent the last lot of patches -to, or anything about the change logs. + information is still there, safely stored in the commits. You don't need + to remember what version you are up to, who you sent the last lot of patches + to, or anything about the change logs. 2. If you put tags in the subject, patman will Cc the maintainers -automatically in many cases. + automatically in many cases. 3. If you want to keep the commits from each series you sent so that you can -compare change and see what you did, you can either create a new branch for -each version, or just tag the branch before you start changing it: + compare change and see what you did, you can either create a new branch for + each version, or just tag the branch before you start changing it: - git tag sent/us-cmd-rfc - ...later... - git tag sent/us-cmd-v2 +.. code-block:: bash + + git tag sent/us-cmd-rfc + # ...later... + git tag sent/us-cmd-v2 4. If you want to modify the patches a little before sending, you can do -this in your editor, but be careful! + this in your editor, but be careful! 5. If you want to run git send-email yourself, use the -n flag which will -print out the command line patman would have used. + print out the command line patman would have used. 6. It is a good idea to add the change log info as you change the commit, -not later when you can't remember which patch you changed. You can always -go back and change or remove logs from commits. + not later when you can't remember which patch you changed. You can always + go back and change or remove logs from commits. 7. Some mailing lists have size limits and when we add binary contents to -our patches it's easy to exceed the size limits. Use "--no-binary" to -generate patches without any binary contents. You are supposed to include -a link to a git repository in your "Commit-notes", "Series-notes" or -"Cover-letter" for maintainers to fetch the original commit. + our patches it's easy to exceed the size limits. Use "--no-binary" to + generate patches without any binary contents. You are supposed to include + a link to a git repository in your "Commit-notes", "Series-notes" or + "Cover-letter" for maintainers to fetch the original commit. 8. Patches will have no changelog entries for revisions where they did not -change. For clarity, if there are no changes for this patch in the most -recent revision of the series, a note will be added. For example, a patch -with the following tags in the commit + change. For clarity, if there are no changes for this patch in the most + recent revision of the series, a note will be added. For example, a patch + with the following tags in the commit:: - Series-version: 5 - Series-changes: 2 - - Some change + Series-version: 5 + Series-changes: 2 + - Some change - Series-changes: 4 - - Another change + Series-changes: 4 + - Another change -would have a changelog of +would have a changelog of::: (no changes since v4) @@ -622,8 +679,9 @@ would have a changelog of Changes in v2: - Some change + Other thoughts -============== +-------------- This script has been split into sensible files but still needs work. Most of these are indicated by a TODO in the code. @@ -633,6 +691,8 @@ It would be nice if this could handle the In-reply-to side of things. The tests are incomplete, as is customary. Use the 'test' subcommand to run them: +.. code-block:: bash + $ tools/patman/patman test Error handling doesn't always produce friendly error messages - e.g. @@ -641,9 +701,3 @@ putting an incorrect tag in a commit may provide a confusing message. There might be a few other features not mentioned in this README. They might be bugs. In particular, tags are case sensitive which is probably a bad thing. - - -Simon Glass <sjg@chromium.org> -v1, v2, 19-Oct-11 -revised v3 24-Nov-11 -revised v4 Independence Day 2020, with Patchwork integration |