diff options
| -rw-r--r-- | doc/release-notes/skiboot-6.2-rc1.rst | 893 |
1 files changed, 893 insertions, 0 deletions
diff --git a/doc/release-notes/skiboot-6.2-rc1.rst b/doc/release-notes/skiboot-6.2-rc1.rst new file mode 100644 index 0000000..e930a4c --- /dev/null +++ b/doc/release-notes/skiboot-6.2-rc1.rst @@ -0,0 +1,893 @@ +.. _skiboot-6.2-rc1: + +skiboot-6.2-rc1 +=============== + +skiboot v6.2-rc1 was released on Monday November 19th 2018. It is the first +release candidate of skiboot 6.2, which will become the new stable release +of skiboot following the 6.1 release, first released July 11th 2018. + +Skiboot 6.2 will mark the basis for op-build v2.2. + +skiboot v6.2-rc1 contains all bug fixes as of :ref:`skiboot-6.0.13`, +and :ref:`skiboot-5.4.10` (the currently maintained +stable releases). + +For how the skiboot stable releases work, see :ref:`stable-rules` for details. + +This release has been a longer cycle than typical for a variety of reasons. It +also contains a lot of cleanup work and minor bug fixes (much like skiboot 6.1 +did). + +Over skiboot 6.1, we have the following changes: + +General +------- + +- cpu: Quieten OS endian switch messages + + Users see these when loading an OS from Petitboot: :: + + [ 119.486794100,5] OPAL: Switch to big-endian OS + [ 120.022302604,5] OPAL: Switch to little-endian OS + + Which is expected and doesn't provide any information the user can act + on. Switch them to PR_INFO so they still appear in the log, but not on + the serial console. +- Recognise signed VERSION partition + + A few things need to change to support a signed VERSION partition: + + - A signed VERSION partition will be 4K + SECURE_BOOT_HEADERS_SIZE (4K). + - The VERSION partition needs to be loaded after secure/trusted boot is + set up, and therefore after nvram_init(). + - Added to the trustedboot resources array. + + This also moves the ipmi_dt_add_bmc_info() call to after + flash_dt_add_fw_version() since it adds info to ibm,firmware-versions. +- Run pollers in time_wait() when not booting + + This only bit us hard with hiomap in one scenario. + + Our OPAL API has been OPAL_POLL_EVENTS may be needed to make forward + progress on ongoing operations, and the internal to skiboot API has been + that time_wait() of a suitable time will run pollers (on at least one + CPU) to help ensure forward progress can be made. + + In a perfect world, interrupts are used but they may: a) be disabled, or + b) the thing we're doing can't use interrupts because computers are + generally terrible. + + Back in 3db397ea5892a (circa 2015), we changed skiboot so that we'd run + pollers only on the boot CPU, and not if we held any locks. This was to + reduce the chance of programming code that could deadlock, as well as to + ensure that we didn't just thrash all the cachelines for running pollers + all over a large system during boot, or hard spin on the same locks on + all secondary CPUs. + + The problem arises if the OS we're booting makes an OPAL call early on, + with interrupts disabled, that requires a poller to run to make forward + progress. An example of this would be OPAL_WRITE_NVRAM early in Linux + boot (where Linux sets up the partitions it wants) - something that + occurs iff we've had to reformat NVRAM this boot (i.e. first boot or + corrupted NVRAM). + + The hiomap implementation should arguably *not* rely on synchronous IPMI + messages, but this is a future improvement (as was for mbox before it). + The mbox-flash code solved this problem by spinning on check_timers(). + + More generically though, the approach of running the pollers when no + longer booting means we behave more in line with what the API is meant + to be, rather than have this odd case of "time_wait() for a condition + that could also be tripped by an interrupt works fine unless the OS is + up and running but hasn't set interrupts up yet". +- ipmi: Reduce ipmi_queue_msg_sync() polling loop time to 10ms + + On a plain boot, this reduces the time spent in OPAL by ~170ms on + p9dsu. This is due to hiomap (currently) using synchronous IPMI + messages. + + It will also *significantly* reduce latency on runtime flash + operations for hiomap, as we'll spend typically 10-20ms in OPAL + rather than 100-200ms. It's not an ideal solution to that, but + it's a quick and obvious win for jitter. +- core/device: NULL pointer dereference fix +- core/flash: NULL pointer dereference fixes +- core/cpu: Call memset with proper cpu_thread offset +- libflash: Add ipmi-hiomap, and prefer it for PNOR access + + ipmi-hiomap implements the PNOR access control protocol formerly known + as "the mbox protocol" but uses IPMI instead of the AST LPC mailbox as a + transport. As there is no-longer any mailbox involved in this alternate + implementation the old protocol name is quite misleading, and so it has + been renamed to "the hiomap protoocol" (Host I/O Mapping protocol). The + same commands and events are used though this client-side implementation + assumes v2 of the protocol is supported by the BMC. + + The code is a heavily-reworked copy of the mbox-flash source and is + introduced this way to allow for the mbox implementation's eventual + removal. + + mbox-flash should in theory be renamed to mbox-hiomap for consistency, + but as it is on life-support effective immediately we may as well just + remove it entirely when the time is right. +- opal/hmi: Handle early HMIs on thread0 when secondaries are still in OPAL. + + When primary thread receives a CORE level HMI for timer facility errors + while secondaries are still in OPAL, thread 0 ends up in rendez-vous + waiting for secondaries to get into hmi handling. This is because OPAL + runs with MSR(EE=0) and hence HMIs are delayed on secondary threads until + they are given to Linux OS. Fix this by adding a check for secondary + state and force them in hmi handling by queuing job on secondary threads. + + I have tested this by injecting HDEC parity error very early during Linux + kernel boot. Recovery works fine for non-TB errors. But if TB is bad at + this very eary stage we already doomed. + + Without this patch we see: :: + + [ 285.046347408,7] OPAL: Start CPU 0x0843 (PIR 0x0843) -> 0x000000000000a83c + [ 285.051160609,7] OPAL: Start CPU 0x0844 (PIR 0x0844) -> 0x000000000000a83c + [ 285.055359021,7] HMI: Received HMI interrupt: HMER = 0x0840000000000000 + [ 285.055361439,7] HMI: [Loc: U78D3.ND1.WZS004A-P1-C48]: P:8 C:17 T:0: TFMR(2e12002870e14000) Timer Facility Error + [ 286.232183823,3] HMI: Rendez-vous stage 1 timeout, CPU 0x844 waiting for thread 1 (sptr=0000ccc1) + [ 287.409002056,3] HMI: Rendez-vous stage 1 timeout, CPU 0x844 waiting for thread 2 (sptr=0000ccc1) + [ 289.073820164,3] HMI: Rendez-vous stage 1 timeout, CPU 0x844 waiting for thread 3 (sptr=0000ccc1) + [ 290.250638683,3] HMI: Rendez-vous stage 1 timeout, CPU 0x844 waiting for thread 1 (sptr=0000ccc2) + [ 291.427456821,3] HMI: Rendez-vous stage 1 timeout, CPU 0x844 waiting for thread 2 (sptr=0000ccc2) + [ 293.092274807,3] HMI: Rendez-vous stage 1 timeout, CPU 0x844 waiting for thread 3 (sptr=0000ccc2) + [ 294.269092904,3] HMI: Rendez-vous stage 1 timeout, CPU 0x844 waiting for thread 1 (sptr=0000ccc3) + [ 295.445910944,3] HMI: Rendez-vous stage 1 timeout, CPU 0x844 waiting for thread 2 (sptr=0000ccc3) + [ 297.110728970,3] HMI: Rendez-vous stage 1 timeout, CPU 0x844 waiting for thread 3 (sptr=0000ccc3) + + After this patch: :: + + [ 259.401719351,7] OPAL: Start CPU 0x0841 (PIR 0x0841) -> 0x000000000000a83c + [ 259.406259572,7] OPAL: Start CPU 0x0842 (PIR 0x0842) -> 0x000000000000a83c + [ 259.410615534,7] OPAL: Start CPU 0x0843 (PIR 0x0843) -> 0x000000000000a83c + [ 259.415444519,7] OPAL: Start CPU 0x0844 (PIR 0x0844) -> 0x000000000000a83c + [ 259.419641401,7] HMI: Received HMI interrupt: HMER = 0x0840000000000000 + [ 259.419644124,7] HMI: [Loc: U78D3.ND1.WZS004A-P1-C48]: P:8 C:17 T:0: TFMR(2e12002870e04000) Timer Facility Error + [ 259.419650678,7] HMI: Sending hmi job to thread 1 + [ 259.419652744,7] HMI: Sending hmi job to thread 2 + [ 259.419653051,7] HMI: Received HMI interrupt: HMER = 0x0840000000000000 + [ 259.419654725,7] HMI: Sending hmi job to thread 3 + [ 259.419654916,7] HMI: Received HMI interrupt: HMER = 0x0840000000000000 + [ 259.419658025,7] HMI: Received HMI interrupt: HMER = 0x0840000000000000 + [ 259.419658406,7] HMI: [Loc: U78D3.ND1.WZS004A-P1-C48]: P:8 C:17 T:2: TFMR(2e12002870e04000) Timer Facility Error + [ 259.419663095,7] HMI: [Loc: U78D3.ND1.WZS004A-P1-C48]: P:8 C:17 T:3: TFMR(2e12002870e04000) Timer Facility Error + [ 259.419655234,7] HMI: [Loc: U78D3.ND1.WZS004A-P1-C48]: P:8 C:17 T:1: TFMR(2e12002870e04000) Timer Facility Error + [ 259.425109779,7] OPAL: Start CPU 0x0845 (PIR 0x0845) -> 0x000000000000a83c + [ 259.429870681,7] OPAL: Start CPU 0x0846 (PIR 0x0846) -> 0x000000000000a83c + [ 259.434549250,7] OPAL: Start CPU 0x0847 (PIR 0x0847) -> 0x000000000000a83c + +- core/cpu: Fix memory allocation for job array + + fixes: 7a3f307e core/cpu: parallelise global CPU register setting jobs + + This bug would result in boot-hang on some configurations due to + cpu_wait_job() endlessly waiting for the last bogus jobs[cpu->pir] pointer. +- i2c: Fix multiple-enqueue of the same request on NACK + + i2c_request_send() will retry the request if the error is a NAK, + however it forgets to clear the "ud.done" flag. It will thus + loop again and try to re-enqueue the same request causing internal + request list corruption. +- i2c: Ensure ordering between i2c_request_send() and completion + + i2c_request_send loops waiting for a flag "uc.done" set by + the completion routine, and then look for a result code + also set by that same completion. + + There is no synchronization, the completion can happen on another + processor, so we need to order the stores to uc and the reads + from uc so that uc.done is stored last and tested first using + memory barriers. +- pci: Clarify power down logic + + Currently pci_scan_bus() unconditionally calls pci_slot_set_power_state() + when it's finished scanning a bus. This is one of those things that + makes you go "WHAT?" when you first see it and frankly the skiboot PCI + code could do with less of that. + +Fast Reboot +^^^^^^^^^^^ + +- fast-reboot: parallel memory clearing + + Arbitrarily pick 16GB as the unit of parallelism, and + split up clearing memory into jobs and schedule them + node-local to the memory (or on node 0 if we can't + work that out because it's the memory up to SKIBOOT_BASE) + + This seems to cut at least ~40% time from memory zeroing on + fast-reboot on a 256GB Boston system. + + For many systems, scanning PCI takes about as much time as + zeroing all of RAM, so we may as well do them at the same time + and cut a few seconds off the total fast reboot time. +- fast-reboot: verify firmware "romem" checksum + + This takes a checksum of skiboot memory after boot that should be + unchanged during OS operation, and verifies it before allowing a + fast reboot. + + This is not read-only memory from skiboot's point of view, beause + it includes things like the opal branch table that gets populated + during boot. + + This helps to improve the integrity of firmware against host and + runtime firmware memory scribble bugs. + +- core/fast-reboot: print the fast reboot disable reason + + Once things start to go wrong, disable_fast_reboot can be called a + number of times, so make the first reason sticky, and also print it + to the console at disable time. This helps with making sense of + fast reboot disables. +- Add fast-reboot property to /ibm,opal DT node + + this means that if it's permanently disabled on boot, the test suite can + pick that up and not try a fast reboot test. + +Utilities +--------- + +- pflash: Add --skip option for reading + + Add a --skip=N option to pflash to skip N number of bytes when reading. + This would allow users to print the VERSION partition without the STB + header by specifying the --skip=4096 argument, and it's a more generic + solution rather than making pflash depend on secure/trusted boot code. +- xscom-utils: Rework getsram + + Allow specifying a file on the command line to read OCC SRAM data into. + If no file is specified then we print it to stdout as text. This is a + bit inconsistent, but it retains compatibility with the existing tool. +- xscom-utils/getsram: Make it work on P9 + + The XSCOM base address of the OCC control registers changed slightly + between P8 and P9. Fix this up and add a bit of PVR checking so we look + in the right place. +- opal-prd: Fix opal-prd crash + + Presently callback function from HBRT uses r11 to point to target function + pointer. r12 is garbage. This works fine when we compile with "-no-pie" option + (as we don't use r12 to calculate TOC). + + As per ABIv2 : "r12 : Function entry address at global entry point" + + With "-pie" compilation option, we have to set r12 to point to global function + entry point. So that we can calculate TOC properly. + + Crash log without this patch: :: + + opal-prd[2864]: unhandled signal 11 at 0000000000029320 nip 00000 00102012830 lr 0000000102016890 code 1 + + +Development and Debugging +------------------------- + +- core/lock: Use try_lock_caller() in lock_caller() to capture owner + + Otherwise we can get reports of core/lock.c owning the lock, which is + not helpful when tracking down ownership issues. +- core/flash: Emit a warning if Skiboot version doesn't match + + This means you'll get a warning that you've modified skiboot separately + to the rest of the PNOR image, which can be useful in determining what + firmware is actually running on a machine. +- gcov: link in ctors* as newer GCC doesn't group them all + + It seems that newer toolchains get us multiple ctors sections to link in + rather than just one. If we discard them (as we were doing), then we + don't have a working gcov build (and we get the "doesn't look sane" + warning on boot). +- core/flash: Log return code when ffs_init() fails + + Knowing the return code is at least better than not knowing the return + code. +- gcov: Fix building with GCC8 +- travis/ci: rework Dockerfiles to produce build artifacts + + ubuntu-latest was also missing clang, as ubuntu-latest is closer to + ubuntu 18.04 than 16.04 +- cpu: add cpu_queue_job_on_node() + + Add a job scheduling API which will run the job on the requested + chip_id (or return failure). +- opal-ci: Build old dtc version for fedora 28 + + There are patches that will go into dtc to fix the issues we hit, but + for the moment let's just build and use a slightly older version. +- mem_region: Merge similar allocations when dumping + + Currently we print one line for each allocation done at runtime when + dumping the memory allocations. We do a few thousand allocations at + boot so this can result in a huge amount of text being printed which + is a) slow to print, and b) Can result in the log buffer overflowing + which destroys otherwise useful information. + + This patch adds a de-duplication to this memory allocation dump by + merging "similar" allocations (same location, same size) into one. + + Unfortunately, the algorithm used to do the de-duplication is quadratic, + but considering we only dump the allocations in the event of a fatal + error I think this is acceptable. I also did some benchmarking and found + that on a ZZ it takes ~3ms to do a dump with 12k allocations. On a Zaius + it's slightly longer at about ~10ms for 10k allocs. However, the + difference there was due to the output being written to the UART. + + This patch also bumps the log level to PR_NOTICE. PR_INFO messages are + suppressed at the default log level, which probably isn't something you + want considering we only dump the allocations when we run out of skiboot + heap space. +- core/lock: fix timeout warning causing a deadlock false positive + + If a lock waiter exceeds the warning timeout, it prints a message + while still registered as requesting the lock. Printing the message + can take locks, so if one is held when the owner of the original + lock tries to print a message, it will get a false positive deadlock + detection, which brings down the system. + + This can easily be hit when there is a lot of HMI activity from a + KVM guest, where the timebase was not returned to host timebase + before calling the HMI handler. +- hw/p8-i2c: Print the set error bits + + This is purely to save me from having to look it up every time someone + gets an I2C error. +- init: Fix starting stripped kernel + + Currently if we try to run a raw/stripped binary kernel (ie. without + the elf header) we crash with: :: + + [ 0.008757768,5] INIT: Waiting for kernel... + [ 0.008762937,5] INIT: platform wait for kernel load failed + [ 0.008768171,5] INIT: Assuming kernel at 0x20000000 + [ 0.008779241,3] INIT: ELF header not found. Assuming raw binary. + [ 0.017047348,5] INIT: Starting kernel at 0x0, fdt at 0x3044b230 14339 bytes + [ 0.017054251,0] FATAL: Kernel is zeros, can't execute! + [ 0.017059054,0] Assert fail: core/init.c:590:0 + [ 0.017065371,0] Aborting! + + This is because we haven't set kernel_entry correctly in this path. + This fixes it. +- cpu: Better output when waiting for a very long job + + Instead of printing at the end if the job took more than 1s, + print in the loop every 30s along with a backtrace. This will + give us some output if the job is deadlocked. +- lock: Fix interactions between lock dependency checker and stack checker + + The lock dependency checker does a few nasty things that can cause + re-entrancy deadlocks in conjunction with the stack checker or + in fact other debug tests. + + A lot of it revolves around taking a new lock (dl_lock) as part + of the locking process. + + This tries to fix it by making sure we do not hit the stack + checker while holding dl_lock. + + We achieve that in part by directly using the low-level __try_lock + and manually unlocking on the dl_lock, and making some functions + "nomcount". + + In addition, we mark the dl_lock as being in the console path to + avoid deadlocks with the UART driver. + + We move the enabling of the deadlock checker to a separate config + option from DEBUG_LOCKS as well, in case we chose to disable it + by default later on. +- xscom-utils/adu_scoms.py: run 2to3 over it +- clang: -Wno-error=ignored-attributes + +Mambo Platform +^^^^^^^^^^^^^^ + +- mambo: Merge PMEM_DISK and PMEM_VOLATILE code + + PMEM_VOLATILE and PMEM_DISK can't be used together and are basically + copies of the same code. + + This merges the two and allows them used together. Same API is kept. +- hw/chiptod: test QUIRK_NO_CHIPTOD in opal_resync_timebase + + This allows some test coverage of deep stop states in Linux with + Mambo. +- core/mem_region: mambo reserve kernel payload areas + + Mambo image payloads get overwritten by the OS and by + fast reboot memory clearing because they have no region + defined. Add them, which allows fast reboot to work. + +Qemu platform +^^^^^^^^^^^^^ + +- nx: Don't abort on missing NX when using a QEMU machine + + These don't have an NX node (and probably never will) as they + don't provide any coprocessor. However, the DARN instruction + works so this abort is unnecessary. + +POWER8 Platforms +---------------- +- SBE-p8: Do all sbe timer update with xscom lock held + + Without this, on some P8 platforms, we could (falsely) think the SBE timer + had stalled getting the dreaded "timer stuck" message. + + The code was doing the mftb() to set the start of the timeout period while + *not* holding the lock, so the 1ms timeout started sometime when somebody + else had the xscom lock. + + The simple solution is to just do the whole routine holding the xscom lock, + so do it that way. + +Vesnin Platform +^^^^^^^^^^^^^^^ +- platforms/astbmc/vesnin: Send list of PCI devices to BMC through IPMI + + Implements sending a list of installed PCI devices through IPMI protocol. + Each PCI device description is sent as a standalone IPMI message. + A list of devices can be gathered from separate messages using the + session identifier. The session Id is an incremental counter that is + updated at the start of synchronization session. + + +POWER9 Platforms +---------------- + +- STOP API: API conditionally supports 255 SCOM restore entries for each quad. +- hdata/i2c: Skip unknown device type + + Do not add unknown I2C devices to device tree. +- hdata/i2c: Add whitelisting for Host I2C devices + + Many of the devices that we get information about through HDAT are for + use by firmware rather than the host operating system. This patch adds + a boolean flag to hdat_i2c_info structure that indicates whether devices + with a given purpose should be reserved for use inside of OPAL (or some + other firmware component, such as the OCC). +- hdata/iohub: Fix Cumulus Hub ID number +- opal/hmi: Wakeup the cpu before reading core_fir + + When stop state 5 is enabled, reading the core_fir during an HMI can + result in a xscom read error with xscom_read() returning an + OPAL_XSCOM_PARTIAL_GOOD error code and core_fir value of all FFs. At + present this return error code is not handled in decode_core_fir() + hence the invalid core_fir value is sent to the kernel where it + interprets it as a FATAL hmi causing a system check-stop. + + This can be prevented by forcing the core to wake-up using before + reading the core_fir. Hence this patch wraps the call to + read_core_fir() within calls to dctl_set_special_wakeup() and + dctl_clear_special_wakeup(). +- xive: Disable block tracker + + Due to some HW errata, the block tracking facility (performance optimisation + for large systems) should be disabled on Nimbus chips. Disable it unconditionally + for now. +- opal/hmi: Ignore debug trigger inject core FIR. + + Core FIR[60] is a side effect of the work around for the CI Vector Load + issue in DD2.1. Usually this gets delivered as HMI with HMER[17] where + Linux already ignores it. But it looks like in some cases we may happen + to see CORE_FIR[60] while we are already in Malfunction Alert HMI + (HMER[0]) due to other reasons e.g. CAPI recovery or NPU xstop. If that + happens then just ignore it instead of crashing kernel as not recoverable. +- hdata: Make sure reserved node name starts with "ibm, " + + HDAT does not provide consistent label format for reserved memory label. + Few starts with "ibm," while few other starts with component name. +- hdata: Fix dtc warnings + + Fix dtc warnings related to mcbist node. :: + + Warning (reg_format): "reg" property in /xscom@623fc00000000/mcbist@1 has invalid length (4 bytes) (#address-cells == 1, #size-cells == 1) + Warning (reg_format): "reg" property in /xscom@623fc00000000/mcbist@2 has invalid length (4 bytes) (#address-cells == 1, #size-cells == 1) + Warning (reg_format): "reg" property in /xscom@603fc00000000/mcbist@1 has invalid length (4 bytes) (#address-cells == 1, #size-cells == 1) + Warning (reg_format): "reg" property in /xscom@603fc00000000/mcbist@2 has invalid length (4 bytes) (#address-cells == 1, #size-cells == 1) + + Ideally we should add proper xscom range here... but we are not getting that + information in HDAT today. Lets fix warning until we get proper data in HDAT. + +PHB4 +^^^^ + +- phb4: Generate checkstop on AIB ECC corr/uncorr for DD2.0 parts + + On DD2.0 parts, PCIe ECC protection is not warranted in the response + data path. Thus, for these parts, we need to flag any ECC errors + detected from the adjacent AIB RX Data path so the part can be + replaced. + + This patch configures the FIRs so that we escalate these AIB ECC + errors to a checkstop so the parts can be replaced. +- phb4: Reset pfir and nfir if new errors reported during ETU reset + + During fast-reboot new PEC errors can be latched even after ETU-Reset + is asserted. This will result in values of variables nfir_cache and + pfir_cache to be out of sync. + + During step-2 of CRESET nfir_cache and pfir_cache values are used to + bring the PHB out of reset state. However if these variables are out + as noted above of date the nfir/pfir registers are never reset + completely and ETU still remains frozen. + + Hence this patch updates step-2 of phb4_creset to re-read the values of + nfir/pfir registers to check if any new errors were reported after + ETU-reset was asserted, report these new errors and reset the + nfir/pfir registers. This should bring the ETU out of reset + successfully. +- phb4: Disable nodal scoped DMA accesses when PB pump mode is enabled + + By default when a PCIe device issues a read request via the PHB it is first + issued with nodal scope. When accessing GPU memory the NPU does not know at the + time of response if the requested memory page is off node or not. Therefore + every read of GPU memory by a PHB is retried with larger scope which introduces + bandwidth and latency issues. + + On smaller boxes which have pump mode enabled nodal and group scoped reads are + treated the same and both types of request are broadcast to one chip. Therefore + we can avoid the retry by disabling nodal scope on the PHB for these boxes. On + larger boxes nodal (single chip) and group (multiple chip) scoped reads are + treated differently. Therefore we avoid disabling nodal scope on large boxes + which have pump mode disabled to avoid all PHB requests being broadcast to + multiple chips. +- phb4/capp: Only reset FIR bits that cause capp machine check + + During CAPP recovery do_capp_recovery_scoms() will reset the CAPP Fir + register just after CAPP recovery is completed. This has an + unintentional side effect of preventing PRD from analyzing and + reporting this error. If PRD tries to read the CAPP FIR after opal has + already reset it, then it logs a critical error complaining "No active + error bits found". + + To prevent this from happening we update do_capp_recovery_scoms() to + only reset fir bits that cause CAPP machine check (local xstop). This + is done by reading the CAPP Fir Action0/1 & Mask registers and + generating a mask which is then written on CAPP_FIR_CLEAR register. + +- phb4: Check for RX errors after link training + + Some PHB4 PHYs can get stuck in a bad state where they are constantly + retraining the link. This happens transparently to skiboot and Linux + but will causes PCIe to be slow. Resetting the PHB4 clears the + problem. + + We can detect this case by looking at the RX errors count where we + check for link stability. This patch does this by modifying the link + optimal code to check for RX errors. If errors are occurring we + retrain the link irrespective of the chip rev or card. + + Normally when this problem occurs, the RX error count is maxed out at + 255. When there is no problem, the count is 0. We chose 8 as the max + rx errors value to give us some margin for a few errors. There is also + a knob that can be used to set the error threshold for when we should + retrain the link. ie :: + + nvram -p ibm,skiboot --update-config phb-rx-err-max=8 + +- hw/phb4: Add a helper to dump the PELT-V + + The "Partitionable Endpoint Lookup Table (Vector)" is used by the PHB + when processing EEH events. The PELT-V defines which PEs should be + additionally frozen in the event of an error being flagged on a + given PE. Knowing the state of the PELT-V is sometimes useful for + debugging PHB issues so this patch adds a helper to dump it. + +- hw/phb4: Print the PEs in the EEH dump in hex + + Linux always displays the PE number in hexidecimal while skiboot + displays the PEST index (PE number) in decimal. This makes correlating + errors between Skiboot and Linux more annoying than it should be so + this patch makes Skiboot print the PEST number in hex. + +- phb4: Reallocate PEC2 DMA-Read engines to improve GPU-Direct bandwidth + + We reallocate additional 16/8 DMA-Read engines allocated to stack0/1 + on PEC2 respectively. This is needed to improve bandwidth available to + the Mellanox CX5 adapter when trying to read GPU memory (GPU-Direct). + + If kernel cxl driver indicates a request to allocate maximum possible + DMA read engines when calling enable_capi_mode() and card is attached + to PEC2/stack0 slot then we assume its a Mellanox CX5 adapter. We then + allocate additional 16/8 extra DMA read engines to stack0 and stack1 + respectively on PEC2. This is done by populating the + XPEC_PCI_PRDSTKOVR and XPEC_NEST_READ_STACK_OVERRIDE as suggested by + the h/w team. +- phb4: Enable PHB MMIO-0/1 Bars only when mmio window exists + + Presently phb4_probe_stack() will always enable PHB MMIO0/1 windows + even if they doesn't exist in phy_map. Hence we do some minor shuffling + in the phb4_probe_stack() so that MMIO-0/1 Bars are only enabled if + there corresponding MMIO window exists in the phy_map. In case phy_map + for an mmio window is '0' we set the corresponding BAR register to + '0'. +- hw/phb4: Use local_alloc for phb4 structures + + Struct phb4 is fairly heavyweight at 283664 bytes. On systems with + 6x PHBs per socket this results in using 3.2MB of heap space the PHB + structures alone. This is a fairly large chunk of our 12MB heap and + on systems with particularly large PCIe topologies, or additional + PHBs we can fail to boot because we cannot allocate space for the + FDT blob. + + This patch switches to using local_alloc() for the PHB structures + so they don't consume too large a portion of our 12MB heap space. +- phb4: Fix typo in disable lane eq code + + In this commit :: + + commit 737c0ba3d72b8aab05a765a9fc111a48faac0f75 + Author: Michael Neuling <mikey@neuling.org> + Date: Thu Feb 22 10:52:18 2018 +1100 + phb4: Disable lane eq when retrying some nvidia GEN3 devices + + We made a typo and set PH2 twice. This fixes it. + + It worked previously as if only phase 2 (PH2) is set it, skips phase 2 + and phase 3 (PH3). +- phb4: Don't probe a PHB if its garded + + Presently phb4_probe_stack() causes an exception while trying to probe + a PHB if its garded. This causes skiboot to go into a reboot loop with + following exception log: :: + + *********************************************** + Fatal MCE at 000000003006ecd4 .probe_phb4+0x570 + CFAR : 00000000300b98a0 + <snip> + Aborting! + CPU 0018 Backtrace: + S: 0000000031cc37e0 R: 000000003001a51c ._abort+0x4c + S: 0000000031cc3860 R: 0000000030028170 .exception_entry+0x180 + S: 0000000031cc3a40 R: 0000000000001f10 * + S: 0000000031cc3c20 R: 000000003006ecb0 .probe_phb4+0x54c + S: 0000000031cc3e30 R: 0000000030014ca4 .main_cpu_entry+0x5b0 + S: 0000000031cc3f00 R: 0000000030002700 boot_entry+0x1b8 + + This is caused as phb4_probe_stack() will ignore all xscom read/write + errors to enable PHB Bars and then tries to perform an mmio to read + PHB Version registers that cause the fatal MCE. + + We fix this by ignoring the PHB probe if the first xscom_write() to + populate the PHB Bar register fails, which indicates that there is + something wrong with the PHB. +- phb4: Workaround PHB errata with CFG write UR/CA errors + + If the PHB encounters a UR or CA status on a CFG write, it will + incorrectly freeze the wrong PE. Instead of using the PE# specified + in the CONFIG_ADDRESS register, it will use the PE# of whatever + MMIO occurred last. + + Work around this disabling freeze on such errors +- phb4: Handle allocation errors in phb4_eeh_dump_regs() + + If the zalloc fails (and it can be a rather large allocation), + we will overwite memory at 0 instead of failing. +- phb4: Don't try to access non-existent PEST entries + + In a POWER9 chip, some PHB4s have 256 PEs, some have 512. + + Currently, the diagnostics code retrieves 512 unconditionally, + which is wrong and causes us to incorrectly report bogus values + for the "high" PEs on the small PHBs. + + Use the actual number of implemented PEs instead + +CAPI2 +^^^^^ + +- phb4/capp: Use link width to allocate STQ engines to CAPP + + Update phb4_init_capp_regs() to allocates STQ Engines to CAPP/PEC2 + based on link width instead of always assuming it to x8. + + Also re-factor the function slightly to evaluate the link-width only + once and cache it so that it can also be used to allocate DMA read + engines. +- phb4/capp: Update DMA read engines set in APC_FSM_READ_MASK based on link-width + + Commit 47c09cdfe7a3("phb4/capp: Calculate STQ/DMA read engines based + on link-width for PEC") update the CAPP init sequence by calculating + the needed STQ/DMA-read engines based on link width and populating it + in XPEC_NEST_CAPP_CNTL register. This however needs to be synchronized + with the value set in CAPP APC FSM Read Machine Mask Register. + + Hence this patch update phb4_init_capp_regs() to calculate the link + width of the stack on PEC2 and populate the same values as previously + populated in PEC CAPP_CNTL register. +- capp: Fix the capp recovery timeout comparison + + The current capp recovery timeout control loop in + do_capp_recovery_scoms() uses a wrong comparison for return value of + tb_compare(). This may cause do_capp_recovery_scoms() to report an + timeout earlier than the 168ms stipulated time. + + The patch fixes this by updating the loop timeout control branch in + do_capp_recovery_scoms() to use the correct enum tb_cmpval. +- phb4: Disable 32-bit MSI in capi mode + + If a capi device does a DMA write targeting an address lower than 4GB, + it does so through a 32-bit operation, per the PCI spec. In capi mode, + the first TVE entry is configured in bypass mode, so the address is + valid. But with any (bad) luck, the address could be 0xFFFFxxxx, thus + looking like a 32-bit MSI. + + We currently enable both 32-bit and 64-bit MSIs, so the PHB will + interpret the DMA write as a MSI, which very likely results in an EEH + (MSI with a bad payload size). + + We can fix it by disabling 32-bit MSI when switching the PHB to capi + mode. Capi devices are 64-bit. + +NVLINK2 +^^^^^^^ +- npu2: Add support for relaxed-ordering mode + + Some device drivers support out of order access to GPU memory. This does + not affect the CPU view of memory but it does affect the GPU view of + memory. It should only be enabled if the GPU driver has requested it. + + Add OPAL APIs allowing the driver to query relaxed ordering state or + request it to be set for a device. Current hardware only allows relaxed + ordering to be enabled per PCIe root port. So the code here doesn't + enable relaxed ordering until it has been explicitly requested for every + device on the port. +- Add the other 7 ATSD registers to the device tree. +- npu2/hw-procedures: Don't open code NPU2_NTL_MISC_CFG2_BRICK_ENABLE + + Name this bit properly. There's a lot more cleanup like this to be done, + but I'm catching this one now as part of some related changes. +- npu2/hw-procedures: Enable parity and credit overflow checks + + Enable these error checking features by setting the appropriate bits in + our one-off initialization of each "NTL Misc Config 2" register. + + The exception is NDL RX parity checking, which should be disabled during + the link training procedures. +- npu2: Use correct kill type for TCE invalidation + + kill_type is enum of OPAL_PCI_TCE_KILL_PAGES, OPAL_PCI_TCE_KILL_PE, + OPAL_PCI_TCE_KILL_ALL and phb4_tce_kill() gets it right but + npu2_tce_kill() uses OPAL_PCI_TCE_KILL which is an OPAL API token. + + This fixes an obvious mistype. + +OpenCAPI +^^^^^^^^ + +- Support OpenCAPI on Witherspoon platform +- npu2-opencapi: Enable presence detection on ZZ + + Presence detection for opencapi adapters was broken for ZZ planars v3 + and below. All ZZ systems currently used in the lab have had their + planar upgraded, so we can now remove the override we had to force + presence and activate presence detection. Which should improve boot + time. + + Considering the state of opal support on ZZ, this is really only for + lab usage on BML. The opencapi enablement team has okay'd the + change. In the unlikely case somebody tries opencapi on an old ZZ, the + presence detection through i2c will show that no adapter is present + and skiboot won't try to access or train the link. +- npu2-opencapi: Don't send commands to NPU when link is down + + Even if an opencapi link is down, we currently always try to issue a + config read operation when probing for PCI devices, because of the + default scan map used for an opencapi PHB. The config operation fails, + as expected, but it can also raise a FIR bit and trigger an HMI. + + For opencapi, there's no root device like for a "normal" PCI PHB, so + there's no reason to do the config operation. To fix it, we keep the + scan map blank by default, and only add a device once the link is + trained. +- opal/hmi: Catch NPU2 HMIs for opencapi + + HMIs for NPU2 are filtered with the 'compatible' string of the PHB, so + add opencapi to the mix. +- occ: Wait if OCC GPU presence status not immediately available + + It takes a few seconds for the OCC to set everything up in order to read + GPU presence. At present, we try to kick off OCC initialisation as early as + possible to maximise the time it has to read GPU presence. + + Unfortunately sometimes that's not enough, so add a loop in + occ_get_gpu_presence() so that on the first time we try to get GPU presence + we keep trying for up to 2 seconds. Experimentally this seems to be + adequate. +- hw/npu2-hw-procedures: Enable RX auto recal on OpenCAPI links + + The RX_RC_ENABLE_AUTO_RECAL flag is required on OpenCAPI but not NVLink. + + Traditionally, Hostboot sets this value according to the machine type. + However, now that Witherspoon supports both NVLink and OpenCAPI, it can't + tell whether or not a link is OpenCAPI. + + So instead, set it in skiboot, where it will only be triggered after we've + done device detection and found an OpenCAPI device. +- hw/npu2-opencapi: Fix setting of supported OpenCAPI templates + + In opal_npu_tl_set(), we made a typo that means the OPAL_NPU_TL_SET call + may not clear the enable bits for templates that were previously enabled + but are now disabled. + + Fix the typo so we clear NPU2_OTL_CONFIG1_TX_TEMP2_EN as well as + TEMP{1,3}_EN. + +Barreleye G2 and Zaius platforms +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +- zaius: Add a slot table +- zaius: Add slots for the Barreleye G2 HDD rack + + The Barreleye G2 is distinct from the Zaius in that it features a 24 + Bay NVMe/SATA HDD rack. To provide meaningful slot names for each NVMe + device we need to define a slot table for the NVMe capable HDD bays. + + Unfortunately this isn't straightforward because the PCIe path to the + NVMe devices isn't fixed. The PCIe topology is something like: + P9 -> HBA card -> 9797 switch -> 20x NVMe HDD slots + + The 9797 switch is partitioned into two (or four) virtual switches which + allow multiple HBA cards to be used (e.g. one per socket). As a result + the exact BDFN of the ports will vary depending on how the system is + configured. + + That said, the virtual switch configuration of the 9797 does not change + the device and function numbers of the switch downports. This means that + we can define a single slot table that maps switch ports to the NVMe bay + names. + + Unfortunately we still need to guess which bus to use this table on, so + we assume that any switch downport we find with the PEX9797 VDID is part + of the 9797 that supports the HDD rack. + +FSP based platforms (firenze and ZZ) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +- phb4/capp: Update the expected Eye-catcher for CAPP ucode lid + + Currently on a FSP based P9 system load_capp_code() expects CAPP ucode + lid header to have eye-catcher magic of 'CAPPPSLL'. However skiboot + currently supports CAPP ucode only lids that have a eye-catcher magic + of 'CAPPLIDH'. This prevents skiboot from loading the ucode with this + error message: :: + + CAPP: ucode header invalid + + We fix this issue by updating load_capp_ucode() to use the eye-catcher + value of 'CAPPLIDH' instead of 'CAPPPSLL'. + +- FSP: Improve Reset/Reload log message + + Below message is confusing. Lets make it clear. + + FSP sends "R/R complete notification" whenever there is a dump. We use `flag` + to identify whether its its R/R completion -OR- just new dump notification. :: + + [ 483.406351956,6] FSP: SP says Reset/Reload complete + [ 483.406354278,5] DUMP: FipS dump available. ID = 0x1a00001f [size: 6367640 bytes] + [ 483.406355968,7] A Reset/Reload was NOT done + +Witherspoon platform +^^^^^^^^^^^^^^^^^^^^ + +- platforms/astbmc/witherspoon: Implement OpenCAPI support + + OpenCAPI on Witherspoon is slightly more involved than on Zaius and ZZ, due + to the OpenCAPI links using the SXM2 connectors that are used for NVLink + GPUs. + + This patch adds the regular OpenCAPI platform information, and also a + Witherspoon-specific presence detection callback that uses the previously + added OCC GPU presence detection to figure out the device types plugged + into each SXM2 socket. + + The SXM2 connectors are capable of carrying 2 OpenCAPI links, and future + OpenCAPI devices are expected to make use of this. However, we don't yet + support ganged links and the various implications that has for handling + things like device reset, so for now, we only enable 1 brick per device. |