aboutsummaryrefslogtreecommitdiff
path: root/hw/phb4.c
AgeCommit message (Collapse)AuthorFilesLines
2019-02-18phb4: Rework BDFN filtering in phb4_set_pe()Oliver O'Halloran1-41/+17
General cleanup. For a function that does nothing more than a mask-and-compare the current implementation is way more convoluted than it has any right to be. Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Signed-off-by: Oliver O'Halloran <oohall@gmail.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2019-01-25Remove POWER9N DD1 supportNicholas Piggin1-215/+67
This is not a shipping product and is no longer supported by Linux or other firmware components. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2019-01-25phb4: remove POWER9N DD1 creset workaroundNicholas Piggin1-5/+0
Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2019-01-18sparse: Make tree 'constant is so big' warning cleanStewart Smith1-24/+24
Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2019-01-16capp/phb4: Prevent HMI from getting triggered when disabling CAPPVaibhav Jain1-0/+11
While disabling CAPP an HMI gets triggered as soon as ETU is put in reset mode. This is caused as before we can disabled CAPP, it detects PHB link going down and triggers an HMI requesting Opal to perform CAPP recovery. This has an un-intended side effect of spamming the Opal logs with malfunction alert messages and may also confuse the user. To prevent this we mask the CAPP FIR error 'PHB Link Down' Bit(31) when we are disabling CAPP just before we put ETU in reset in phb4_creset(). Also now since bringing down the PHB link now wont trigger an HMI and CAPP recovery, hence we manually set the PHB4_CAPP_RECOVERY flag on the phb to force recovery during creset. Reviewed-by: Frederic Barrat <fbarrat@linux.ibm.com> Reviewed-by: Christophe Lombard <clombard@linux.vnet.ibm.com> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2019-01-16phb4/capp: Implement sequence to disable CAPP and enable fast-resetVaibhav Jain1-3/+64
We implement h/w sequence to disable CAPP in disable_capi_mode() and with it also enable fast-reset for CAPI mode in phb4_set_capi_mode(). Sequence to disable CAPP is executed in three phases. The first two phase is implemented in disable_capi_mode() where we reset the CAPP registers followed by PEC registers to their init values. The final third final phase is to reset the PHB CAPI Compare/Mask Register and is done in phb4_init_ioda3(). The reason to move the PHB reset to phb4_init_ioda3() is because by the time Opal PCI reset state machine reaches this function the PHB is already un-fenced and its configuration registers accessible via mmio. Reviewed-by: Frederic Barrat <fbarrat@linux.ibm.com> Reviewed-by: Christophe Lombard <clombard@linux.vnet.ibm.com> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2019-01-16capp/phb4: Introduce PHB4 flag, PHB4_CAPP_DISABLE to disable CAPPVaibhav Jain1-1/+96
This patch introduces a PHB4 flag PHB4_CAPP_DISABLE and scaffolding necessary to handle it during CRESET flow. The flag is set when CAPP is request to switch to PCIe mode via call to phb4_set_capi_mode() with mode OPAL_PHB_CAPI_MODE_PCIE. This starts the below sequence that ultimately ends in newly introduced phb4_slot_sm_run_completed() 1. Set PHB4_CAPP_DISABLE to phb4->flags. 2. Start a CRESET on the phb slot. This also starts the opal pci reset state machine. 3. Wait for slot state to be PHB4_SLOT_CRESET_WAIT_CQ. 4. Perform CAPP recovery as PHB is still fenced, by calling do_capp_recovery_scoms(). 5. Call newly introduced 'disable_capi_mode()' to disable CAPP. 6. Wait for slot reset to complete while it transitions to PHB4_SLOT_FRESET and optionally to PHB4_SLOT_LINK_START. 7. Once slot reset is complete opal pci-core state machine will call slot->ops.completed_sm_run(). 8. For PHB4 this branches newly introduced 'phb4_slot_sm_run_completed()'. 9. Inside this function we mark the CAPP as disabled and un-register the opal syncer phb4_host_sync_reset(). 10. Optionally if the slot reset was unsuccessful disable fast-reboot. **************************** Notes: **************************** a. Function 'disable_capi_mode()' performs various sanity tests on CAPP to to determine if its ok to disable it and perform necessary xscoms to disable it. However the current implementation proposed in this patch is a skeleton one that just does sanity tests. A followup patch will be proposed that implements the xscoms necessary to disable CAPP. b. The sequence expects that Opal PCI reset state machine makes forward progress hence needs someone to call slot->ops.run_sm(). This can be either from phb4_host_sync_reset() or opal_pci_poll(). Reviewed-by: Frederic Barrat <fbarrat@linux.ibm.com> Reviewed-by: Christophe Lombard <clombard@linux.vnet.ibm.com> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2019-01-16capp/phb4: Force CAPP to PCIe mode during kernel shutdownVaibhav Jain1-0/+34
This patch introduces a new opal syncer for PHB4 named phb4_host_sync_reset(). We register this opal syncer when CAPP is activated successfully in phb4_set_capi_mode() so that it will be called at kernel shutdown during fast-reset. During kernel shutdown the function will then repeatedly call phb->ops->set_capi_mode() to switch switch CAPP to PCIe mode. In case set_capi_mode() indicates its OPAL_BUSY, which indicates that CAPP is still transitioning to new state; it calls slot->ops.run_sm() to ensure that Opal slot reset state machine makes forward progress. Reviewed-by: Frederic Barrat <fbarrat@linux.ibm.com> Reviewed-by: Christophe Lombard <clombard@linux.vnet.ibm.com> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2019-01-16phb4/capp: Update and re-factor phb4_set_capi_mode()Vaibhav Jain1-35/+53
Presently phb4_set_capi_mode() performs certain CAPP checks like, checking of CAPP ucode loaded or checks if CAPP is still in recovery, even when the requested mode is to switch to PCI mode. Hence this patch updates and re-factors phb4_set_capi_mode() to make sure CAPP related checks are only performed when request to enable CAPP is made by mode==OPAL_PHB_CAPI_MODE_CAPI/DMA_TVT1. We also update other possible modes requests to return a more appropriate status code based on if CAPP is activated or not. Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Reviewed-by: Christophe Lombard <clombard@linux.vnet.ibm.com> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Reviewed-by: Frederic Barrat <fbarrat@linux.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2019-01-16capp/phb: Introduce 'struct capp' to hold capp related info in 'struct phb'Vaibhav Jain1-7/+47
Previously struct proc_chip member 'capp_phb3_attached_mask' was used for Power-8 to keep track of PHB attached to the single CAPP on the chip. CAPP on that chip supported a flexible PHB assignment scheme. However since then new chips only support a static assignment i.e a CAPP can only be attached to a specific PEC. Hence instead of using 'proc_chip.capp_phb4_attached_mask' to manage CAPP <-> PEC assignments which needs a global lock (capi_lock) to be updated, we introduce a new struct named 'capp' a pointer to which resides inside struct 'phb4'. Since updates to struct 'phb4' already happen in context of phb_lock; this eliminates the need to use mutex 'capi_lock' while updating 'capp_phb4_attached_mask'. This struct is also used to hold CAPP specific variables such as pointer to the 'struct phb' to which the CAPP is attached, 'capp_xscom_offset' which is the xscom offset to be added to CAPP registers in case there are more than 1 on the chip, 'capp_index' which is the index of the CAPP on the chip, and attached_pe' which is the process endpoint index to which CAPP is attached. Finally member 'chip_id' holds the chip-id thats used for performing xscom read/writes. Also new helpers named capp_xscom_read()/write() are introduced to make access to CAPP xscom registers easier. Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Reviewed-by: Christophe Lombard <clombard@linux.vnet.ibm.com> Reviewed-by: Frederic Barrat <fbarrat@linux.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-11-01phb4/capp: Only reset FIR bits that cause capp machine checkVaibhav Jain1-0/+17
During CAPP recovery do_capp_recovery_scoms() will reset the CAPP Fir register just after CAPP recovery is completed. This has an unintentional side effect of preventing PRD from analyzing and reporting this error. If PRD tries to read the CAPP FIR after opal has already reset it, then it logs a critical error complaining "No active error bits found". To prevent this from happening we update do_capp_recovery_scoms() to only reset fir bits that cause CAPP machine check (local xstop). This is done by reading the CAPP Fir Action0/1 & Mask registers and generating a mask which is then written on CAPP_FIR_CLEAR register. Cc: stable Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-11-01phb4: Check for RX errors after link trainingOliver O'Halloran1-3/+26
Some PHB4 PHYs can get stuck in a bad state where they are constantly retraining the link. This happens transparently to skiboot and Linux but will causes PCIe to be slow. Resetting the PHB4 clears the problem. We can detect this case by looking at the RX errors count where we check for link stability. This patch does this by modifying the link optimal code to check for RX errors. If errors are occurring we retrain the link irrespective of the chip rev or card. Normally when this problem occurs, the RX error count is maxed out at 255. When there is no problem, the count is 0. We chose 8 as the max rx errors value to give us some margin for a few errors. There is also a knob that can be used to set the error threshold for when we should retrain the link. ie nvram -p ibm,skiboot --update-config phb-rx-err-max=8 Signed-off-by: Oliver O'Halloran <oohall@gmail.com> Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-10-16phb4: Enable PHB MMIO-0/1 Bars only when mmio window existsVaibhav Jain1-2/+1
Presently phb4_probe_stack() will always enable PHB MMIO0/1 windows even if they doesn't exist in phy_map. Hence we do some minor shuffling in the phb4_probe_stack() so that MMIO-0/1 Bars are only enabled if there corresponding MMIO window exists in the phy_map. In case phy_map for an mmio window is '0' we set the corresponding BAR register to '0'. Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Reviewed-by: Oliver O'Halloran <oohall@gmail.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-10-16phb4/capp: Update the expected Eye-catcher for CAPP ucode lidVaibhav Jain1-2/+2
Currently on a FSP based P9 system load_capp_code() expects CAPP ucode lid header to have eye-catcher magic of 'CAPPPSLL'. However skiboot currently supports CAPP ucode only lids that have a eye-catcher magic of 'CAPPLIDH'. This prevents skiboot from loading the ucode with this error message: CAPP: ucode header invalid We fix this issue by updating load_capp_ucode() to use the eye-catcher value of 'CAPPLIDH' instead of 'CAPPPSLL'. Cc: stable Fixes: e50764d4f2b1("capi: Load capp microcode") Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Acked-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Reviewed-by: Christophe Lombard <clombard@linux.vnet.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-10-16phb4/capp: Use link width to allocate STQ engines to CAPPVaibhav Jain1-17/+29
Update phb4_init_capp_regs() to allocates STQ Engines to CAPP/PEC2 based on link width instead of always assuming it to x8. Also re-factor the function slightly to evaluate the link-width only once and cache it so that it can also be used to allocate DMA read engines. Cc: stable Fixes: 47c09cdfe7a3("phb4/capp: Calculate STQ/DMA read engines based on link-width for PEC") Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Reviewed-by: Frederic Barrat <fbarrat@linux.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-09-27phb4: Generate checkstop on AIB ECC corr/uncorr for DD2.0 partsMichael Neuling1-9/+34
On DD2.0 parts, PCIe ECC protection is not warranted in the response data path. Thus, for these parts, we need to flag any ECC errors detected from the adjacent AIB RX Data path so the part can be replaced. This patch configures the FIRs so that we escalate these AIB ECC errors to a checkstop so the parts can be replaced. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-09-20phb4: Re-factor phb4_fenced() and introduce phb4_dump_pec_err_regs()Vaibhav Jain1-30/+38
Couple of places in 'phb4.c' where we may want to dump the PEC's error registers. Hence we introduce a phb4_dump_pec_err_regs() that dumps all the PEC error registers and also update phb4->nfir_cache & phb4->pfir_cache for later use. Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Reviewed-by: Oliver O'Halloran <oohall@gmail.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-09-20phb4: Reset pfir and nfir if new errors reported during ETU resetVaibhav Jain1-0/+19
During fast-reboot new PEC errors can be latched even after ETU-Reset is asserted. This will result in values of variables nfir_cache and pfir_cache to be out of sync. During step-2 of CRESET nfir_cache and pfir_cache values are used to bring the PHB out of reset state. However if these variables are out as noted above of date the nfir/pfir registers are never reset completely and ETU still remains frozen. Hence this patch updates step-2 of phb4_creset to re-read the values of nfir/pfir registers to check if any new errors were reported after ETU-reset was asserted, report these new errors and reset the nfir/pfir registers. This should bring the ETU out of reset successfully. Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Tested-By: Vasant Hegde <hegdevasant@linux.vnet.ibm.com> Reviewed-by: Oliver O'Halloran <oohall@gmail.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-09-17phb4: Fix typo in disable lane eq codeMichael Neuling1-1/+1
In this commit commit 737c0ba3d72b8aab05a765a9fc111a48faac0f75 Author: Michael Neuling <mikey@neuling.org> Date: Thu Feb 22 10:52:18 2018 +1100 phb4: Disable lane eq when retrying some nvidia GEN3 devices We made a typo and set PH2 twice. This fixes it. It worked previously as if only phase 2 (PH2) is set it, skips phase 2 and phase 3 (PH3). Reported-by: Meng Li <shlimeng@cn.ibm.com> Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-09-13phb4: Don't probe a PHB if its gardedVaibhav Jain1-2/+11
Presently phb4_probe_stack() causes an exception while trying to probe a PHB if its garded. This causes skiboot to go into a reboot loop with following exception log: *********************************************** Fatal MCE at 000000003006ecd4 .probe_phb4+0x570 CFAR : 00000000300b98a0 <snip> Aborting! CPU 0018 Backtrace: S: 0000000031cc37e0 R: 000000003001a51c ._abort+0x4c S: 0000000031cc3860 R: 0000000030028170 .exception_entry+0x180 S: 0000000031cc3a40 R: 0000000000001f10 * S: 0000000031cc3c20 R: 000000003006ecb0 .probe_phb4+0x54c S: 0000000031cc3e30 R: 0000000030014ca4 .main_cpu_entry+0x5b0 S: 0000000031cc3f00 R: 0000000030002700 boot_entry+0x1b8 This is caused as phb4_probe_stack() will ignore all xscom read/write errors to enable PHB Bars and then tries to perform an mmio to read PHB Version registers that cause the fatal MCE. We fix this by ignoring the PHB probe if the first xscom_write() to populate the PHB Bar register fails, which indicates that there is something wrong with the PHB. Cc: stable Fixes: dc21b4db3a2e('hw/phb4: Add initial support') Reviewed-by: Oliver O'Halloran <oohall@gmail.com> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.vnet.ibm.com>
2018-09-13phb4: Workaround PHB errata with CFG write UR/CA errorsBenjamin Herrenschmidt1-1/+5
If the PHB encounters a UR or CA status on a CFG write, it will incorrectly freeze the wrong PE. Instead of using the PE# specified in the CONFIG_ADDRESS register, it will use the PE# of whatever MMIO occurred last. Work around this disabling freeze on such errors Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> Tested-By: Oliver O'Halloran <oohall@gmail.com> Signed-off-by: Stewart Smith <stewart@linux.vnet.ibm.com>
2018-09-13phb4: Handle allocation errors in phb4_eeh_dump_regs()Benjamin Herrenschmidt1-0/+4
If the zalloc fails (and it can be a rather large allocation), we will overwite memory at 0 instead of failing. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> Signed-off-by: Stewart Smith <stewart@linux.vnet.ibm.com>
2018-09-13phb4: Don't try to access non-existent PEST entriesBenjamin Herrenschmidt1-3/+3
In a POWER9 chip, some PHB4s have 256 PEs, some have 512. Currently, the diagnostics code retrieves 512 unconditionally, which is wrong and causes us to incorrectly report bogus values for the "high" PEs on the small PHBs. Use the actual number of implemented PEs instead Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> Signed-off-by: Stewart Smith <stewart@linux.vnet.ibm.com>
2018-08-15phb4: Disable 32-bit MSI in capi modeFrederic Barrat1-0/+9
If a capi device does a DMA write targeting an address lower than 4GB, it does so through a 32-bit operation, per the PCI spec. In capi mode, the first TVE entry is configured in bypass mode, so the address is valid. But with any (bad) luck, the address could be 0xFFFFxxxx, thus looking like a 32-bit MSI. We currently enable both 32-bit and 64-bit MSIs, so the PHB will interpret the DMA write as a MSI, which very likely results in an EEH (MSI with a bad payload size). We can fix it by disabling 32-bit MSI when switching the PHB to capi mode. Capi devices are 64-bit. Cc: stable Signed-off-by: Frederic Barrat <fbarrat@linux.ibm.com> Reviewed-by: Christophe Lombard <clombard@linux.vnet.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-08-15capp: Fix the capp recovery timeout comparisonVaibhav Jain1-1/+1
The current capp recovery timeout control loop in do_capp_recovery_scoms() uses a wrong comparison for return value of tb_compare(). This may cause do_capp_recovery_scoms() to report an timeout earlier than the 168ms stipulated time. The patch fixes this by updating the loop timeout control branch in do_capp_recovery_scoms() to use the correct enum tb_cmpval. Cc: Stable #6.0+ Fixes: 09b853cae0aa0("capi: Poll Err/Status register during CAPP recovery") Reported-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Reviewed-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com> Reviewed-by: Christophe Lombard <clombard@linux.vnet.ibm.com> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-08-13phb4/capp: Update DMA read engines set in APC_FSM_READ_MASK based on link-widthVaibhav Jain1-4/+18
Commit 47c09cdfe7a3("phb4/capp: Calculate STQ/DMA read engines based on link-width for PEC") update the CAPP init sequence by calculating the needed STQ/DMA-read engines based on link width and populating it in XPEC_NEST_CAPP_CNTL register. This however needs to be synchronized with the value set in CAPP APC FSM Read Machine Mask Register. Hence this patch update phb4_init_capp_regs() to calculate the link width of the stack on PEC2 and populate the same values as previously populated in PEC CAPP_CNTL register. Cc: stable # v5.7+ Fixes: 47c09cdfe7a3("phb4/capp: Calculate STQ/DMA read engines based on link-width for PEC") Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Reviewed-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-08-06phb4: Use the return value of phb4_fenced() in phb4_get_diag_data()Cyril Bur1-2/+3
phb4_get_diag_data() checks the flags for the PHB4_AIB_FENCED after having called phb4_fenced(). This information is returned by phb4_fenced(). This patch was prompted by an unused return value warning in Coverity. Fixes: CID 163734 Signed-off-by: Cyril Bur <cyril.bur@au1.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-08-06phb4: Track PEC index in dt and phb4 structReza Arbab1-0/+2
Knowing the PEC index is going to be important when we enable relaxed ordering, so store this value for later use. Signed-off-by: Alistair Popple <alistair@popple.id.au> [arbab@linux.ibm.com: Rebase/refactor original changes] Signed-off-by: Reza Arbab <arbab@linux.ibm.com> Reviewed-By: Alistair Popple <alistair@popple.id.au> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-08-01hw/phb4: Use local_alloc for phb4 structuresOliver O'Halloran1-2/+6
Struct phb4 is fairly heavyweight at 283664 bytes. On systems with 6x PHBs per socket this results in using 3.2MB of heap space the PHB structures alone. This is a fairly large chunk of our 12MB heap and on systems with particularly large PCIe topologies, or additional PHBs we can fail to boot because we cannot allocate space for the FDT blob. This patch switches to using local_alloc() for the PHB structures so they don't consume too large a portion of our 12MB heap space. Signed-off-by: Ryan Grimm <grimm@linux.vnet.ibm.com> Signed-off-by: Oliver O'Halloran <oohall@gmail.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-07-26hw/phb4: Fix unused value/parameter warningsAndrew Donnellan1-20/+27
Remove the phb4.c-specific CFLAGS that disable the unused value and unused parameter warnings, and cleanup the ensuing warnings. Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-07-26hw/phb4: Add helpers to dump the IODA tablesOliver O'Halloran1-0/+89
The IODA tables are stored inside the PHB itself rather than in memory. This makes accessing them slightly tedious, but the process is more or less the same for every table. This patch adds a helper function for dumping the contents of the IODA tables to help with debugging PHB issues. Signed-off-by: Oliver O'Halloran <oohall@gmail.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-07-26hw/phb4: Add a helper to dump the PELT-VOliver O'Halloran1-0/+40
The "Partitionable Endpoint Lookup Table (Vector)" is used by the PHB when processing EEH events. The PELT-V defines which PEs should be additionally frozen in the event of an error being flagged on a given PE. Knowing the state of the PELT-V is sometimes useful for debugging PHB issues so this patch adds a helper to dump it. Signed-off-by: Oliver O'Halloran <oohall@gmail.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-07-26hw/phb4: Print the PEs in the EEH dump in hexOliver O'Halloran1-1/+1
Linux always displays the PE number in hexidecimal while skiboot displays the PEST index (PE number) in decimal. This makes correlating errors between Skiboot and Linux more annoying than it should be so this patch makes Skiboot print the PEST number in hex. Signed-off-by: Oliver O'Halloran <oohall@gmail.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-07-19phb4: Reallocate PEC2 DMA-Read engines to improve GPU-Direct bandwidthVaibhav Jain1-3/+37
We reallocate additional 16/8 DMA-Read engines allocated to stack0/1 on PEC2 respectively. This is needed to improve bandwidth available to the Mellanox CX5 adapter when trying to read GPU memory (GPU-Direct). If kernel cxl driver indicates a request to allocate maximum possible DMA read engines when calling enable_capi_mode() and card is attached to PEC2/stack0 slot then we assume its a Mellanox CX5 adapter. We then allocate additional 16/8 extra DMA read engines to stack0 and stack1 respectively on PEC2. This is done by populating the XPEC_PCI_PRDSTKOVR and XPEC_NEST_READ_STACK_OVERRIDE as suggested by the h/w team. Signed-off-by: Christophe Lombard <clombard@linux.vnet.ibm.com> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-07-17phb4: Disable nodal scoped DMA accesses when PB pump mode is enabledAlistair Popple1-0/+11
By default when a PCIe device issues a read request via the PHB it is first issued with nodal scope. When accessing GPU memory the NPU does not know at the time of response if the requested memory page is off node or not. Therefore every read of GPU memory by a PHB is retried with larger scope which introduces bandwidth and latency issues. On smaller boxes which have pump mode enabled nodal and group scoped reads are treated the same and both types of request are broadcast to one chip. Therefore we can avoid the retry by disabling nodal scope on the PHB for these boxes. On larger boxes nodal (single chip) and group (multiple chip) scoped reads are treated differently. Therefore we avoid disabling nodal scope on large boxes which have pump mode disabled to avoid all PHB requests being broadcast to multiple chips. Signed-off-by: Alistair Popple <alistair@popple.id.au> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-07-10capi: Select the correct IODA table entry for the mbt cache.Christophe Lombard1-9/+9
With the current code, the capi mmio window is not correctly configured in the IODA table entry. The first entry (generally the non-prefetchable BAR) is overwrriten. This patch sets the capi window bar at the right place. Signed-off-by: Christophe Lombard <clombard@linux.vnet.ibm.com> Reviewed-by: Vaibhav Jain <vaibhav@linux.ibm.com> Reviewed-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com> Acked-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-07-04phb4/capp: Calculate STQ/DMA read engines based on link-width for PECVaibhav Jain1-9/+27
Presently in CAPI mode the number of STQ/DMA-read engines allocated on PEC2 for CAPP is fixed to 6 and 0-30 respectively irrespective of the PCI link width. These values are only suitable for x8 cards and quickly run out if a x16 card is plugged to a PEC2 attached slot. This usually manifests as CAPP reporting TLBI timeout due to these messages getting stalled due to insufficient STQs. To fix this we update enable_capi_mode() to check if PEC2 chiplet is in x16 mode and if yes then we allocate 4/0-47 STQ/DMA-read engines for the CAPP traffic. Cc: stable # v5.7+ Fixes: 37ea3cfdc852("capi: Enable capi mode for PHB4") Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-07-03phb4: Delay training till after PERST is deassertedMichael Neuling1-0/+14
This helps some cards train on the second PERST (ie fast-reboot). The reason is not clear why but it helps, so YOLO! Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-07-03phb4: Move training trace logging to next state.Michael Neuling1-2/+2
I'm going to defer training to this state soon, so move the tracing first. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-07-03phb4: Minimise wait when moving through FRESET statesMichael Neuling1-1/+1
We want to get through this as fast as possible so minimise by removing msecs_to_tb() call. Changes number passed from 512 -> 1. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-05-22capi: Add a comment for the Transport Control RegisterChristophe Lombard1-1/+5
The transport control register needs to be loaded in two steps: Once the register values have been set, we have to write bit 63 to a '1', which loads the register values into the ci store buffer logic. Bit 63 always reads back as a zero but to load the ci store buffer values in capp the transition of 0 to 1 of bit 63 must be seen. A new comment is added in the code to avoid confusion and to precise the feature of this register. Signed-off-by: Christophe Lombard <clombard@linux.vnet.ibm.com> Reviewed-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Acked-by: Vaibhav Jain <vaibhav@linux.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-05-11phb4: Print WOF registers on fence detectRussell Currey1-1/+7
Without the WOF registers it's hard to figure out what went wrong first, so print those when we print the FIRs when a fence is detected. Suggested-by: Mike Perez <perezma@us.ibm.com> Signed-off-by: Russell Currey <ruscur@russell.cc> Reviewed-by: Vaibhav Jain <vaibhav@linux.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-05-02phb4: Hardware init updatesRussell Currey1-3/+3
CFG Write Request Timeout was incorrectly set to informational and not fatal for both non-CAPI and CAPI, so set it to fatal. This was a mistake in the specification. Correcting this fixes a niche bug in escalation (which is necessary on pre-DD2.2) that can cause a checkstop due to a NCU timeout. In addition, set the values in the timeout control registers to match. This fixes an extremely rare and unreproducible bug, though the current timings don't make sense since they're higher than the NCU timeout (16) which will checkstop the machine anyway. Signed-off-by: Russell Currey <ruscur@russell.cc> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> # CAPI Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-04-11phb4: Restore bus numbers after CRSMichael Neuling1-12/+1
Currently we restore PCIe bus numbers right after the link is up. Unfortunately as this point we haven't done CRS so config space may not be accessible. This moves the bus number restore till after CRS has happened. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-04-11phb4: Enable the PCIe slotcap on pluggable slotsOliver O'Halloran1-0/+20
Enables reporting of slot status information, etc in the config space of the root complex. Currently this is only used to set the slot power limit in our generic PCI code, but we might use it for other things later on. Signed-off-by: Oliver O'Halloran <oohall@gmail.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-04-11interrupts: Create an "interrupts" property in the OPAL nodeBenjamin Herrenschmidt1-1/+1
Deprecate the old "opal-interrupts", it's still there, but the new property follows the standard and allow us to specify whether an interrupt is level or edge sensitive. Similarly create "interrupt-names" whose content is identical to "opal-interrupts-names". Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-04-11capi: Keep the current mmio windows in the mbt cache table.Christophe Lombard1-32/+36
When the phb is used as a CAPI interface, the current mmio windows list is cleaned before adding the capi and the prefetchable memory (M64) windows, which implies that the non-prefetchable BAR is no more configured. This patch allows to set only the mbt bar to pass capi mmio window and to keep, as defined, the other mmio values (M32 and M64). Signed-off-by: Christophe Lombard <clombard@linux.vnet.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-04-10phb4: Do not set the PBCQ Tunnel BAR register when enabling capi mode.Philippe Bergheaud1-19/+0
The cxl driver will set the capi value, like other drivers already do. Signed-off-by: Philippe Bergheaud <felix@linux.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-04-10phb4: Quieten and improve "Timeout waiting for electrical link"Benjamin Herrenschmidt1-3/+2
This happens normally if a slot doesn't have a working HW presence detect and relies instead of inband presence detect. The message we display is scary and not very useful unless ou are debugging, so quiten it up and change it to something more meaningful. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> Acked-By: Michael Neuling <mikey@neuling.org> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
2018-04-09phb4: set TVT1 for tunneled operations in capi modePhilippe Bergheaud1-15/+7
The ASN indication is used for tunneled operations (as_notify and atomics). Tunneled operation messages can be sent in PCI mode as well as CAPI mode. The address field of as_notify messages is hijacked to encode the LPID/PID/TID of the target thread, so those messages should not go through address translation. Therefore bit 59 is part of the ASN indication. This patch sets TVT#1 in bypass mode when capi mode is enabled, to prevent as_notify messages from being dropped. Signed-off-by: Philippe Bergheaud <felix@linux.ibm.com> Reviewed-by: Christophe Lombard <clombard@linux.vnet.ibm.com> Signed-off-by: Stewart Smith <stewart@linux.ibm.com>