xive/p10: Add a XIVE2 driver

The XIVE2 interrupt controller of the POWER10 processor follows the
same logic than on POWER9 but the HW interface has been largely
reviewed.  It has a new register interface, different BARs, extra
VSDs, new layout for the XIVE structures, and a set of new features
which are described below.

The OPAL XIVE2 driver code activating this controller was duplicated
from P9 for clarity as the registers and structures have changed
considerably. The same OPAL interface is implemented for OS
compatibility and it should not impact existing Linux kernels, KVM
included. Guest OS is not impacted either.

Support for new features will be implemented in time and will require
new support from the OS.

* XIVE2 BARS

The interrupt controller BARs have a different layout outlined below.
Each sub-engine has now own its range and the indirect TIMA access was
replaced with a set of pages, one per CPU, under the IC BAR:

  - IC BAR (Interrupt Controller)
    . 4 pages, one per sub-engine
    . 128 indirect TIMA pages
  - TM BAR (Thread Interrupt Management Area)
    . 4 pages
  - ESB BAR (ESB pages for IPIs)
    . up to 1TB
  - END BAR (ESB pages for ENDs)
    . up to 2TB
  - NVC BAR (Notification Virtual Crowd)
    . up to 128
  - NVPG BAR (Notification Virtual Process and Group)
    . up to 1TB
  - Direct mapped Thread Context Area (reads & writes)

OPAL does not use the grouping and crowd capability.

* Virtual Structure Tables

XIVE2 adds new tables types and also changes the field layout of the END
and NVP Virtualization Structure Descriptors.

  - EAS
  - END new layout
  - NVT was splitted in :
    . NVP (Processor), 32B
    . NVG (Group), 32B
    . NVC (Crowd == P9 block group) 32B
  - IC for remote configuration
  - SYNC for cache injection
  - ERQ for event input queue

The setup is slighly different on XIVE2 because the indexing has changed
for some of the tables, block ID or the chip topology ID can be used.

* XIVE2 features

SCOM and MMIO registers have a new layout and XIVE2 adds a new global
capability and configuration registers.

The lowlevel hardware offers a set of new features among which :

  - cache injection mechanism
  - 4 cache watch engines
  - a configurable number of priorities : 1 -8
  - StoreEOI with load-after-store ordering is activated by default
  - new sync/kill operations for cache operations

Other features will have some impact on the Hypervisor and guest OS
when activated, but this is not required for initial support of the
controller.

  - Gen2 TIMA layout
  - A P9-compat mode, or Gen1, TIMA toggle bit for SW compatibility
  - Automatic Context save & restore
  - increase to 24bit for VP number
  - New escalations schems : ESB, Adaptive, CPPR

POWER10 adds support for User interrupts. When configured, the XIVE2
controller can notify directly user processes using the Event Based
Branch exception line of the thread. If not running, the OS is
notified through an escalation event. New OPAL and PAPR interfaces
will be required and OS support needs to be studied.

* XIVE2 P9-compat mode, or Gen1

The thread interrupt management area (TIMA) is a set of pages mapped
in the Hypervisor and in the guest OS address space giving access to
the interrupt thread context registers for interrupt management, ACK,
EOI, CPPR, etc.

XIVE2 changes slightly the TIMA layout with extra bits for the new
features, larger CAM lines and the controller provides configuration
switches for backward compatibility. This is called the XIVE2
P9-compat mode, of Gen1 TIMA. It impacts the layout of the TIMA and
the availability of the internal features associated with it,
Automatic Save & Restore for instance. Using a P9 layout also means
setting the controller in such a mode at init time.

The XIVE2 driver in OPAL chooses to initialize the XIVE2 controller
with a XIVE2/P10 TIMA directly because the layouts are compatible with
the Linux PowerNV and the guest OSes expectations.

For KVM support, the OPAL calls abstract the HW interface and no
assumption is made on the OS CAM line width.

* Activating new XIVE2 features

Everything related to OPAL internals such as the use of the new cache
sync mechanism can be implemented in time without impact on the OS.

Other features will require new device tree properties exposed to the
OS and extra support for the OS. Automatic Context save & restore is
one of the first feature which should be looked at.

* XICS-over-XICS driver (P8 compatibility)

The P8 emulation mode is an OPAL compat interface used for Linux
kernels which did not have XIVE native support. This was useful for
POWER9 bringup but it is much less now. As it was adding a lot of
complexity and reducing the interrupt controller resources, this mode
is not available in the XIVE2 driver for POWER10.

It will still be possible to add this compat mode in the future if
required. The OS will have to reset the driver at boot time, like on
POWER9.

* Impact on other drivers (PSI, PHB, NPU)

Interrupts are allocated in a very similar way. Each controller might
have different ESB characteristics, StoreEOI support, 64K pages for
PSI. All is in place to support these changes already.

PHB5 will have support for "address-based trigger mode", probably in
the DD2.0 time frame when verification is completed. When activated,
the XIVE IC ESB pages will be used instead of the PHB ESB pages for a
lower interrupt latency.

LSI will still use old fashion triggers without StoreEOI.

* Yet to be addressed :

 - OPAL P10 interface incomplete (stop states)
 - Clarify the PHB5 strategy regarding the use of the XIVE IC ESB
   pages instead of the PHB ones when address-based trigger mode is
   supported.

Signed-off-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Vasant Hegde <hegdevasant@linux.vnet.ibm.com>

2 files changed, 13 insertions, 3 deletions

diff --git a/core/fast-reboot.c b/core/fast-reboot.c
index ac9b3b2..9f92525 100644
--- a/core/fast-reboot.c
+++ b/core/fast-reboot.c
@@ -262,6 +262,8 @@ static void cleanup_cpu_state(void)
 
 	if (proc_gen == proc_gen_p9)
 		xive_cpu_reset();
+	else if (proc_gen == proc_gen_p10)
+		xive2_cpu_reset();
 
 	/* Per core cleanup */
 	if (cpu_is_thread0(cpu) || cpu_is_core_chiplet_primary(cpu)) {
@@ -381,6 +383,8 @@ void __noreturn fast_reboot_entry(void)
 
 	if (proc_gen == proc_gen_p9)
 		xive_reset();
+	else if (proc_gen == proc_gen_p10)
+		xive2_reset();
 
 	/* Let the CPU layer do some last minute global cleanups */
 	cpu_fast_reboot_complete();
diff --git a/core/init.c b/core/init.c
index 0bf4ab2..e389695 100644
--- a/core/init.c
+++ b/core/init.c
@@ -1225,8 +1225,11 @@ void __noreturn __nomcount main_cpu_entry(const void *fdt)
 	if (proc_gen == proc_gen_p8)
 		cpu_set_ipi_enable(true);
 
-	/* On P9, initialize XIVE */
-	init_xive();
+	/* On P9 and P10, initialize XIVE */
+	if (proc_gen == proc_gen_p9)
+		init_xive();
+	else if (proc_gen == proc_gen_p10)
+		xive2_init();
 
 	/* Grab centaurs from device-tree if present (only on FSP-less) */
 	centaur_init();
@@ -1437,7 +1440,10 @@ void __noreturn __secondary_cpu_entry(void)
 	mtmsrd(MSR_RI, 1);
 
 	/* Some XIVE setup */
-	xive_cpu_callin(cpu);
+	if (proc_gen == proc_gen_p9)
+		xive_cpu_callin(cpu);
+	else if (proc_gen == proc_gen_p10)
+		xive2_cpu_callin(cpu);
 
 	/* Wait for work to do */
 	while(true) {