Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

Interrupt is allowed during exception handling.
There are warning messages if the kernel enables the configuration
'CONFIG_DEBUG_ATOMIC_SLEEP=y'.

BUG: sleeping function called from invalid context at kernel/locking/rwsem.c:23
in_atomic(): 0, irqs_disabled(): 1, pid: 43, name: ash
CPU: 0 PID: 43 Comm: ash Tainted:  G	 W	 4.15.0-rc8-00089-g89ffdae-dirty #17
Call Trace:
[<000000009abb1587>] walk_stackframe+0x0/0x7a
[<00000000d4f3d088>] ___might_sleep+0x102/0x11a
[<00000000b1fd792a>] down_read+0x18/0x28
[<000000000289ec01>] do_page_fault+0x86/0x2f6
[<00000000012441f6>] _do_fork+0x1b4/0x1e0
[<00000000f46c3e3b>] ret_from_syscall+0xa/0xe

Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Zong Li <zong@andestech.com>
Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>

There isn't a hard dependency of the Xilinx AXI-PCIe host bridge on any
architecture.  For example: at SiFive we map RISC-V cores to Xilinx
FPGAs and connect the Xilinx IP via a TileLink adapter, so the RISC-V
Linux port will need to be able to enable PCIE_XILINX in order to have
PCIe support.

This patch decouples the PCIE_XILINX support from ARCH.  Instead it just
depends on OF_PCI, which I believe is the only true dependency.

Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>

This isn't actually how I want to do it, I just needed something right
now.

Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>

The RISC-V ISA defines a per-hart real-time clock and timer, which is
present on all systems.  The clock is accessed via the 'rdtime'
pseudo-instruction (which reads a CSR), and the timer is set via an SBI
call.

This driver attempts to split out the RISC-V ISA specific mechanisms of
accessing the hardware from the clocksource driver by taking a pair of
function pointers to issue the actual RISC-V specific instructions.

Signed-off-by: Dmitriy Cherkasov <dmitriy@oss-tech.org>
Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>

This fixes: [    0.010000] cpu cpu0: Error -2 creating of_node link
... which you get for every CPU on all architectures with a OF cpu/ node.

This affects riscv, nios, etc.

Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>

This patch adds a driver for the Platform Level Interrupt Controller
(PLIC) specified as part of the RISC-V supervisor level ISA manual.
The PLIC connocts global interrupt sources to the local interrupt
controller on each hart.  A PLIC is present on all RISC-V systems.

Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>

This patch adds documentation for the platform-level interrupt
controller (PLIC) found in all RISC-V systems.  This interrupt
controller routes interrupts from all the devices in the system to each
hart-local interrupt controller.

Note: the DTS bindings for the PLIC aren't set in stone yet, as we might
want to change how we're specifying holes in the hart list.

Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>

This patch adds a driver that manages the local interrupts on each
RISC-V hart, as specifiec by the RISC-V supervisor level ISA manual.
The local interrupt controller manages software interrupts, timer
interrupts, and hardware interrupts (which are routed via the
platform level interrupt controller).  Per-hart local interrupt
controllers are found on all RISC-V systems.

Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>

This patch adds documentation on the RISC-V local interrupt controller,
which is a per-hart interrupt controller that manages all interrupts
entering a RISC-V hart.  This interrupt controller is present on all
RISC-V systems.

Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>

It appears that openrisc copied arm64's GENERIC_IRQ_MULTI_HANDLER code
(which came from arm).  I wanted to make this generic so I could use it
in the RISC-V port.  This patch converts the openrisc code to use the
generic version.

Acked-by: Stafford Horne <shorne@gmail.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

It appears arm64 copied arm's GENERIC_IRQ_MULTI_HANDLER code, but made
it unconditional.  I wanted to make this generic so it could be used by
the RISC-V port.  This patch converts the arm64 code to use the new
generic code, which simply consists of deleting the arm64 code and
setting MULTI_IRQ_HANDLER instead.

Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

This converts the ARM port to use the recently added
GENERIC_IRQ_MULTI_HANDLER, which is essentially just a copy of ARM's
existhing MULTI_IRQ_HANDLER.  The only changes are:

* handle_arch_irq is now defined in a generic C file instead of an
  arm-specific assembly file.
* handle_arch_irq is not marked as __ro_after_init.

Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

The old mechanism for handling IRQs on RISC-V was pretty ugly: the arch
code looked at the Kconfig entry for our first-level irqchip driver and
called into it directly.

This patch uses the new generic IRQ handling infastructure, which
essentially just deletes a bunch of code.  This does add an additional
load to the interrupt latency, but there's a lot of tuning left to be
done there on RISC-V so I think it's OK for now.

Reviewed-by: Christoph Hellwig <hch@lst.de>
Acked-by: Stafford Horne <shorne@gmail.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

It looks like the arm irqchip registration mechanism has been copied
into a handful of ports, including arm64 and openrisc.  I want to use
this in the RISC-V port, so I thought it would be good to make this
generic instead.

This patch copies the arm definition of CONFIG_MULTI_IRQ_HANDLER into
kernel/irq under CONFIG_GENERIC_MULTI_IRQ_HANDLER.  This patch is
currently all dead code, but it will be enabled in the various other
architectures in subsequent patches.

Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

We use the asm-generic implementation, so it will always be there.

XXX: this works around the fact that the probe trips up an unresolved
pagefaul with split supervisor/user address spaces.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

This code lives entirely within the RISC-V arch code.  I've left it
within an "#ifdef CONFIG_EARLY_PRINTK" despite always having
EARLY_PRINTK support on RISC-V just in case someone wants to remove
it.

Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

RISC-V doesn't currently specify a mechanism for enabling or disabling
CPUs.  Instead, we assume that all CPUs are enabled on boot, and if
someone wants to save power we instead put a CPU to sleep via a WFI
loop.  Future systems may have an explicit mechanism for putting a CPU
to sleep, so we're standardizing the device tree entry for when that
happens.

We're not defining a spin-table based interface to the firmware, as the
plan is to handle this entirely within the kernel instead.

CC: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

Someone must have read the device tree specification incorrectly,
because we were putting timebase-frequency in the wrong place.  This
corrects the issue, moving it from

/ {
        cpus {
                timebase-frequency = X;
        }
}

to

/ {
        cpus {
                cpu@0 {
                        timebase-frequency = X;
                }
        }
}

This is great, because the timer's frequency should really be a per-cpu
quantity on RISC-V systems since there's a timer per CPU.  This should
lead to some cleanups in our timer driver.

CC: Wesley Terpstra <wesley@sifive.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

The RISC-V ISA defines a simple console that is availiable via SBI calls
on all systems.  This patch adds a driver for this console interface
that can act as both a target for early printk and as the system
console.  The core arch code already enables the early printk support
when CONFIG_HVC_RISCV_SBI is defined.

There is one checkpatch.pl warning here: to check the MAINTAINERS file.
They're all matched by the "K: riscv" line.

Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

satp is the name used by the current privileged spec 1.10, use it
instead of the old name.  The most recent release binutils release
(2.29) doesn't know about the satp name yet, so stick to the name from
the previous privileged ISA release and comment on the fact.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

init_mm.pgd (aka swapped_pgd) gets relocated like all other kernel
symbols by the elf loader, so there is no need to reload it from satp.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

All RISC-V platforms today lack an IOMMU. However, legacy PCI devices sometimes
require DMA-memory to be in the low 32 bits. To make this work, we enable the
software-based bounce buffers from swiotlb. They only impose overhead when the
device in question cannot address the full 64-bit address space, so a perfect fit.

This patch assumes that DMA is coherent with the processor and the PCI bus. It
also assumes that the processor and devices share a common address space. This is
true for all RISC-V platforms so far.

This patch allows devices that require memory that can be addressed
using 32-bit addresses to work easily on RISC-V systems.  The newly
improved dma-direct ops will tap into this pool automatically for
32-bit addressing.

Based on an earlier patch from Wesley W. Terpstra.

CC: Wesley W. Terpstra <terpstra@sifive.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

RISC-V systems perform TLB shootdows via the SBI, which currently
performs an IPI to each of the remote harts which then performs a local
TLB flush.  This process is a bit on the slow side, but we can at least
speed it up for some common cases by restricting the set of harts to
shoot down to the actual set of harts that are currently participating
in the given mm context, as opposed to the entire system.

This should provide a measurable performance increase, but we haven't
measured it.  Regardless, it seems like obviously the right thing to do
here.

Signed-off-by: Andrew Waterman <andrew@sifive.com>
Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>

The SUM bit is enabled at the beginning of the copy_{to,from}_user and
{get,put}_user routines, and cleared before they return.  But these user
copy helper can be interrupted by exceptions, in which case the SUM bit
will remain set, which leads to elevated privileges for the code running
in exception context, as that can now access userspace address space
unconditionally.  This frequently happens when the user copy routines
access freshly allocated user memory that hasn't been faulted in, and a
pagefault needs to be taken before the user copy routines can continue.

Fix this by unconditionally clearing SUM when the exception handler is
called - the restore code will automatically restore it based on the
saved value.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>