\chapter{Hypervisor Extension, Version 0.4} \label{hypervisor} {\bf Warning! This draft specification is likely to change before being accepted as standard by the RISC-V Foundation.} This chapter describes the RISC-V hypervisor extension, which virtualizes the supervisor-level architecture to support the efficient hosting of guest operating systems atop a type-1 or type-2 hypervisor. The hypervisor extension changes supervisor mode into {\em hypervisor-extended supervisor mode} (HS-mode, or {\em hypervisor mode} for short), where a hypervisor or a hosting-capable operating system runs. The hypervisor extension also adds another level of address translation, from {\em guest physical addresses} to supervisor physical addresses, to virtualize the memory and memory-mapped I/O subsystems for a guest operating system. HS-mode acts the same as S-mode, but with additional instructions and CSRs that control the new level of address translation and support hosting a guest OS in virtual S-mode (VS-mode). Regular S-mode operating systems can execute without modification either in HS-mode or as VS-mode guests. In HS-mode, an OS or hypervisor interacts with the machine through the same SBI as an OS normally does from S-mode. An HS-mode hypervisor is expected to implement the SBI for its VS-mode guest. The hypervisor extension is enabled by setting bit 7 in the {\tt misa} CSR, which corresponds to the letter H. When {\tt misa}[7] is clear, the hart behaves as though this extension were not implemented, and attempts to use hypervisor CSRs or instructions raise an illegal instruction exception. Implementations that include the hypervisor extension are encouraged not to hardwire {\tt misa}[7], so that the extension may be disabled. \begin{commentary} This draft is based on earlier proposals by John Hauser and Paolo Bonzini. \end{commentary} \begin{commentary} The baseline privileged architecture is designed to simplify the use of classic virtualization techniques, where a guest OS is run at user-level, as the few privileged instructions can be easily detected and trapped. The hypervisor extension improves virtualization performance by reducing the frequency of these traps. The hypervisor extension has been designed to be efficiently emulable on platforms that do not implement the extension, by running the hypervisor in S-mode and trapping into M-mode for hypervisor CSR accesses and to maintain shadow page tables. The majority of CSR accesses for type-2 hypervisors are valid S-mode accesses so need not be trapped. Hypervisors can support nested virtualization analogously. \end{commentary} \section{Privilege Modes} The current {\em virtualization mode}, denoted V, indicates whether the hart is currently executing in a guest. When V=1, the hart is either in virtual S-mode (VS-mode), or in virtual U-mode (VU-mode) under a guest OS running in VS-mode. When V=0, the hart is either in M-mode, in HS-mode, or in U-mode under an OS running in HS-mode. The virtualization mode also indicates whether two-level address translation is active (V=1) or inactive (V=0). Table~\ref{h-operating-modes} lists the possible operating modes of a RISC-V hart with the hypervisor extension. \begin{table*}[h!] \begin{center} \begin{tabular}{|c|c||l|l|l|} \hline Virtualization & Privilege & \multirow{2}{*}{Abbreviation} & \multirow{2}{*}{Name} & Two-Level \\ Mode (V) & Encoding & & & Translation \\ \hline 0 & 0 & U-mode & User mode & Off \\ 0 & 1 & HS-mode & Hypervisor-extended supervisor mode & Off \\ 0 & 3 & M-mode & Machine mode & Off \\ \hline 1 & 0 & VU-mode & Virtual user mode & On \\ 1 & 1 & VS-mode & Virtual supervisor mode & On \\ \hline \end{tabular} \end{center} \caption{Operating modes with the hypervisor extension.} \label{h-operating-modes} \end{table*} \section{Hypervisor and Virtual Supervisor CSRs} An OS or hypervisor running in HS-mode uses the supervisor CSRs to interact with the exception, interrupt, and address-translation subsystems. Additional CSRs are provided to HS-mode, but not to VS-mode, to manage two-level address translation and to control the behavior of a VS-mode guest: {\tt hstatus}, {\tt hedeleg}, {\tt hideleg}, {\tt hcounteren}, and {\tt hgatp}. Furthermore, several {\em virtual supervisor} CSRs (VS CSRs) are replicas of the normal supervisor CSRs. For example, {\tt vsstatus} is the VS CSR that duplicates the usual {\tt sstatus} CSR. When V=1, the VS CSRs substitute for the corresponding supervisor CSRs, taking over all functions of the usual supervisor CSRs except as specified otherwise. Instructions that normally read or modify a supervisor CSR shall instead access the corresponding VS CSR. In VS-mode, an attempt to read or write a VS CSR directly by its own separate CSR address causes an illegal instruction exception. The VS CSRs can be directly accessed only from M-mode or HS-mode. While V=1, the normal HS-level supervisor CSRs retain their values but do not affect the behavior of the machine unless specifically documented to do so. Conversely, when V=0, the VS CSRs do not ordinarily affect the behavior of the machine other than being readable and writable by CSR instructions. A few standard supervisor CSRs ({\tt scounteren} and, if the N extension is implemented, {\tt sedeleg} and {\tt sideleg}) have no matching VS CSR. These supervisor CSRs continue to have their usual function and accessibility even when V=1, except with VS-mode and VU-mode substituting for HS-mode and U-mode. Hypervisor software is expected to manually swap the contents of these registers as needed. In this section, we use the term {\em HSXLEN} to refer to the effective XLEN when executing in HS-mode, and {\em VSXLEN} to refer to the effective XLEN when executing in VS-mode. \subsection{Hypervisor Status Register ({\tt hstatus})} The {\tt hstatus} register is an HSXLEN-bit read/write register formatted as shown in Figure~\ref{hstatusreg-rv32} when HSXLEN=32 and Figure~\ref{hstatusreg} when HSXLEN=64. The {\tt hstatus} register provides facilities analogous to the {\tt mstatus} register that track and control the exception behavior of a VS-mode guest. \begin{figure*}[h!] {\footnotesize \begin{center} \setlength{\tabcolsep}{4pt} \begin{tabular}{YcccRcccccWc} \\ \instbitrange{31}{23} & \instbit{22} & \instbit{21} & \instbit{20} & \instbitrange{19}{10} & \instbit{9} & \instbit{8} & \instbit{7} & \instbit{6} & \instbit{5} & \instbitrange{4}{1} & \instbit{0} \\ \hline \multicolumn{1}{|c|}{\wpri} & \multicolumn{1}{c|}{VTSR} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{VTVM} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{SP2V} & \multicolumn{1}{c|}{SP2P} & \multicolumn{1}{c|}{SPV} & \multicolumn{1}{c|}{STL} & \multicolumn{1}{c|}{VSBE} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{SPRV} \\ \hline 9 & 1 & 1 & 1 & 10 & 1 & 1 & 1 & 1 & 1 & 4 & 1 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Hypervisor-mode status register ({\tt hstatus}) for RV32.} \label{hstatusreg-rv32} \end{figure*} \begin{figure*}[h!] {\footnotesize \begin{center} \setlength{\tabcolsep}{4pt} \begin{tabular}{MFScccc} \\ \instbitrange{HSXLEN-1}{34} & \instbitrange{33}{32} & \instbitrange{31}{23} & \instbit{22} & \instbit{21} & \instbit{20} & \\ \hline \multicolumn{1}{|c|}{\wpri} & \multicolumn{1}{c|}{VSXL[1:0]} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{VTSR} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{VTVM} & \\ \hline HSXLEN-34 & 2 & 9 & 1 & 1 & 1 & \\ \end{tabular} \begin{tabular}{cOcccccFc} \\ & \instbitrange{19}{10} & \instbit{9} & \instbit{8} & \instbit{7} & \instbit{6} & \instbit{5} & \instbitrange{4}{1} & \instbit{0} \\ \hline & \multicolumn{1}{|c|}{\wpri} & \multicolumn{1}{c|}{SP2V} & \multicolumn{1}{c|}{SP2P} & \multicolumn{1}{c|}{SPV} & \multicolumn{1}{c|}{STL} & \multicolumn{1}{c|}{VSBE} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{SPRV} \\ \hline & 10 & 1 & 1 & 1 & 1 & 1 & 4 & 1 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Hypervisor-mode status register ({\tt hstatus}) for RV64.} \label{hstatusreg} \end{figure*} The VSXL field controls the effective XLEN for VS-mode (known as VSXLEN), which may differ from the XLEN for HS-mode (HSXLEN). When HSXLEN=32, the VSXL field does not exist, and VSXLEN=32. When HSXLEN=64, VSXL is a \warl\ field that is encoded the same as the MXL field of {\tt misa}, shown in Table~\ref{misabase} on page~\pageref{misabase}. In particular, the implementation may hardwire VSXL so that VSXLEN=HSXLEN. If HSXLEN is changed from 32 to a wider width, and if field VSXL is not hardwired to a forced value, it gets the value corresponding to the widest supported width not wider than the new HSXLEN. The {\tt hstatus} fields VTSR and VTVM are defined analogously to the {\tt mstatus} fields TSR and TVM, but affect the trapping behavior of the SRET and virtual-memory management instructions only when V=1. The SPV bit (Supervisor Previous Virtualization Mode) is written by the implementation whenever a trap is taken into HS-mode. Just as the SPP bit in {\tt sstatus} is set to the privilege mode at the time of the trap, the SPV bit in {\tt hstatus} is set to the value of the virtualization mode V at the time of the trap. When an SRET instruction is executed when V=0, V is set to SPV. When a trap is taken into HS-mode, bits SP2V and SP2P are set to the values that SPV and the HS-level SPP had before the trap. When an SRET instruction is executed when V=0, the reverse assignments occur: after SPV and {\tt sstatus}.SPP have supplied the new virtualization and privilege modes, they are written with SP2V and SP2P, respectively. The STL bit (Supervisor Translation Level), which indicates which address-translation level caused an access-fault or page-fault exception, is also written by the implementation whenever a trap is taken into HS-mode. On an access or page fault due to guest physical address translation, STL is set to 1. For any other trap into HS-mode, STL is set to 0. The VSBE bit is a \warl\ field that controls the endianness of explicit memory accesses made from VS-mode. If VSBE=0, explicit load and store memory accesses made from VS-mode are little-endian, and if VSBE=1, they are big-endian. An implementation may hardwire VSBE to specify always the same endianness as for HS-mode. The SPRV bit modifies the privilege with which loads and stores execute in HS-mode. When SPRV=0, translation and protection behave as normal. When SPRV=1, load and store memory addresses are translated and protected, and endianness is applied, as though the current virtualization mode were set to {\tt hstatus}.SPV and the current privilege mode were set to the HS-level SPP. Table~\ref{h-sprv} enumerates the cases. \begin{table*}[h!] \begin{center} \begin{tabular}{|c|c|c||p{4.7in}|} \hline SPRV & SPV & SPP & Effect \\ \hline \hline 0 & -- & -- & Normal access; current privilege and virtualization modes apply. \\ \hline 1 & 0 & 0 & U-level access with HS-level translation and protection only. \\ \hline 1 & 0 & 1 & HS-level access with HS-level translation and protection only. \\ \hline 1 & 1 & 0 & VU-level access with two-level translation and protection. The HS-level MXR bit makes any executable page readable. {\tt vsstatus}.MXR makes readable those pages marked executable at the VS translation level, but only if readable at the guest-physical translation level. \\ \hline 1 & 1 & 1 & VS-level access with two-level translation and protection. The HS-level MXR bit makes any executable page readable. {\tt vsstatus}.MXR makes readable those pages marked executable at the VS translation level, but only if readable at the guest-physical translation level. {\tt vsstatus}.SUM applies instead of the HS-level SUM bit. \\ \hline \end{tabular} \end{center} \caption{Effect of SPRV on load and store translation and protection.} \label{h-sprv} \end{table*} An MRET or SRET instruction that changes the operating mode to U-mode, VS-mode, or VU-mode also sets SPRV=0. \subsection{Hypervisor Trap Delegation Registers ({\tt hedeleg} and {\tt hideleg})} By default, all traps at any privilege level are handled in M-mode, though M-mode usually uses the {\tt medeleg} and {\tt mideleg} CSRs to delegate some traps to HS-mode. The {\tt hedeleg} and {\tt hideleg} CSRs allow these traps to be further delegated to a VS-mode guest; their layout is the same as {\tt medeleg} and {\tt mideleg}. \begin{figure}[h!] {\footnotesize \begin{center} \begin{tabular}{@{}U} \instbitrange{HSXLEN-1}{0} \\ \hline \multicolumn{1}{|c|}{Synchronous Exceptions (\warl)} \\ \hline HSXLEN \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Hypervisor Exception Delegation Register {\tt hedeleg}.} \label{hedelegreg} \end{figure} \begin{figure}[h!] {\footnotesize \begin{center} \begin{tabular}{@{}U} \instbitrange{HSXLEN-1}{0} \\ \hline \multicolumn{1}{|c|}{Interrupts (\warl)} \\ \hline HSXLEN \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Hypervisor Interrupt Delegation Register {\tt hideleg}.} \label{hidelegreg} \end{figure} The {\tt hedeleg} and {\tt hideleg} registers are only active when V=1. When V=1, a trap that has been delegated to HS-mode (using {\tt medeleg} or {\tt mideleg}) is further delegated to VS-mode if the corresponding {\tt hedeleg} or {\tt hideleg} bit is set. When an access-fault or page-fault exception is caused by guest physical address translation, the trap is not delegated beyond HS-mode, regardless of the setting of {\tt hedeleg}. \subsection{Hypervisor Counter-Enable Register ({\tt hcounteren})} The counter-enable register {\tt hcounteren} is a 32-bit register that controls the availability of the hardware performance monitoring counters to the guest virtual machine. \begin{figure*}[h!] {\footnotesize \begin{center} \setlength{\tabcolsep}{4pt} \begin{tabular}{cccMcccccc} \instbit{31} & \instbit{30} & \instbit{29} & \instbitrange{28}{6} & \instbit{5} & \instbit{4} & \instbit{3} & \instbit{2} & \instbit{1} & \instbit{0} \\ \hline \multicolumn{1}{|c|}{HPM31} & \multicolumn{1}{c|}{HPM30} & \multicolumn{1}{c|}{HPM29} & \multicolumn{1}{c|}{...} & \multicolumn{1}{c|}{HPM5} & \multicolumn{1}{c|}{HPM4} & \multicolumn{1}{c|}{HPM3} & \multicolumn{1}{c|}{IR} & \multicolumn{1}{c|}{TM} & \multicolumn{1}{c|}{CY} \\ \hline 1 & 1 & 1 & 23 & 1 & 1 & 1 & 1 & 1 & 1 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Hypervisor counter-enable register ({\tt hcounteren}).} \label{hcounteren} \end{figure*} When the CY, TM, IR, or HPM{\em n} bit in the {\tt hcounteren} register is clear, attempts to read the {\tt cycle}, {\tt time}, {\tt instret}, or {\tt hpmcounter}{\em n} register while V=1 will cause an illegal instruction exception. When one of these bits is set, access to the corresponding register is permitted when V=1, unless prevented for some other reason. In VU-mode, a counter is not readable unless the applicable bits are set in both {\tt hcounteren} and {\tt scounteren}. {\tt hcounteren} must be implemented. However, any of the bits may contain a hardwired value of zero, indicating reads to the corresponding counter will cause an exception when V=1. Hence, they are effectively \warl\ fields. \subsection{Hypervisor Time Delta Registers ({\tt htimedelta}, {\tt htimedeltah})} The {\tt htimedelta} CSR is a read/write register that contains the delta between the value of the {\tt time} CSR and the value returned in VS-mode or VU-mode. That is, reading the {\tt time} CSR in VS or VU mode returns the sum of the contents of {\tt htimedelta} and the actual value of {\tt time}. \begin{figure*}[h!] {\footnotesize \begin{center} \begin{tabular}{@{}J} \instbitrange{63}{0} \\ \hline \multicolumn{1}{|c|}{\tt htimedelta} \\ \hline 64 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Hypervisor time delta register, HSXLEN=64.} \label{hdeltareg} \end{figure*} For HSXLEN=32 only, {\tt htimedelta} holds the lower 32 bits of the delta, and {\tt htimedeltah} holds the upper 32 bits of the delta. \begin{figure*}[h!] {\footnotesize \begin{center} \begin{tabular}{@{}J} \instbitrange{31}{0} \\ \hline \multicolumn{1}{|c|}{\tt htimedelta} \\ \hline \multicolumn{1}{|c|}{\tt htimedeltah} \\ \hline 32 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Hypervisor time delta registers, HSXLEN=32.} \label{hdeltahreg} \end{figure*} \subsection{Hypervisor Guest Address Translation and Protection Register ({\tt hgatp})} \label{sec:hgatp} The {\tt hgatp} register is an HSXLEN-bit read/write register, formatted as shown in Figure~\ref{rv32hgatp} for HSXLEN=32 and Figure~\ref{rv64hgatp} for HSXLEN=64, which controls guest physical address translation and protection. Similar to CSR {\tt satp}, this register holds the physical page number (PPN) of the guest-physical root page table; a virtual machine identifier (VMID), which facilitates address-translation fences on a per-virtual-machine basis; and the MODE field, which selects the address-translation scheme for guest physical addresses. When {\tt mstatus}.TVM=1, attempts to read or write {\tt hgatp} while executing in HS-mode will raise an illegal instruction exception. \begin{figure}[h!] {\footnotesize \begin{center} \begin{tabular}{cY@{}E@{}K} \instbit{31} & \instbitrange{30}{29} & \instbitrange{28}{22} & \instbitrange{21}{0} \\ \hline \multicolumn{1}{|c|}{MODE} & \multicolumn{1}{c|}{0 (\warl)} & \multicolumn{1}{c|}{VMID (\warl)} & \multicolumn{1}{c|}{PPN (\warl)} \\ \hline 1 & 2 & 7 & 22 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{RV32 Hypervisor guest address translation and protection register {\tt hgatp}.} \label{rv32hgatp} \end{figure} \begin{figure}[h!] {\footnotesize \begin{center} \begin{tabular}{@{}S@{}Y@{}E@{}K} \instbitrange{63}{60} & \instbitrange{59}{58} & \instbitrange{57}{44} & \instbitrange{43}{0} \\ \hline \multicolumn{1}{|c|}{MODE (\warl)} & \multicolumn{1}{c|}{0 (\warl)} & \multicolumn{1}{c|}{VMID (\warl)} & \multicolumn{1}{c|}{PPN (\warl)} \\ \hline 4 & 2 & 14 & 44 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{RV64 Hypervisor guest address translation and protection register {\tt hgatp}, for MODE values Bare, Sv39x4, and Sv48x4.} \label{rv64hgatp} \end{figure} Table~\ref{tab:hgatp-mode} shows the encodings of the MODE field for RV32 and RV64. When MODE=Bare, guest physical addresses are equal to supervisor physical addresses, and there is no further memory protection for a guest virtual machine beyond the physical memory protection scheme described in Section~\ref{sec:pmp}. In this case, the remaining fields in {\tt hgatp} have no effect. For RV32, the only other valid setting for MODE is Sv32x4, which is a modification of the usual Sv32 paged virtual-memory scheme, extended to support 34-bit guest physical addresses. For RV64, modes Sv39x4 and Sv48x4 are defined as modifications of the Sv39 and Sv48 paged virtual-memory schemes. All of these paged virtual-memory schemes are described in Section~\ref{sec:guest-addr-translation}. An additional RV64 scheme, Sv57x4, may be defined in a later version of this specification. The remaining MODE settings for RV64 are reserved for future use and may define different interpretations of the other fields in {\tt hgatp}. \begin{table}[h] \begin{center} \begin{tabular}{|c|c|l|} \hline \multicolumn{3}{|c|}{RV32} \\ \hline Value & Name & Description \\ \hline 0 & Bare & No translation or protection. \\ 1 & Sv32x4 & Page-based 34-bit virtual addressing (2-bit extension of Sv32). \\ \hline \hline \multicolumn{3}{|c|}{RV64} \\ \hline Value & Name & Description \\ \hline 0 & Bare & No translation or protection. \\ 1--7 & --- & {\em Reserved} \\ 8 & Sv39x4 & Page-based 41-bit virtual addressing (2-bit extension of Sv39). \\ 9 & Sv48x4 & Page-based 50-bit virtual addressing (2-bit extension of Sv48). \\ 10 & {\em Sv57x4} & {\em Reserved for page-based 59-bit virtual addressing.} \\ 11--15 & --- & {\em Reserved} \\ \hline \end{tabular} \end{center} \caption{Encoding of {\tt hgatp} MODE field.} \label{tab:hgatp-mode} \end{table} RV64 implementations are not required to support all defined RV64 MODE settings. A write to {\tt hgatp} with an unsupported MODE value is not ignored as it is for {\tt satp}. Instead, the fields of {\tt hgatp} are {\warl} in the normal way, when so indicated. As explained in Section~\ref{sec:guest-addr-translation}, for the paged virtual-memory schemes (Sv32x4, Sv39x4, and Sv48x4), the root page table is 16~KiB and must be aligned to a 16-KiB boundary. In these modes, the lowest two bits of the physical page number (PPN) in {\tt hgatp} always read as zeros. An implementation that supports only the defined paged virtual-memory schemes and/or Bare may hardwire PPN[1:0] to zero. The number of supervisor physical address bits is implementation-defined; any unimplemented address bits are hardwired to zero in {\tt hgatp}.PPN. The number of VMID bits is also implementation-defined and may be zero. The number of implemented VMID bits, termed {\mbox {\em VMIDLEN}}, may be determined by writing one to every bit position in the VMID field, then reading back the value in {\tt hgatp} to see which bit positions in the VMID field hold a one. The least-significant bits of VMID are implemented first: that is, if VMIDLEN~$>$~0, VMID[VMIDLEN-1:0] is writable. The maximal value of VMIDLEN, termed VMIDMAX, is 7 for Sv32x4 or 14 for Sv39x4 and Sv48x4. Note that writing {\tt hgatp} does not imply any ordering constraints between page-table updates and subsequent guest physical address translations. If the new virtual machine's guest physical page tables have been modified, it may be necessary to execute an HFENCE.GVMA instruction (see Section~\ref{sec:hfence.vma}) before or after writing {\tt hgatp}. \subsection{Virtual Supervisor Status Register ({\tt vsstatus})} The {\tt vsstatus} register is a VSXLEN-bit read/write register that is VS-mode's version of supervisor register {\tt sstatus}, formatted as shown in Figure~\ref{vsstatusreg} when VSXLEN=32 and Figure~\ref{vsstatusreg} when VSXLEN=64. When V=1, {\tt vsstatus} substitutes for the usual {\tt sstatus}, so instructions that normally read or modify {\tt sstatus} actually access {\tt vsstatus} instead. \begin{figure*}[h!] {\footnotesize \begin{center} \setlength{\tabcolsep}{4pt} \scalebox{0.95}{ \begin{tabular}{cWcccccWccccWcc} \\ \instbit{31} & \instbitrange{30}{20} & \instbit{19} & \instbit{18} & \instbit{17} & \instbitrange{16}{15} & \instbitrange{14}{13} & \instbitrange{12}{9} & \instbit{8} & \instbit{7} & \instbit{6} & \instbit{5} & \instbitrange{4}{2} & \instbit{1} & \instbit{0} \\ \hline \multicolumn{1}{|c|}{SD} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{MXR} & \multicolumn{1}{c|}{SUM} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{XS[1:0]} & \multicolumn{1}{c|}{FS[1:0]} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{SPP} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{UBE} & \multicolumn{1}{c|}{SPIE} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{SIE} & \multicolumn{1}{c|}{\wpri} \\ \hline 1 & 11 & 1 & 1 & 1 & 2 & 2 & 4 & 1 & 1 & 1 & 1 & 3 & 1 & 1 \\ \end{tabular}} \end{center} } \vspace{-0.1in} \caption{Virtual supervisor status register ({\tt vsstatus}) for RV32.} \label{vsstatusreg-rv32} \end{figure*} \begin{figure*}[h!] {\footnotesize \begin{center} \setlength{\tabcolsep}{4pt} \begin{tabular}{cMFScccc} \\ \instbit{VSXLEN-1} & \instbitrange{VSXLEN-2}{34} & \instbitrange{33}{32} & \instbitrange{31}{20} & \instbit{19} & \instbit{18} & \instbit{17} & \\ \hline \multicolumn{1}{|c|}{SD} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{UXL[1:0]} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{MXR} & \multicolumn{1}{c|}{SUM} & \multicolumn{1}{c|}{\wpri} & \\ \hline 1 & VSXLEN-35 & 2 & 12 & 1 & 1 & 1 & \\ \end{tabular} \begin{tabular}{cWWFccccWcc} \\ & \instbitrange{16}{15} & \instbitrange{14}{13} & \instbitrange{12}{9} & \instbit{8} & \instbit{7} & \instbit{6} & \instbit{5} & \instbitrange{4}{2} & \instbit{1} & \instbit{0} \\ \hline & \multicolumn{1}{|c|}{XS[1:0]} & \multicolumn{1}{c|}{FS[1:0]} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{SPP} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{UBE} & \multicolumn{1}{c|}{SPIE} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{SIE} & \multicolumn{1}{c|}{\wpri} \\ \hline & 2 & 2 & 4 & 1 & 1 & 1 & 1 & 3 & 1 & 1 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Virtual supervisor status register ({\tt vsstatus}) for RV64.} \label{vsstatusreg} \end{figure*} The UXL field controls the effective XLEN for VU-mode, which may differ from the XLEN for VS-mode (VSXLEN). When VSXLEN=32, the UXL field does not exist, and VU-mode XLEN=32. When VSXLEN=64, UXL is a \warl\ field that is encoded the same as the MXL field of {\tt misa}, shown in Table~\ref{misabase} on page~\pageref{misabase}. In particular, the implementation may hardwire field UXL so that VU-mode XLEN=VSXLEN. If VSXLEN is changed from 32 to a wider width, and if field UXL is not hardwired to a forced value, it gets the value corresponding to the widest supported width not wider than the new VSXLEN. When V=1, both {\tt vsstatus}.FS and the HS-level {\tt sstatus}.FS are in effect. Attempts to execute a floating-point instruction when either field is 0 (Off) raise an illegal-instruction exception. Modifying the floating-point state when V=1 causes both fields to be set to 3 (Dirty). \begin{commentary} For a hypervisor to benefit from the extension context status, it must have its own copy in the HS-level {\tt sstatus}, maintained independently of a guest OS running in VS-mode. While a version of the extension context status obviously must exist in {\tt vsstatus} for VS-mode, a hypervisor cannot rely on this version being maintained correctly, given that VS-level software can change {\tt vsstatus}.FS arbitrarily. If the HS-level {\tt sstatus}.FS were not independently active and maintained by the hardware in parallel with {\tt vsstatus}.FS while V=1, hypervisors would always be forced to conservatively swap all floating-point state when context-switching between virtual machines. \end{commentary} Read-only fields SD and XS summarize the extension context status as it is visible to VS-mode only. For example, the value of the HS-level {\tt sstatus}.FS does not affect {\tt vsstatus}.SD. When V=0, {\tt vsstatus} does not directly affect the behavior of the machine, unless the MPRV feature in the {\tt mstatus} register or the SPRV feature in the {\tt hstatus} register is used to execute a load or store {\em as though} V=1. \subsection{Virtual Supervisor Interrupt Registers ({\tt vsip} and {\tt vsie})} The {\tt vsip} register is a VSXLEN-bit read/write register that is VS-mode's version of supervisor register {\tt sip}, formatted as shown in Figure~\ref{vsipreg}. The {\tt vsip} register indicates pending VS-level interrupts. When V=1, {\tt vsip} substitutes for the usual {\tt sip}, so instructions that normally read or modify {\tt sip} actually access {\tt vsip} instead. However, pending HS-level interrupts continue to be indicated in the HS-level {\tt sip} register, not in {\tt vsip}. \note{Need to describe how {\tt vsip}.SEIP interacts with PLIC. Current thinking is that {\tt vsip}.SEIP should purely be a read-write storage bit to emulate the PLIC for VS-mode; the PLIC should not be wired into {\tt vsip}.SEIP. A future revision of the PLIC is expected to provide direct support for VS-level interrupts to reduce virtualization overhead.} \begin{figure*}[h!] {\footnotesize \begin{center} \setlength{\tabcolsep}{4pt} \begin{tabular}{KcFcFcc} \instbitrange{VSXLEN-1}{10} & \instbit{9} & \instbitrange{8}{6} & \instbit{5} & \instbitrange{4}{2} & \instbit{1} & \instbit{0} \\ \hline \multicolumn{1}{|c|}{\wpri} & \multicolumn{1}{c|}{SEIP} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{STIP} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{SSIP} & \multicolumn{1}{c|}{\wpri} \\ \hline VSXLEN-10 & 1 & 3 & 1 & 3 & 1 & 1 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Virtual supervisor interrupt-pending register ({\tt vsip}).} \label{vsipreg} \end{figure*} The {\tt vsie} register is a VSXLEN-bit read/write register that is VS-mode's version of supervisor register {\tt sie}, formatted as shown in Figure~\ref{vsiereg}. The {\tt vsie} register contains interrupt enable bits for VS-level interrupts. When V=1, {\tt vsie} substitutes for the usual {\tt sie}, so instructions that normally read or modify {\tt sie} actually access {\tt vsie} instead. However, the enables for HS-level interrupts continue to be provided by the HS-level {\tt sie} register, not by {\tt vsie}. \begin{figure*}[h!] {\footnotesize \begin{center} \setlength{\tabcolsep}{4pt} \begin{tabular}{KcFcFcc} \instbitrange{VSXLEN-1}{10} & \instbit{9} & \instbitrange{8}{6} & \instbit{5} & \instbitrange{4}{2} & \instbit{1} & \instbit{0} \\ \hline \multicolumn{1}{|c|}{\wpri} & \multicolumn{1}{c|}{SEIE} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{STIE} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{SSIE} & \multicolumn{1}{c|}{\wpri} \\ \hline VSXLEN-10 & 1 & 3 & 1 & 3 & 1 & 1 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Virtual supervisor interrupt-enable register ({\tt vsie}).} \label{vsiereg} \end{figure*} \subsection{Virtual Supervisor Trap Vector Base Address Register ({\tt vstvec})} The {\tt vstvec} register is a VSXLEN-bit read/write register that is VS-mode's version of supervisor register {\tt stvec}, formatted as shown in Figure~\ref{vstvecreg}. When V=1, {\tt vstvec} substitutes for the usual {\tt stvec}, so instructions that normally read or modify {\tt stvec} actually access {\tt vstvec} instead. When V=0, {\tt vstvec} does not directly affect the behavior of the machine. \begin{figure*}[h!] {\footnotesize \begin{center} \begin{tabular}{J@{}R} \instbitrange{VSXLEN-1}{2} & \instbitrange{1}{0} \\ \hline \multicolumn{1}{|c|}{BASE[VSXLEN-1:2] (\warl)} & \multicolumn{1}{c|}{MODE (\warl)} \\ \hline VSXLEN-2 & 2 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Virtual supervisor trap vector base address register ({\tt vstvec}).} \label{vstvecreg} \end{figure*} \subsection{Virtual Supervisor Scratch Register ({\tt vsscratch})} The {\tt vsscratch} register is a VSXLEN-bit read/write register that is VS-mode's version of supervisor register {\tt sscratch}, formatted as shown in Figure~\ref{vsscratchreg}. When V=1, {\tt vsscratch} substitutes for the usual {\tt sscratch}, so instructions that normally read or modify {\tt sscratch} actually access {\tt vsscratch} instead. The contents of {\tt vsscratch} never directly affect the behavior of the machine. \begin{figure*}[h!] {\footnotesize \begin{center} \begin{tabular}{@{}J} \instbitrange{VSXLEN-1}{0} \\ \hline \multicolumn{1}{|c|}{\tt vsscratch} \\ \hline VSXLEN \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Virtual supervisor scratch register ({\tt vsscratch}).} \label{vsscratchreg} \end{figure*} \subsection{Virtual Supervisor Exception Program Counter ({\tt vsepc})} The {\tt vsepc} register is a VSXLEN-bit read/write register that is VS-mode's version of supervisor register {\tt sepc}, formatted as shown in Figure~\ref{vsepcreg}. When V=1, {\tt vsepc} substitutes for the usual {\tt sepc}, so instructions that normally read or modify {\tt sepc} actually access {\tt vsepc} instead. When V=0, {\tt vsepc} does not directly affect the behavior of the machine. {\tt vsepc} is a \warl\ register that must be able to hold the same set of values that {\tt sepc} can hold. \begin{figure*}[h!] {\footnotesize \begin{center} \begin{tabular}{@{}J} \instbitrange{VSXLEN-1}{0} \\ \hline \multicolumn{1}{|c|}{\tt vsepc} \\ \hline VSXLEN \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Virtual supervisor exception program counter ({\tt vsepc}).} \label{vsepcreg} \end{figure*} \subsection{Virtual Supervisor Cause Register ({\tt vscause})} The {\tt vscause} register is a VSXLEN-bit read/write register that is VS-mode's version of supervisor register {\tt scause}, formatted as shown in Figure~\ref{vscausereg}. When V=1, {\tt vscause} substitutes for the usual {\tt scause}, so instructions that normally read or modify {\tt scause} actually access {\tt vscause} instead. When V=0, {\tt vscause} does not directly affect the behavior of the machine. {\tt vscause} is a \wlrl\ register that must be able to hold the same set of values that {\tt scause} can hold. \begin{figure*}[h!] {\footnotesize \begin{center} \begin{tabular}{c@{}U} \instbit{VSXLEN-1} & \instbitrange{VSXLEN-2}{0} \\ \hline \multicolumn{1}{|c|}{Interrupt} & \multicolumn{1}{c|}{Exception Code (\wlrl)} \\ \hline 1 & VSXLEN-1 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Virtual supervisor cause register ({\tt vscause}).} \label{vscausereg} \end{figure*} \subsection{Virtual Supervisor Trap Value Register ({\tt vstval})} The {\tt vstval} register is a VSXLEN-bit read/write register that is VS-mode's version of supervisor register {\tt stval}, formatted as shown in Figure~\ref{vstvalreg}. When V=1, {\tt vstval} substitutes for the usual {\tt stval}, so instructions that normally read or modify {\tt stval} actually access {\tt vstval} instead. When V=0, {\tt vstval} does not directly affect the behavior of the machine. {\tt vstval} is a \warl\ register that must be able to hold the same set of values that {\tt stval} can hold. \begin{figure*}[h!] {\footnotesize \begin{center} \begin{tabular}{@{}J} \instbitrange{VSXLEN-1}{0} \\ \hline \multicolumn{1}{|c|}{\tt vstval} \\ \hline VSXLEN \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Virtual supervisor trap value register ({\tt vstval}).} \label{vstvalreg} \end{figure*} \subsection{Virtual Supervisor Address Translation and Protection Register ({\tt vsatp})} The {\tt vsatp} register is a VSXLEN-bit read/write register that is VS-mode's version of supervisor register {\tt satp}, formatted as shown in Figure~\ref{rv32vsatpreg} for VSXLEN=32 and Figure~\ref{rv64vsatpreg} for VSXLEN=64. When V=1, {\tt vsatp} substitutes for the usual {\tt satp}, so instructions that normally read or modify {\tt satp} actually access {\tt vsatp} instead. {\tt vsatp} controls VS-level address translation, the first stage of two-level translation for guest virtual addresses (see Section~\ref{sec:two-level-translation}). When V=0, a write to {\tt vsatp} with an unsupported MODE value is not ignored as it is for {\tt satp}. Instead, the fields of {\tt vsatp} are {\warl} in the normal way. When V=0, {\tt vsatp} does not directly affect the behavior of the machine, unless the MPRV feature in the {\tt mstatus} register or the SPRV feature in the {\tt hstatus} register is used to execute a load or store {\em as though} V=1. \begin{figure}[h!] {\footnotesize \begin{center} \begin{tabular}{c@{}E@{}K} \instbit{31} & \instbitrange{30}{22} & \instbitrange{21}{0} \\ \hline \multicolumn{1}{|c|}{MODE (\warl)} & \multicolumn{1}{c|}{ASID (\warl)} & \multicolumn{1}{c|}{PPN (\warl)} \\ \hline 1 & 9 & 22 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{RV32 virtual supervisor address translation and protection register {\tt vsatp}.} \label{rv32vsatpreg} \end{figure} \begin{figure*}[h!] {\footnotesize \begin{center} \begin{tabular}{@{}S@{}T@{}U} \instbitrange{63}{60} & \instbitrange{59}{44} & \instbitrange{43}{0} \\ \hline \multicolumn{1}{|c|}{MODE (\warl)} & \multicolumn{1}{c|}{ASID (\warl)} & \multicolumn{1}{c|}{PPN (\warl)} \\ \hline 4 & 16 & 44 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{RV64 virtual supervisor address translation and protection register {\tt vsatp}, for MODE values Bare, Sv39, and Sv48.} \label{rv64vsatpreg} \end{figure*} \section{Hypervisor Instructions} The hypervisor extension adds two new privileged fence instructions. \subsection{Hypervisor Memory-Management Fence Instructions} \label{sec:hfence.vma} \vspace{-0.2in} \begin{center} \begin{tabular}{O@{}R@{}R@{}F@{}R@{}S} \\ \instbitrange{31}{25} & \instbitrange{24}{20} & \instbitrange{19}{15} & \instbitrange{14}{12} & \instbitrange{11}{7} & \instbitrange{6}{0} \\ \hline \multicolumn{1}{|c|}{funct7} & \multicolumn{1}{c|}{rs2} & \multicolumn{1}{c|}{rs1} & \multicolumn{1}{c|}{funct3} & \multicolumn{1}{c|}{rd} & \multicolumn{1}{c|}{opcode} \\ \hline 7 & 5 & 5 & 3 & 5 & 7 \\ HFENCE.GVMA & vmid & gaddr & PRIV & 0 & SYSTEM \\ HFENCE.VVMA & asid & vaddr & PRIV & 0 & SYSTEM \\ \end{tabular} \end{center} The hypervisor memory-management fence instructions, HFENCE.GVMA and HFENCE.VVMA, are valid only in HS-mode when {\tt mstatus}.TVM=0, or in M-mode (irrespective of {\tt mstatus}.TVM). These instructions perform a function similar to SFENCE.VMA (Section~\ref{sec:sfence.vma}), except applying to the guest-physical memory-management data structures controlled by CSR {\tt hgatp} (HFENCE.GVMA) or the VS-level memory-management data structures controlled by CSR {\tt vsatp} (HFENCE.VVMA). Instruction SFENCE.VMA applies only to the memory-management data structures controlled by the current {\tt satp} (either the HS-level {\tt satp} when V=0 or {\tt vsatp} when V=1). If an HFENCE.VVMA instruction executes without trapping, its effect is much the same as temporarily entering VS-mode and executing SFENCE.VMA. Executing an HFENCE.VVMA guarantees that any previous stores already visible to the current hart are ordered before all subsequent implicit reads by that hart of the VS-level memory-management data structures, when those implicit reads are for instructions that \begin{compactitem} \item are subsequent to the HFENCE.VVMA, and \item execute when {\tt hgatp}.VMID has the same setting as it did when HFENCE.VVMA executed. \end{compactitem} Implicit reads need not be ordered when {\tt hgatp}.VMID is different than at the time HFENCE.VVMA executed. If operand {\em rs1}$\neq${\tt x0}, it specifies a single guest virtual address, and if operand {\em rs2}$\neq${\tt x0}, it specifies a single guest address-space identifier (ASID). \begin{commentary} An HFENCE.VVMA instruction applies only to a single virtual machine, identified by the setting of {\tt hgatp}.VMID when HFENCE.VVMA executes. \end{commentary} When {\em rs2}$\neq${\tt x0}, bits XLEN-1:ASIDMAX of the value held in {\em rs2} are reserved for future use and should be zeroed by software and ignored by current implementations. Furthermore, if ASIDLEN~$<$~ASIDMAX, the implementation shall ignore bits ASIDMAX-1:ASIDLEN of the value held in {\em rs2}. \begin{commentary} Simpler implementations of HFENCE.VVMA can ignore the guest virtual address in {\em rs1} and the guest ASID value in {\em rs2}, as well as {\tt hgatp}.VMID, and always perform a global fence for the VS-level memory management of all virtual machines, or even a global fence for all memory-management data structures. \end{commentary} Executing an HFENCE.GVMA instruction guarantees that any previous stores already visible to the current hart are ordered before all subsequent implicit reads by that hart of guest-physical memory-management data structures done for instructions that follow the HFENCE.GVMA. If operand {\em rs1}$\neq${\tt x0}, it specifies a single guest physical address, shifted right by 2~bits, and if operand {\em rs2}$\neq${\tt x0}, it specifies a single virtual machine identifier (VMID). \begin{commentary} For HFENCE.GVMA, a guest physical address specified in {\em rs1} is shifted right by 2~bits to accommodate addresses wider than the current XLEN. For RV32, the hypervisor extension permits guest physical addresses as wide as 34 bits, and {\em rs1} specifies bits 33:2 of such an address. This shift-by-2 encoding of guest physical addresses matches the encoding of physical addresses in PMP address registers (Section~\ref{sec:pmp}) and in page table entries (Sections \ref{sec:sv32}, \ref{sec:sv39}, and~\ref{sec:sv48}). \end{commentary} When {\em rs2}$\neq${\tt x0}, bits XLEN-1:VMIDMAX of the value held in {\em rs2} are reserved for future use and should be zeroed by software and ignored by current implementations. Furthermore, if VMIDLEN~$<$~VMIDMAX, the implementation shall ignore bits VMIDMAX-1:VMIDLEN of the value held in {\em rs2}. \begin{commentary} Simpler implementations of HFENCE.GVMA can ignore the guest physical address in {\em rs1} and the VMID value in {\em rs2} and always perform a global fence for the guest-physical memory management of all virtual machines, or even a global fence for all memory-management data structures. \end{commentary} \section{Machine-Level CSRs} The hypervisor extension augments machine status registers {\tt mstatus} and (for RV32 only) {\tt mstatush}. \subsection{Machine Status Registers ({\tt mstatus} and {\tt mstatush})} The hypervisor extension adds two fields, MPV and MTL, to the machine-level {\tt mstatus} or {\tt mstatush} CSR, and modifies the behavior of several existing {\tt mstatus} fields. Figure~\ref{hypervisor-mstatus} shows the modified {\tt mstatus} register when the hypervisor extension is provided and MXLEN=64. When MXLEN=32, the hypervisor extension adds fields not to {\tt mstatus} but to {\tt mstatush}, which must exist. Figure~\ref{hypervisor-mstatush} shows the {\tt mstatush} register when the hypervisor extension is provided and MXLEN=32. \begin{figure*}[h!] {\footnotesize \begin{center} \setlength{\tabcolsep}{4pt} \begin{tabular}{cMccccFFc} \\ \instbit{MXLEN-1} & \instbitrange{MXLEN-2}{40} & \instbit{39} & \instbit{38} & \instbit{37} & \instbit{36} & \instbitrange{35}{34} & \instbitrange{33}{32} & \\ \hline \multicolumn{1}{|c|}{SD} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{MPV} & \multicolumn{1}{c|}{MTL} & \multicolumn{1}{c|}{MBE} & \multicolumn{1}{c|}{SBE} & \multicolumn{1}{c|}{SXL[1:0]} & \multicolumn{1}{c|}{UXL[1:0]} & \\ \hline 1 & MXLEN-41 & 1 & 1 & 1 & 1 & 2 & 2 & \\ \end{tabular} \begin{tabular}{cEccccccWWc} \\ & \instbitrange{31}{23} & \instbit{22} & \instbit{21} & \instbit{20} & \instbit{19} & \instbit{18} & \instbit{17} & \instbitrange{16}{15} & \instbitrange{14}{13} & \\ \hline & \multicolumn{1}{|c|}{\wpri} & \multicolumn{1}{c|}{TSR} & \multicolumn{1}{c|}{TW} & \multicolumn{1}{c|}{TVM} & \multicolumn{1}{c|}{MXR} & \multicolumn{1}{c|}{SUM} & \multicolumn{1}{c|}{MPRV} & \multicolumn{1}{c|}{XS[1:0]} & \multicolumn{1}{c|}{FS[1:0]} & \\ \hline & 9 & 1 & 1 & 1 & 1 & 1 & 1 & 2 & 2 & \\ \end{tabular} \begin{tabular}{cFWcccccccccc} \\ & \instbitrange{12}{11} & \instbitrange{10}{9} & \instbit{8} & \instbit{7} & \instbit{6} & \instbit{5} & \instbit{4} & \instbit{3} & \instbit{2} & \instbit{1} & \instbit{0} \\ \hline & \multicolumn{1}{|c|}{MPP[1:0]} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{SPP} & \multicolumn{1}{c|}{MPIE} & \multicolumn{1}{c|}{UBE} & \multicolumn{1}{c|}{SPIE} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{MIE} & \multicolumn{1}{c|}{\wpri} & \multicolumn{1}{c|}{SIE} & \multicolumn{1}{c|}{\wpri} \\ \hline & 2 & 2 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Machine-mode status register ({\tt mstatus}) for RV64 when the hypervisor extension is provided.} \label{hypervisor-mstatus} \end{figure*} \begin{figure*}[h!] {\footnotesize \begin{center} \setlength{\tabcolsep}{4pt} \begin{tabular}{LccccF} \\ \instbitrange{31}{8} & \instbit{7} & \instbit{6} & \instbit{5} & \instbit{4} & \instbitrange{3}{0} \\ \hline \multicolumn{1}{|c|}{\wpri} & \multicolumn{1}{c|}{MPV} & \multicolumn{1}{c|}{MTL} & \multicolumn{1}{c|}{MBE} & \multicolumn{1}{c|}{SBE} & \multicolumn{1}{c|}{\wpri} \\ \hline 24 & 1 & 1 & 1 & 1 & 4 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Additional machine-mode status register ({\tt mstatush}) for RV32 when the hypervisor extension is provided. The format of {\tt mstatus} is unchanged for RV32.} \label{hypervisor-mstatush} \end{figure*} The MPV bit (Machine Previous Virtualization Mode) is written by the implementation whenever a trap is taken into M-mode. Just as the MPP bit is set to the privilege mode at the time of the trap, the MPV bit is set to the value of the virtualization mode V at the time of the trap. When an MRET instruction is executed, the virtualization mode V is set to MPV, unless MPP=3, in which case V remains 0. The MTL bit (Machine Translation Level), which indicates which address-translation level caused an access-fault or page-fault exception, is also written by the implementation whenever a trap is taken into M-mode. On an access or page fault due to guest physical address translation, MTL is set to 1. For any other trap into M-mode, MTL is set to 0. The TSR and TVM fields of {\tt mstatus} affect execution only in HS-mode, not in VS-mode. The TW field affects execution in all modes except M-mode. The hypervisor extension changes the behavior of the the Modify Privilege field, MPRV, of {\tt mstatus}. When MPRV=0, translation and protection behave as normal. When MPRV=1, loads and stores are translated and protected as though the current privilege mode were set to MPP and the current virtualization mode were set to MPV. Table~\ref{h-mprv} enumerates the cases. \begin{table*}[h!] \begin{center} \begin{tabular}{|c|c|c||p{4.5in}|} \hline MPRV & MPV & MPP & Effect \\ \hline \hline 0 & -- & -- & Normal access; current privilege and virtualization modes apply. \\ \hline 1 & 0 & 0 & U-level access with HS-level translation and protection only. \\ \hline 1 & 0 & 1 & HS-level access with HS-level translation and protection only. \\ \hline 1 & -- & 3 & M-level access with no translation. \\ \hline 1 & 1 & 0 & VU-level access with two-level translation and protection. The HS-level MXR bit makes any executable page readable. {\tt vsstatus}.MXR makes readable those pages marked executable at the VS translation level, but only if readable at the guest-physical translation level. \\ \hline 1 & 1 & 1 & VS-level access with two-level translation and protection. The HS-level MXR bit makes any executable page readable. {\tt vsstatus}.MXR makes readable those pages marked executable at the VS translation level, but only if readable at the guest-physical translation level. {\tt vsstatus}.SUM applies instead of the HS-level SUM bit. \\ \hline \end{tabular} \end{center} \caption{Effect of MPRV on load and store translation and protection. When MPRV=1, MPP$\neq$3, and {\tt hstatus}.SPRV=1, the effective privilege is further modified: {\tt hstatus}.SPV applies instead of MPV, and the HS-level SPP applies instead of MPP.} \label{h-mprv} \end{table*} The {\tt mstatus} register is a superset of the HS-level {\tt sstatus} register but is not a superset of {\tt vsstatus}. \section{Two-Level Address Translation} \label{sec:two-level-translation} Whenever the current virtualization mode V is 1 (and assuming {\tt mstatus}.MPRV=0), two-level address translation and protection is in effect. For any virtual memory access, the original virtual address is first converted by VS-level address translation, as controlled by the {\tt vsatp} register, into a {\em guest physical address}. The guest physical address is then converted by guest physical address translation, as controlled by the {\tt hgatp} register, into a supervisor physical address. Although there is no option to disable two-level address translation when V=1, either level of translation can be effectively disabled by zeroing the corresponding {\tt vsatp} or {\tt hgatp} register. The {\tt vsstatus} field MXR, which makes execute-only pages readable, only overrides VS-level page protection. Setting MXR at VS-level does not override guest-physical page protections. Setting MXR at HS-level, however, overrides both VS-level and guest-physical execute-only permissions. When V=1, memory accesses that would normally bypass address translation are subject to guest physical address translation alone. This includes memory accesses made in support of VS-level address translation, such as reads and writes of VS-level page tables. Machine-level physical memory protection applies to supervisor physical addresses and is in effect regardless of virtualization mode. \subsection{Guest Physical Address Translation} \label{sec:guest-addr-translation} The mapping of guest physical addresses to supervisor physical addresses is controlled by CSR {\tt hgatp} (Section~\ref{sec:hgatp}). When the address translation scheme selected by the MODE field of {\tt hgatp} is Bare, guest physical addresses are equal to supervisor physical addresses without modification, and no memory protection applies in the trivial translation of guest physical addresses to supervisor physical addresses. When {\tt hgatp}.MODE specifies a translation scheme of Sv32x4, Sv39x4, or Sv48x4, guest physical address translation is a variation on the usual page-based virtual address translation scheme of Sv32, Sv39, or Sv48, respectively. In each case, the size of the incoming address is widened by 2~bits (to 34, 41, or 50 bits). To accommodate the 2~extra bits, the root page table (only) is expanded by a factor of four to be 16~KiB instead of the usual 4~KiB. Matching its larger size, the root page table also must be aligned to a 16~KiB boundary instead of the usual 4~KiB page boundary. Except as noted, all other aspects of Sv32, Sv39, or Sv48 are adopted unchanged for guest physical address translation. Non-root page tables and all page table entries (PTEs) have the same formats as documented in Sections \ref{sec:sv32}, \ref{sec:sv39}, and~\ref{sec:sv48}. For Sv32x4, an incoming guest physical address is partitioned into a virtual page number (VPN) and page offset as shown in Figure~\ref{sv32x4va}. This partitioning is identical to that for an Sv32 virtual address as depicted in Figure~\ref{sv32va} (page~\pageref{sv32va}), except with 2 more bits at the high end in VPN[1]. (Note that the fields of a partitioned guest physical address also correspond one-for-one with the structure that Sv32 assigns to a physical address, depicted in Figure~\ref{rv32va}.) \begin{figure*}[h!] {\footnotesize \begin{center} \begin{tabular}{@{}E@{}O@{}E} \instbitrange{33}{22} & \instbitrange{21}{12} & \instbitrange{11}{0} \\ \hline \multicolumn{1}{|c|}{VPN[1]} & \multicolumn{1}{c|}{VPN[0]} & \multicolumn{1}{c|}{page offset} \\ \hline 12 & 10 & 12 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Sv32x4 virtual address (guest physical address).} \label{sv32x4va} \end{figure*} For Sv39x4, an incoming guest physical address is partitioned as shown in Figure~\ref{sv39x4va}. This partitioning is identical to that for an Sv39 virtual address as depicted in Figure~\ref{sv39va} (page~\pageref{sv39va}), except with 2 more bits at the high end in VPN[2]. Address bits 63:41 must all be zeros, or else a page-fault exception occurs, attributed to guest physical address translation. \begin{figure*}[h!] {\footnotesize \begin{center} \begin{tabular}{@{}E@{}O@{}O@{}O} \instbitrange{40}{30} & \instbitrange{29}{21} & \instbitrange{20}{12} & \instbitrange{11}{0} \\ \hline \multicolumn{1}{|c|}{VPN[2]} & \multicolumn{1}{c|}{VPN[1]} & \multicolumn{1}{c|}{VPN[0]} & \multicolumn{1}{c|}{page offset} \\ \hline 11 & 9 & 9 & 12 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Sv39x4 virtual address (guest physical address).} \label{sv39x4va} \end{figure*} For Sv48x4, an incoming guest physical address is partitioned as shown in Figure~\ref{sv48x4va}. This partitioning is identical to that for an Sv48 virtual address as depicted in Figure~\ref{sv48va} (page~\pageref{sv48va}), except with 2 more bits at the high end in VPN[3]. Address bits 63:50 must all be zeros, or else a page-fault exception occurs, attributed to guest physical address translation. \begin{figure*}[h!] {\footnotesize \begin{center} \begin{tabular}{@{}E@{}O@{}O@{}O@{}O} \instbitrange{49}{39} & \instbitrange{38}{30} & \instbitrange{29}{21} & \instbitrange{20}{12} & \instbitrange{11}{0} \\ \hline \multicolumn{1}{|c|}{VPN[3]} & \multicolumn{1}{c|}{VPN[2]} & \multicolumn{1}{c|}{VPN[1]} & \multicolumn{1}{c|}{VPN[0]} & \multicolumn{1}{c|}{page offset} \\ \hline 11 & 9 & 9 & 9 & 12 \\ \end{tabular} \end{center} } \vspace{-0.1in} \caption{Sv48x4 virtual address (guest physical address).} \label{sv48x4va} \end{figure*} \begin{commentary} The page-based guest physical address translation scheme for RV32, Sv32x4, is defined to support a 34-bit guest physical address so that an RV32 hypervisor need not be limited in its ability to virtualize real 32-bit RISC-V machines, even those with 33-bit or 34-bit physical addresses. This may include the possibility of a machine virtualizing itself, if it happens to use 33-bit or 34-bit physical addresses. Multiplying the size and alignment of the root page table by a factor of four is the cheapest way to extend Sv32 to cover a 34-bit address. The possible wastage of 12~KiB for an unnecessarily large root page table is expected to be of negligible consequence for most (maybe all) real uses. A consistent ability to virtualize machines having as much as four times the physical address space as virtual address space is believed to be of some utility also for RV64. For a machine supporting 39-bit virtual addresses (Sv39), for example, this allows the hypervisor extension to support up to a 41-bit guest physical address space without either necessitating hardware support for 48-bit virtual addresses (Sv48) or falling back to emulating the larger address space with shadow page tables. \end{commentary} The conversion of an Sv32x4, Sv39x4, or Sv48x4 guest physical address is accomplished with the same algorithm used for Sv32, Sv39, or Sv48, as presented in Section~\ref{sv32algorithm}, except that: \begin{compactitem} \item in step~1, $a = \mbox{{\tt hgatp}.PPN}\times\mbox{PAGESIZE}$; and \item the current privilege mode is always taken to be U-mode. \end{compactitem} For guest physical address translation, all memory accesses (including those made to access data structures for VS-level address translation) are considered to be user-level accesses, as though executed in U-mode. Access type permissions---readable, writable, or executable---are checked during guest physical address translation the same as for VS-level address translation. For a memory access made to support VS-level address translation (such as to read/write a VS-level page table), permissions are checked as though for a load or store, not for the original access type. However, any exception is always reported for the original access type (instruction, load, or store/AMO). Access faults and page faults raised by guest physical address translation are treated as HS-level exceptions for the purpose of exception delegation, so are not delegated to VS-mode, regardless of the setting of the {\tt hedeleg} register. \subsection{Memory-Management Fences} The behavior of the SFENCE.VMA instruction is affected by the current virtualization mode V. When V=0, the virtual-address argument is an HS-level virtual address, and the ASID argument is an HS-level ASID. The instruction orders stores only to HS-level address-translation structures with subsequent HS-level address translations. When V=1, the virtual-address argument to SFENCE.VMA is a guest virtual address within the current virtual machine, and the ASID argument is a VS-level ASID within the current virtual machine. The current virtual machine is identified by the VMID field of CSR {\tt hgatp}, and the effective ASID can be considered to be the combination of this VMID with the VS-level ASID. The SFENCE.VMA instruction orders stores only to the VS-level address-translation structures with subsequent VS-level address translations for the same virtual machine, i.e., only when {\tt hgatp}.VMID is the same as when the SFENCE.VMA executed. Hypervisor instructions HFENCE.GVMA and HFENCE.VVMA provide additional memory-management fences to complement SFENCE.VMA. These instructions are described in Section~\ref{sec:hfence.vma}. Section~\ref{pmp-vmem} discusses the intersection between physical memory protection (PMP) and page-based address translation. It is noted there that, when PMP settings are modified in a manner that affects either the physical memory that holds page tables or the physical memory to which page tables point, M-mode software must synchronize the PMP settings with the virtual memory system. For HS-level address translation, this is accomplished by executing in M-mode an SFENCE.VMA instruction with {\em rs1}={\tt x0} and {\em rs2}={\tt x0}, after the PMP CSRs are written. If guest physical address translation is in use, synchronization with its data structures is also needed. When PMP settings are modified in a manner that affects either the physical memory that holds guest-physical page tables or the physical memory to which guest-physical page tables point, an HFENCE.GVMA instruction with {\em rs1}={\tt x0} and {\em rs2}={\tt x0} must be executed in M-mode after the PMP CSRs are written. An HFENCE.VVMA instruction is not required. \section{WFI in Virtual Operating Modes} Executing instruction WFI in VS-mode or VU-mode causes an illegal instruction exception, unless it completes within an implementation-specific, bounded time limit. \begin{commentary} The behavior required of WFI in VS-mode and VU-mode is the same as required of it in U-mode when S-mode exists. \end{commentary} \section{Traps} The hypervisor extension augments the environment-call exception cause encoding. Environment calls from HS-mode use cause 9, whereas environment calls from VS-mode use cause 10. Table~\ref{hcauses} lists the possible M-mode and HS-mode exception codes when the hypervisor extension is present. \begin{table*}[p] \begin{center} \begin{tabular}{|r|r|l|l|} \hline Interrupt & Exception Code & Description \\ \hline 1 & 0 & {\em Reserved} \\ 1 & 1 & Supervisor software interrupt \\ 1 & 2 & {\em Reserved} \\ 1 & 3 & Machine software interrupt \\ \hline 1 & 4 & {\em Reserved} \\ 1 & 5 & Supervisor timer interrupt \\ 1 & 6 & {\em Reserved} \\ 1 & 7 & Machine timer interrupt \\ \hline 1 & 8 & {\em Reserved} \\ 1 & 9 & Supervisor external interrupt \\ 1 & 10 & {\em Reserved} \\ 1 & 11 & Machine external interrupt \\ \hline 1 & 12--15 & {\em Reserved} \\ 1 & $\ge$16 & {\em Available for platform use} \\ \hline 0 & 0 & Instruction address misaligned \\ 0 & 1 & Instruction access fault \\ 0 & 2 & Illegal instruction \\ 0 & 3 & Breakpoint \\ 0 & 4 & Load address misaligned \\ 0 & 5 & Load access fault \\ 0 & 6 & Store/AMO address misaligned \\ 0 & 7 & Store/AMO access fault \\ 0 & 8 & Environment call from U-mode or VU-mode \\ 0 & 9 & Environment call from HS-mode \\ 0 & 10 & Environment call from VS-mode \\ 0 & 11 & Environment call from M-mode \\ 0 & 12 & Instruction page fault \\ 0 & 13 & Load page fault \\ 0 & 14 & {\em Reserved} \\ 0 & 15 & Store/AMO page fault \\ 0 & 16--23 & {\em Reserved} \\ 0 & 24--31 & {\em Available for custom use} \\ 0 & 32--47 & {\em Reserved} \\ 0 & 48--63 & {\em Available for custom use} \\ 0 & $\ge$64 & {\em Reserved} \\ \hline \end{tabular} \end{center} \caption{Supervisor and machine cause register ({\tt scause} and {\tt mcause}) values when the hypervisor extension is enabled.} \label{hcauses} \end{table*} \begin{commentary} HS-mode and VS-mode ECALLs use different cause values so they can be delegated separately. \end{commentary} When a trap occurs in HS-mode or U-mode, it goes to M-mode, unless delegated by {\tt medeleg} or {\tt mideleg}, in which case it goes to HS-mode. When a trap occurs in VS-mode or VU-mode, it goes to M-mode, unless delegated by {\tt medeleg} or {\tt mideleg}, in which case it goes to HS-mode, unless further delegated by {\tt hedeleg} or {\tt hideleg}, in which case it goes to VS-mode. When a trap is taken into M-mode, virtualization mode V gets set to~0, and {\tt mstatus}.MPV and {\tt mstatus}.MPP are set according to Table~\ref{h-mpp}. \begin{table*}[h!] \begin{center} \begin{tabular}{|l|c|c|} \hline Previous Mode & MPV & MPP \\ \hline U-mode & 0 & 0 \\ HS-mode & 0 & 1 \\ M-mode & 0 & 3 \\ \hline VU-mode & 1 & 0 \\ VS-mode & 1 & 1 \\ \hline \end{tabular} \end{center} \caption{Value of {\tt mstatus} fields MPV and MPP after a trap into M-mode. Upon trap return, MPV is ignored when MPP=3.} \label{h-mpp} \end{table*} When a trap is taken into HS-mode, virtualization mode V is first set to~0, then {\tt hstatus}.SP2V is set to {\tt hstatus}.SPV, {\tt hstatus}.SP2P is set to {\tt sstatus}.SPP, and lastly {\tt hstatus}.SPV and {\tt sstatus}.SPP are set according to Table~\ref{h-spp}. \begin{table*}[h!] \begin{center} \begin{tabular}{|l|c|c|} \hline Previous Mode & SPV & SPP \\ \hline U-mode & 0 & 0 \\ HS-mode & 0 & 1 \\ \hline VU-mode & 1 & 0 \\ VS-mode & 1 & 1 \\ \hline \end{tabular} \end{center} \caption{Value of {\tt hstatus} field SPV and {\tt sstatus} field SPP after a trap into HS-mode.} \label{h-spp} \end{table*} When a trap is taken into VS-mode, {\tt vsstatus}.SPP is set according to Table~\ref{h-vspp}. Register {\tt hstatus} and the HS-level {\tt sstatus} are not modified, and the virtualization mode V remains~1. \begin{table*}[h!] \begin{center} \begin{tabular}{|l|c|c|} \hline Previous Mode & SPP \\ \hline VU-mode & 0 \\ VS-mode & 1 \\ \hline \end{tabular} \end{center} \caption{Value of {\tt vsstatus} field SPP after a trap into VS-mode.} \label{h-vspp} \end{table*} \FloatBarrier \section{Trap Return} The MRET instruction is used to return from a trap taken into M-mode. MRET first determines what the new operating mode will be according to the values of MPP and MPV in {\tt mstatus}, as encoded in Table~\ref{h-mpp}. MRET then in {\tt mstatus} sets MPV=0, MPP=0, MIE=MPIE, and MPIE=1. If the new operating mode will be U, VS, or VU, MRET also sets {\tt hstatus}.SPRV=0. Lastly, MRET sets the virtualization and privilege modes as previously determined, and sets {\tt pc}={\tt mepc}. The SRET instruction is used to return from a trap taken into HS-mode or VS-mode. Its behavior depends on the current virtualization mode. When executed in M-mode or HS-mode (i.e., V=0), SRET first determines what the new operating mode will be according to the values in {\tt hstatus}.SPV and {\tt sstatus}.SPP, as encoded in Table~\ref{h-spp}. SRET then sets {\tt hstatus}.SPV={\tt hstatus}.SP2V, {\tt sstatus}.SPP={\tt hstatus}.SP2P, {\tt hstatus}.SP2V=0, {\tt hstatus}.SP2P=0, {\tt sstatus}.SIE={\tt sstatus}.SPIE, and {\tt sstatus}.SPIE=1. If the new operating mode will be U, VS, or VU, SRET also sets {\tt hstatus}.SPRV=0. Lastly, SRET sets the virtualization and privilege modes as previously determined, and sets {\tt pc}={\tt sepc}. When executed in VS-mode (i.e., V=1), SRET sets the privilege mode according to Table~\ref{h-vspp}, then in {\tt vsstatus} sets SPP=0, SIE=SPIE, and SPIE=1, and lastly sets {\tt pc}={\tt vsepc}.