aboutsummaryrefslogtreecommitdiff
path: root/tcg
AgeCommit message (Collapse)AuthorFilesLines
2021-06-19tcg/tci: Implement andc, orc, eqv, nand, norRichard Henderson2-10/+50
These were already present in tcg-target.c.inc, but not in the interpreter. Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-19tcg/tci: Implement movcondRichard Henderson3-6/+24
When this opcode is not available in the backend, tcg middle-end will expand this as a series of 5 opcodes. So implementing this saves bytecode space. Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-19tcg/tci: Implement goto_ptrRichard Henderson5-2/+44
This operation is critical to staying within the interpretation loop longer, which avoids the overhead of setup and teardown for many TBs. The check in tcg_prologue_init is disabled because TCI does want to use NULL to indicate exit, as opposed to branching to a real epilogue. Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-19tcg/tci: Change encoding to uint32_t unitsRichard Henderson4-559/+380
This removes all of the problems with unaligned accesses to the bytecode stream. With an 8-bit opcode at the bottom, we have 24 bits remaining, which are generally split into 6 4-bit slots. This fits well with the maximum length opcodes, e.g. INDEX_op_add2_i32, which have 6 register operands. We have, in previous patches, rearranged things such that there are no operations with a label which have more than one other operand. Which leaves us with a 20-bit field in which to encode a label, giving us a maximum TB size of 512k -- easily large. Change the INDEX_op_tci_movi_{i32,i64} opcodes to tci_mov[il]. The former puts the immediate in the upper 20 bits of the insn, like we do for the label displacement. The later uses a label to reference an entry in the constant pool. Thus, in the worst case we still have a single memory reference for any constant, but now the constants are out-of-line of the bytecode and can be shared between different moves saving space. Change INDEX_op_call to use a label to reference a pair of pointers in the constant pool. This removes the only slightly dodgy link with the layout of struct TCGHelperInfo. The re-encode cannot be done in pieces. Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-19tcg/tci: Remove tci_write_regRichard Henderson1-11/+2
Inline it into its one caller, tci_write_reg64. Drop the asserts that are redundant with tcg_read_r. Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-19tcg/tci: Emit setcond before brcondRichard Henderson2-85/+35
The encoding planned for tci does not have enough room for brcond2, with 4 registers and a condition as input as well as the label. Resolve the condition into TCG_REG_TMP, and relax brcond to one register plus a label, considering the condition to always be reg != 0. Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-19tcg/tci: Reserve r13 for a temporaryRichard Henderson2-0/+2
We're about to adjust the offset range on host memory ops, and the format of branches. Both will require a temporary. Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-19tcg/tci: Use ffi for callsRichard Henderson4-104/+150
This requires adjusting where arguments are stored. Place them on the stack at left-aligned positions. Adjust the stack frame to be at entirely positive offsets. Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-19tcg/tci: Move call-return regs to end of tcg_target_reg_alloc_orderRichard Henderson1-2/+2
As the only call-clobbered regs for TCI, these should receive the least priority. Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-19tcg/tci: Improve tcg_target_call_clobber_regsRichard Henderson1-2/+8
The current setting is much too pessimistic. Indicating only the one or two registers that are actually assigned after a call should avoid unnecessary movement between the register array and the stack array. Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-19tcg: Build ffi data structures for helpersRichard Henderson2-1/+65
Add libffi as a build requirement for TCI. Add libffi to the dockerfiles to satisfy that requirement. Construct an ffi_cif structure for each unique typemask. Record the result in a separate hash table for later lookup; this allows helper_table to stay const. Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-19tcg: Add tcg_call_funcRichard Henderson2-3/+7
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-19tcg: Store the TCGHelperInfo in the TCGOp for callRichard Henderson2-29/+34
This will give us both flags and typemask for use later. We also fix a dumping bug, wherein calls generated for plugins fail tcg_find_helper and print (null) instead of either a name or the raw function pointer. Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-19tcg: Add tcg_call_flagsRichard Henderson3-9/+13
We're going to change how to look up the call flags from a TCGop, so extract it as a helper. Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-19tcg: Combine dh_is_64bit and dh_is_signed to dh_typecodeRichard Henderson1-30/+41
We will shortly be interested in distinguishing pointers from integers in the helper's declaration, as well as a true void return. We currently have two parallel 1 bit fields; merge them and expand to a 3 bit field. Our current maximum is 7 helper arguments, plus the return makes 8 * 3 = 24 bits used within the uint32_t typemask. Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-13tcg/arm: Fix tcg_out_op function signatureJose R. Ziviani1-1/+2
Commit 5e8892db93 fixed several function signatures but tcg_out_op for arm is missing. This patch fixes it as well. Signed-off-by: Jose R. Ziviani <jziviani@suse.de> Message-Id: <20210610224450.23425-1-jziviani@suse.de> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-13tcg: Introduce tcg_remove_ops_afterRichard Henderson1-0/+13
Introduce a function to remove everything emitted since a given point. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-13tcg: Move tcg_init_ctx and tcg_ctx from accel/tcg/Richard Henderson2-0/+4
These variables belong to the jit side, not the user side. Since tcg_init_ctx is no longer used outside of tcg/, move the declaration to tcg-internal.h. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Suggested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-13tcg: When allocating for !splitwx, begin with PROT_NONERichard Henderson1-10/+9
There's a change in mprotect() behaviour [1] in the latest macOS on M1 and it's not yet clear if it's going to be fixed by Apple. In this case, instead of changing permissions of N guard pages, we change permissions of N rwx regions. The same number of syscalls are required either way. [1] https://gist.github.com/hikalium/75ae822466ee4da13cbbe486498a191f Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-13tcg: Merge buffer protection and guard page protectionRichard Henderson1-14/+31
Do not handle protections on a case-by-case basis in the various alloc_code_gen_buffer instances; do it within a single loop in tcg_region_init. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-13tcg: Round the tb_size default from qemu_get_host_physmemRichard Henderson1-26/+21
If qemu_get_host_physmem returns an odd number of pages, then physmem / 8 will not be a multiple of the page size. The following was observed on a gitlab runner: ERROR qtest-arm/boot-serial-test - Bail out! ERROR:../util/osdep.c:80:qemu_mprotect__osdep: \ assertion failed: (!(size & ~qemu_real_host_page_mask)) Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-13tcg: Sink qemu_madvise call to common codeRichard Henderson1-7/+7
Move the call out of the N versions of alloc_code_gen_buffer and into tcg_region_init. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-13tcg: Return the map protection from alloc_code_gen_bufferRichard Henderson1-30/+33
Change the interface from a boolean error indication to a negative error vs a non-negative protection. For the moment this is only interface change, not making use of the new data. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-13tcg: Allocate code_gen_buffer into struct tcg_region_stateRichard Henderson1-37/+27
Do not mess around with setting values within tcg_init_ctx. Put the values into 'region' directly, which is where they will live for the lifetime of the program. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11tcg: Move in_code_gen_buffer and tests to region.cRichard Henderson2-23/+34
Shortly, the full code_gen_buffer will only be visible to region.c, so move in_code_gen_buffer out-of-line. Move the debugging versions of tcg_splitwx_to_{rx,rw} to region.c as well, so that the compiler gets to see the implementation of in_code_gen_buffer. This leaves exactly one use of in_code_gen_buffer outside of region.c, in cpu_restore_state. Which, being on the exception path, is not performance critical. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11tcg: Tidy split_cross_256mbRichard Henderson1-10/+9
Return output buffer and size via output pointer arguments, rather than returning size via tcg_ctx->code_gen_buffer_size. Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11tcg: Tidy tcg_n_regionsRichard Henderson1-17/+12
Compute the value using straight division and bounds, rather than a loop. Pass in tb_size rather than reading from tcg_init_ctx.code_gen_buffer_size, Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11tcg: Rename region.start to region.after_prologueRichard Henderson1-7/+8
Give the field a name reflecting its actual meaning. Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11tcg: Replace region.end with region.total_sizeRichard Henderson1-12/+18
A size is easier to work with than an end point, particularly during initial buffer allocation. Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11tcg: Move MAX_CODE_GEN_BUFFER_SIZE to tcg-target.hRichard Henderson10-28/+23
Remove the ifdef ladder and move each define into the appropriate header file. Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11tcg: Introduce tcg_max_ctxsRichard Henderson3-17/+15
Finish the divorce of tcg/ from hw/, and do not take the max cpu value from MachineState; just remember what we were passed in tcg_init. Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11accel/tcg: Pass down max_cpus to tcg_initRichard Henderson3-28/+16
Start removing the include of hw/boards.h from tcg/. Pass down the max_cpus value from tcg_init_machine, where we have the MachineState already. Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11tcg: Create tcg_initRichard Henderson2-1/+9
Perform both tcg_context_init and tcg_region_init. Do not leave this split to the caller. Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11accel/tcg: Move alloc_code_gen_buffer to tcg/region.cRichard Henderson1-5/+426
Buffer management is integral to tcg. Do not leave the allocation to code outside of tcg/. This is code movement, with further cleanups to follow. Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11tcg: Split out region.cRichard Henderson4-544/+613
Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11tcg: Split out tcg_region_prologue_setRichard Henderson1-15/+22
This has only one user, but will make more sense after some code motion. Always leave the tcg_init_ctx initialized to the first region, in preparation for tcg_prologue_init(). This also requires that we don't re-allocate the region for the first cpu, lest we hit the assertion for total number of regions allocated . Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11tcg: Split out tcg_region_initial_allocRichard Henderson1-3/+10
This has only one user, and currently needs an ifdef, but will make more sense after some code motion. Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11tcg: Remove error return from tcg_region_initial_alloc__lockedRichard Henderson1-13/+6
All callers immediately assert on error, so move the assert into the function itself. Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11tcg: Re-order tcg_region_init vs tcg_prologue_initRichard Henderson1-33/+19
Instead of delaying tcg_region_init until after tcg_prologue_init is complete, do tcg_region_init first and let tcg_prologue_init shrink the first region by the size of the generated prologue. Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-11meson: Split out tcg/meson.buildRichard Henderson1-0/+13
Reviewed-by: Luis Pires <luis.pires@eldorado.org.br> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-04tcg/arm: Implement TCG_TARGET_HAS_rotv_vecRichard Henderson1-1/+34
Implement via expansion, so don't actually set TCG_TARGET_HAS_rotv_vec. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-04tcg/arm: Implement TCG_TARGET_HAS_roti_vecRichard Henderson3-0/+17
Implement via expansion, so don't actually set TCG_TARGET_HAS_roti_vec. For NEON, this is shift-right followed by shift-left-and-insert. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-04tcg/arm: Implement TCG_TARGET_HAS_shv_vecRichard Henderson2-1/+63
The three vector shift by vector operations are all implemented via expansion. Therefore do not actually set TCG_TARGET_HAS_shv_vec, as none of shlv_vec, shrv_vec, sarv_vec may actually appear in the instruction stream, and therefore also do not appear in tcg_target_op_def. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-04tcg/arm: Implement TCG_TARGET_HAS_bitsel_vecRichard Henderson3-3/+22
NEON has 3 instructions implementing this 4 argument operation, with each insn overlapping a different logical input onto the destination register. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-04tcg/arm: Implement TCG_TARGET_HAS_minmax_vecRichard Henderson2-1/+25
This is minimum and maximum, signed and unsigned. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-04tcg/arm: Implement TCG_TARGET_HAS_sat_vecRichard Henderson2-1/+25
This is saturating add and subtract, signed and unsigned. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-04tcg/arm: Implement TCG_TARGET_HAS_mul_vecRichard Henderson2-1/+7
Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-04tcg/arm: Implement TCG_TARGET_HAS_shi_vecRichard Henderson2-1/+28
This consists of the three immediate shifts: shli, shri, sari. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-04tcg/arm: Implement andc, orc, abs, neg, not vector operationsRichard Henderson3-5/+44
These logical and arithmetic operations are optional, but are trivial to accomplish with the existing infrastructure. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-04tcg/arm: Implement minimal vector operationsRichard Henderson4-6/+202
Implementing dup2, add, sub, and, or, xor as the minimal set. This allows us to actually enable neon in the header file. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
generated by cgit v1.1 at 2024-08-23 06:28:36 +0000