| 7364d60c | 28-Feb-2023 |
Jiaxun Yang <jiaxun.yang@flygoat.com> |
bpf, mips: Implement R4000 workarounds for JIT
For R4000 erratas around multiplication and division instructions,
as our use of those instructions are always followed by mflo/mfhi
instructions, the
bpf, mips: Implement R4000 workarounds for JIT
For R4000 erratas around multiplication and division instructions,
as our use of those instructions are always followed by mflo/mfhi
instructions, the only issue we need care is
"MIPS R4000PC/SC Errata, Processor Revision 2.2 and 3.0" Errata 28:
"A double-word or a variable shift may give an incorrect result if
executed while an integer multiplication is in progress."
We just emit a mfhi $0 to ensure the operation is completed after
every multiplication instruction according to workaround suggestion
in the document.
Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Johan Almbladh <johan.almbladh@anyfinetworks.com>
Link: https://lore.kernel.org/bpf/20230228113305.83751-3-jiaxun.yang@flygoat.com
show more ...
|
| 431bfb9e | 11-Oct-2021 |
Tiezhu Yang <yangtiezhu@loongson.cn> |
bpf, mips: Fix comment on tail call count limiting
In emit_tail_call() of bpf_jit_comp32.c, "blez t2" (t2 <= 0) is
not consistent with the comment "t2 < 0", update the comment to
keep consistency.
bpf, mips: Fix comment on tail call count limiting
In emit_tail_call() of bpf_jit_comp32.c, "blez t2" (t2 <= 0) is
not consistent with the comment "t2 < 0", update the comment to
keep consistency.
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Johan Almbladh <johan.almbladh@anyfinetworks.com>
Link: https://lore.kernel.org/bpf/1633915150-13220-3-git-send-email-yangtiezhu@loongson.cn
show more ...
|
| bbf731b3 | 07-Oct-2021 |
Johan Almbladh <johan.almbladh@anyfinetworks.com> |
mips, bpf: Optimize loading of 64-bit constants
This patch shaves off a few instructions when loading sparse 64-bit
constants to register. The change is covered by additional tests in
lib/test_bpf.c
mips, bpf: Optimize loading of 64-bit constants
This patch shaves off a few instructions when loading sparse 64-bit
constants to register. The change is covered by additional tests in
lib/test_bpf.c.
Signed-off-by: Johan Almbladh <johan.almbladh@anyfinetworks.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20211007142828.634182-1-johan.almbladh@anyfinetworks.com
show more ...
|
| ebcbacfa | 05-Oct-2021 |
Johan Almbladh <johan.almbladh@anyfinetworks.com> |
mips, bpf: Remove old BPF JIT implementations
This patch removes the old 32-bit cBPF and 64-bit eBPF JIT implementations.
They are replaced by a new eBPF implementation that supports both 32-bit
and
mips, bpf: Remove old BPF JIT implementations
This patch removes the old 32-bit cBPF and 64-bit eBPF JIT implementations.
They are replaced by a new eBPF implementation that supports both 32-bit
and 64-bit MIPS CPUs.
Signed-off-by: Johan Almbladh <johan.almbladh@anyfinetworks.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20211005165408.2305108-8-johan.almbladh@anyfinetworks.com
show more ...
|
| 01bdc58e | 05-Oct-2021 |
Johan Almbladh <johan.almbladh@anyfinetworks.com> |
mips, bpf: Enable eBPF JITs
This patch enables the new eBPF JITs for 32-bit and 64-bit MIPS. It also
disables the old cBPF JIT to so cBPF programs are converted to use the
new JIT.
Workarounds for
mips, bpf: Enable eBPF JITs
This patch enables the new eBPF JITs for 32-bit and 64-bit MIPS. It also
disables the old cBPF JIT to so cBPF programs are converted to use the
new JIT.
Workarounds for R4000 CPU errata are not implemented by the JIT, so the
JIT is disabled if any of those workarounds are configured.
Signed-off-by: Johan Almbladh <johan.almbladh@anyfinetworks.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20211005165408.2305108-7-johan.almbladh@anyfinetworks.com
show more ...
|
| 72570224 | 05-Oct-2021 |
Johan Almbladh <johan.almbladh@anyfinetworks.com> |
mips, bpf: Add JIT workarounds for CPU errata
This patch adds workarounds for the following CPU errata to the MIPS
eBPF JIT, if enabled in the kernel configuration.
- R10000 ll/sc weak ordering
mips, bpf: Add JIT workarounds for CPU errata
This patch adds workarounds for the following CPU errata to the MIPS
eBPF JIT, if enabled in the kernel configuration.
- R10000 ll/sc weak ordering
- Loongson-3 ll/sc weak ordering
- Loongson-2F jump hang
The Loongson-2F nop errata is implemented in uasm, which the JIT uses,
so no additional mitigations are needed for that.
Signed-off-by: Johan Almbladh <johan.almbladh@anyfinetworks.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Link: https://lore.kernel.org/bpf/20211005165408.2305108-6-johan.almbladh@anyfinetworks.com
show more ...
|
| fbc802de | 05-Oct-2021 |
Johan Almbladh <johan.almbladh@anyfinetworks.com> |
mips, bpf: Add new eBPF JIT for 64-bit MIPS
This is an implementation on of an eBPF JIT for 64-bit MIPS III-V and
MIPS64r1-r6. It uses the same framework introduced by the 32-bit JIT.
Signed-off-by
mips, bpf: Add new eBPF JIT for 64-bit MIPS
This is an implementation on of an eBPF JIT for 64-bit MIPS III-V and
MIPS64r1-r6. It uses the same framework introduced by the 32-bit JIT.
Signed-off-by: Johan Almbladh <johan.almbladh@anyfinetworks.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20211005165408.2305108-5-johan.almbladh@anyfinetworks.com
show more ...
|
| 36366e36 | 05-Dec-2019 |
Paul Burton <paulburton@kernel.org> |
MIPS: BPF: Restore MIPS32 cBPF JIT
Commit 716850ab104d ("MIPS: eBPF: Initial eBPF support for MIPS32
architecture.") enabled our eBPF JIT for MIPS32 kernels, whereas it has
previously only been avai
MIPS: BPF: Restore MIPS32 cBPF JIT
Commit 716850ab104d ("MIPS: eBPF: Initial eBPF support for MIPS32
architecture.") enabled our eBPF JIT for MIPS32 kernels, whereas it has
previously only been availailable for MIPS64. It was my understanding at
the time that the BPF test suite was passing & JITing a comparable
number of tests to our cBPF JIT [1], but it turns out that was not the
case.
The eBPF JIT has a number of problems on MIPS32:
- Most notably various code paths still result in emission of MIPS64
instructions which will cause reserved instruction exceptions & kernel
panics when run on MIPS32 CPUs.
- The eBPF JIT doesn't account for differences between the O32 ABI used
by MIPS32 kernels versus the N64 ABI used by MIPS64 kernels. Notably
arguments beyond the first 4 are passed on the stack in O32, and this
is entirely unhandled when JITing a BPF_CALL instruction. Stack space
must be reserved for arguments even if they all fit in registers, and
the callee is free to assume that stack space has been reserved for
its use - with the eBPF JIT this is not the case, so calling any
function can result in clobbering values on the stack & unpredictable
behaviour. Function arguments in eBPF are always 64-bit values which
is also entirely unhandled - the JIT still uses a single (32-bit)
register per argument. As a result all function arguments are always
passed incorrectly when JITing a BPF_CALL instruction, leading to
kernel crashes or strange behavior.
- The JIT attempts to bail our on use of ALU64 instructions or 64-bit
memory access instructions. The code doing this at the start of
build_one_insn() incorrectly checks whether BPF_OP() equals BPF_DW,
when it should really be checking BPF_SIZE() & only doing so when
BPF_CLASS() is one of BPF_{LD,LDX,ST,STX}. This results in false
positives that cause more bailouts than intended, and that in turns
hides some of the problems described above.
- The kernel's cBPF->eBPF translation makes heavy use of 64-bit eBPF
instructions that the MIPS32 eBPF JIT bails out on, leading to most
cBPF programs not being JITed at all.
Until these problems are resolved, revert the removal of the cBPF JIT
performed by commit 716850ab104d ("MIPS: eBPF: Initial eBPF support for
MIPS32 architecture."). Together with commit f8fffebdea75 ("MIPS: BPF:
Disable MIPS32 eBPF JIT") this restores MIPS32 BPF JIT behavior back to
the same state it was prior to the introduction of the broken eBPF JIT
support.
[1] https://lore.kernel.org/linux-mips/MWHPR2201MB13583388481F01A422CE7D66D4410@MWHPR2201MB1358.namprd22.prod.outlook.com/
Signed-off-by: Paul Burton <paulburton@kernel.org>
Fixes: 716850ab104d ("MIPS: eBPF: Initial eBPF support for MIPS32 architecture.")
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Hassan Naveed <hnaveed@wavecomp.com>
Cc: Tony Ambardar <itugrok@yahoo.com>
Cc: bpf@vger.kernel.org
Cc: netdev@vger.kernel.org
Cc: linux-mips@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
show more ...
|