- Jul 30, 2021
-
-
Quentin Monnet authored
Split BTF objects are typically BTF objects for kernel modules, which are incrementally built on top of kernel BTF instead of redefining all kernel symbols they need. We can use bpftool with its -B command-line option to dump split BTF objects. It works well when the handle provided for the BTF object to dump is a "path" to the BTF object, typically under /sys/kernel/btf, because bpftool internally calls btf__parse_split() which can take a "base_btf" pointer and resolve the BTF reconstruction (although in that case, the "-B" option is unnecessary because bpftool performs autodetection). However, it did not work so far when passing the BTF object through its id, because bpftool would call btf__get_from_id() which did not provide a way to pass a "base_btf" pointer. In other words, the following works: # bpftool btf dump file /sys/kernel/btf/i2c_smbus -B /sys/kernel/btf/vmlinux But this was not possible: # bpftool btf dump id 6 -B /sys/kernel/btf/vmlinux The libbpf API has recently changed, and btf__get_from_id() has been deprecated in favour of btf__load_from_kernel_by_id() and its version with support for split BTF, btf__load_from_kernel_by_id_split(). Let's update bpftool to make it able to dump the BTF object in the second case as well. Signed-off-by:Quentin Monnet <quentin@isovalent.com> Signed-off-by:
Andrii Nakryiko <andrii@kernel.org> Acked-by:
John Fastabend <john.fastabend@gmail.com> Link: https://lore.kernel.org/bpf/20210729162028.29512-9-quentin@isovalent.com
-
Quentin Monnet authored
Add a new API function btf__load_from_kernel_by_id_split(), which takes a pointer to a base BTF object in order to support split BTF objects when retrieving BTF information from the kernel. Reference: https://github.com/libbpf/libbpf/issues/314 Signed-off-by:
-
Quentin Monnet authored
Replace the calls to function btf__get_from_id(), which we plan to deprecate before the library reaches v1.0, with calls to btf__load_from_kernel_by_id() in tools/ (bpftool, perf, selftests). Update the surrounding code accordingly (instead of passing a pointer to the btf struct, get it as a return value from the function). Signed-off-by:
-
Quentin Monnet authored
Make sure to call btf__free() (and not simply free(), which does not free all pointers stored in the struct) on pointers to struct btf objects retrieved at various locations. These were found while updating the calls to btf__get_from_id(). Fixes: 999d82cb ("tools/bpf: enhance test_btf file testing to test func info") Fixes: 254471e5 ("tools/bpf: bpftool: add support for func types") Fixes: 7b612e29 ("perf tools: Synthesize PERF_RECORD_* for loaded BPF programs") Fixes: d56354dc ("perf tools: Save bpf_prog_info and BTF of new BPF programs") Fixes: 47c09d6a ("bpftool: Introduce "prog profile" command") Fixes: fa853c4b ("perf stat: Enable counting events for BPF programs") Signed-off-by:
-
Quentin Monnet authored
Rename function btf__get_from_id() as btf__load_from_kernel_by_id() to better indicate what the function does. Change the new function so that, instead of requiring a pointer to the pointer to update and returning with an error code, it takes a single argument (the id of the BTF object) and returns the corresponding pointer. This is more in line with the existing constructors. The other tools calling the (soon-to-be) deprecated btf__get_from_id() function will be updated in a future commit. References: - https://github.com/libbpf/libbpf/issues/278 - https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#btfh-apis Signed-off-by:
-
Quentin Monnet authored
As part of the effort to move towards a v1.0 for libbpf, rename btf__load() function, used to "upload" BTF information into the kernel, as btf__load_into_kernel(). This new name better reflects what the function does. References: - https://github.com/libbpf/libbpf/issues/278 - https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#btfh-apis Signed-off-by:
-
Quentin Monnet authored
Variable "err" is initialised to -EINVAL so that this error code is returned when something goes wrong in libbpf_find_prog_btf_id(). However, a recent change in the function made use of the variable in such a way that it is set to 0 if retrieving linear information on the program is successful, and this 0 value remains if we error out on failures at later stages. Let's fix this by setting err to -EINVAL later in the function. Fixes: e9fc3ce9 ("libbpf: Streamline error reporting for high-level APIs") Signed-off-by:
-
Yonghong Song authored
To avoid kernel build failure due to some missing .BTF-ids referenced functions/types, the patch ([1]) tries to fill btf_id 0 for these types. In bpf verifier, for percpu variable and helper returning btf_id cases, verifier already emitted proper warning with something like verbose(env, "Helper has invalid btf_id in R%d\n", regno); verbose(env, "invalid return type %d of func %s#%d\n", fn->ret_type, func_id_name(func_id), func_id); But this is not the case for bpf_iter context arguments. I hacked resolve_btfids to encode btf_id 0 for struct task_struct. With `./test_progs -n 7/5`, I got, 0: (79) r2 = *(u64 *)(r1 +0) func 'bpf_iter_task' arg0 has btf_id 29739 type STRUCT 'bpf_iter_meta' ; struct seq_file *seq = ctx->meta->seq; 1: (79) r6 = *(u64 *)(r2 +0) ; struct task_struct *task = ctx->task; 2: (79) r7 = *(u64 *)(r1 +8) ; if (task == (void *)0) { 3: (55) if r7 != 0x0 goto pc+11 ... ; BPF_SEQ_PRINTF(seq, "%8d %8d\n", task->tgid, task->pid); 26: (61) r1 = *(u32 *)(r7 +1372) Type '(anon)' is not a struct Basically, verifier will return btf_id 0 for task_struct. Later on, when the code tries to access task->tgid, the verifier correctly complains the type is '(anon)' and it is not a struct. Users still need to backtrace to find out what is going on. Let us catch the invalid btf_id 0 earlier and provide better message indicating btf_id is wrong. The new error message looks like below: R1 type=ctx expected=fp ; struct seq_file *seq = ctx->meta->seq; 0: (79) r2 = *(u64 *)(r1 +0) func 'bpf_iter_task' arg0 has btf_id 29739 type STRUCT 'bpf_iter_meta' ; struct seq_file *seq = ctx->meta->seq; 1: (79) r6 = *(u64 *)(r2 +0) ; struct task_struct *task = ctx->task; 2: (79) r7 = *(u64 *)(r1 +8) invalid btf_id for context argument offset 8 invalid bpf_context access off=8 size=8 [1] https://lore.kernel.org/bpf/20210727132532.2473636-1-hengqi.chen@gmail.com/ Signed-off-by:Yonghong Song <yhs@fb.com> Signed-off-by:
-
Hengqi Chen authored
Kernel functions referenced by .BTF_ids may be changed from global to static and get inlined or get renamed/removed, and thus disappears from BTF. This causes kernel build failure when resolve_btfids do id patch for symbols in .BTF_ids in vmlinux. Update resolve_btfids to emit warning messages and patch zero id for missing symbols instead of aborting kernel build process. Suggested-by:
Hengqi Chen <hengqi.chen@gmail.com> Signed-off-by:
-
- Jul 28, 2021
-
-
Stanislav Fomichev authored
Current max cgroup storage value size is 4k (PAGE_SIZE). The other local storages accept up to 64k (BPF_LOCAL_STORAGE_MAX_VALUE_SIZE). Let's align max cgroup value size with the other storages. For percpu, the max is 32k (PCPU_MIN_UNIT_SIZE) because percpu allocator is not happy about larger values. netcnt test is extended to exercise those maximum values (non-percpu max size is close to, but not real max). v4: * remove inner union (Andrii Nakryiko) * keep net_cnt on the stack (Andrii Nakryiko) v3: * refine SIZEOF_BPF_LOCAL_STORAGE_ELEM comment (Yonghong Song) * anonymous struct in percpu_net_cnt & net_cnt (Yonghong Song) * reorder free (Yonghong Song) v2: * cap max_value_size instead of BUILD_BUG_ON (Martin KaFai Lau) Signed-off-by:
Stanislav Fomichev <sdf@google.com> Signed-off-by:
Martin KaFai Lau <kafai@fb.com> Link: https://lore.kernel.org/bpf/20210727222335.4029096-1-sdf@google.com
-
Martynas Pumputis authored
When loading in parallel multiple programs which use the same to-be pinned map, it is possible that two instances of the loader will call bpf_object__create_maps() at the same time. If the map doesn't exist when both instances call bpf_object__reuse_map(), then one of the instances will fail with EEXIST when calling bpf_map__pin(). Fix the race by retrying reusing a map if bpf_map__pin() returns EEXIST. The fix is similar to the one in iproute2: e4c4685fd6e4 ("bpf: Fix race condition with map pinning"). Before retrying the pinning, we don't do any special cleaning of an internal map state. The closer code inspection revealed that it's not required: - bpf_object__create_map(): map->inner_map is destroyed after a successful call, map->fd is closed if pinning fails. - bpf_object__populate_internal_map(): created map elements is destroyed upon close(map->fd). - init_map_slots(): slots are freed after their initialization. Signed-off-by:Martynas Pumputis <m@lambda.lt> Signed-off-by:
-
Jason Wang authored
Remove the repeated word 'the' in line 48. Signed-off-by:
Jason Wang <wangborong@cdjrlc.com> Signed-off-by:
-
Juhee Kang authored
There are recently added xdp samples (xdp_redirect_map_multi and xdpsock_ctrl_proc) which are not managed by .gitignore. This commit adds these files to .gitignore. Signed-off-by:
Juhee Kang <claudiajkang@gmail.com> Signed-off-by:
-
Juhee Kang authored
The current behavior of 'tracex7' doesn't consist with other bpf samples tracex{1..6}. Other samples do not require any argument to run with, but tracex7 should be run with btrfs device argument. (it should be executed with test_override_return.sh) Currently, tracex7 doesn't have any description about how to run this program and raises an unexpected error. And this result might be confusing since users might not have a hunch about how to run this program. // Current behavior # ./tracex7 sh: 1: Syntax error: word unexpected (expecting ")") // Fixed behavior # ./tracex7 ERROR: Run with the btrfs device argument! In order to fix this error, this commit adds logic to report a message and exit when running this program with a missing argument. Additionally in test_override_return.sh, there is a problem with multiple directory(tmpmnt) creation. So in this commit adds a line with removing the directory with every execution. Signed-off-by:
-
- Jul 27, 2021
-
-
Jussi Maki authored
In the tc_redirect test only use ping6 if it's available and otherwise fall back to using "ping -6". Signed-off-by:Jussi Maki <joamaki@gmail.com>
-
Andrii Nakryiko authored
Alexei Starovoitov says: ==================== From: Alexei Starovoitov <ast@kernel.org> Split CO-RE processing logic from libbpf into separate file with an interface that doesn't dependend on libbpf internal details. As the next step relo_core.c will be compiled with libbpf and with the kernel. The _internal_ interface between libbpf/CO-RE and kernel/CO-RE will be: int bpf_core_apply_relo_insn(const char *prog_name, struct bpf_insn *insn, int insn_idx, const struct bpf_core_relo *relo, int relo_idx, const struct btf *local_btf, struct bpf_core_cand_list *cands); where bpf_core_relo and bpf_core_cand_list are simple types prepared by kernel and libbpf. Though diff stat shows a lot of lines inserted/deleted they are moved lines. Pls review with diff.colorMoved. ==================== Signed-off-by: -
Alexei Starovoitov authored
Move CO-RE logic into separate file. The internal interface between libbpf and CO-RE is through bpf_core_apply_relo_insn() function and few structs defined in relo_core.h. Signed-off-by:
Alexei Starovoitov <ast@kernel.org> Signed-off-by:
-
Alexei Starovoitov authored
In order to make a clean split of CO-RE logic move its types into independent header file. Signed-off-by:
-
Alexei Starovoitov authored
bpf_core_apply_relo() doesn't need to know bpf_program internals and hashmap details. Signed-off-by:
-
Alexei Starovoitov authored
CO-RE processing functions don't need to know 'struct bpf_program' details. Cleanup the layering to eventually be able to move CO-RE logic into a separate file. Signed-off-by:
-
- Jul 24, 2021
-
-
Johan Almbladh authored
Each test case can have a set of sub-tests, where each sub-test can run the cBPF/eBPF test snippet with its own data_size and expected result. Before, the end of the sub-test array was indicated by both data_size and result being zero. However, most or all of the internal eBPF tests has a data_size of zero already. When such a test also had an expected value of zero, the test was never run but reported as PASS anyway. Now the test runner always runs the first sub-test, regardless of the data_size and result values. The sub-test array zero-termination only applies for any additional sub-tests. There are other ways fix it of course, but this solution at least removes the surprise of eBPF tests with a zero result always succeeding. Signed-off-by:
Johan Almbladh <johan.almbladh@anyfinetworks.com> Signed-off-by:
-
Johan Almbladh authored
This test now operates on DW as stated instead of W, which was already covered by another test. Signed-off-by:
-
Evgeniy Litvinenko authored
Add a list of vmtest script dependencies to make it easier for new contributors to get going. Signed-off-by:
Evgeniy Litvinenko <evgeniyl@fb.com> Signed-off-by:
-
Evgeniy Litvinenko authored
Add bpf_map__pin_path, so that the inconsistently named bpf_map__get_pin_path can be deprecated later. This is part of the effort towards libbpf v1.0: https://github.com/libbpf/libbpf/issues/307 Also, add a selftest for the new function. Signed-off-by:
-
Andrii Nakryiko authored
Martin KaFai says: ==================== This set is to allow bpf tcp iter to call bpf_(get|set)sockopt. With bpf-tcp-cc, new algo rollout happens more often. Instead of restarting the applications to pick up the new tcp-cc, this set allows the bpf tcp iter to call bpf_(get|set)sockopt(TCP_CONGESTION). It is not limited to TCP_CONGESTION, the bpf tcp iter can call bpf_(get|set)sockopt() with other options. The bpf tcp iter can read into all the fields of a tcp_sock, so there is a lot of flexibility to select the desired sk to do setsockopt(), e.g. it can test for TCP_LISTEN only and leave the established connections untouched, or check the addr/port, or check the current tcp-cc name, ...etc. Patch 1-4 are some cleanup and prep work in the tcp and bpf seq_file. Patch 5 is to have the tcp seq_file iterate on the port+addr lhash2 instead of the port only listening_hash. Patch 6 is to have the bpf tcp iter doing batching which then allows lock_sock. lock_sock is needed for setsockopt. Patch 7 allows the bpf tcp iter to call bpf_(get|set)sockopt. v2: - Use __GFP_NOWARN in patch 6 - Add bpf_getsockopt() in patch 7 to give a symmetrical user experience. selftest in patch 8 is changed to also cover bpf_getsockopt(). - Remove CAP_NET_ADMIN check in patch 7. Tracing bpf prog has already required CAP_SYS_ADMIN or CAP_PERFMON. - Move some def macros to bpf_tracing_net.h in patch 8 ==================== Signed-off-by: -
Martin KaFai Lau authored
This patch adds tests for the batching and bpf_(get|set)sockopt in bpf tcp iter. It first creates: a) 1 non SO_REUSEPORT listener in lhash2. b) 256 passive and active fds connected to the listener in (a). c) 256 SO_REUSEPORT listeners in one of the lhash2 bucket. The test sets all listeners and connections to bpf_cubic before running the bpf iter. The bpf iter then calls setsockopt(TCP_CONGESTION) to switch each listener and connection from bpf_cubic to bpf_dctcp. The bpf iter has a random_retry mode such that it can return EAGAIN to the usespace in the middle of a batch. Signed-off-by:
Eric Dumazet <edumazet@google.com> Acked-by:
Kuniyuki Iwashima <kuniyu@amazon.co.jp> Acked-by:
-
Martin KaFai Lau authored
This patch allows bpf tcp iter to call bpf_(get|set)sockopt. To allow a specific bpf iter (tcp here) to call a set of helpers, get_func_proto function pointer is added to bpf_iter_reg. The bpf iter is a tracing prog which currently requires CAP_PERFMON or CAP_SYS_ADMIN, so this patch does not impose other capability checks for bpf_(get|set)sockopt. Signed-off-by:
-
Martin KaFai Lau authored
This patch does batching and lock_sock for the bpf tcp iter. It does not affect the proc fs iteration. With bpf-tcp-cc, new algo rollout happens more often. Instead of restarting the application to pick up the new tcp-cc, the next patch will allow bpf iter to do setsockopt(TCP_CONGESTION). This requires locking the sock. Also, unlike the proc iteration (cat /proc/net/tcp[6]), the bpf iter can inspect all fields of a tcp_sock. It will be useful to have a consistent view on some of the fields (e.g. the ones reported in tcp_get_info() that also acquires the sock lock). Double lock: locking the bucket first and then locking the sock could lead to deadlock. This patch takes a batching approach similar to inet_diag. While holding the bucket lock, it batch a number of sockets into an array first and then unlock the bucket. Before doing show(), it then calls lock_sock_fast(). In a machine with ~400k connections, the maximum number of sk in a bucket of the established hashtable is 7. 0.02% of the established connections fall into this bucket size. For listen hash (port+addr lhash2), the bucket is usually very small also except for the SO_REUSEPORT use case which the userspace could have one SO_REUSEPORT socket per thread. While batching is used, it can also minimize the chance of missing sock in the setsockopt use case if the whole bucket is batched. This patch will start with a batch array with INIT_BATCH_SZ (16) which will be enough for the most common cases. bpf_iter_tcp_batch() will try to realloc to a larger array to handle exception case (e.g. the SO_REUSEPORT case in the lhash2). Signed-off-by:
-
Martin KaFai Lau authored
This patch moves the tcp seq_file iteration on listeners from the port only listening_hash to the port+addr lhash2. When iterating from the bpf iter, the next patch will need to lock the socket such that the bpf iter can call setsockopt (e.g. to change the TCP_CONGESTION). To avoid locking the bucket and then locking the sock, the bpf iter will first batch some sockets from the same bucket and then unlock the bucket. If the bucket size is small (which usually is), it is easier to batch the whole bucket such that it is less likely to miss a setsockopt on a socket due to changes in the bucket. However, the port only listening_hash could have many listeners hashed to a bucket (e.g. many individual VIP(s):443 and also multiple by the number of SO_REUSEPORT). We have seen bucket size in tens of thousands range. Also, the chance of having changes in some popular port buckets (e.g. 443) is also high. The port+addr lhash2 was introduced to solve this large listener bucket issue. Also, the listening_hash usage has already been replaced with lhash2 in the fast path inet[6]_lookup_listener(). This patch follows the same direction on moving to lhash2 and iterates the lhash2 instead of listening_hash. Signed-off-by:
-
Martin KaFai Lau authored
The current listening_get_next() is overloaded by passing NULL to the 2nd arg, like listening_get_next(seq, NULL), to mean get_first(). This patch moves some logic from the listening_get_next() into a new function listening_get_first(). It will be equivalent to the current established_get_first() and established_get_next() setup. get_first() is to find a non empty bucket and return the first sk. get_next() is to find the next sk of the current bucket and then resorts to get_first() if the current bucket is exhausted. The next patch is to move the listener seq_file iteration from listening_hash (port only) to lhash2 (port+addr). Separating out listening_get_first() from listening_get_next() here will make the following lhash2 changes cleaner and easier to follow. Signed-off-by:
-
Martin KaFai Lau authored
A following patch will create a separate struct to store extra bpf_iter state and it will embed the existing tcp_iter_state like this: struct bpf_tcp_iter_state { struct tcp_iter_state state; /* More bpf_iter specific states here ... */ } As a prep work, this patch removes the "struct tcp_seq_afinfo *bpf_seq_afinfo" where its purpose is to tell if it is iterating from bpf_iter instead of proc fs. Currently, if "*bpf_seq_afinfo" is not NULL, it is iterating from bpf_iter. The kernel should not filter by the addr family and leave this filtering decision to the bpf prog. Instead of adding a "*bpf_seq_afinfo" pointer, this patch uses the "seq->op == &bpf_iter_tcp_seq_ops" test to tell if it is iterating from the bpf iter. The bpf_iter_(init|fini)_tcp() is left here to prepare for the change of a following patch. Signed-off-by: -
Martin KaFai Lau authored
This patch refactors the net and family matching into two new helpers, seq_sk_match() and seq_file_family(). seq_file_family() is in the later part of the file to prepare the change of a following patch. Signed-off-by:
-
Martin KaFai Lau authored
st->bucket stores the current bucket number. st->offset stores the offset within this bucket that is the sk to be seq_show(). Thus, st->offset only makes sense within the same st->bucket. These two variables are an optimization for the common no-lseek case. When resuming the seq_file iteration (i.e. seq_start()), tcp_seek_last_pos() tries to continue from the st->offset at bucket st->bucket. However, it is possible that the bucket pointed by st->bucket has changed and st->offset may end up skipping the whole st->bucket without finding a sk. In this case, tcp_seek_last_pos() currently continues to satisfy the offset condition in the next (and incorrect) bucket. Instead, regardless of the offset value, the first sk of the next bucket should be returned. Thus, "bucket == st->bucket" check is added to tcp_seek_last_pos(). The chance of hitting this is small and the issue is a decade old, so targeting for the next tree. Fixes: a8b690f9 ("tcp: Fix slowness in read /proc/net/tcp") Signed-off-by:
-
- Jul 23, 2021
-
-
Jiri Olsa authored
Export bpf_program__attach_kprobe_opts as a public API. Rename bpf_program_attach_kprobe_opts to bpf_kprobe_opts and turn it into OPTS struct. Suggested-by:
Jiri Olsa <jolsa@kernel.org> Signed-off-by:
Alan Maguire <alan.maguire@oracle.com> Tested-by:
-
Jiri Olsa authored
Allow to specify decimal offset in SEC macro, like: SEC("kprobe/bpf_fentry_test7+5") Add selftest for that. Suggested-by: -
Jiri Olsa authored
Add missing free() for func pointer in attach_kprobe function. Fixes: a2488b5f ("libbpf: Allow specification of "kprobe/function+offset"") Reported-by:
-
Martynas Pumputis authored
Previously, the newly introduced test case in test_map_in_map(), which checks whether the inner map is destroyed after unsuccessful creation of the outer map, logged the following harmless and expected error: libbpf: map 'mim': failed to create: Invalid argument(-22) libbpf: failed to load object './test_map_in_map_invalid.o' To avoid any possible confusion, mute the logging during loading of the prog. Fixes: 08f71a1e ("selftests/bpf: Check inner map deletion") Reported-by: -
Colin Ian King authored
The variable stype is being initialized with a value that is never read, it is being updated later on. The assignment is redundant and can be removed. Addresses-Coverity: ("Unused value") Signed-off-by:Colin Ian King <colin.king@canonical.com> Signed-off-by:
-
Arnd Bergmann authored
kp->addr is a pointer, so it cannot be cast directly to a 'u64' when it gets interpreted as an integer value: kernel/trace/bpf_trace.c: In function '____bpf_get_func_ip_kprobe': kernel/trace/bpf_trace.c:968:21: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast] 968 | return kp ? (u64) kp->addr : 0; Use the uintptr_t type instead. Fixes: 9ffd9f3f ("bpf: Add bpf_get_func_ip helper for kprobe programs") Signed-off-by:
Arnd Bergmann <arnd@arndb.de> Signed-off-by:
-
- Jul 21, 2021
-
-
Andrii Nakryiko authored
Alan Maguire says: ==================== This series aims to resolve further issues with the BTF typed data dumping interfaces in libbpf. Compilation failures with use of __int128 on 32-bit platforms were reported [1]. As a result, the use of __int128 in libbpf typed data dumping is replaced with __u64 usage for bitfield manipulations. In the case of 128-bit integer values, they are simply split into two 64-bit hex values for display (patch 1). Tests are added for __int128 display in patch 2, using conditional compilation to avoid problems with a lack of __int128 support. Patch 3 resolves an issue Andrii noted about error propagation when handling enum data display. More followup work is required to ensure multi-dimensional char array display works correctly. [1] https://lore.kernel.org/bpf/1626362126-27775-1-git-send-email-alan.maguire@oracle.com/T/#mc2cb023acfd6c3cd0b661e385787b76bb757430d Changes since v1: - added error handling for bitfield size > 64 bits by changing function signature for bitfield retrieval to return an int error value and to set bitfield value via a __u64 * argument (Andrii) ==================== Signed-off-by:
-
