Commit graph

163 commits

Author SHA1 Message Date
Warner Losh b144e70a33 Remove $FreeBSD$: two-line nroff pattern
Remove /^\.\\"\n\.\\"\s*\$FreeBSD\$$\n/

Similar commit in main:
(cherry picked from commit fa9896e082)
2023-08-23 11:43:31 -06:00
Warner Losh 023fc80ee3 Remove $FreeBSD$: one-line sh pattern
Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

Similar commit in main:
(cherry picked from commit d0b2dbfa0e)
2023-08-23 11:43:30 -06:00
Warner Losh 3d497e17eb Remove $FreeBSD$: one-line .c pattern
Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Similar commit in main:
(cherry picked from commit 1d386b48a5)
2023-08-23 11:43:26 -06:00
Warner Losh 17da660ad5 Remove $FreeBSD$: two-line .h pattern
Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

Similar commit in main:
(cherry picked from commit b3e7694832)
2023-08-23 11:43:21 -06:00
Warner Losh caa41f6417 spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD
The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with:		pfg
MFC After:		3 days
Sponsored by:		Netflix

(cherry picked from commit 4d846d260e)
2023-07-25 09:13:49 -06:00
Mitchell Horne c6741d6aa7 pmcstat(8): Drop .Ud
This utility has existed for a long time and should not be advertised as
"currently under development".

While here, fix the one other warning from igor about using a newline
for a new sentence.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation

(cherry picked from commit a5b3e56330)
2023-07-17 13:48:58 -03:00
Mitchell Horne e6246bd85a pmcstat: augment an error message
It also applies to the -t argument.

Reviewed by:	jkoshy
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D39907

(cherry picked from commit bf66159342)
2023-06-09 15:14:59 -03:00
Piotr Pawel Stefaniak a7ce2cf0ff Update a sysctl name to nbuffers_pcpu in hwpmc.4 and pmcstat.c
This change was missed in r333509 (e6b475e0af).

(cherry picked from commit 0939f965d8)
2021-09-25 10:34:38 +02:00
Mitchell Horne d87ee7b97f pmcstat: fix the usage message for -t
A slight mix-up of the flags means this case isn't triggered when it
should be. Now, `pmcstat -s event -t 1234` will print the correct
error message.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation

(cherry picked from commit c4473add1d)
2021-04-29 10:21:19 -03:00
Mitchell Horne 112e303018 pmcstat: set initial counter value to zero
For an infrequent event, pmcstat may report (u_long)-1 for CPUs where
the counter was never incremented. Just set this to zero, instead.

ev->ev_count is passed as the 'count' argument to pmc_allocate(3), but
this wasn't always the case.

Reviewed by:	gnn
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D29887

(cherry picked from commit edcf9e59c3)
2021-04-29 10:19:20 -03:00
Andrew Turner 9fd96b416c Add the missing pmcstat log entries
Sponsored by:	Innovate UK
2021-01-05 10:03:05 +00:00
Mateusz Piotrowski d3231fbdd2 pmcstat: Fix a typo in the usage message
Reviewed by:	emaste
Approved by:	emaste
Differential Revision:	https://reviews.freebsd.org/D26082
2020-11-04 15:44:59 +00:00
Mateusz Piotrowski 51ae4071cb pmcstat.8: Improve description of -T 2020-08-17 10:51:03 +00:00
Mateusz Guzik ec3b2a79b5 pmcstat: fix build on non-64 bit platforms 2020-08-05 20:42:08 +00:00
Mateusz Guzik 94e9ef85c5 pmcstat: implement showing offsets into symbols in top mode
The -I option (and hotkey) is reused for this. Skipping symbol resolution is
moved to the new -A option (and hotkey).

While arguably this violates POLA I think it's a change for the better.
ALso note the -I option was added in head.

Differential Revision:	https://reviews.freebsd.org/D21658
2020-08-05 19:05:49 +00:00
Emmanuel Vadot ee55186dfd pmc: Add include path for libpmcstat as it is an internallib
Reviewed by:	bapt
Differential Revision:	https://reviews.freebsd.org/D24173
2020-03-25 01:32:16 +00:00
Ryan Moeller 797711a84f libpmcstat: Try /boot/modules if module not found
Modules from ports/pkg are commonly installed to /boot/modules rather than to
the same directory the kernel resides in.  Look there if a module is not found
next to the kernel.

Submitted by:	mmacy
Reported by:	Nick Principe <nap@iXsystems.com>
Approved by:	mmacy (mentor)
MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2020-03-12 23:04:40 +00:00
Warner Losh 52467047aa Regularize the Netflix copyright
Use recent best practices for Copyright form at the top of
the license:
1. Remove all the All Rights Reserved clauses on our stuff. Where we
   piggybacked others, use a separate line to make things clear.
2. Use "Netflix, Inc." everywhere.
3. Use a single line for the copyright for grep friendliness.
4. Use date ranges in all places for our stuff.

Approved by: Netflix Legal (who gave me the form), adrian@ (pmc files)
2019-02-04 21:28:25 +00:00
Matt Macy f992dd4b5c pmc: convert native to jsonl and track TSC value of samples
- add '-j' options to filter to enable converting native pmc
  log format to json lines format to enable the use of scripts
  and external tooling

% pmc filter -j pmc.log pmc.jsonl

- Record the tsc value in sampling interrupts as opposed to
  recording nanotime when the sample is copied to a global log
  in hardclock - potentially many milliseconds later.

- At initialize record the tsc_freq and the time of day to give
  us an offset for translating the tsc values in callchain records
2018-06-07 02:03:22 +00:00
Matt Macy b2ca2e50b9 hwpmc: add summary command and further metadata extensions
metadata changes:
- log pmc sample rate with pmcallocate
- log proc flags with thread / process logging
  to identify user vs kernel threads

fixes:
- use log cpuid to translate event id to event name

Implement rudimentary summary command to track sample
counts by thread and process name within a pmc log.

% make -j4 buildkernel >& /dev/null &
% sudo pmcstat -S unhalted_core_cycles -S llc-misses -O foo sleep 15
% pmc summary foo
cpu_clk_unhalted.thread_p_any:
        idle: 138108207162
        clang-6.0: 105336158004
        sh: 72340108510
        make: 8642012963
        kernel: 7754011631
longest_lat_cache.miss:
        clang-6.0: 87502625
        sh: 40901227
        make: 5500165
        kernel: 3300099
        awk: 2000060

%  pmc summary -f ~/foo
idx: 278 name: cpu_clk_unhalted.thread_p_any rate: 2000003
idle: 69054
clang-6.0: 52668
sh: 36170
make: 4321
kernel: 3877
hwpmc: proc(7445): 3319
awk: 1289
xargs: 357
rand_harvestq: 181
mtree: 102
intr: 53
zfskern: 31
usb: 7
pagedaemon: 4
ntpd: 3
syslogd: 1
acpi_thermal: 1
logger: 1
syncer: 1
snmptrapd: 1
sleep: 1
idx: 17 name: longest_lat_cache.miss rate: 100003
clang-6.0: 875
sh: 409
make: 55
kernel: 33
awk: 20
hwpmc: proc(7445): 14
xargs: 9
idle: 8
intr: 3
zfskern: 2
2018-06-06 02:48:09 +00:00
Matt Macy 5de96e33d6 hwpmc: support sampling both kernel and user stacks when interrupted in kernel
This adds the -U options to pmcstat which will attribute in-kernel samples
back to the user stack that invoked the system call. It is not the default,
because when looking at kernel profiles it is generally more desirable to
merge all instances of a given system call together.

Although heavily revised, this change is directly derived from D7350 by
Jonathan T. Looney.

Obtained from: jtl
Sponsored by: Juniper Networks, Limelight Networks
2018-06-04 01:10:23 +00:00
Matt Macy fbf962e6bb libpmc: allow substring for list and add function for printing event details 2018-06-01 00:45:53 +00:00
Matt Macy ae266f9f94 pmcstat: suppress uninitialized warning of event 2018-05-27 17:49:27 +00:00
Matt Macy 959826ca1b pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the
vendor provided pmu-events tables and sundry cleanups.

The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:

- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system

Update man page with newer sample types and remove unused sample type.
2018-05-26 19:29:19 +00:00
Matt Macy 5506ceb87f Revert r334242 "pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the"
because of squash commit messages
2018-05-26 19:26:19 +00:00
Matt Macy 4928135658 pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the
vendor provided pmu-events tables and sundry cleanups.

The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:

- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system

Update man page with newer sample types and remove unused sample type.

Squashed commit of the following:

commit 4459d43eff815bec08ccc5533dbe5de846f03128
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Sat May 26 00:06:31 2018 -0700

    libpmc: fix pmu function signatures for non amd64

commit a2cb8bbc586c65d41f9b291430a2261ec67b59fe
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:38:11 2018 -0700

    pmcstat: fix indentation of usage

commit f686954b15ff56a833ac80404898977cb80a265b
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:19:49 2018 -0700

    pmclog(3): add callchain and pmcallocatedyn, remove pcsample

commit 73e13a0d2e9498c81c150d14d022050cee7511bb
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:19:00 2018 -0700

    pmclog.h: GC pcsample field

commit 3e93ffd65da641fa657539dad3c48e281f8b5798
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:05:57 2018 -0700

    hwpmc: make Intel core CPUs use external event tables

commit 634f5fae1e1644ac324003136c66cd9c619d1c93
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:00:06 2018 -0700

    pmclog: update log record types, bump PMC_MAJOR
    - explicitly make log record types a multiple of 8 bytes
    - hook in pmu event types for pmc_allocate records
    - remove references to no longer PCSAMPLE record

commit 83d84fcd2d65bdf6ddcb2e155a22f0cfa2a9c225
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 21:52:10 2018 -0700

    libpmc: add support for having vendor table driven pmc_allocate

commit 9e6ad63c40c2fce8404847ace5078ca6cb33a736
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 19:11:33 2018 -0700

    hwpmc_core: add accessors for EVSEL & UMASK, make IAP_UMASK useful to user

commit 859dceb93daa6419a48c794db99b6758e5b041c9
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 19:09:45 2018 -0700

    pmcstat: update usage and man page as well as make -L consistent with pmccontrol

commit 79c7d8597e28c2eb13f5f9113e65ec2792ca57b1
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 18:07:03 2018 -0700

    pmu_util: add support for all current intel event keywords

commit d8089c7f6a6c8527f38324252b1ffb47004694c6
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 17:45:00 2018 -0700

    add description for new arguments

commit 058336740bab53c62ec88a3a026ea848cf3878c6
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 17:38:15 2018 -0700

    libpmc: move pmu_events table and pmu_utils out of libpmcstat so that they can be used by pmc_allocate

commit 049b66b382e2f833c3f47bc8df9e750cb265709f
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:12:41 2018 -0700

    pmcstat: hook pmu_events counter description utility routines in

commit f5e01e7b37a691dc045e1aa16b3ebdd162515de8
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:11:59 2018 -0700

    pmu_events: add utility routines for listing counters and their descriptions

commit cba4d4f8907f772279f86f18f915e0d74d33ac56
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:09:50 2018 -0700

    pmu-events: expand out skylake regex to simplify string matches
2018-05-26 18:12:50 +00:00
Sean Bruno e22ee9b247 pmcstat(8)
- Document per thread filtering.

Submitted by:	kbowling
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15549
2018-05-25 18:11:13 +00:00
Matt Macy 0a87e3ff64 pmcstat: add usage strings for -L and -I 2018-05-24 07:18:49 +00:00
Matt Macy 4bd2a4b58e pmcstat top mode + -I - use fully qualified address 2018-05-24 06:47:04 +00:00
Matt Macy 357405d985 pmcstat: don't compare signed and unsigned 2018-05-24 04:43:40 +00:00
Matt Macy e98bbcf9ca libpmcstat: compile in events based on json description 2018-05-24 04:30:06 +00:00
Matt Macy 95104040a3 pmc: bump PMC major version to reflect ABI breakage and make warning not require verbose 2018-05-23 19:55:47 +00:00
Matt Macy 0b5dc7f64f hwpmc: add thread id field to callchain and context switch records to allow filtering
on thread in post-processing.

To generate stacks for just ${THREADID}:

pmcstat -R ${PREFIX}.pmcstat -L ${THREADID} -z100 -G ${PREFIX}.stacks

Sponsored by:	Limelight Networks
2018-05-23 17:25:00 +00:00
Matt Macy 821a352a77 pmcstat: add option to not decode the leaf function in top mode
-I will allow the user to see the hot instruction in question
as opposed getting the name of the function
2018-05-22 04:45:46 +00:00
Fabien Thomas cb43ed2815 Only call flush in pipe mode.
It fixes a crash with a socket in top mode.

Ex:
# pmcstat -R 127.0.0.1:8080 -T -w1
then
# pmcstat -n1 -Sclock.prof -Slock.failed -O 127.0.0.1:8080

MFC after:	1 week
Sponsored by:	Stormshield
2018-01-17 16:55:35 +00:00
Pedro F. Giffuni 1de7b4b805 various: general adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.
2017-11-27 15:37:16 +00:00
Andriy Voskoboinyk 7d41856e52 pmcstat(8): unbreak build with LDFLAGS+=-Wl,--as-needed
libpmcstat.a requires libelf.so; reorder them accordingly.
2017-11-03 00:12:32 +00:00
Bryan Drewery ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Ruslan Bukin d27927f731 Extract a set of pmcstat functions and interfaces to the new internal
library -- libpmcstat.

This includes PMC logging module, symbols lookup functions,
ELF parsing, process management, PMC attachment, etc.

This allows to reuse code while building new hwpmc(4)-based applications.

Also add pmcstat_symbol_search_by_name() function that allows to find
mapped IP range for a given function name.

Reviewed by:	kib
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D12718
2017-10-24 16:28:00 +00:00
Konstantin Belousov e28f262a86 Fix off by one in calculation of the number of buckets for the pc
addresses.

Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D11864
2017-08-04 18:02:54 +00:00
Pedro F. Giffuni 780154cbaf pmcstat(8); unsign some allocation variables and use reallocarray(3).
Use unsigned values in some internal variables that will be used during
allocation. The variables are used in reduced scope and have no chance of
becoming negative.

Provide bounds checking through reallocarray(3).

MFC after:	2 weeks
2017-04-22 16:02:40 +00:00
Mark Johnston 3176d87c9d Remove an unused field from struct pmcstat_image.
MFC after:	3 days
2016-10-15 23:46:55 +00:00
Stephen J. Kiernan 3ad36a25cf Just remove freeing memory in pmcstat_cleanup, since we only clean up
when we are exiting anyway.

Add NULL checks for all malloc and strdup returns.

Reviewed by:	gnn
Approved by:	sjg (mentor)
Obtained from:	Juniper Networks, Inc.
Differential Revision:	https://reviews.freebsd.org/D8002
2016-10-07 21:03:12 +00:00
Ed Schouten d93c912801 Make code compile when basename() is POSIX compliant.
The POSIX basename() function is allowed to modify its input buffer,
which means its argument is "char *". Pull a copy of the input string
before computing the base.

Reviewed by:	jtl
Differential Revision:	https://reviews.freebsd.org/D6465
2016-05-25 08:45:03 +00:00
Pedro F. Giffuni d7b1cc0a3c pmcstat: minor spelling fixes.
Mostly comments bur also a user-visible string.

MFC after:	2 weeks
2016-05-01 16:40:56 +00:00
Jonathan T. Looney cc2ccfbe20 Fix the date on the pmcstat(8) man page from r291016.
Approved by:	gnn (mentor)
MFC after:	1 month
X-MFC with:	r291016
Pointy hat to:	jtl
2015-11-18 15:12:24 +00:00
Jonathan T. Looney 465dadb59b Support a wider history counter in pmcstat(8) gmon output
pmcstat(8) contains an option to output sampling data in a gmon format
compatible with gprof(1). Currently, it uses the default histcounter,
which is an (unsigned short). With large sets of sampling data, it
is possible to overflow the maximum value provided by an (unsigned
short).

This change adds the -e argument to pmcstat. If -e and -g are both
specified, pmcstat will use a histcounter type of uint64_t.

Differential Revision:	https://reviews.freebsd.org/D4151
Reviewed by:	jhb, bjk
Approved by:	gnn (mentor)
MFC after:	1 month
Sponsored by:	Juniper Networks
2015-11-18 14:52:01 +00:00
Jonathan T. Looney e64ba55dd2 Change the driver stats to what they really are: unsigned values.
When pmcstat exits after some samples were dropped, give the user an
idea of how many were lost. (Granted, these are global numbers, but
they may still help quantify the scope of the loss.)

Differential Revision:	https://reviews.freebsd.org/D4123
Approved by:	gnn (mentor)
MFC after:	1 month
Sponsored by:	Juniper Networks
2015-11-16 15:16:09 +00:00
Fabien Thomas 66ca84a7ba Fix for r288176 changes related to debug symbols move.
Reviewed by: emaste
Sponsored by:	Stormshield
2015-10-08 09:46:35 +00:00
John Baldwin 7816cb4a0b Fix an off by one error in r283613: Like regular ffs(), CPU_FFS() returns
1 for CPU 0, etc. so the return value must be decremented to obtain the
first valid CPU ID.

Submitted by:	fabient
MFC after:	1 week
2015-09-01 17:52:43 +00:00