Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

Add two new kernel options to control memory locality on NUMA hardware.
- UMA_XDOMAIN enables an additional per-cpu bucket for freed memory that
was freed on a different domain from where it was

Add two new kernel options to control memory locality on NUMA hardware.
- UMA_XDOMAIN enables an additional per-cpu bucket for freed memory that
was freed on a different domain from where it was allocated. This is
only used for UMA_ZONE_NUMA (first-touch) zones.
- UMA_FIRSTTOUCH sets the default UMA policy to be first-touch for all
zones. This tries to maintain locality for kernel memory.

Reviewed by: gallatin, alc, kib
Tested by: pho, gallatin
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20929

show more ...

lib: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
pr

lib: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

show more ...

MFC @ r266724

An SVM update will follow this.

Merge head

IFC @264767

MFH: tracking commit

Sponsored by: The FreeBSD Foundation

Merge head up to r262222 (last merge was incomplete).

Expose real size of UMA allocations via libmemstat(3).

Sponsored by: Nginx, Inc.

IFC @ r225592

sys/dev/bvm/bvm_console.c - move up to the new alt-break order.

Integrate FreeBSD/head into projects/zfsd/head as of SVN revision 224870.

Get rid of MAXCPU knowledge used for internal needs only. Switch to
dynamic memory allocation to hold per-CPU memory types data (sized to
mp_maxid for UMA, and to mp_maxcpus for malloc to match the k

Get rid of MAXCPU knowledge used for internal needs only. Switch to
dynamic memory allocation to hold per-CPU memory types data (sized to
mp_maxid for UMA, and to mp_maxcpus for malloc to match the kernel).

That fixes libmemstat with arbitrary large MAXCPU values and therefore
eliminates MEMSTAT_ERROR_TOOMANYCPUS error type.

Reviewed by: jhb
Approved by: re (kib)

show more ...

Revert MAXCPU introduction. In userland it is always 1.

Noted by: marcel

Remove the redefinition of MEMSTAT_MAXCPU and just use MAXCPU for that.

Reviewed by: sbruno

Add a new column to the output of vmstat -z to indicate the number
of times the system was forced to sleep when requesting a new allocation.

Expand the debugger hook, db_show_uma, to display these r

Add a new column to the output of vmstat -z to indicate the number
of times the system was forced to sleep when requesting a new allocation.

Expand the debugger hook, db_show_uma, to display these results as well.

This has proven to be very useful in out of memory situations when
it is not known why systems have become sluggish or fail in odd ways.

Reviewed by: rwatson alc
Approved by: scottl (mentor) peter
Obtained from: Yahoo Inc.

show more ...

Correct two libmemstat(3) bugs:

- Move memory_type_list flushing logic from memstat_mtl_free() to
_memstat_mtl_empty(), a libmemstat-internal function that can
be called from other parts of the

Correct two libmemstat(3) bugs:

- Move memory_type_list flushing logic from memstat_mtl_free() to
_memstat_mtl_empty(), a libmemstat-internal function that can
be called from other parts of the library. Invoke
_memstat_mtl_empty() from memstat_mtl_free(), which also frees
the containing list structure.

Invoke _memstat_mtl_empty() instead of memstat_mtl_free() in
various error cases in memstat_malloc.c and memstat_uma.c, which
previously resulted in the list being freed prematurely.

- Reverse the order of updating the mt_kegfree and mt_free fields
of the memory_type in memstat_uma.c, otherwise keg free items
won't be counted properly for non-secondary zones.

MFC after: 3 days

show more ...

Introduce more formal error handling for libmemstat(3):

- Define a set of libmemstat(3) error constants, which are used by all
libmemstat(3) methods except for memstat_mtl_alloc(), which allocates

Introduce more formal error handling for libmemstat(3):

- Define a set of libmemstat(3) error constants, which are used by all
libmemstat(3) methods except for memstat_mtl_alloc(), which allocates
a memory type list and may return ENOMEM via errno.

- Define a per-memory_type_list current error value, which is set when a
call associated with a memory list fails. This requires wrapping a
structure around the queue(9) list head data structure, but this change
is not visible to libmemstat(3) consumers due to using access methods.

- Add a new accessor method, memstat_mtl_geterror() to retrieve the error
number.

- Consistently set the error number in a number of failure modes where
previously some combination of setting errno and printf'ing error
descriptions was used. libmemstat(3) will now no longer print to stdio
under any circumstances. Returns of NULL/-1 for errors remain the
same.

This avoids use of stdio, misuse of error numbers, and should make it
easier to program a libmemstat(3) consumer able to print useful error
messages. Currently, no error-to-string function is provided, as I'm
unsure how to address internationalization concerns.

MFC after: 1 day

show more ...

Prefix two non-static libmemstat(3) internal functions with '_' symbols, to
try and discourage use outside the library.

Remove duplicate declaration of memstat_mtl_free() from memstat_internal.h,
as

Prefix two non-static libmemstat(3) internal functions with '_' symbols, to
try and discourage use outside the library.

Remove duplicate declaration of memstat_mtl_free() from memstat_internal.h,
as it's not internal, and the memstat.h definition suffices.

show more ...

UMA supports "secondary" zones, in which a second zone can be layered
on top of a primary zone, sharing the same allocation "keg". When
reporting statistics for zones, do not report the free items i

UMA supports "secondary" zones, in which a second zone can be layered
on top of a primary zone, sharing the same allocation "keg". When
reporting statistics for zones, do not report the free items in the
keg as part of the free items in the zone, or those free items will
be reported more than once: for the primary zone, and then any
secondary zones off the primary zone. Separately record and maintain
a kegfree statistic, and export via memstat_get_kegfree(), which is
available for use if needed. Since items free'd back to the keg are
not fully initialized, and hence may not actually be available (since
secondary zone ctor-time initialization can fail), this makes some
amount of sense.

This change corrects a bug made visible in the libmemstat(3)
modifications to netstat: mbufs freed back to the keg from the
packet zone would be counted twice, resulting in negative values
being printed in the mbuf free count.

Some further refinement of reporting relating to secondary zones may
still be required.

Reported by: ssouhlal
MFC after: 3 days

show more ...

Increase the number of caller memory storage slots from 2 or 4 to
MEMSTAT_MAXCALLER (8), and expose MEMSTAT_MAXCALLER via memstat.h so
that applications can check their assumptions about how many slo

Increase the number of caller memory storage slots from 2 or 4 to
MEMSTAT_MAXCALLER (8), and expose MEMSTAT_MAXCALLER via memstat.h so
that applications can check their assumptions about how many slots
are available.

Remove 'spare' memory storage in struct malloc_type, since we now
don't expose the data structure internals to applications and rely
on accessor methods, this approach to ABI stability isn't required.

MFC after: 7 days

show more ...

Add libmemstat(3), a library for use by debugging and monitoring
applications in tracking kernel memory statistics. It provides an
abstracted interface to uma(9) and malloc(9) statistics, wrapped
ar

Add libmemstat(3), a library for use by debugging and monitoring
applications in tracking kernel memory statistics. It provides an
abstracted interface to uma(9) and malloc(9) statistics, wrapped
around the recently added binary stream sysctls for the allocators.

Using this interface, it is easy to build monitoring tools, query
specific memory types for usage information, etc. Facilities are
provided for binding caller-provided data to memory types,
incremental updates of memory types, and queries that span multiple
allocators.

Support for additional allocators is (relatively) easy to add.

The API for libmemstat(3) will probably change some over time as
consumers are written, and requirements evolve. It is written to
avoid encoding ABIs for data structure layout into consuming
applications for this reason.

MFC after: 1 week

show more ...

Correct two libmemstat(3) bugs:

- Move memory_type_list flushing logic from memstat_mtl_free() to
_memstat_mtl_empty(), a libmemstat-internal function that can
be called from other parts of the

Correct two libmemstat(3) bugs:

- Move memory_type_list flushing logic from memstat_mtl_free() to
_memstat_mtl_empty(), a libmemstat-internal function that can
be called from other parts of the library. Invoke
_memstat_mtl_empty() from memstat_mtl_free(), which also frees
the containing list structure.

Invoke _memstat_mtl_empty() instead of memstat_mtl_free() in
various error cases in memstat_malloc.c and memstat_uma.c, which
previously resulted in the list being freed prematurely.

- Reverse the order of updating the mt_kegfree and mt_free fields
of the memory_type in memstat_uma.c, otherwise keg free items
won't be counted properly for non-secondary zones.

MFC after: 3 days

show more ...

Introduce more formal error handling for libmemstat(3):

- Define a set of libmemstat(3) error constants, which are used by all
libmemstat(3) methods except for memstat_mtl_alloc(), which allocates

Introduce more formal error handling for libmemstat(3):

- Define a set of libmemstat(3) error constants, which are used by all
libmemstat(3) methods except for memstat_mtl_alloc(), which allocates
a memory type list and may return ENOMEM via errno.

- Define a per-memory_type_list current error value, which is set when a
call associated with a memory list fails. This requires wrapping a
structure around the queue(9) list head data structure, but this change
is not visible to libmemstat(3) consumers due to using access methods.

- Add a new accessor method, memstat_mtl_geterror() to retrieve the error
number.

- Consistently set the error number in a number of failure modes where
previously some combination of setting errno and printf'ing error
descriptions was used. libmemstat(3) will now no longer print to stdio
under any circumstances. Returns of NULL/-1 for errors remain the
same.

This avoids use of stdio, misuse of error numbers, and should make it
easier to program a libmemstat(3) consumer able to print useful error
messages. Currently, no error-to-string function is provided, as I'm
unsure how to address internationalization concerns.

MFC after: 1 day

show more ...

Prefix two non-static libmemstat(3) internal functions with '_' symbols, to
try and discourage use outside the library.

Remove duplicate declaration of memstat_mtl_free() from memstat_internal.h,
as

Prefix two non-static libmemstat(3) internal functions with '_' symbols, to
try and discourage use outside the library.

Remove duplicate declaration of memstat_mtl_free() from memstat_internal.h,
as it's not internal, and the memstat.h definition suffices.

show more ...