xref: /titanic_41/usr/src/lib/libast/common/man/vmalloc.3 (revision 3e14f97f673e8a630f076077de35afdd43dc1587)
.fp 5 CW \f5\\$1 ..
VMALLOC 3 "1 May 1998"
NAME
vmalloc - virtual memory allocation
SYNOPSIS
.MW "#include <vmalloc.h>"
Regions
.MW "Vmalloc_t* vmopen(Vmdisc_t* disc, Vmethod_t* meth, int flags);"
.MW "int vmclose(Vmalloc_t*);"
.MW "int vmclear(Vmalloc_t*);"
.MW "int vmcompact(Vmalloc_t* region);"
.MW "int vmset(Vmalloc_t* region, int flags, int type);"
.MW "Vmalloc_t* Vmheap;"
.MW "Vmdisc_t* vmdisc(Vmalloc_t* region, Vmdisc_t* disc);"
.MW "Vmalloc_t* vmmopen(char* file, Void_t* base, size_t round);"
.MW "Void_t* vmmset(Vmalloc_t* vm, int key, Void_t* data, int set);"
"Allocation functions"
.MW "Void_t* vmalloc(Vmalloc_t* region, size_t size);"
.MW "Void_t* vmalign(Vmalloc_t* region, size_t size, size_t align);"
.MW "Void_t* vmresize(Vmalloc_t* region, Void_t* addr, size_t size, int type);"
.MW "int vmfree(Vmalloc_t* region, Void_t* addr);"
.MW "Void_t* vmnewof(Vmalloc_t* region, Void_t* addr, type, size_t n, size_t x);"
.MW "Void_t* vmoldof(Vmalloc_t* region, Void_t* addr, type, size_t n, size_t x);"
.MW "Void_t* vmgetmem(Vmalloc_t* region, Void_t* addr, size_t size);"
"Debugging"
.MW "int vmdebug(int);"
.MW "int vmdbcheck(Vmalloc_t* vm);"
.MW "int vmdbwatch(Void_t* addr);"
.MW "static void vmdbwarn(Vmalloc_t*, char* mesg, int n);"
"Profiling"
.MW "void vmprofile(Vmalloc_t* vm, int fd);"
"Information and statistics"
.MW "Vmalloc_t* vmregion(Void_t* addr);"
.MW "Void_t* vmsegment(Vmalloc_t* region, Void_t* addr);"
.MW "int vmwalk(Vmalloc_t* region, int(*walkf)(Vmalloc_t*, Void_t*, size_t, size_t, Vmdisc_t*, Void_t*), Void_t* handle);"
.MW "long vmaddr(Vmalloc_t* region, Void_t* addr);"
.MW "long vmsize(Vmalloc_t* region, Void_t* addr);"
.MW "int vmstat(Vmalloc_t* vm, Vmstat_t* statb);"
.MW "int vmtrace(int fd);"
.MW "int vmtrbusy(Vmalloc_t* vm);"
"Malloc-compatible functions"
.MW "Void_t* malloc(size_t size);"
.MW "Void_t* realloc(Void_t* addr, size_t size);"
.MW "Void_t* calloc(size_t n_obj, size_t s_obj);"
.MW "int cfree(Void_t* addr);"
.MW "void free(Void_t* addr);"
.MW "Void_t* memalign(size_t align, size_t size);"
.MW "Void_t* valloc(size_t size);"
DESCRIPTION
These functions for dynamic storage allocation work in regions of memory. Each region has an allocation method for parceling out blocks of storage and a memory discipline for obtaining raw space. Automatic locking prevents interference by reentrant access to a region.

Pointers to space have type \f5Void_t* where \f5Void_t is \f5#defined as \f5void if possible, otherwise \f5char. Space is counted in type \f5size_t.

Regions
Regions have type \f5Vmalloc_t. Two predefined regions are pointed to by:

.MW Vmheap A general-purpose region, with best-fit allocation, and Unix memory discipline \f5Vmdcsbrk.

These functions manipulate regions:

vmopen creates a region with memory discipline disc, allocation method meth, and a setting for control flags. It returns a pointer to the region on success and \f5NULL on failure. The flags, represented by bit values or-ed together, are:

.MW VM_TRUST Disable locking and consistency checks, except under method \f5Vmdebug.

.MW VM_TRACE Place tracing messages for each allocation event on the tracing file established by vmtrace. This turns off \f5VM_TRUST.

\f5VM_DBCHECK, \f5VM_DBABORT

See Debugging below.

vmclose closes a region and releases all associated memory according to the region's discipline. The first segment obtained from the discipline's \f5memoryf function (see `Disciplines' below) will be the last released. vmclose returns -1 on failure and a non-negative value otherwise.

vmclear frees all allocated blocks in region regardless of methods. It returns -1 on failure and a non-negative value otherwise.

vmcompact releases as much of a region's free space to its discipline's \f5memoryf function as possible. It returns a nonnegative value on success and -1 on failure.

vmset adjusts and queries a region's flags. The indicated flags are turned on if type is nonzero, off if zero. vmset returns the previous value of all flags. Thus, \f5vmset(region,0,0) queries the flags without changing them. In addition to the settable flags, one of \f5VM_MTBEST, \f5VM_MTDEBUG, \f5VM_MTPROFILE, \f5VM_MTPOOL, or \f5VM_MTLAST is returned to indicate the method used in creating the region.

vmdisc changes the discipline of region to the given new discipline disc if disc is not \f5NULL and its \f5memoryf function is the same as the current discipline. If the current discipline has an \f5exceptf function, it will be called with event \f5VM_DISC. This function always returns the current discipline.

vmmopen creates a region to allocate memory given by the system call mmap(2). The given file is the backing store for the map. If base is not \f5NULL, it is the address to map the data to. The round argument asserts that the size of the region is always a multiple of this value. However, note that vmmopen has an internally defined minimum round value (typically 64K) which may be used if the given value is too small.

vmmset sets a data value associated with a key if \f5set is non-zero. In this case, it returns the current data (before setting) if key is previously set; otherwise, it returns the new data value. If the argument set is zero, the call returns the current data value associated with key, if any.

"Allocation functions"
vmalloc returns a pointer to a block of the requested size in a region, aligned to the strictest alignment that is suitable for the needs of any basic data type. It returns \f5NULL on failure.

vmalign works like vmalloc, but returns a block aligned to a common multiple of align and the strictest alignment.

vmresize attempts to change the length of the block pointed to by addr to the specified size. If that is impossible and type has at least one of \f5VM_RSMOVE and \f5VM_RSCOPY, a new block is allocated and the old block is freed. The bit \f5VM_RSCOPY also causes the new block to be initialized with as much of the old contents as will fit. When a resized block gets larger, the new space will be cleared if type has the bit \f5VM_RSZERO. vmresize returns a pointer to the final block, or \f5NULL on failure. If addr is \f5NULL, vmresize behaves like vmalloc; otherwise, if size is 0, it behaves like vmfree.

vmfree makes the currently allocated block pointed to by addr available for future allocations in its region. If addr is \f5NULL, vmfree does nothing. It returns -1 on error, and nonnegative otherwise.

vmnewof is a macro function that attempts to change the length of the block pointed to by addr to the size \f5n*sizeof(type)+x. If the block is moved, new space will be initialized with as much of the old content as will fit. Additional space will be set to zero.

vmoldof is similar to vmnewof but it neither copies data nor clears space.

vmgetmem provides a handy function to creat/close regions and allocate/free memory based on chunks of memory obtained from the heap region Vmheap.

.MW "vmgetmem(0,0,0)" This call opens a new region.

.MW "vmgetmem(region, 0, 0)" This call closes the given \f5region.

.MW "vmgetmem(region,0,n)" This call allocates a block of length \f5n and clears it to zeros.

.MW "vmgetmem(region,p,0)" This call frees the block \f5p.

.MW "vmgetmem(region,p,n)" This call resizes the block \f5p to length \f5n and clears the new memory to zeros if the block grows. The block may be moved as deemed necessary by the allocator.

"Memory disciplines"
Memory disciplines have type \f5Vmdisc_t, a structure with these members:
.MW "Void_t* (*memoryf)(Vmalloc_t *region, Void_t* obj,"
.MW "size_t csz, size_t nsz, Vmdisc_t *disc);"
.MW "int (*exceptf)(Vmalloc_t *region, int type, Void_t* obj, Vmdisc_t *disc);"
.MW "int round;"

.MW round If this value is positive, all size arguments to the \f5memoryf function will be multiples of it.

.MW memoryf Points to a function to get or release segments of space for the region.

.MW exceptf If this pointer is not \f5NULL, the function it points to is called to announce events in a region.

There are two standard disciplines. In both, \f5round is 0, and \f5exceptf is \f5NULL.

.MW Vmdcsbrk A discipline whose \f5memoryf function gets space from sbrk(2).

.ns

.MW Vmdcheap A discipline whose \f5memoryf function gets space from the region \f5Vmheap. A region with \f5Vmdcheap discipline and \f5Vmlast allocation is good for building throwaway data structures.

A memoryf function returns a pointer to a memory segment on success, and \f5NULL on failure. If csz is 0, the function returns a new segment of size nsz. Otherwise, the function attempts to change the length of the segment pointed to by addr from csz to nsz. If this is successful, \f5memoryf should return addr (even if nsz is 0).

An exceptf function is called for events identified by type, which is coded thus:

.MW VM_OPEN A new region is being opened. If exceptf returns a zero value, the region opening proceeds normally. A negative return value causes vmopen to terminate with failure. A positive return value indicates that the new region is to manipulate memory already initialized by a previous vmopen call (perhaps by a different process on persistent or shared memory). In this case, the argument \f5(Void_t**)obj should return the initial segment (which is of type \f5(Void_t*)). vmopen will return failure if this segment is not returned or if it has not been properly initialized.

.MW VM_CLOSE The region is being closed. The return value of \f5exceptf is significant as follows. If negative, vmclose immediately returns with failure. If zero, vmclose proceeds normally by calling \f5memoryf to free all allocated memory segments and also freeing the region itself. Finally, if positive, vmclose will only free the region without deallocating the allocated segments.

.MW VM_NOMEM An attempt to extend the region by the amount \f5(size_t)obj failed. The region is unlocked, so the exceptf function may free blocks. If the function returns a positive value the memory request will be repeated. If zero, the allocation method will again invoke memoryf to get space. If negative, the allocation request will fail.

.MW VM_BADADDR Address obj, given to vmfree or vmresize, does not point to an allocated block from the region. The respective call will fail.

.MW VM_ALLOC Announce that a memory allocation request is finished and returning.

.MW VM_FREE Announce that a memory freeing request is finished and returning.

.MW VM_RESIZE Announce that a memory resizing request is finished and returning.

"Allocation methods"
There are five methods, of type \f5Vmethod_t*:

.MW Vmbest An approximately best-fit allocation strategy.

.MW Vmlast A strategy for building structures that are only deleted in whole. Only the latest allocated block can be freed. This means that as soon as a block \f5a is allocated, vmfree calls on blocks other than \c5a are ignored.

.MW Vmpool A strategy for blocks of one size, set by the first vmalloc call after vmopen or vmclear.

.MW Vmdebug An allocation strategy with extra-stringent checking and locking regardless of the \f5VM_TRUST flag. It is useful for finding misuses of dynamically allocated memory, such as writing beyond the boundary of a block, or freeing a block twice.

.MW Vmprofile An allocation method that records and prints summaries of memory usage.

Debugging
The method \f5Vmdebug is used to debug common memory violation problems. When a problem is found, a warning message is written to file descriptor 2 (standard error). In addition, if flag \f5VM_DBABORT is on, the program is terminated by calling abort(2). Each message is a line of self-explanatory fields separated by colons. The optional flag \f5-DVMFL, if used during compilation, enables recording of file names and line numbers. The following functions work with method \f5Vmdebug.

vmdebug resets the file descriptor to write out warnings to the given argument. By default, this file descriptor is 2, the standard error. vmdebug returns the previous file descriptor.

vmdbcheck checks a region using \f5Vmdebug or \f5Vmbest for integrity. If \f5Vmdebug, this also checks for block overwriting errors. On errors, vmdbwarn is called. If flag \f5VM_DBCHECK is on, vmdbcheck is called at each invocation of vmalloc, vmfree, or vmresize.

vmdbwatch causes address addr to be watched, and reported whenever met in vmalloc, vmresize or vmfree. The watch list has finite size and if it becomes full, watches will be removed in a first-in-first-out fashion. If addr is \f5NULL, all current watches are canceled. vmdbwatch returns the watch bumped out due to an insertion into a full list or \f5NULL otherwise.

vmdbwarn is an internal function that processes warning messages for discovered errors. It can't be called from outside the vmalloc package, but is a good place to plant debugger traps because control goes there at every trouble.

"Profiling"
The method \f5Vmprofile is used to profile memory usage. Profiling data are maintained in private memory of a process so \f5Vmprofile should be avoided from regions manipulating persistent or shared memory. The optional flag \f5-DVMFL, if used during compilation, enables recording of file names and line numbers.

vmprofile prints memory usage summary. The summary is restricted to region vm if vm is not \f5NULL; otherwise, it is for all regions created with \f5Vmprofile. Summary records are written to file descriptor fd as lines with colon-separated fields. Here are some of the fields:

n_alloc,n_free: Number of allocation and free calls respectively. Note that a resize operation is coded as a free and an allocation.

s_alloc,s_free: Total amounts allocated and freed. The difference between these numbers is the amount of space not yet freed.

max_busy, extent: These fields are only with the summary record for region. They show the maximum busy space at any time and the extent of the region.

"Information and statistics"
vmregion returns the region to which the block pointed to by addr belongs. This works only in regions that allocate with \f5Vmbest, \f5Vmdebug or \f5Vmprofile. If multiple regions manipulate the same segment of memory, vmregion returns the region that causes the creation that memory segment.

vmsegment finds if some segment of memory in region contains the address addr. It returns the address of a found segment or \f5NULL if none found.

vmwalk walks all segments in region or if region is \f5NULL, all segments in all regions. At each segment, (*walkf)(vm,addr,size,segno,disc,handle) is called where vm is the region, addr is the segment, size is the size of the segment, segno is the segment ordinal counting from 0, disc is the region's discipline, and handle is the user supplied vmwalk handle. If walkf returns a negative value, the walk stops and returns the same value. On success, vmwalk returns 0; otherwise, it returns -1.

vmaddr checks whether addr points to an address within some allocated block of the given region. If not, it returns -1. If so, it returns the offset from the beginning of the block. The function does not work for a \f5Vmlast region except on the latest allocated block.

vmsize returns the size of the allocated block pointed to by addr. It returns -1 if addr does not point to a valid block in the region. Sizes may be padded beyond that requested; in particular no block has size 0. The function does not work for a \f5Vmlast region except on the latest allocated block.

vmstat gathers statistics on the given region and returns that information in the \f5Vmstat_t structure pointed to by statb. A \f5Vmstat_t structure has at least these members:

.MW "int n_busy; /* number of busy blocks */
.MW "int n_free; /* number of free blocks */
.MW "size_t s_busy; /* total busy space */
.MW "size_t s_free; /* total free space */
.MW "size_t m_busy; /* maximum size of busy block */
.MW "size_t m_free; /* maximum size of free block */
.MW "int n_seg; /* number of segments in region */
.MW "size_t extent; /* total size of the region */

Bookeeping overhead is counted in \f5extent, but not in \f5s_busy or \f5s_free.

vmtrace establishes file descriptor fd as the trace file and returns the previous value of the trace file descriptor. The trace descriptor is initially invalid. Output is sent to the trace file by successful allocation events when flag \f5VM_TRACE is on.

Tools for analyzing traces are described in mtreplay(1). The trace record for an allocation event is a line with colon-separated fields, four numbers and one string.

old Zero for a fresh allocation; the address argument for freeing and resizing.

new Zero for freeing; the address returned by allocation or resizing.

size The size argument for allocation or resizing; the size freed by freeing. Sizes may differ due to padding for alignment.

region The address of the affected region.

method A string that tells the region's method: \f5best, \f5last, \f5pool, \f5profile, or \f5debug.

vmtrbusy outputs a trace of all currently busy blocks in region \f5vm. This only works with the \f5Vmbest, \f5Vmdebug and \f5Vmprofile methods.

"Malloc-compatible functions"
Functions in this set provide the behaviors of malloc(3). The functions memalign and valloc allocate aligned blocks; valloc further restricts alignment to page boundaries.

The malloc functions are instrumented for run-time debugging, profiling and tracing. When these modes are enabled, time and space performance will be affected. For accurate reporting of files and line numbers, code should include \f5vmalloc.h and compile with either \f5-DVMFL or \f5-D_BLD_debug (defined by nmake(1) for \f5-g compilations). The VMALLOC_OPTIONS environment variable should be set before any memory allocation (e.g., before a process starts). Its value is a space-separated list of [no]name[=value] options that control runtime debugging/profiling and other modes:

abort If Vmregion==Vmdebug then VM_DBABORT is set, otherwise _BLD_debug enabled assertions abort() on failure.

check If Vmregion==Vmbest then the region is checked after every operation.

method =method Sets Vmregion=method if not already set. method (Vm prefix optional) may be one of:

best

debug

last

profile

mmap Prefer mmap() over brk() for region allocation.

period =n Sets Vmregion=Vmdebug if not already set. If Vmregion==Vmdebug then the region is checked every n operations.

profile =file Sets Vmregion=Vmprofile if not already set. If Vmregion==Vmprofile then profile info is printed to file.

region If Vmregion==Vmbest then region block frees verify that the block belongs to the region. start =n Sets Vmregion=Vmdebug if not already set. If Vmregion==Vmdebug then region checking starts after n operations.

trace =file Operation trace info is printed to file.

warn =file Sets Vmregion=Vmdebug if not already set. If Vmregion==Vmdebug then warnings are printed to file.

watch =address Sets Vmregion=Vmdebug if not already set. If Vmregion==Vmdebug then operations on address are traced.

Output files are created if they don't exist. & n and /dev/fd/ n name the file descriptor n which must be open for writing. The pattern %p in a file name is replaced by the current process ID. VMALLOC_OPTIONS combines the features of these previously used env vars { "VMDEBUG VMETHOD VMPROFILE VMTRACE" }. For compatibility, if VMALLOC_OPTIONS is not defined, then { "VMDEBUG VMETHOD VMPROFILE VMTRACE" } will be checked -- this will be dropped in a future release.

RECENT CHANGES
\f5Vmlast: allocated blocks are now allowed to be resized (1998-09). VMALLOC_OPTIONS now controls all runtime settings (2010-01).
SEE ALSO
mtreplay(1), malloc(3).
AUTHOR
Kiem-Phong Vo, kpv@research.att.com