README.md
1# tests/common - C and C++ symbol visibility test program
2
3## Overview
4
5This directory contains the source for the symbol visibility test program:
6
7- `symbol_test.py` - test driver for C and C++ (supports parallel compilation)
8- `test_parse_env_cfg.py` - unit test for environment config parser
9- `test_parse_sym_cfg.py` - unit test for symbols config parser
10- `test_gen_probe.py` - unit test for test/probe program generation
11
12The `symbol_test.py` program:
13- reads an `environment config` file
14- reads a `symbols config` file
15- runs short compiler jobs for each combination of environement and symbol,
16 verifying that it exists or does not exist as indicated by the data from
17 the `symbols config` file.
18- generates include-only tests where needed to ensure the primary header has
19 at least one expected-success compilation in each environment selected for
20 positive coverage.
21
22The first header named by a symbols configuration is its primary header. The
23first header of later entries is expected to be the same. Additional headers
24after the first one in an entry are supporting headers and do not affect
25positive coverage. Multiple symbols configuration files may be supplied;
26they are effectively concatenated and must have the same primary header.
27
28The config file format is documented in `../cfg/README`,
29`../cfg/c-symbols/README`, and `../cfg/cxx-symbols/README`.
30
31## Program Operation
32
33- Compiler discovery: tries `g++` then `clang++`, or accepts `-c compiler`.
34- C++ include path setup: queries the compiler for its internal include
35 directory and constructs the `-isystem` paths needed to find C++ standard
36 headers without picking up GCC's fixincludes copies of system headers.
37
38## Probe program generation
39
40For each test entry, the program generates a minimal C or C++ program.
41For notes on "probe" program generation, see `test_gen_probe.py`
42
43#### `func` - function call test
44
45Config line:
46```
47func | log | double | double | math.h | C99+
48```
49Generated program (C):
50```c
51#include <math.h>
52double
53test_func(double a0)
54{
55 return log(a0);
56}
57```
58
59In the C++ program, the return value uses a `RESULT()` macro that selects
60brace-initialisation (C++11 and later) or plain assignment (C++98):
61```cpp
62#if __cplusplus >= 201103L
63#define RESULT(v) result{v}
64#else
65#define RESULT(v) result = (v)
66#endif
67double
68test_func(double a0)
69{
70 double RESULT(log(a0));
71 return result;
72}
73```
74Brace-initialisation prohibits narrowing conversions, so a missing `float`
75or `long double` overload that would silently promote through `double` is
76caught as a compile error rather than a silent pass. C++98 uses plain
77assignment since brace-init is a C++11 extension.
78Function pointer return types use plain `return` in the program since
79brace-init does not compose with declarator syntax.
80
81#### `type` - type existence test
82
83Config line:
84```
85type | size_t | stddef.h | C11+
86```
87Generated program:
88```c
89#include <stddef.h>
90size_t test_type;
91```
92
93#### `value` - constant/variable access test
94
95Config line:
96```
97value | M_PI | double | math.h | C99+
98```
99Generated program:
100```c
101#include <math.h>
102double test_value;
103void
104test_func(void)
105{
106 test_value = M_PI;
107}
108```
109
110#### `define` - preprocessor macro test
111
112Config line:
113```
114define | INFINITY | | math.h | C99+
115```
116Generated program:
117```c
118#include <math.h>
119#if !defined(INFINITY)
120#error INFINITY is not defined or has the wrong value
121#endif
122```
123An optional value field checks strict equality:
124```
125define | FLT_RADIX | 2 | float.h | C99+
126```
127```c
128#include <float.h>
129#if !defined(FLT_RADIX) || FLT_RADIX != 2
130#error FLT_RADIX is not defined or has the wrong value
131#endif
132```
133
134## Command-line options
135
136### Usage
137
138 python3 symbol_test.py --lang c|c++ -m64|-m32 [options] env_cfg sym_cfg...
139
140Required arguments:
141
142 --lang c|c++ Language to test
143 -m64 | -m32 Target ABI
144
145Options:
146
147 -C Check compiler only, do not run tests
148 -f Force: continue after failures
149 -d Debug: print probe and compiler output on failures
150 -D Extra debug: also print the compiler command and probe
151 program for every test, not just failures (implies -d)
152 -c compiler Use the specified compiler instead of auto-detecting
153 -s sym Run only the test for the named symbol
154 -e ENV Run only tests for the named environment
155 -j N Number of parallel compile jobs (default: 4 or
156 the environment variable SYMBOL_TEST_JOBS)
157 --positive-coverage=NAME
158 Ensure that at least one expected-success test compiles the
159 primary header in every environment named by NAME. Additional
160 include-only tests are generated where needed. By default,
161 all declared environments require positive coverage.
162 -R ROOT Alternate root directory (e.g. a proto area) whose
163 ROOT/usr/include is tested instead of the default
164 (default: $HEADER_TEST_ROOT/usr/include if that
165 environment variable is set, else /usr/include)
166
167When running tests by hand (not via the `setup` script), you may
168pass `-f` to see all failures instead of stopping on errors.
169
170## Extending the tests
171
172To add tests for a new header:
173
1741. Add a new `.cfg` file in `../cfg/c-symbols/` or `../cfg/cxx-symbols/`.
1752. Add a wrapper script (hardlink or copy of `setup`) in the corresponding
176 `../c-symbols/` or `../cxx-symbols/` subdirectory, named after the cfg file
177 without `.cfg`.
1783. Add the new files to the `../cfg/Makefile` and the IPS manifest.
179
180To add a new compilation environment or group, edit
181`../cfg/c-symbols-env.cfg` or `../cfg/cxx-symbols-env.cfg`.
182
183## Developer Notes
184
185Unit tests for the Python components (`symbol_test.py`) live alongside the
186source in this directory. They use canned string input and require no
187external files or build products.
188
189Run individual test modules from `tests/common/`:
190
191```
192python3 test_parse_env_cfg.py -v
193python3 test_parse_sym_cfg.py -v
194python3 test_gen_probe.py -v
195```
196
197Or run all unit tests at once from the repository root:
198
199```
200python3 -m unittest discover -s usr/src/test/header-tests/tests/common -p 'test_*.py' -v
201```
202