xref: /freebsd/contrib/atf/NEWS (revision c203bd70b5957f85616424b6fa374479372d06e3)
1c243e490SMarcel MoolenaarMajor changes between releases                  Automated Testing Framework
2c243e490SMarcel Moolenaar===========================================================================
3c243e490SMarcel Moolenaar
4*c203bd70SAlex RichardsonChanges in version 0.22
5*c203bd70SAlex Richardson***********************
6*c203bd70SAlex Richardson
7*c203bd70SAlex RichardsonSTILL UNDER DEVELOPMENT; NOT RELEASED YET.
8*c203bd70SAlex RichardsonDON'T FORGET TO BUMP THE -version-info PRE-RELEASE IF NECESSARY!
9*c203bd70SAlex Richardson
10*c203bd70SAlex Richardson* Issue #23: Fix double-free triggered by atf_map_insert in low memory
11*c203bd70SAlex Richardson  scenarios, caused by an overlook in the atf_list code.
12*c203bd70SAlex Richardson
13*c203bd70SAlex Richardson* Issue #29: Fixed various typos and formatting errors in manual pages.
14*c203bd70SAlex Richardson
15*c203bd70SAlex Richardson* Issue #31: Added require.progs metadata properties to the tests that
16*c203bd70SAlex Richardson  need a compiler to run.
17*c203bd70SAlex Richardson
18*c203bd70SAlex Richardson* Added the atf_check_not_equal function to atf-sh to check for
19*c203bd70SAlex Richardson  unequal values.
20*c203bd70SAlex Richardson
21*c203bd70SAlex Richardson
220677dfd1SJulio MerinoChanges in version 0.21
230677dfd1SJulio Merino***********************
240677dfd1SJulio Merino
250677dfd1SJulio MerinoReleased on October 23rd, 2014.
260677dfd1SJulio Merino
270677dfd1SJulio Merino* Restored the atf(7) manual page to serve as a reference to all the other
280677dfd1SJulio Merino  manual pages shipped by ATF.
290677dfd1SJulio Merino
300677dfd1SJulio Merino* Added the -s flag to atf-sh to support specifying the shell interpreter
310677dfd1SJulio Merino  to be used.
320677dfd1SJulio Merino
330677dfd1SJulio Merino* Removed ATF_WORKDIR.  The only remaining consumers have been converted to
340677dfd1SJulio Merino  use the standard TMPDIR environment variable.  As a benefit, and because
350677dfd1SJulio Merino  Kyua forces the TMPDIR to live within the test case's work directory,
360677dfd1SJulio Merino  any stale files left behind by ATF will be automatically cleaned up.
370677dfd1SJulio Merino
380677dfd1SJulio Merino* Documented the environment variables recognized by each component in the
390677dfd1SJulio Merino  relevant manual pages.  This information was lost with the atf-config(1)
400677dfd1SJulio Merino  removal.
410677dfd1SJulio Merino
420677dfd1SJulio Merino* Added a new "require.diskspace" metadata property to test cases so that
430677dfd1SJulio Merino  they can specify the minimum amount of disk space required for the test
440677dfd1SJulio Merino  to run.
450677dfd1SJulio Merino
460677dfd1SJulio Merino* Renamed the atf-{c,c++,sh}-api(3) manual pages to atf-{c,c++,sh}(3) for
470677dfd1SJulio Merino  discoverability purposes.  Symbolic links are provided for the time
480677dfd1SJulio Merino  being to still make the old names visible.
490677dfd1SJulio Merino
500677dfd1SJulio Merino* Issue #5: Recommend the (expected, actual) idiom for calls to the test
510677dfd1SJulio Merino  macros in the manual pages.
520677dfd1SJulio Merino
530677dfd1SJulio Merino* Issue #7: Stopped catching unhandled exceptions in atf-c++ tests.  This
540677dfd1SJulio Merino  propagates the crash to the caller, which in turn allows it to obtain
550677dfd1SJulio Merino  proper debugging information.  In particular, Kyua should now be able to
560677dfd1SJulio Merino  extract a stacktrace pinpointing the problem.
570677dfd1SJulio Merino
580677dfd1SJulio Merino* Issue #8: Fixed atf-c/macros_test:use test failures spotted by the clang
590677dfd1SJulio Merino  that ships with FreeBSD 11.0-CURRENT.
600677dfd1SJulio Merino
610677dfd1SJulio Merino* Issue #12: Improved documentation of atf-sh(3) and atf-check(1) by better
620677dfd1SJulio Merino  explaining how they relate to each other.
630677dfd1SJulio Merino
640677dfd1SJulio Merino* Issue #14: Stopped setting 'set -e' in atf-sh.  This setting was
650677dfd1SJulio Merino  initially added as a way to enable a "strict" mode in the library and to
660677dfd1SJulio Merino  make test cases fail fast when they run unprotected commands.  However,
670677dfd1SJulio Merino  doing so in the library is surprising as the responsibility of enabling
680677dfd1SJulio Merino  'set -e' should be on the user's code.  Also, 'set -e' introduces
690677dfd1SJulio Merino  inconsistent behavior on subshells and users do not expect that.
700677dfd1SJulio Merino
710677dfd1SJulio Merino* Issue #15: Fixed atf_utils_{fork,wait} to support nested calls.
720677dfd1SJulio Merino
730677dfd1SJulio Merino* Issue #16: Fixed test failures (by removing a long-standing hack) on
740677dfd1SJulio Merino  systems that lack \e support in printf(1).
750677dfd1SJulio Merino
760677dfd1SJulio Merino* Issue #19: Removed stale references to atf-config and atf-run.
770677dfd1SJulio Merino
78c243e490SMarcel Moolenaar
791a61beb0SJulio MerinoChanges in version 0.20
801a61beb0SJulio Merino***********************
811a61beb0SJulio Merino
821a61beb0SJulio MerinoExperimental version released on February 7th, 2014.
831a61beb0SJulio Merino
841a61beb0SJulio MerinoThis is the first release without the code for the deprecated tools.  If
851a61beb0SJulio Merinoyou require such code, please fetch a copy of the 0.19 release and extract
861a61beb0SJulio Merinothe 'tools' directory for your own consumption.
871a61beb0SJulio Merino
881a61beb0SJulio Merino* Removed the deprecated tools.  This includes atf-config, atf-report,
891a61beb0SJulio Merino  atf-run and atf-version.
901a61beb0SJulio Merino
91dfdc97aeSCraig Rodrigues* Issue #8: Fixed atf-c/macros_test:use test failures spotted by the clang
92dfdc97aeSCraig Rodrigues  that ships with FreeBSD 11.0-CURRENT.
93dfdc97aeSCraig Rodrigues
941a61beb0SJulio Merino
951a61beb0SJulio MerinoChanges in version 0.19
961a61beb0SJulio Merino***********************
971a61beb0SJulio Merino
981a61beb0SJulio MerinoExperimental version released on February 7th, 2014.
991a61beb0SJulio Merino
1001a61beb0SJulio MerinoThis is the last release to bundle the code for the deprecated tools.
1011a61beb0SJulio MerinoThe next release will drop their code and will stop worrying about
1021a61beb0SJulio Merinobackwards compatibility between the ATF libraries and what the old tools
1031a61beb0SJulio Merinomay or may not support.
1041a61beb0SJulio Merino
1051a61beb0SJulio MerinoIf you still require the old tools for some reason, grab a copy of the
1061a61beb0SJulio Merino'tools' directory now.  The code in this directory is standalone and
1071a61beb0SJulio Merinodoes not depend on any internal details of atf-c++ any longer.
1081a61beb0SJulio Merino
1091a61beb0SJulio Merino* Various fixes and improvements to support running as part of the FreeBSD
1101a61beb0SJulio Merino  test suite.
1111a61beb0SJulio Merino
1121a61beb0SJulio Merino* Project hosting moved from Google Code (as a subproject of Kyua) to
1131a61beb0SJulio Merino  GitHub (as a first-class project).  The main reason for the change is
1141a61beb0SJulio Merino  the suppression of binary downloads in Google Code on Jan 15th, 2014.
1151a61beb0SJulio Merino  See https://github.com/jmmv/atf/
1161a61beb0SJulio Merino
1171a61beb0SJulio Merino* Removed builtin help from atf-sh(1) and atf-check(1) for simplicity
1181a61beb0SJulio Merino  reasons.  In other words, their -h option is gone.
1191a61beb0SJulio Merino
1201a61beb0SJulio Merino* Moved the code of the deprecated tools into a 'tools' directory and
1211a61beb0SJulio Merino  completely decoupled their code from the internals of atf-c++.  The
1221a61beb0SJulio Merino  reason for this is to painlessly allow a third-party to maintain a
1231a61beb0SJulio Merino  copy of these tools after we delete them because upcoming changes to
1241a61beb0SJulio Merino  atf-c++ would break the stale tools.
1251a61beb0SJulio Merino
1261a61beb0SJulio Merino
127a18eacbeSJulio MerinoChanges in version 0.18
128a18eacbeSJulio Merino***********************
129a18eacbeSJulio Merino
130a18eacbeSJulio MerinoExperimental version released on November 16th, 2013.
131a18eacbeSJulio Merino
132a18eacbeSJulio Merino* Issue 45: Added require.memory support in atf-run for FreeBSD.
133a18eacbeSJulio Merino
134a18eacbeSJulio Merino* Fixed an issue with the handling of cin with libc++.
135a18eacbeSJulio Merino
136a18eacbeSJulio Merino* Issue 64: Fixed various mandoc formatting warnings.
137a18eacbeSJulio Merino
138a18eacbeSJulio Merino* NetBSD PR bin/48284: Made atf-check flush its progress message to
139a18eacbeSJulio Merino  stdout so that an interrupted test case always shows the last message
140a18eacbeSJulio Merino  being executed.
141a18eacbeSJulio Merino
142a18eacbeSJulio Merino* NetBSD PR bin/48285: Fixed atf_check examples in atf-sh-api(3).
143a18eacbeSJulio Merino
144a18eacbeSJulio Merino
145a18eacbeSJulio MerinoChanges in version 0.17
146a18eacbeSJulio Merino***********************
147a18eacbeSJulio Merino
148a18eacbeSJulio MerinoExperimental version released on February 14th, 2013.
149a18eacbeSJulio Merino
150a18eacbeSJulio Merino* Added the atf_utils_cat_file, atf_utils_compare_file,
151a18eacbeSJulio Merino  atf_utils_copy_file, atf_utils_create_file, atf_utils_file_exists,
152a18eacbeSJulio Merino  atf_utils_fork, atf_utils_grep_file, atf_utils_grep_string,
153a18eacbeSJulio Merino  atf_utils_readline, atf_utils_redirect and atf_utils_wait utility
154a18eacbeSJulio Merino  functions to atf-c-api.  Documented the already-public
155a18eacbeSJulio Merino  atf_utils_free_charpp function.
156a18eacbeSJulio Merino
157a18eacbeSJulio Merino* Added the cat_file, compare_file, copy_file, create_file, file_exists,
158a18eacbeSJulio Merino  fork, grep_collection, grep_file, grep_string, redirect and wait
159a18eacbeSJulio Merino  functions to the atf::utils namespace of atf-c++-api.  These are
160a18eacbeSJulio Merino  wrappers around the same functions added to the atf-c-api library.
161a18eacbeSJulio Merino
162a18eacbeSJulio Merino* Added the ATF_CHECK_MATCH, ATF_CHECK_MATCH_MSG, ATF_REQUIRE_MATCH and
163a18eacbeSJulio Merino  ATF_REQUIRE_MATCH_MSG macros to atf-c to simplify the validation of a
164a18eacbeSJulio Merino  string against a regular expression.
165a18eacbeSJulio Merino
166a18eacbeSJulio Merino* Miscellaneous fixes for manpage typos and compilation problems with
167a18eacbeSJulio Merino  clang.
168a18eacbeSJulio Merino
169a18eacbeSJulio Merino* Added caching of the results of those configure tests that rely on
170a18eacbeSJulio Merino  executing a test program.  This should help crossbuild systems by
171a18eacbeSJulio Merino  providing a mechanism to pre-specify what the results should be.
172a18eacbeSJulio Merino
173a18eacbeSJulio Merino* PR bin/45690: Make atf-report convert any non-printable characters to
174a18eacbeSJulio Merino  a plain-text representation (matching their corresponding hexadecimal
175a18eacbeSJulio Merino  entities) in XML output files.  This is to prevent the output of test
176a18eacbeSJulio Merino  cases from breaking xsltproc later.
177a18eacbeSJulio Merino
178a18eacbeSJulio Merino
179c243e490SMarcel MoolenaarChanges in version 0.16
180c243e490SMarcel Moolenaar***********************
181c243e490SMarcel Moolenaar
182c243e490SMarcel MoolenaarExperimental version released on July 10th, 2012.
183c243e490SMarcel Moolenaar
184c243e490SMarcel Moolenaar* Added a --enable-tools flag to configure to request the build of the
185c243e490SMarcel Moolenaar  deprecated ATF tools, whose build is now disabled by default.  In order
186c243e490SMarcel Moolenaar  to continue running tests, you should migrate to Kyua instead of enabling
187c243e490SMarcel Moolenaar  the build of the deprecated tools.  The kyua-atf-compat package provides
188c243e490SMarcel Moolenaar  transitional compatibility versions of atf-run and atf-report built on
189c243e490SMarcel Moolenaar  top of Kyua.
190c243e490SMarcel Moolenaar
191c243e490SMarcel Moolenaar* Tweaked the ATF_TEST_CASE macro of atf-c++ so that the compiler can
192c243e490SMarcel Moolenaar  detect defined but unused test cases.
193c243e490SMarcel Moolenaar
194c243e490SMarcel Moolenaar* PR bin/45859: Fixed some XSLT bugs that resulted in the tc-time and
195c243e490SMarcel Moolenaar  tp-time XML tags leaking into the generated HTML file.  Also improved
196c243e490SMarcel Moolenaar  the CSS file slightly to correct alignment and color issues with the
197c243e490SMarcel Moolenaar  timestamps column.
198c243e490SMarcel Moolenaar
199c243e490SMarcel Moolenaar* Optimized atf-c++/macros.hpp so that GNU G++ consumes less memory during
200c243e490SMarcel Moolenaar  compilation with GNU G++.
201c243e490SMarcel Moolenaar
202c243e490SMarcel Moolenaar* Flipped the default to building shared libraries for atf-c and atf-c++,
203c243e490SMarcel Moolenaar  and started versioning them.  As a side-effect, this removes the
204c243e490SMarcel Moolenaar  --enable-unstable-shared flag from configure that appears to not work any
205c243e490SMarcel Moolenaar  more (under NetBSD).  Additionally, some distributions require the use of
206c243e490SMarcel Moolenaar  shared libraries for proper dependency tracking (e.g. Fedora), so it is
207c243e490SMarcel Moolenaar  better if we do the right versioning upstream.
208c243e490SMarcel Moolenaar
209c243e490SMarcel Moolenaar* Project hosting moved from an adhoc solution (custom web site and
210c243e490SMarcel Moolenaar  Monotone repository) to Google Code (standard wiki and Git).  ATF now
211c243e490SMarcel Moolenaar  lives in a subcomponent of the Kyua project.
212c243e490SMarcel Moolenaar
213c243e490SMarcel Moolenaar
214c243e490SMarcel MoolenaarChanges in version 0.15
215c243e490SMarcel Moolenaar***********************
216c243e490SMarcel Moolenaar
217c243e490SMarcel MoolenaarExperimental version released on January 16th, 2012.
218c243e490SMarcel Moolenaar
219c243e490SMarcel Moolenaar* Respect stdin in atf-check.  The previous release silenced stdin for any
220c243e490SMarcel Moolenaar  processes spawned by atf, not only test programs, which caused breakage
221c243e490SMarcel Moolenaar  in tests that pipe data through atf-check.
222c243e490SMarcel Moolenaar
223c243e490SMarcel Moolenaar* Performance improvements to atf-sh.
224c243e490SMarcel Moolenaar
225c243e490SMarcel Moolenaar* Enabled detection of unused parameters and variables in the code and
226c243e490SMarcel Moolenaar  fixed all warnings.
227c243e490SMarcel Moolenaar
228c243e490SMarcel Moolenaar* Changed the behavior of "developer mode".  Compiler warnings are now
229c243e490SMarcel Moolenaar  enabled unconditionally regardless of whether we are in developer mode or
230c243e490SMarcel Moolenaar  not; developer mode is now only used to perform strict warning checks and
231c243e490SMarcel Moolenaar  to enable assertions.  Additionally, developer mode is now only
232c243e490SMarcel Moolenaar  automatically enabled when building from the repository, not for formal
233c243e490SMarcel Moolenaar  releases.
234c243e490SMarcel Moolenaar
235c243e490SMarcel Moolenaar* Added new Autoconf M4 macros (ATF_ARG_WITH, ATF_CHECK_C and
236c243e490SMarcel Moolenaar  ATF_CHECK_CXX) to provide a consistent way of defining a --with-arg flag
237c243e490SMarcel Moolenaar  in configure scripts and detecting the presence of any of the ATF
238c243e490SMarcel Moolenaar  bindings.  Note that ATF_CHECK_SH was already introduced in 0.14, but it
239c243e490SMarcel Moolenaar  has now been modified to also honor --with-atf if instantiated.
240c243e490SMarcel Moolenaar
241c243e490SMarcel Moolenaar* Added timing support to atf-run / atf-report.
242c243e490SMarcel Moolenaar
243c243e490SMarcel Moolenaar* Added support for a 'require.memory' property, to specify the minimum
244c243e490SMarcel Moolenaar  amount of physical memory needed by the test case to yield valid results.
245c243e490SMarcel Moolenaar
246c243e490SMarcel Moolenaar* PR bin/45690: Force an ISO-8859-1 encoding in the XML files generated by
247c243e490SMarcel Moolenaar  atf-report so that invalid data in the output of test cases does not
248c243e490SMarcel Moolenaar  mangle our report.
249c243e490SMarcel Moolenaar
250c243e490SMarcel Moolenaar
251c243e490SMarcel MoolenaarChanges in version 0.14
252c243e490SMarcel Moolenaar***********************
253c243e490SMarcel Moolenaar
254c243e490SMarcel MoolenaarExperimental version released on June 14th, 2011.
255c243e490SMarcel Moolenaar
256c243e490SMarcel Moolenaar* Added a pkg-config file for atf-sh and an aclocal file to ease the
257c243e490SMarcel Moolenaar  detection of atf-sh from autoconf scripts.
258c243e490SMarcel Moolenaar
259c243e490SMarcel Moolenaar* Made the default test case body defined by atf_sh fail.  This is to
260c243e490SMarcel Moolenaar  ensure that test cases are properly defined in test programs and helps
261c243e490SMarcel Moolenaar  in catching typos in the names of the body functions.
262c243e490SMarcel Moolenaar
263c243e490SMarcel Moolenaar* PR bin/44882: Made atf-run connect the stdin of test cases to /dev/zero.
264c243e490SMarcel Moolenaar  This provides more consistent results with "normal" execution (in
265c243e490SMarcel Moolenaar  particular, when tests are executed detached from a terminal).
266c243e490SMarcel Moolenaar
267c243e490SMarcel Moolenaar* Made atf-run hardcode TZ=UTC for test cases.  It used to undefine TZ, but
268c243e490SMarcel Moolenaar  that does not take into account that libc determines the current timezone
269c243e490SMarcel Moolenaar  from a configuration file.
270c243e490SMarcel Moolenaar
271c243e490SMarcel Moolenaar* All test programs will now print a warning when they are not run through
272c243e490SMarcel Moolenaar  atf-run(1) stating that this is unsupported and may deliver incorrect
273c243e490SMarcel Moolenaar  results.
274c243e490SMarcel Moolenaar
275c243e490SMarcel Moolenaar* Added support for the 'require.files' test-case property.  This allows
276c243e490SMarcel Moolenaar  test cases to specify installed files that must be present for the test
277c243e490SMarcel Moolenaar  case to run.
278c243e490SMarcel Moolenaar
279c243e490SMarcel Moolenaar
280c243e490SMarcel MoolenaarChanges in version 0.13
281c243e490SMarcel Moolenaar***********************
282c243e490SMarcel Moolenaar
283c243e490SMarcel MoolenaarExperimental version released on March 31st, 2011.
284c243e490SMarcel Moolenaar
285c243e490SMarcel MoolenaarThis is the first release after the creation of the Kyua project, a more
286c243e490SMarcel Moolenaarmodular and reliable replacement for ATF.  From now on, ATF will change to
287c243e490SMarcel Moolenaaraccomodate the transition to this new codebase, but ATF will still continue
288c243e490SMarcel Moolenaarto see development in the short/medium term.  Check out the project page at
289c243e490SMarcel Moolenaarhttp://code.google.com/p/kyua/ for more details.
290c243e490SMarcel Moolenaar
291c243e490SMarcel MoolenaarThe changes in this release are:
292c243e490SMarcel Moolenaar
293c243e490SMarcel Moolenaar* Added support to run the tests with the Kyua runtime engine (kyua-cli), a
294c243e490SMarcel Moolenaar  new package that aims to replace atf-run and atf-report.  The ATF tests
295c243e490SMarcel Moolenaar  can be run with the new system by issuing a 'make installcheck-kyua' from
296c243e490SMarcel Moolenaar  the top-level directory of the project (assuming the 'kyua' binary is
297c243e490SMarcel Moolenaar  available during the configuration stage of ATF).
298c243e490SMarcel Moolenaar
299c243e490SMarcel Moolenaar* atf-run and atf-report are now in maintenance mode (but *not* deprecated
300c243e490SMarcel Moolenaar  yet!).  Kyua already implements a new, much more reliable runtime engine
301c243e490SMarcel Moolenaar  that provides similar features to these tools.  That said, it is not
302c243e490SMarcel Moolenaar  complete yet so all development efforts should go towards it.
303c243e490SMarcel Moolenaar
304c243e490SMarcel Moolenaar* If GDB is installed, atf-run dumps the stack trace of crashing test
305c243e490SMarcel Moolenaar  programs in an attempt to aid debugging.  Contributed by Antti Kantee.
306c243e490SMarcel Moolenaar
307c243e490SMarcel Moolenaar* Reverted default timeout change in previous release and reset its value
308c243e490SMarcel Moolenaar  to 5 minutes.  This was causing several issues, specially when running
309c243e490SMarcel Moolenaar  the existing NetBSD test suite in qemu.
310c243e490SMarcel Moolenaar
311c243e490SMarcel Moolenaar* Fixed the 'match' output checker in atf-check to properly validate the
312c243e490SMarcel Moolenaar  last line of a file even if it does not have a newline.
313c243e490SMarcel Moolenaar
314c243e490SMarcel Moolenaar* Added the ATF_REQUIRE_IN and ATF_REQUIRE_NOT_IN macros to atf-c++ to
315c243e490SMarcel Moolenaar  check for the presence (or lack thereof) of an element in a collection.
316c243e490SMarcel Moolenaar
317c243e490SMarcel Moolenaar* PR bin/44176: Fixed a race condition in atf-run that would crash atf-run
318c243e490SMarcel Moolenaar  when the cleanup of a test case triggered asynchronous modifications to
319c243e490SMarcel Moolenaar  its work directory (e.g. killing a daemon process that cleans up a pid
320c243e490SMarcel Moolenaar  file in the work directory).
321c243e490SMarcel Moolenaar
322c243e490SMarcel Moolenaar* PR bin/44301: Fixed the sample XSLT file to report bogus test programs
323c243e490SMarcel Moolenaar  instead of just listing them as having 0 test cases.
324c243e490SMarcel Moolenaar
325c243e490SMarcel Moolenaar
326c243e490SMarcel MoolenaarChanges in version 0.12
327c243e490SMarcel Moolenaar***********************
328c243e490SMarcel Moolenaar
329c243e490SMarcel MoolenaarExperimental version released on November 7th, 2010.
330c243e490SMarcel Moolenaar
331c243e490SMarcel Moolenaar* Added the ATF_REQUIRE_THROW_RE to atf-c++, which is the same as
332c243e490SMarcel Moolenaar  ATF_REQUIRE_THROW but allows checking for the validity of the exception's
333c243e490SMarcel Moolenaar  error message by means of a regular expression.
334c243e490SMarcel Moolenaar
335c243e490SMarcel Moolenaar* Added the ATF_REQUIRE_MATCH to atf-c++, which allows checking for a
336c243e490SMarcel Moolenaar  regular expression match in a string.
337c243e490SMarcel Moolenaar
338c243e490SMarcel Moolenaar* Changed the default timeout for test cases from 5 minutes to 30 seconds.
339c243e490SMarcel Moolenaar  30 seconds is long enough for virtually all tests to complete, and 5
340c243e490SMarcel Moolenaar  minutes is a way too long pause in a test suite where a single test case
341c243e490SMarcel Moolenaar  stalls.
342c243e490SMarcel Moolenaar
343c243e490SMarcel Moolenaar* Deprecated the use.fs property.  While this seemed like a good idea in
344c243e490SMarcel Moolenaar  the first place to impose more control on what test cases can do, it
345c243e490SMarcel Moolenaar  turns out to be bad.  First, use.fs=false prevents bogus test cases
346c243e490SMarcel Moolenaar  from dumping core so after-the-fact debugging is harder.  Second,
347c243e490SMarcel Moolenaar  supporting use.fs adds a lot of unnecessary complexity.  atf-run will
348c243e490SMarcel Moolenaar  now ignore any value provided to use.fs and will allow test cases to
349c243e490SMarcel Moolenaar  freely access the file system if they wish to.
350c243e490SMarcel Moolenaar
351c243e490SMarcel Moolenaar* Added the atf_tc_get_config_var_as_{bool,long}{,_wd} functions to the atf-c
352c243e490SMarcel Moolenaar  library.  The 'text' module became private in 0.11 but was being used
353c243e490SMarcel Moolenaar  externally to simplify the parsing of configuration variables.
354c243e490SMarcel Moolenaar
355c243e490SMarcel Moolenaar* Made atf-run recognize the 'unprivileged-user' configuration variable
356c243e490SMarcel Moolenaar  and automatically drop root privileges when a test case sets
357c243e490SMarcel Moolenaar  require.user=unprivileged.  Note that this is, by no means, done for
358c243e490SMarcel Moolenaar  security purposes; this is just for user convenience; tests should, in
359c243e490SMarcel Moolenaar  general, not be blindly run as root in the first place.
360c243e490SMarcel Moolenaar
361c243e490SMarcel Moolenaar
362c243e490SMarcel MoolenaarChanges in version 0.11
363c243e490SMarcel Moolenaar***********************
364c243e490SMarcel Moolenaar
365c243e490SMarcel MoolenaarExperimental version released on October 20th, 2010.
366c243e490SMarcel Moolenaar
367c243e490SMarcel Moolenaar* The ATF_CHECK* macros in atf-c++ were renamed to ATF_REQUIRE* to match
368c243e490SMarcel Moolenaar  their counterparts in atf-c.
369c243e490SMarcel Moolenaar
370c243e490SMarcel Moolenaar* Clearly separated the modules in atf-c that are supposed to be public
371c243e490SMarcel Moolenaar  from those that are implementation details.  The header files for the
372c243e490SMarcel Moolenaar  internal modules are not installed any more.
373c243e490SMarcel Moolenaar
374c243e490SMarcel Moolenaar* Made the atf-check tool private.  It is only required by atf-sh and being
375c243e490SMarcel Moolenaar  public has the danger of causing confusion.  Also, making it private
376c243e490SMarcel Moolenaar  simplifies the public API of atf.
377c243e490SMarcel Moolenaar
378c243e490SMarcel Moolenaar* Changed atf-sh to enable per-command error checking (set -e) by default.
379c243e490SMarcel Moolenaar  This catches many cases in which a test case is broken but it is not
380c243e490SMarcel Moolenaar  reported as such because execution continues.
381c243e490SMarcel Moolenaar
382c243e490SMarcel Moolenaar* Fixed the XSTL and CSS stylesheets to support expected failures.
383c243e490SMarcel Moolenaar
384c243e490SMarcel Moolenaar
385c243e490SMarcel MoolenaarChanges in version 0.10
386c243e490SMarcel Moolenaar***********************
387c243e490SMarcel Moolenaar
388c243e490SMarcel MoolenaarExperimental version released on July 2nd, 2010.
389c243e490SMarcel Moolenaar
390c243e490SMarcel MoolenaarMiscellaneous features
391c243e490SMarcel Moolenaar
392c243e490SMarcel Moolenaar* Added expected failures support to test cases and atf-run.  These
393c243e490SMarcel Moolenaar  include, for example, expected clean exits, expected reception of fatal
394c243e490SMarcel Moolenaar  signals, expected timeouts and expected errors in condition checks.
395c243e490SMarcel Moolenaar  These statuses can be used to denote test cases that are known to fail
396c243e490SMarcel Moolenaar  due to a bug in the code they are testing.  atf-report reports these
397c243e490SMarcel Moolenaar  tests separately but they do not count towards the failed test cases
398c243e490SMarcel Moolenaar  amount.
399c243e490SMarcel Moolenaar
400c243e490SMarcel Moolenaar* Added the ATF_CHECK_ERRNO and ATF_REQUIRE_ERRNO to the C library to
401c243e490SMarcel Moolenaar  allow easy checking of call failures that update errno.
402c243e490SMarcel Moolenaar
403c243e490SMarcel Moolenaar* Added the has.cleanup meta-data property to test caes that specifies
404c243e490SMarcel Moolenaar  whether the test case has a cleanup routine or not; its value is
405c243e490SMarcel Moolenaar  automatically set.  This property is read by atf-run to know if it has to
406c243e490SMarcel Moolenaar  run the cleanup routine; skipping this run for every test case
407c243e490SMarcel Moolenaar  significantly speeds up the run time of test suites.
408c243e490SMarcel Moolenaar
409c243e490SMarcel Moolenaar* Reversed the order of the ATF_CHECK_THROW macro in the C++ binding to
410c243e490SMarcel Moolenaar  take the expected exception as the first argument and the statement to
411c243e490SMarcel Moolenaar  execute as the second argument.
412c243e490SMarcel Moolenaar
413c243e490SMarcel MoolenaarChanges in atf-check
414c243e490SMarcel Moolenaar
415c243e490SMarcel Moolenaar* Changed atf-check to support negating the status and output checks by
416c243e490SMarcel Moolenaar  prefixing them with not- and added support to specify multiple checkers
417c243e490SMarcel Moolenaar  for stdout and stderr, not only one.
418c243e490SMarcel Moolenaar
419c243e490SMarcel Moolenaar* Added the match output checker to atf-check to look for regular
420c243e490SMarcel Moolenaar  expressions in the stdout and stderr of commands.
421c243e490SMarcel Moolenaar
422c243e490SMarcel Moolenaar* Modified the exit checks in atf-check to support checking for the
423c243e490SMarcel Moolenaar  reception of signals.
424c243e490SMarcel Moolenaar
425c243e490SMarcel MoolenaarCode simplifications and cleanups
426c243e490SMarcel Moolenaar
427c243e490SMarcel Moolenaar* Removed usage messages from test programs to simplify the
428c243e490SMarcel Moolenaar  implementation of every binding by a significant amount.  They just now
429c243e490SMarcel Moolenaar  refer the user to the appropriate manual page and do not attempt to wrap
430c243e490SMarcel Moolenaar  lines on terminal boundaries.  Test programs are not supposed to be run
431c243e490SMarcel Moolenaar  by users directly so this minor interface regression is not important.
432c243e490SMarcel Moolenaar
433c243e490SMarcel Moolenaar* Removed the atf-format internal utility, which is unused after the
434c243e490SMarcel Moolenaar  change documented above.
435c243e490SMarcel Moolenaar
436c243e490SMarcel Moolenaar* Removed the atf-cleanup internal utility.  It has been unused since the
437c243e490SMarcel Moolenaar  test case isolation was moved to atf-run in 0.8
438c243e490SMarcel Moolenaar
439c243e490SMarcel Moolenaar* Splitted the Makefile.am into smaller files for easier maintenance and
440c243e490SMarcel Moolenaar  dropped the use of M4.  Only affects users building from the repository
441c243e490SMarcel Moolenaar  sources.
442c243e490SMarcel Moolenaar
443c243e490SMarcel Moolenaar* Intermixed tests with the source files in the source tree to provide
444c243e490SMarcel Moolenaar  them more visibility and easier access.  The tests directory is gone from
445c243e490SMarcel Moolenaar  the source tree and tests are now suffixed by _test, not prefixed by t_.
446c243e490SMarcel Moolenaar
447c243e490SMarcel Moolenaar* Simplifications to the atf-c library: removed the io, tcr and ui
448c243e490SMarcel Moolenaar  modules as they had become unnecessary after all simplifications
449c243e490SMarcel Moolenaar  introduced since the 0.8 release.
450c243e490SMarcel Moolenaar
451c243e490SMarcel Moolenaar* Removed the application/X-atf-tcr format introduced in 0.8 release.
452c243e490SMarcel Moolenaar  Tests now print a much simplified format that is easy to parse and nicer
453c243e490SMarcel Moolenaar  to read by end users.  As a side effect, the default for test cases is
454c243e490SMarcel Moolenaar  now to print their results to stdout unless otherwise stated by providing
455c243e490SMarcel Moolenaar  the -r flag.
456c243e490SMarcel Moolenaar
457c243e490SMarcel Moolenaar* Removed XML distribution documents and replaced them with plain-text
458c243e490SMarcel Moolenaar  documents.  They provided little value and introduced a lot of complexity
459c243e490SMarcel Moolenaar  to the build system.
460c243e490SMarcel Moolenaar
461c243e490SMarcel Moolenaar* Simplified the output of atf-version by not attempting to print a
462c243e490SMarcel Moolenaar  revision number when building form a distfile.  Makes the build system
463c243e490SMarcel Moolenaar  easier to maintain.
464c243e490SMarcel Moolenaar
465c243e490SMarcel Moolenaar
466c243e490SMarcel MoolenaarChanges in version 0.9
467c243e490SMarcel Moolenaar**********************
468c243e490SMarcel Moolenaar
469c243e490SMarcel MoolenaarExperimental version released on June 3rd, 2010.
470c243e490SMarcel Moolenaar
471c243e490SMarcel Moolenaar* Added atf-sh, an interpreter to process test programs written using
472c243e490SMarcel Moolenaar  the shell API.  This is not really a shell interpreter by itself though:
473c243e490SMarcel Moolenaar  it is just a wrapper around the system shell that eases the loading of
474c243e490SMarcel Moolenaar  the necessary ATF libraries.
475c243e490SMarcel Moolenaar
476c243e490SMarcel Moolenaar* Removed atf-compile in favour of atf-sh.
477c243e490SMarcel Moolenaar
478c243e490SMarcel Moolenaar* Added the use.fs metadata property to test case, which is used to
479c243e490SMarcel Moolenaar  specify which test cases require file system access.  This is to
480c243e490SMarcel Moolenaar  highlight dependencies on external resources more clearly and to speed up
481c243e490SMarcel Moolenaar  the execution of test suites by skipping the creation of many unnecessary
482c243e490SMarcel Moolenaar  work directories.
483c243e490SMarcel Moolenaar
484c243e490SMarcel Moolenaar* Fixed test programs to get a sane default value for their source
485c243e490SMarcel Moolenaar  directory.  This means that it should not be necessary any more to pass
486c243e490SMarcel Moolenaar  -s when running test programs that do not live in the current directory.
487c243e490SMarcel Moolenaar
488c243e490SMarcel Moolenaar* Defining test case headers became optional.  This is trivial to achieve
489c243e490SMarcel Moolenaar  in shell-based tests but a bit ugly in C and C++.  In C, use the new
490c243e490SMarcel Moolenaar  ATF_TC_WITHOUT_HEAD macro to define the test case, and in C++ use
491c243e490SMarcel Moolenaar  ATF_TEST_CASE_WITHOUT_HEAD.
492c243e490SMarcel Moolenaar
493c243e490SMarcel Moolenaar
494c243e490SMarcel MoolenaarChanges in version 0.8
495c243e490SMarcel Moolenaar**********************
496c243e490SMarcel Moolenaar
497c243e490SMarcel MoolenaarExperimental version released on May 7th, 2010.
498c243e490SMarcel Moolenaar
499c243e490SMarcel Moolenaar* Test programs no longer run several test cases in a row.  The execution
500c243e490SMarcel Moolenaar  of a test program now requires a test case name, and that single test
501c243e490SMarcel Moolenaar  case is executed.  To execute several test cases, use the atf-run utility
502c243e490SMarcel Moolenaar  as usual.
503c243e490SMarcel Moolenaar
504c243e490SMarcel Moolenaar* Test programs no longer fork a subprocess to isolate the execution of
505c243e490SMarcel Moolenaar  test cases.  They run the test case code in-process, and a crash of the
506c243e490SMarcel Moolenaar  test case will result in a crash of the test program.  This is to ease
507c243e490SMarcel Moolenaar  debugging of faulty test cases.
508c243e490SMarcel Moolenaar
509c243e490SMarcel Moolenaar* Test programs no longer isolate their test cases.  This means that they
510c243e490SMarcel Moolenaar  will not create temporary directories nor sanitize the environment any
511c243e490SMarcel Moolenaar  more.  Yes: running a test case that depends on system state by hand will
512c243e490SMarcel Moolenaar  most likely yield different results depending on where (machine,
513c243e490SMarcel Moolenaar  directory, user environment, etc.) it is run.  Isolation has been moved
514c243e490SMarcel Moolenaar  to atf-run.
515c243e490SMarcel Moolenaar
516c243e490SMarcel Moolenaar* Test programs no longer print a cryptic format (application/X-atf-tcs)
517c243e490SMarcel Moolenaar  on a special file channel.  They can now print whatever they want on the
518c243e490SMarcel Moolenaar  screen.  Because test programs can now only run one test case every time,
519c243e490SMarcel Moolenaar  providing controlled output is not necessary any more.
520c243e490SMarcel Moolenaar
521c243e490SMarcel Moolenaar* Test programs no longer write their status into a special file
522c243e490SMarcel Moolenaar  descriptor.  Instead, they create a file with the results, which is later
523c243e490SMarcel Moolenaar  parsed by atf-run.  This changes the semantics of the -r flag.
524c243e490SMarcel Moolenaar
525c243e490SMarcel Moolenaar* atf-run has been adjusted to perform the test case isolation.  As a
526c243e490SMarcel Moolenaar  result, there is now a single canonical place that implements the
527c243e490SMarcel Moolenaar  isolation of test caes.  In previous releases, the three language
528c243e490SMarcel Moolenaar  bindings (C, C++ and shell) had to be kept in sync with each other (read:
529c243e490SMarcel Moolenaar  not a nice thing to do at all).  As a side effect of this change, writing
530c243e490SMarcel Moolenaar  bindings for other languages will be much, much easier from now on.
531c243e490SMarcel Moolenaar
532c243e490SMarcel Moolenaar* atf-run forks test programs on a test case basis, instead of on a test
533c243e490SMarcel Moolenaar  program basis as it did before.  This is to provide the test case
534c243e490SMarcel Moolenaar  isolation that was before implemented by the test programs themselves.
535c243e490SMarcel Moolenaar
536c243e490SMarcel Moolenaar* Removed the atf-exec tool.  This was used to implement test case
537c243e490SMarcel Moolenaar  isolation in atf-sh, but it is now unnecessary.
538c243e490SMarcel Moolenaar
539c243e490SMarcel Moolenaar* It is now optional to define the descr meta-data property.  It has been
540c243e490SMarcel Moolenaar  proven to be mostly useless, because test cases often carry a descriptive
541c243e490SMarcel Moolenaar  name of their own.
542c243e490SMarcel Moolenaar
543c243e490SMarcel Moolenaar
544c243e490SMarcel MoolenaarChanges in version 0.7
545c243e490SMarcel Moolenaar**********************
546c243e490SMarcel Moolenaar
547c243e490SMarcel MoolenaarExperimental version released on December 22nd, 2009.
548c243e490SMarcel Moolenaar
549c243e490SMarcel Moolenaar* Added build-time checks to atf-c and atf-c++.  A binding for atf-sh
550c243e490SMarcel Moolenaar  will come later.
551c243e490SMarcel Moolenaar
552c243e490SMarcel Moolenaar* Migrated all build-time checks for header files to proper ATF tests.
553c243e490SMarcel Moolenaar  This demonstrates the use of the new feature described above.
554c243e490SMarcel Moolenaar
555c243e490SMarcel Moolenaar* Added an internal API for child process management.
556c243e490SMarcel Moolenaar
557c243e490SMarcel Moolenaar* Converted all plain-text distribution documents to a Docbook canonical
558c243e490SMarcel Moolenaar  version, and include pre-generated plain text and HTML copies in the
559c243e490SMarcel Moolenaar  distribution file.
560c243e490SMarcel Moolenaar
561c243e490SMarcel Moolenaar* Simplified the contents of the Makefile.am by regenerating it from a
562c243e490SMarcel Moolenaar  canonical Makefile.am.m4 source.  As a side-effect, some dependency
563c243e490SMarcel Moolenaar  specifications were fixed.
564c243e490SMarcel Moolenaar
565c243e490SMarcel Moolenaar* Migrated all checks from the check target to installcheck, as these
566c243e490SMarcel Moolenaar  require ATF to be installed.
567c243e490SMarcel Moolenaar
568c243e490SMarcel Moolenaar* Fixed sign comparison mismatches triggered by the now-enabled
569c243e490SMarcel Moolenaar  -Wsign-compare.
570c243e490SMarcel Moolenaar
571c243e490SMarcel Moolenaar* Fixed many memory and object leaks.
572c243e490SMarcel Moolenaar
573c243e490SMarcel Moolenaar
574c243e490SMarcel MoolenaarChanges in version 0.6
575c243e490SMarcel Moolenaar**********************
576c243e490SMarcel Moolenaar
577c243e490SMarcel MoolenaarExperimental version released on January 18th, 2009.
578c243e490SMarcel Moolenaar
579c243e490SMarcel Moolenaar* Make atf-exec be able to kill its child process after a certain period
580c243e490SMarcel Moolenaar  of time; this is controlled through the new -t option.
581c243e490SMarcel Moolenaar
582c243e490SMarcel Moolenaar* Change atf-sh to use atf-exec's -t option to control the test case's
583c243e490SMarcel Moolenaar  timeouts, instead of doing it internally.  Same behavior as before, but
584c243e490SMarcel Moolenaar  noticeably faster.
585c243e490SMarcel Moolenaar
586c243e490SMarcel Moolenaar* atf-exec's -g option and atf-killpg are gone due to the previous
587c243e490SMarcel Moolenaar  change.
588c243e490SMarcel Moolenaar
589c243e490SMarcel Moolenaar* Added the atf-check(1) tool, a program that executes a given command
590c243e490SMarcel Moolenaar  and checks its exit code against a known value and allows the management
591c243e490SMarcel Moolenaar  of stdout and stderr in multiple ways.  This replaces the previous
592c243e490SMarcel Moolenaar  atf_check function in the atf-sh library and exposes this functionality
593c243e490SMarcel Moolenaar  to both atf-c and atf-c++.
594c243e490SMarcel Moolenaar
595c243e490SMarcel Moolenaar* Added the ATF_REQUIRE family of macros to the C interface.  These help
596c243e490SMarcel Moolenaar  in checking for fatal test conditions.  The old ATF_CHECK macros now
597c243e490SMarcel Moolenaar  perform non-fatal checks only.  I.e. by using ATF_CHECK, the test case
598c243e490SMarcel Moolenaar  can now continue its execution and the failures will not be reported
599c243e490SMarcel Moolenaar  until the end of the whole run.
600c243e490SMarcel Moolenaar
601c243e490SMarcel Moolenaar* Extended the amount of ATF_CHECK_* C macros with new ones to provide
602c243e490SMarcel Moolenaar  more features to the developer.  These also have their corresponding
603c243e490SMarcel Moolenaar  counterparts in the ATF_REQUIRE_* family.  The new macros (listing the
604c243e490SMarcel Moolenaar  suffixes only) are: _EQ (replaces _EQUAL), _EQ_MSG, _STREQ and
605c243e490SMarcel Moolenaar  _STREQ_MSG.
606c243e490SMarcel Moolenaar
607c243e490SMarcel Moolenaar
608c243e490SMarcel MoolenaarChanges in version 0.5
609c243e490SMarcel Moolenaar**********************
610c243e490SMarcel Moolenaar
611c243e490SMarcel MoolenaarExperimental version released on May 1st, 2008.
612c243e490SMarcel Moolenaar
613c243e490SMarcel Moolenaar* Clauses 3 and 4 of the BSD license used by the project were dropped.
614c243e490SMarcel Moolenaar  All the code is now under a 2-clause BSD license compatible with the GNU
615c243e490SMarcel Moolenaar  General Public License (GPL).
616c243e490SMarcel Moolenaar
617c243e490SMarcel Moolenaar* Added a C-only binding so that binary test programs do not need to be
618c243e490SMarcel Moolenaar  tied to C++ at all.  This binding is now known as the atf-c library.
619c243e490SMarcel Moolenaar
620c243e490SMarcel Moolenaar* Renamed the C++ binding to atf-c++ for consistency with the new atf-c.
621c243e490SMarcel Moolenaar
622c243e490SMarcel Moolenaar* Renamed the POSIX shell binding to atf-sh for consistency with the new
623c243e490SMarcel Moolenaar  atf-c and atf-c++.
624c243e490SMarcel Moolenaar
625c243e490SMarcel Moolenaar* Added a -w flag to test programs through which it is possible to
626c243e490SMarcel Moolenaar  specify the work directory to be used.  This was possible in prior
627c243e490SMarcel Moolenaar  releases by defining the workdir configuration variable (-v workdir=...),
628c243e490SMarcel Moolenaar  but was a conceptually incorrect mechanism.
629c243e490SMarcel Moolenaar
630c243e490SMarcel Moolenaar* Test programs now preserve the execution order of test cases when they
631c243e490SMarcel Moolenaar  are given in the command line.  Even those mentioned more than once are
632c243e490SMarcel Moolenaar  executed multiple times to comply with the user's requests.
633c243e490SMarcel Moolenaar
634c243e490SMarcel Moolenaar
635c243e490SMarcel MoolenaarChanges in version 0.4
636c243e490SMarcel Moolenaar**********************
637c243e490SMarcel Moolenaar
638c243e490SMarcel MoolenaarExperimental version released on February 4th, 2008.
639c243e490SMarcel Moolenaar
640c243e490SMarcel Moolenaar* Added two new manual pages, atf-c++-api and atf-sh-api, describing the
641c243e490SMarcel Moolenaar  C++ and POSIX shell interfaces used to write test programs.
642c243e490SMarcel Moolenaar
643c243e490SMarcel Moolenaar* Added a pkg-config file, useful to get the flags to build against the
644c243e490SMarcel Moolenaar  C++ library or to easily detect the presence of ATF.
645c243e490SMarcel Moolenaar
646c243e490SMarcel Moolenaar* Added a way for test cases to require a specific architecture and/or
647c243e490SMarcel Moolenaar  machine type through the new 'require.arch' and 'require.machine'
648c243e490SMarcel Moolenaar  meta-data properties, respectively.
649c243e490SMarcel Moolenaar
650c243e490SMarcel Moolenaar* Added the 'timeout' property to test cases, useful to set an
651c243e490SMarcel Moolenaar  upper-bound limit for the test's run time and thus prevent global test
652c243e490SMarcel Moolenaar  program stalls due to the test case's misbehavior.
653c243e490SMarcel Moolenaar
654c243e490SMarcel Moolenaar* Added the atf-exec(1) internal utility, used to execute a command
655c243e490SMarcel Moolenaar  after changing the process group it belongs to.
656c243e490SMarcel Moolenaar
657c243e490SMarcel Moolenaar* Added the atf-killpg(1) internal utility, used to kill process groups.
658c243e490SMarcel Moolenaar
659c243e490SMarcel Moolenaar* Multiple portability fixes.  Of special interest, full support for
660c243e490SMarcel Moolenaar  SunOS (Solaris Express Developer Edition 2007/09) using the Sun Studio 12
661c243e490SMarcel Moolenaar  C++ compiler.
662c243e490SMarcel Moolenaar
663c243e490SMarcel Moolenaar* Fixed a serious bug that prevented atf-run(1) from working at all
664c243e490SMarcel Moolenaar  under Fedora 8 x86_64.  Due to the nature of the bug, other platforms
665c243e490SMarcel Moolenaar  were likely affected too.
666c243e490SMarcel Moolenaar
667c243e490SMarcel Moolenaar
668c243e490SMarcel MoolenaarChanges in version 0.3
669c243e490SMarcel Moolenaar**********************
670c243e490SMarcel Moolenaar
671c243e490SMarcel MoolenaarExperimental version released on November 11th, 2007.
672c243e490SMarcel Moolenaar
673c243e490SMarcel Moolenaar* Added XML output support to atf-report.  This is accompanied by a DTD
674c243e490SMarcel Moolenaar  for the format's structure and sample XSLT/CSS files to post-process this
675c243e490SMarcel Moolenaar  output and convert it to a plain HTML report.
676c243e490SMarcel Moolenaar
677c243e490SMarcel Moolenaar* Changed atf-run to add system information to the report it generates.
678c243e490SMarcel Moolenaar  This is currently used by atf-report's XML output only, and is later
679c243e490SMarcel Moolenaar  printed in the HTML reports in a nice and useful summary table.  The user
680c243e490SMarcel Moolenaar  and system administrator are allowed to tune this feature by means of
681c243e490SMarcel Moolenaar  hooks.
682c243e490SMarcel Moolenaar
683c243e490SMarcel Moolenaar* Removed the test cases' 'isolated' property.  This was intended to
684c243e490SMarcel Moolenaar  avoid touching the file system at all when running the related test case,
685c243e490SMarcel Moolenaar  but this has not been true for a long while: some control files are
686c243e490SMarcel Moolenaar  unconditionally required for several purposes, and we cannot easily get
687c243e490SMarcel Moolenaar  rid of them.  This way we remove several critical and delicate pieces of
688c243e490SMarcel Moolenaar  code.
689c243e490SMarcel Moolenaar
690c243e490SMarcel Moolenaar* Improved atf-report's CSV output format to include information about
691c243e490SMarcel Moolenaar  test programs too.
692c243e490SMarcel Moolenaar
693c243e490SMarcel Moolenaar* Fixed the tests that used atf-compile to not require this tool as a
694c243e490SMarcel Moolenaar  helper.  Avoids systems without build-time utilities to skip many tests
695c243e490SMarcel Moolenaar  that could otherwise be run.  (E.g. NetBSD without the comp.tgz set
696c243e490SMarcel Moolenaar  installed.)
697c243e490SMarcel Moolenaar
698c243e490SMarcel Moolenaar* Many general cleanups: Fixed many pieces of code marked as ugly and/or
699c243e490SMarcel Moolenaar  incomplete.
700c243e490SMarcel Moolenaar
701c243e490SMarcel Moolenaar
702c243e490SMarcel MoolenaarChanges in version 0.2
703c243e490SMarcel Moolenaar**********************
704c243e490SMarcel Moolenaar
705c243e490SMarcel MoolenaarExperimental version released on September 20th, 2007.
706c243e490SMarcel Moolenaar
707c243e490SMarcel Moolenaar* Test cases now get a known umask on entry.
708c243e490SMarcel Moolenaar
709c243e490SMarcel Moolenaar* atf-run now detects many unexpected failures caused by test programs and
710c243e490SMarcel Moolenaar  reports them as bogus tests.  atf-report is able to handle these new
711c243e490SMarcel Moolenaar  errors and nicely reports them to the user.
712c243e490SMarcel Moolenaar
713c243e490SMarcel Moolenaar* All the data formats read and written by the tools have been
714c243e490SMarcel Moolenaar  documented and cleaned up.  These include those grammars that define how
715c243e490SMarcel Moolenaar  the different components communicate with each other as well as the
716c243e490SMarcel Moolenaar  format of files written by the developers and users: the Atffiles and the
717c243e490SMarcel Moolenaar  configuration files.
718c243e490SMarcel Moolenaar
719c243e490SMarcel Moolenaar* Added the atf-version tool, a utility that displays information about
720c243e490SMarcel Moolenaar  the currently installed version of ATF.
721c243e490SMarcel Moolenaar
722c243e490SMarcel Moolenaar* Test cases can now define an optional cleanup routine to undo their
723c243e490SMarcel Moolenaar  actions regardless of their exit status.
724c243e490SMarcel Moolenaar
725c243e490SMarcel Moolenaar* atf-report now summarizes the list of failed (bogus) test programs
726c243e490SMarcel Moolenaar  when using the ticker output format.
727c243e490SMarcel Moolenaar
728c243e490SMarcel Moolenaar* Test programs now capture some termination signals and clean up any
729c243e490SMarcel Moolenaar  temporary files before exiting the program.
730c243e490SMarcel Moolenaar
731c243e490SMarcel Moolenaar* Multiple bug fixes and improvements all around.
732c243e490SMarcel Moolenaar
733c243e490SMarcel Moolenaar
734c243e490SMarcel MoolenaarChanges in version 0.1
735c243e490SMarcel Moolenaar**********************
736c243e490SMarcel Moolenaar
737c243e490SMarcel MoolenaarExperimental version released on August 20th, 2007.
738c243e490SMarcel Moolenaar
739c243e490SMarcel Moolenaar* First public version.  This was released coinciding with the end of the
740c243e490SMarcel Moolenaar  Google Summer of Code 2007 program.
741c243e490SMarcel Moolenaar
742c243e490SMarcel Moolenaar
743c243e490SMarcel Moolenaar===========================================================================
744c243e490SMarcel Moolenaarvim: filetype=text:textwidth=75:expandtab:shiftwidth=2:softtabstop=2
745