mkuzip: drop support for executable uzip imagesA uzip image has a 128-byte header, historically, this header could beexecuted as a shell script to mount the uzip image to a user providedmountpoin
mkuzip: drop support for executable uzip imagesA uzip image has a 128-byte header, historically, this header could beexecuted as a shell script to mount the uzip image to a user providedmountpoint.The embedded shell commands only work for uzip images that were createdwith zlib or zstd compression that contained an ISO-9660 file system.Given the limited space available in the uzip header, it is notpractical to extend this feature to include other file systems or toprovide sensible error handling and error messages to the user.For these reasons, abandon the embedded shell script in the uzip imageheader.To maintain backwards compatibility, the shebang and shell must residein the 128-byte header.This change of behavior is documented in mkuzip(8) and an examplehas been provided for creating/mounting uzip images.PR: 276174
show more ...
Remove residual blank line at start of MakefileThis is a residual of the $FreeBSD$ removal.MFC After: 3 days (though I'll just run the command on the branches)Sponsored by: Netflix
usr.bin: Automated cleanup of cdefs and other formattingApply the following automated changes to try to eliminateno-longer-needed sys/cdefs.h includes as well as now-emptyblank lines in a row.R
usr.bin: Automated cleanup of cdefs and other formattingApply the following automated changes to try to eliminateno-longer-needed sys/cdefs.h includes as well as now-emptyblank lines in a row.Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/Remove /\n+#if.*\n#endif.*\n+/Remove /^#if.*\n#endif.*\n/Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/Sponsored by: Netflix
Remove $FreeBSD$: two-line nroff patternRemove /^\.\\"\n\.\\"\s*\$FreeBSD\$$\n/
Remove $FreeBSD$: one-line sh patternRemove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/
Remove $FreeBSD$: one-line .c patternRemove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/
Remove $FreeBSD$: two-line .h patternRemove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
Mark usage function as __dead2 in programs where it does not returnIn most cases, usage does not return, so mark them as __dead2. For thecases where they do return, they have not been marked __dea
Mark usage function as __dead2 in programs where it does not returnIn most cases, usage does not return, so mark them as __dead2. For thecases where they do return, they have not been marked __dead2.Reviewed by: impPull Request: https://github.com/freebsd/freebsd-src/pull/735
mkuzip: Remove set but unused variable.Reported by: GCC
spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSDThe SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catchup to that fact and revert to their recommended match of
spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSDThe SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catchup to that fact and revert to their recommended match of BSD-2-Clause.Discussed with: pfgMFC After: 3 daysSponsored by: Netflix
Update/fix Makefile.depend for userland
geom_uzip(4), mkuzip(8): Add Zstd image modeThe Zstd format bumps the CLOOP major number to 4 to avoid incompatibilitywith older systems. Support in geom_uzip(4) is conditional on the ZSTDIOkern
geom_uzip(4), mkuzip(8): Add Zstd image modeThe Zstd format bumps the CLOOP major number to 4 to avoid incompatibilitywith older systems. Support in geom_uzip(4) is conditional on the ZSTDIOkernel option, which is enabled in amd64 GENERIC, but not all in-treeconfigurations.mkuzip(8) was modified slightly to always initialize the nblocks + 1'thoffset in the CLOOP file format. Previously, it was only initialized in thecase where the final compressed block happened to be unaligned w.r.t.DEV_BSIZE. The "Fake" last+1 block change in r298619 means that the finalcompressed block's 'blen' was never correct unless the compressed uzip imagehappened to be BSIZE-aligned. This happened in about 1 out of every 512cases. The zlib and lzma decompressors are probably tolerant of extra trashfollowing the frame they were told to decode, but Zstd complains that theinput size is incorrect.Correspondingly, geom_uzip(4) was modified slightly to avoid trashing thenblocks + 1'th offset when it is known to be initialized to a good value.This corrects the calculated final real cluster compressed length to matchthat printed by mkuzip(8).mkuzip(8) was refactored somewhat to reduce code duplication and increaseease of adding other compression formats. * Input block size validation was pulled out of individual compression init routines into main(). * Init routines now validate a user-provided compression level or select an algorithm-specific default, if none was provided. * A new interface for calculating the maximal compressed size of an incompressible input block was added for each driver. The generic code uses it to validate against MAXPHYS as well as to allocate compression result buffers in the generic code. * Algorithm selection is now driven by a table lookup, to increase ease of adding other formats in the future.mkuzip(8) gained the ability to explicitly specify a compression level with'-C'. The prior defaults -- 9 for zlib and 6 for lzma -- are maintained.The new zstd default is 9, to match zlib.Rather than select lzma or zlib with '-L' or its absense, respectively, anew argument '-A <algorithm>' is provided to select 'zlib', 'lzma', or'zstd'. '-L' is considered deprecated, but will probably never be removed.All of the new features were documented in mkuzip.8; the page was alsocleaned up slightly.Relnotes: yes
Minor cleanup for mkuzip(8) man page.List all single-letter options in summary.Order options alphabetically.Sponsored by: Mellanox TechnologiesMFC after: 3 days
various: general adoption of SPDX licensing ID tags.Mainly focus on files that use BSD 2-Clause license, however the tool Iwas using misidentified many licenses so this was mostly a manual - error
various: general adoption of SPDX licensing ID tags.Mainly focus on files that use BSD 2-Clause license, however the tool Iwas using misidentified many licenses so this was mostly a manual - errorprone - task.The Software Package Data Exchange (SPDX) group provides a specificationto make it easier for automated tools to detect and summarize well knownopensource licenses. We are gradually adopting the specification, notingthat the tags are considered only advisory and do not, in any way,superceed or replace the license texts.No functional change intended.
DIRDEPS_BUILD: Update dependencies.Sponsored by: Dell EMC Isilon
Doh, fix some botched "fix" in r320277.Reported by: cemMFC after: 6 weeks
Don't leak file descriptor in some cases.Reported by: cemMFC after: 6 weeks
o Move logic that determines size of the input image into its own file. That logic has grown quite significantly now;o add a special handling for the snapshot images. Those have some extra head
o Move logic that determines size of the input image into its own file. That logic has grown quite significantly now;o add a special handling for the snapshot images. Those have some extra headers at the end of the image and we don't need those in the output image really.MFC after: 6 weeks
Don't depend on assert(3) getting evaluatedReported by: impMFC after: 3 weeksX-MFC-With: 318141, 318143Sponsored by: Spectra Logic Corp
strcpy => strlcpyReported by: CoverityCID: 1352771MFC after: 3 weeksSponsored by: Spectra Logic Corp
Remove spaces at EOL and sort correctly the SEE ALSO sectionReported by: make manlintMFC after: 2 days
DIRDEPS_BUILD: Update dependencies.Sponsored by: EMC / Isilon Storage Division
Try to make gcc builds happy again by removing a redundant declaration.
GC duplicate define.
Improve performance in a few key areas: o Split the compression across several worker threads. By default, "several" matches number of CPUs, capped at 24 for sanity when running on a very big
Improve performance in a few key areas: o Split the compression across several worker threads. By default, "several" matches number of CPUs, capped at 24 for sanity when running on a very big hardwares. Provide option to set that number manually; o Fix bug inherited from the mkulzma (R.I.P) which degraded already slow LZMA compression even further by calling function to release compression state after processing each block. It is neither documented as required nor actually required by the LZMA library. This caused spree of system calls to release memory and then map it again for every block. LZMA compression is more than 2x faster after this change alone; o Record time it takes to do compression and report throughput achieved. o Add simple first-level 256 entry hash table for de-dup code, so it's not becoming a bottleneck at big files.
123