1# 2# CDDL HEADER START 3# 4# The contents of this file are subject to the terms of the 5# Common Development and Distribution License (the "License"). 6# You may not use this file except in compliance with the License. 7# 8# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE 9# or http://www.opensolaris.org/os/licensing. 10# See the License for the specific language governing permissions 11# and limitations under the License. 12# 13# When distributing Covered Code, include this CDDL HEADER in each 14# file and include the License file at usr/src/OPENSOLARIS.LICENSE. 15# If applicable, add the following below this CDDL HEADER, with the 16# fields enclosed by brackets "[]" replaced with your own identifying 17# information: Portions Copyright [yyyy] [name of copyright owner] 18# 19# CDDL HEADER END 20# 21 22# 23# Copyright 2008 Sun Microsystems, Inc. All rights reserved. 24# Use is subject to license terms. 25# 26# ident "%Z%%M% %I% %E% SMI" 27 28 29 30 31Why 32-bit libelf is not Large File Aware 32----------------------------------------- 33 34The ELF format uses unsigned 32-bit integers for offsets, so the 35theoretical limit on a 32-bit ELF object is 4GB. However, libelf 36imposes a 2GB limit on the objects it can create. The Solaris 37link-editor and related tools are all based on libelf, so the 3832-bit version of the link-editor also has a 2GB limit, despite 39the theoretical limit of 4GB. 40 41Large file support (LFS) is a half step between the 32 and 64-bit 42worlds, in which an otherwise 32-bit limited process is allowed to 43read and write data to a file that can be larger than 2GB (the extent 44of a signed 32-bit integer, as represented by the system type off_t). 45LFS is useful if the program only needs to access a small subset of 46the file data at any given time (e.g. /usr/bin/cat). It is less useful 47if the program needs to access a large amount of data at once --- having 48been freed from the file limit, the program will simply hit the virtual 49memory limit (4GB). 50 51In particular, the link-editor generally requires twice as much 52memory as the size of the output object, half to hold the input 53objects, and half to hold the result. This means that a 32-bit 54link-editor process will hit the 2GB file size limit and the 4GB 55address space limit at roughly the same time. As a result, a 56large file aware 32-bit version of libelf has no significant value. 57Despite this, the question of what it would take to make libelf 58large file aware comes up from time to time. 59 60The first step would be to provide alternative versions of 61all public data structures that involve the off_t data type. 62These structs, found in /usr/include/libelf.h, are: 63 64 /* 65 * Archive member header 66 */ 67 typedef struct { 68 char *ar_name; 69 time_t ar_date; 70 uid_t ar_uid; 71 gid_t ar_gid; 72 mode_t ar_mode; 73 off_t ar_size; 74 char *ar_rawname; 75 } Elf_Arhdr; 76 77 78 /* 79 * Data descriptor 80 */ 81 typedef struct { 82 Elf_Void *d_buf; 83 Elf_Type d_type; 84 size_t d_size; 85 off_t d_off; /* offset into section */ 86 size_t d_align; /* alignment in section */ 87 unsigned d_version; /* elf version */ 88 } Elf_Data; 89 90As off_t is a signed type, these alternative versions would have to use 91an off64_t type instead. 92 93In addition to providing alternative large file aware Elf_Arhdr and 94Elf_Data types, it would be necessary to implement large file aware 95versions of the public functions that use them, also found in 96/usr/include/libelf.h: 97 98 /* 99 * Function declarations 100 */ 101 unsigned elf_flagdata(Elf_Data *, Elf_Cmd, unsigned); 102 Elf_Arhdr *elf_getarhdr(Elf *); 103 off_t elf_getbase(Elf *); 104 Elf_Data *elf_getdata(Elf_Scn *, Elf_Data *); 105 Elf_Data *elf_newdata(Elf_Scn *); 106 Elf_Data *elf_rawdata(Elf_Scn *, Elf_Data *); 107 off_t elf_update(Elf *, Elf_Cmd); 108 Elf_Data *elf32_xlatetof(Elf_Data *, const Elf_Data *, unsigned); 109 Elf_Data *elf32_xlatetom(Elf_Data *, const Elf_Data *, unsigned); 110 Elf_Data *elf64_xlatetof(Elf_Data *, const Elf_Data *, unsigned); 111 Elf_Data *elf64_xlatetom(Elf_Data *, const Elf_Data *, unsigned); 112 113It is important to note that these new versions cannot replace the 114original definitions. Those must continue to be available to support 115non-largefile-aware programs. These new types and functions would be in 116addition to the pre-existing versions. 117 118When you make code like this large file aware, it is necessary to undertake 119a careful analysis of the code to ensure that all the surrounding code uses 120variable types large enough to handle the increased range. Hence, this work 121is more complicated than simply supplying variants that use a bigger 122off_t and rebuilding --- that is just the first step. 123 124There are two standard preprocessor definitions used to control 125large file support: 126 127 _LARGEFILE64_SOURCE 128 _FILE_OFFSET_BITS 129 130These preprocessor definitions would be used to determine whether 131a given program linked against libelf would see the regular, or 132the large file aware versons of the above types and routines. 133This is the same approach used in other large file capable software, 134such as libc. 135 136Finally, all the applications that rely on libelf would need to be made 137large file aware. As with libelf itself, there is more to such an effort 138than recompiling with preprocessor macros set. The code in these 139applications would need to be examined carefully. Some of these programs 140are very old, and were not originally written with such type portability 141in mind. Such code can be difficult to transition. 142 143To work around the 2GB limit in 32-bit libelf: 144 145 - The fundamental limits of a 32-bit address space mean 146 that a program this large should be 64-bit. Only a 64-bit 147 address space has enough room for that much code, plus the 148 stack and heap needed to do useful work with it. 149 150 - The 64-bit version of libelf is also able to process 151 32-bit objects, and does not have a 2GB file size limit. 152 Therefore, the 64-bit link-editor can be used to build a 32-bit 153 executable which is >2GB. The resulting program will consume over 154 half the available address space just to start running. However, 155 there may be enough address space left for it to do useful work. 156 157 Note that the 32-bit limit for sharable objects remains at 158 2GB --- imposed by the runtime linker, which is also not large 159 file aware. 160