The following has been sent out as email to some people to hint them about the largefile problems. It gives a quick overview of what the problems might be and what's that about the testscript.
Summary The Unix98 standard requires largefile support and in fact many of the latest operating systems do. However some systems chose to still make it not the default resulting in two models where some parts of the system use the traditional 32bit off_t while others already compiled with a largefile 64bit off_t. Mixing libraries and plugins is not a good idea. The 64on32 While systems like FreeBSD and Darwin do simply use a 64bit off_t as the default, there are also systems like Linux and Solaris that do not. Instead they do implement what's called the `transitional API` in the largefile specifications - many calls are given a 64-cousin, so that there are both "open" and "open64" in the C library, and also "lseek" and "lseek64". Using a define like -D_FILE_OFFSET_BITS=64 will make for some magic that traditional calls are remapped to the transitional API - so that your source code might read `open(...)` and `lseek(...)` but it will get linked to the symbols "open64" and "lseek64" for real. Also, it will make off_t a 64bit entity. In headers As a result however it is highly dangerous to use off_t in header files in largefile sensitive systems. Most software writers do not expect that an integral type like off_t can change its size, it is just used in the exported interface like in making up a new call `off_t my_lseek(int,off_t,int)`. In reality we get a scent from old DOS-modes with a "small" mode and "large" mode to compile source code - the library code might be compiled with a 64bit off_t while the application code using the library is compiled with 32bit off_t - possibly ending up in a callframe mismatch. A similar problem arrives when using off_t in exported structures as these can have differeng sizes and offsets for the member variables therein. A library maker should take measures to defend against improper off_t size, possibly making up dualmode func/func64 like the C library does, but in reality many software writers have been not aware of the actual problem so far. seek problem Another problem is described in the largefile documents in the section about holes in the protection system - it stems from the fact that some file descriptors might be opened in largefile mode while others are not, and they can even be transferred from a non-largefile application into largefile libraries and vice versa. The 64on32 transitional API is trying to support this scheme, mostly by introducing a new error code EOVERFLOW that will be returned when a "small"file application access a file that has grown beyond the 2GiB limit due to calls from other software parts compiled as "large"file. However, most "small"file software does not expect this error-code and it is known that many software writers do not check the return value of lseek - it can easily lead to data corruption when the file-pointer is not actually moved. mixing up Most of the software problems arise on the side of "small"file applications. As a general advice one should compile all software as largefile as soon as the system provides these interfaces. That is pretty easy, AC_SYS_LARGEFILE in autoconf'ed software can do it, or just some -D_FILE_OFFSET_BITS=64 to be defined somewhere. A lot of software however is not aware of a need to enable largefile mode somewhere, hundreds of opensource applications are compiled with 32bit off_t by default. It's been simply forgotten, and it would create a lot of work and publicity to make everyone aware - with the only result that the next new developer misses it just again. As a result, we better use technical support tools to track the problem area of mixing up compiled code from largefile sides and those which do not yet do so. check mismatch The perl script to do that can be fetched from http://ac-archive.sf.net/largefile - it will try to classify binaries and libraries whether they are using "-32-" or "-64-" modes, it does so by looking for fopen() .vs. fopen64() to present in the list of dynamic symbols. Each argument binary is checked plus the dynamic dependencies it has - and if there are mismatches then it prints a list of them. Furthermore, the script can detect when a library is trying to exhibit itself as dualmode exporting both func() and func64() calls, e.g. libgstreamer does so. For these it is okay that software may be in either of -32- or -64- mode when linking to these. So actually, only three combinations are rejected: -64- depends on -32-, -32- depends on -64-, and 3264 dualmode libraries to depend on simple -32- libraries. distro problem When the script is run in a contemporary Linux system on /usr/lib/*.so or just /usr/bin then it will detect a lot of largefile mismatches. The common user will not experience any problems with that as long no files are handled being possibly larger than 2 GibiByte. And note that the Unix98 mandates base utilities like `cat` and `cp` to be compiled as largefile ones for sure. Real opensource OS distributions however carry a lot more code from very different sides, and particularly there is a number of graphical frontend of the filemanager type that is not compiled in largefile mode. Sooner or later, the problem will come up. It would be best if no rpm/deb/whatever binary package to exist that has a largefile mismatch in the first place. That can be helped if the packagers and distro makers would check the binary packages while making them. That could be easy when integrating the check-routine into the post-%files tools (as it is called in rpm) which needs to check the libraries and binaries anyway for dependent libraries as to do a `chrpath` on them since they have been relinked in the DESTDIR staging area. Future The future should be to see all packages get compiled in largefile mode thereby eliminating any problems from mixing up libraries from different sides. A distro maker can ensure that, and if that needs a few patches then it is good anyway as it makes the very same software more portable to freebsd/darwin. And a little more beyond, one should really think about dropping the 32bit off_t default altogether - as it was done on FreeBSD obviously. The Linux 2.4 and glibc 2.2 should be all ready fine to do the step, and leaving the old times of "small"files behind. links http://ac-archive.sourceforge.net/largefile http://ftp.sas.com/standards/large.file/ http://unix.org/version2/whatsnew/lfs.html http://www.suse.de/~aj/linux_lfs.html P.S. If you are a programmer then be noted that it does not hurt to compile with AC_SYS_LARGEFILE by default - all the traditional 32bit calls of the C library are just wrappers to forward the arguments to the 64bit calls. That's simply because the internals and the OS kernel are using 64bit off_t anyway. In effect, your program might run even slightly faster, it will need to cope with less errorcode, and gets more portable to systems like freebsd that use a 64bit off_t anyway.