- ...
- 2010-03-27
introducing the largefile64_source.h where off64_t is ensured.
This stands on top of exporting both implementations on all systems,
i.e. the largefile variant with 64bit offset values and a traditional
variant using a "long" offset value. The rest ist just an ifdef-else
in the headers for a third function without suffix while the binary has
only two functions func32 and func64.
- 2010-03-25
decribing the compat32 approach to achieve an easier 64on32 dualmode.
Thomas Orgis has pointed me to < ahref="http://bugs.winehq.org/show_bug.cgi?id=22132">
http://bugs.winehq.org/show_bug.cgi?id=22132 which did show a
problem with twinlibs with respect to dynaloading the instance. So,
in order to make a binary compatible variant of a libary that is not
as bad to maintain one can try to care for a compat32 lib variant.
This is now considered the best practice.
- 2006-06-05
updating the testscripts - changed python to my new style.
I changed the python style for match/replace to my new style using
classes Match/MatchReplace. While going over the source I did find
some minor glitches like missing "r" on regex pattersn. Additionally
I added an "ignore:" when the `ldd` did resolve to a non-library
artifact as an import (which might need to be expanded on).
- 2006-01-13
updating the testscripts and adding
broken-builds page.
Renewed intest has shown that there was a small bug in the testscript.
It is fixed it and additionally I was reformatting the output of the
testscript run to make it better for postprocessing with third party
tools. Another report did point out problems when using config.h
based on a general problem space described on the broken-builds page.
- 2004-04-21
using mksite.sh formatter
Created a sitefile and rebuilding the website with the formatter.
Here we get a sitemap as well as a printer-friendly mode for free.
In the course some markup was modified, e.g. all paragraphs should
now be justified and indented slightly.
- 2003-12-20
adding conversion pages
To get an easy overview how to support largefile with a library
these pages tell the necessary changes - step by step. It's been
given from memory but let's see if someone notes me of minor
problems and buglets. Atleast it should give people an easier start.
- 2003-11-15
little site updates
during porting the script to python, I did also change the perl script
to keep them in sync. Well, it did turn out have a char missing in the
end. Tobias Brausen was kind to send a patch that fixes a bit more and
also adds "-w" in the header, that's a good choice I think.
Otherwise, a little discussion about largefile in win32 was there on
on the autoconf mailinglist. I did copy some parts to a new page on
win32libc.html, and may be you want to
read the full thread here:
mail.gnu.org/archive/html/autoconf/2003-11/msg00039.html
- 2003-08-15
ported script to python
and a few updates to the website were added as well. Likewise,
the build system was redone to allow proper snapshot building
plus the ability to install the scripts and docs and to build
rpm packages. That should allow for even wider distribution.
- 2003-08-11
refreshing the website
Using a new makefile setup, the website is given a uniform look for sure.
The last refresh of 08-02 was wrong since code2html was installed,
now we accept gnu source-highlight as an alternative.
- 2003-08-02
_int/_int64 not a pair for 3264
While checking newer distros I became aware that libgnomevfs-2 is
not a dualmode library - the perl script did think it was since it
did saw a pair of *_read_int/*_read_int64. We exclude that pair now.
Furthermore I did notice that some routines reference both the
"fopen" and "fopen64" routine. This can not be anything else than
an error, a miscompiled binary. We already have been marking these
libraries internally, now we show a list classified as "DANGER".
Finally, we add six other symbols to the list of known imported
symbols that make a library to be classified.
- 2003-05-11
_LARGEFILE_SOURCE == _WE_DO_USE_OFF_T_
From problems of zziplib on
solaris-8, I have learned that the -D_LARGEFILE_SOURCE define
does only select that the program sources want to use the XOpen
extension of `off_t` instead of the old `long`. As per default
however there is a `typedef long off_t` so it does not make a
difference but for one thing: solaris (and glibc) do only export
the symbol `fseeko` then which happens to be introduced with the
`off_t` type as well. The only correct way in Unix98 to shift a
32bit default off_t into a 64bit off_t happens to use a setting
of the -D_FILE_OFFSET_BITS=64 and everything else might just not
work. Here I need to check all webpages here to adapt to this.
- 2003-03-24
off_t_headers.html update.
From the experiences, I am reworking the off_t_headers page with
a reference to ac_sys_largefile_sensitive, some proper example
source, and a hint on howto fix an existing off_t-mismatch
problem quickly and incrementally. I'm going to add a hint on
the kde bug id below.
- 2003-03-23
http://bugs.kde.org/show_bug.cgi?id=52843 confirmed!
Stephan Kulow (coolo) has looked into the matter and simply
comments: "I guess this needs to be fixed asap". The bug id
is now "assigned" to him, so perhaps this will be ironed out
very soon now.
- 2003-03-17 fseeko bug -
http://sources.redhat.com/ml/bug-glibc/2003-03/msg00067.html
The fseeko() function is not defined with AC_SYS_LARGEFILE alone,
you need to add _LARGEFILE_SOURCE somewhere. The AC_FUNC_FSEEKO
will do that - however, it makes it somewhat impossible to write
software that runs in both modes, non-LFS and LFS in a system.
(and what about mixed-mode libraries?). The link above tells of
a glibc bug around it: the off_t will be made 64bit wide but
your library will link to 32bit fseeko() instead of fseek64().
Note that glibc is not alone with this: a quick check into the
solaris8 headers gives me another horror. It seems the LFS spec
is non-realistic in this very situation - and probably needs to
be fixed.
- 2003-01-27
linuxbase.org submission sent.
The LSB ships a tool
`appchk`
to test conformance of binaries including its imports. It would be
natural to add a check for largefile-mismatch in there. Similar could
be done for `libchk` and the sampel rpm scripts shipped by linuxbase.org.
I did also send an mail to the lsb-test mailing list regarding some
action items for largefile support. It does basically list the
three basic steps to get away from 64on32 completely within a few years.
- 2003-01-26
freshmeat editorial published and
slashdotted
The talkback leads me to add two new pages:
win32/other lists behaviour of the
win32-API and some pre-unix98 systems. The
environ changes tells about
possibilities in fixing the compile-time environment of
systems in the future to get rid of the problems of the
64on32 world.
- 2003-01-25
freshmeat - almost
So, to get some awareness for the largefile problem, I had been
preparing an editorial ready to be posted to freshmeat. The
policy requires to contact Jeff Covey which I did with
sending an e-mail on the 13th. Today he did reply along with
a html version of the editorial text and a request to send some
little bio information that should go with it. Well, let's see
when it gets posted, looking forward to that day :-)
- 2003-01-15
testscript off_t_problem.pl cleanup/sourcedocs
clean up the code so that it does document itself - may be some
people would be afraid to just run some perl script on their system.
Furthermore, a "--noncleanall" option was added to show all the
symbols that could be offending - and looking closer, I do see now
that some more offendings must be also silenced down. The thing
starts to get smart...
- 2003-01-13
testscript off_t_problem.pl --smart
It took about a day to create the website, and after that I've
been looking agtain into the testscript - it did seem to me that
sometimes the largefile mismatch is not important. Therefore, I did
add a routine that can detect common binding that are simply
acceptable - the are marked as "(clean)" in the output list and
running the script as "--smart" or even "--silent" will suppress
these entries from the final output list altogether. Furthermore,
using "--nonclean" can help to detect which symbols were offending
(it only shows the first imported symbol that is not in the good-list).
- 2003-01-12
http://ac-archive/largefile
- and
testscript off_t_problem.pl
Creating a perl script that checks binaries/libraries installed in
system whether they have mismatching off_t types. That may be later
used by system vendors to extend they post-rpm-build scripts to
check for mismatches.
Starting to create a whole website around the topic and placing
all relevant info just there. It's also the start of this
history document that you just read.
- 2003-01-11
http://article.gmane.org/gmane.comp.sysutils.autoconf.general/1935
The discussion on the autoconf mailinglist comes to the conclusion that
it should be part of the distro maker and packaging process to ensure
that binaries are not mixed up between 32bit-off_t mode and 64bit-off_t
mode.
That is also from a note from the largefile support group that states
there are holes in the mixed-style support for libc filedescriptors.
After all, it is not only a callframe-mismatch problem but also about
file-descriptor inheritance and passing file-descriptors from one
library/plugin to the next (e.g. zlib's gzdopen).
- 2003-01-11
http://advogato.org/person/guidod/
Starting a diary entry on advogato.org that summarizes the problemspace
a bit in the way it did exhibit itself up to this. The discussion on
mailinglist has heatened up and it makes me more and more worry that
it is actually a very bad idea to mix binaries of 32bit-off_t with
those of 64bit-off_t.
- 2003-01-10
http://bugs.kde.org/show_bug.cgi?id=52843
Filing a bug report for KDE3 - the <kio/jobclasses.h>
header makes use of off_t without guards. It seems that even an
experienced programmer like Stephan Kulow himself has missed the
problemspace about off_t.
Note that most of the /usr/lib/kde3/* plugins are actually
compiled as 32bit while some are compiled as 64bit-off_t - that
brings the kde plugin system at risk.
- 2003-01-10
zlib/gzip and
libaudiofile
Grepping a bit through header files, I get to note that zlib itself
uses off_t for its function pair of gztell/gzseek. The zlib on my
system is compiled as 32bit-off_t while the gzip utility is compiled
as 64bit-off_t - only by luck the gzip does not use the function pair
that got affected. So... sending a bug report to zlib-at-gzip-org.
Furthermore, I notice that in mandrake9 about all files have some
dependency on libaudiofile which makes heavy use of off_t in its
header files but has not guarded against largefile-sensitivy. Again,
sending a bug report to the oss.sgi.com mailinglist for it.
- 2003-01-07
GFileManager /
LinNeighborhood Bugs
Sending a bug-report mail to the maintainers of these projects that
they have a problem - these frontends are compiled as 32bit off_t.
See below that Samba has already
implemented 64bit SMB extensions, and even the libsmbclient software
is compiled as 64bit-off_t with it.
- 2003-01-06
http://article.gmane.org/gmane.comp.sysutils.autoconf.general/1893
Making the LARGEFILE_SENSITIVE macro available on the
ac-archive.sf.net site,
and starting a thread on the
autoconf
Mailinglist about largefile support.
Note that samba uses a much more elaborate autoconf check to enable
64bit-off_t support in the various systems that goes beyond what the
original autoconf macro AC_SYS_LARGEFILE does provide.
http://cvs.samba.org/cgi-bin/cvsweb/samba/source/configure.in?rev=1.383
- 2003-01-05
http://freshmeat.net/releases/108451/
Making zziplib 0.10.77 into a
dualmode library offering both 32bit and 64bit off_t interfaces.
- 2003-01-03
http://unix.org/version2/unix98fm.html
Checking a bit deeper, I learn that Unix98-system are actually
required to support largefile system - it just happens that
some systems still use a 32bit off_t default.
http://unix.org/version2/whatsnew/lfs.html
more links:
http://unix.org/whitepapers/64bit.html
http://www.google.de/search?q=_XBS5_ILP32_OFFBIG
http://docsrv.caldera.com/cgi-bin/man/man?Intro+2#LargeFileSupport
http://www.llnl.gov/asci/platforms/bluepac/large_file_prog.html
http://www.google.de/search?q=_LARGEFILE_SOURCE
- 2003-01-03
http://groups.google.com/groups?q=large.file.support.in.mandrake.9
The original incident that made me think about the problemspace - a user
of LinNeighborhood had
problems to copy files to his local computer when they are slightly
bigger than 2GiB - in fact, this graphical frontend did show the
filesize numbers all incorrect, and even that copying started, it broke
after somewhat beyond 2GiB.