[syslinux] ISOLINUX and readdir: Performance

[Gene Cumm]

On Sat, Mar 28, 2009 at 4:26 PM, H. Peter Anvin <hpa at zytor.com> wrote:
> Gene Cumm wrote:
>> Since my last e-mail, I've done some more exploring on the code.  I've
>> discovered that all variants currently use at least 3 64 kiB segments.
>>
>> I was surprised to see that ISOLINUX does not implement any cache
>> segment yet SYSLINUX and EXTLINUX, the disk-based variants do.  Are
>> there any ideas on why the faster disk-based variants have cache?  Is
>> it as simple as no one has attempted to dedicate the time to it?
>
> Pretty much.  As you may notice, too, EXTLINUX doesn't cache the
> contents of directories, either.

OK.

> The big issue is actually that it means having to have a scheme for
> separating cacheable and noncachable reads -- we don't have enough of a
> cache to make everything worth caches.
>

Hmm... basically need to specify this type of data block should be
cachable while another is not.  If a function is going to request a
data block, the function will know the type and should then call for
the block according to the scheme.

What about adding another parameter to getcachesector?  If the
parameter states that it is cachable, store it as normal.  If not
cachable, store it either in a designated volatile cache block or as
the least recently used block.  This would allow the caller to not
need their own buffer block but the non-cachable block would be less
likely to be in cache very long compared to the rest of the cached
blocks and at least be overwritten the next time some function needs a
non-cachable block.  If I have readdir call getcachesector with the
non-cacheable flag set and call readdir 10 times in a row with no
other disk requests, it will have cache hits on the 9 later calls.

If a function has their own buffer, it could either call
getcachesector (probably with the non-cachable flag; cache is too
valuable if you have your own buffer) and copy to its own buffer (a
little wasteful for memory copies but not too bad) or call the
FETCH_SECTOR_TO_BUFFER function (getonesec, getonesec_ext, getlinsec)
directly (the latter probably being the preferred route).

>> With that in mind and a statement by HPA that performance is almost a
>> non-issue except when it comes to disk-like device I/O (floppy, hdd,
>> optical CD/DVD) and graphics, it seems that it may be time to
>> implement the cache.  If the cache were to be implemented, it would
>> either have to have pass through routines when in debug mode ("%define
>> DEBUG_MESSAGES 1") to reduce size or some of the uninitialized data in
>> the first segment needs to move elsewhere, probably a new data segment
>> (for a total of 5 segments in ISOLINUX).
>
> I don't think that helps -- the issue with DEBUG_MESSAGES is mainly
> overflow of the first (2048 byte) sector.
>

Which files should I examine to verify this and how?  The .lsr files
in the second column (the 32b Hex number with leading 0s) and finding
rl_checkpt? It looks like the issue is .adv moving up but I'm
uncertain if I'm examining the correct files or examining the files
correctly.

-Gene