[syslinux] [RFC] COMBOOT: readdir: st_mode or d_type

[Thomas Schmitt]

Hi,

> I think Syslinux can only buffer about 64k (1 segment) if I'm reading
> core/cache.inc correctly.  The next question is how much data would a
> user need cached if they were performing ls repeatedly on a normal
> size boot system.  I'd expect that it should be under half that but
> I'm uncertain.

Typically the whole directory tree is written before
any data block. So one can estimate its size from
the address of the lowest data block.

Some arbitray Live CDs which i got with Linux Magazin:

-  Grml 1.0 "Meilenschwein"
It has 57 directory entries. The lowest data block
address is 44 (file /boot/isolinux/isolinux.bin).
The directory can at most begin at block 32.
So it's 24 KB.
  
- Debian 4.0 rev0
1997 entries, lowest address is 955
(file /isolinux/isolinux.bin).
That's nearly 2 MB.

- Ubuntu 7.10 Server Edition
1998 entries, lowest address 1472
(/isolinux/isolinux.bin).
Nearly 3 MB.

- CentOS 5.1 Live CD
14 entries, lowest address 37
(/isolinux/isolinux.bin)
10 KB.


With less than 100 files, performance should
only be hampered by drive latency. It can need
several seconds until it wakes up from dozing.
2000 files with no caching at all might become
annoying, though.

Obstacles i encoutered when implementing the
reader of libisoburn:

- libisofs, the consumer, reads single 2 KB blocks.
  The drives usually read larger chunks.
  Some use their 1 or 2 MB RAM for read caching,
  some use it only for writing. Those are awfully
  slow when single 2 KB are read.

  So our reader software uses a larger read buffer
  size in order to enforce rudimentary caching.
  32 KB is minimum with DVD. 64 KB with BD.

- This buffer was read much too often because
  cache misses are frequent but often one of the
  next read addresses is again in the buffer
  that was just discarded.
 
  So the reader has not only one buffer tile
  in its cache but multiple ones. If a new
  one is to be read then the one is discarded
  which has not been hit for the longest time.
  16 tiles of 128 MB each turned out to be a
  good compromise between memory usage and read
  acceleration. Tested with images of about 1e5
  files.


Have a nice day :)

Thomas