[syslinux] [RFC] COMBOOT: readdir: st_mode or d_type

[Gene Cumm]

On Mon, Mar 2, 2009 at 5:49 PM, Jeffrey Hutzelman <jhutz at cmu.edu> wrote:
> --On Sunday, March 01, 2009 05:31:23 PM -0500 Gene Cumm
> <gene.cumm at gmail.com> wrote:
>
>>> Since you don't have anything resembling UNIX users, there's no use for
>>> UNIX access modes and really very little point in emulating them, other
>>> than for informational purposes.  So...
>>
>> It's either emulate them at the COMBOOT level or the COM32 level.
>
> Why?  As far as I can tell, the only possible use is for displaying
> information to the user, so we should be as informative as possible
> _and no more_.  That means providing UNIX modes is fine, but that
> trying to wedge a representation of something else into them is of
> limited use, unless the user can tell what you meant.  So, reporting
> the real UNIX mode from filesystems that have them is useful.  And
> reporting the presence of the read-only bit on FAT is useful.
>
>>  The
>> COM32 APIs I've built have been aimed at a Unix-like environment in
>> that the APIs are compatible.  Not multi-user, of course.  I'd either
>> have to stick with a Unix-compatible format at the COMBOOT level or
>> make my own for all file systems at the COMBOOT level.
>
> The UNIX directory-reading API's do not report information about each file,
> other than its inode number and, maybe, a very coarse type.  The reason is
> actually the same reason your API should not do so -- getting any
> additional information is _expensive_, because it requires locating and
> reading in the inode.
>
> That said, I certainly don't object to returning this information, if it
> can be obtained cheaply.
>
>
>> The other part of the reasoning was that implementing it for EXTLINUX
>> would be essentially no code (copy data to register and done).
>
> Actually, it would (should).  If you're reading a directory, you have only
> the directory entry, which does not contain this information.  Getting the
> mode requires looking up the inode; it can't be stored in the directory
> because it's mutable and an inode doesn't contain information about where
> all of the directory entries are that point to it, so updating them when it
> changed would be prohibitively expensive.
>
>>> If the filesystem stores a UNIX mode, it should be reported as-is.
>>> Otherwise, report everything as either 777 or 555, depending on whether
>>> the file is writable or not (that is, for filesystems like FAT that have
>>> a "readonly" bit, report that by turning off the 222 bits.
>>
>> Except for the fact that fstat() already uses 0444.  That would sort
>> of break the pattern if 0111 was added.
>
> Yeah, you could use 444/666 instead.  When actually emulating for something
> that's going to do access control based on those bits, the 111 bits are
> important because without them you can't execute anything.  In this case,
> consistency with the existing fstat() is clearly more important.
>
>>> I would also note that returning the UNIX mode from readdir is not nearly
>>> as useful as returning filetype information.  So if you have to choose,
>>> return the filetype.
>>
>> Which filetype are you suggesting?  Unix struct dirent d_type, raw FAT
>> filetype (only useful for FAT, not ISO-9660 or EXT2/3) or a custom
>> format?  It needs to be usable for FAT, ISO-9660, EXT2/3 and any other
>> fielsystem that the Syslinux project may support in the future.
>
> Something along the lines of d_type.  Roughly, "file", "directory", maybe
> "symlink", or "thing we have no support for and cannot access" (which
> covers device files, UNIX-domain sockets, FIFO's, and probably some special
> types in other filesystems as well).

Well, you've made me think about what was happening with what I'm
trying to accomplish and how the inner workings of each file system in
question are implemented.

The most important thing to return is the filename.  Then the basic
file type (the list you had was good: file, folder, symlink and other)
and its size followed by read-only (only useful for the user in a case
of "Oh, I made the file I want read-only but got the name wrong" as we
will _never_ write anything except the ADV or other dedicated
structure; the "what-if" of broken BIOSes that Peter brought up a
while ago) and last, maybe its full file type, something resembling an
inode number, and a date (probably modified).

If I'm reading things correctly, this is what I can gather.  FAT and
ISO9660 dirents contain name, size, type and any other attributes.
EXT2, being inode-based, stores only the inode number and the name in
the dirent while the type, size and other attributes are all in the
inode.  In all three, dirents from a directory are grouped together.
EXT2 inodes are yet a different category in where and how they are
stored.

Based upon this, it's debatable which way is better:
1) provide everything from readdir, providing better performance for
FAT or ISO9660 in the case of a cache miss.
2) provide only the name but then also provide a stat() function, either by
a) using open(), fstat(), close(), opendir() and closedir(), (simple
implementation, costly execution) or
b) providing a stat capability in the COMBOOT API;
Either way, giving only names and a stat()/fstat() function only
provides an advantage to EXT2 when only the names are requested (ls by
itself, not long, not classified, etc).

There is minimal difference in the resource cost of return just one
attribute (other than name and inode) like size versus all attributes
(which is more than most may want) as the biggest resource cost is in
retrieving a disk sector after a cache miss.  There is a difference
between attributes and name/inode but only in (most) Inode
filesystems.  As you've said, some file systems do store a basic type
in the dirent but it looks like EXT2 is not one of them.  If it did,
it would be all we need and that would probably be the preferred
route.

At the moment, the COM32 C struct dirent can hold all of this
information, far more than most UNIX systems.  Thinking about what
you've said, it may be beneficial to either remove those members or
insert a flag stating if they're valid.  Doing so would mean that for
FAT and ISO9660, the data could be in a dirent but EXT2 might require
a stat()/fstat() call.

-- 
-Gene