Contents |
DBIFS - The Database Filesystem for Fuse
DBIFS is a fully-threaded, database-agnostic filesystem for Fuse. DBIFS can use any database that it has permission to, that has a Perl DBI/DBD module.
Actively Developed:
- MySQL 5 (including MySQL Cluster)
- Oracle (including RAC)
Tried, Seem OK:
- LDAP
- ODBC
DBIFS is being designed with the goal of providing web application server clusters a potentially stupid fast, highly-available, highly-redundant, highly-scalable "file system" to do what they do: Serve the webby bits to the Internets.
DBIFS allows any number of systems to mount the file system at the same time, and comes with its own cluster daemon to properly handle:
- distributed file-locking
- fencing
- load awareness
You may be surprised or scoff at the fact that DBIFS is written in Perl instead of a traditional language like C. After the initial mount of the DBIFS, performance metrics of Perl vs. C for access to both MySQL and Oracle databases are comparable, with the Perl DBI interface winning out in a number of categories when considering the lines of written code necessary.
DBIFS is still in pre-release development.
NOTES
0 dev device number of filesystem
1 ino inode number
2 mode file mode (type and permissions)
3 nlink number of (hard) links to the file
4 uid numeric user ID of file’s owner
5 gid numeric group ID of file’s owner
6 rdev the device identifier (special files only)
7 size total size of file, in bytes
8 atime last access time in seconds since the epoch
9 mtime last modify time in seconds since the epoch
10 ctime inode change time (NOT creation time!) in seconds
since the epoch
11 blksize preferred block size for file system I/O
12 blocks actual number of blocks allocated
xattr list... name:value pairs
DLM
The Distibuted Lock Manager (DLM) is pretty damned simple. Listens on a port and launches threads to handle connection requests.
There is more than one way to tackle this. I chose semaphores and a spinlock, could be done with mutexes or all spins, or a number of other mechanisms. Assuming your lock durations are generally short (as they are IRL) and lock collisions sparse (as they are IRL), this method is exceptionally scalable.
Conventions
This DLM is written such that is can coordinate the locking of anything, not just files. You can "LOCK bob" if you want, as long as anything else that wants to operate on 'bob' knows that it may be locked, and requests a lock itself.
For the purposes of DBIFS, we're generally going to use the convention:
FILESYSTEM_NAME:/path/to/object
Where object could be a folder or a file. One thing to note is that the only locks we care about are write locks. A lock does not prevent a read for the simple reason that we're trusting the database to always give us a full-record, as that's why we mess around with RDBMSes to begin with, isn't it?
Engineering Debates
- There is some debate whether a lock should match anything "below" it. In other words should a lock of "/" actually write-lock the entire file system? Currently, it does not, nor have the debaters convinced me otherwise. It would be trivial to implement however, so it might show up as an option.
- Currently, DLM expects that once something connects to it and creates a lock, that thing will stay connected at least until it unlocks whatever it has locked. As such, upon disconnect all of the held locks are auto-unlocked. There is debate as to whether this is a fair assumption, and the flexibility should exist for Connect->Lock->Disconnect...Connect->Unlock scenarios should be supported. For the record, currently the lock can be unlocked by any connecting session.
Commands
LOCK objectname
The lock command creates a lock on an unlocked object, or spins (yielding politely each spin) until it can get a lock.
UNLOCK objectname
The unlock command removes a lock on a locked object. Does nothing if the object isn't already locked.
LIST
The list command lists all locks in "objectname -> threadID" format.
NOTE: The order in which the locks are listed is not indicative of anything, and shouldn't be used to assume chronology or anything other than existance.
QUIT
The quite command signals DLM that a client is done and the connect should be terminated.