From 849369d6c66d3054688672f97d31fceb8e8230fb Mon Sep 17 00:00:00 2001 From: root Date: Fri, 25 Dec 2015 04:40:36 +0000 Subject: initial_commit --- Documentation/DocBook/filesystems.tmpl | 426 +++++++++++++++++++++++++++++++++ 1 file changed, 426 insertions(+) create mode 100644 Documentation/DocBook/filesystems.tmpl (limited to 'Documentation/DocBook/filesystems.tmpl') diff --git a/Documentation/DocBook/filesystems.tmpl b/Documentation/DocBook/filesystems.tmpl new file mode 100644 index 00000000..f51f2853 --- /dev/null +++ b/Documentation/DocBook/filesystems.tmpl @@ -0,0 +1,426 @@ + + + + + + Linux Filesystems API + + + + This documentation is free software; you can redistribute + it and/or modify it under the terms of the GNU General Public + License as published by the Free Software Foundation; either + version 2 of the License, or (at your option) any later + version. + + + + This program is distributed in the hope that it will be + useful, but WITHOUT ANY WARRANTY; without even the implied + warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. + See the GNU General Public License for more details. + + + + You should have received a copy of the GNU General Public + License along with this program; if not, write to the Free + Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, + MA 02111-1307 USA + + + + For more details see the file COPYING in the source + distribution of Linux. + + + + + + + + The Linux VFS + The Filesystem types +!Iinclude/linux/fs.h + + The Directory Cache +!Efs/dcache.c +!Iinclude/linux/dcache.h + + Inode Handling +!Efs/inode.c +!Efs/bad_inode.c + + Registration and Superblocks +!Efs/super.c + + File Locks +!Efs/locks.c +!Ifs/locks.c + + Other Functions +!Efs/mpage.c +!Efs/namei.c +!Efs/buffer.c +!Efs/bio.c +!Efs/seq_file.c +!Efs/filesystems.c +!Efs/fs-writeback.c +!Efs/block_dev.c + + + + + The proc filesystem + + sysctl interface +!Ekernel/sysctl.c + + + proc filesystem interface +!Ifs/proc/base.c + + + + + Events based on file descriptors +!Efs/eventfd.c + + + + The Filesystem for Exporting Kernel Objects +!Efs/sysfs/file.c +!Efs/sysfs/symlink.c +!Efs/sysfs/bin.c + + + + The debugfs filesystem + + debugfs interface +!Efs/debugfs/inode.c +!Efs/debugfs/file.c + + + + + + The Linux Journalling API + + + + Roger + Gammans + +
+ rgammans@computer-surgery.co.uk +
+ + + + + + + Stephen + Tweedie + +
+ sct@redhat.com +
+ + + + + + 2002 + Roger Gammans + + + + The Linux Journalling API + + + Overview + + Details + +The journalling layer is easy to use. You need to +first of all create a journal_t data structure. There are +two calls to do this dependent on how you decide to allocate the physical +media on which the journal resides. The journal_init_inode() call +is for journals stored in filesystem inodes, or the journal_init_dev() +call can be use for journal stored on a raw device (in a continuous range +of blocks). A journal_t is a typedef for a struct pointer, so when +you are finally finished make sure you call journal_destroy() on it +to free up any used kernel memory. + + + +Once you have got your journal_t object you need to 'mount' or load the journal +file, unless of course you haven't initialised it yet - in which case you +need to call journal_create(). + + + +Most of the time however your journal file will already have been created, but +before you load it you must call journal_wipe() to empty the journal file. +Hang on, you say , what if the filesystem wasn't cleanly umount()'d . Well, it is the +job of the client file system to detect this and skip the call to journal_wipe(). + + + +In either case the next call should be to journal_load() which prepares the +journal file for use. Note that journal_wipe(..,0) calls journal_skip_recovery() +for you if it detects any outstanding transactions in the journal and similarly +journal_load() will call journal_recover() if necessary. +I would advise reading fs/ext3/super.c for examples on this stage. +[RGG: Why is the journal_wipe() call necessary - doesn't this needlessly +complicate the API. Or isn't a good idea for the journal layer to hide +dirty mounts from the client fs] + + + +Now you can go ahead and start modifying the underlying +filesystem. Almost. + + + + +You still need to actually journal your filesystem changes, this +is done by wrapping them into transactions. Additionally you +also need to wrap the modification of each of the buffers +with calls to the journal layer, so it knows what the modifications +you are actually making are. To do this use journal_start() which +returns a transaction handle. + + + +journal_start() +and its counterpart journal_stop(), which indicates the end of a transaction +are nestable calls, so you can reenter a transaction if necessary, +but remember you must call journal_stop() the same number of times as +journal_start() before the transaction is completed (or more accurately +leaves the update phase). Ext3/VFS makes use of this feature to simplify +quota support. + + + +Inside each transaction you need to wrap the modifications to the +individual buffers (blocks). Before you start to modify a buffer you +need to call journal_get_{create,write,undo}_access() as appropriate, +this allows the journalling layer to copy the unmodified data if it +needs to. After all the buffer may be part of a previously uncommitted +transaction. +At this point you are at last ready to modify a buffer, and once +you are have done so you need to call journal_dirty_{meta,}data(). +Or if you've asked for access to a buffer you now know is now longer +required to be pushed back on the device you can call journal_forget() +in much the same way as you might have used bforget() in the past. + + + +A journal_flush() may be called at any time to commit and checkpoint +all your transactions. + + + +Then at umount time , in your put_super() (2.4) or write_super() (2.5) +you can then call journal_destroy() to clean up your in-core journal object. + + + +Unfortunately there a couple of ways the journal layer can cause a deadlock. +The first thing to note is that each task can only have +a single outstanding transaction at any one time, remember nothing +commits until the outermost journal_stop(). This means +you must complete the transaction at the end of each file/inode/address +etc. operation you perform, so that the journalling system isn't re-entered +on another journal. Since transactions can't be nested/batched +across differing journals, and another filesystem other than +yours (say ext3) may be modified in a later syscall. + + + +The second case to bear in mind is that journal_start() can +block if there isn't enough space in the journal for your transaction +(based on the passed nblocks param) - when it blocks it merely(!) needs to +wait for transactions to complete and be committed from other tasks, +so essentially we are waiting for journal_stop(). So to avoid +deadlocks you must treat journal_start/stop() as if they +were semaphores and include them in your semaphore ordering rules to prevent +deadlocks. Note that journal_extend() has similar blocking behaviour to +journal_start() so you can deadlock here just as easily as on journal_start(). + + + +Try to reserve the right number of blocks the first time. ;-). This will +be the maximum number of blocks you are going to touch in this transaction. +I advise having a look at at least ext3_jbd.h to see the basis on which +ext3 uses to make these decisions. + + + +Another wriggle to watch out for is your on-disk block allocation strategy. +why? Because, if you undo a delete, you need to ensure you haven't reused any +of the freed blocks in a later transaction. One simple way of doing this +is make sure any blocks you allocate only have checkpointed transactions +listed against them. Ext3 does this in ext3_test_allocatable(). + + + +Lock is also providing through journal_{un,}lock_updates(), +ext3 uses this when it wants a window with a clean and stable fs for a moment. +eg. + + + + + journal_lock_updates() //stop new stuff happening.. + journal_flush() // checkpoint everything. + ..do stuff on stable fs + journal_unlock_updates() // carry on with filesystem use. + + + +The opportunities for abuse and DOS attacks with this should be obvious, +if you allow unprivileged userspace to trigger codepaths containing these +calls. + + + +A new feature of jbd since 2.5.25 is commit callbacks with the new +journal_callback_set() function you can now ask the journalling layer +to call you back when the transaction is finally committed to disk, so that +you can do some of your own management. The key to this is the journal_callback +struct, this maintains the internal callback information but you can +extend it like this:- + + + struct myfs_callback_s { + //Data structure element required by jbd.. + struct journal_callback for_jbd; + // Stuff for myfs allocated together. + myfs_inode* i_commited; + + } + + + +this would be useful if you needed to know when data was committed to a +particular inode. + + + + + + Summary + +Using the journal is a matter of wrapping the different context changes, +being each mount, each modification (transaction) and each changed buffer +to tell the journalling layer about them. + + + +Here is a some pseudo code to give you an idea of how it works, as +an example. + + + + journal_t* my_jnrl = journal_create(); + journal_init_{dev,inode}(jnrl,...) + if (clean) journal_wipe(); + journal_load(); + + foreach(transaction) { /*transactions must be + completed before + a syscall returns to + userspace*/ + + handle_t * xct=journal_start(my_jnrl); + foreach(bh) { + journal_get_{create,write,undo}_access(xact,bh); + if ( myfs_modify(bh) ) { /* returns true + if makes changes */ + journal_dirty_{meta,}data(xact,bh); + } else { + journal_forget(bh); + } + } + journal_stop(xct); + } + journal_destroy(my_jrnl); + + + + + + + Data Types + + The journalling layer uses typedefs to 'hide' the concrete definitions + of the structures used. As a client of the JBD layer you can + just rely on the using the pointer as a magic cookie of some sort. + + Obviously the hiding is not enforced as this is 'C'. + + Structures +!Iinclude/linux/jbd.h + + + + + Functions + + The functions here are split into two groups those that + affect a journal as a whole, and those which are used to + manage transactions + + Journal Level +!Efs/jbd/journal.c +!Ifs/jbd/recovery.c + + Transasction Level +!Efs/jbd/transaction.c + + + + See also + + + + Journaling the Linux ext2fs Filesystem, LinuxExpo 98, Stephen Tweedie + + + + + + + Ext3 Journalling FileSystem, OLS 2000, Dr. Stephen Tweedie + + + + + + + + + splice API + + splice is a method for moving blocks of data around inside the + kernel, without continually transferring them between the kernel + and user space. + +!Ffs/splice.c + + + + pipes API + + Pipe interfaces are all for in-kernel (builtin image) use. + They are not exported for use by modules. + +!Iinclude/linux/pipe_fs_i.h +!Ffs/pipe.c + + + -- cgit v1.2.3