1======================================================= 2Configfs - Userspace-driven Kernel Object Configuration 3======================================================= 4 5Joel Becker <joel.becker@oracle.com> 6 7Updated: 31 March 2005 8 9Copyright (c) 2005 Oracle Corporation, 10 Joel Becker <joel.becker@oracle.com> 11 12 13What is configfs? 14================= 15 16configfs is a ram-based filesystem that provides the converse of 17sysfs's functionality. Where sysfs is a filesystem-based view of 18kernel objects, configfs is a filesystem-based manager of kernel 19objects, or config_items. 20 21With sysfs, an object is created in kernel (for example, when a device 22is discovered) and it is registered with sysfs. Its attributes then 23appear in sysfs, allowing userspace to read the attributes via 24readdir(3)/read(2). It may allow some attributes to be modified via 25write(2). The important point is that the object is created and 26destroyed in kernel, the kernel controls the lifecycle of the sysfs 27representation, and sysfs is merely a window on all this. 28 29A configfs config_item is created via an explicit userspace operation: 30mkdir(2). It is destroyed via rmdir(2). The attributes appear at 31mkdir(2) time, and can be read or modified via read(2) and write(2). 32As with sysfs, readdir(3) queries the list of items and/or attributes. 33symlink(2) can be used to group items together. Unlike sysfs, the 34lifetime of the representation is completely driven by userspace. The 35kernel modules backing the items must respond to this. 36 37Both sysfs and configfs can and should exist together on the same 38system. One is not a replacement for the other. 39 40Using configfs 41============== 42 43configfs can be compiled as a module or into the kernel. You can access 44it by doing:: 45 46 mount -t configfs none /config 47 48The configfs tree will be empty unless client modules are also loaded. 49These are modules that register their item types with configfs as 50subsystems. Once a client subsystem is loaded, it will appear as a 51subdirectory (or more than one) under /config. Like sysfs, the 52configfs tree is always there, whether mounted on /config or not. 53 54An item is created via mkdir(2). The item's attributes will also 55appear at this time. readdir(3) can determine what the attributes are, 56read(2) can query their default values, and write(2) can store new 57values. Don't mix more than one attribute in one attribute file. 58 59There are two types of configfs attributes: 60 61* Normal attributes, which similar to sysfs attributes, are small ASCII text 62 files, with a maximum size of one page (PAGE_SIZE, 4096 on i386). Preferably 63 only one value per file should be used, and the same caveats from sysfs apply. 64 Configfs expects write(2) to store the entire buffer at once. When writing to 65 normal configfs attributes, userspace processes should first read the entire 66 file, modify the portions they wish to change, and then write the entire 67 buffer back. 68 69* Binary attributes, which are somewhat similar to sysfs binary attributes, 70 but with a few slight changes to semantics. The PAGE_SIZE limitation does not 71 apply, but the whole binary item must fit in single kernel vmalloc'ed buffer. 72 The write(2) calls from user space are buffered, and the attributes' 73 write_bin_attribute method will be invoked on the final close, therefore it is 74 imperative for user-space to check the return code of close(2) in order to 75 verify that the operation finished successfully. 76 To avoid a malicious user OOMing the kernel, there's a per-binary attribute 77 maximum buffer value. 78 79When an item needs to be destroyed, remove it with rmdir(2). An 80item cannot be destroyed if any other item has a link to it (via 81symlink(2)). Links can be removed via unlink(2). 82 83Configuring FakeNBD: an Example 84=============================== 85 86Imagine there's a Network Block Device (NBD) driver that allows you to 87access remote block devices. Call it FakeNBD. FakeNBD uses configfs 88for its configuration. Obviously, there will be a nice program that 89sysadmins use to configure FakeNBD, but somehow that program has to tell 90the driver about it. Here's where configfs comes in. 91 92When the FakeNBD driver is loaded, it registers itself with configfs. 93readdir(3) sees this just fine:: 94 95 # ls /config 96 fakenbd 97 98A fakenbd connection can be created with mkdir(2). The name is 99arbitrary, but likely the tool will make some use of the name. Perhaps 100it is a uuid or a disk name:: 101 102 # mkdir /config/fakenbd/disk1 103 # ls /config/fakenbd/disk1 104 target device rw 105 106The target attribute contains the IP address of the server FakeNBD will 107connect to. The device attribute is the device on the server. 108Predictably, the rw attribute determines whether the connection is 109read-only or read-write:: 110 111 # echo 10.0.0.1 > /config/fakenbd/disk1/target 112 # echo /dev/sda1 > /config/fakenbd/disk1/device 113 # echo 1 > /config/fakenbd/disk1/rw 114 115That's it. That's all there is. Now the device is configured, via the 116shell no less. 117 118Coding With configfs 119==================== 120 121Every object in configfs is a config_item. A config_item reflects an 122object in the subsystem. It has attributes that match values on that 123object. configfs handles the filesystem representation of that object 124and its attributes, allowing the subsystem to ignore all but the 125basic show/store interaction. 126 127Items are created and destroyed inside a config_group. A group is a 128collection of items that share the same attributes and operations. 129Items are created by mkdir(2) and removed by rmdir(2), but configfs 130handles that. The group has a set of operations to perform these tasks 131 132A subsystem is the top level of a client module. During initialization, 133the client module registers the subsystem with configfs, the subsystem 134appears as a directory at the top of the configfs filesystem. A 135subsystem is also a config_group, and can do everything a config_group 136can. 137 138struct config_item 139================== 140 141:: 142 143 struct config_item { 144 char *ci_name; 145 char ci_namebuf[UOBJ_NAME_LEN]; 146 struct kref ci_kref; 147 struct list_head ci_entry; 148 struct config_item *ci_parent; 149 struct config_group *ci_group; 150 struct config_item_type *ci_type; 151 struct dentry *ci_dentry; 152 }; 153 154 void config_item_init(struct config_item *); 155 void config_item_init_type_name(struct config_item *, 156 const char *name, 157 struct config_item_type *type); 158 struct config_item *config_item_get(struct config_item *); 159 void config_item_put(struct config_item *); 160 161Generally, struct config_item is embedded in a container structure, a 162structure that actually represents what the subsystem is doing. The 163config_item portion of that structure is how the object interacts with 164configfs. 165 166Whether statically defined in a source file or created by a parent 167config_group, a config_item must have one of the _init() functions 168called on it. This initializes the reference count and sets up the 169appropriate fields. 170 171All users of a config_item should have a reference on it via 172config_item_get(), and drop the reference when they are done via 173config_item_put(). 174 175By itself, a config_item cannot do much more than appear in configfs. 176Usually a subsystem wants the item to display and/or store attributes, 177among other things. For that, it needs a type. 178 179struct config_item_type 180======================= 181 182:: 183 184 struct configfs_item_operations { 185 void (*release)(struct config_item *); 186 int (*allow_link)(struct config_item *src, 187 struct config_item *target); 188 void (*drop_link)(struct config_item *src, 189 struct config_item *target); 190 }; 191 192 struct config_item_type { 193 struct module *ct_owner; 194 struct configfs_item_operations *ct_item_ops; 195 struct configfs_group_operations *ct_group_ops; 196 struct configfs_attribute **ct_attrs; 197 struct configfs_bin_attribute **ct_bin_attrs; 198 }; 199 200The most basic function of a config_item_type is to define what 201operations can be performed on a config_item. All items that have been 202allocated dynamically will need to provide the ct_item_ops->release() 203method. This method is called when the config_item's reference count 204reaches zero. 205 206struct configfs_attribute 207========================= 208 209:: 210 211 struct configfs_attribute { 212 char *ca_name; 213 struct module *ca_owner; 214 umode_t ca_mode; 215 ssize_t (*show)(struct config_item *, char *); 216 ssize_t (*store)(struct config_item *, const char *, size_t); 217 }; 218 219When a config_item wants an attribute to appear as a file in the item's 220configfs directory, it must define a configfs_attribute describing it. 221It then adds the attribute to the NULL-terminated array 222config_item_type->ct_attrs. When the item appears in configfs, the 223attribute file will appear with the configfs_attribute->ca_name 224filename. configfs_attribute->ca_mode specifies the file permissions. 225 226If an attribute is readable and provides a ->show method, that method will 227be called whenever userspace asks for a read(2) on the attribute. If an 228attribute is writable and provides a ->store method, that method will be 229called whenever userspace asks for a write(2) on the attribute. 230 231struct configfs_bin_attribute 232============================= 233 234:: 235 236 struct configfs_bin_attribute { 237 struct configfs_attribute cb_attr; 238 void *cb_private; 239 size_t cb_max_size; 240 }; 241 242The binary attribute is used when the one needs to use binary blob to 243appear as the contents of a file in the item's configfs directory. 244To do so add the binary attribute to the NULL-terminated array 245config_item_type->ct_bin_attrs, and the item appears in configfs, the 246attribute file will appear with the configfs_bin_attribute->cb_attr.ca_name 247filename. configfs_bin_attribute->cb_attr.ca_mode specifies the file 248permissions. 249The cb_private member is provided for use by the driver, while the 250cb_max_size member specifies the maximum amount of vmalloc buffer 251to be used. 252 253If binary attribute is readable and the config_item provides a 254ct_item_ops->read_bin_attribute() method, that method will be called 255whenever userspace asks for a read(2) on the attribute. The converse 256will happen for write(2). The reads/writes are bufferred so only a 257single read/write will occur; the attributes' need not concern itself 258with it. 259 260struct config_group 261=================== 262 263A config_item cannot live in a vacuum. The only way one can be created 264is via mkdir(2) on a config_group. This will trigger creation of a 265child item:: 266 267 struct config_group { 268 struct config_item cg_item; 269 struct list_head cg_children; 270 struct configfs_subsystem *cg_subsys; 271 struct list_head default_groups; 272 struct list_head group_entry; 273 }; 274 275 void config_group_init(struct config_group *group); 276 void config_group_init_type_name(struct config_group *group, 277 const char *name, 278 struct config_item_type *type); 279 280 281The config_group structure contains a config_item. Properly configuring 282that item means that a group can behave as an item in its own right. 283However, it can do more: it can create child items or groups. This is 284accomplished via the group operations specified on the group's 285config_item_type:: 286 287 struct configfs_group_operations { 288 struct config_item *(*make_item)(struct config_group *group, 289 const char *name); 290 struct config_group *(*make_group)(struct config_group *group, 291 const char *name); 292 void (*disconnect_notify)(struct config_group *group, 293 struct config_item *item); 294 void (*drop_item)(struct config_group *group, 295 struct config_item *item); 296 }; 297 298A group creates child items by providing the 299ct_group_ops->make_item() method. If provided, this method is called from 300mkdir(2) in the group's directory. The subsystem allocates a new 301config_item (or more likely, its container structure), initializes it, 302and returns it to configfs. Configfs will then populate the filesystem 303tree to reflect the new item. 304 305If the subsystem wants the child to be a group itself, the subsystem 306provides ct_group_ops->make_group(). Everything else behaves the same, 307using the group _init() functions on the group. 308 309Finally, when userspace calls rmdir(2) on the item or group, 310ct_group_ops->drop_item() is called. As a config_group is also a 311config_item, it is not necessary for a separate drop_group() method. 312The subsystem must config_item_put() the reference that was initialized 313upon item allocation. If a subsystem has no work to do, it may omit 314the ct_group_ops->drop_item() method, and configfs will call 315config_item_put() on the item on behalf of the subsystem. 316 317Important: 318 drop_item() is void, and as such cannot fail. When rmdir(2) 319 is called, configfs WILL remove the item from the filesystem tree 320 (assuming that it has no children to keep it busy). The subsystem is 321 responsible for responding to this. If the subsystem has references to 322 the item in other threads, the memory is safe. It may take some time 323 for the item to actually disappear from the subsystem's usage. But it 324 is gone from configfs. 325 326When drop_item() is called, the item's linkage has already been torn 327down. It no longer has a reference on its parent and has no place in 328the item hierarchy. If a client needs to do some cleanup before this 329teardown happens, the subsystem can implement the 330ct_group_ops->disconnect_notify() method. The method is called after 331configfs has removed the item from the filesystem view but before the 332item is removed from its parent group. Like drop_item(), 333disconnect_notify() is void and cannot fail. Client subsystems should 334not drop any references here, as they still must do it in drop_item(). 335 336A config_group cannot be removed while it still has child items. This 337is implemented in the configfs rmdir(2) code. ->drop_item() will not be 338called, as the item has not been dropped. rmdir(2) will fail, as the 339directory is not empty. 340 341struct configfs_subsystem 342========================= 343 344A subsystem must register itself, usually at module_init time. This 345tells configfs to make the subsystem appear in the file tree:: 346 347 struct configfs_subsystem { 348 struct config_group su_group; 349 struct mutex su_mutex; 350 }; 351 352 int configfs_register_subsystem(struct configfs_subsystem *subsys); 353 void configfs_unregister_subsystem(struct configfs_subsystem *subsys); 354 355A subsystem consists of a toplevel config_group and a mutex. 356The group is where child config_items are created. For a subsystem, 357this group is usually defined statically. Before calling 358configfs_register_subsystem(), the subsystem must have initialized the 359group via the usual group _init() functions, and it must also have 360initialized the mutex. 361 362When the register call returns, the subsystem is live, and it 363will be visible via configfs. At that point, mkdir(2) can be called and 364the subsystem must be ready for it. 365 366An Example 367========== 368 369The best example of these basic concepts is the simple_children 370subsystem/group and the simple_child item in 371samples/configfs/configfs_sample.c. It shows a trivial object displaying 372and storing an attribute, and a simple group creating and destroying 373these children. 374 375Hierarchy Navigation and the Subsystem Mutex 376============================================ 377 378There is an extra bonus that configfs provides. The config_groups and 379config_items are arranged in a hierarchy due to the fact that they 380appear in a filesystem. A subsystem is NEVER to touch the filesystem 381parts, but the subsystem might be interested in this hierarchy. For 382this reason, the hierarchy is mirrored via the config_group->cg_children 383and config_item->ci_parent structure members. 384 385A subsystem can navigate the cg_children list and the ci_parent pointer 386to see the tree created by the subsystem. This can race with configfs' 387management of the hierarchy, so configfs uses the subsystem mutex to 388protect modifications. Whenever a subsystem wants to navigate the 389hierarchy, it must do so under the protection of the subsystem 390mutex. 391 392A subsystem will be prevented from acquiring the mutex while a newly 393allocated item has not been linked into this hierarchy. Similarly, it 394will not be able to acquire the mutex while a dropping item has not 395yet been unlinked. This means that an item's ci_parent pointer will 396never be NULL while the item is in configfs, and that an item will only 397be in its parent's cg_children list for the same duration. This allows 398a subsystem to trust ci_parent and cg_children while they hold the 399mutex. 400 401Item Aggregation Via symlink(2) 402=============================== 403 404configfs provides a simple group via the group->item parent/child 405relationship. Often, however, a larger environment requires aggregation 406outside of the parent/child connection. This is implemented via 407symlink(2). 408 409A config_item may provide the ct_item_ops->allow_link() and 410ct_item_ops->drop_link() methods. If the ->allow_link() method exists, 411symlink(2) may be called with the config_item as the source of the link. 412These links are only allowed between configfs config_items. Any 413symlink(2) attempt outside the configfs filesystem will be denied. 414 415When symlink(2) is called, the source config_item's ->allow_link() 416method is called with itself and a target item. If the source item 417allows linking to target item, it returns 0. A source item may wish to 418reject a link if it only wants links to a certain type of object (say, 419in its own subsystem). 420 421When unlink(2) is called on the symbolic link, the source item is 422notified via the ->drop_link() method. Like the ->drop_item() method, 423this is a void function and cannot return failure. The subsystem is 424responsible for responding to the change. 425 426A config_item cannot be removed while it links to any other item, nor 427can it be removed while an item links to it. Dangling symlinks are not 428allowed in configfs. 429 430Automatically Created Subgroups 431=============================== 432 433A new config_group may want to have two types of child config_items. 434While this could be codified by magic names in ->make_item(), it is much 435more explicit to have a method whereby userspace sees this divergence. 436 437Rather than have a group where some items behave differently than 438others, configfs provides a method whereby one or many subgroups are 439automatically created inside the parent at its creation. Thus, 440mkdir("parent") results in "parent", "parent/subgroup1", up through 441"parent/subgroupN". Items of type 1 can now be created in 442"parent/subgroup1", and items of type N can be created in 443"parent/subgroupN". 444 445These automatic subgroups, or default groups, do not preclude other 446children of the parent group. If ct_group_ops->make_group() exists, 447other child groups can be created on the parent group directly. 448 449A configfs subsystem specifies default groups by adding them using the 450configfs_add_default_group() function to the parent config_group 451structure. Each added group is populated in the configfs tree at the same 452time as the parent group. Similarly, they are removed at the same time 453as the parent. No extra notification is provided. When a ->drop_item() 454method call notifies the subsystem the parent group is going away, it 455also means every default group child associated with that parent group. 456 457As a consequence of this, default groups cannot be removed directly via 458rmdir(2). They also are not considered when rmdir(2) on the parent 459group is checking for children. 460 461Dependent Subsystems 462==================== 463 464Sometimes other drivers depend on particular configfs items. For 465example, ocfs2 mounts depend on a heartbeat region item. If that 466region item is removed with rmdir(2), the ocfs2 mount must BUG or go 467readonly. Not happy. 468 469configfs provides two additional API calls: configfs_depend_item() and 470configfs_undepend_item(). A client driver can call 471configfs_depend_item() on an existing item to tell configfs that it is 472depended on. configfs will then return -EBUSY from rmdir(2) for that 473item. When the item is no longer depended on, the client driver calls 474configfs_undepend_item() on it. 475 476These API cannot be called underneath any configfs callbacks, as 477they will conflict. They can block and allocate. A client driver 478probably shouldn't calling them of its own gumption. Rather it should 479be providing an API that external subsystems call. 480 481How does this work? Imagine the ocfs2 mount process. When it mounts, 482it asks for a heartbeat region item. This is done via a call into the 483heartbeat code. Inside the heartbeat code, the region item is looked 484up. Here, the heartbeat code calls configfs_depend_item(). If it 485succeeds, then heartbeat knows the region is safe to give to ocfs2. 486If it fails, it was being torn down anyway, and heartbeat can gracefully 487pass up an error. 488