============================= FUSE-Python_ bindings new API ============================= .. _FUSE-Python: http://fuse.sourceforge.net/wiki/index.php/FusePython :Author: Csaba Henk :copyright: This document has been placed in the public domain. I've made several changes on the FUSE Python interface as we knew it. We will review here how this effects the usage of the module -- both from the end user and the developer POV. *This is not a reference.* This document just wants to show the big picture. If you want to write code using this module, then read this document first, and then take a look at the example filesystems under ``example/`` (``example/xmp.py`` is a pretty complete demo of the usage of the FUSE binding). For the rest, you can get away with the usual resources like in-code documentation (ie., docstrings) and the code itself. .. contents:: Old API =================== Enforcing compatibility ----------------------- There are lot of existing Python based FUSE based filesystems out there. They won't work with the current fuse-py code as is; however, we'd like to keep on using them. What can we do? Easy it is: just set ``fuse.fuse_python_api`` to ``(0, 1)`` before you invoke your filesystem. This can be achieved externally too, by setting the ``FUSE_PYTHON_API`` environment variable to ``0.1``. [#]_ [#]_ .. [#] Setting ``fuse.compat_0_1`` to ``True`` or having ``FUSE_PYTHON_COMPAT = 0.1`` in the environment still works, but it's deprecated. .. [#] Cf. `Long-term compatibility`_. What's incompatible, anyway? ---------------------------- - ``Fuse`` instance initialization - Handling of command line options - Transferring structured system data (file/filesystem attributes, directory entries) to the FUSE library - `getdir` fs method ditched in favor of `readdir` That is, to upgrade your filesystem to the new API, you will have to rewrite: - all your code between instantiating ``Fuse`` and calling the ``main()`` method of the instance (and the ``__init__()``, ``main()`` methods of your ``Fuse`` derived class, if you have overwritten the original) - the following fs methods: `getattr`, `statfs`, `getdir`. New API ======= The basic layout is the same: to start with, you get a class, ``fuse.Fuse``. You implement a filesystem by subclassing this class and add the filesystem as class methods. You can mount your filesystem class by calling the ``main()`` method of one of its instances. The list of possible filesystem methods is available as ``fuse.Fuse._attrs``. [#]_ So what's new? The new API has put emphasis on the following themes: - FUSE is a command line driven library. Handling ``sys.argv`` and the FUSE command line should be integrated. - Object based interface to system structures. - Wrap stateful I/O in OO. - Add support for all FUSE features which are available at the level the library is interfaced (the *high-level interface*). - Reflection. Let's see how these are implemented. .. [#] See ``examples/xmp.py`` for the argument list of the fs methods. Regarding return values: in each method, you can signal success or error by returning ``0`` (*succes*), a negative number (*error*, interpreted as negated errno), not returning anything (*success*) or by raising (not catching) an exception (*error*, Python infers an errno from the nature of the exception). For most methods this just does the job (eg., you don't need to pass back anything after handling an `mkdir` request, only success/error). Some others require data being passed on from your handler to FUSE (eg., `read`, `getattr`). Then just return with a suitable object (a string for `read`, a stat result alike for `getattr`, cf. `Simple objects to represent system structures`_). If the object is not suitable, then FUSE will signal an EINVAL to the system; you won't get feedback about it. (The badness shows up in FUSE's inner context, out of Python's, so raising an exception makes no sense. We could wrap some fs methods into format valifiers; currently we don't do that.) FUSE and the command line ------------------------- Crudely there can be two ways to provide configuration hooks to some C library: - One is having a config structure as a part of the API, which is to be filled and pass to some constructor or initializator when you start interfacing with the library. Such an interface can be easily used from C, but it is very fragile wrt. changing the config options. - Other is to use some kind of markup or domain specific language (*DSL*). This is flexible, but there should be provided a parser/generator for this language to be possible to make use of it. FUSE chose the latter way. Instead of using XML or some other widely used config format, FUSE made a simple decision: let the command line be our DSL -- we have to grok the command line anyway. [#]_ So, there are two command lines in the game. One is the actual command line (``sys.argv``), the other is the FUSE command line: the library can be initialized with an ``(argc, argv)`` pair. This makes the library user to want urgently: - A way to easily generate a FUSE compatible command line from an abstract spec. - A way to easily extract such an abstract spec from the actual command line. (... and these two procedures should interfere *only via the spec*.) The new API does this as follows: - Now it's the Python code's duty to put together a complete FUSE command line (in the form of a Python sequence). [#]_ - ``FuseArgs`` is the class for the abstract specification: the ``mountpoint``, ``set_mod()``, and ``add()`` attributes/methods enable you to set up such a beast; ``assemble()`` dumps a complete FUSE command line. - ``Fuse`` got a ``parser`` attribute. It's an instance of ``FuseOptParse``, which is derived from the ``OptionParser`` class of optparse_. [#]_ ``FuseOptParse`` groks a new kind of option (a subclass of ``Option``), which takes no short or long opts; it matches or not based on its ``mountopt`` attribute, which is looked for among the comma-separated members of a ``-o`` option. You can specify handlers these mountopts, just like to ordinary options. The unhandled suboptions are collected in a ``FuseArgs`` instance. - Calling ``Fuse``'s ``parse()`` method performs the parsing, and makes a note of the resulting ``FuseArgs`` instance. When you invoke ``Fuse``'s ``main()``, the FUSE command line will be inferred from this instance. .. [#] Originally this idea seemed as simple as there was no dedicated parser/generator interface provided with the library. With FUSE 2.5 we finally got the ``fuse_opt`` subAPI to make the command line more accessible. That's for C programming, so we don't deal with it here. .. _optparse: http://docs.python.org/lib/module-optparse.html .. [#] It wasn't like so: in earlier versions, Python passed down several partially parsed pieces of the FUSE command line to the C code, which used these directly in low level functions of the library, getting behind the main commandline parsing routine of the FUSE lib with no real reason. .. [#] To be precise, we have the ``SubbedOptParse`` subclass of ``OptionParser`` and ``FuseOptParse`` is further derived from ``SubbedOptParse``. ``SubbedOptParse`` is a generic class for parsing and handling suboptions. Simple objects to represent system structures --------------------------------------------- In old Pythons, ``os.stat()`` returned file attributes as a tuple, and for the convenient access of the stat values, you got a bunch of constats with it (so you queried file size like ``os.stat("foofile")[stat.ST_SIZE]``). While this approach still works, and if you print a stat result, it looks like a tuple, *it is, in fact, not a tuple*. It's an object which is immutable and provides the sequence protocol, just like tuple, but it has direct stat field accessors. That is, you can do it now like ``os.stat("foofile").st_size``. The same is the case with the FUSE bindings: for `getattr`, you are to return an object which has attributes like those of an ``os.stat()`` result, and for `statfs`, you are to return an object which has attributes like those of an ``os.statvfs()`` result. This, of course, can be achieved by calling ``os.stat()``, resp. ``os.statvfs()`` and passing on the result of this call. But you might feel like starting from scratch. You can build on the ``fuse.Stat`` and ``fuse.StatVfs`` classes. Subclass and/or instantiate them and specify the stat/statvfs attributes. Similarly, when listing directories, you have to return a sequence of ``fuse.Direntry`` objects which can be constructed from filenames (``fuse.Direntry("foofile")``). Does the above senctence make sense? I hope so. Anyway, *it's not true as is*. (Truth has been sacrified for making it short.) Don't worry, we uncover the lies immediately: - *You don't necessarily have to return a sequence*. You just have to return an object which implements the *iterator protocol*. In practice, this means that you can *yield* the direntries one by one, instead of aggregating them into a sequence. - The direntries don't have to be instances of ``fuse.Direntry``, they are just required to have some attributes. The ones other than ``name`` are probably not interesting for you. If you have large directories, you might want to specify a unique ``offset`` value for the direntries. This makes it possible for the system to read your dir in several chunks, and in each turn, reading can be continued from where it has been put off (for this to work, you have to be able to decode an ``offset`` and find the direntry which it belongs to). Filehandles can also be objects if you want ------------------------------------------- The FUSE library (and the Python new API) supports stateful I/O. That is, when you open a file, you can choose return an arbitrary object, a so called *filehandle*. [#]_ FUSE internally will allocate a (FUSE) filehandle upon open, and keep a record of your (Python) filehandle. When the system will want to use the FUSE filehandle for I/O, the respective Python method will get the (py-)filehandle as an argument. Ie., you can use the filehandle to preserve a state. You might as well want the filehandle to be an instance of a dedicated class, and want the filesystem methods get delegated to the filehandle. The new API can arrange this for you: set up a class, say ``Myfile``, which implements the I/O related methods (`read`, `write`, ...), and set ``foose.file_class = Myfile`` before calling ``foose.main()`` (where ``foose`` is an instance of ``Fuse``). This will also imply that the `open` fs method will be handled by instantiating ``Myfile``. Also note that the *path* argument will be stripped upon delegation (except for init time). You can do the same for directories, too. Directory I/O methods have similar names to file ones, just postfixed with `dir` (like `readdir`), and there are not that many of them (there is no `writedir`). You can register a directory class by setting the ``dir_class`` ``Fuse`` attribute. I bet you don't wanna use this feature, though. Another use of filehandles is that they can be used for adjusting some FUSE tunables *filewise*. That is, if you return a py-filehandle object so that it has a ``keep_cache`` or ``direct_io`` attribute of value ``True``, then the respective option will be enabled for the given file by FUSE [#]_. As a special case, if the returned py-filehandle is an instance of ``fuse.FuseFileInfo``, it will be used for nothing else apart from testing the ``keep_cache`` / ``direct_io`` attributes (after which it will be disposed). .. [#] although it should not be an integer, as integers are treated as error values .. [#] See the meaning of these options eg. in standard FUSE help message, which you can read by, eg., running ``example/xmp.py -h`` from the root of the FUSE Python bindings source tree. Complete support for hi-lib --------------------------- The Python bindings support all highlevel (pathname based) methods of the Fuse library as of API revision 26, including `create`, `access`, `flush` and *extended attributes*. Reflection ---------- In order to use the stateful I/O features as described above, the FUSE library on your system has to be recent enough. It's very likely that it will be so, as stateful I/O is around since a while, but if not... let's try proactively prevent cryptic bug reports. Therefore there is ``fuse.feature_assert()`` at your disposal. While there are several possible features you can assert, the form you will most likely use is ``feature_assert("stateful_files")``. This will raise an exception if stateful I/O on files is not supported. When it comes to reflection, we see that the command line based FUSE config machinery is sadly unidirectional [#]_. There is no simple way for querying the option list recognized by the lib. The best we have is that we can dump a help message. The new Python API tries to make use of this: it can mangle the help output into an instance of the aforementioned ``FuseArgs`` class (``Fuse.fuseoptref()``). The most convenient way to use this as follows: take a ``FuseArgs`` instance, eg. as its yielded by parsing with ``FuseOptParse``, and call its ``filter()`` method. This returns a new ``FuseArgs`` with the *rejected* options (which are not understood by the lib, according to the help message), and also purges out these from self, so the remainder can be safely passed down to FUSE. .. [#] We can argue that it's not that sad. We just pass on to FUSE what we get from the user and that either eats it or blows up. Why would we want more sophistication? Long-term compatibility ----------------------- You are suggested to declare explicitly the API you use by setting ``fuse.fuse_python_api`` to the value of ``fuse.FUSE_PYTHON_API_VERSION`` as it's definied in the fuse.py instance you code your filesystem against. This ensures that your code will keep working even if further API revisions take place.