Notes on the storage of attributes. =================================== Resident attributes =================== Resident attributes are small enough to be stored in the mft record itself. When an attribute becomes too big to fit in the mft record or when the number of attributes grows so large that there is no more space in the mft record, the largest attribute(s) is(are) made non-resident. (Note, that some attributes cannot be non-resident.) Non-resident attributes ======================= Non-resident attributes contain only their non-resident attribute header in the mft record and the actual attribute value is stored anywhere on the volume (but not in other mft records). The value of the attribute is saved as a sequence of clusters, named virtual clusters, beginning at virtual cluster number (vcn) zero. Each vcn corresponds to a cluster on the volume, or a logical cluster number (lcn). The lcns on the volume begin with lcn zero for the very first cluster on the volume. The location of the attribute value is described by the mapping pairs array present in the non-resident attribute header structure. The mapping pairs array contains, in compressed form, the mapping between the attribute's vcns and the corresponding lcns on the volume. When the mapping pairs array of a non-resident attribute becomes too large to fit in the mft record or when there are so many attributes in the mft record that, even though all attributes that can be non-resident have been made non-resident, they still do not fit in the mft record, the larger non-resident attributes are moved away from the mft record to make space. This is done by moving whole non-resident attribute header structures to other mft records and/or by splitting the mapping pairs array of attributes into several non-resident attribute headers, each containing a chunk of the original mapping pairs array. These non-resident attribute headers each describing the same attribute value (but different parts of it) are called extents. Attribute list attribute ======================== The attribute list attribute is then added to the base mft record to describe the location of each attribute. The attribute list attribute lists all attributes that belong to this mft record (with the exception of itself). Each entry in the attribute list describes the attribute listed and in which mft record its attribute header can be found. For resident attributes this will be the same number as the base mft record in which the attribute list attribute is located itself. For non-resident attributes, this will be another mft record, called an extension mft record. Naturally, all extension mft records point back to their base mft record. Only one attribute is stored in an extension mft record, even if the attribute is very small. At least this is the case with Windows NT4 SP6a driver. Should the mapping pairs array of an attribute become so large as to not fit into an extenstion mft record, even though its attribute is the only attribute in this extension record, then the attribute is splitt into several extents. The first extent starts at vcn 0 and has its lowest vcn value set to zero and continues up to its highest vcn value. This is determined by splitting up the mapping pairs array into chunks which just fit into an extension mft record each. Thus the first mapping pairs array chunk will determine the value of the highest vcn for the first extent. The attribute list will contain an entry for this extent. Then, a second extent is created which has its lowest vcn value set to the highest vcn of the previous extent + 1 and the next chunk of the mapping pairs array is inserted into this extent. Again, an entry for this extent is placed into the attribute list, and so on, until the whole non-resident attribute's mapping pairs array has been accomodated. Should the attribute list become too big to fit inside the base mft record it is made non-resident. However, it's maximum value size is determined by the Windows cache manager to be 256kb (the size of a VACB). Also, the mapping pairs array of the attribute list has to fit inside the base mft record, as an attribute list can't be described by itself and attribute lists are not allowed to be nested. Compressed attributes ===================== The attribute value of each $DATA attribute can be compressed to save space. If this is the case, the ATTR_IS_COMPRESSED flag is set. See the discussion on compression in include/attrib.h for more details. FIXME: The discussion belongs here! (AIA) Sparse attributes (NTFS 3.0+) ============================= The attribute value of each $DATA attribute can be sparse to save space. If this is the case, the ATTR_IS_SPARSE flag is set. Note, that compressed attributes are by definition sparse, as well as compressed, without having the ATTR_IS_SPARSE flag set. See the discussion on compression in include/attrib.h for more details. FIXME: The discussion belongs here! (AIA) Encrypted attributes (NTFS 3.0+) ================================ Since NTFS 3.0, the attribute value of each $DATA attribute can be encrypted, to protect the contents from spying eyes. If this is the case, the ATTR_IS_ENCRYPTED flag is set. FIXME: Write notes on attribute encryption. The discussions from the articles "Inside the Encrypting File System" in Windows NT magazine (?) are very good starting points. (AIA)
