Table of Contents
List of Figures
List of Tables
List of Examples
Directories
Table of Contents
EntityFS is a Java library for working with file system entities – files and directories – in an object-oriented fashion. For an introduction to EntityFS, see the Architecture overview.
This book is the Programmer's guide. It is written for programmers who want to use EntityFS in their applications. It gives an overview of EntityFS concepts and features, with examples and pointers to the API documentation, which serves as the reference documentation.
Feedback and contributions from users is essential for making EntityFS a better library. Please share your thoughts and opinions on EntityFS with the rest of the community through the mailing lists.
EntityFS is Copyright 2007-2009 Karl Gustafsson. It is licensed under the Gnu Lesser General Public License, version 3. If you want other licensing terms, contact Holocene Software – www.holocene.se.
Support for EntityFS can be found on the users mailing list. See the EntityFS site.
Table of Contents
Get EntityFS from http://www.entityfs.org.
There are two different distributions of EntityFS – the binary distribution and the source distribution. The binary distribution contains everything necessary for using EntityFS. The source distribution contains everything that the binary archive does as well as unit test classes, test data and the complete EntityFS source code in an Eclipse workspace.
Unzip the distribution into a directory.
EntityFS Jar files are in the lib
sub directory
of the distribution. The entityfs-full-1.2.jar
file contains all EntityFS classes, except for the unit tests. The other Jar files contains
different subsets of all EntityFS classes. Together they contain all of the classes in the "full" Jar. See the
README.txt
file in the lib
directory for a listing of what the different Jar files contain.
The unit test suite runs all EntityFS unit tests for many capability permutations for two different file systems. The total number of permutations for this release is 14534, so all unit tests take a long time to run – approximately four to five hours on a dual core CPU.
To run unit tests, you have to have the build tool
Schmant installed, the EntityFS source distribution
and some six gigabytes free space on the hard disk partition where the platform's
temporary directory (java.io.tmpdir
) is.
Open a command window (terminal, cmd) and change directory to the distribution's
build
directory. Set the
JAVA_HOME
environment variable to point to a
Java JDK 6 installation. (Just a JRE won't do.)
On Unix, run:
$ # For instance $ export JAVA_HOME=/opt/java6 $ schmant.sh -cp lib/entityfs_build.jar run_unit_tests.js
On Windows, run:
> rem For instance. Note the absence of quotes in JAVA_HOME > set JAVA_HOME=c:\Program Files\Java\jdk1.6.0_24 > schmant -cp lib\entityfs_build.jar run_unit_tests.js
To use one or several other JVM:s than the system's default for running unit
tests, set the javaCmds
property to a path
separator (: on Unix, ; on Windows) separated list of all Java installations to use.
$ # For instance $ export JAVA_HOME=/opt/java6 $ schmant.sh -cp lib/entityfs_build.jar \ -p javaCmds=/opt/java6/bin/java:/opt/jrockit6/bin/java run_unit_tests.js
Or, on Windows:
> rem For instance. Note the absence of quotes > set JAVA_HOME=c:\Program Files\Java\jdk1.6.0_24 > rem Note the quotes around javaHomes > schmant -cp lib\entityfs_build.jar \ -p "javaCmds=c:\Program\Java\jdk1.6.0_24\bin\javaw.exe;c:\Program\Java\jdk1.5.0_22\bin\javaw.exe" \ run_unit_tests.js
Don't be scared by all "errors" about disappeared files or files that will not be overwritten. They are from the tests that test that those conditions are handled correctly.
On Windows, anti-virus software may interfere with the tests. Run the tests with the anti-virus software disabled.
Run the performance test suite by opening a command window and change directory to the EntityFS distribution root directory.
On Unix, run:
java -server -Xmx2048M -Xincgc \ -cp lib/entityfs-full-1.2.jar:build/lib/entityfs_build.jar \ org.entityfs.build.perf.PerformanceTestRunner [Zip file]
Or, on Windows:
java -server -Xmx2048M -Xincgc \ -cp lib\entityfs-full-1.2.jar;build\lib\entityfs_build.jar \ org.entityfs.build.perf.PerformanceTestRunner [Zip file]
The Zip file argument is the path to any sufficiently large Zip file. The distribution's performance tests are run on a Zip file with the Linux kernel 2.6.30.1 sources.
Table of Contents
Well, what is a file, anyway? The answer is, of course, it depends. It depends on what you intend it to do. Sometimes a file is just a container of binary data that a client can read from, and sometimes it is a container of data with a unique identity that can be manipulated and moved around in the context of a file system.
EntityFS supports several different ways of viewing files and directories, all depending on what the client intends to do with them. This chapter describes the simple, low-level file and directory interfaces, Chapter 4, File systems describes what a file system is in EntityFS and Chapter 5, Entities describes what it means when a file or a directory is an entity.
Clients are encouraged to take the simplest possible view on the objects that they use. If a client just wants to read data from a file, it should use a ReadableFile instead of the full file entity (EFile). This makes mocking and testing much easier.
The sections below contain a walkthrough through the simple low-level interfaces for files and directories. Graphical inheritance charts are in Inheritance chart for EFile, Inheritance chart for Directory and Inheritance chart for all entity objcets.
Implemented by: Directory, EFile
Lockable is a marker interface that identifies the implementing object as being capable of locking for exclusive access by a thread or a process somehow.
Entity locking is described in Chapter 7, Locking and multi-threaded access.
Implements: Lockable.
Implemented by: Directory, EFile
ReadLockable gives the implementing object the capability of being locked for exclusive read access by a thread, several reading threads or a process. It is up to the implementing object to decide whether to actually support locking and, if so, the locking semantics to use.
Entity locking is described in Chapter 7, Locking and multi-threaded access.
Implements: Lockable.
Implemented by: Directory, EFile
WriteLockable gives the implementing object the capability of being locked for exclusive write access by a thread or a process. It is up to the implementing object to decide whether to actually support locking and, if so, the locking semantics to use.
Entity locking is described in Chapter 7, Locking and multi-threaded access.
Implements: Lockable, ReadLockable, WriteLockable.
Implemented by: Directory, EFile
ReadWriteLockable combines the ReadLockable and WriteLockable interfaces.
Implements: Lockable, ReadLockable.
Implemented by: Directory, EFile
Named is implemented by objects that has a name that can be read, such as a file name.
Implements: Lockable, WriteLockable.
Implemented by: Directory, EFile
Deletable is implemented by objects that can be deleted.
Implements: Lockable, ReadLockable.
Implemented by: EFile
ReadableFile defines a file that can be repeatedly opened for read only access and queried for its size. It is ReadLockable, so an implementing object may be able to be locked for exclusive read access, if it chooses to implement that.
In many ways, it is often better to use a ReadableFile as a method argument instead of, as is customarily done in Java libraries today, an InputStream. The reasons for that are explained in the Introduction section of the Architecture overview.
Table 3.1. Important ReadableFile implementations
Class | Description |
---|---|
ByteArrayReadableFile | A file that reads data from a byte array. |
CharSequenceReadableFile | A file that reads data from a CharSequence
(such as a String ). |
GZipReadableFile | A file containing data compressed with the gzip algorithm. The data is transparently decompressed when the file is read. |
Table 3.2. ReadableFile adapters
Class | Description |
---|---|
FileReadableFile | Adapts a File to the NamedReadableFile interface. |
NamedReadableFileAdapter | Adapts a ReadableFile to the NamedReadableFile interface. |
ReadWritableFileAdapter | Adapts a File to the ReadWritableFile interface. |
See also NamedReadableFile.
Implements: Lockable, Named, ReadableFile, ReadLockable.
Implemented by: EFile
The NamedReadableFile is a ReadableFile that has a name.
Table 3.3. Important NamedReadableFile implementations
Class | Description |
---|---|
ClasspathNamedReadableFile | A file that reads data from a resource on the classpath. |
ManualNamedReadableFile | A file that reads its data from a byte array or a
CharSequence (such as a
String ). |
UrlReadableFile | A file that reads its data from the target of a URL, such as a hyperlink. |
Table 3.4. NamedReadableFile adapters
Class | Description |
---|---|
FileReadableFile | Adapts a File to the NamedReadableFile interface. |
NamedReadableFileAdapter | Adapts a ReadableFile to the NamedReadableFile interface. |
ReadWritableFileAdapter | Adapts a File to the ReadWritableFile interface. |
Implements: Lockable, WriteLockable.
Implemented by: EFile
WritableFile defines a file that can be written or appended to. It is WriteLockable, so an implementing object may be able to be locked for exclusive write access, if it chooses to implement that.
In many ways, it is often better to use a WritableFile as a method argument instead of, as is customarily done in Java libraries today, an OutputStream. The reasons for that are explained in the Introduction section of the Architecture overview.
Table 3.5. Important WritableFile implementations
Class | Description |
---|---|
ByteArrayWritableFile | A file that writes data to a byte array. |
GZipWritableFile | A file that will gzip compress all data written to it. |
Table 3.6. WritableFile adapters
Class | Description |
---|---|
ReadWritableFileAdapter | Adapts a File to the ReadWritableFile interface. |
Implements: Lockable, ReadLockable, ReadWriteLockable, WriteLockable.
Implemented by: EFile
RandomlyAccessibleFile defines a file that can be opened for random access. It is ReadWriteLockable, so an implementing object may be able to be locked for exclusive read or write access, if it chooses to implement that.
EntityFS' RandomAccess object and not Java's RandomAccessFile is used to represent an open random access object. The API for RandomAccess is similar to RandomAccessFile, but it is an interface instead of a class.
Table 3.7. Important RandomlyAccessibleFile implementations
Class | Description |
---|---|
ByteArrayReadableFile | A read-only file that reads data from a byte array. |
Table 3.8. RandomlyAccessibleFile adapters
Class | Description |
---|---|
ReadWritableFileAdapter | Adapts a File to the ReadWritableFile interface. |
Implements: Deletable, Lockable, RandomlyAccessibleFile, ReadableFile, ReadLockable, ReadWriteLockable, WritableFile, WriteLockable.
Implemented by: EFile
ReadWritableFile combines all the other file interfaces and adds methods for copying data and for querying the file's size.
Table 3.9. ReadWritableFile adapters
Class | Description |
---|---|
ReadWritableFileAdapter | Adapts a File to the ReadWritableFile interface. |
Implements: Lockable, ReadLockable.
Implemented by: Directory
EntityHolder is a container that stores entities by their names. Clients can query the object for individual entities, but not list all entities that it contains.
Table 3.10. Important EntityHolder implementations
Class | Description |
---|---|
AggregatingEntityRepository | An EntityRepository that gives an aggregated view of the entities from zero or more other entity repositories. |
CompoundEntityHolder | Entity holder that combines zero or more other entity holders. |
Implements: Iterable<EntityView>, Lockable, ReadLockable.
Implemented by: Directory
EntityListable is a container that stores entities. Clients can use the object to get a list of the stored entities, but cannot query it for individual entities.
Table 3.11. Important EntityListable implementations
Class | Description |
---|---|
AggregatingEntityRepository | An EntityRepository that gives an aggregated view of the entities from zero or more other entity repositories. |
ManualEntityListable | Entity listable that uses a list of entities that can be manipulated by the client. |
Implements: EntityHolder, EntityListable, Iterable<EntityView>, Lockable, ReadLockable.
Implemented by: Directory
EntityRepository combines the EntityHolder and EntityListable interfaces to provide a read only repository for entities.
Table 3.12. Important EntityRepository implementations
Class | Description |
---|---|
AggregatingEntityRepository | An EntityRepository that gives an aggregated view of the entities from zero or more other entity repositories. |
The FileSystem interface represents a file system. It contains a directory hierarchy containing EFile and Directory entities, and it has a set of policy and strategy objects. It always holds a reference to the root Directory.
A FileSystem is created from a FileSystemBuilder object[1].
Table 4.1. File system implementations
Capabilities are discussed in Chapter 10, Capabilities
All setter methods of a FileSystemBuilder return the builder object. This makes it possibly to chain setter method invocations together as is shown in many of the examples below.
File systems have an optional name that, if set, should be unique within the current system. [2] The name can be used by applications to identify different file system instances. It is not used by EntityFS at all.
Example 4.1. Creating a file system-backed file system
// Create a file system with default settings except for a custom name FileSystem fs1 = new FSRWFileSystemBuilder(). setName("My first file system"). setRoot(new File("/tmp/fs1")). create(); // Create a file system with no event handling and relaxed validity controls // for entities. (See FSRWFileSystemBuilder API docs.) FileSystem fs2 = new FSRWFileSystemBuilder(). setName("My second file system"). setRoot(new File("/tmp/fs2")). disableEntityValidityControls(). // Enable entity locking enableLocking(). create();
Example 4.2. Creating a Jar file-backed file system on a file in another file system
// fs is another file system EFile jarFile = Directories.getFile(fs.getRootDirectory(), "myapp.jar"); FileSystem jfs = new JarFileSystemBuilder(). setName("My Jar file system"). setJarFile(jarFile). create();
By default, file systems are created with a minimal set of features. This can be tuned by configuring the file system builder differently. Also, the default validity control settings for the file system may be more defensive than what is required by the client. By setting less safe values, the file system's performance may increase somewhat.
The following is a list of common file system builder properties:
accessController
A file system can use an AccessController to limit access to entities. It can be used by applications to implement their access control policies. See Chapter 13, Access controls. Access controls are disabled by default.
bufferSize
The size of temporary in-memory buffers used by the file system. Different file system implementations use this for different things. The default value is 8192 bytes.
entityValidityControlStrategy
The strategy used for verifying that entities are still
present in the backend whenever they are used. The default strategy for this is
file system-specific. If it is known when the file system is created that no
other processes will ever modify its contents, this can be disabled by calling
disableEntityValidityControls
on the file
system builder.
events
By calling enableEvents
, a
client can make the entire FileSystem as well as individual entities
Observable for EntityEvent:s. See
Chapter 9, Entity events.
lockAcquiringStrategy
The strategy for acquiring file system
EntityLock:s. See
Chapter 7, Locking and multi-threaded access. Entity locking is
disabled by default. It can be
enabled by calling enableLocking
on the file system builder.
lockCommandExecutor
Utility class for locking entity locks in accordance with the
file system's locking strategy. Entity locking is disabled by default. It can be
enabled by calling enableLocking
on the file system builder. See
Chapter 7, Locking and multi-threaded access.
lockAdapterFactory
A factory used for creating locks. Entity locking can be
enabled by calling enableLocking
on the file system builder. See
Chapter 7, Locking and multi-threaded access.
Different file system implementations may also have their own, implementation-specific tunable parameters. For details, see the API documentation for each file system implementation.
File system implementations that use backend resources, for instance the Zip
file system, require that the client closes the file system when it is not
needed anymore. Calling close
makes the file
system release all open resources.
Table of Contents
A file system contains Entity:s such as files and directories. Entity objects are entities in the object-oriented sense of the word[3]. This means that, for instance for EFile (the same holds for Directory), as far as the client application is concerned, is the file itself. At any given time, in one FileSystem, there can only be one EFile object referencing that file. If the file is moved or renamed, it is still represented by the same EFile object. Compare this with how Java's File works.
An entity object always lives within the context of one and the same FileSystem. It cannot be moved to another file system.
The methods of file entities are defined by the EFile interface and the interfaces that it inherits. Directory methods are defined by the Directory interface and its inherited interfaces. The super interfaces of EFile and Directory each capture some aspect of files or directories. Just like when working with the Java Collections API, clients are recommended to use the simplest possible interface that meets their needs. For instance, a client just wanting to read from a file can use an implementation of the ReadableFile interface instead of the full EFile. This makes the client code easier to test since it is easier to replace a ReadableFile with a mock implementation. See Chapter 3, What is a file, anyway? and Figure 5.1, “Simplified inheritance chart for entity objects”.
Some entity types (Directory:s) are ViewCapable. For a Directory, this means that a client can create a view of the directory that uses one or several entity Filter:s to hide some of its child entities. EntityView:s such as the DirectoryView are not entities like the Entity objects; there can be several different active DirectoryView:s for a Directory entity, for instance.
CapabilityProvider:s may give entities or file systems extra capabilities, such as metadata support or support for entity attributes. See Chapter 10, Capabilities.
Many of the examples in this and the following chapters do entity locking as described in Chapter 7, Locking and multi-threaded access.
An entity's AbsoluteLocation, its absolute path, is its unique location in its file system. An entity also has a different RelativeLocation relative to every directory in its file system.
The Entity interface is what identifies an object as an entity, with all the properties discussed above. It defines all common properties and methods for entity objects. It is augmented by static utility methods implemented in the Entities and EntityViews classes.
For the complete interface inheritance chart, see the Inheritance chart for all entity objcets.
Example 5.1. Moving an entity in a locking file system
// d1 and d2 are directories, e1 is an entity of any type that currently is in // the directory d1. Collection<EntityLock> locks = new ArrayList<EntityLock>(); try { locks.add(d1.lockForWriting()); // The write lock on d1 is also a read lock. Get the entity to move. EntityView e1 = d1.getEntityOrNull("e1"); locks.add(e1.lockForWriting()); locks.add(d2.lockForWriting()); // Move the entity e1.rename(d2, "e1"); } finally { // Use the utility class EntityLocks to unlock all locks EntityLocks.unlockAll(locks); } // The entity is still represented by the variable e1, even though it has // moved.
Example 5.2. Moving an entity using utility methods
// d1 and d2 are directories, e1 is an entity of any type that currently is in // the directory d1. // // In a locking file system, the utility classes make sure to lock all entities // that need to be locked to perform the requested operation. See the chapter // on locking. EntityView e1 = Directories.getEntity(d1, "e1"); Entities.move(e1, d2);
In the second example, no locks are held on
d1
between e1
is retrieved and moved. Another thread may move or even delete
e1
between the calls to
getEntity
and move
.
If this is a real risk (this depends on the application), lock
d1
for writing before calling
getEntity
and release the lock after calling
move
. Locks are reentrant, so the locking
performed by the utility classes will not be disturbed by that. See
Chapter 7, Locking and multi-threaded access.
A Directory is a container for other entities, child entities. Basic directory operations are defined in the EntityHolder and EntityListable interfaces.
For the Directory inheritance chart, see the Inheritance chart for Directory.
The utility class Directories implements utility methods for working with directories.
Example 5.3. Creating a file in the root directory
// fs is a locking file system. Get the root directory Directory root = fs.getRootDirectory(); EFile myFile; // Lock the root directory for writing EntityLock wl = root.lockForWriting(); try { // Create an entity of the type ETFile.TYPE called myFile myFile = (EFile) root.newEntity(ETFile.TYPE, "myFile", null); } finally { // Unlock the directory wl.unlock(); }
Example 5.4. Creating a file in the root directory using Directories
// fs is a file system. Get the root directory Directory root = fs.getRootDirectory(); // Create a file called "myFile". EFile myFile = Directories.newFile(root, "myFile");
Example 5.5. Getting an entity in a subdirectory
// d is a directory // Create a directory under d with an empty file in it. This invocation // returns a reference to the file, but we ignore it for the sake // of the example Directories.newFile(org.entityfs.util.Directories.newDirectory(d, "d"), "f"); // Directories.getDirectory returns a DirectoryView. Since the argument d // is a Directory, the result can safely be cast to a Directory. Directory dd = (Directory) Directories.getDirectory(d, "d"); // (We could have used the entity reference returned by newFile above.) EFile f = Directories.getFile(d, new RelativeLocation("d/f"));
Example 5.6. Listing entities in a directory
Set<EntityView> s; // d is a directory in a locking file system. Lock it for reading. EntityLock rl = d.lockForReading(); try { s = d.listEntities(); } finally { // Unlock the directory. rl.unlock(); }
Example 5.7. Listing entities in a directory using
Directories
// d is a directory Set<EntityView> s = Directories.listEntities(d);
A DirectoryView is a directory viewed through one or several entity Filter:s. The filter hides all child entities that do not match the filter criteria. Read more about entity filters in the section called “Entity filters”.
Example 5.8. A directory view
// d is a directory EFile f1 = Directories.newFile(d, "f1"); DirectoryView d1 = Directories.newDirectory(d, "d1"); // Create a view that only shows files DirectoryView dv = d.newView(EFileFilter.FILTER); Set<EntityView> s = Directories.listEntities(dv);
The returned set will only contain f1
.
The Directory interface inherits DirectoryView, so all directories can be considered to be views of themselves with no filters.
All EntityListable objects, including directories and directory views, are
Iterable<EntityView>
. They have an
iterator()
method that returns an
Iterator<EntityView>
for iterating
over all visible child entities. This means that directories
can be used directly in a for loop like in the examples below.
Example 5.9. Using a Directory directly in a for loop
// d is a directory EntityLock rl = d.lockForReading(); try { for (EntityView ev : d) { // process child entities... } } finally { rl.unlock(); }
Example 5.10. Using a DirectoryView directly in a for loop
// d is a directory (or a view) // Create a view that only shows XML files. DirectoryView dv = d.newView(new EFileNameExtensionFilter("xml")); // Even though this is called on the view, it locks the viewed directory Lock rl = dv.lockForReading(); try { for (EntityView ev : dv) { EFile xmlFile = (EFile) ev; // process the XML files... } } finally { rl.unlock(); }
There are utility classes that implement other types of entity iterators, for instance recursive iterators over entity hierarchies. Read more about them in the section called “Recursive iterators” and the section called “Iterator utilities”.
Example 5.11. Using a depth-first iterator
// d is a directory Directory dd = (Directory) Directories.newDirectory(d, "d"); Directory ddd = (Directory) Directories.newDirectory(dd, "d"); EFile f = Directories.newFile(ddd, "f"); Iterator<EntityView> itr = Directories.getDepthFirstIterator(d);
The iterator will return the entities f
,
ddd
, dd
and
d
, in that order.
View settings are inherited. If a view capable child entity is retrieved from a directory view, the returned object is a view of that entity with the same filters (filter object instances) as the directory view.
Example 5.12. Views settings are inherited
// d is a directory EFile f1txt = Directories.newFile(d, "f1.txt"); EFile f1xml = Directories.newFile(d, "f1.xml"); Directory d1 = (Directory) Directories.newDirectory(d, "d1"); EFile d1f1txt = Directories.newFile(d1, "d1f1.txt"); EFile d1f1xml = Directories.newFile(d1, "d1f1.xml"); // Create a view that only shows directories and XML files DirectoryView dv = d.newView( DirectoryFilter.FILTER. or(new EFileNameExtensionFilter("xml"))); Iterator<EntityView> itr = Directories.getDepthLastIterator(dv);
itr
will return the entities
/
(view of), f1.xml
,
d1
(view of)
and d1f1.xml
(d1
may be returned before f1.xml
).
In the the inherited view example above, a FilteringIterator may be used if you don't want the iterator to return any directories. Create the iterator like this instead:
Iterator itr = new FilteringIterator( Directories.getDepthLastIterator(dv), DirectoryFilter.FILTER.not());
File entities are containers for byte data. Basic file operations are defined in the ReadableFile, WritableFile and RandomlyAccessibleFile interfaces. They are inherited by the file entity interface, EFile.
For the EFile inheritance chart, see the Inheritance chart for EFile.
Example 5.13. Creating and opening a file
// d is a directory EFile f = Directories.newFile(d, "f"); // Lock the file for writing EntityLock wl = f.lockForWriting(); try { OutputStream os = f.openForWrite(); try { try { ... write to os } finally { os.close(); } } catch (IOException e) { ... error handling } } finally { wl.unlock(); }
The class Files contains utility methods for working with file entities.
Example 5.14. Creating and writing to a text file
// d is a directory EFile f = Directories.newFile(d, "f"); Files.writeText(f, "Ceterum censeo Carthaginem esse delendam");
Sometimes an open input or output stream on a file has to be passed as an argument to another method. If using a locking file system, to make sure that the lock on the file is released when the stream is closed, use the lock-aware LockAwareInputStream or a lock-aware stream from Files.openForRead like in the example below.
Example 5.15. Using a lock-aware stream
// f is a file myStreamUsingMethod(Files.openForRead(f)); // The openForRead method locks the file for reading. The lock is released when // the stream is closed. (And that is a responsibility of the stream-using // class.)
Methods that only are interested in files as sources or sinks of data, not of their entity qualities, are encouraged to use the ReadableFile and WritableFile interfaces, just like most of the methods in Files do. They are easier to mock for testing than the EFile interface.
Table of Contents
All EntityFS exceptions inherit FileSystemException, either directly or via some subclass. FileSystemException inherits RuntimeException which makes all EntityFS exceptions unchecked (see the Wikipedia article on checked exceptions).
In general, EntityFS methods do not try to handle any exceptions that they encounter; it is up to the client application to implement a strategy for that. Some classes with longer-running methods, such as the IteratorCopier, support an optional ErrorHandlingStrategy.
Every FileSystem uses a LogAdapter instance for logging. It is an adapter to the client application's logging system.
The following adapters are implemented:
Logs to stdout and stderr.
Logs to the Logger with a configurable name. The default name
is org.entityfs.log
.
Logs to the org.apache.commons.logging.Log
with
a configurable name. The default name is
org.entityfs.log
.
Adapters to other logging systems can be created by implementing the LogAdapter interface.
The file system's current log adapter is always referenced by its LogAdapterHolder instance. The same holder object is used throughout the file system's lifetime and it is shared among all of the file system's objects that need to log.
By keeping the log configuration in an object instance, rather than in static class configuration like Commons Logging and JDK 1.4 logging do, the configuration of a LogAdapter instance is only visible to the objects that have a reference to it. This has the advantage of automatically limiting the scope of configuration changes without having to bother with different class loaders, thread locals and such. This is important in environments such as application servers where several different applications may share the same JVM and hence, often, the same class loaders. The obvious drawback is that an instance of the logger object has to be passed around to everyone that needs it. In EntityFS, this is not a big problem since most objects have a reference to their parent file system object anyway, and can get a LogAdapter from there.
Table of Contents
EntityFS entities are designed to be used concurrently by several threads. To avoid simultaneous, conflicting operations by different threads, entities can be locked for exclusive access by the thread holding the lock.
Locking is supported by read/write file system, but by default locking is
disabled. It can be enabled by calling
enableLocking
on the
FileSystemBuilder.
Locks are either read/write locks (the same semantics
as Java's ReadWriteLock)
or single locks (the same semantics as Java's
synchronized
keyword). Locks can also be
fair (first come, first
served for threads wanting to lock the entity, but with a performance penalty),
or non-fair.
All lock types are reentrant, so the same lock can safely be locked several times by
the thread that owns it.
Every file system uses one lock type and one lock acquiring strategy for all of its entities. The type of locks is determined by the file system's LockAdapterFactory.
Table 7.1. Lock types
Lock type | LockAdapterFactory |
---|---|
Fair read/write locks | FairReadWriteEntityLockAdapterFactory |
Fair single locks | FairSingleEntityLockAdapterFactory |
Non-fair read/write locks | NonFairReadWriteEntityLockAdapterFactory |
Non-fair single locks | NonFairSingleEntityLockAdapterFactory |
How a file system's locks are acquired is determined by its LockAcquiringStrategy:
Table 7.2. Lock acquiring strategies
Lock acquiring strategy | Description |
---|---|
SimpleLockAcquiringStrategy | Locks are acquired by calling their lock
method. Threads can wait indefinitely for a lock. Deadlocks will freeze the
affected threads. This is the default strategy. |
TimeoutLockAcquiringStrategy | Locks are acquired by calling their timed lock
method, which times out, generating a
LockTimeoutException, if a thread waits too long for a lock. |
Entity locks, regardless of their type, implement the EntityLock interface, which is an extension of Java's Lock interface. An EntityLock is aware of the Lockable object that it protects.
The EntityLocks utility class contains static utility methods for working with entity locks.
Entity locks should always be released in a finally
clause of a try-finally
block. In this way,
the locks are always release even in the case of an error happening
in the try
block.
Example 7.1. Using try-finally for locks
// d is a directory EntityLock rl = d.lockForReading(); try { d.listEntities(); } finally { // This will always be called. rl.unlock(); }
Example 7.2. Releasing several locks in finally
Collection<EntityLock> locks = new ArrayList<EntityLock>(); try { // d1 and d2 are directories locks.add(d1.lockForReading()); locks.add(d2.lockForWriting()); // do stuff... } finally { EntityLocks.unlockAll(locks); }
To avoid deadlocks, every application must have a strategy for in which order entity locks are locked. (This is another strategy than the LockAcquiringStrategy discussed above.) For a discussion on locking strategies, see for instance the book Java concurrency in practice. Entities do not assume any particular locking strategy. Utility classes, however, use a strategy where:
Parent entities are locked before child entities.
Entities in the same directory are locked in hash code order.
LockCommand:s are command objects for locking entities.
Several commands may be run by calling the LockCommandExecutor's
execute
method. The
executor makes sure that locks are acquired in a sequence that follows a
the lock acquiring strategy that it implements.
Table 7.3. Lock commands
Lock command | Description |
---|---|
LockForReadingCommand | Lock an entity for reading. |
LockForWritingCommand | Lock an entity for writing. |
LockParentForReadingCommand | Lock the parent directory of an entity for reading. |
LockParentForWritingCommand | Lock the parent directory of an entity for writing. |
A locking FileSystem has its own LockCommandExecutor instance that clients may use to execute LockCommand:s in an order that is compliant with that file system's locking strategy. The default implementation, LockCommandExecutorImpl, implements the utility classes' locking strategy that were described above.
The example below shows how a LockCommandExecutor can be used to lock all locks necessary for moving and then writing to an entity when using an AccessController that require locked entities. (See Chapter 13, Access controls.)
Example 7.3. Using a LockCommandExecutor
// Move the file f1 from directory d1 to directory d2 and append the text // "moved" to it. FileSystem fs = f1.getFileSystem(); AccessController ac = fs.getAccessController(); Collection<LockCommand> cmds = new ArrayList<LockCommand>(); // Add the lock commands that are needed for moving the entity. // Keep this lock command for future reference LockCommand f1lc = new LockForWritingCommand(f1); cmds.add(f1lc); cmds.add(new LockForWritingCommand(d1)); cmds.add(new LockForWritingCommand(d2)); // Add the lock commands that are needed for performing access controls. // Keep these lock commands for future reference Collection<? extends LockCommand> f1acmds = ac.getLockCommandsForAccessControl(f1); cmds.addAll(f1acmds); cmds.addAll(ac.getLockCommandsForAccessControl(d1)); cmds.addAll(ac.getLockCommandsForAccessControl(d2)); // Lock all locks LockCollection lc = fs.getLockCommandExecutor().execute(cmds); try { // Move the entity f1.rename(d2, f1.getName()); // Keep all locks required for appending text to the file in a new lock // collection. LockCollection lc2 = new LockCollection(); try { // The write lock on f1 lc2.addLock(f1lc, lc.removeLock(f1lc)); // The locks required for access controls on f1 lc2.addLocks(lc.removeLocks(f1acmds)); // Unlock all other locks lc.unlockAll(); Writer w = new OutputStreamWriter( f1.openForAppend(), Charset.forName("utf8")); try { w.write("moved"); } finally { w.close(); } } finally { lc2.unlockAll(); } } finally { // It is safe to call unlockAll several times an a LockCollection lc.unlockAll(); }
An application can create new locks for an entity in addition to the
standard read and write locks by calling its createLock
method.
This can be used as an alternative to using synchronized
or any other locking method and it makes it possible for the application to use the
same LockAcquiringStrategy for its own locks as for EntityFS locks.
When creating new locks for an entity, give careful thought about how the new locks relate to the standard read and write locks. For instance, an application may require that a thread must hold at least a read lock on the entity before it lets it lock the custom lock. Without a good strategy, deadlocks may occur.
The EntityFS methods that can be interrupted throw the unchecked exception UncheckedInterruptedException instead of InterruptedException. UncheckedInterruptedException:s should be handled in the same way as InterruptedException:s. See, for instance, Java concurrency in practice.
Table of Contents
Entity utility classes contain static methods for working with entities. The entity interfaces are designed to be as small as possible, so they contain just the bare essentials for implementing an entity. They are complemented by utility classes which implement methods to make it easier for client code to work with the entities. Compare this with the role of Java's Collections class.
Example 8.1. Creating a new file
Compare the Directory entity method for creating a new file:
// d is a directory EFile f; EntityLock wl = d.lockForWriting(); try { f = (EFile) d.newEntity(ETFile.TYPE, "f", null); } finally { wl.unlock(); }
With the Directories utility method:
EFile f = Directories.newFile(d, "f");
In the example above, the utility class takes care of locking all required locks before creating the file. All utility classes use the locking strategy that were described in Chapter 7, Locking and multi-threaded access, which should be fine with most applications.
Since locks are reentrant, that the utility classes do their own locking does not prevent client code from doing its own locking too. For instance, client code might want to hold a write lock on an entity that is modified through several utility method invocations to be sure that no other thread accesses the entity in between calls.
An entity utility class is designed to work with one specific entity interface. There are also capability utility classes with designed for working with capability objects, see Chapter 10, Capabilities. The following utility classes are implemented for entity interfaces:
Example 8.2. Holding a write lock for several utility method calls
// d is a directory EntityLock wl = d.lockForWriting(); try { EFile f1 = Directories.newFile(d, "f1"); // Since we hold a write lock on d here, no other thread can get access to // f1 since that would involve reading d. // writeDataToFile is a method that writes data to the file... writeDataToFile(f1); EFile f2 = Directories.newFile(d, "f2"); } finally { wl.unlock(); }
In addition to the iterators that iterate over a directory's child entities, there are two types of iterators that iterate recursively over the entities in an entire directory hierarchy – the DepthFirstIterator and the DepthLastIterator[4]. Both iterators can be instantiated directly or created by using Directories utility methods.
There are implementations of depth-first and breadth-first iterators, apart from the default implementations DepthFirstIteratorImpl and DepthLastIteratorImpl, that implement a strategy for locking the entities that they return. All implementations are listed in the table below.
Table 8.2. Recursive iterators
Iterator class | Description |
---|---|
DepthFirstIteratorImpl | A depth-first iterator that does not lock the entities that it returns. |
DepthLastIteratorImpl | A depth-last (breadth-first) iterator that does not lock the entities that it returns. |
LocationAwareDepthFirstIteratorImpl | A depth-first iterator that returns both entities and their locations relative to the iteration base directory. |
LocationAwareDepthLastIteratorImpl | A depth-last (breadth-first) iterator that returns both entities and their locations relative to the iteration base directory. |
OptimisticLockingDepthLastIterator | A depth-last (breadth-first) iterator that locks entities as they are returned from the iterator. |
PessimisticLockingDepthFirstIterator | A depth-first iterator that locks all entities that it will return before the first entity is returned. |
PessimisticLockingDepthLastIterator | A depth-last (breadth-first) iterator that locks all entities that it will return before the first entity is returned. |
The example below shows a simple alternative implementation of the IteratorDeleter class – an utility object that deletes entities from a directory tree.
Example 8.3. Simple IteratorDeleter implementation
// d is a directory (or view) Iterator<EntityView> itr = Directories.getDepthFirstIterator(d); while(itr.hasNext()) { EntityView ev = itr.next(); DirectoryView parent; // Get a temporary read lock on ev. We need it to get ev's parent. EntityLock rl = ev.lockForReading(); try { // getParent requires a read lock (a write lock would also do) parent = ev.getParent(); } finally { rl.unlock(); } // parent == null means the root directory. We cannot delete that. if (parent != null) { // Lock the parent before ev. EntityLock pwl = parent.lockForWriting(); try { EntityLock wl = ev.lockForWriting(); try { // Is the parent still the parent of the entity to delete? if (ev.getParent() == parent) { ev.delete(); } else { ev.getFileSystem().getLogAdapter(). logWarning("Cannot delete " + ev + ". It has been moved!"); } } finally { wl.unlock(); } } finally { pwl.unlock(); } } }
EntityFS comes with some utility classes for working with iterators:
Table 8.3. Iterator utilities
Utility class | Description |
---|---|
AggregatingIterator | An iterator that aggregates the results from a list of other iterators. |
FilteringIterator | An entity iterator that only returns the entities that passes its Filter. |
IteratorCopier | Copies entities returned from an iterator to another directory tree. The directory hierarchies are preserved. |
IteratorDeleter | Deletes entities returned from an iterator. This is useful for deleting whole or partial directory hierarchies. |
Iterators | Contains static methods for working with iterators. |
JarCreator | Creates a Jar file containing entities that are returned from an iterator. See Chapter 14, Zip and Jar files. |
ZipCreator | Creates a Zip file containing entities that are returned from an iterator. See Chapter 14, Zip and Jar files. |
EntityFS comes with a generic filter implementation defined by the Filter and ConvenientFilter interfaces. In EntityFS, the filters are used for filtering entities to, for instance, create entity views or select which entities an iterator should return. (Other projects such as Schmant and At4J use the filter library for other purposes.)
There are several implementations of entity filters, as shown in the table below. Different filters can be combined using logical operators by using the methods defined in the ConvenientFilter interface. All entity filters also implement the EntityFilter interface to make them easier to find in the API documentation.
Table 8.4. Entity filters
Entity filter class | Description |
---|---|
DirectoryContainsFilter | Matches directories that contain at least x entities that match a supplied entity filter, where x is a configurable number. |
DirectoryEmptyFilter | Matches empty directories. |
DirectoryFilter | Matches directories. |
EFileFilter | Matches files. |
EFileNameExtensionFilter | Matches files with specific file name extensions. |
EntityIdentityFilter | Matches a specific entity object. |
EntityLatestModificationTimeFilter | Matches entities that were last modified before a specific point in time. It can be negated to match entities that were modified after a specific point in time. |
EntityLocationGlobFilter | Matches entities whose locations relative to a base
directory match a Glob pattern, for instance
*/test/*.xml . |
EntityLocationRegexpFilter | Matches entities whose locations relative to a base
directory match a Java regexp , for instance
.*/test/.*\.xml . |
EntityNameFilter | Matches entities with a specific name. |
EntityNameGlobFilter | Matches entities with names that match a Glob pattern, for instance
test_[ab]*.xml . |
EntityNamePrefixFilter | Matches entities with names that starts with a specific prefix. |
EntityNameRegexpFilter | Matches entities with names that match a Java regular expression
, for instance test_[ab].*.xml .
|
EntityRecentModificationFilter | Matches entities that have been modified recently. It can be negated to match entities that have not been recently modified. |
EntityTypeFilter | Matches entities of a specific EntityType (files, directories, etc.). |
FalseEntityFilter | Does not match any entities at all. |
FGrepFilter | Matches text files that contain a certain text string. |
GrepFilter | Matches text files that contain text that match a regular expression Pattern. |
LockedEntityFilter | Matches entities that are locked by the current thread. |
ParentFilter | A filter that applies another filter to an entity's parent directory. |
ReadLockedEntityFilter | Matches entities that are read locked by the current thread. |
SuperParentAndFilter | This filter matches if all of an entity's parent directories (up to the file system root directory) matches a filter. |
SuperParentOrFilter | This filter matches if any of an entity's parent directories (up to the file system root directory) matches a filter. |
SymbolicLinkFilter | Matches symbolic link entities. |
TrueEntityFilter | Matches all entities. |
WriteLockedEntityFilter | Matches entities that are write locked by the current thread. |
For more information on glob, see The Wikipedia article on glob.
Example 8.4. An iterator for iterating recursively over XML files
// d is a directory (or view) Iterator<EntityView> itr = new FilteringIterator( Directories.getDepthLastIterator(d), new EFileNameExtensionFilter("xml"));
All Filter implementations in EntityFS also implement the
ConvenientFilter interface.
It adds the methods and
,
or
, xor
and
not
that can be used instead of the
AndFilter, OrFilter,
XorFilter and NotFilter to combine different
filters. This gives a more compact and readable syntax.
The EntityFilters class contains static utility methods for working with entity filters.
EntityFS defines its own Properties interface.
It extends Map<String,String>
and
defines methods for accessing properties of different data types. Static property
utilities are implemented in the PropertiesUtil
class.
Example 8.5. Working with properties
// f is a property file Properties p = PropertiesUtil.loadFromFile(f); int i = p.getIntValue("myIntProperty"); // The object value is stored as the object serialized to a // Base64-encoded String. Object o = p.getObjectValue("myObjectProperty"); // If no default value is given, all getter methods throw // a NoSuchPropertyException if a requested property is missing. // Define a default value to use if the property is missing. long l = p.getLongValue("myOptionalLongProperty", 17L);
Adapters are used for adapting the interface of one object to another interface. There are several adapter implementations in EntityFS.
Table 8.5. Adapter implementations
Class | From | To | Comments |
---|---|---|---|
FileReadableFile | File | NamedReadableFile | |
DataSinkToOutputStreamAdapter | DataSink | OutputStream | |
DataSourceToInputStreamAdapter | DataSource | InputStream | |
GatheringByteChannelAdapter | WritableByteChannel | GatheringByteChannel | |
InputStreamBackedRandomAccess | InputStream | RandomAccess | Supports random access by opening new input streams whenever the client moves backwards. |
InputStreamToDataSourceAdapter | InputStream | DataSource | |
NamedReadableFileAdapter | ReadableFile | NamedReadableFile | This can also be used to give a NamedReadableFile a new name. |
OutputStreamToDataSinkAdapter | OutputStream | DataSink | |
RandomAccessAdapter | RandomAccessFile | RandomAccess | |
RandomAccessToDataInputOutputAdapter | RandomAccess | DataInput, DataOutput | |
RandomAccessToInputStreamAdapter | RandomAccess | InputStream | |
RandomAccessToOutputStreamAdapter | RandomAccess | OutputStream | |
ReadWritableFileAdapter | File | ReadWritableFile | |
ScatteringByteChannelAdapter | ReadableByteChannel | ScatteringByteChannel |
EntityFS comes with several useful implementations of Java's InputStream and OutputStream. It also has some utility implementations of the RandomAccess interface.
Table 8.6. InputStream implementations
Implementation | Description |
---|---|
Base64InputStream | Reads and decodes Base64 encoded data. |
ChecksumInputStream | Calculates a checksum over the data read from a proxied stream. |
CountingInputStream | Counts the number of bytes that are read from a proxied stream. |
LockAwareFileInputStream | A FileInputStream that releases an EntityLock when the stream is closed. |
LockAwareInputStream | An input stream that releases an EntityLock when the stream is closed. |
RangeInputStream | Reads data from a portion of a RandomAccess object. |
Table 8.7. OutputStream implementations
Implementation | Description |
---|---|
Base64OutputStream | Base64 encodes data that is written to it. |
CountingOutputStream | Counts the number of bytes that are written to a proxied stream. |
LockAwareFileOutputStream | A FileOutputStream that releases an EntityLock when the stream is closed. |
LockAwareOutputStream | An output stream that releases an EntityLock when the stream is closed. |
MultiplexingOutputStream | An output stream that writes its output to one or more proxied output streams. |
Table 8.8. RandomAccess implementations
Implementation | Description |
---|---|
ByteArrayRandomAccess | A read only RandomAccess object that reads data from a byte array. |
EmptyRandomAccess | A read only RandomAccess object that does not contain any data. |
LockAwareRandomAccess | A RandomAccess that releases an EntityLock when it is closed. |
RangeRandomAccess | Reads and writes data to a portion of another RandomAccess object. |
TempFileBackedRandomAccess | This is used for opening RandomAccess objects on files that don't support it. A temporary copy is made of the file and the RandomAccess object is opened on that copy. |
Table 8.9. Other utility classes
Utility class | Description |
---|---|
DirectoryDifferentiator | Analyzes differences between a master Directory and a target Directory. Parent class of the DirectorySynchronizer. |
DirectorySynchronizer | One way synchronization of the contents of a target directory with that of a master directory. |
DirectoryTreeDifferentiator | Analyzes differences between a master Directory hierarchy and a target Directory hierarchy. Parent class of the DirectoryTreeSynchronizer. |
DirectoryTreeSynchronizer | One way synchronization of the contents of a target directory hierarchy with that of a master directory hierarchy. See the example below. |
EntityClassLoader | Class loader that loads resources from an EntityFS file system. |
EntityLocations | Methods for working with EntityLocation:s. |
EntityLocks | Methods for working with EntityLock:s. |
StreamUtil | Low-level utility methods for working with I/O streams and NIO channels. |
ZipFiles | Methods for working with Zip files. See Chapter 14, Zip and Jar files |
Example 8.6. Synchronizing directory hierarchies
// Synchronize the directory hierarchy under the directory target with the // hierarchy under the directory master. // // Assume the following hierarchy under master: // master // + f1 // + d1 // + f2 // + f3 new DirectoryTreeSynchronizer(master, target).run(); // target now mirrors the directory hierarchy under master, regardless of what // it contained before synchronizing.
Table of Contents
Entities as well as the entire file system, may be configured to be Observable for events. Clients that want to receive notification of entity updates, register a listener of the type EntityObserver on the entity or on the file system object. The listener will then receive all EntityEvent:s that involve the observed entity, or for a file system listener, all events in that file system.
Every event has a sender which is the entity that generated the event.
Example 9.1. Registering an observer on a directory
// d is a directory. Register a simple observer on it. d.addObserver( new EntityObserver() { public void update(EntityEvent<?> ee) { if (ee instanceof ChildEntityAddedEvent) { System.out.println("Child entity " + ((ChildEntityAddedEvent) ee).getChildEntity() + " added to directory " + ee.getSender()); } else { System.out.println("Event: " + ee); } } }); Directories.newFile(d, "f1");
The observer will print out that the file f1
was
added.
Table 9.1. EntityEvent types
EntityEvent | Description |
---|---|
ChildEntityAddedEvent | An entity has been added to a Directory. |
ChildEntityDisappearedEvent | The EntityValidityControlStrategy has detected that an entity has disappeared from a Directory. The entity was not removed through using the file system's methods. It might have been removed by another process altogether. |
ChildEntityModifiedEvent | An entity in a Directory has been modified in a way that updated its last modification time property. For modified files, this event is generated when the client closes its stream or channel on the file. |
ChildEntityRemovedEvent | A child entity has been removed from a Directory, either by deleting it or moving it to another directory. |
ChildEntityRenamedEvent | An entity in a Directory has been renamed within the same directory. |
EntityDeletedEvent | The entity has been deleted. |
EntityDisappearedEvent | The FileSystem's EntityValidityControlStrategy
has detected that the entity has disappeared. It was not removed through using the file
system's methods. The entity is not valid (its isValid
method will return false ) when the observer
receives the event. |
EntityModifiedEvent | The entity was modified in a way that updated its latest modification time. For modified files, this event is generated when the client closes its stream or channel on the file. |
The EntityObserver's update
method is called in the thread that generated the event. This means that, if the
file system is locking, the calling thread is holding a write lock on the
event's sender when update
is called. It also
means that update
should return quickly. If
there is a lot of work to do in response to an event, use an
Executor to schedule the work for execution in another thread.
The following examples show which events that are generated when the file system is modified in different ways.
Example 9.2. Adding a file to a directory
Assume the following file system layout:
+ d (directory) + d1 (directory)
Adding a file f
to the directory
d1
will generate the following events, in this
order:
A ChildEntityAddedEvent for
d1
.
A EntityModifiedEvent for
d1
.
A ChildEntityModifiedEvent for
d
.
A NewEntityEvent for
f
.
Example 9.3. Modifying a file
Assuming the file system layout from the previous example, if the file
f
is opened and written to, when it is closed,
the following events will be generated, in this order:
A EntityModifiedEvent for
f
.
A ChildEntityModifiedEvent for
d1
.
(If f
just had been opened and closed, without
any data being written to it, the same events had been generated anyway.)
Example 9.4. Moving an entity
Assume the following file system layout:
+ d (directory) + d1 (directory) | + d10 (directory) | + f (file) + d2 (directory) + d20 (directory)
Moving the file f
from
d10
to d20
will generate the following events, in this order:
A ChildEntityRemovedEvent for
d10
.
A EntityModifiedEvent for
d10
.
A ChildEntityModifiedEvent for
d1
.
A ChildEntityAddedEvent for
d20
.
A EntityModifiedEvent for
d20
.
A ChildEntityModifiedEvent for
d2
.
A EntityRenamedEvent for
f
.
For directories that can be modified by someone that does not use its file system's methods, for instance by another process, a DirectoryMonitorer can be used to detect updates. Generally, it is not recommended to let entities be modified without using their entity objects, but in some cases, such as when monitoring a directory for incoming files, it can be useful. It requires a good EntityValidityControlStrategy for the file system, though.
Example 9.5. Using a polling directory monitorer to watch for incoming XML files
// d is the directory to monitor. // Create a view that only shows XML files. DirectoryView dv = d.newView(new EFileNameExtensionFilter("xml")); // Create a timer that will tell the monitorer when to scan the directory // for updates. Any old Observable should work; the monitorer will // scan its directory every time the Observable generates an event. // Scan every minute. TimerTicker tt = new TimerTicker(60000); // Create a polling monitorer on the directory view. PollingDirectoryMonitorer mon = new PollingDirectoryMonitorer(dv, tt); // Register an observer on the monitorer mon.addObserver(new EntityObserver() { public void update(EntityEvent ee) { if (ee instanceof ChildEntitiesAddedEvent) { System.out.println("New child entities: " + ee); } } }); // Start the monitorer mon.start(); // Start the timer ticker. tt.start(); // This will generate an event sometime in the next minute Directories.newFile(d, "foo.xml"); // This will not Directories.newFile(d, "bar.txt"); // Neither will this Directories.newDirectory(d, "baz.xml"); // But this will new File( ECFileResolvableUtil.getFileObject(d) + File.separator + "doh.xml").createNewFile();
Standard file system and entity functionality is extended by capabilities. The capabilities that a file system and its entities have is decided when the file system is created by adding CapabilityProvider:s to its builder. All file system implementations also have built-in capabilities, see Table 4.1, “File system implementations”.
When mixing some capabilities, the order in which the capability providers are added to the file system may be important. Restrictions are documented in each capability provider's API documentation.
Capabilities are used by first retrieving a capability object from a file system or from an entity. The capability is identified by a capability-specific FileSystemCapabilityType or EntityCapabilityType constant.
Below is an example of how the ECUnixAttributes entity capability can be used to get the creation time for an entity. Entity attributes are explained more in detail in Chapter 12, Entity attributes.
Example 10.1. Using Unix entity attributes to get an entity's creation time
// Use the Unix entity attributes capability to find out when the EFile f was // created // First get the capability object for the file ECUnixAttributes eua = f.getCapability(ECTUnixAttributes.TYPE); long timestamp = eua.getAttributes().getCreationTime();
Most capabilities have their own utility classes that can be used to make the client code shorter and more readable. The example below accomplishes just the same as the previous example, but with less code.
Example 10.2. Using Unix entity attributes utility class to get an entity's creation time
// Use the Unix entity attributes capability utility object to find out when the // EFile f was created long timestamp = ECUnixAttributesUtil.getCreationTime(f);
The name of an entity capability object shows for which entity types it is
used. If the name starts with FC
, the capability
is used for EFile:s. If it starts with DC
,
it is used for Directory:s. An entity capability name starting with
EC
is used for all entity types.
Table 10.1. Entity capabilities
Capability | Utility class | Description |
---|---|---|
DCSymbolicLink | none | SymbolicLink entity support. This is still experimental. |
ECFileResolvable | ECFileResolvableUtil | For entities that are backed by a File object, this capability gives access to that File. |
ECJarEntry | ECJarEntryUtil | For entities that have a JarEntry object, this capability gives access to that object. This is true for all entities in a Jar file system. See Chapter 14, Zip and Jar files. |
ECMetadata | ECMetadataUtil | Metadata support for entities. This capability give an entity one metadata container. See Chapter 11, Metadata support. |
ECNamedMetadata | ECNamedMetadataUtil | Metadata support for entities. This capability give an entity any number of metadata containers. See Chapter 11, Metadata support. |
ECNamedEntityAttributes | none | This capability is supported by entities that have several, named EntityAttributes objects. This is currently not implemented by any capability provider. |
ECNtfsAttributes | ECNtfsAttributesUtil | NtfsAttributes support for entities. See Chapter 12, Entity attributes. |
ECUnixAttributes | ECUnixAttributesUtil | UnixAttributes support for entities. See Chapter 12, Entity attributes. |
ECUriResolvable | ECUriResolvableUtil | For entities whose locations can be represented by a URI, this capability gives access to that URI. |
ECZipEntry | ECZipEntryUtil | For entities that have a ZipEntry object, this capability gives access to that object. This is true for all entities in a Zip or a Jar file system. See Chapter 14, Zip and Jar files. |
FCAppendable | none | This capability marks that a file can be appended to. It does not have any methods. |
FCFileBacked | FCFileBackedUtil | For files that are backed by a File object, this capability has methods for manipulating that File. |
FCRandomAccess | none | This capability marks that a file can be opened for random access. It does not have any methods. |
Table 10.2. File system capabilities
Capability | Utility class | Description |
---|---|---|
FSCAppendableFiles | none | This capability is supported by file systems whose files can be appended to. It does not have any methods. |
FSCCompression | none | This capability is supported by file systems that compress file data. It does not have any methods. |
FSCDirectoryMetadata | none | This capability is supported by file systems that support metadata for its directories. It does not have any methods. |
FSCFileResolvable | FSCFileResolvableUtil | Capability supported by file systems whose entities can be resolved using File:s. |
FSCJarFileBacked | FSCJarFileBackedUtil | Capability supported by file systems whose entities are backed by JarFile:s. |
FSCMetadata | none | This capability is supported by file systems that support metadata for its entities. It does not have any methods. |
FSCNamedEntityAttributes | none | This capability is supported by file systems that support named entity attributes objects for its entities. It does not have any methods. |
FSCNtfsAttributes | none | This capability is supported by file systems that support NTFS attributes for its entities. It does not have any methods. |
FSCPersistent | none | This capability is supported by file systems that store entity data in some kind of persistent storage, such as in an ordinary file system. |
FSCRandomAccessFiles | none | This capability is supported by file systems with support for random access to its files. |
FSCSymbolicLink | none | This capability is supported by file systems that support symbolic link entities. |
FSCUnixAttributes | none | This capability is supported by file systems that support Unix attributes for its entities. It does not have any methods. |
FSCUriResolvable | FSCUriResolvableUtil | Capability supported by file systems whose entities can be resolved using URI:s. |
FSCZipFileBacked | FSCZipFileBackedUtil | Capability supported by file systems whose entities are backed by ZipFile:s. |
Table 10.3. Capability providers
Capability provider | ||
---|---|---|
Adds capabilities | Disables capabilities | Description |
FileBasedDirectoryMetadataCapabilityProvider | ||
FSCDirectoryMetadata, ECMetadata (for directories), ECNamedMetadata (for directories) | Capability that make directory entities capable of hosting metadata. See Chapter 11, Metadata support. | |
FileBasedMetadataCapabilityProvider | ||
FSCMetadata, FSCDirectoryMetadata, ECMetadata, ECNamedMetadata | Capability that make all entities capable of hosting metadata. See Chapter 11, Metadata support. | |
GZipCompressionCapabilityProvider | ||
FSCCompression | ECFileResolvable, ECUriResolvable, FCFileBacked, FSCRandomAccessFiles | Capability that compress file data using the gzip algorithm. |
MetadataCustomAttributesCapabilityProvider | ||
ECEntityAttributes, FSCEntityAttributes | Capability that adds custom EntityAttributes to entities. See Chapter 12, Entity attributes and Chapter 13, Access controls. | |
MetadataNtfsAttributesCapabilityProvider | ||
ECEntityAttributes, ECNtfsAttributes, FSCEntityAttributes, FSCNtfsAttributes | Capability that adds NtfsAttributes to entities. See Chapter 12, Entity attributes and Chapter 13, Access controls. | |
MetadataUnixAttributesCapabilityProvider | ||
ECEntityAttributes, ECUnixAttributes, FSCEntityAttributes, FSCUnixAttributes | Capability that adds UnixAttributes to entities. See Chapter 12, Entity attributes and Chapter 13, Access controls. | |
RamSymbolicLinkCapabilityProvider | ||
FSCSymbolicLink, DCSymbolicLink | SymbolicLink support for the Ram file system. This is still experimental. |
Example 10.3. Creating a Ram file system with gzip compression of file data
FileSystem fs = new RamFileSystemBuilder(). addCapabilityProvider( new GZipCompressionCapabilityProvider()).create();
Metadata support for entities is defined by the ECMetadata and the ECNamedMetadata capability interfaces. ECMetadata uses a single metadata container for each entity, while ECNamedMetadata supports any number of named metadata containers per entity. Each named metadata container is identified by a name that is unique for the entity that it is associated with. A metadata container is represented as a ReadWritableFile.
There are two file system capabilities that indicate that a file system supports metadata; FSCMetadata and FSCDirectoryMetadata. FSCMetadata indicate metadata support for all entity types, while FSCDirectoryMetadata indicate metadata support for directory entities only. All file systems that support FSCMetadata also support FSCDirectoryMetadata.
Two capability providers provide metadata support:
Table 11.1. Metadata capability providers
Capability provider | Adds capabilities | Description |
---|---|---|
FileBasedDirectoryMetadataCapabilityProvider | FSCDirectoryMetadata ECMetadata (for directories) ECNamedMetadata (for directories) | Provides metadata support for directory entities. Metadata is stored in invisible files in the directory. This capability provider does not modify the backing file system in the way that FileBasedMetadataCapabilityProvider do. |
FileBasedMetadataCapabilityProvider | FSCMetadata FSCDirectoryMetadata ECMetadata ECNamedMetadata | Provides metadata support for all entity types. This capability provider transparently changes the file system layout in the backing file system. |
If the file system that the metadata capability is added to is FSCPersistent, the metadata will also be persistent.
When copying entities with metadata to a file system that does not support
metadata, the metatada capability provider's MetadataCopyStrategy
determines what happens. If it is set to IGNORE
,
which is the default, metadata will not be copied. If it is set to
COPY
, metadata is copied and put in one or
several separate files in the target file system.
ECMetadataUtil and ECNamedMetadataUtil contain static utility methods for working with metadata capabilities.
Example 11.1. Creating a metadata enabled file system
FileSystem fs = new FSRWFileSystemBuilder(). setRoot(new File("/tmp/fs1")). addCapabilityProvider( new FileBasedMetadataCapabilityProvider(). // Put metadata in separate files if copying entities to a metadata // unaware file system. setCopyStrategy(MetadataCopyStrategy.COPY)). create();
Example 11.2. Creating a file system with metadata support for directories
FileSystem fs = new FSRWFileSystemBuilder(). setRoot(new File("/tmp/fs1")). addCapabilityProvider( new FileBasedDirectoryMetadataCapabilityProvider(). // Don't copy metadata if copying directories with metadata to a file // system that does not support metadata. setCopyStrategy(MetadataCopyStrategy.IGNORE)). create();
Example 11.3. Setting a metadata property on an entity
// a is an entity (file or directory) EntityLock wl = a.lockForWriting(); try { Properties p = new FileBackedProperties( ECMetadataUtil.getCapability(a)); p.putStringValue( "lastSeen", new SimpleDateFormat("yyyyMMdd").format(new Date())); } finally { wl.unlock(); }
Example 11.4. Reading data from a named metadata file
// a is an entity (file or directory) EntityLock rl = a.lockForReading(); try { // The metadata file is protected by it's parent entity's read lock InputStream is = ECNamedMetadataUtil.getMetadataFile(a, "icon64x64.bmp"). openForRead(); try { BufferedImage img = ImageIO.read(is); } finally { is.close(); } } finally { rl.unlock(); }
Table of Contents
An EntityAttributes object captures a collection of attributes for an Entity, such as the information on who owns it. It can be used for representing metadata in a structured way.
The entity attributes of an entity are properties of its parent directory. The entity attributes for all of a directory's child entities are stored in one of the directory's metadata files, so all entity attribute capability implementations require that the file system they are used for support the FSCDirectoryMetadata capability.
The UnixAttributes contains metadata information about entities that can be found in a Unix file system. The attributes are the user and group id:s of the entity's owner, the UnixEntityMode of the entity and the entity's creation time.
UnixAttributes support is provided by the MetadataUnixAttributesCapabilityProvider. It uses a B+ Tree HeliDB database stored in a metadata file for a directory to store all the entity attributes for that directory's child entities. This has the consequence that the root directory itself does not support entity attributes.
The NtfsAttributes contains a subset of the metadata information about entities that can be found in an NTFS file system. The attributes are the user and group id:s of the entity's owner, the UnixEntityMode of the entity, the entity's creation time and the set of NtFileAttributes.
NtfsAttributes support is provided by the MetadataNtfsAttributesCapabilityProvider. It uses a B+ Tree HeliDB database stored in a metadata file for a directory to store all the entity attributes for that directory's child entities. This has the consequence that the root directory itself does not support entity attributes.
The MetadataCustomAttributesCapabilityProvider can be used to have custom EntityAttributes for entities. The client must provide factory objects for default attributes for different entity types and an object that is used to parse the entity attributes metadata file, a HeliDB Serializer.
For entity attributes that have records of a fixed size in the attributes metadata file, there are abstract stub implementations of the required classes. For entity attributes that don't have fixed record size, some more effort is required. See the API documentation for details.
The rather long example below shows how a simple EntityAttributes object and all other classes required to use it together with the MetadataCustomAttributesCapabilityProvider are implemented.
Example 12.1. Using the custom entity attributes capability
// Our custom entity attributes // Implement the attributes object to be immutable. public class IndexAndBackupAttributes implements EntityAttributes { private final Date m_indexDate; private final Date m_backupDate; public IndexAndBackupAttributes(Date indexDate, Date backupDate) { // Null checks indexDate.getClass(); backupDate.getClass(); m_indexDate = indexDate; m_backupDate = backupDate; } public Date getIndexDate() { return m_indexDate; } public Date getBackupDate() { return m_backupDate; } // EntityAttributes inherits Cloneable public Object clone() { return new IndexAndBackupAttributes(m_indexDate, m_backupDate); } // EntityAttributes recommends that we implement equals @Override public boolean equals(Object o) { if (o instanceof IndexAndBackupAttributes) { IndexAndBackupAttributes a2 = (IndexAndBackupAttributes) o; return a2.m_indexDate.equals(m_indexDate) && a2.m_backupDate.equals(m_backupDate); } else { return false; } } // We implemented equals, so now we have to implement hashCode too... @Override public int hashCode() { // Implement as suggested in Joshua Bloch's Effective Java int res = ((int) m_indexDate.getTime() ^ (int) (m_indexDate.getTime() >>> 32)); return 31 * res + (int) (m_backupDate.getTime() ^ (int) (m_backupDate.getTime() >>> 32)); } } // This is the Serializer that is used to serialize // attributes to and interpret attributes from the file where they are stored. // One instance is used by all objects. public class IndexAndBackupSerializer implements Serializer<IndexAndBackupAttributes> { // The size of a serialized object in bytes. // A long is 8 bytes long private static final int SERIALIZED_SIZE = 8 + 8; // Get the size in bytes of the object when serialized. public int getSerializedSize() { return SERIALIZED_SIZE; } public boolean isNullValuesPermitted() { // No. return false; } // This is used by both serialization methods private void serializeInternal(IndexAndBackupAttributes o, byte[] barr, int offset) // We are allowed to throw this throws ArrayIndexOutOfBoundsException { // Serialize the dates to long:s. Use the LongSerializer. int pos = offset; LongSerializer.INSTANCE.serialize(o.getIndexDate().getTime(), barr, pos); pos += LongSerializer.DATA_SIZE; LongSerializer.INSTANCE.serialize(o.getBackupDate().getTime(), barr, pos); } public byte[] serialize(IndexAndBackupAttributes o) { byte[] res = new byte[SERIALIZED_SIZE]; serializeInternal(o, res, 0); return res; } public int serialize(IndexAndBackupAttributes o, byte[] barr, int offset) { serializeInternal(o, barr, offset); return SERIALIZED_SIZE; } private void validateSize(int size) { if (size != SERIALIZED_SIZE) { throw new SerializationException( "Invalid size of data " + size + ". Must be " + SERIALIZED_SIZE + " bytes"); } } // This is used by both interpret methods. private IndexAndBackupAttributes interpretInternal(byte[] barr, int offset) { // Use the LongSerializer to read the dates // that are serialized as longs int pos = offset; long indexDate = LongSerializer.INSTANCE.interpret(barr, pos, LongSerializer.DATA_SIZE); pos += LongSerializer.DATA_SIZE; long backupDate = LongSerializer.INSTANCE.interpret(barr, pos, LongSerializer.DATA_SIZE); return new IndexAndBackupAttributes( new Date(indexDate), new Date(backupDate)); } public IndexAndBackupAttributes interpret(byte[] barr) throws SerializationException { validateSize(barr.length); return interpretInternal(barr, 0); } public IndexAndBackupAttributes interpret(byte[] barr, int offset, int length) throws SerializationException { validateSize(length); return interpretInternal(barr, offset); } public IndexAndBackupAttributes read(InputStream is, int size) { validateSize(size); byte[] barr = new byte[SERIALIZED_SIZE]; try { int noRead = is.read(barr); if (noRead != SERIALIZED_SIZE) { throw new NotEnoughDataException(SERIALIZED_SIZE, noRead); } return interpret(barr); } catch (IOException e) { throw new WrappedIOException(e); } } public IndexAndBackupAttributes read(RandomAccess ra, int size) { validateSize(size); byte[] barr = new byte[SERIALIZED_SIZE]; int noRead = ra.read(barr); if (noRead != SERIALIZED_SIZE) { throw new NotEnoughDataException(SERIALIZED_SIZE, noRead); } return interpret(barr); } } // This ObjectFactory is used to create default values of the attributes // for different entity types. public class IndexAndBackupAttributesFactory implements ObjectFactory<IndexAndBackupAttributes> { public IndexAndBackupAttributes create() { // Initialize the object with the current time. Date d = new Date(); return new IndexAndBackupAttributes(d, d); } } // This object creates a file system whose entities have // IndexAndBackupAttributes, creates a couple of entities and set their // attribute values. public class MyIndexAndBackupAttributesUsingClass { public FileSystem createAndInitFileSystem(File root) { // Default attributes factories for the different entity types. IndexAndBackupAttributesFactory af = new IndexAndBackupAttributesFactory(); Map<EntityType, ObjectFactory<IndexAndBackupAttributes>> daf = new HashMap <EntityType, ObjectFactory<IndexAndBackupAttributes>>(); daf.put(ETFile.TYPE, af); daf.put(ETDirectory.TYPE, af); FileSystem res = new FSRWFileSystemBuilder(). setRoot(root). // Add the necessary directory metadata capability provider first. addCapabilityProvider( new FileBasedDirectoryMetadataCapabilityProvider()). // Add the custom entity attributes capability provider addCapabilityProvider( new MetadataCustomAttributesCapabilityProvider(). // Don't store entities' attributes lazily (this is the default, so // this invocation is unnecessary) setStoreAttributesLazily(false). setDefaultAttributesFactories(daf). // Set the types of attributes that will be handled by the // capability setAttributesType(IndexAndBackupAttributes.class). // This factory creates ChildEntityAttributeManager // objects for all directories. setChildEntityAttributeManagerFactory( // This can be used for capabilities that store their attributes // in a HeliDB database. new DatabaseChildEntityAttributeManagerFactory<IndexAndBackupAttributes>( new IndexAndBackupSerializer(), daf, 8192))).create(); Directory fsroot = res.getRootDirectory(); EFile f = Directories.newFile(fsroot, "f"); // This has not been backed up since the 28th of January 1982. Sorry. Calendar cal = Calendar.getInstance(); cal.clear(); cal.set(Calendar.YEAR, 1982); cal.set(Calendar.MONTH, 0); cal.set(Calendar.DAY_OF_MONTH, 28); IndexAndBackupAttributes iab = new IndexAndBackupAttributes(new Date(), cal.getTime()); ECEntityAttributesUtil.setAttributes(f, iab); return res; } }
File systems have support for using an AccessController to control access to its entities. Access controllers are tools for helping file system clients in setting access policies for their file systems. They are not full-blown security solutions in themselves, though, since they can always be circumvented by malicious code running in the same JVM.
Most access controllers use Subject objects to
identify who is trying to access an entity. A Subject is
assigned to a running thread by calling any of the Subject's
doAs
methods.
The simplest AccessController implementation is the
AccessGranterAccessController. It uses the callback
method hasAccess
of an AccessGranter
to decide if a client has access to an entity.
The example below shows how a custom AccessGranter can be used with custom Subject objects to limit access to entities.
Example 13.1. A custom access granter
// The access granter public class MyAccessGranter implements AccessGranter { public boolean hasAccess(Subject subj, Permission p) { // Alice may only write in directories on thursdays if (p instanceof WritePermission) { WritePermission wp = (WritePermission) p; // Is the entity we are controlling access to a directory entity? if (wp.getEntity().getType() == ETDirectory.TYPE) { String subjName = ((MySubject) subj).getName(); if (subjName.equals("Alice")) { return Calendar.getInstance().get(Calendar.DAY_OF_WEEK) == Calendar.THURSDAY; } } } // All other access is permitted return true; } } // A custom Subject implementation that has a name. public class MySubject extends SimpleAccessControllerSubject { private final String m_name; public MySubject(AccessController ac, String name) { super(ac); m_name = name; } public String getName() { return m_name; } } // A PrivilegedAction that creates a new temporary file in a directory public class MyWriteToDirectoryAction implements PrivilegedAction<EFile> { private final Directory m_directory; public MyWriteToDirectoryAction(Directory d) { m_directory = d; } public EFile run() { return Directories.newTempFile(m_directory, "privileged", ".tmp"); } } // A class that uses the privileged action to try to create a temporary file. public class MyDirectoryWriter { public EFile createTempFile(MySubject subj, Directory dir) { try { return subj.doAs(new MyWriteToDirectoryAction(dir)); } catch (AccessDeniedException e) { dir.getFileSystem().getLogAdapter().logWarning("Not today"); throw e; } } }
The UnixEntityModeAccessController uses an UidGidSubject, the owner user and group id:s, and the UnixEntityMode of an entity to decide if the subject has access to it. The UidGidSubject is a subject that has a user id, UID, a primary group id, GID, and zero or more secondary GID:s. The UnixEntityMode specifies what kind of access the entity's owner, members of the same group as the entity belongs to and all other subjects have to the entity.
The UnixEntityModeAccessController requires that the file system supports any of the ECNtfsAttributes or ECUnixAttributes capabilities. They are supplied by the MetadataNtfsAttributesCapabilityProvider or the MetadataUnixAttributesCapabilityProvider.
The following example shows how a UnixEntityModeAccessController can be used in a non-locking file system.
Example 13.2. The Unix entity mode access controller on a non-locking file system
// A PrivilegedAction that creates a new text file in a directory public class NewTextFileAction implements PrivilegedAction<EFile> { private final Directory m_directory; public NewTextFileAction(Directory d) { m_directory = d; } public EFile run() { EFile res = (EFile) m_directory.newEntity(ETFile.TYPE, "t.txt", null); Files.writeText(res, "Quidquid latine dictum sit, altum sonatur"); return res; } } // A class that makes sure that the file is written public class MyFileWriter { public EFile createFile(Directory d) { // Create a subject with UID = 1, primary GID = 2 and one secondary GID = 3 UidGidSubject subj = new SimpleAccessControllerUidGidSubject( d.getFileSystem().getAccessController(), 1, 2, Arrays.asList(new Integer[] { Integer.valueOf(3) })); // Get the entity attributes for the directory. Make no assumption about // what exact kind of entity attributes that the directory has, other than // that it implements UidGidConfigurable and UnixEntityModeConfigurable EntityAttributes ea = ECEntityAttributesUtil.getAttributes(d); // Set the directory's owner to be the new subject ((UidGidConfigurable) ea).setUid(1); // Set the directory's mode to permit write and execution by its // owner. 0755 is the mode's octal representation. It will allow writes only // by the directory's owner. See UnixEntityMode for more documentation. ((UnixEntityModeConfigurable) ea).setUnixEntityMode(0755); // Save the directory's attributes ECEntityAttributesUtil.setAttributes(d, ea); // Create the text file. It will automatically be owned by the creating // Subject return subj.doAs(new NewTextFileAction(d)); } }
Since the entity attributes of an entity is a property of its parent directory, the parent directory has to be locked for reading every time an access control is performed on the entity, in other words, every time any of the access controlling entity methods, which are most of them, are called. A consequence of this is that the locking strategy of the utility classes does not work with this access controller since the utility classes do not acquire locks on an entity's parent directory before calling an entity's methods. File system clients have to do their own entity locking, as is shown in the example below.
The following example does the same as the example above on a file system that is locking. The file system's LockCommandExecutor (see Chapter 7, Locking and multi-threaded access) is used for locking all locks in an order that is compliant with the file system's locking strategy.
Example 13.3. The Unix entity mode access controller on a locking file system
// A PrivilegedAction that creates a new text file in a directory public class NewTextFileAction implements PrivilegedAction<EFile> { private final Directory m_directory; public NewTextFileAction(Directory d) { m_directory = d; } public EFile run() { FileSystem fs = m_directory.getFileSystem(); // Create the file. // Use the file system's LockCommandExecutor to // execute the lock commands according to the file system's locking strategy. Collection<LockCommand> lcmds = new ArrayList<LockCommand>(); // We need lock commands for access controls on the directory lcmds.addAll(fs.getAccessController().getLockCommandsForAccessControl(m_directory)); // ...and lock commands for locking the directory for writing lcmds.add(new LockForWritingCommand(m_directory)); EFile res; LockCollection lc = fs.getLockCommandExecutor().execute(lcmds); try { res = (EFile) m_directory.newEntity(ETFile.TYPE, "t.txt", null); } finally { lc.unlockAll(); } // Write to the file lcmds = new ArrayList<LockCommand>(); // Lock commands for access controls on the file lcmds.addAll(fs.getAccessController().getLockCommandsForAccessControl(res)); // ... and for locking the file for writing lcmds.add(new LockForWritingCommand(res)); lc = fs.getLockCommandExecutor().execute(lcmds); try { Files.writeText(res, "Vescere bracis meis!"); } finally { lc.unlockAll(); } return res; } } // A class that makes sure that the file is written public class MyFileWriter { public EFile createFile(Directory d) { // Create a subject with UID = 1, primary GID = 2 and one secondary GID = 3 UidGidSubject subj = new SimpleAccessControllerUidGidSubject( d.getFileSystem().getAccessController(), 1, 2, Arrays.asList(new Integer[] { Integer.valueOf(3) })); // Get the entity attributes for the directory. Make no assumption about // what exact kind of entity attributes that the directory has, other than // that it implements UidGidConfigurable and UnixEntityModeConfigurable EntityAttributes ea = ECEntityAttributesUtil.getAttributes(d); // Set the directory's owner to be the new subject ((UidGidConfigurable) ea).setUid(1); // Set the directory's mode to permit write and execution by its owner. // 0755 is the mode's octal representation. It will allow writes only by the // directory's owner. See UnixEntityMode for more documentation. ((UnixEntityModeConfigurable) ea).setUnixEntityMode(0755); // Save the directory's attributes ECEntityAttributesUtil.setAttributes(d, ea); // Create the text file. It will automatically be owned by the creating // Subject return subj.doAs(new NewTextFileAction(d)); } }
EntityFS has support for using Zip and Jar files as backends for file systems, and also support for creating new Zip and Jar files.
There are three different Zip file systems
This file system, built by the At4jZipFileSystemBuilder, supports more kinds of Zip files than the file system backed by Java's Zip implementation. It requires the At4J library on the classpath.
The file system supports random access (FCRandomAccess) to files that are stored uncompressed in the Zip file. This makes it possible to build a new Zip file system on an uncompressed file in another Zip file system.
This Zip file system, built by the ZipFileSystemBuilder, supports the kinds of Zip files that Java's Zip implementation supports. It must be built on a File.
This file system, built by the JarFileSystemBuilder extends the Zip file system with support for JAR metadata.
A ZipCreator object can be used to create a new Zip file. It builds the file using ZipEntryInfo objects that are returned from an Iterator. A ZipEntryInfo object contains information that the ZipCreator uses for creating a ZipEntry for a file or a directory in the Zip file.
EntityFS ZipCreator uses Java's ZipOutputStream to create Zip files. It is somewhat limited in how the created file can be configured. The At4J project has a Zip builder that can create more types of Zip files.
The EntityIteratorToZipEntryInfoIteratorAdapter can be used to adapt any kind of EntityView iterator to a ZipEntryInfo Iterator.
Below is a simple example of how a Zip file can be created.
Example 14.1. Building a Zip file
// Build a Zip file containing all files and directories under the directory d1. // Put the new file zip.zip in the directory d2. new ZipCreator( new EntityIteratorToZipEntryInfoIteratorAdapter(d1, Directories.getDepthLastIterator(d1)), d2, "zip.zip").create(); // The ZipCreator will acquire read locks on the files and directories // added to the Zip file when it needs them.
The next example is a slightly more advanced example that builds a Zip file from files in two different directory hierarchies.
Example 14.2. Building a Zip file from files in two directory hierarchies
// Build a Zip file containing all files and directories under the directory d1, // stored under /dirs/d1 and all directories and XML files under the directory // d2, stored under /xmldirs/d2 in the Zip file. Put the new file zip.zip in the // directory d3. new ZipCreator( new AggregatingIterator( new EntityIteratorToZipEntryInfoIteratorAdapter(d1, Directories.getDepthLastIterator(d1), new AbsoluteLocation("/dirs/d1")), new EntityIteratorToZipEntryInfoIteratorAdapter(d2, new FilteringIterator( Directories.getDepthLastIterator(d2), new EFileNameExtensionFilter("xml")), new AbsoluteLocation("/xmldirs/d2"))), d3, "zip.zip").create();
A Jar file can be built just like a Zip file. The JarCreator
works just like the ZipCreator, with the added
setManifest
method for setting a Manifest.
Example 14.3. Building a Jar file
// Build a Jar file containing all files and directories under the directory d1. // Use a custom manifest. Put the new file jar.jar in the directory d2. new JarCreator( new EntityIteratorToJarEntryInfoIteratorAdapter(d1, Directories.getDepthLastIterator(d1)), d2, "jar.jar"). setManifest( new Manifest( new ByteArrayInputStream( // Note the trailing \n after the last row... "Manifest-Version: 1.0\nCreated-By: 1.0 me\n".getBytes()))).create();
To read the contents from a Zip or Jar file, create a Zip or Jar file system on the file and read entities from the file system.
Example 14.4. Reading entities from a Zip file
// Read the file d1/text2.txt from the Zip file texts.zip in the directory d FileSystem fs = new ZipFileSystemBuilder(). setZipFile(Directories.getFile(d, "texts.zip")). create(); try { info("Contents of text2.txt: " + Files.readTextFile( Directories.getFile( fs.getRootDirectory(), new RelativeLocation("d1/text2.txt")))); } finally { // Don't forget to close the Zip file system when you're done with it. fs.close(); }
If you want to just unzip a Zip or Jar archive, use any of
ZipFiles's unzip
methods.
Example 14.5. Unzipping a Zip file
// Extract all text files from the Zip file texts.zip in directory d into a // directory tree under the directory d1. ZipFiles.unzip( Directories.getFile(d, "texts.zip"), d1, new EFileNameExtensionFilter("txt"));
Table of Contents
An application often has to deal with both EntityFS entities and plain, old Java IO File:s[5]. This appendix shows how you can move between the two worlds with the least effort.
Depending on what the File should be used for, there are several methods available that can represent it as an EntityFS entity.
This method gives most flexibility in configuring how the desired entities should behave, but it is perhaps also the method requiring the most effort to use.
Example A.1. Creating a custom file system on the File
// f is a file File called f // Create a read/write file system on the file's parent directory. FileSystem fs = new FSRWFileSystemBuilder(). setRoot(f.getParentFile()). create(); // This is the file, represented as a file entity EFile ef = Directories.getFile(fs.getRootDirectory(), "f");
The utility class FileSystems has the
getEntityForDirectory
and
getEntityForFile
shortcut methods for creating
file systems on File objects.
Example A.2. Creating a file system on the File using a FileSystems method
// f is a file File called f // The returned EFile entity is in a file system created using the default // settings of FSRWFileSystemBuilder EFile ef = FileSystems.getEntityForFile(f, false);
If the File references a location in an existing
FileSystem, the FSCFileResolvableUtil
methods getEntityForFile
or
getEntityLocationForFile
can be used to retrieve the
referenced Entity.
Example A.3. Getting an existing Entity using a FSCFileResolvableUtil method
// f is a file File that references the file entity /d/f in the file // system fs. EFile ef = (EFile) FSCFileResolvableUtil.getEntityForFile(fs, f);
A File can be adapted different interfaces using an adapter object.
Table A.1. File adapters
File adapter | Implements interface |
---|---|
FileReadableFile | ReadableFile |
ReadWritableFileAdapter | ReadWritableFile |
Example A.4. Adapting a File to the ReadableFile interface
// f is a File containing text // Use the Files utility class to read the contents of the text file // to a String String s = Files.readTextFile(new FileReadableFile(f));
Conversions from entity objects to File objects can be useful when
working with API:s that use File arguments. File objects can
be created for an entities in a file system that is
FSCFileResolvable using the
FSCFileResolvableUtil method
getFileForEntity
.
Example A.5. Getting a File object referencing an entity
// Get a File object referencing the EFile entity ef File f = FSCFileResolvableUtil.getFileForEntity(ef);
[GoF] 0-201-63361-2. Addison-Wesley. Design Patterns: Elements of Reusable Object-Oriented Software.
[Wikipedia article on checked exceptions] Wikipedia article on checked exceptions.
[Domain Driven Design] 0-321-12521-5. Addison Wesley. Domain Driven Design, Tackling Complexity in the Heart of Software.
[Architecture overview] EntityFS Architecture Overview.
[Inheritance chart for all entity objcets] Inheritance chart for all entity objects.
[Inheritance chart for EFile] Inheritance chart for EFile.
[Inheritance chart for Directory] Inheritance chart for Directory.
[The Wikipedia article on glob] Wikipedia article on glob.