Class NativeUnixDirectory

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable

    public class NativeUnixDirectory
    extends FSDirectory
    A Directory implementation for all Unixes that uses DIRECT I/O to bypass OS level IO caching during merging. For all other cases (searching, writing) we delegate to the provided Directory instance.

    See Overview for more details.

    To use this you must compile NativePosixUtil.cpp (exposes Linux-specific APIs through JNI) for your platform, by running ant build-native-unix, and then putting the resulting libNativePosixUtil.so (from lucene/build/native) onto your dynamic linker search path.

    WARNING: this code is very new and quite easily could contain horrible bugs. For example, here's one known issue: if you use seek in IndexOutput, and then write more than one buffer's worth of bytes, then the file will be wrong. Lucene does not do this today (only writes small number of bytes after seek), but that may change.

    This directory passes Solr and Lucene tests on Linux and OS X; other Unixes should work but have not been tested! Use at your own risk.

    • Field Detail

      • DEFAULT_MERGE_BUFFER_SIZE

        public static final int DEFAULT_MERGE_BUFFER_SIZE
        Default buffer size before writing to disk (256 KB); larger means less IO load but more RAM and direct buffer storage space consumed during merging.
        See Also:
        Constant Field Values
      • DEFAULT_MIN_BYTES_DIRECT

        public static final long DEFAULT_MIN_BYTES_DIRECT
        Default min expected merge size before direct IO is used (10 MB):
        See Also:
        Constant Field Values
      • mergeBufferSize

        private final int mergeBufferSize
      • minBytesDirect

        private final long minBytesDirect
    • Constructor Detail

      • NativeUnixDirectory

        public NativeUnixDirectory​(java.nio.file.Path path,
                                   int mergeBufferSize,
                                   long minBytesDirect,
                                   LockFactory lockFactory,
                                   Directory delegate)
                            throws java.io.IOException
        Create a new NIOFSDirectory for the named location.
        Parameters:
        path - the path of the directory
        lockFactory - to use
        mergeBufferSize - Size of buffer to use for merging. See DEFAULT_MERGE_BUFFER_SIZE.
        minBytesDirect - Merges, or files to be opened for reading, smaller than this will not use direct IO. See DEFAULT_MIN_BYTES_DIRECT
        delegate - fallback Directory for non-merges
        Throws:
        java.io.IOException - If there is a low-level I/O error
      • NativeUnixDirectory

        public NativeUnixDirectory​(java.nio.file.Path path,
                                   LockFactory lockFactory,
                                   Directory delegate)
                            throws java.io.IOException
        Create a new NIOFSDirectory for the named location.
        Parameters:
        path - the path of the directory
        lockFactory - the lock factory to use
        delegate - fallback Directory for non-merges
        Throws:
        java.io.IOException - If there is a low-level I/O error
      • NativeUnixDirectory

        public NativeUnixDirectory​(java.nio.file.Path path,
                                   Directory delegate)
                            throws java.io.IOException
        Create a new NIOFSDirectory for the named location with FSLockFactory.getDefault().
        Parameters:
        path - the path of the directory
        delegate - fallback Directory for non-merges
        Throws:
        java.io.IOException - If there is a low-level I/O error
    • Method Detail

      • openInput

        public IndexInput openInput​(java.lang.String name,
                                    IOContext context)
                             throws java.io.IOException
        Description copied from class: Directory
        Opens a stream for reading an existing file. This method must throw either NoSuchFileException or FileNotFoundException if name points to a non-existing file.
        Specified by:
        openInput in class Directory
        Parameters:
        name - the name of an existing file.
        Throws:
        java.io.IOException - in case of I/O error
      • createOutput

        public IndexOutput createOutput​(java.lang.String name,
                                        IOContext context)
                                 throws java.io.IOException
        Description copied from class: Directory
        Creates a new, empty file in the directory and returns an IndexOutput instance for appending data to this file. This method must throw FileAlreadyExistsException if the file already exists.
        Overrides:
        createOutput in class FSDirectory
        Parameters:
        name - the name of the file to create.
        Throws:
        java.io.IOException - in case of I/O error