Class BufferedUpdatesStream

  • All Implemented Interfaces:
    Accountable

    final class BufferedUpdatesStream
    extends java.lang.Object
    implements Accountable
    Tracks the stream of FrozenBufferedUpdates. When DocumentsWriterPerThread flushes, its buffered deletes and updates are appended to this stream and immediately resolved (to actual docIDs, per segment) using the indexing thread that triggered the flush for concurrency. When a merge kicks off, we sync to ensure all resolving packets complete. We also apply to all segments when NRT reader is pulled, commit/close is called, or when too many deletes or updates are buffered and must be flushed (by RAM usage or by count). Each packet is assigned a generation, and each flushed or merged segment is also assigned a generation, so we can track which BufferedDeletes packets to apply to any given segment.
    • Field Detail

      • nextGen

        private long nextGen
      • bytesUsed

        private final java.util.concurrent.atomic.AtomicLong bytesUsed
      • numTerms

        private final java.util.concurrent.atomic.AtomicInteger numTerms
    • Constructor Detail

      • BufferedUpdatesStream

        BufferedUpdatesStream​(InfoStream infoStream)
    • Method Detail

      • getPendingUpdatesCount

        int getPendingUpdatesCount()
      • clear

        void clear()
        Only used by IW.rollback
      • any

        boolean any()
      • numTerms

        int numTerms()
      • ramBytesUsed

        public long ramBytesUsed()
        Description copied from interface: Accountable
        Return the memory usage of this object in bytes. Negative values are illegal.
        Specified by:
        ramBytesUsed in interface Accountable
      • waitApplyAll

        void waitApplyAll​(IndexWriter writer)
                   throws java.io.IOException
        Waits for all in-flight packets, which are already being resolved concurrently by indexing threads, to finish. Returns true if there were any new deletes or updates. This is called for refresh, commit.
        Throws:
        java.io.IOException
      • stillRunning

        boolean stillRunning​(long delGen)
        Returns true if this delGen is still running.
      • finishedSegment

        void finishedSegment​(long delGen)
      • finished

        void finished​(FrozenBufferedUpdates packet)
        Called by indexing threads once they are fully done resolving all deletes for the provided delGen. We track the completed delGens and record the maximum delGen for which all prior delGens, inclusive, are completed, so that it's safe for doc values updates to apply and write.
      • getCompletedDelGen

        long getCompletedDelGen()
        All frozen packets up to and including this del gen are guaranteed to be finished.
      • waitApplyForMerge

        void waitApplyForMerge​(java.util.List<SegmentCommitInfo> mergeInfos,
                               IndexWriter writer)
                        throws java.io.IOException
        Waits only for those in-flight packets that apply to these merge segments. This is called when a merge needs to finish and must ensure all deletes to the merging segments are resolved.
        Throws:
        java.io.IOException
      • waitApply

        private void waitApply​(java.util.Set<FrozenBufferedUpdates> waitFor,
                               IndexWriter writer)
                        throws java.io.IOException
        Throws:
        java.io.IOException
      • getNextGen

        long getNextGen()
      • checkDeleteStats

        private boolean checkDeleteStats()