Package org.apache.lucene.analysis.fa
Class PersianAnalyzer
- java.lang.Object
-
- org.apache.lucene.analysis.Analyzer
-
- org.apache.lucene.analysis.StopwordAnalyzerBase
-
- org.apache.lucene.analysis.fa.PersianAnalyzer
-
- All Implemented Interfaces:
java.io.Closeable
,java.lang.AutoCloseable
public final class PersianAnalyzer extends StopwordAnalyzerBase
Analyzer
for Persian.This Analyzer uses
PersianCharFilter
which implies tokenizing around zero-width non-joiner in addition to whitespace. Some persian-specific variant forms (such as farsi yeh and keheh) are standardized. "Stemming" is accomplished via stopwords.- Since:
- 3.1
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description private static class
PersianAnalyzer.DefaultSetHolder
Atomically loads the DEFAULT_STOP_SET in a lazy fashion once the outer class accesses the static final set the first time.;-
Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer
Analyzer.ReuseStrategy, Analyzer.TokenStreamComponents
-
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String
DEFAULT_STOPWORD_FILE
File containing default Persian stopwords.static java.lang.String
STOPWORDS_COMMENT
The comment character in the stopwords file.-
Fields inherited from class org.apache.lucene.analysis.StopwordAnalyzerBase
stopwords
-
Fields inherited from class org.apache.lucene.analysis.Analyzer
GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY
-
-
Constructor Summary
Constructors Constructor Description PersianAnalyzer()
Builds an analyzer with the default stop words:DEFAULT_STOPWORD_FILE
.PersianAnalyzer(CharArraySet stopwords)
Builds an analyzer with the given stop words
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description protected Analyzer.TokenStreamComponents
createComponents(java.lang.String fieldName)
CreatesAnalyzer.TokenStreamComponents
used to tokenize all the text in the providedReader
.static CharArraySet
getDefaultStopSet()
Returns an unmodifiable instance of the default stop-words set.protected java.io.Reader
initReader(java.lang.String fieldName, java.io.Reader reader)
Wraps the Reader withPersianCharFilter
protected TokenStream
normalize(java.lang.String fieldName, TokenStream in)
Wrap the givenTokenStream
in order to apply normalization filters.-
Methods inherited from class org.apache.lucene.analysis.StopwordAnalyzerBase
getStopwordSet, loadStopwordSet, loadStopwordSet, loadStopwordSet
-
Methods inherited from class org.apache.lucene.analysis.Analyzer
attributeFactory, close, getOffsetGap, getPositionIncrementGap, getReuseStrategy, getVersion, initReaderForNormalization, normalize, setVersion, tokenStream, tokenStream
-
-
-
-
Field Detail
-
DEFAULT_STOPWORD_FILE
public static final java.lang.String DEFAULT_STOPWORD_FILE
File containing default Persian stopwords. Default stopword list is from http://members.unine.ch/jacques.savoy/clef/index.html The stopword list is BSD-Licensed.- See Also:
- Constant Field Values
-
STOPWORDS_COMMENT
public static final java.lang.String STOPWORDS_COMMENT
The comment character in the stopwords file. All lines prefixed with this will be ignored- See Also:
- Constant Field Values
-
-
Constructor Detail
-
PersianAnalyzer
public PersianAnalyzer()
Builds an analyzer with the default stop words:DEFAULT_STOPWORD_FILE
.
-
PersianAnalyzer
public PersianAnalyzer(CharArraySet stopwords)
Builds an analyzer with the given stop words- Parameters:
stopwords
- a stopword set
-
-
Method Detail
-
getDefaultStopSet
public static CharArraySet getDefaultStopSet()
Returns an unmodifiable instance of the default stop-words set.- Returns:
- an unmodifiable instance of the default stop-words set.
-
createComponents
protected Analyzer.TokenStreamComponents createComponents(java.lang.String fieldName)
CreatesAnalyzer.TokenStreamComponents
used to tokenize all the text in the providedReader
.- Specified by:
createComponents
in classAnalyzer
- Parameters:
fieldName
- the name of the fields content passed to theAnalyzer.TokenStreamComponents
sink as a reader- Returns:
Analyzer.TokenStreamComponents
built from aStandardTokenizer
filtered withLowerCaseFilter
,DecimalDigitFilter
,ArabicNormalizationFilter
,PersianNormalizationFilter
and Persian Stop words
-
normalize
protected TokenStream normalize(java.lang.String fieldName, TokenStream in)
Description copied from class:Analyzer
Wrap the givenTokenStream
in order to apply normalization filters. The default implementation returns theTokenStream
as-is. This is used byAnalyzer.normalize(String, String)
.
-
initReader
protected java.io.Reader initReader(java.lang.String fieldName, java.io.Reader reader)
Wraps the Reader withPersianCharFilter
- Overrides:
initReader
in classAnalyzer
- Parameters:
fieldName
- IndexableField name being indexedreader
- original Reader- Returns:
- reader, optionally decorated with CharFilter(s)
-
-