LuceneDataSearchEngine (JCMS API)

java.lang.Object
- com.jalios.jcms.search.LuceneDataSearchEngine

All Implemented Interfaces:

JcmsConstants, JaliosConstants

Direct Known Subclasses:

LuceneCategorySearchEngine, LucenePublicationSearchEngine
```
public abstract class LuceneDataSearchEngine
extends java.lang.Object
implements JcmsConstants
```
This class provides a base class to index JCMS Data using Lucene.

Since:

jcms-5.5.0

Version:

$Revision: 77430 $

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`LuceneDataSearchEngine.IndexingLatch` CountDownLatch used to be notified of the end of the next indexing operation taking place on a Data.

Field Summary

Fields
Modifier and Type	Field and Description
`protected AlarmManager`	`alarmMgr`
`protected Channel`	`channel`
`protected java.lang.String`	`directoryName`
`static java.lang.String`	`ID_FIELD`
`protected java.lang.Object`	`indexAccessLock`
`static java.lang.String`	`INDEXING_DATE_EXTRAINFO`
`static java.lang.String`	`INDEXING_DATE_FIELD`
`protected java.util.List<java.lang.String>`	`langList` Languages in which the engine index and search its content.
`protected java.util.Map<java.lang.String,org.apache.lucene.store.FSDirectory>`	`langToIndexDirMap` Lucene FSDirectory in which the engine index and search its content.
`static java.lang.String`	`MAX_BUFFERED_DOCS` Optionnal integer property name to define the maxBufferedDocs when writing in lucene index
`static java.lang.String`	`MAX_FIELD_LENGTH` Optionnal integer property name to define the maxFieldLength when writing in lucene index
`static java.lang.String`	`MAX_MERGE_DOCS` Optionnal integer property name to define the maxMergeDocs when writing in lucene index
`static java.lang.String`	`MERGE_FACTOR` Optionnal integer property name to define the mergeFactor when writing in lucene index
`protected boolean`	`multilingual`
`static java.lang.String`	`REVISION`

Fields inherited from interface com.jalios.jcms.JcmsConstants
ADATE_SEARCH, ADMIN_NOTES_PROP, ADVANCED_TAB, AJAX_MODE_ATTR, ARCHIVES_DIR, ASCII_WIDTH, CATEGORY_TAB, CDATE_SEARCH, COMMON_ALARM, CONTENT_TAB, COOKIE_MAX_AGE, COUNTRY_SPRITE, CTRL_TOPIC_INTERNAL, CTRL_TOPIC_REF, CTRL_TOPIC_VALUE, CTRL_TOPIC_WRITE, CUSTOM_PROP, DOCCHOOSER_HEIGHT, DOCCHOOSER_WIDTH, DOCS_DIR, EDATE_SEARCH, EMAIL_REGEXP, ERROR_MSG, FORBIDDEN_FILE_ACCESS, FORBIDDEN_REDIRECT, FORCE_REDIRECT, GLYPH_ICON_PREFIX, ICON_ARCHIVE, ICON_ICON_PREFIX, ICON_LOCK, ICON_LOCK_STRONG, ICON_PREFIX_PROP, ICON_WARN, ICON_WH_BOOK_CLOSED, ICON_WH_BOOK_OPEN, INFORMATION_MSG, JALIOS_JUNIT_PROP, JCMS_CADDY, JCMS_MSG_LIST, JSYNC_DOWNLOAD_DIR, JSYNC_SYNC_ALARM, LANG_SPRITE, LOG_FILE, LOG_TOPIC_SECURITY, LOGGER_PROP, LOGGER_XMLPROP, MBR_PHOTO_DIR, MDATE_SEARCH, MONITOR_XML, OP_CREATE, OP_CREATE_STR, OP_DEEP_COPY, OP_DEEP_COPY_STR, OP_DEEP_DELETE, OP_DEEP_DELETE_STR, OP_DELETE, OP_DELETE_STR, OP_MERGE, OP_MERGE_STR, OP_UPDATE, OP_UPDATE_STR, PDATE_SEARCH, PHOTO_DIR, PHOTO_ICON, PHOTO_ICON_HEIGHT, PHOTO_ICON_PROP_PREFIX, PHOTO_ICON_WIDTH, PHOTO_LARGE, PHOTO_LARGE_HEIGHT, PHOTO_LARGE_PROP_PREFIX, PHOTO_LARGE_WIDTH, PHOTO_NORMAL, PHOTO_NORMAL_HEIGHT, PHOTO_NORMAL_PROP_PREFIX, PHOTO_NORMAL_WIDTH, PHOTO_SMALL, PHOTO_SMALL_HEIGHT, PHOTO_SMALL_PROP_PREFIX, PHOTO_SMALL_WIDTH, PHOTO_TINY, PHOTO_TINY_HEIGHT, PHOTO_TINY_PROP_PREFIX, PHOTO_TINY_WIDTH, PREVIOUS_TAB, PRINT_VIEW, PRIVATE_FILE_ACCESS, PUBLIC_FILE_ACCESS, READ_RIGHT_TAB, SDATE_SEARCH, SEARCHENGINE_ALARM, SESSION_AUTHORIZED_FILENAMES_SET, SPRITE_ICON_PREFIX, STATS_REPORT_DIR, STATUS_PROP, STORE_XML, SUCCESS_MSG, TEMPLATE_TAB, THUMBNAIL_LARGE_HEIGHT, THUMBNAIL_LARGE_WIDTH, THUMBNAIL_SMALL_HEIGHT, THUMBNAIL_SMALL_WIDTH, TTCARD_MEDIA_HEIGHT, TTCARD_MEDIA_WIDTH, TYPES_ICON_ALT_PROP, TYPES_ICON_SUFFIX_PROP, TYPES_ICON_TITLE_PROP, TYPES_PREFIX_PROP, TYPES_THUMB_SUFFIX_PROP, UDATE_SEARCH, UPDATE_RIGHT_TAB, UPLOAD_DIR, URL_REGEXP, WARNING_MSG, WEBAPP_PROP, WFEXPRESS_ALARM, WFREMINDER_ALARM, WORKFLOW_TAB, WORKFLOW_XML

Fields inherited from interface com.jalios.util.JaliosConstants
CRLF, MILLIS_IN_ONE_DAY, MILLIS_IN_ONE_HOUR, MILLIS_IN_ONE_MINUTE, MILLIS_IN_ONE_MONTH, MILLIS_IN_ONE_SECOND, MILLIS_IN_ONE_WEEK, MILLIS_IN_ONE_YEAR

Constructor Summary

Constructors
Modifier	Constructor and Description
`protected`	`LuceneDataSearchEngine(java.lang.String directoryName, boolean multilingual)` Construct a new Lucene Data Search Engine given a directory name.

Method Summary

Methods
Modifier and Type	Method and Description
`protected void`	`addData(Data data)` Add given `Data` to this lucene search engine.
`protected void`	`addDataCollection(java.util.Collection<? extends Data> coll)` Add given `Collection` of `Data` to this lucene search engine.
`void`	`clearIndices()` Delete all `Document` from all indices (overwrite existing index with a new one).
`protected void`	`clearSearcher()` Close current searchers and clear it for future renewal.
`protected void`	`deleteData(Data data)` Delete given `Data` from this lucene search engine.
`protected void`	`deleteDataCollection(java.util.Collection<? extends Data> coll)` Delete given `Collection` of `Data` from this lucene search engine.
`protected abstract com.jalios.jcms.search.DataIterator<Data>`	`getAllDataIterator()` This methods must be implemented by the LuceneSearchEngine.
`org.apache.lucene.store.FSDirectory`	`getDirectory(java.lang.String lang)` Returns the lucene directory used for the specified language.
`java.util.Date`	`getIndexingDate(Data data)` Retrieve the Date at which the specified Data was indexed in the main language of the site.
`java.util.Date`	`getIndexingDate(Data data, java.lang.String lang)` Returns the date at which the specified Data has been indexed for the specified language.
`LuceneDataSearchEngine.IndexingLatch`	`getIndexingLatch(Data data)` Retrieve a new IndexingLatch useful to be notified of the end of the next indexing operation to take place on the specified Data.
`java.util.Date`	`getLastOptimizeDateSinceRestart()`
`long`	`getLastOptimizeDuration()`
`java.util.Date`	`getLastReindexDateSinceRestart()`
`long`	`getLastReindexDuration()`
`protected abstract org.apache.log4j.Logger`	`getLogger()` This methods must be implemented by the LuceneSearchEngine.
`org.apache.lucene.document.Document`	`getLuceneDocument(Data data, java.lang.String lang)` Returns the lucene Document corresponding to the specified Data in the index of the the specified language.
`long`	`getOperationStartTime()`
`protected org.apache.lucene.index.Term`	`getPrimaryTerm(Data data)` Retrieve the a lucene Terme suitable for use as primary key when searching/removing/updating a unique lucene document for the specified data
`int`	`getProgressState()`
`protected org.apache.lucene.search.Searcher`	`getSearcher(java.lang.String[] languages)` Retrieve a valid lucene Searcher instance
`protected void`	`index(org.apache.lucene.store.Directory directory, java.util.Collection<? extends Data> coll, java.lang.String lang)` Index a `Collection` of `Data` into lucene.
`protected void`	`index(org.apache.lucene.store.Directory directory, java.util.Iterator<? extends Data> iterator, java.lang.String lang)` Index all `Data` returned by the specified `Iterator` into lucene.
`protected abstract void`	`indexData(org.apache.lucene.index.IndexWriter writer, Data data, java.lang.String lang)` This methods must be implemented by the LuceneSearchEngine.
`boolean`	`isOperationRunning()`
`void`	`optimizeIndices()` Optimize all incides of the LuceneSearchEngine.
`void`	`reindexAll()` Clears the lucene indices of this searchengine, reindex all content retrieved using protected method getAllDataIterator().
`void`	`remove(org.apache.lucene.store.Directory directory, java.util.Collection<? extends Data> coll)` Remove a `Collection` of `Data` from the lucene index.
`static void`	`setIndexWriterOptions(org.apache.lucene.index.IndexWriter writer)` Set IndexWriter options retrieved from hooks and channel properties.
`protected void`	`updateData(Data data)` Update given `Data` in this lucene search engine.
`protected void`	`updateDataCollection(java.util.Collection<? extends Data> coll)` Update given `Collection` of `Data` in this lucene search engine.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - REVISION
```
public static final java.lang.String REVISION
```
    See Also:
    Constant Field Values
  - ID_FIELD
```
public static final java.lang.String ID_FIELD
```
    See Also:
    Constant Field Values
  - INDEXING_DATE_FIELD
```
public static final java.lang.String INDEXING_DATE_FIELD
```
    See Also:
    Constant Field Values
  - INDEXING_DATE_EXTRAINFO
```
public static final java.lang.String INDEXING_DATE_EXTRAINFO
```
    See Also:
    Constant Field Values
  - channel
```
protected final Channel channel
```
  - alarmMgr
```
protected final AlarmManager alarmMgr
```
  - directoryName
```
protected final java.lang.String directoryName
```
  - multilingual
```
protected final boolean multilingual
```
  - langList
```
protected final java.util.List<java.lang.String> langList
```
    Languages in which the engine index and search its content.
    - all site languages for multilingual engine
    - default site language for monolingual engine
  - langToIndexDirMap
```
protected final java.util.Map<java.lang.String,org.apache.lucene.store.FSDirectory> langToIndexDirMap
```
    Lucene FSDirectory in which the engine index and search its content.
    - one Directory per site languages for multilingual engine
    - only one Directory in the default site language, for monolingual engine
  - indexAccessLock
```
protected final java.lang.Object indexAccessLock
```
  - MERGE_FACTOR
```
public static final java.lang.String MERGE_FACTOR
```
    Optionnal integer property name to define the mergeFactor when writing in lucene index
  - MAX_MERGE_DOCS
```
public static final java.lang.String MAX_MERGE_DOCS
```
    Optionnal integer property name to define the maxMergeDocs when writing in lucene index
  - MAX_BUFFERED_DOCS
```
public static final java.lang.String MAX_BUFFERED_DOCS
```
    Optionnal integer property name to define the maxBufferedDocs when writing in lucene index
  - MAX_FIELD_LENGTH
```
public static final java.lang.String MAX_FIELD_LENGTH
```
    Optionnal integer property name to define the maxFieldLength when writing in lucene index
- Constructor Detail
  - LuceneDataSearchEngine
```
protected LuceneDataSearchEngine(java.lang.String directoryName,
                      boolean multilingual)
                          throws java.lang.Exception
```
    Construct a new Lucene Data Search Engine given a directory name.
    
    Parameters:
    directoryName - the name of the directory to create
    multilingual - true to use one index per language, false to use only one index
    
    Throws:
    
    java.lang.Exception - on any error
- Method Detail
  - setIndexWriterOptions
```
public static void setIndexWriterOptions(org.apache.lucene.index.IndexWriter writer)
```
    Set IndexWriter options retrieved from hooks and channel properties.
    
    Parameters:
    writer - the IndexWriter for option will be modified
    Since:
    
    jcms-6.0.3, jcms-6.1.2
  - index
```
protected void index(org.apache.lucene.store.Directory directory,
         java.util.Collection<? extends Data> coll,
         java.lang.String lang)
              throws java.io.IOException
```
    Index a Collection of Data into lucene.
    This method is NOT synchronized, the caller is responsible to do it!
    
    Throws:
    
    java.io.IOException
  - index
```
protected void index(org.apache.lucene.store.Directory directory,
         java.util.Iterator<? extends Data> iterator,
         java.lang.String lang)
              throws java.io.IOException
```
    Index all Data returned by the specified Iterator into lucene.
    This method is NOT synchronized, the caller is responsible to do it!
    
    Throws:
    
    java.io.IOException
  - remove
```
public void remove(org.apache.lucene.store.Directory directory,
          java.util.Collection<? extends Data> coll)
            throws java.io.IOException
```
    Remove a Collection of Data from the lucene index.
    This method is NOT synchronized, the caller is responsible to do it!
    
    Parameters:
    directory - The Lucene directory in which to remove the Datas
    coll - a collection of Data, must not be null
    
    Throws:
    
    java.io.IOException - if the directory could not be opened or deletion could not be performed
  - getPrimaryTerm
```
protected org.apache.lucene.index.Term getPrimaryTerm(Data data)
```
    Retrieve the a lucene Terme suitable for use as primary key when searching/removing/updating a unique lucene document for the specified data
    
    Returns:
    a Term instance, must not return null
  - getDirectory
```
public org.apache.lucene.store.FSDirectory getDirectory(java.lang.String lang)
```
    Returns the lucene directory used for the specified language.
    Warning!!! you should not modify the index (LuceneDataSearchEngine rely on its own modification for optimization purpose), use this method only to access the directory in readonly !.
    
    Parameters:
    lang - the language of JCMS (ISO-639) in which to retrieve the Directory
    
    Returns:
    the FSDirectory of the specified language or null if no Directory is available for this language.
  - getLuceneDocument
```
public org.apache.lucene.document.Document getLuceneDocument(Data data,
                                                    java.lang.String lang)
```
    Returns the lucene Document corresponding to the specified Data in the index of the the specified language.
    
    Parameters:
    data - the Data being looked for
    lang - the language in which to check
    
    Returns:
    a lucene Document or null if could not be found
    Since:
    
    jcms-6.0.1
  - getIndexingDate
```
public java.util.Date getIndexingDate(Data data,
                             java.lang.String lang)
```
    Returns the date at which the specified Data has been indexed for the specified language.
    
    Parameters:
    data - the Data being looked for
    lang - the language in which to check
    
    Returns:
    a Date or null if it could not be found
    Since:
    
    jcms-6.0.1
  - getIndexingDate
```
public java.util.Date getIndexingDate(Data data)
```
    Retrieve the Date at which the specified Data was indexed in the main language of the site.
    
    Parameters:
    data - the Data for which to retrieve the indexing date.
    
    Returns:
    the indexing date of the Data or null if was not indexed.
    Since:
    
    jcms-6.0.1
  - addData
```
protected void addData(Data data)
```
    Add given Data to this lucene search engine. This method is asynchronous, the given data may not be (and will certainly not be) available immediately after call.
  - updateData
```
protected void updateData(Data data)
```
    Update given Data in this lucene search engine. This method is asynchronous, the given data may not be (and will certainly not be) available immediately after call.
  - deleteData
```
protected void deleteData(Data data)
```
    Delete given Data from this lucene search engine. This method is asynchronous, the given data may not be (and will certainly not be) available immediately after call.
  - addDataCollection
```
protected void addDataCollection(java.util.Collection<? extends Data> coll)
```
    Add given Collection of Data to this lucene search engine. This method is asynchronous, the given datas may not be (and will certainly not be) available immediately after call.
  - updateDataCollection
```
protected void updateDataCollection(java.util.Collection<? extends Data> coll)
```
    Update given Collection of Data in this lucene search engine. This method is asynchronous, the given datas may not be (and will certainly not be) available immediately after call.
  - deleteDataCollection
```
protected void deleteDataCollection(java.util.Collection<? extends Data> coll)
```
    Delete given Collection of Data from this lucene search engine. This method is asynchronous, the given datas may not be (and will certainly not be) available immediately after call.
  - clearIndices
```
public void clearIndices()
```
    Delete all Document from all indices (overwrite existing index with a new one). Warning: this operation is undoable! It is run against the indexing thread, it will not return as long as the indexing process is not done, and will block the indexing thread from running when doing its job.
  - optimizeIndices
```
public void optimizeIndices()
```
    Optimize all incides of the LuceneSearchEngine. Warning: This is a potentially long and heavy process on large index, do not call without being sure of what you do. It is run against the indexing thread, it will not return as long as the indexing process is not done, and will block the indexing thread from running when doing its job.
  - getLastOptimizeDateSinceRestart
```
public java.util.Date getLastOptimizeDateSinceRestart()
```
    Returns:
    a date indicating the last time the optimize was done, or null if no optimization was done.
  - getLastOptimizeDuration
```
public long getLastOptimizeDuration()
```
    Returns:
    a duration in millisecond indicating the duration of the last optimize operation since restart (or 0 if none occured).
  - reindexAll
```
public void reindexAll()
                throws java.io.IOException
```
    Clears the lucene indices of this searchengine, reindex all content retrieved using protected method getAllDataIterator(). It is run against the indexing thread, it will not return as long as the indexing process is not done, and will block the indexing thread from running when doing its job. You can access status regarding operation progress using : isOperationRunning() and getProgressState() .
    
    Throws:
    
    java.io.IOException - if an error occurs during indexing
  - getLastReindexDateSinceRestart
```
public java.util.Date getLastReindexDateSinceRestart()
```
    Returns:
    a date indicating the last time the reindex was done, or null if no reindex was done.
  - getLastReindexDuration
```
public long getLastReindexDuration()
```
    Returns:
    a duration in millisecond indicating the duration of the last reindex operation since restart (or 0 if none occured).
  - getSearcher
```
protected org.apache.lucene.search.Searcher getSearcher(java.lang.String[] languages)
                                                 throws java.io.IOException
```
    Retrieve a valid lucene Searcher instance
    
    Parameters:
    languages - The languages for which a searcher must be retrieved
    
    Returns:
    current available searcher or create a new one if an index changed as occured.
    
    Throws:
    
    java.io.IOException
  - clearSearcher
```
protected void clearSearcher()
```
    Close current searchers and clear it for future renewal. Called after index change.
  - getIndexingLatch
```
public LuceneDataSearchEngine.IndexingLatch getIndexingLatch(Data data)
```
    Retrieve a new IndexingLatch useful to be notified of the end of the next indexing operation to take place on the specified Data.
```
   Data original = ...
   Data update = origina.getUpdateInstance();
   // update.set(...)
   IndexingLatch latch = searchEngine.getIndexingLatch(original);
   update.performUpdate()...;
   latch.await();
 
```
    Parameters:
    data - the Data that must be monitored for indexing
    
    Returns:
    an IndexingLatch instance, or null if specified Data was null
    Since:
    
    jcms-8.0.2, jcms-9, JCMS-3805
  - isOperationRunning
```
public boolean isOperationRunning()
```
    Returns:
    true if an operation of which its progess is being watched (reindexing, optimizing)
    See Also:
    getProgressState()
  - getProgressState
```
public int getProgressState()
```
    Returns:
    a percentage showing current state of operation, or 100 if no operation is running
    See Also:
    isOperationRunning()
  - getOperationStartTime
```
public long getOperationStartTime()
```
    Returns:
    the time at which the current operation was started, or 0 if no operation is running
    See Also:
    isOperationRunning()
  - getLogger
```
protected abstract org.apache.log4j.Logger getLogger()
```
    This methods must be implemented by the LuceneSearchEngine. It must return the logger to be used for log messages.
    
    Returns:
    Logger of this engine.
  - getAllDataIterator
```
protected abstract com.jalios.jcms.search.DataIterator<Data> getAllDataIterator()
```
    This methods must be implemented by the LuceneSearchEngine. It must return a DataIterator used to iterate on all Data to index. Used by reindexAll().
  - indexData
```
protected abstract void indexData(org.apache.lucene.index.IndexWriter writer,
             Data data,
             java.lang.String lang)
                           throws java.io.IOException
```
    This methods must be implemented by the LuceneSearchEngine. It must index the given data in the given language, into the given index writer.
    
    Throws:
    
    java.io.IOException

Class LuceneDataSearchEngine

Nested Class Summary

Field Summary

Fields inherited from interface com.jalios.jcms.JcmsConstants

Fields inherited from interface com.jalios.util.JaliosConstants

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

REVISION

ID_FIELD

INDEXING_DATE_FIELD

INDEXING_DATE_EXTRAINFO

channel

alarmMgr

directoryName

multilingual

langList

langToIndexDirMap

indexAccessLock

MERGE_FACTOR

MAX_MERGE_DOCS

MAX_BUFFERED_DOCS

MAX_FIELD_LENGTH

Constructor Detail

LuceneDataSearchEngine

Method Detail

setIndexWriterOptions

index

index

remove

getPrimaryTerm

getDirectory

getLuceneDocument

getIndexingDate

getIndexingDate

addData

updateData

deleteData

addDataCollection

updateDataCollection

deleteDataCollection

clearIndices

optimizeIndices

getLastOptimizeDateSinceRestart

getLastOptimizeDuration

reindexAll

getLastReindexDateSinceRestart

getLastReindexDuration

getSearcher

clearSearcher

getIndexingLatch

isOperationRunning

getProgressState

getOperationStartTime

getLogger

getAllDataIterator

indexData