Package org.exoplatform.documents.storage.jcr.search

Class DocumentPagedTextExtractor

java.lang.Object
org.exoplatform.documents.storage.jcr.search.DocumentPagedTextExtractor
public class DocumentPagedTextExtractor extends Object
Reads a document's text part by part, the whole text being extracted once per version of the file into a node-local cache file, so a document far larger than what is extracted on the fly, a book full of pictures for instance, can be read in parts, or one section at a time.

The file is copied to a temporary file and parsed from it. A Word document, a presentation, a PDF and a spreadsheet get their outline from the same walk that writes their text, the PDF being parsed with its buffers in temporary files and the spreadsheet streamed a row at a time, its shared strings held in memory. Any other supported type, an older Office or an OpenDocument file for instance, is read by the platform's document readers, without an outline; most of them load the whole file, so reading one takes a heap of at least the file's size, up to the largest file read, the same limit as the largest file a user can attach in the UI; and the one that builds its whole text before giving any is not stopped at the timeout, which is checked as the text is written: it keeps the extraction thread until it ends. One file is extracted at a time; a caller waits for its extraction a bounded time, the extraction going on after it so a later call finds the text. The text is stored as UTF-16, two bytes a character, so a part is read at its offset without reading what comes before it. The cache is a node's own: another node of a cluster extracts the file again on its first part.

  • Constructor Summary

    Constructors
    Constructor
    Description
    DocumentPagedTextExtractor(org.exoplatform.services.jcr.RepositoryService repositoryService, org.exoplatform.services.document.DocumentReaderService documentReaderService, DocumentTextExtractor formats, long maxFileSizeBytes, long maxChars, long timeoutMillis, long waitMillis, long maxCacheBytes, Path cacheDirectory)
    Builds the extractor.

  • Method Summary

    Modifier and Type
    Method
    Description
    protected PagedTextExtraction
    extract(String documentId, String key)
    Extracts the whole text of a file into a cache file.
    org.exoplatform.documents.model.DocumentTextPage
    getPage(String documentId, long offset, int length)
    Reads a part of a document's text, with its length and its outline.
    protected javax.jcr.Session
    openSystemSession()
    Opens a system session on the collaboration workspace, the caller having checked the user can read the document.
    void
    shutdown()
    Stops the extraction thread.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • DocumentPagedTextExtractor

      public DocumentPagedTextExtractor(org.exoplatform.services.jcr.RepositoryService repositoryService, org.exoplatform.services.document.DocumentReaderService documentReaderService, DocumentTextExtractor formats, long maxFileSizeBytes, long maxChars, long timeoutMillis, long waitMillis, long maxCacheBytes, Path cacheDirectory) throws IOException
      Builds the extractor.
      Parameters:
      repositoryService - the repository holding the files
      documentReaderService - the platform's document readers
      formats - the on-the-fly extractor, whose supported types and type resolution are shared
      maxFileSizeBytes - the largest file read
      maxChars - the most characters kept from one file
      timeoutMillis - the longest one extraction runs
      waitMillis - the longest a caller waits for an extraction
      maxCacheBytes - the most bytes the cache files take
      cacheDirectory - the directory of the cache files, emptied here
      Throws:
      IOException - when the directory can't be created or emptied

  • Method Details

    • getPage

      public org.exoplatform.documents.model.DocumentTextPage getPage(String documentId, long offset, int length)
      Reads a part of a document's text, with its length and its outline.
      Parameters:
      documentId - the document id
      offset - the first character to read
      length - the most characters to read, 0 for none; a part that would hold only halves of surrogate pairs gets one more character, so it gives a pair whole
      Returns:
      the part, or why there is no text

    • shutdown

      public void shutdown()
      Stops the extraction thread.

    • extract

      protected PagedTextExtraction extract(String documentId, String key) throws Exception
      Extracts the whole text of a file into a cache file.
      Parameters:
      documentId - the file id
      key - the version key naming the cache file
      Returns:
      the extraction
      Throws:
      Exception - when the repository fails

    • openSystemSession

      protected javax.jcr.Session openSystemSession() throws javax.jcr.RepositoryException
      Opens a system session on the collaboration workspace, the caller having checked the user can read the document.
      Returns:
      the session
      Throws:
      javax.jcr.RepositoryException - when the repository fails