<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
<section id="JCR.QueryHandlerConfiguration">
  <title>QueryHandler configuration</title>

  <para>This section shows you how to configure
  <emphasis>QueryHandler</emphasis>: <link
  linkend="JCR.QueryHandlerConfiguration.Indexing_in_clustered_environment">Indexing
  in clustered environment</link>.</para>

  <section id="JCR.QueryHandlerConfiguration.Indexing_in_clustered_environment">
    <title>Indexing in clustered environment</title>

    <para>JCR offers multiple indexing strategies. They include both for
    standalone and clustered environments using the advantages of running in a
    single JVM or doing the best to use all resources available in cluster.
    JCR uses Lucene library as underlying search and indexing engine, but it
    has several limitations that greatly reduce possibilities and limits the
    usage of cluster advantages. That's why eXo JCR offers three strategies
    that are suitable for it's own usecases. They are standalone, clustered
    with shared index, clustered with local indexes and RSync-based. Each one
    has it's pros and cons.</para>

    <para>Stanadlone strategy provides a stack of indexes to achieve greater
    performance within single JVM.</para>

    <mediaobject>
      <imageobject>
        <imagedata align="center"
                   fileref="images/diagram-standalone-index.png" />
      </imageobject>
    </mediaobject>

    <para>It combines in-memory buffer index directory with delayed
    file-system flushing. This index is called "Volatile" and it is invoked in
    searches also. Within some conditions volatile index is flushed to the
    persistent storage (file system) as new index directory. This allows to
    achieve great results for write operations.</para>

    <para>Clustered implementation with local indexes is built upon same
    strategy with volatile in-memory index buffer along with delayed flushing
    on persistent storage.</para>

    <mediaobject>
      <imageobject>
        <imagedata align="center" fileref="images/diagram-local-index.png" />
      </imageobject>
    </mediaobject>

    <para>As this implementation designed for clustered environment it has
    additional mechanisms for data delivery within cluster. Actual text
    extraction jobs done on the same node that does content operations (i.e.
    write operation). Prepared "documents" (Lucene term that means block of
    data ready for indexing) are replicated withing cluster nodes and
    processed by local indexes. So each cluster instance has the same index
    content. When new node joins the cluster it has no initial index, so it
    must be created. There are some supported ways of doing this operation.
    The simplest is to simply copy the index manually but this is not intended
    for use. If no initial index found JCR uses automated sceneries. They are
    controlled via configuration (see "index-recovery-mode" parameter)
    offering full re-indexing from database or copying from another cluster
    node.</para>

    <para>For some reasons having a multiple index copies on each instance can
    be costly. So shared index can be used instead (see diagram below).</para>

    <mediaobject>
      <imageobject>
        <imagedata align="center" fileref="images/diagram-shared-index.png" />
      </imageobject>
    </mediaobject>

    <para>This indexing strategy combines advantages of in-memory index along
    with shared persistent index offering "near" real time search
    capabilities. This means that newly added content is accessible via search
    practically immediately. This strategy allows nodes to index data in their
    own volatile (in-memory) indexes, but persistent indexes are managed by
    single "coordinator" node only. Each cluster instance has a read access
    for shared index to perform queries combining search results found in own
    in-memory index also. Take in account that shared folder must be
    configured in your system environment (i.e. mounted NFS folder). But this
    strategy in some extremely rare cases can have a bit different volatile
    indexes within cluster instances for a while. In a few seconds they will
    be up2date.</para>

    <para>Shared index is consistent and stable enough, but slow, while local
    index is fast, but requires much time for re-synchronization, when cluster
    node is leaving a cluster for a small period of time. RSync-based index
    solves this problem along with local file system advantages in term of
    speed.</para>

    <mediaobject>
      <imageobject>
        <imagedata align="center" fileref="images/diagram-rsync-index.png" />
      </imageobject>
    </mediaobject>

    <para>This strategy is the same shared index, but stores actual data on
    local file system, instead of shared. Eventually triggering a
    synchronization job, that woks on the level of file blocks, synchronizing
    only modified data. Diagram shows it in action. Only single node in the
    cluster is responsible for modifying index files, this is the Coordinator
    node. When data persisted, corresponding command fired, starting
    synchronization jobs all over the cluster.</para>

    <para>See more about <link linkend="JCR.SearchConfiguration">Search
    Configuration</link>.</para>
  </section>

  <section id="JCR.QueryHandlerConfiguration.Configuration">
    <title>Configuration</title>

    <section id="JCR.QueryHandlerConfiguration.Configuration.Query-handlerConfigurationOverview">
      <title>Query-handler configuration overview</title>

      <para>Configuration example:</para>

      <programlisting language="xml">&lt;workspace name="ws"&gt;
   &lt;query-handler class="org.exoplatform.services.jcr.impl.core.query.lucene.SearchIndex"&gt;
      &lt;properties&gt;
         &lt;property name="index-dir" value="shareddir/index/db1/ws" /&gt;
         &lt;property name="changesfilter-class"
            value="org.exoplatform.services.jcr.impl.core.query.ispn.ISPNIndexChangesFilter" /&gt;
         &lt;property name="infinispan-configuration" value="infinispan-indexer.xml" /&gt;
         &lt;property name="jgroups-configuration" value="udp-mux.xml" /&gt;
         &lt;property name="infinispan-cluster-name" value="JCR-cluster" /&gt;
         &lt;property name="max-volatile-time" value="60" /&gt;
         &lt;property name="rdbms-reindexing" value="true" /&gt;
         &lt;property name="reindexing-page-size" value="1000" /&gt;
         &lt;property name="index-recovery-mode" value="from-coordinator" /&gt;
         &lt;property name="index-recovery-filter" value="org.exoplatform.services.jcr.impl.core.query.lucene.DocNumberRecoveryFilter" /&gt;
         &lt;property name="indexing-thread-pool-size" value="16" /&gt;
      &lt;/properties&gt;
   &lt;/query-handler&gt;
&lt;/workspace&gt;
</programlisting>

      <table>
        <title>Config properties description</title>

        <tgroup cols="2">
          <thead>
            <row>
              <entry>Property name</entry>

              <entry>Description</entry>
            </row>
          </thead>

          <tbody>
            <row>
              <entry>index-dir</entry>

              <entry>path to index</entry>
            </row>

            <row>
              <entry>changesfilter-class</entry>

              <entry>The FQN of the class to use to indicate the policy to use
              to manage the lucene indexes changes. This class must extend
              <emphasis>org.exoplatform.services.jcr.impl.core.query.IndexerChangesFilter</emphasis>.
              This must be set in cluster environment to define the clustering
              strategy to adopt. To use the Shared Indexes Strategy, you can
              set it to
              org.exoplatform.services.jcr.impl.core.query.ispn.ISPNIndexChangesFilter.
              I you prefer the Local Indexes Strategy, you can set it to
              org.exoplatform.services.jcr.impl.core.query.ispn.LocalIndexChangesFilter.</entry>
            </row>

            <row>
              <entry>infinispan-configuration</entry>

              <entry>template of Infinispan configuration for all
              query-handlers in repository (search, cache, locks)</entry>
            </row>

            <row>
              <entry>jgroups-configuration</entry>

              <entry>This is the path to JGroups configuration that should not
              be anymore jgroups' stack definitions but a normal jgroups
              configuration format with the shared transport configured by
              simply setting the jgroups property singleton_name to a unique
              name (it must remain unique from one portal container to
              another). This file is also pre-bundled with templates and is
              recommended for use.</entry>
            </row>

            <row>
              <entry>infinispan-cluster-name</entry>

              <entry>cluster name (must be unique)</entry>
            </row>

            <row>
              <entry>max-volatile-time</entry>

              <entry>max time to live for Volatile Index</entry>
            </row>

            <row>
              <entry>rdbms-reindexing</entry>

              <entry>Indicates whether the rdbms re-indexing mechanism must be
              used, the default value is true.</entry>
            </row>

            <row>
              <entry>reindexing-page-size</entry>

              <entry>maximum amount of nodes which can be retrieved from
              storage for re-indexing purpose, the default value is
              100</entry>
            </row>

            <row>
              <entry>index-recovery-mode</entry>

              <entry>If the parameter has been set to
              <command>from-indexing</command>, so a full indexing will be
              automatically launched, if the parameter has been set to
              <command>from-coordinator</command> (default behavior), the
              index will be retrieved from coordinator</entry>
            </row>

            <row>
              <entry>index-recovery-filter</entry>

              <entry>Defines implementation class or classes of
              RecoveryFilters, the mechanism of index synchronization for
              Local Index strategy.</entry>
            </row>

            <row>
              <entry>async-reindexing</entry>

              <entry>Controls the process of re-indexing on JCR's startup. If
              flag set, indexing will be launched asynchronously, without
              blocking the JCR. Default is "false".</entry>
            </row>

            <row>
              <entry>indexing-thread-pool-size</entry>

              <entry>Defines the total amount of indexing threads.</entry>
            </row>

            <row>
              <entry>max-volatile-size</entry>

              <entry>The maximum volatile index size in bytes until it is
              written to disk. The default value is 1048576 (1MB).</entry>
            </row>

            <row>
              <entry>indexing-load-batching-threshold-property</entry>

              <entry>The total amount of properties from which the application
              will decide to get by name all the properties of a node to be
              indexed using one single query instead of one query per
              property. The query used is the equivalent of
              <emphasis>getProperties(String namePattern)</emphasis>. The
              default value is -1 which actually disables this feature. The
              expected value is an integer.</entry>
            </row>

            <row>
              <entry>indexing-load-batching-threshold-node</entry>

              <entry>The total amount of nodes to index within the same
              transaction from which the application will decide to get all
              the properties of the remaining nodes to be indexed using one
              single query instead of one query per property and a query that
              will get the list of properties. The query used is the
              equivalent of <emphasis>getProperties()</emphasis>. The default
              value is -1 which actually disables this feature. The expected
              value is an integer.</entry>
            </row>

            <row>
              <entry>indexing-load-batching-threshold-dynamic</entry>

              <entry>In case
              <emphasis>indexing-load-batching-threshold-property</emphasis>
              and/or
              <emphasis>indexing-load-batching-threshold-node</emphasis> have
              been enabled, you could expect to see the thresholds to be
              updated automatically in order to better match with the current
              performances of the database used. This is possible if you set
              this parameter to <emphasis>true</emphasis> knowing that the
              default value is <emphasis>false</emphasis> and if you enable
              the JCR statistics. Based on the JCR statistics, the application
              will be able to set the best possible values for your thresholds
              to get the best possible performances.</entry>
            </row>

            <row>
              <entry>indexing-load-batching-threshold-ttl</entry>

              <entry>In case
              <emphasis>indexing-load-batching-threshold-property</emphasis>
              and/or
              <emphasis>indexing-load-batching-threshold-node</emphasis>,
              <emphasis>indexing-load-batching-threshold-dynamic</emphasis>
              and the JCR statistics have been enabled, the application will
              regularily update if needed the thresholds. This parameter
              defines the periodicity of the task that will update the
              thresholds. The default value is 5 minutes. The expected value
              is a time expressed in milliseconds.</entry>
            </row>
          </tbody>
        </tgroup>
      </table>

      <note>
        <para>If you use postgreSQL and the parameter rdbms-reindexing is set
        to true, the performances of the queries used while indexing can be
        improved by setting the parameter "enable_seqscan" to "off" or
        "default_statistics_target" to at least "50" in the configuration of
        your database. Then you need to restart DB server and make analyze of
        the JCR_SVALUE (or JCR_MVALUE) table.</para>
      </note>

      <note>
        <para>If you use DB2 and the parameter rdbms-reindexing is set to
        true, the performance of the queiries used while indexing can be
        improved by making statisticks on tables by running "RUNSTATS ON TABLE
        &lt;scheme&gt;.&lt;table&gt; WITH DISTRIBUTION AND INDEXES ALL" for
        JCR_SITEM (or JCR_MITEM) and JCR_SVALUE (or JCR_MVALUE) tables.</para>
      </note>
    </section>

    <section id="JCR.QueryHandlerConfiguration.Configuration.StandaloneStrategy">
      <title>Standalone strategy</title>

      <para>When running JCR in standalone usually standalone indexing is used
      also. Such parameters as "changesfilter-class", "jgroups-configuration"
      and all the "infinispan-*" must be skipped and not defined. Like the
      configuration below.</para>

      <programlisting language="xml">&lt;workspace name="ws"&gt;
   &lt;query-handler class="org.exoplatform.services.jcr.impl.core.query.lucene.SearchIndex"&gt;
      &lt;properties&gt;
         &lt;property name="index-dir" value="shareddir/index/db1/ws" /&gt;
         &lt;property name="max-volatile-time" value="60" /&gt;
         &lt;property name="rdbms-reindexing" value="true" /&gt;
         &lt;property name="reindexing-page-size" value="1000" /&gt;
         &lt;property name="index-recovery-mode" value="from-coordinator" /&gt;
      &lt;/properties&gt;
   &lt;/query-handler&gt;
&lt;/workspace&gt;</programlisting>
    </section>

    <section id="JCR.QueryHandlerConfiguration.Configuration.Cluster-readyIndexingStrategies">
      <title>Cluster-ready indexing strategies</title>

      <section id="JCR.QueryHandlerConfiguration.Configuration.Cluster-readyIndexingStrategies.SharedIndex">
        <title>Shared Index</title>

        <para>For both cluster-ready implementations Infinispan, JGroups and
        Changes Filter values must be defined. Shared index requires some kind
        of remote or shared file system to be attached in a system (i.e. NFS,
        SMB or etc). Indexing directory ("indexDir" value) must point to it.
        Setting "changesfilter-class" to
        "org.exoplatform.services.jcr.impl.core.query.ispn.ISPNIndexChangesFilter"
        will enable shared index implementation.</para>

        <programlisting language="xml">&lt;query-handler class="org.exoplatform.services.jcr.impl.core.query.lucene.SearchIndex"&gt;
   &lt;properties&gt;
      &lt;property name="index-dir" value="/mnt/nfs_drive/index/db1/ws" /&gt;
      &lt;property name="changesfilter-class"
         value="org.exoplatform.services.jcr.impl.core.query.ispn.ISPNIndexChangesFilter" /&gt;
      &lt;property name="infinispan-configuration" value="infinispan-indexer.xml" /&gt;
      &lt;property name="jgroups-configuration" value="udp-mux.xml" /&gt;
      &lt;property name="infinispan-cluster-name" value="JCR-cluster" /&gt;
      &lt;property name="max-volatile-time" value="60" /&gt;
      &lt;property name="rdbms-reindexing" value="true" /&gt;
      &lt;property name="reindexing-page-size" value="1000" /&gt;
      &lt;property name="index-recovery-mode" value="from-coordinator" /&gt;
   &lt;/properties&gt;
&lt;/query-handler&gt;
</programlisting>
      </section>

      <section id="JCR.QueryHandlerConfiguration.Configuration.Cluster-readyIndexingStrategies.RSyncIndex">
        <title>RSync Index</title>

        <section id="JCR.QueryHandlerConfiguration.Configuration.Cluster-readyIndexingStrategies.RSyncIndex.SystemRequirements">
          <title>System requirements</title>

          <para>Mandatory requirement for Rsync-based indexing strategy is an
          installed and properly configured RSync utility. It must be
          accessible by calling "rsync" without defining it's full path, in
          addition each cluster node should have a running RSync Server
          supporting "rsync://" protocol. For more details, please refer to
          RSync and operation system documentations. Sample RSync Server
          configuration will be shown below. There are some additional
          limitations also. Path for index for each workspace must be the same
          across the cluster, i.e.
          "/var/data/index/&lt;repository-name&gt;/&lt;workspace-name&gt;".
          Next limitation is RSync Server configuration. It must share some of
          index's parent folders. For example, "/var/data/index". In other
          words, index is stored inside of RSync Server shared folder.
          Configuration details are give below.</para>
        </section>

        <section id="JCR.QueryHandlerConfiguration.Configuration.Cluster-readyIndexingStrategies.RSyncIndex.Configuration">
          <title>Configuration</title>

          <para>Configuration has much in common with Shared Index, it just
          requires some additional parameters for RSync options. If they are
          present, JCR switches from shared to RSync-based index. Here is an
          example configuration:</para>

          <programlisting language="xml">&lt;query-handler class="org.exoplatform.services.jcr.impl.core.query.lucene.SearchIndex"&gt;
   &lt;properties&gt;
      &lt;property name="index-dir" value="/var/data/index/repository1/production" /&gt;
      &lt;property name="changesfilter-class"
         value="org.exoplatform.services.jcr.impl.core.query.ispn.ISPNIndexChangesFilter" /&gt;
      &lt;property name="infinispan-configuration" value="jar:/conf/portal/cluster/infinispan-indexer.xml" /&gt;
      &lt;property name="jgroups-configuration" value="jar:/conf/portal/cluster/udp-mux.xml" /&gt;
      &lt;property name="infinispan-cluster-name" value="JCR-cluster" /&gt;
      &lt;property name="max-volatile-time" value="60" /&gt; 
      &lt;property name="rsync-entry-name" value="index" /&gt;
      &lt;property name="rsync-entry-path" value="/var/data/index" /&gt;
      &lt;property name="rsync-port" value="8085" /&gt;
      &lt;property name="rsync-user" value="rsyncexo" /&gt;
      &lt;property name="rsync-password" value="exo" /&gt;
   &lt;/properties&gt;
&lt;/query-handler&gt;
</programlisting>

          <para>Let's start with authentication: "rsync-user" and
          "rsync-password". They are optional and can be skipped if RSync
          Server configured to accept anonymous identity. Before reviewing
          other RSync index options need to have a look at RSync Server
          configuration. Sample RSync Server (rsyncd) Configuration</para>

          <programlisting>uid = nobody
gid = nobody
use chroot = no
port = 8085
log file = rsyncd.log
pid file = rsyncd.pid
[index]
        path = /var/data/index
        comment = indexes
        read only = true
        auth users = rsyncexo
        secrets file= rsyncd.secrets
</programlisting>

          <para>This sample configuration shares folder "/var/data/index" as
          an entry "index". Those parameters should match corresponding
          properties in JCR configuration. Respectively "rsync-entry-name",
          "rsync-entry-path", "rsync-port" properties. Notice! Make sure
          "index-dir" is a descendant folder of RSync shared folder and those
          paths are the same on each cluster node.</para>
        </section>
      </section>

      <section id="JCR.QueryHandlerConfiguration.Configuration.Cluster-readyIndexingStrategies.LocalIndex">
        <title>Local Index</title>

        <para>In order to use cluster-ready strategy based on local indexes,
        when each node has own copy of index on local file system, the
        following configuration must be applied. Indexing directory must point
        to any folder on local file system and "changesfilter-class" must be
        set to
        "org.exoplatform.services.jcr.impl.core.query.ispn.LocalIndexChangesFilter".</para>

        <programlisting language="xml">&lt;query-handler class="org.exoplatform.services.jcr.impl.core.query.lucene.SearchIndex"&gt;
   &lt;properties&gt;
      &lt;property name="index-dir" value="/mnt/nfs_drive/index/db1/ws" /&gt;
      &lt;property name="changesfilter-class"
         value="org.exoplatform.services.jcr.impl.core.query.ispn.LocalIndexChangesFilter" /&gt;
      &lt;property name="infinispan-configuration" value="infinispan-indexer.xml" /&gt;
      &lt;property name="jgroups-configuration" value="udp-mux.xml" /&gt;
      &lt;property name="infinispan-cluster-name" value="JCR-cluster" /&gt;
      &lt;property name="max-volatile-time" value="60" /&gt;
      &lt;property name="rdbms-reindexing" value="true" /&gt;
      &lt;property name="reindexing-page-size" value="1000" /&gt;
      &lt;property name="index-recovery-mode" value="from-coordinator" /&gt;
   &lt;/properties&gt;
&lt;/query-handler&gt;
</programlisting>

        <section id="JCR.QueryHandlerConfiguration.Configuration.Cluster-readyIndexingStrategies.LocalIndexRecoveryFilters">
          <title>Local Index Recovery Filters</title>

          <para>Common usecase for all cluster-ready applications is a hot
          joining and leaving of processing units. Node that is joining
          cluster for the first time or node joining after some downtime, they
          all must be in a synchronized state. When having a deal with shared
          value storages, databases and indexes, cluster nodes are
          synchronized anytime. But it's an issue when local index strategy
          used. If new node joins cluster, having no index it is retrieved or
          recreated. Node can be restarted also and thus index not empty.
          Usually existing index is thought to be actual, but can be outdated.
          JCR offers a mechanism called RecoveryFilters that will
          automatically retrieve index for the joining node on startup. This
          feature is a set of filters that can be defined via QueryHandler
          configuration:</para>

          <programlisting language="xml">&lt;property name="index-recovery-filter" value="org.exoplatform.services.jcr.impl.core.query.lucene.DocNumberRecoveryFilter" /&gt;</programlisting>

          <para>Filter number is not limited so they can be combined:</para>

          <programlisting language="xml">&lt;property name="index-recovery-filter" value="org.exoplatform.services.jcr.impl.core.query.lucene.DocNumberRecoveryFilter" /&gt;
&lt;property name="index-recovery-filter" value="org.exoplatform.services.jcr.impl.core.query.lucene.SystemPropertyRecoveryFilter" /&gt;
</programlisting>

          <para>If any one fires, the index is re-synchronized. Please take in
          account, that DocNumberRecoveryFilter is used in cases when no
          filter configured. So, if resynchronization should be blocked, or
          strictly required on start, then ConfigurationPropertyRecoveryFilter
          can be used.</para>

          <para>This feature uses standard index recovery mode defined by
          previously described parameter (can be "from-indexing" or
          "from-coordinator" (default value))</para>

          <programlisting language="xml">&lt;property name="index-recovery-mode" value="from-coordinator" /&gt;
</programlisting>

          <para>There are couple implementations of filters:</para>

          <itemizedlist>
            <listitem>
              <para>org.exoplatform.services.jcr.impl.core.query.lucene.DummyRecoveryFilter:
              always returns true, for cases when index must be force
              resynchronized (recovered) each time;</para>
            </listitem>

            <listitem>
              <para>org.exoplatform.services.jcr.impl.core.query.lucene.SystemPropertyRecoveryFilter
              : return value of system property
              "org.exoplatform.jcr.recoveryfilter.forcereindexing". So index
              recovery can be controlled from the top without changing
              documentation using system properties;</para>
            </listitem>

            <listitem>
              <para>org.exoplatform.services.jcr.impl.core.query.lucene.ConfigurationPropertyRecoveryFilter
              : return value of QueryHandler configuration property
              "index-recovery-filter-forcereindexing". So index recovery can
              be controlled from configuration separately for each workspace.
              I.e:</para>

              <programlisting language="xml">&lt;property name="index-recovery-filter" value="org.exoplatform.services.jcr.impl.core.query.lucene.ConfigurationPropertyRecoveryFilter" /&gt;
&lt;property name="index-recovery-filter-forcereindexing" value="true" /&gt;
</programlisting>
            </listitem>

            <listitem>
              <para>org.exoplatform.services.jcr.impl.core.query.lucene.DocNumberRecoveryFilter
              : checks number of documents in index on coordinator side and
              self-side. Return true if differs. Advantage of this filter
              comparing to other, it will skip reindexing for workspaces where
              index wasn't modified. I.e. there is 10 repositories with 3
              workspaces in each one. Only one is really heavily used in
              cluster : frontend/production. So using this filter will only
              reindex those workspaces that are really changed, without
              affecting other indexes thus greatly reducing startup
              time.</para>
            </listitem>
          </itemizedlist>
        </section>
      </section>
    </section>

    <section id="JCR.QueryHandlerConfiguration.Configuration.ISPNTemplateConfiguration">
      <title>Infinispan template configuration</title>

      <para>Infinispan template configuration for query handler is about the
      same for both clustered strategies.</para>

      <para>infinispan-indexer.xml</para>

      <programlisting language="xml">&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;infinispan
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
      xsi:schemaLocation="urn:infinispan:config:5.2 http://www.infinispan.org/schemas/infinispan-config-5.2.xsd"
      xmlns="urn:infinispan:config:5.2"&gt;

    &lt;global&gt;
      &lt;evictionScheduledExecutor factory="org.infinispan.executors.DefaultScheduledExecutorFactory"&gt;
        &lt;properties&gt;
          &lt;property name="threadNamePrefix" value="EvictionThread"/&gt;
        &lt;/properties&gt;
      &lt;/evictionScheduledExecutor&gt;

      &lt;globalJmxStatistics jmxDomain="exo" enabled="true" allowDuplicateDomains="true"/&gt;

      &lt;transport transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport" clusterName="${infinispan-cluster-name}" distributedSyncTimeout="20000"&gt;
        &lt;properties&gt;
          &lt;property name="configurationFile" value="${jgroups-configuration}"/&gt;
        &lt;/properties&gt;
      &lt;/transport&gt;
    &lt;/global&gt;

    &lt;default&gt;
      &lt;clustering mode="replication"&gt;
        &lt;stateTransfer timeout="20000" fetchInMemoryState="false" /&gt;
        &lt;sync replTimeout="20000"/&gt;
      &lt;/clustering&gt;

      &lt;locking isolationLevel="READ_COMMITTED" lockAcquisitionTimeout="20000" writeSkewCheck="false" concurrencyLevel="500" useLockStriping="false"/&gt;
      &lt;transaction transactionManagerLookupClass="org.exoplatform.services.transaction.infinispan.JBossStandaloneJTAManagerLookup" syncRollbackPhase="true" syncCommitPhase="true" transactionMode="TRANSACTIONAL"/&gt;
      &lt;jmxStatistics enabled="true"/&gt;
      &lt;eviction strategy="NONE"/&gt;

      &lt;loaders passivation="false" shared="false" preload="false"&gt;
        &lt;store class="${infinispan-cachestore-classname}" fetchPersistentState="true" ignoreModifications="false" purgeOnStartup="false"&gt;
          &lt;async enabled="false"/&gt;
        &lt;/store&gt;
      &lt;/loaders&gt;
   &lt;/default&gt;
&lt;/infinispan&gt;</programlisting>

      <para>See more about template configurations <link
      linkend="JCR.ISPNConfigurationTemplates">here</link>.</para>
    </section>
  </section>

  <section id="JCR.QueryHandlerConfiguration.Asynchronous_reindexing">
    <title>Asynchronous reindexing</title>

    <para>Managing a big set of data using JCR in production environment
    sometimes requires special operations with Indexes, stored on File System.
    One of those maintenance operations is a recreation of it. Also called
    "re-indexing". There are various usecases when it's important to do. They
    include hardware faults, hard restarts, data-corruption, migrations and
    JCR updates that brings new features related to index. Usually index
    re-creation requested on server's startup or in runtime.</para>

    <section id="JCR.QueryHandlerConfiguration.Asynchronous_reindexing.OnStartupIndexing">
      <title>On startup indexing</title>

      <para>Common usecase for updating and re-creating the index is to stop
      the server and manually remove indexes for workspaces requiring it. When
      server will be started, missing indexes are automatically recovered by
      re-indexing. JCR Supports direct RDBMS re-indexing, that usually is
      faster than ordinary and can be configured via QueryHandler parameter
      "rdbms-reindexing" set to "true" (for more information please refer to
      "Query-handler configuration overview"). New feature to introduce is
      asynchronous indexing on startup. Usually startup is blocked until
      process is finished. Block can take any period of time, depending on
      amount of data persisted in repositories. But this can be resolved by
      using an asynchronous approaches of startup indexation. Saying briefly,
      it performs all operations with index in background, without blocking
      the repository. This is controlled by the value of "async-reindexing"
      parameter in QueryHandler configuration. With asynchronous indexation
      active, JCR starts with no active indexes present. Queries on JCR still
      can be executed without exceptions, but no results will be returned
      until index creation completed. Checking index state is possible via
      QueryManagerImpl:</para>

      <para><programlisting lang="java">boolean online = ((QueryManagerImpl)Workspace.getQueryManager()).getQueryHandeler().isOnline();</programlisting></para>

      <para>"OFFLINE" state means that index is currently re-creating. When
      state changed, corresponding log event is printed. From the start of
      background task index is switched to "OFFLINE", with following log event
      :</para>

      <programlisting>[INFO] Setting index OFFLINE (repository/production[system]).</programlisting>

      <para>When process finished, two events are logged :</para>

      <programlisting>[INFO] Created initial index for 143018 nodes (repository/production[system]).
[INFO] Setting index ONLINE (repository/production[system]).</programlisting>

      <para>Those two log lines indicates the end of process for workspace
      given in brackets. Calling isOnline() as mentioned above, will also
      return true.</para>
    </section>

    <section id="JCR.QueryHandlerConfiguration.Asynchronous_reindexing.HotAsynchronousWorkspaceReindexingViaJMX">
      <title>Hot Asynchronous Workspace Reindexing via JMX</title>

      <para>Some hard system faults, error during upgrades, migration issues
      and some other factors may corrupt the index. Most likely end customers
      would like the production systems to fix index issues in run-time,
      without delays and restarts. Current versions of JCR supports "Hot
      Asynchronous Workspace Reindexing" feature. It allows end-user (Service
      Administrator) to launch the process in background without stopping or
      blocking whole application by using any JMX-compatible console (see
      screenshot below, "JConsole in action").<mediaobject>
          <imageobject>
            <imagedata align="center" fileref="images/jmx-jconsole.png" />
          </imageobject>
        </mediaobject>Server can continue working as expected while index is
      recreated. This depends on the flag "allow queries", passed via JMX
      interface to reindex operation invocation. If the flag set, then
      application continues working. But there is one critical limitation the
      end-users must be aware. Index is frozen while background task is
      running. It meant that queries are performed on index present on the
      moment of task startup and data written into repository after startup
      won't be available through the search until process finished. Data added
      during re-indexation is also indexed, but will be available only when
      task is done. Briefly, JCR makes the snapshot of indexes on asynch task
      startup and uses it for searches. When operation finished, stale indexes
      replaced by newly created including newly added data. If flag "allow
      queries" is set to false, then all queries will throw an exception while
      task is running. Current state can be acquired using the following JMX
      operation:</para>

      <itemizedlist>
        <listitem>
          <para>getHotReindexingState() - returns information about latest
          invocation: start time, if in progress or finish time if
          done.</para>
        </listitem>
      </itemizedlist>
    </section>

    <section id="JCR.QueryHandlerConfiguration.Asynchronous_reindexing.Notices">
      <title>Notices</title>

      <para>First of all, can't launch Hot re-indexing via JMX if index is
      already in offline mode. It means that index is currently is invoked in
      some operations, like re-indexing at startup, copying in cluster to
      another node or whatever. Another important this is Hot Asynchronous
      Reindexing via JMX and "on startup" reindexing are completely different
      features. So you can't get the state of startup reindexing using command
      getHotReindexingState in JMX interface, but there are some common JMX
      operations:</para>

      <itemizedlist>
        <listitem>
          <para>getIOMode - returns current index IO mode (READ_ONLY /
          READ_WRITE), belongs to clustered configuration states;</para>
        </listitem>

        <listitem>
          <para>getState - returns current state: ONLINE / OFFLINE.</para>
        </listitem>
      </itemizedlist>
    </section>
  </section>

  <section id="JCR.QueryHandlerConfiguration.Advanced_tuning">
    <title>Advanced tuning</title>

    <section id="JCR.QueryHandlerConfiguration.Advanced_tuning.LuceneTuning">
      <title>Lucene tuning</title>

      <para>As mentioned above, JCR Indexing is based on Lucene indexing
      library as underlying search engine. It uses Directories to store index
      and manages access to index by Lock Factories. By default JCR
      implementation uses optimal combination of Directory implementation and
      Lock Factory implementation. When running on OS different from Windows,
      NIOFSDirectory implementation used. And SimpleFSDirectory for Windows
      stations. NativeFSLockFactory is an optimal solution for wide variety of
      cases including clustered environment with NFS shared resources. But
      those default can be overridden with the help of system properties.
      There are two properties:
      "org.exoplatform.jcr.lucene.store.FSDirectoryLockFactoryClass" and
      "org.exoplatform.jcr.lucene.FSDirectory.class" that are responsible for
      changing default behavior. First one defines implementation of abstract
      Lucene LockFactory class and the second one sets implementation class
      for FSDirectory instances. For more information please refer to Lucene
      documentation. But be sure You know what You are changing. JCR allows
      end users to change implementation classes of Lucene internals, but
      doesn't guarantee it's stability and functionality.</para>
    </section>

    <section>
      <title>Index optimization</title>

      <para>From time to time, the Lucene index needs to be optimized. The
      process is essentially a defragmentation. Until an optimization is
      triggered Lucene only marks deleted documents as such, no physical
      deletions are applied. During the optimization process the deletions
      will be applied. Optimizing the Lucene index speeds up searches but has
      no effect on the indexation (update) performance. First of all ensure
      repository is suspended to avoid any possible inconsistency. It is
      recommended to schedule optimization. Also checking for pending
      deletions is supported. If it is so, it is a first signal to index
      optimization. All operation are available via JMX:</para>

      <mediaobject>
        <imageobject>
          <imagedata fileref="images/search-manager-optimize.png" />
        </imageobject>
      </mediaobject>
    </section>
  </section>
</section>
