Open topic with navigation

# Field-Based Indexing

You can use field-based indexing to distribute the indexing load between IDOL servers. This mode is similar to reference-based indexing, except that you configure the document fields that determine which child server to send the document to. Data indexing uses the value of the specified field in the documents being indexed or in which you replace fields.

When you enable field-based indexing, it applies to the `DREADD`, `DREADDDATA`, and `DREREPLACE` index actions. HPE DIH sends `DREREPLACE` index actions to all child servers, because the `DREREPLACE` action does not contain the information required to determine which child server contains the original document.

When you use field-based indexing, you cannot alter the number of child servers.

Note: Field-based indexing might prevent deduplication of documents with different field values. You can use field-based indexing only with a `KillDuplicates=NONE` setting in the `[Server]` section of the IDOL Server configuration file.

To enable field-based indexing

1. Open the HPE DIH configuration file in a text editor.
2. In the `[Server]` section, set the `DistributeByFields` parameter to `True`.

3. In the `[Server]` section, Set `DistributeByFieldsCSVs` to a comma-separated list of fields that HPE DIH uses to distribute index data between child servers. For example:

       DistributeByFieldsCSVs=*/DeDupeHash,*/SecondDistributeField

4. Save and close the configuration file. Restart the HPE DIH for your changes to take effect.

For example:

    [Server]
    Port=16000
    DIHPort=16001
    IndexClients=*.*.*.*
    MirrorMode=False
    DistributeByFields=True
    DistributeByFieldsCSVs=*/DeDupeHash,*/SecondDistributeField

Note: You can use `DistributeByFields` only when `MirrorMode` is set to `False`. HPE DIH will not start if `DistributeByFields` and `MirrorMode` are both set to `True`.

You can also set the `BalanceDistributeByFields` configuration parameter to balance the distribution of documents that do not contain the specified distribution fields. In this option, HPE DIH sends documents to a random child server if they lack the specified distribution fields.

## Field Value-Based Indexing

By default, HPE DIH internally determines how to distribute documents between child servers. This process ensures that HPE DIH always sends duplicate documents to the same child server.

Instead, you can configure HPE DIH to distribute documents to specific child servers when the field contains a specific value.

To enable field value-based indexing

1. Open the HPE DIH configuration file in a text editor.
2. In the `[Server]` section, set the `DistributeByFields` parameter to `True`.
3. In the `[Server]` section, set the `DistributeByFieldsCSVs` parameter to a comma-separated list of fields that HPE DIH uses to distribute index data between child servers.

4. In the `[IDOLServerN]` or `[DIHEngineN]` section for each group of child servers, set `DistributeByFieldsValues` to a comma-separated list of field values. If a document contains a field listed in the `DistributeByFieldsCSVs` parameter with this value, indexes it to this child server. For example:

       [DIHEngine2]
       DistributeByFieldsValues=backup,France

   Note: You can configure each field value in the list for only one child server. If a field value occurs in the list for multiple child servers, indexes matching documents into the child server with the lowest ID.

5. In the `[Server]` section, set the `UnknownFieldValueAction` parameter to the action that HPE DIH takes if the fields listed in the `DistributeByFieldsCSVs` parameter contain unknown values. The following actions are available:

   `Distribute` | HPE DIH uses a hash of the field values to distribute the document, as with conventional field-based indexing.
   `Ignore` | HPE DIH ignores the document and logs a warning.
   `Default` | HPE DIH sends the document to the server specified by the `UnknownFieldValueDefaultEngine` configuration parameter.

   For example:

       UnknownFieldValueAction=Distribute

6. In the `[Server]` section, set `UnknownFieldValueDefaultEngine` to the number of the child server that acts as the default server. Set this parameter only if you have set `UnknownFieldValueAction` to `Default`.
7. In the `[Server]` section, set `DistributeOnMultipleFieldValues` to `True` if you want to index documents into each server group that matches the particular field values. Set this parameter to `False` if you want the document to index only into the server with the lowest number.
8. Save and close the configuration file. Restart the HPE DIH for your changes to take effect.

For example:

    [Server]
    DistributeByFields=True
    DistributeByFieldsCSVs=*/database,*/country
    UnknownFieldValueAction=Default
    UnknownFieldValueDefaultEngine=0
    DistributeOnMultipleFieldValues=True

    [DIHEngines]
    Number=3

    [DIHEngine0]
    DistributeByFieldsValues=main

    [DIHEngine1]
    DistributeByFieldsValues=uk

    [DIHEngine2]
    DistributeByFieldsValues=backup,france

© 2016 Hewlett Packard Enterprise Development LP | Send documentation feedback to HPE Close Send Documentation Feedback To open the configured email client on this computer, open an email window. Otherwise, copy the information below to a web mail client, and send this email to AutonomyTPFeedback@hpe.com. We appreciate your feedback! Product: Topic Title: Help Topic: Feedback:

_HP_HTML5_bannerTitle.htm