Open topic with navigation

# Manipulate and Enrich Documents

HPE CFS provides features to manipulate and enrich documents. This means that you can add additional information to the documents, and improve the quality of the information, before the documents are indexed into IDOL. For example, you can:

- Add additional fields to a document.
- Extract content from HTML pages, discarding irrelevant content such as headers, sidebars, advertisements, and scripts.
- Split long documents into multiple sections. This can improve performance when you query IDOL, because IDOL can return a specific part of a document in response to a query.
- Reject documents that do not contain content in a specific language.
- Perform Eduction on document fields. Eduction extracts entities from a document, and writes them to specific document fields. An entity can be a word, phrase, or block of information - for example an address or telephone number.
- Perform Optical Character Recognition (OCR), face recognition, and object recognition on image and video files and add the results to the document.
- Extract speech from audio and video files, and add the transcription to the document content.

The simplest way to manipulate documents is to use the import tasks and index tasks that are included with CFS. For information about the tasks that are available, see Manipulate and Enrich Documents. You can configure these tasks by modifying configuration parameters in the HPE CFS configuration file.

Import tasks can call other services, for example Media Server or Speech Server. You can use Media Server to perform Optical Character Recognition, face recognition, and object recognition on images and video. You can use Speech Server to extract speech from audio and video files. This enables IDOL to use the data in images, video, and speech for retrieval, clustering, and other operations.

HPE CFS also supports Lua, an embedded scripting language. You can write Lua scripts to manipulate documents and define custom processing rules. For information about the Lua functions that are provided with HPE CFS, refer to the HPE Connector Framework Server Reference.

Related Topics

- Import Process
- Write and Run Lua Scripts

© 2016 Hewlett Packard Enterprise Development LP | Send documentation feedback to HPE Close Send Documentation Feedback To open the configured email client on this computer, open an email window. Otherwise, copy the information below to a web mail client, and send this email to AutonomyTPFeedback@hpe.com. We appreciate your feedback! Product: Topic Title: Help Topic: Feedback:

_HP_HTML5_bannerTitle.htm