Documentation Center

Configuring linguistic analyzers

Linguistic analyzers manipulate raw data before it is stored in the search index. Depending on the language and nature of the attribute a fitting analyzer should be chosen.

The Overview linguistic analyzers page displays:

  • Name—the name of the analyzer.
  • ID—determines the analyzer: Character Removal Filter, Character Transformation Filter, Advanced Tokenizer, Lowercase Filter, Stop Filter, Stem Filter, and so on.
  • Delete box for removing the analyzer.
  • Add pipeline button for developing custom text analysis components in addition to the pre-configured analyzers.

Fredhopper has several pre-configured analyzers (pipelines) onboard that you can attach to Search Fields. The configuration of Analyzers is global and they can be applied in each universe/locale. You can also develop custom text analysis components if needed by clicking Add pipeline and specifying the following:

Details
  • ID—indicates the pipeline as a number.
  • Display name—indicates the name of the pipeline.
Components
  • Component—choose among Russian, CJK, Simple, Greek, Dutch, and so on.
  • Description—explanation for the component
  • Type—character filter, Tokenizer, Token filter
  • Ranking—the bigger the number, the lower the place in the order.
  • Action—option to add or remove components.