Regenerate shingles based on current TM settings
The shingle generation process driven from the lookup process needs to be synchronized with the shingles that were generated during the TM entry storage process. Changes to the stemming configuration options can have the result that the saved shingles no longer match the shingles that will be generated for the search text (and for new TM entries added to the translation memory). In essence, this can happen when the TM is not empty and one of the following is true:
- the enabling or disabling of stemming has been toggled
- a stemmer has been assigned to a language that was not formally assigned a stemmer (and that language already contains entries in the TM)
- a stemmer has been changed or removed for a language already containing TM entries
- the stemming implementation has been updated
When the stored shingles are not synchronized with the current stemming configuration for the TM, leverage will most likely be negatively affected. (Note that this only applies to fuzzy matches within the TM, but to all standard matches within the TD.) As a result, the TM (and TD) configuration page presents the option to regenerate the shingles for the TM for the entries impacted. All processes accessing the TM or TD will be either blocked (by database locking) until the update is completed, or subject to leverage impact until the process is completed.