vendredi 4 décembre 2020

Solr: Searching with/without spaces in keywords

I am experiencing an issue when spaces are introduced to keywords, for example:

  • We have a product with the title "Sony Playstation 4 Camera V2 PS4 (PSVR)"

  • Searching for "playstation" or "playstation camera" brings back this product

  • Searching for "play station" or "play station camera" does not bring back this product (notice the space)

Here is the fieldType being used:

    <fieldType name="text_en_splitting" class="solr.TextField" positionIncrementGap="100" autoGeneratePhraseQueries="true">
        <analyzer type="index">
            <tokenizer class="solr.WhitespaceTokenizerFactory"/>
            <filter class="solr.HyphenatedWordsFilterFactory"/>
            <filter class="solr.EnglishMinimalStemFilterFactory"/>
            <filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_en.txt"/>
            <filter class="solr.SynonymGraphFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
            <filter class="solr.WordDelimiterGraphFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" preserveOriginal="1"/>
            <filter class="solr.LowerCaseFilterFactory"/>
        </analyzer>
        <analyzer type="query">
            <tokenizer class="solr.WhitespaceTokenizerFactory"/>
            <filter class="solr.HyphenatedWordsFilterFactory"/>
            <filter class="solr.EnglishMinimalStemFilterFactory"/>
            <filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_en.txt"/>
            <filter class="solr.SynonymGraphFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
            <filter class="solr.WordDelimiterGraphFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" preserveOriginal="1"/>
            <filter class="solr.LowerCaseFilterFactory"/>
        </analyzer>
    </fieldType>

How can I fix this, and make both "playstation" and "play station" match? This is only limited to PlayStation for my example, but it can happen to any search term e.g. "cyberpunk", "cyber punk". So solutions that require alot of manual work such as adding a synonym for play station => playstation are not feasible.

Things I have tried, but not managed to make work:

  • N-GRAM filter and tokenizer
  • Fuzzy search
  • Removing whitespace
  • Escaping whitespace



Aucun commentaire:

Enregistrer un commentaire