Home/ Questions/Q 8103853
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-05T23:40:07+00:00

I am indexing some URIs in SOLR using the UAX29URLEmailTokenizerFactory tokenizer. The problem is

  • 0

I am indexing some URIs in SOLR using the UAX29URLEmailTokenizerFactory tokenizer. The problem is that some of my URIs are containing plus characters, which SOLR interprets as whitespaces and splits the URI. Can this problem be solved by some clever escaping of the ‘+’ char? I tried ‘+’ in the analyzer, but got the same results.
Here is my exact configuration of a field:

<fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
    <analyzer type="index">
    <tokenizer class="solr.UAX29URLEmailTokenizerFactory"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
  <analyzer type="query">
    <tokenizer class="solr.UAX29URLEmailTokenizerFactory"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
    <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
</fieldType>
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    The solution I came up with finally makes use of the CharacterFilter as suggested above. The trick was to substitute it with the encoded char ‘%2B’. This had the effect of keeping the URI as a single token and returning it to my application in a proper state – as a ‘+’. Here is the fieldtype definition I came up with:

    <fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
  <analyzer type="index">
    <charFilter class="solr.PatternReplaceCharFilterFactory" pattern="\+" replacement="%2B"/>
    <tokenizer class="solr.UAX29URLEmailTokenizerFactory"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
    <!-- in this example, we will only use synonyms at query time
    <filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
    -->
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
  <analyzer type="query">
    <charFilter class="solr.PatternReplaceCharFilterFactory" pattern="\+" replacement="%2B"/>
    <tokenizer class="solr.UAX29URLEmailTokenizerFactory"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
    <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
</fieldType>
    • 0