Question

I have Solr 4.0 up and running and using DataImportHandler to import data from MySQL.

I have notcied that if I point DataImportHandler at MySQL 5.5 data source everyhting works as expected. However when using exactly the same Solr/DataImportHandler config and exactly the same database but running on MySQL 5.0 certain fields come back base64 encoded.

Relevant entries in data-config.xml

       <dataSource type="JdbcDataSource" 
                    driver="com.mysql.jdbc.Driver" 
                    name="DB-SOURCE"
                    url="jdbc:mysql://dbhost/dbname"
                    user="user"    
                    password="password"
    />

   <document name="articles">
<entity name="article_ph" transformer="HTMLStripTransformer" dataSource="DB-SOURCE" pk="article_id" 
                            query="SELECT 'Politics Home' AS article_site, 
                                            CONCAT('ph-article-', article_id) AS article_id,
                                            article_title,
                                            article_text_plain AS article_content,
                                            article_articletype_id,
                                            article_datetime AS article_date,
                                            'Uncategorised' AS article_section,
                                            'Non Member' AS article_source
                                            FROM articles 
                                            WHERE 
                                            article_datetime!='0000-00-00 00:00:00' 
                                            AND article_datetime is NOT NULL 
                                            AND article_live=1 
                                            AND article_text_plain!='' 
                                            AND article_text_plain IS NOT NULL 
                                            AND article_title is NOT NULL
                                            AND article_title !=''">
            <field column="ARTICLE_SITE" name="article_site" />
            <field column="ARTICLE_ID" name="article_id" />
            <field column="ARTICLE_TITLE" name="article_title" />
            <field column="ARTICLE_CONTENT" name="article_content" stripHTML="true" />
            <field column="ARTICLE_DATE" name="article_date" />
            <field column="ARTICLE_SECTION" name="article_section" />
            <field column="ARTICLE_SOURCE" name="article_source" />
            <entity name="articletype_name" dataSource="DB-SOURCE" 
                            query="SELECT 
                            articletype_name 
                            FROM articletypes 
                            WHERE articletype_id='${article_ph.article_articletype_id}'">
    <field column="articletype_name" name="article_type"/>
            </entity>
</entity>

When I run import pointing at MySQL 5.5 I get :

<arr name="article_id"><str>ph-article-124</str></arr>

When I run import pointing at MySQL 5.0 I get articles with base64 encode IDs :

<arr name="article_id"><str>cGgtYXJ0aWNsZS0xMjQ=</str></arr>

All other fields come back correctly.

Collation and character sets on both DBs are the same.

Any help appreciated.

Answer 1

Try converting it back to string

CONCAT('ph-article-', CAST(article_id AS CHAR(50))