Question

I am trying to index into a document a field with one term that has a payload.
Since the only constructor of Field that can work for me takes a TokenStream, I decided to inherit from this class and give the most basic implementation for what I need:

public class MyTokenStream : TokenStream
{
    TermAttribute termAtt;
    PayloadAttribute payloadAtt;
    bool moreTokens = true;
    public MyTokenStream()
    {
        termAtt = (TermAttribute)GetAttribute(typeof(TermAttribute));
        payloadAtt = (PayloadAttribute)GetAttribute(typeof(PayloadAttribute));
    }
    public override bool IncrementToken()
    {
        if (moreTokens)
        {
            termAtt.SetTermBuffer("my_val");
            payloadAtt.SetPayload(new Payload(/*bye[] data*/));
            moreTokens = false;
        }

        return false;
    }     
}

The code which was used while indexing:

        IndexWriter writer = //init tndex writer...

        Document d = new Document();
        d.Add(new Field("field_name", new MyTokenStream()));
        writer.AddDocument(d);
        writer.Commit();

And the code that was used during the search:

        IndexSearcher searcher = //init index searcher
        Query query = new TermQuery(new Term("field_name", "my_val"));
        TopDocs result = searcher.Search(query, null, 10);

I used the debugger to verify that call to IncrementToken() actually sets the TermBuffer.
My problem is that the returned TopDocs instance returns no documents, and I cant understand why… Actually I started from TermPositions (which gives me approach to the Payload…), but it also gave me no results.
Can someone explain to me what am I doing wrong?
I am currently using Lucene .NET 2.9.2

Answer 1

After you set the TermBuffer you need to return true from IncrementToken, you return false when you have nothing to feed the TermBuffer with anymore