Question

I have a class Type which cannot be copied nor it contains default constructor.
I have second class A that acts as a set of the above classes. This second class gives access via iterators and my iterator has dereference operator:

class A {
    class iterator {
        [...]
      public:
        Type & operator*()
        { 
            return instance;
        }
      private:
        Type instance;
    }
    [...]
};

Now to expose that I wrote a boost::python code that looks like that:

class_<A>("A", [...])
    .def("__iter__", iterator<A, return_internal_reference<> >())
    .def("__len__", container_length_no_diff<A, A::iterator>)
;

After adding print messages to all iterator operations (construction, assignment, dereferences, destruction) for code Python like this:

for o in AInstance:
    print o.key

I get output (trimmed to important part):

construct 0xffffffff7fffd3e8
dereference: 0xffffffff7fffd3e8
destroy 0xffffffff7fffd3e8
get key 0xffffffff7fffd3e8

In above code those addresses are just addresses of instance member (or this in method call).
First three lines are produced by iterator, the fourth line is printed by getter method in Type. So somehow boost::python wraps everything in such manner that it:

1. creates iterator
2. dereferences iterator and stores reference
3. destroys iterator (and object it contains)
4. uses reference obtained in step two

So clearly return_internal_reference does not behave like stated (note that it actually is just typedef over with_custodian_and_ward_postcall<>) where it should keep object as long as result of method call is referenced.

So my question is how do I expose such an iterator to Python with boost::python?

edit:

As it was pointed out it might not be clear: the original container does not contain objects of type Type. It contains some BaseType objects from which I am able to construct/modify Type object. So iterator in above example acts like transform_iterator.

Answer 1

I think the whole problem was that I did not fully understand what semantics should iterator class provide.
It seems that value returned by iterator has to be valid as long as container exists, not iterator.

This means that boost::python behaves correctly and there are two solutions to that:

• use boost::shared_ptr
• return by value

A bit less efficient approaches than what I tried to do, but looks like there is no other way.

edit:
I have worked out a solution (not only possible, but it seems to be working nicely): Boost python container, iterator and item lifetimes