Question

I have written a browser helper object to get the text between the tags and use it for data mining purpose. I tried using it on igoogle (basically to test its capability on gadgets) and it failed in some of the cases where an <iframe> is present with some external source.

I can get the <div> and its child <iframe> but fail to get the body.

I get the frame collection from this API HRESULT IHTMLDocument2::get_frames(IHTMLFramesCollection2 **p);

The problem can be re-created in igoogle and firefox using the loan calculator gadget. You will also need the fire bug extension to debug the page. For reference purpose I am pasting the sample here…

<div class="modboxin" id="m_8_b"><div style="border: 0pt none; padding: 0pt; margin: 0pt; width: 100%;" id="remote_8">
<iframe scrolling="no" frameborder="0" onload="_ifr_ol(this)" style="border: 0pt none; padding: 0pt; margin: 0pt; width: 100%; height: 100px; overflow: hidden;" name="remote_iframe_8" id="remote_iframe_8" src="http://8.ig.gmodules.com/gadgets/ifr?exp_rpc_js=1&amp;exp_track_js=1&amp;v=682f3db70d7cfff515d7c64fd24923&amp;container=ig&amp;view=default&amp;debug=0&amp;mid=8&amp;lang=en&amp;url=http%3A%2F%2Fwww.nova.edu%2F%7Ewillheat%2Floan.xml&amp;country=US&amp;parent=http://www.google.com&amp;libs=core:core.io:core.iglegacy:auth-refresh&amp;synd=ig&amp;view=default#st=...B27zWVKsnJu6OviCNnzXoPjkDsbPg95yZNMwfmMaLnwWoRxGaRArxTpOqK4TiH87uGUiHnYkkaqU9NE1sOyms6sg/Jwi&amp;gadgetId=116809661812082345195&amp;gadgetOwner=105250506097979753968&amp;gadgetViewer=105250506097979753968&amp;rpctoken=422312139&amp;ifpctok=422312139">
</iframe>
</div>

The link is not complete as I have replaced some part of the src with .... Now as you can see that there is no body for the although it is getting rendered in the browser..

As per this post ( http://stackoverflow.com/questions/957133/does-body-onload-wait-for-iframes ) the onload event on body does not wait for frames to complete.

So I can conclude that I have to use some sort onload listener for the <iframe>… but I am not sure how …

Kindly suggest a way/snippet to retrieve the body of the <iframe> using ATL/COM APIs…

** Update **

I am using the following code to get the <iframes>. Although i get the iframe collection but when i try to get their body it fails… may be because they are not loaded by that time ?!

void testFrame(IHTMLDocument2* pDocument)
{
    CComQIPtr<IHTMLFramesCollection2> col;
    HRESULT hr = pDocument->get_frames(&col);
    if((hr == S_OK) && (col != NULL))
    {
        long counter = 0;
        hr = col->get_length(&counter);
        if((hr == S_OK) && (counter > 0))
        {
            for (int i = 0; i < counter; i++)
            {
                VARIANT     v1, v2;
                v1.vt = VT_I4; v1.lVal = i;
                VariantClear (&v2);
                hr = col->item(&v1, &v2);

                if (hr == S_OK && (v2.vt == VT_DISPATCH))
                {
                    CComPtr<IDispatch> pDispatch = v2.pdispVal;
                    CComQIPtr<IHTMLWindow2, &IID_IHTMLWindow2> pFrame = pDispatch;

                    if(pFrame)
                    {
                        CComPtr<IHTMLDocument2> spHTML;
                        hr = pFrame->get_document (&spHTML);

                        if((hr == S_OK) && (spHTML != NULL))
                        {
                            CComQIPtr<IHTMLElement> elem;
                            hr = spHTML->get_body(&elem);
                            if((hr == S_OK) && (elem != NULL))
                            {
                                CComBSTR str;
                                hr = elem->get_innerHTML(&str);
                                if((hr == S_OK) && (str != NULL))
                                {
                                    box(str);
                                }else if(hr != S_OK) {
                                    box(_T("hr is not ok"));
                                }else if(str == NULL){
                                    box(_T("STR is null"));
                                }else
                                    box(_T("Failed"));
                            }
                        }
                    }
                }
            }
        }
    }
}

And,

void box(LPCWSTR msg)
{
    MessageBox(NULL,msg,_T("..BOX.."),MB_OK);
}

Any suggestions, how to get the iframe body …. by the way I am handling this in OnDocumentComplete event…

Thanks,

Answer 1

Instead of updating my own question.. I am putting this as an answer. Though I would really love to see an alternate answer…

–Solution–

My basic assumptions are:

1. I know about the urls to handle..
2. A page can be divided in two main events (there could be other events too but these two will do)
   • The completion of the main page
   • Completion of the <iframes>

Code

void STDMETHODCALLTYPE CSafeMaskBHO::OnDocumentComplete(IDispatch *pDisp, VARIANT *pvarURL)
{
    CComQIPtr<IWebBrowser2> spTempWebBrowser = pDisp;

    CComBSTR url = NULL;
    HRESULT hr = spTempWebBrowser->get_LocationURL(&url); // You can also take the url from pvarURL .. 

    if((hr == S_OK) && (url != NULL))
    {
        /*
            I know which url's I am looking for
        */
        if(!(wcsstr(url,_T("www.example.com")) != NULL) && !((wcsstr(url,_T("www.test.com")) != NULL))){
            return;
        }       

        CComPtr<IDispatch> frameDocDisp;
        hr = spTempWebBrowser->get_Document(&frameDocDisp);
        if((hr == S_OK) && (frameDocDisp != NULL))
        {
            CComQIPtr<IHTMLDocument3> spHTMLDoc = frameDocDisp;
            // ... Do someting useful ...

        }

    }else if(spTempWebBrowser && m_spWebBrowser && m_spWebBrowser.IsEqualObject(spTempWebBrowser))
    {
        CComPtr<IDispatch> spDispDoc;
        hr = m_spWebBrowser->get_Document(&spDispDoc);

        if ((hr == S_OK) && (spDispDoc != NULL))
        {
            CComQIPtr<IHTMLDocument2> spHTMLDoc = spDispDoc;
            if(spHTMLDoc)
            {
                // ... Do someting useful ...
            }
        }
    }
}

If you think that you have anything to share (suggestions/corrections/alternatives) then please do so.. 🙂

Thanks,