Question

I’m experimenting with the Twitter streaming api, and am trying to open a stream for a user to consume events as they happen. I’m using a standard set of classes for making REST api calls to twitter. When using https://userstream.twitter.com/2/user.json in a “GET” call the response stream never ends… I’m opening a StreamReader and reading the response as I would with any other REST call to Twitter. This is probably obvious to others, but how do I “consume” this stream… Is there a way to read the StreamReader as it’s reading (meaning before it closes)? or maybe there a different method I can user to consume this stream…. again, I apologize if this seams to be elementary to some, but I can’t figure it out at the moment… Thanks in advance for any advise or help!!!

Here is the original source I started troubleshooting this with… This method was fabricated from a set of C# Classes I found in a forum on LinkedIn. At the line that reads “responseData = responseReader.ReadToEnd()” the method starts to “drink” the stream… but does so like a bottomless cup… reading this stream of data in real time before it closes (which is essentially until I stop debugging or kill the process) is the question I’m tackling.

Private Function WebResponseGet(ByVal webRequest As HttpWebRequest) As String
    Dim responseReader As StreamReader = Nothing
    Dim responseData As String = ""

    Try
        responseReader = New StreamReader(webRequest.GetResponse().GetResponseStream())
        responseData = responseReader.ReadToEnd()
    Catch
        Throw
    Finally
        webRequest.GetResponse().GetResponseStream().Close()
        responseReader.Close()
        responseReader = Nothing
    End Try

    Return responseData
End Function

UPDATE & RELATED QUESTION:

So, I figured out the following way to keep a stream open, and write it to a file (this won’t be the final approach, I’m just testing, a developing the best way of doing this 🙂

Private Sub DrinkIt(ByVal webRequest As HttpWebRequest)

    Dim coder As Encoding = Encoding.UTF8

    Dim stream_reader As New StreamReader(webRequest.GetResponse().GetResponseStream(), coder, True, 1024)

    Do While 0 < 1

        Dim w As IO.StreamWriter

        w = File.AppendText(targetFile)

        Dim c(5) As Char

        stream_reader.Read(c, 0, c.Length)

        w.Write(c)

        w.Close()

        w.Dispose()

    Loop

    stream_reader.Close()

    stream_reader.DiscardBufferedData()

    stream_reader.Dispose()

End Sub

This writes the opened twitter stream to a file, and every time I Tweet, Retweet, Delete, Direct Message… and so on… The file grows with a JSON objects appended to the text. I used the Do While 0 < 1 for testing here, because I just wanted to see it working. I see on MSDN StreamReader Constructor Description that the New constructor is supposed to accept a Boolean value for “leaveOpen”, but no such argument allowed when I try to add this to the constructor… Does anyone have a good example of how to do this with forcing and infinite loop or just a better approach than this… I would like to simply read new updates sent each time from Twitter, and address them accordingly? There is obviously a way, I’m just new to the concept of consuming a stream like this with out it being closed.. (**btw, Thanks to Dr. Evil’s suggestion, I was lead in this direction… It’s not exactly what he suggested, but is what lead me here)

Answer 1

One remark upfront: The leaveOpen parameter you saw is only present in .NET 4.5 and up

As to how to deal with the Twitter streaming API:

• http://www.voiceoftech.com/swhitley/index.php/2010/04/open-source-net-c-twitter-streaming-api-client/
• http://code.google.com/p/twitterstreamclient/
• http://www.codeproject.com/Articles/75003/Creating-a-Simple-Twitter-Client-Application.aspx

The above links lead you to opensource libraries and documentation on how to access the Twitter API (both the streaming API and the REST API) with .NET / C#.