Diving into the Twitter Stream

May 21st, 2010 Posted by: Collin - posted under:Tutorials

Hey iCoders. As I posted earlier I recently made an iPad app called . I just put out a new release of the app with a big new feature. The app now has a scrolling timeline of the tweets it is seeing as they come in. In order to make this app I took advantage of the that is provided by twitter. This API creates a persistent connection between the Twitter servers and your application. We will essentially start a stream of incoming NSData object to an NSURLConnection that you create querying the stream. We will look into the different search parameters which can be passed into the request, the way in which our code responds to authentication requests from Twitter, and the logic we must use to ensure that the data we have received is a complete XML element and not chopped off. There are three major steps to taking advantage of this in your app.

  1. Create an NSURLConnection to request an XML response from the Stream.
  2. Create a parameter string for the HTTP Body.
  3. Respond to the authentication challenge with user credentials.
  4. Append the data as it comes in and when a complete element is received parse the Tweets.

So lets dive in.

Creating the Class

The first thing that we need to do is establish a connection with the Twitter stream. I will not be creating a Twitter client class that can be plugged into any application you choose. You will be able to find the class as a download at the end of the post. We will call the class TwitterStreamClient. Lets first define the header file of the class.

#import "TouchXML.h"
#import "Tweet.h"
@interface TwitterClient : NSObject {
        NSString *searchString;
        NSString *locationString;
        NSMutableString *holderString;
        id mapController;
        NSURLConnection *connection;
        NSMutableURLRequest *request;
@property (nonatomic, assign) id mapController;
-(NSString*)locationStringForLongitude:(double)_long Latitude:(double)_lat;

You will need to have TouchXML installed for this class to work. You can find installation instructions . Let move onto the .m file of the TwitterStreamClient. First thing to do is create the initializer. Ours will look like this:

-init {
        if([self = [super init]) {
                holderString = [[NSMutableString alloc] init];
        return self;

This holder string is what we are going to use to synchronize our incoming NSData from the service. We will see the reason for this in the next step.

Connect to Service

The Twitter stream provides many different parameters for users to pass into their request. You can see the full list , but we will be focusing on user specific stream, keyword specific stream and location specific stream. We are going to be hardcoding what these will be searching for but in use you can pass in whatever values you like. We will create an NSMutableURLRequest and fill in its HTTP body with the appropriate request. First we will make a method to create the stream request.

-(void)makeMyRequest {
        request = [[NSMutableURLRequest alloc] initWithURL:[NSURL URLWithString:@"http://stream.twitter.com/1/statuses/filter.xml"]];
        [request setHTTPMethod:@"POST"];

With this done we will create three different methods that can be called. One that bases its search on keywords, another based on users and a final one based on location.

Request by keyword

-(void)searchByTerm {
        searchString = @"track=Love,Hate,Want,Need";
        NSString *httpBody = searchString;
        [request setHTTPBody:[httpBody dataUsingEncoding:NSUTF8StringEncoding]];
        connection = [[NSURLConnection alloc] initWithRequest:request delegate:self startImmediately:YES];
        [request release];

Request by User ID

-(void)searchByUser {
        searchString = @"follow=14402149,29089557,807095,19058681";
        NSString *httpBody = searchString;
        [request setHTTPBody:[httpBody dataUsingEncoding:NSUTF8StringEncoding]];
        connection = [[NSURLConnection alloc] initWithRequest:request delegate:self startImmediately:YES];
        [request release];

The final method will create a location string. The twitter stream requires passing in 4 coordinates, 2 coordinate pairs. This will define the southwest most point and northeast most point. The square that the coordinates define has a maximum of 1 degree length for any side of the defined box. My helper method takes in a longitude and latitude as a center points and creates as large an area as possible around it and returns the 4 coordinates as a string. Paste in the following code to search with center points on Tempe, AZ and New York, NY.

-(NSString*)locationStringForLongitude:(double)_long Latitude:(double)_lat {
        NSString *returnString = [NSString stringWithFormat:@"%f,%f,%f,%f",(_long-.5),(_lat-.5),(_long+.5),(_lat+.5)];
        return returnString;
-(void)searchByLocation {
        searchString = [NSString stringWithFormat:@"locations=%@,%@",
                                          [self locationStringForLongitude:-111.932898 Latitude:33.419265],
                                          [self locationStringForLongitude:-74.0 Latitude:40.7]];
        NSString *httpBody = searchString;
        [request setHTTPBody:[httpBody dataUsingEncoding:NSUTF8StringEncoding]];
        connection = [[NSURLConnection alloc] initWithRequest:request delegate:self startImmediately:YES];
        [request release];

Answering Authentication

The NSURLConnection will connect to the Twitter stream and then Twitter will ask for a username and password for the request. With the NSURLConnection’s delegate set to self we will implement the following method to answer the TwitterRequest.

- (void)connection:(NSURLConnection *)connection didReceiveAuthenticationChallenge:(NSURLAuthenticationChallenge *)challenge {
        NSURLCredential *cred = [[NSURLCredential alloc] initWithUser:@"USERNAME" password:@"PASSWORD" persistence:NSURLCredentialPersistencePermanent];
        [[challenge sender] useCredential:cred forAuthenticationChallenge:challenge];
        NSLog(@"Received Challenge");
        [cred release];

This should satisfy the authentication challenge and the stream should start sending you data.

Receiving Data

Now the stream will start sending you data, it will coming into another NSURLConnection delegate method. The stream will send NSData into this method. The NSData can be parsed into a string and if you print it you will see output like this:

2010-05-21 14:05:17.491 TweetMapper[72280:207] Recieved Data from Stream:
 2010-05-21 14:05:17.919 TweetMapper[72280:207] Recieved Data from Stream:
  Fri May 21 21:05:17 +0000 2010
  Need some followers yo I jus got this twitter its pretty str8
  <a href="http://mobile.twitter.com" rel="nofollow">Twitter for Android</a>
    David Powell
    I'm a chill dude that love to play ball, hang out,  and uhhhh chillax...holla at me and we can be cool
    Mon May 17 16:17:15 +0000 2010
2010-05-21 14:05:23.073 TweetMapper[72280:207] Recieved Data from Stream: e_background_color>
    Fri Aug 01 08:10:36 +0000 2008
    Pacific Time (US & Canada)

A response from the Twitter stream comes in as XML. An entire XML element that represents a single is a element. The status element seen here took two different calls to the didRecieveData method. Because of this we need to build in logic to recognize when a complete Tweet XML object is received and pass that onto our parsing method. This is what we use the NSMutableSting holderString for. You can see the completed method below.

- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data {
        NSString *dataString = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
        [holderString appendString:dataString];
        [dataString release];
        if([holderString rangeOfString:@""].location != NSNotFound && [holderString rangeOfString:@""].location != NSNotFound) {
                if([holderString rangeOfString:@""].location < [holderString rangeOfString:@""].location) {
                        NSRange start = [holderString rangeOfString:@""];
                        NSRange end = [holderString rangeOfString:@""];
                        NSRange range;
                        range.location = start.location;
                        range.length = (end.location+end.length)-start.location;
                        NSString *xmlString = [holderString substringWithRange:range];
                        [self parseXMLString:xmlString];
                        [holderString deleteCharactersInRange:range];

Here we receive data and append it to our holder string. We then check that both an opening and closing element tag is seen. If both tags are seen and the opening tag occurs before the closing tag, we create a substring in that range, pass the completed element to a method called parseXMLString and delete the characters from the holder string.

Parsing Data

Now that we have completed elements being grabbed out of the data we receive we need to parse out what we find as important. For the sake of extensibility I created a helper object called Tweet. A Tweet takes in tweet text, an author name, the authors Twitter URL, a timestamp, an image URL and a coordinate. This is the header for the class

@interface Tweet : NSObject {
        NSString *tweet;
        NSString *author;
        NSURL *authorURL;
        NSString *timeStamp;
        NSURL *authorImageURL;
        CLLocationCoordinate2D coord;
@property (nonatomic, retain) NSString *tweet;
@property (nonatomic, retain) NSURL *authorImageURL;
@property (nonatomic, retain) NSString *author;
@property (nonatomic, retain) NSURL *authorURL;
@property (nonatomic, retain) NSString *timeStamp;
@property (nonatomic, assign) CLLocationCoordinate2D coord;

I implement the class like this. I include a method called TweetLink which will return an NSURL of any url within the tweet text.

#import "Tweet.h"
@implementation Tweet
@synthesize tweet;
@synthesize authorImageURL;
@synthesize author;
@synthesize authorURL;
@synthesize timeStamp;
@synthesize coord;
-init {
        if([super init]) {
        return self;
-(NSURL*)tweetLink {
        NSRange httpRange = [[self tweet] rangeOfString:@"http://"];
        if(httpRange.location == NSNotFound) {
                return nil;
        else {
                httpRange.length = [[self tweet] length] - httpRange.location;
                NSString *customString = [[self tweet] substringWithRange:httpRange];
                httpRange = [customString rangeOfString:@" "];
                if(httpRange.location == NSNotFound) {
                        httpRange.location = 0;
                        httpRange.length = [customString length];
                else {
                        httpRange.length = httpRange.location;
                        httpRange.location = 0;
                NSLog(@"URL I am returning: %@", [customString substringWithRange:httpRange]);
                return [NSURL URLWithString:[customString substringWithRange:httpRange]];
-(UIImage*)authorPhoto {
        return [[UIImage alloc] initWithData:[NSData dataWithContentsOfURL:authorURL]];
-(NSString*)description {
        return [NSString stringWithFormat:@"Tweet:%@\nAuthor:%@\nAuthor URL:%@\nPublished:%@\nLongitude:%f\nLatitude:%f\nAuthor Image URL:%@", tweet, author, authorURL, timeStamp,coord.longitude,coord.latitude, authorImageURL];

I will use this class to create a single object holding all of the data I parse out of the XML elements I am looking at. All that is left to do is finish my parse method to get create these Tweet objects. This is my parse method that takes advantage of Touch XML and its awesome XML parsing methods.

-(void)parseXMLString:(NSString*)xmlString {
        CXMLDocument *document = [[CXMLDocument alloc] initWithXMLString:xmlString options:0 error:nil];
        if([[[[document rootElement] elementsForName:@"geo"] objectAtIndex:0] childCount] > 0) {
                Tweet *tweet = [[Tweet alloc] init];
                [tweet setTweet:[[[[document rootElement] elementsForName:@"text"] objectAtIndex:0] stringValue]];
                [tweet setAuthor:[NSString stringWithFormat:@"@%@", [[[[[[document rootElement] elementsForName:@"user"] objectAtIndex:0] elementsForName:@"screen_name"] objectAtIndex:0] stringValue]]];
                [tweet setAuthorURL:[NSURL URLWithString:[[[[[[document rootElement] elementsForName:@"user"] objectAtIndex:0] elementsForName:@"url"] objectAtIndex:0] stringValue]]];
                [tweet setAuthorImageURL:[NSURL URLWithString:[[[[[[document rootElement] elementsForName:@"user"] objectAtIndex:0] elementsForName:@"profile_image_url"] objectAtIndex:0] stringValue]]];
                NSString *coordinateString = [[[[[[document rootElement] elementsForName:@"geo"] objectAtIndex:0] elementsForName:@"point"] objectAtIndex:0] stringValue];
                NSRange range = [coordinateString rangeOfString:@" "];
                NSRange lon;
                lon.location = 0;
                lon.length = range.location;
                NSRange lat;
                lat.location = range.location+range.length;
                lat.length = [coordinateString length] - lat.location;
                double longit = [[coordinateString substringWithRange:lon] doubleValue];
                double latit = [[coordinateString substringWithRange:lat] doubleValue];
                CLLocationCoordinate2D coord;
                coord.longitude = latit;
                coord.latitude = longit;
                [tweet setCoord:coord];
                [tweet release];


Make sure that you install Touch XML and import the MapKit framework when using this class. Customize the terms, users and coordinates for the stream method to fit what you need it for. All that will be required to use the class is the following to create and begin parsing:

TwitterClient *client = [[TwitterClient alloc] init];
[client makeMyRequest];
[client searchByTerm];
[client searchByLocation];
[client searchByUser];

Fillin the very end of the parsing method to send the tweets off to whatever part of the application you want to use them in. I hope this introduction will help you guys add live twitter streams into your apps. Happy coding!

Source Downloads


  • Caleb

    Thanks Collin, how would one go about integrating this into an application for displaying incoming Tweets full screen?

  • http://icode.dreamvision-soft.com/blog/?p=49 Diving into the Twitter Stream | iCode

    [...] Original post on iCodeBlog [...]

  • Dirk-Jan

    Thanks for the post. Just what I was looking for.

    One question though. Why is there a double check for the same thing in the didReceiveData loop?

    if([holderString rangeOfString:@""].location != NSNotFound && [holderString rangeOfString:@""].location != NSNotFound) {

    It seems to be checking the same thing twice… few seconds later… The downloadable code explains. :) The <status> dissapears in the post. I always prefer on typing the code myself so I know what is happening.

    Thanks again!

  • http://terumoinfo.com/wordpress/?p=41 ★TeRuMoJapan★

    Over two atomic mushroom…

    Japan was bombed twice over the mushrooms 堕Shi materialistic nation had an opportunity to defeat in World War II Speech is dominated by the media over the result I lost a spiritual leader Lose sight of the national outlook on life but she was unable fi…

  • http://terumoinfo.com/wordpress/?p=44 ★TeRuMoJapan★

    Kan trick saw through policy speech…

    Kan coffee trick saw through policy speech ↓ Are deceiving the public will pass Cheating on defense this national crisis to hide true feelings The economy was silent cheats education For example, economic Social Security strong strong strong economic f…