Talking Puffin is a full blown desktop Twitter client, but it also includes an independently consumable Twitter API. This provides a Scala implementation of most Twitter API functions, buildable with Maven and includable in other projects.

The first step in using the Twitter API is getting source. You can either download this from http://github.com/dcbriccetti/talking-puffin/ or use git to clone your own repository.

The next step is building the code. Go to the base directory of the project (the one with README.md), and execute mvn install. This will run the scala compiler on both the desktop and Twitter API projects and install them into your Maven repository. Note that you can also do this from the twitter-api subdirectory if you only want to build the Twitter API.

After code has successfully built you can interact with the Twitter API via the scala console. To do this, go to the twitter-api directory and run mvn scala:console. You should see something like the following

mcmac:twitter-api mmcbride$ mvn scala:console
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'scala'.
[INFO] ------------------------------------------------------------------------
[INFO] Building TalkingPuffin Twitter API
[INFO]    task-segment: [scala:console]
[INFO] ------------------------------------------------------------------------
[INFO] Preparing scala:console
[INFO] [resources:resources]
[INFO] Using default encoding to copy filtered resources.
[INFO] [compiler:compile]
[INFO] Nothing to compile - all classes are up to date
[INFO] [scala:compile {execution: default}]
[INFO] Checking for multiple versions of scala
[INFO] Nothing to compile - all classes are up to date
[INFO] [scala:console]
[INFO] Checking for multiple versions of scala
Welcome to Scala version 2.7.3.final (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_07).
Type in expressions to have them evaluated.
Type :help for more information.

scala>

In the console you can now import the Twitter API package and create a session.

cala> import org.talkingpuffin.twitter._
import org.talkingpuffin.twitter._
import org.talkingpuffin.twitter._
 
scala> val sess = TwitterSession(twitterUser,twitterPassword)
val sess = TwitterSession(twitterUser,twitterPassword)
sess: org.talkingpuffin.twitter.AuthenticatedSession = org.talkingpuffin.twitter.AuthenticatedSession@9fbc913

Once you have a session, you can call twitter methods… this example shows us getting messages from the public timeline

scala> val timeline = sess.getFriendsTimeline("mccv")
val timeline = sess.getFriendsTimeline("mccv")
timeline: List[org.talkingpuffin.twitter.TwitterStatus] = List(org.talkingpuffin.twitter.TwitterStatus@73ec8c2, org.talkingpuffin.twitter.TwitterStatus@2aee3c45, org.talkingpuffin.twitter.TwitterStatus@7eb6ec07, org.talkingpuffin.twitter.TwitterStatus@1b42008f, org.talkingpuffin.twitter.TwitterStatus@a32ba44, org.talkingpuffin.twitter.TwitterStatus@862cb97, org.talkingpuffin.twitter.T...

The object returned is a List of TwitterStatus objects. The TwitterStatus object is a container for a single tweet, and has fields for most of the elements of the XML returned from the Twitter API. The following example shows a way to iterate over this list and print out the user and text. Note that the user field of the TwitterStatus object is a TwitterUser object, which contains info about a user.

scala> timeline foreach {(x) => println(x.user.name + "--\t" + x.text)}
timeline foreach {(x) => println(x.user.name + "--\t" + x.text)}
Dave Winer--	Switched at Birth, Women Find New Identity. http://tr.im/lixe
Dave Winer--	Yes, there is something super-ironic about a long list of URL shorteners. :-) http://tr.im/li3i
Robert Dempsey--	Yes, I'm on a bus in Austin.   http://yfrog.com/emz3oj
Dave Winer--	The NY Times/Twitter feed (Scripting News). http://tr.im/lixY
Dave Winer--	No LOST spoilers!!
Twitter API--	Starting something serious on the API? Want to launch your project May 26? Launch at @140tc: http://bit.ly/8jj8x ^DW
Dave Briccetti--	Trying to get more RAM on my shit dedicated server, davebsoft.com. Shit because OLM wanted a fortune per month to add dirt-cheap RAM.
Michael Arrington--	FriendFeed Enables People/Group Tracking http://tcrn.ch/1t4 by @parislemon
rick--	OMG LOST finale in a couple hours
Steve Spalding--	A hard lesson to learn is that you can't hand people success. The best you can do is give them the tools.
Michael Arrington--	Google's Last MySpace Payment: $75 Million On June 20, 2010 http://tcrn.ch/1t9 by @arrington
Alex Payne--	On my way to have a beer with two of the @dropbox guys. Once tipsy, I may request more jiggabytes.
Robert Dempsey--	OH: "They all run in the same circles. They fight, they hang out, they have transvestite beauty pageants."
Albert Wenger--	On the ground at SFO - meeting up with @joshu for dinner.
NewsGang--	Calling all iPhones! Emergency scanner apps on the loose! (from Steven Sande) : Filed under: Software, Odds and .. http://tinyurl.com/qpom96
NewsGang--	Quite Possibly the Coolest Hot Tub You Have Ever Seen | dornob (from dornob.com) http://tinyurl.com/p2r54h
Dave Winer--	Red tree: http://tr.im/liMR
Dave Winer--	Guerilla Cafe, Berkeley: http://tr.im/liMS
Dave Winer--	Tree with red leaves in sunset: http://tr.im/liMT
Dave Troy--	There will be a 2010 re-enactment of the 1871 Baltimore parade celebrating the ratification of the 15th amendment (my college thesis).
scala>

Note that we can get a bit fancier with this and use for comprehensions to do more or less the same thing. The following sample uses the API to filter the friends timeline, looking for any text with the word “Berkeley”

scala> for(x <- sess.getFriendsTimeline("mccv") if x.text.indexOf("Berkeley") > 0) println(x.text)
for(x <- sess.getFriendsTimeline("mccv") if x.text.indexOf("Berkeley") > 0) println(x.text)
Guerilla Cafe, Berkeley: http://tr.im/liMS
 
scala>

This should give you a pretty decent start on using the API. There are many more functions available, and you can take a look by generating the scala doc and checking out the API methods. To generate the site, go to your twitter-api directory and run “mvn site”. This should generate a set of HTML pages in target/site. Open index.html, click project reports, click scala docs, and look at the generated documentation for AuthenticatedSession.

I’ve recently been working on the talking puffin project, particularly on the lower level twitter API. At the plumbing level, we want a class that allows us to fetch XML from a URL. In Java doing this sort of stuff usually involves quite a few checked exceptions. The interwebs are flakey, and ignoring that fact usually leads to really bad things happening. However Scala doesn’t have checked exceptions… and that’s a nice thing for flexibility, conciseness, etc. So what to do?

My two basic approaches were

1. Throw exceptions on errors. this allows more concise code when you don’t care about the errors (maybe you’re dealing with them higher up)
2. Use case classes with subclasses indicating successful and unsuccessful execution. This forces you to deal with error conditions right away, which may lead to more stable code

Let’s take a look at implementations of both. First we set up a simple base class to deal with HTTP plumbing. This just provides a way to open a URL and a helper method to read the error stream into a string.

class HttpBase{
  /**
  * Open the specified URL and return a tuple containing the corresponding
  * HTTP response code and HttpURLConnection
  */
  def openConnection(urlStr:String) = {
    val url = new URL(urlStr)
    val conn = (url.openConnection).asInstanceOf[HttpURLConnection]
    val responseCode = conn.getResponseCode()
    (responseCode,conn)
  }
 
  /**
  * Utility to open a connection's error stream and read it into a string'
  */
  def getErrMsg(conn:HttpURLConnection) = {
    var errMsg = ""
    val reader = new BufferedReader(
          new InputStreamReader(conn.getErrorStream()))
    var line = reader.readLine()
    while(line != null){
      errMsg += line
      line = reader.readLine()
    }
    errMsg
  }
}

Now let’s try to implement a class that actually fetches XML, and throws an exception if errors are encountered.

case class HttpException(val msg:String, val code:Int) extends Exception
 
/**
* Provides a getXML method that fetches XML from a URL.
* Throws an exception if any errors are encountered
*/
class HttpXMLExceptions extends HttpBase{
  def getXML(urlStr:String):Node = {
    try{
      openConnection(urlStr) match {
        case (200,conn) => XML.load(conn.getInputStream())
        case (code,conn) => throw HttpException(getErrMsg(conn),code)
      }
    }catch{
      case e => throw HttpException(e.toString,-1)
    }
  }
}

And here’s a usage sample. This is concise, but (by design) we aren’t forced to think about nasty things like 404s, 401s, dropped connections, broken pipes, and all sorts of other networky hobgoblins.

    val excs = new HttpXMLExceptions()
    val url = "http://twitter.com/statuses/public_timeline.xml"
    val content = excs.getXML(url)

Let’s try to force people to think about them. Instead of throwing an exception, we can define a hierarchy of case classes for our responses, like so.

case class Response
case class Success(val content:Node) extends Response
case class Error(val msg:String,val code:Int) extends Response

Our method will return a type of Response. If our request is successful we’ll get a Success object that has a content field. If we run into any errors we’ll instead get an Error object back with a response code and error message from the response body. This is similar to using Scala’s Option type, however this allows us to provide information on failure instead of simple returning the None instance. Here we go…

/**
* Proveds a getXML method that fetches XML from a URL.
* Always returns a Response object, regardless of success/failure
*/
class HttpXMLMatches extends HttpBase{
  def getXML(urlStr:String):Response = {
    try{
      openConnection(urlStr) match {
        case (200,conn) => Success(XML.load(conn.getInputStream()))
        case (code,conn) => Error(getErrMsg(conn),code)
      }
    }catch{
      case e => Error(e.toString,-1)
    }
  }
}

This is pleasantly similar in implementation… However usage looks quite a bit different…

    val matches = new HttpXMLMatches()
    val url = "http://twitter.com/statuses/public_timeline.xml"
    val content = matches.getXML(url) match {
      case Success(node) => node
      case Error(msg,code) => 
        <error><msg>{msg}</msg><code>{code}</code></error>
    }

This is quite a bit more verbose. However it does send a message that developers need to be conscious of the potential error case. Let’s look at side by side usage when we add error handling for an unknown host

    val excs = new HttpXMLExceptions()
    val badUrl = "http://nohost.twitter.com/statuses/public_timeline.xml"
    val noContent = try{
      excs.getXML(badUrl)
    }catch{
      case HttpException(msg,code) => 
        <error><msg>{msg}</msg><code>{code}</code></error>
    }
 
    val matches = new HttpXMLMatches()
    val noMatchContent = matches.getXML(badUrl) match {
      case Success(node) => node
      case Error(msg,code) => 
        <error><msg>{msg}</msg><code>{code}</code></error>
    }

This usage actually looks pretty similar too. This is due to the fact that a try/catch evaluates like a pattern match. In the exception case your error conditions are separated from the main logic, which some may like and some may not.

The primary difference is that using the Response class hierarchy forces the user to consider the the unsunny day scenarios. I don’t think there’s a black and white rule on when to use either approach, but case classes provide you a way to expose error conditions more visibly than unchecked exceptions. In the talking puffin project I think we’ll stick with exceptions, but it’s nice to have options.

So I went to the last half of Scala Lift Off on Saturday (only half, because the first half was taken up by my final MBA class.  Ever.).  I went primarily out of curiosity, not knowing much about Scala or Lift.  The main draw was the built in comet support for Lift, which seems to not be a focus in other frameworks… at least not for Rails.  We currently use Juggernaut for comet support, but depending on flash is something of a liability (see: iPhone), and Juggernaut itself isn’t as smoothly integrated with Rails as i’d like.

I came away extremely impressed.  Scala is relatively unheralded in the world of alternative JVM languages (see Groovy, Jython and JRuby publicity), but shows a lot of promise.  It’s a functional language with an expressive syntax that allows you to easily create code that looks DSL-ish.  These are the primary features that drew me to Ruby (ok, Ruby isn’t a functional language, but you can sorta fake it).  But Scala has a better integration story with existing Java libraries, is strongly typed, and has a stronger functional bent.

I’m a big believer in the right tool for the job, and as such don’t fall into a pure-dynamic or pure-static language camp.  I also don’t fall into a single language camp. I really enjoy Ruby for quick prototyping, and love Rails for quick prototyping of webapps, and maintaining a nimble production face on web applications.  But Rails falls down when I need to run background processing.  The times I think hardest about moving back to a Java webapp environment are when I need to go write something that doesn’t just receive a web request and terminate.  This is where concurrency issues get painful to deal with, Ruby daemons/DRb are painful, and starting up a whole Rails env for simple processing is rough.

So I’m hoping Scala/Lift fills that void.  I’m mentally sketching out a replacement of our background processing jobs (Twitter integration, email processing, etc.) with Scala, and in particular the Actors library.  These are relatively simple processing tasks, and should give me a decent feel for the language.  It should also improve the stability and scalability of our background processing.  It may also yield some reasonable libraries to contribute back (Scala Twitter library, Scala ActiveRecord bridge). 

Once I have that nailed down, an evaluation of how Lift can/should fit into the framework is in order… or maybe I’ll have to start my Rettiwt side project based on Lift.