Introduction
I just read a magnificent blog post from Joseph Smarr over at Plaxo that inspired me to share some of my experiences in helping to build and test TripIt's OAuth protected API.
I've written this post as a guide that lets you manually walk through all of the steps an OAuth consumer must implement in order to be able to make authenticated calls to TripIt's API. At each step of the process I've referenced a set of command line utilities that you can use to run through the examples as well as explanations for what those utilities are doing under the covers. All of these utilities can be found in TripIt's Python binding/wrapper library here:
http://groups.google.com/group/api_tripit/web/tripit-api-language-bindings
The Python binding for the TripIt API is actually a complete OAuth consumer implementation in addition to a convenient wrapper for the TripIt API. The command line utilities in this package enable you to conveniently execute each step of the OAuth authorization process from the command line. I recommend you go through this process a few times by hand as it should help you (it definitely helped me) get your head around all of the different steps involved in getting a user to authorize your application. Note: executing any of these commands w/o command line arguments will print a useful usage message.
With the specific example I walk through in this document I've included all of the public and private portions of all request and access tokens as well as the consumer key and secret. If you need to re-implement an OAuth consumer in another language you can use the inputs and outputs for each step of the process in this post to test your code to see if your code is working.
Also, please note that the discussion is very TripIt focused. While we implemented the provider to the specification we didn't implement everything (e.g. the only signing algorithm we built was HMAC-SHA1).
To frame the overall discussion, imagine the requests coming from a consumer application (Alice) and a content publisher (Bob). OAuth channels the authentication for this dialogue in a highly stylized way.
Alice: I would like to develop a TripIt application. [Registers application]
Bob: Thanks. Here are the permanent Consumer Token and the Consumer Secret values for that application.
Alice: Now, I would like some private information from you about trips. Here is my Consumer Token and Secret. Also, since an eavesdropper might try to replay my request later, I'll also add a random Nonce and a Timestamp. Finally, I have signed this request in a very specific way.
Bob: Everything is in order here, thanks. Here is an Unauthorized Request Token, good for one use only.
Alice: The user must ok the next step. Browser, here's a url with that Unauthorized Request Token, plus a Callback URL argument.
Web Browser: Tripit, here's that URL.
[User logs in & grants access. ]
Bob: All is good. I'm creating an Authorized Request Token, valid just for that user and your application. Now I'm redirecting to the callback URL.
Alice: Here's my Consumer Token and Secret as well as my Request Token and Secret. Give me the Authorized Request Token and matching Secret you just generated, please.
Bob: Here they are.
Alice: Sweet. All future requests from me will have the four values:
- Consumer Token
- Consumer Secret
- Authorized Request Token
- Authorized Request Secret
Bob: We stand ready for all your travel information needs.
Although simplified to an extreme degree, this shows the fundamentals of the protocol.
Before getting started I wanted to give a shout-out to my colleague Travis W. who's a very smart guy that didn't know anything about OAuth before helping me proof-read this post. He contributed the idea for the OAuth discussion you just read and claims this run-through helped him wrap his head around how to write a consumer. I hope he wasn't just being nice. Let's get started!
Getting an Unauthorized Request Token
The first step is to get an unauthorized request token. An unauthorized request token is just a string that enables an OAuth consumer to "ask" a user to grant it an authorized access token. To get an unauthorized request token a consumer must have a consumer key and secret. To generate a set of these to work with the TripIt API you simply go through the process of creating an application here: http://www.tripit.com/developer (Note: you must have a TripIt account to do this but that's free so what's stopping you?!).
IMPLEMENTATION NOTE: You need to store the the consumer key/secret somewhere so that your application can get to them everytime it needs to generate a new request token or obtain an authorized access token.
For this example I'm going to use the following consumer key/secret:
| Key: | 5dbf348aa966c5f7f07e8ce2ba5e7a3badc234bc |
|---|---|
| Secret: | fceb3aedb960374e74f559caeabab3562efe97b4 |
Once you have your consumer key/secret you can use the get_request_token.py command to generate a new request token like this:
$ consumer_token=5dbf348aa966c5f7f07e8ce2ba5e7a3badc234bc $ consumer_secret=fceb3aedb960374e74f559caeabab3562efe97b4 $ python get_request_token.py https://api.tripit.com $consumer_token $consumer_secret RESPONSE: oauth_token=df919acd38722bc0bd553651c80674fab2b46508&oauth_token_secret=1370adbe858f9d726a43211afea2b2d9928ed878
Here is what the get_request_token.py script is doing under the covers:
In order to make any OAuth request a consumer must know how to properly form the request. The first part of understanding how to form a proper request is understanding what request parameters are required. Here is a list of all OAuth the request parameters, what they represent, and how to generate them:
- oauth_nonce
A nonce is just a Number you use ONCE. That's actually a bit of a misnomer because a nonce doesn't really need to be a number at all. It just needs to be a random and globally unique string. The nonce is used along with the timestamp to uniquely identify each request made to an OAuth provider and helps the provider detect whether or not an attacker is trying to "replay" an intercepted API call. For a more in-depth description of the security risks around a replay attack check out Eran Hammer-Lahav's post on OAuth security architecture.
The OAuth consumer code in the TripIt library creates nonces using the OAuthConsumer::_generate_nonce() method from tripit.py by computing the hex form of an md5 checksum on a concatenated string that contains the current epoch time in seconds plus a string of 40 random numbers between 0 and 9. For those who read Python:
def _generate_nonce(self):
random_number = ''.join(str(random.randint(0, 9)) for i in range(40))
m = md5.new(str(time.time()) + str(random_number))
return m.hexdigest()
The important thing to make sure about the code you write to create the nonce is that it will not generate the same string more than once across all requests.
The timestamp is just the integer representation of the number of seconds since the epoch. It's used in conjunction with the nonce to make sure the API doesn't respond to requests that have already been made.
The consumer key is the string you obtained when you registered your application with the OAuth provider. In TripIt's case you can do that here: http://www.tripit.com/developer. In the example we are working through here the consumer key is:
5dbf348aa966c5f7f07e8ce2ba5e7a3badc234bc
The OAuth spec allows for 3 different signing algorithms (i.e. HMAC-SHA1, RSA-SHA1, and PLAINTEXT). TripIt only supports HMAC-SHA1 and so that is the only signing algorithm I am going to describe in any detail.
At this time there is only one version of the OAuth specification (version 1.0). Therefore, the value of the version parameter should be "1.0".
The signature is where most people who are trying to make an OAuth request run in to trouble. In order for a signature to be "valid" both the consumer and provider must implement the same signing algorithm so that when the provider re-computes a signature for a request it matches exactly the one posted to it in the request by the consumer. If they is even a single character difference, the request will fail with a 401 "Invalid Signature" return code.
Here's a description of the algorithm used to generate a signature (all of this is documented in code in the tripit.py implementation of OAuthConsumer::generate_oauth_parameters() so you can follow along if you read Python):
- Collect all request parameters
- Normalize the request parameters
- Sort the parameter dictionary by key name (Note: I'm not concerned about redundantly named keys in this implementation because it's not a concern for TripIt. If you needed to consider this the OAuth spec says sort by parameter name and for parameter names that are equivalent, sort by value)
- URL Encode each key/value pair
- Create a list of <key>=<value> pairs and join them all together with &'s
- URL encode the whole thing
- Construct the signature base string
- Generate a key
- Compute the oauth_signature
The first step in computing a signature is to collect all the parameters in the request. All parameters means all of the oauth_* parameters (i.e. oauth_consumer_key, oauth_nonce, oauth_timestamp, etc...) plus any parameters that are part of either the query string and/or the body of a POST request (e.g. xml= or json=). The implementation in tripit.py simply creates a Python dictionary with all the OAuth parameters and combines those with the args in the query string (i.e. url_args) and the POST body (i.e. post_args):
parameters = self._oauth_parameters
if url_args is not None:
parameters.update(url_args)
if post_args is not None:
parameters.update(post_args)
According to the OAuth spec all request parameters must be sorted using lexicographical byte value ordering, url encoded, and then concatenated together with an '&' separating each value. Here's the code from tripit.py that does this:
normalized_parameters = self._escape('&'.join(['%s=%s' % (self._escape(str(k)), self._escape(str(parameters[k]))) for k in sorted(parameters)]))
So, in English:
Take the sorted, escaped, and concatenated list of URL parameters you just generated and prepend to it the normalized HTTP method name and URL all separated by '&'. Here's the code that generates the signature base string:
signature_base_string = '&'.join([normalized_http_method, normalized_http_url, normalized_parameters])
The normalized HTTP method is simply the HTTP method (i.e. GET or POST) in all caps. The normalized HTTP URL is the resource URL in your request, URL encoded. Here is what the signature base string looks like in this example:
POST&https%3A%2F%2Fapi.tripit.com%2Foauth%2Frequest_token&oauth_consumer_key%3D5dbf348aa966c5f7f07e8ce2ba5e7a3badc234bc%26oauth_nonce%3D39100e296c709a592600c8d1a3ee69dd%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1235272437%26oauth_version%3D1.0
Up until now all the consumer has done has been to format the "non-secret" portions of the request. The consumer must now generate a key that it will ultimately use to compute the signature. For API requests such as the one to get an unauthorized request token, the key is simply the oauth_consumer_secret followed by a single '&'. For API requests for protected resources that are being made after an authorized access token has been obtained, the key is the oauth_consumer_secret and the authorized oauth_token_secret separated by a '&'. Here's what the key looks like for the request to get an unauthorized request token in this example:
fceb3aedb960374e74f559caeabab3562efe97b4&
Once you have the signature base string and the key, it's time to compute the signature. The signature will be sent to the provider with the request and if all goes well the provider will go through an identical set of steps to re-compute the signature. If you did your job right, the signature string the provider computes will be an exact match for the one computed by the consumer. TripIt only supports HMAC-SHA1 so here's some Python code that does that for you:
try:
import hashlib
hashed = hmac.new(key, signature_base_string, hashlib.sha1)
except:
import sha
hashed = hmac.new(key, signature_base_string, sha)
self._oauth_parameters['oauth_signature'] = base64.b64encode(hashed.digest())
For our example, the value of the signature is:
KlTlU95CdzFYo5tfrJjaPz5RA6g=
Once all the OAuth parameters are generated you will need to construct an HTTP Authorization header to send along as part of the request. The OAuth specification says in section 5.4 that the Authorization header is the preferred authorization scheme and it's the only one that TripIt supports. Here's what the authorization header looks like for this example:
Authorization: OAuth realm="https://api.tripit.com",oauth_nonce="39100e296c709a592600c8d1a3ee69dd",oauth_timestamp="1235272437",oauth_consumer_key="5dbf348aa966c5f7f07e8ce2ba5e7a3badc234bc",oauth_signature_method="HMAC-SHA1",oauth_version="1.0",oauth_signature="KlTlU95CdzFYo5tfrJjaPz5RA6g%3D"
In Python, using the urllib2 library, you add the header to the request like this:
request.add_header('Authorization', authorization_header);
If all goes well the server responds with an unauthorized request token as well as the unauthorized request token secret. The consumer's job is now to have the user authorize those request tokens. Here's what the response from the server looks like:
oauth_token=df919acd38722bc0bd553651c80674fab2b46508&oauth_token_secret=1370adbe858f9d726a43211afea2b2d9928ed878
IMPLEMENTATION NOTE: You need to store these values someplace where your application can retrieve them so that you will be able to retrieve the authorized access token later on. For this example, save the values you get from get_request_token.py as you'll need them to continue following this example.
Authorizing an Unauthorized Request Token
You've just completed the process of obtaining an unauthorized request token and secret. This token is only good for one purpose and that is to enable the consumer to ask the user to grant it an authorized access token. To mimic this part of the process you will need a web browser. When building an OAuth consumer the call to TripIt's API is usually immediately followed by an HTTP redirect to the following URL:
https://www.tripit.com/oauth/authorize?oauth_token=df919acd38722bc0bd553651c80674fab2b46508&oauth_callback=http%3A%2F%2Fwww.tripit.com%2Fhome
Naturally the specific value of the oauth_token depends on what you received when you requested a request token. The value of the oauth_callback is usually a URL in your application that implements the code I will talk about in the next section to download the authorized access token.
To mimic the behavior of your application, just go to the above URL w/ your web browser. Make sure to replace the value of oauth_token with whatever you got from get_request_token.py in the last section!
If you aren't logged in to TripIt when you access the URL you will be prompted to do so. Once logged in you'll see a screen that looks like this:
Click the 'Grant Access' button and notice that you are redirected back to http://www.tripit.com/home (i.e. the oauth_callback URL). The provider now has an authorized access token for your consumer and all you need to do is go and get it!
Obtaining the Authorized Access Token
Congratulations, you're nearly ready to start querying the API! All that's left to do is get the authorized request token and then use it. To obtain the authorized access token the TripIt Python binding package has another utility called get_authorized_token.py that can help you mimic what an OAuth consumer does to download the authorized access token after a user has clicked the 'Grant Access'
button on the /oauth/authorize page and has been redirected back to the consumer application. Here's what a sample run of the get_authorized_token.py utility looks like:
$ consumer_token=5dbf348aa966c5f7f07e8ce2ba5e7a3badc234bc $ consumer_secret=fceb3aedb960374e74f559caeabab3562efe97b4 $ request_token=df919acd38722bc0bd553651c80674fab2b46508 $ request_token_secret=1370adbe858f9d726a43211afea2b2d9928ed878 $ python get_authorized_token.py https://api.tripit.com $consumer_token $consumer_secret $request_token $request_token_secret RESPONSE: oauth_token=b865676c95c736c4cbb90c652cd896dec022ba86&oauth_token_secret=f3ce720ebb4c0ddc3469662c246f78ef4d99f1cb
IMPLEMENTATION NOTE: Save the authorized access oauth_token and oauth_token_secret values somewhere your application can get to them. You will need them to make API calls! It is now safe to discard the request token and token secret you obtained earlier.
The process of getting the authorized access token is exactly the same as getting a request token except for two things. First, the oauth_token_secret of the request token is included in the request parameters and therefore in both the signature base string and the key used to sign the request. This means that the
signature base string for this request looks like this:
POST&https%3A%2F%2Fapi.tripit.com%2Foauth%2Faccess_token&oauth_consumer_key%3D5dbf348aa966c5f7f07e8ce2ba5e7a3badc234bc%26oauth_nonce%3D45a4e8dc2000e5c3fb27aefeb4769326%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1235272542%26oauth_token%3Ddf919acd38722bc0bd553651c80674fab2b46508%26oauth_token_secret%3D1370adbe858f9d726a43211afea2b2d9928ed878%26oauth_version%3D1.0
Note the inclusion of the oauth_token_secret parameter. When generating the key that's used to create the oauth_signature parameter it should look like this:
fceb3aedb960374e74f559caeabab3562efe97b4&1370adbe858f9d726a43211afea2b2d9928ed878
The string after the & is the oauth_token_secret obtained along with the unauthorized request token.
The second difference is the URL you request the authorized access token from is https://api.tripit.com/oauth/access_token.
Aside from those two relatively minor differences, the process of requesting an authorized access token is identical to that of requesting the request token. token.
Using an Authorized Access Token
If all went well, your application now has an authorized access token that can be used to make API calls. While it's beyond the scope of this post to document the TripIt API, let's go through a simple example so you can see how to form an OAuth request properly. For a complete description of all the methods and their return
values you should check out TripIt's API documentation. Here:
http://groups.google.com/group/api_tripit/web/tripit-api-documentation---v1
To form a simple request to get a user's upcoming trips you would make a GET request to the protected resource URL: https://api.tripit.com/v1/list/trip.
To mimic this request on the command line use the make_request.py utility included in the TripIt Python library. Here's what a sample run of the make_request.py utility looks like:
$ consumer_token=5dbf348aa966c5f7f07e8ce2ba5e7a3badc234bc $ consumer_secret=fceb3aedb960374e74f559caeabab3562efe97b4 $ auth_token=b865676c95c736c4cbb90c652cd896dec022ba86 $ auth_token_secret=f3ce720ebb4c0ddc3469662c246f78ef4d99f1cb $ python make_request.py https://api.tripit.com/v1/list/trip $consumer_token $consumer_secret $auth_token $auth_secret RESPONSE: <Response><timestamp>1235272610</timestamp><Trip><id>1180136</id><relative_url>/trip/show/id/1180136</relative_url><start_date>2010-01-21</start_date><end_date>2010-01-30</end_date><display_name>Honolulu, HI, January 2010</display_name><image_url>http://www.tripit.com/images/places/general.jpg</image_url><is_private>true</is_private><is_traveler>true</is_traveler><primary_location>Honolulu, HI</primary_location></Trip></Response>
Under the covers, a request to a protected resource is basically the same as that to obtain an authorized access token. By now you should be very familiar with how an OAuth request works so I'll just describe the core components used to generate the request. Here's the signature base string:
GET&https%3A%2F%2Fapi.tripit.com%2Fv1%2Flist%2Ftrip&oauth_consumer_key%3D5dbf348aa966c5f7f07e8ce2ba5e7a3badc234bc%26oauth_nonce%3D720201c8b047528e4fcc119a98cffc8e%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1235272610%26oauth_token%3Db865676c95c736c4cbb90c652cd896dec022ba86%26oauth_token_secret%3Df3ce720ebb4c0ddc3469662c246f78ef4d99f1cb%26oauth_version%3D1.0
Note that since you are asking for upcoming trips and that's done via an HTTP GET to the URL https://api.tripit.com/v1/list/trip the signature base string starts out with a 'GET'.
The key, like the one used to sign the request for an authorized access token, includes both the oauth_consumer key and authorized access token oauth_token_secret. It should look like this:
fceb3aedb960374e74f559caeabab3562efe97b4&f3ce720ebb4c0ddc3469662c246f78ef4d99f1cb
The Authorization header should look like this:
Authorization: OAuth realm="https://api.tripit.com/v1/list/trip",oauth_nonce="720201c8b047528e4fcc119a98cffc8e",oauth_timestamp="1235272610",oauth_token_secret="f3ce720ebb4c0ddc3469662c246f78ef4d99f1cb",oauth_consumer_key="5dbf348aa966c5f7f07e8ce2ba5e7a3badc234bc",oauth_signature_method="HMAC-SHA1",oauth_version="1.0",oauth_token="b865676c95c736c4cbb90c652cd896dec022ba86",oauth_signature="QlHuyGIbtPNNra7qBEAcdbqQuGc%3D"
And the response will be an XML document that looks like this:
<Response>
<timestamp>1235272610</timestamp>
<Trip>
... XML that describes a Trip ...
</Trip>
</Response>
Conclusions
Congratulations, you've gone through the OAuth authorization flow and learned how to build an OAuth consumer that implements that flow. Implementing an OAuth consumer can be a slightly tedious task as there are a number of steps and all have to be executed very precisely to guarantee success. I highly recommend that you use the command line utilities provided with the TripIt Python library and go through the process a couple of times until you really understand the order of events and what each step means before you go trying to implement an OAuth consumer on your own. Once all of the steps are in your head, implementing them should be relatively
straightforward.
Good luck!
