How to Use Restful Web APIs in Python

This tutorial will introduce you to web APIs and teach you how to use the requests Python library to fetch and update information in web APIs. You will also learn how to interact with the Twitter API as a working example.

Introduction to Web APIs

An API (Application Programming Interface) is a framework for building HTTP services that can be consumed by a wide variety of clients. Web APIs use HTTP protocol to handle requests between the client and the web server.

Some of the most common APIs that enable developers to integrate and use their infrastructure include:

Google APIs
Twitter API
Amazon API
Facebook API

One of the most important reasons to use an API as opposed to other static data sources is because it's real time. For example, the Twitter API we are going to use will fetch real-time data from the social network.

Another advantage is that the data keeps changing, so if you were to download it at intervals, it would be time-consuming.

Using the Requests Library

In order to use an API, you will need to install the requests Python library. Requests is an HTTP library in Python that enables you to send HTTP requests in Python.

Install Requests

In your terminal, type:

pip install requests

To check if the installation has been successful, issue the following command in your Python interpreter or the terminal:

import requests

If there are no errors, the installation has been successful.

How to Get Information From a Web API

The GET method is used to get information from a web server. Let's see how to make a GET request to get GitHub's public timeline.

We use the variable req to store the response from our request.

import requests
req = requests.get('https://github.com/timeline.json')

Now that we have made a request to the GitHub timeline, let's get the encoding and the content contained in the response.

import requests
req = requests.get('https://github.com/timeline.json')
req.text
u'{"message":"Hello there, wayfaring stranger. If you\u2019re reading this then you probably didn\u2019t see our blog post a couple of years back announcing that this API would go away: https://git.io/17AROg Fear not, you should be able to get what you need from the shiny new Events API instead.","documentation_url":"https://developer.github.com/v3/activity/events/#list-public-events"}

import requests
req = requests.get('https://github.com/timeline.json')
req.encoding
'utf-8'

Requests has a built-in JSON decode which you can use to get the response of a request in JSON format.

import requests
import json
req = requests.get('https://github.com/timeline.json')
req.json()
{u'documentation_url': u'https://developer.github.com/v3/activity/events/#list-public-events', u'message': u'Hello there, wayfaring stranger. If you\u2019re reading this then you probably didn\u2019t see our blog post a couple of years back announcing that this API would go away: http://git.io/17AROg Fear not, you should be able to get what you need from the shiny new Events API instead.'}

How to Create and Update Information on the Web API

The POST and PUT methods are both used to create and update data. Despite the similarities, it's important to note that using a POST request to update data will result in two entries in the data store if two identical items are submitted.

Create data (POST request):

r = requests.post('http://127.0.0.1/api/v1/add_item', data = {'task':'Shopping'})

Update data (PUT request):

r = requests.put('http://127.0.0.1/api/v1/add_item', data = {'task':'Shopping at 2'})

Working With the Twitter REST API

In this section, you are going to learn how to obtain Twitter API credentials, authenticate to the Twitter API, and interact with the Twitter API using Python.

You will also be able to retrieve information from public Twitter accounts, like tweets, followers, etc.

Authenticating With Twitter

We need to authenticate with the Twitter API before we can interact with it. To do this, follow the following steps:

Go to the Twitter Apps page.
Click on Create New App (you need to be logged in to Twitter to access this page). If you don't have a Twitter account, create one.

3. Create a name and description for your app and a website placeholder.

4. Locate the Keys and Access Tokens Tab and create your access token.

5. You need to take note of the Access token and Access Token secret since you will need them for the authentication process.

6. You also need to take note of the Consumer Key and Consumer Secret.

There are a few libraries that we can use to access the Twitter API, but we are going to use the python-twitter library in this tutorial.

Install python-twitter

To install python-twitter, use:

$ pip install python-twitter

The Twitter API is exposed via the twitter.Api class, so let's create the class by passing our tokens and secret keys:

import twitter
api = twitter.Api(consumer_key=[consumer key],
                  consumer_secret=[consumer secret],
                  access_token_key=[access token],
                  access_token_secret=[access token secret])

Replace your credentials above and make sure they are enclosed in quotes, i.e. consumer_key=‘xxxxxxxxxx’, ...)

Querying Twitter

There are many methods of interacting with the Twitter API, including:

>>> api.PostUpdates(status)
>>> api.PostDirectMessage(user, text)
>>> api.GetUser(user)
>>> api.GetReplies()
>>> api.GetUserTimeline(user)
>>> api.GetHomeTimeline()
>>> api.GetStatus(status_id)
>>> api.DestroyStatus(status_id)
>>> api.GetFriends(user)
>>> api.GetFollowers()

To get data from Twitter, we are going to make an API call with the help of the api object we created above.

We will do the following:

Create a user variable and set it equal to a valid Twitter handle (username).
Call the GetUserTimeline() method on the api object and pass in the following arguments.

a valid Twitter handle
the number of tweets you want to retrieve (count)
a flag to exclude retweets (this is done using include_rts = false)

Let's get the latest tweets from the Envato Tuts+ Code timeline, excluding retweets.

import twitter
api = twitter.Api(consumer_key="xxxxxxxxxxxx", consumer_secret="xxxxxxxxxxxxxx",
                  access_token_key="314746354-xxxxx", access_token_secret="xxxxxx")

user = "@TutsPlusCode"
statuses = api.GetUserTimeline(
    screen_name=user, count=30, include_rts=False)
for s in statuses:
    print s.text

The GetUserTimeline() method will return a list of the latest 30 tweets, so we loop through the list and print the most important information (content) from each tweet.

Here are 6 things that make Yarn the best #JavaScript package manager around. https://t.co/N4vzIJmSJi
Find out more about bar charts with part 3 of this series on creating interactive charts using Plotly.js. https://t.co/lyKMxSsicJ
See what's new with Git support in Xcode 9. https://t.co/7gGu0PV1rV
Here's how to create digital signatures with Swift. https://t.co/kCYYjShJkW
In this quick tip, discover how to use Atom as a Git GUI. https://t.co/8rfQyo42xM
Take a look at these 12 useful WordPress plugins for page layouts. https://t.co/T57QUjEpu5
Learn more about line charts in part 2 of our series on creating interactive charts using Plotly.js. https://t.co/nx51wOzSkF
Grab some great freebies with our special 10th birthday offer, https://t.co/AcIGTiC2re
In this latest in our series on coding a real-time app with NativeScript: Push notifications. https://t.co/qESFVGVF4L
Get started with end-to-end testing in Angular using Protractor. https://t.co/TWhQZe7ihE
Learn how to build a to-do API with Node, Express, and MongoDB. https://t.co/R4DvRYiM90
What is the Android activity lifecyle? https://t.co/VUHsucaC1X
Learn all about object-oriented programming with JavaScript. https://t.co/bI7ypANOx3
Have some fun with functions in this latest in our series on Kotlin. https://t.co/r2f2TzA5lM
Here's how to make your JavaScript code robust with Flow. https://t.co/rcdjybKL8L
Build your own to-do API with Node and Restify. https://t.co/gQeTSZ6C5k
Here's how to put your view controllers on a diet with MVVM. https://t.co/oJqNungt1O
Learn how to submit your new iOS app to the App Store. https://t.co/JQwsKovcaI
This guide is the perfect place to build your skills and start writing plugins in Go. https://t.co/68X5lLSNHp
Take a look at how to test components in Angular using Jasmine. https://t.co/V5OTNZgDkR
Learn to build your first #WordPress plugin in this awesome new course. https://t.co/695C6U6D7V
In this latest in our Android architecture components series: LiveData. https://t.co/gleDFbqeAi
Take a deep dive into the Go type system. https://t.co/AUM7ZyanRO
Take a look at serverless logic with realm functions. https://t.co/aYhfeMgAZc
Part 4 of React crash course for beginners is out! https://t.co/aG5NEa6yG9
Learn about social login and Firebase in this new addition to the series. https://t.co/oL5z0krQD3
Creating a blogging app using React part 6 is out! Tags https://t.co/OzUaPQEX8E
What is GenServer and why should you care? https://t.co/EmQeTBggUK
Check out part 3 in React crash course for beginners series. https://t.co/dflLCUqncO
Learn about packages and basic functions in this addition to our Kotlin from scratch series. https://t.co/JAo2ckSgZS

To retrieve followers, we use the GetFriends() method.

import twitter
api = twitter.Api(consumer_key="ftFL8G4yzQXUVzbUCdxGwKSjJ", consumer_secret="KxGwBe6GlgSYyC7PioIVuZ5tFAsZs7q1rseEYCOnTDIjulT0mZ",
                  access_token_key="314746354-Ucq36TRDnfGAxpOVtnK1qZxMfRKzFHFhyRqzNpTx",
                  access_token_secret="7wZ1qHS0qycy0aNjoMDpKhcfzuLm6uAbhB2LilxZzST8w")

user = "@TutsPlusCode"
friends = api.GetFriends(screen_name=user)
for friend in friends:
    print friend.name

Output

Derek Herman
Cyan Ta'eed
Dropbox
Stoyan Stefanov
The JavaScript Ninja
Dan Wellman
Brenley Dueck
Dave Ward
Packt
Karl Swedberg
Envato Tuts+ Web
Dev_Tips
Vahid Ta'eed
Jarel Remick
Envato Dev. Team
🖥 Drew Douglass 📈
Cameron Moll
SitePoint
John Resig
Skellie
Chris Coyier
Envato Tuts+
Envato Tuts+ Design
Collis

Conclusion

Twitter’s API can be used to a great extent in data analytics. It can also be used in complex big data problems and authenticating apps. Read more about the Twitter API at the Twitter developers site.

Also, don’t hesitate to see what we have available for sale and for study in the marketplace, and don't hesitate to ask any questions and provide your valuable feedback using the feed below.