link back to blog
|Deletions are marked like this.||Additions are marked like this.|
|Line 548:||Line 548:|
|(From [[http://blog.launchpad.net/api/three-tips-faster-clients|the blog]]).
launchpadlib is an open-source Python library that lets you treat the HTTP resources published by Launchpad's web service as Python objects responding to a standard set of commands. With launchpadlib you can integrate your applications into Launchpad without knowing a lot about HTTP client programming.
This document shows how to use a Python client to read and write Launchpad's data using the launchpadlib library. It doesn't cover the HTTP requests and responses that go back and forth behind the scenes: for that, see the "hacking" document. This document also doesn't cover the full range of what's possible with Launchpad's web service: for that, see the web service reference documentation.
Launchpad's web service currently exposes the following major parts of Launchpad:
- People and teams
- Team memberships
- Bugs and bugtasks
- The project registry
- Hosted files, such as bug attachments and mugshots.
As new features and capabilities are added to the web service, you'll be able to access most of them without having to update your copy of launchpadlib. You will have to upgrade launchpadlib to get new client-side features (like support for uploaded files). The Launchpad team will put out an announcement whenever a server-side change means you should upgrade launchpadlib.
The launchpadlib in Ubuntu (easy to install)
If you have the latest version of Ubuntu then to install the launchpadlib available in the Ubuntu repositories, open a terminal and run the command:
$ sudo apt-get install python-launchpadlib
And you are done!
If you have an older version of Ubuntu then parts of the instructions below may not work with the version from the repositories but you should be able to install the latest version of launchpadlib manually.
The latest launchpadlib (more difficult to install)
It is recommended that you do manual installations in a chroot or a virtual environment to keep your main installation stable.
The launchpadlib library depends on quite a few other libraries. All of which you can get from Launchpad.
$ bzr branch lp:oauth $ cd oauth $ sudo ./setup.py install
$ bzr branch lp:lazr.uri $ cd lazr.uri $ sudo ./setup.py install
$ bzr branch lp:lazr.restfulclient $ cd lazr.restfulclient $ sudo ./setup.py install
$ bzr branch lp:wadllib $ cd wadllib $ sudo ./setup.py install
Then do the same for launchpadlib.
$ bzr branch lp:launchpadlib $ cd launchpadlib $ sudo ./setup.py install
The first step towards using Launchpad's web service is to choose a cache directory. The documents you retrieve from Launchpad will be stored here, which will save you a lot of time. Run this code in a Python session, substituting an appropriate directory on your computer:
cachedir = "/home/me/.launchpadlib/cache/"
The next step is to set up credentials for your client. For quick read-only access to Launchpad data, you can get anonymous access. Otherwise, you'll need to authenticate with Launchpad using OAuth.
The Launchpad.login_anonymously() method will give you automatic read-only access to public Launchpad data.
from launchpadlib.launchpad import Launchpad launchpad = Launchpad.login_anonymously('just testing', 'production', cachedir)
The first argument to Launchpad.login_anonymously() is a string that identifies the web service client. We use this string to gauge client popularity and detect buggy or inefficient clients. Here, though, we're just testing.
The second argument tells launchpadlib which Launchpad instance to run against. Here, we're using production, which is mapped to the web service root on the production Launchpad server: "https://api.launchpad.net/beta/". Anonymous access cannot change the Launchpad dataset, so there's no concern about a bad test program accidentally overwriting data. (If you want to play it safe, you can use 'staging' instead.)
The login_anonymously() method automatically negotiates a read-only credential with Launchpad. You can use your Launchpad object right away.
bug_one = launchpad.bugs print bug_one.title # Microsoft has a majority market share
You'll get an error if you try to modify the dataset, access private data, or access objects like launchpad.me which assume a particular user is logged in.
Note that login_anonymously() is only available in launchpadlib starting in version 1.5.4.
To get read-write access to Launchpad, or to see a user's private data, you'll need to get an OAuth credential for that user.
from launchpadlib.launchpad import Launchpad launchpad = Launchpad.login_with('just testing', 'staging', cachedir)
Just like Launchpad.login_anonymously, the first argument to Launchpad.login_with() is a string that identifies the web service client. The second argument identifies the web service root. Here, we're using staging. These examples run against the staging data, so that we don't change real data when we're just trying to see how launchpadlib works.
If this complains that 'staging' isn't a URL, then you're not running the most up-to-date launchpadlib. You can replace 'staging' with launchpadlib.launchpad.STAGING_SERVICE_ROOT to make it work in Ubuntu 9.10's launchpadlib.
Once you're ready to write a launchpadlib program that changes real data, you can replace staging with edge or production.
Call login_with() and you'll see a message like this:
The authorization page (https://staging.launchpad.net/+authorize-token?oauth_token=xxxxxxxxx) should be opening in your browser. After you have authorized this program to access Launchpad on your behalf you should come back here and press <Enter> to finish the authentication process.
Your web browser will open to a page at launchpad.net. You'll be asked to login to Launchpad, and grant some level of access to your new credentials. The level of access you choose will determine how much you can do through launchpadlib with these credentials. This lets your users delegate a portion of their Launchpad permissions to your program, without having to trust it completely.
Once you grant access, hit Enter within your Python session. You'll get back a Launchpad object. Now you can access the web service.
Here's some code that retrieves a bug from Launchpad and prints its title.
bug_one = launchpad.bugs print bug_one.title # Microsoft has a majority market share
Launchpad.login_with() cached the credentials, so we can run the script again and not have to log in. That's the cache at work. Note that the cache isn't thread-safe. If you've got multiple threads or processes running at once, give each one its own cache directory.
If you don't know the capabilities of one of the objects you've got, you can call dir() on it. You'll see all of its fields and all the custom methods it supports. Unfortunately, you'll also see a bunch of launchpadlib-specific junk that you don't care about. That's why we've made available these four lists:
lp_attributes: Data fields of this object. You can read from these might be able to write to some of them.
lp_collections: List of launchpad objects associated with this object.
lp_entries: Other Launchpad objects associated with this one.
lp_operations: The names of Launchpad methods you can call on the object.
print sorted(bug_one.lp_attributes) # ['date_created', 'date_last_message', 'date_last_updated', ... 'tags', 'title'] print sorted(bug_one.lp_operations) # ['addAttachment', 'addWatch', 'subscribe', 'unsubscribe']
If you need more detailed help, you can look the object up in the reference documentation. First, find out the type of the object.
print repr(bug_one) # <bug at https://api.staging.launchpad.net/beta/bugs/1>
This is a 'bug' type object. Now you use the type of the object as an anchor into the reference documentation. To find out the capabilities of this object and what data is stored inside it, you'd visit https://launchpad.net/+apidoc#bug.
As you'll see, the reference documentation still needs some work, and it's geared more towards web service hackers than launchpadlib users, but it will tell you about all of this object's attributes and all the supported operations.
- The "Default representation" section tells you about the available attributes.
- The "Custom POST methods" and "Custom GET methods" sections tell you about methods the object supports other than the default methods described below. The methods take whatever parameters are listed in "Request query parameters". (You can ignore the "ws.op" parameter because you're using launchpadlib; that's just the name of the method.)
The top-level objects
The Launchpad object has attributes corresponding to the major parts of Launchpad. These are:
.bugs: All the bugs in Launchpad
.people: All the people in Launchpad
.distributions: All the distributions in Launchpad
.projects: All the projects in Launchpad
.project_groups: All the project groups in Launchpad
As a super special secret, distributions, projects and project_groups are all actually the same thing.
me = launchpad.me print me.name # This should be your user name, e.g. 'salgado'
The launchpad.people attribute gives you access to other people who use Launchpad. This code uses launchpad.people to look up the person with the Launchpad name "salgado".
people = launchpad.people salgado = people['salgado'] print salgado.display_name # Guilherme Salgado
You can search for objects in other ways. Here's another way of finding "salgado".
salgado = people.getByEmail(email="firstname.lastname@example.org") print salgado.display_name # Guilherme Salgado
Some searches return more than one object.
for person in people.find(text="salgado"): print person.name # agustin-salgado # ariel-salgado # axel-salgado # bruno-salgado # camilosalgado # ...
Note that, unlike typical Python methods, these methods--find() and getByEmail()--don't support positional arguments, only keyword arguments. You can't call people.find("salgado"); it has to be people.find(text="salgado").
Bugs, people, projects, team memberships, and most other objects published through Launchpad's web service, all work pretty much the same way. We call all these objects "entries". Each corresponds to a single piece of data within Launchpad.
You can use the web service to discover various facts about an entry. The launchpadlib makes the facts available as attributes of the entry object.
name and display_name are facts about people.
print salgado.name # salgado print salgado.display_name # Guilherme Salgado
private and description are facts about bugs.
print bug_one.private # False print bug_one.description # Microsoft has a majority market share in the new desktop PC marketplace. # This is a bug, which Ubuntu is designed to fix. # ...
Every entry has a self_link attribute. You can treat this as a permanent ID for the entry. If your program needs to keep track of Launchpad objects across multiple runs, a simple way to do it is to keep track of the self_links.
print salgado.self_link # https://api.staging.launchpad.net/beta/~salgado bug_one.self_link # https://api.staging.launchpad.net/beta/bugs/1
Some of an object's attributes are links to other entries. Bugs have an attribute owner, but the owner of a bug is a person, with attributes of its own.
owner = bug_one.owner print repr(owner) # <person at https://api.staging.launchpad.net/beta/~sabdfl> print owner.name # sabdfl print owner.display_name # Mark Shuttleworth
If you have permission, you can change an entry's attributes and write the data back to the server using lp_save().
me = people['my-user-name'] me.display_name = 'A user who edits through the Launchpad web service.' me.lp_save() print people['my-user-name'].display_name # A user who edits through the Launchpad web service.
Having permission means not only being authorized to perform an operation on the Launchpad side, but using a launchpadlib Credentials object that authorizes the operation. If you've set up your launchpadlib Credentials for read-only access, you won't be able to change data through launchpadlib.
Some entries also support special operations--see the reference documentation for details. A bugtask entry supports an operation called transitionToAssignee. This operation takes a single argument called assignee, which should be a Launchpad person. Here it is in action.
task = list(bug_one.bug_tasks) old_assignee = task.assignee print old_assignee # <team at https://api.staging.launchpad.net/beta/~compscibuntu-bugs> task.transitionToAssignee(assignee=me) print task.owner.display_name # A user who edits through the Launchpad web service.
Entries can support special operations just like collections, but again note that, these methods don't support positional arguments, only keyword arguments.
When the Launchpad web service encounters an error, it sends back an error message to launchpadlib, which raises an HTTPError exception. You'll see information about the HTTP request that caused the error, and the server-side error message. Depending on the error, you may be able to recover or change your code and try again.
If you're using an old version of launchpadlib, the HTTPError may not be this helpful. To see the server-side error message, you'll need to print out the .content of the HTTPError exception.
When Launchpad groups similar entries together, we call it a collection. You've already seen one collection: the list of people you get back when you call launchpad.people.find.
for person in launchpad.people.find(text="salgado"): print person.name
That's a collection of people-type entries. You can iterate over a collection as you can any Python list.
Some of an entry's attributes are links to related collections. Bug #1 has a number of associated bug tasks, represented as a collection of 'bug task' entries.
tasks = bug_one.bug_tasks print len(tasks) # 17 for task in tasks: print task.bug_target_display_name # Computer Science Ubuntu # Ichthux # JAK LINUX # ...
The person 'salgado' understands two languages, represented here as a collection of two language entries.
for language in salgado.languages: print language.self_link # https://api.staging.launchpad.net/beta/+languages/en # https://api.staging.launchpad.net/beta/+languages/pt_BR
Because collections can be very large, it's usually a bad idea to iterate over them. Bugs generally have a manageable number of bug tasks, and people understand a manageable number of languages, but Launchpad tracks over 250,000 bugs. If you just iterate over a list, launchpadlib will just keep pulling down entries until it runs out, which might be forever (or, realistically, until your client is banned for making too many requests).
That's why we recommend you slice Launchpad's collections into Python lists, and operate on the lists. Here's code that prints descriptions for the 10 most recently filed bugs.
bugs = launchpad.bugs[:10] for bug in bugs: print bug.description
For performance reasons, we've put a couple restrictions on collection slices that don't apply to slices on regular Python lists. You can only slice from the beginning of a collection, not the end.
launchpad.bugs[-5:] # *** ValueError: Collection slices must have a nonnegative start point.
And your slice needs to have a definite end point: you can't slice to the end of a collection.
bugs[10:] # *** ValueError: Collection slices must have a definite, nonnegative end point. bugs[:-5] # *** ValueError: Collection slices must have a definite, nonnegative end point.
On the plus side, you can include a step number with your slice, as with a normal Python list:
every_other_bug = launchpad.bugs[0:10:2] len(every_other_bug) # 5
Launchpad stores some data in the form of binary files. A good example is people's mugshots. With launchpadlib, you can read and write these binary files programatically.
If you have a launchpadlib reference to one of these hosted files, you can read its data by calling the open() method and treating the result as an open filehandle.
mugshot = launchpad.me.mugshot mugshot_handle = mugshot.open() mugshot_handle.read() # [binary data] mugshot_handle.content_type # 'image/jpeg' mugshot_handle.last_modified # 'Wed, 12 Mar 2008 21:47:05 GMT'
You'll get an error if the file doesn't exist: for instance, if a person doesn't have a mugshot.
launchpad.people['has-no-mugshot'].mugshot # *** HTTPError: HTTP Error 404: Not Found
To create or overwrite a file, open the hosted file object for write. You'll need to provide the access mode ("w"), the MIME type of the file you're sending to Launchpad, and the filename you want to give it on the server side.
mugshot_handle = mugshot.open("w", "image/jpeg", "my-image.jpg") mugshot_handle.write("image data goes here") mugshot_handle.close()
If there's something wrong--maybe you provide a file of the wrong type--you'll get an HTTPError with a status code of 400. The content attribute will contain an error message.
print http_error.content # This image is not exactly 192x192 pixels in size. print http_error.content # The file uploaded was not recognized as an image; please # check it and retry.
Persistent references to Launchpad objects
Every entry and collection has a unique ID: its URL. You can get this unique ID by calling str() on the object.
print str(bug_one) # https://api.staging.launchpad.net/beta/bugs/1
If you need to keep track of Launchpad objects over time, or pass references to Launchpad objects to other programs, use these strings. If you've got one of these strings, you can turn it into the corresponding Launchpad object by calling launchpad.load().
bug_one = launchpad.load("https://api.staging.launchpad.net/beta/bugs/1") print bug_one.title Microsoft has a majority market share
You're bookmarking the Launchpad objects and coming back to them later, just like you'd bookmark pages in your web browser.
Three things to make your client faster
1. Use the latest launchpadlib. (The versions in the current Ubuntu release should be fine; otherwise run from the branch or the latest tarball.)
import httplib2 httplib2.debuglevel = 1
3. Fetch objects only once:
Don't do this:
if bug.person is not None: print bug.person.name
p = bug.person if p is not None: print p.name
(From the blog).
launchpadlib still has deficiencies. We track bugs in the launchpadlib bug tracker (https://bugs.launchpad.net/launchpadlib) and will be working to improve launchpadlib throughout the limited beta.
web service reference documentation for a list of all objects, operations, etc