Diff for "Code/Git"

Not logged in - Log In / Register

Differences between revisions 8 and 31 (spanning 23 versions)
Revision 8 as of 2015-04-30 23:46:39
Size: 6431
Editor: cjwatson
Comment: some MP bugs fixed
Revision 31 as of 2019-01-10 20:09:47
Size: 12992
Editor: cjwatson
Comment: permissions
Deletions are marked like this. Additions are marked like this.
Line 7: Line 7:
Launchpad features experimental support for hosting [[http://git-scm.com/|Git]] repositories. This is distinct from the [[Code/Imports|code import]] facility that Launchpad has included for many years; it will shortly be possible to host Git repositories on Launchpad directly. Launchpad supports hosting [[http://git-scm.com/|Git]] repositories. This is distinct from the [[Code/Imports|code import]] facility that Launchpad has included for many years; it is now possible to host Git repositories on Launchpad directly.
Line 13: Line 13:
As of the end of April 2015, a reasonably complete basic version of Git hosting support is live on our "qastaging" site. This means: This summary is up-to-date as of November 2016.
Line 15: Line 15:
 * all changes made will be reset - this is not yet a persistent hosting service, but is strictly for experimenting and finding bugs in Launchpad
 * the URL format is not guaranteed to be stable
 * everything is on qastaging.launchpad.net and git.qastaging.paddev.net, rather than on the main launchpad.net site
 * the hosting backend is still quite basic and may have scaling problems
Launchpad supports Git hosting. This means that you can:
Line 20: Line 17:
Within these constraints, you can:

* push Git repositories to qastaging over SSH
 * push Git repositories over SSH
Line 24: Line 19:
 * see summary information on repositories and the branches they contain in the Launchpad web UI on qastaging  * see summary information on repositories and the branches they contain in the Launchpad web UI
Line 28: Line 23:

Just don't keep the only copy of your repository on qastaging!
 * link Launchpad bugs to merge proposals
 * add [[API/Webhooks|webhooks]] to notify third-party services when repositories are changed
 * use [[Packaging/SourceBuilds|recipes]] to build packages in PPAs for code in Launchpad-hosted Git repositories
 * mirror repositories from other sites
Line 37: Line 34:
 * mirroring
 * webhooks
Line 41: Line 36:
Our top priority is to polish things up so that this can land on Launchpad production, and to this end we appreciate your feedback on any bugs you find. Here's a short list of known bugs that you don't need to tell us about: Here's a short list of known bugs that you don't need to tell us about:
Line 43: Line 38:
 * working with default repositories is harder than it should be (they don't appear on the target's code page, so you have to guess their URL)
 * there is no web listing of repositories available for a given target
Line 46: Line 39:
 * it is not yet possible to browse private repositories using the cgit interface, as it does not yet have SSO integration
Line 51: Line 43:
[url "git+ssh://USER@git.qastaging.paddev.net/"]
        insteadof = lpqas:
[url "git+ssh://USER@git.launchpad.net/"]
        insteadof = lp:
Line 54: Line 46:
This allows you to type `git clone lpqas:REPOSITORY` instead of `git clone git+ssh://git.qastaging.paddev.net/REPOSITORY`.  (Of course, when we add Git support to production we will update this documentation to recommend `lp:` instead.) This allows you to type `git clone lp:REPOSITORY` instead of `git clone git+ssh://git.launchpad.net/REPOSITORY`.
Line 57: Line 49:

You should check the [[SSHFingerprints|fingerprint]] of git.launchpad.net when prompted to do so by SSH.
Line 61: Line 55:
$ git clone lpqas:PROJECT $ git clone lp:PROJECT
Line 64: Line 58:
For example, `git clone lpqas:launchpad` fetches Launchpad itself. For example, `git clone lp:launchpad` fetches Launchpad itself (or will once we've finished converting it to Git!).
Line 73: Line 67:
$ git remote add lpqas lpqas:PROJECT $ git remote add origin lp:PROJECT
Line 76: Line 70:
Or like this (where `USER` is your Launchpad username), if you do not own the project: {{{
$ git remote add lpqas lpqas:~USER/PROJECT
Or like this (where `USER` is your Launchpad username), if you do not own the project but want to contribute to it: {{{
$ git remote add origin lp:~USER/PROJECT
}}}

Or to push a repository that isn't part of any Launchpad project or package, e.g. an ad-hoc experiment: {{{
$ git remote add origin lp:~USER/+git/REPOSITORY-NAME
Line 81: Line 79:
$ git push lpqas my-changes $ git push origin my-changes
Line 83: Line 81:

== Permissions ==

By default, repository owners may create, push, force-push, or delete any branch or tag in their repositories, and nobody else may modify them in any way.

Repository owners can use the "Manage permissions" page of a repository to change this by protecting branches or tags. By default, protecting a branch implicitly prevents repository owners from force-pushing to it or deleting it, while protecting a tag prevents repository owners from moving it. You can modify these default permission grants to be more restrictive (for example, you might prevent anyone from pushing to an archived branch), or to grant other permissions (for example, you might want to allow a contributor to push to certain branches in your repository without owning it).

You may create rules that match a single branch or tag, or wildcard rules that match a pattern: for example, `*` matches everything, while `stable/*` matches `stable/1.0` but not `master`.

Any owner of a repository can change its permissions, so restricting permissions for other repository owners is not a strict security barrier. However, you can use the "View activity" page of a repository to see what permission changes have been made.

Launchpad works out the effective permissions that a user has on a protected branch or tag as follows:

 1. Take all the rules that match the branch or tag.
 2. For each matching rule, select any grants whose grantee matches the user, as long as the same grantee has not already been seen in an earlier matching rule. (A user can be matched by more than one grantee: for example, they might be in multiple teams.)
 3. If the user is an owner of the repository and there was no previous "Repository owner" grant, then add an implicit grant allowing them to create or push.
 4. The effective permission set is the union of the permissions granted by all the selected grants.
Line 88: Line 103:
  https://code.qastaging.launchpad.net/~OWNER/PROJECT/+git/REPOSITORY:: This identifies a repository for an upstream project.
  https://code.qastaging.launchpad.net/~OWNER/DISTRIBUTION/+source/SOURCE/+git/REPOSITORY:: This identifies a repository for a source package in a distribution.
  https://code.qastaging.launchpad.net/~OWNER/+git/REPOSITORY:: This identifies a "personal" repository with no particular connection to any project or package.
  https://code.launchpad.net/~OWNER/PROJECT/+git/REPOSITORY:: This identifies a repository for an upstream project.
  https://code.launchpad.net/~OWNER/DISTRIBUTION/+source/SOURCE/+git/REPOSITORY:: This identifies a repository for a source package in a distribution.
  https://code.launchpad.net/~OWNER/+git/REPOSITORY:: This identifies a "personal" repository with no particular connection to any project or package (like "+junk" in Launchpad's Bazaar code hosting).
Line 94: Line 109:
  https://code.qastaging.launchpad.net/PROJECT:: This is the default repository for an upstream project.
  https://code.qastaging.launchpad.net/DISTRIBUTION/+source/SOURCE:: This is the default repository for a source package in a distribution.
  https://code.launchpad.net/PROJECT:: This is the default repository for an upstream project.
  https://code.launchpad.net/DISTRIBUTION/+source/SOURCE:: This is the default repository for a source package in a distribution.
Line 99: Line 114:
  https://code.qastaging.launchpad.net/~OWNER/PROJECT:: This is an owner's default repository for an upstream project.
  https://code.qastaging.launchpad.net/~OWNER/DISTRIBUTION/+source/SOURCE:: This is an owner's default repository for a source package in a distribution.
  https://code.launchpad.net/~OWNER/PROJECT:: This is an owner's default repository for an upstream project.
  https://code.launchpad.net/~OWNER/DISTRIBUTION/+source/SOURCE:: This is an owner's default repository for a source package in a distribution.
Line 103: Line 118:

= Linking to bugs =

Git-based merge proposals can be linked to Launchpad bugs. This can be done manually from the web UI for the merge proposal, but normally you should just mention the Launchpad bug in the commit message of one of the commits you want to merge. The required commit message text to link to bugs #XXX and #YYY looks like this: {{{
LP: #XXX, #YYY
}}}

Technically, the commit message needs to match this regular expression, case-insensitively: {{{
/lp:\s+\#\d+(?:,\s*\#\d+)*/
}}} This is the same pattern used to match Launchpad bug references in `debian/changelog` files in source packages.

Bugs are not automatically closed when merge proposals land, because the policy for when that should happen varies from project to project: for example, projects often only close bugs when they make releases, or when their code is deployed to production sites.

Users familiar with Bazaar on Launchpad should note that the model for Git bug linking is slightly different: bugs are linked to merge proposals rather than to individual branches. This difference is mainly because individual branches within a Git repository are often much more ephemeral than Bazaar branches.

If you need a more advanced bug-handling workflow for your project, you can use a [[API/Webhooks|webhook]] to help. See [[https://git.launchpad.net/kicad-git-hook|kicad-git-hook]] for an example contributed by a Launchpad user.

= Mirroring repositories from other sites =

You can tell Launchpad to create a repository which is imported from some other site. There are two ways to set this up.

 1. This method is preferred in the common case of importing the upstream repository for a project.
  * Go to the main page in Launchpad for a project you maintain, and follow the "Code" link under "Configuration options".
  * Set "Version control system" to "Git" if necessary.
  * Select "Import a Git repository hosted somewhere else".
  * Fill in the repository name (this should normally just be the project name).
  * Set the repository owner if necessary (defaults to you, can be any public team you participate in).
  * Fill in the URL of the remote repository.
  * Launchpad will create the repository, set it as the default for your project, and schedule an import.
 1. This method is useful for other cases, such as importing repositories that are not the primary upstream repository for a project.
  * Go to the "[[https://code.launchpad.net/+code-imports/+new|Request a code import]]" page.
  * Select "Git" for both the version control system and the target version control system.
  * Fill in the other details as above.
  * Launchpad will create the repository and schedule an import, but in this case it will '''not''' set it as the default for your project.

In either case, Launchpad will mirror the whole repository from the remote site, and will keep its copy up to date regularly. You won't be able to push directly to the imported repository on Launchpad, but you can create another repository in the same project and push branches to that, and even create merge proposals if you want (though you may have to tell the upstream maintainer about them separately!). You can create source package recipes or snap packages based on branches in the imported repository.

= Converting from Bazaar to Git =

There [[https://www.google.com/search?{google:acceptedSuggestion}oq=convert&sourceid=chrome&client=ubuntu&channel=cs&ie=UTF-8&q=convert+from+bzr+to+git|are]] [[https://astrofloyd.wordpress.com/2012/09/06/convert-bzr-to-git/|several]] [[http://www.fusonic.net/en/blog/2013/03/26/migrating-from-bazaar-to-git/|useful]] [[http://blog.timmattison.com/archives/2011/06/13/how-to-convert-from-bzr-to-git-on-debianubuntu/|recommendations]] online for how to convert from Bazaar to Git. Here's one way that preserves tags and does a pretty good job for relatively simple Bazaar branches.

{{{
$ cd /some/place # parent directory of Bazaar branch
$ mkdir new-git-repo
$ cd new-git-repo
$ git init .
$ bzr fast-export --export-marks=../marks.bzr ../old-bzr-branch | git fast-import --export-marks=../marks.git
$ git checkout master
}}}

Now the `new-git-repo` directory is a Git repository with history equivalent to your old Bazaar branch. You should push it somewhere, and to ensure that everything is correct you should re-clone it locally to whatever final destination path you want to work in.

If you have several different Bazaar branches that form part of the same project, or if your Bazaar branches constitute packaging for a project whose upstream is in revision control elsewhere, then you may well want to do a more careful conversion. For this, [[http://www.catb.org/~esr/reposurgeon/|reposurgeon]] is an excellent tool: it gives you a language for describing the transformations you want to make to your input branches, so you can run the migration several times with different tweaks before deciding that the result is the one you want to publish to the world.

Once you're ready to use Git by default for your project, you can configure this from https://launchpad.net/PROJECT/+configure-code (which is linked from the "Configuration Progress" section of the main project page on Launchpad).

Launchpad Help > Code > Git

Overview

Launchpad supports hosting Git repositories. This is distinct from the code import facility that Launchpad has included for many years; it is now possible to host Git repositories on Launchpad directly.

Git repositories use a somewhat different model from Bazaar branches: operations such as cloning happen at the level of a repository, but it is common for a single repository to contain many branches. This means that the Launchpad interface needs to be laid out somewhat differently to support that.

What's supported?

This summary is up-to-date as of November 2016.

Launchpad supports Git hosting. This means that you can:

  • push Git repositories over SSH
  • clone repositories over git://, SSH, or HTTPS
  • see summary information on repositories and the branches they contain in the Launchpad web UI
  • follow links from the Launchpad web UI to a full-featured code browser
  • push and clone private repositories, if you have a commercial subscription to Launchpad
  • propose merges from one branch to another, including in a different repository, provided that they are against the same project or package
  • link Launchpad bugs to merge proposals
  • add webhooks to notify third-party services when repositories are changed

  • use recipes to build packages in PPAs for code in Launchpad-hosted Git repositories

  • mirror repositories from other sites

What will be supported?

Launchpad's Bazaar support has grown many features over the years, and it will take some time to bring our Git support up to full parity with it. Here's an incomplete list of some of the features we expect to add:

  • useful subscriptions (currently only attribute change notifications work, which are not usually very interesting in themselves)
  • RSS feeds
  • an integrated code browser

Here's a short list of known bugs that you don't need to tell us about:

  • the interface for registering merge proposals is very rough

Configuring Git

Git identifies repositories using URLs. Unlike Bazaar, there is no built-in abbreviation for repositories hosted on Launchpad, but it is very easy to add such a thing yourself. Edit ~/.gitconfig and add these lines, where USER is your Launchpad username:

[url "git+ssh://USER@git.launchpad.net/"]
        insteadof = lp:

This allows you to type git clone lp:REPOSITORY instead of git clone git+ssh://git.launchpad.net/REPOSITORY.

The rest of this documentation assumes that you have configured Git this way.

You should check the fingerprint of git.launchpad.net when prompted to do so by SSH.

Getting code

You can fetch the default repository for a project like this:

$ git clone lp:PROJECT

For example, git clone lp:launchpad fetches Launchpad itself (or will once we've finished converting it to Git!).

To keep your local clone up to date, run:

$ git pull

Pushing code

You can add a "remote" to your repository like this, if you own the project:

$ git remote add origin lp:PROJECT

Or like this (where USER is your Launchpad username), if you do not own the project but want to contribute to it:

$ git remote add origin lp:~USER/PROJECT

Or to push a repository that isn't part of any Launchpad project or package, e.g. an ad-hoc experiment:

$ git remote add origin lp:~USER/+git/REPOSITORY-NAME

Now, you can push a branch using a command such as this:

$ git push origin my-changes

Permissions

By default, repository owners may create, push, force-push, or delete any branch or tag in their repositories, and nobody else may modify them in any way.

Repository owners can use the "Manage permissions" page of a repository to change this by protecting branches or tags. By default, protecting a branch implicitly prevents repository owners from force-pushing to it or deleting it, while protecting a tag prevents repository owners from moving it. You can modify these default permission grants to be more restrictive (for example, you might prevent anyone from pushing to an archived branch), or to grant other permissions (for example, you might want to allow a contributor to push to certain branches in your repository without owning it).

You may create rules that match a single branch or tag, or wildcard rules that match a pattern: for example, * matches everything, while stable/* matches stable/1.0 but not master.

Any owner of a repository can change its permissions, so restricting permissions for other repository owners is not a strict security barrier. However, you can use the "View activity" page of a repository to see what permission changes have been made.

Launchpad works out the effective permissions that a user has on a protected branch or tag as follows:

  1. Take all the rules that match the branch or tag.
  2. For each matching rule, select any grants whose grantee matches the user, as long as the same grantee has not already been seen in an earlier matching rule. (A user can be matched by more than one grantee: for example, they might be in multiple teams.)
  3. If the user is an owner of the repository and there was no previous "Repository owner" grant, then add an implicit grant allowing them to create or push.
  4. The effective permission set is the union of the permissions granted by all the selected grants.

Repository URLs

Every Git repository hosted on Launchpad has a full "canonical" URL of one of these forms (these are the versions you'd use in a web browser; you only need to change the scheme and host parts for the command-line Git client):

https://code.launchpad.net/~OWNER/PROJECT/+git/REPOSITORY
This identifies a repository for an upstream project.
https://code.launchpad.net/~OWNER/DISTRIBUTION/+source/SOURCE/+git/REPOSITORY
This identifies a repository for a source package in a distribution.
https://code.launchpad.net/~OWNER/+git/REPOSITORY
This identifies a "personal" repository with no particular connection to any project or package (like "+junk" in Launchpad's Bazaar code hosting).

These are unique, but can involve quite a lot of typing, and in most cases there's no need for more than one repository per owner and target (project or package). Launchpad therefore has the notion of "default repositories". A repository can be the default for a target, in which case it has one of these forms:

https://code.launchpad.net/PROJECT
This is the default repository for an upstream project.
https://code.launchpad.net/DISTRIBUTION/+source/SOURCE
This is the default repository for a source package in a distribution.

Or a repository can be a person's or a team's default for a target, in which case it has one of these forms:

https://code.launchpad.net/~OWNER/PROJECT
This is an owner's default repository for an upstream project.
https://code.launchpad.net/~OWNER/DISTRIBUTION/+source/SOURCE
This is an owner's default repository for a source package in a distribution.

We expect that projects hosting their code on Launchpad will normally have their primary repository set as the default for the project, and contributors will normally push to branches in owner-default repositories. The extra flexibility with named repositories allows for situations such as separate private repositories containing embargoed security fixes.

Linking to bugs

Git-based merge proposals can be linked to Launchpad bugs. This can be done manually from the web UI for the merge proposal, but normally you should just mention the Launchpad bug in the commit message of one of the commits you want to merge. The required commit message text to link to bugs #XXX and #YYY looks like this:

LP: #XXX, #YYY

Technically, the commit message needs to match this regular expression, case-insensitively:

/lp:\s+\#\d+(?:,\s*\#\d+)*/

This is the same pattern used to match Launchpad bug references in debian/changelog files in source packages.

Bugs are not automatically closed when merge proposals land, because the policy for when that should happen varies from project to project: for example, projects often only close bugs when they make releases, or when their code is deployed to production sites.

Users familiar with Bazaar on Launchpad should note that the model for Git bug linking is slightly different: bugs are linked to merge proposals rather than to individual branches. This difference is mainly because individual branches within a Git repository are often much more ephemeral than Bazaar branches.

If you need a more advanced bug-handling workflow for your project, you can use a webhook to help. See kicad-git-hook for an example contributed by a Launchpad user.

Mirroring repositories from other sites

You can tell Launchpad to create a repository which is imported from some other site. There are two ways to set this up.

  1. This method is preferred in the common case of importing the upstream repository for a project.
    • Go to the main page in Launchpad for a project you maintain, and follow the "Code" link under "Configuration options".
    • Set "Version control system" to "Git" if necessary.
    • Select "Import a Git repository hosted somewhere else".
    • Fill in the repository name (this should normally just be the project name).
    • Set the repository owner if necessary (defaults to you, can be any public team you participate in).
    • Fill in the URL of the remote repository.
    • Launchpad will create the repository, set it as the default for your project, and schedule an import.
  2. This method is useful for other cases, such as importing repositories that are not the primary upstream repository for a project.
    • Go to the "Request a code import" page.

    • Select "Git" for both the version control system and the target version control system.
    • Fill in the other details as above.
    • Launchpad will create the repository and schedule an import, but in this case it will not set it as the default for your project.

In either case, Launchpad will mirror the whole repository from the remote site, and will keep its copy up to date regularly. You won't be able to push directly to the imported repository on Launchpad, but you can create another repository in the same project and push branches to that, and even create merge proposals if you want (though you may have to tell the upstream maintainer about them separately!). You can create source package recipes or snap packages based on branches in the imported repository.

Converting from Bazaar to Git

There are several useful recommendations online for how to convert from Bazaar to Git. Here's one way that preserves tags and does a pretty good job for relatively simple Bazaar branches.

$ cd /some/place  # parent directory of Bazaar branch
$ mkdir new-git-repo
$ cd new-git-repo
$ git init .
$ bzr fast-export --export-marks=../marks.bzr ../old-bzr-branch | git fast-import --export-marks=../marks.git
$ git checkout master

Now the new-git-repo directory is a Git repository with history equivalent to your old Bazaar branch. You should push it somewhere, and to ensure that everything is correct you should re-clone it locally to whatever final destination path you want to work in.

If you have several different Bazaar branches that form part of the same project, or if your Bazaar branches constitute packaging for a project whose upstream is in revision control elsewhere, then you may well want to do a more careful conversion. For this, reposurgeon is an excellent tool: it gives you a language for describing the transformations you want to make to your input branches, so you can run the migration several times with different tweaks before deciding that the result is the one you want to publish to the world.

Once you're ready to use Git by default for your project, you can configure this from https://launchpad.net/PROJECT/+configure-code (which is linked from the "Configuration Progress" section of the main project page on Launchpad).

Code/Git (last edited 2019-01-10 20:09:47 by cjwatson)