Diff for "Packaging/SourceBuilds/Recipes"

Not logged in - Log In / Register

Differences between revisions 1 and 25 (spanning 24 versions)
Revision 1 as of 2010-07-07 19:06:23
Size: 5155
Editor: abentley
Comment:
Revision 25 as of 2021-06-07 13:41:17
Size: 12122
Editor: cjwatson
Comment: attempt to clarify distinction between short name and git branch name
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
~-[[FrontPage|Launchpad Help]] > [[Packaging]] > [[Packaging/SourceBuilds|Daily builds]] > Recipes -~
Line 2: Line 4:

Line 4: Line 8:
= General information =
"recipes" are descriptions of the steps needed to construct a package from the various bzr branches. It relies on a certain format that specifies
 * where to use the code from (trunk branch, beta branch, etc.), where to get the packaging from (separate branch? ubuntu branch?)
 * the correct package version (so users will still be able to upgrade to the stable version of the distro once it gets released)
 * what to modify to make the source build properly

In this first step, just write it in a text editor. Later on, you'll test it locally, then enable it in Launchpad.
= Overview =

A "recipe" is a description of the steps needed to construct a package from a set of Bazaar or Git branches. Its format specifies:

 * Which branch to use for the source code: trunk branch, beta branch, etc.
 * Which branch to use for the packaging information: separate branch, Ubuntu branch, etc.
 * The correct package version (so people can still upgrade to the new stable version when it's released.
 * What to modify to make the source build properly.

The same recipe format is used for bzr via bzr-builder and git via git-build-recipe.
Line 14: Line 21:
The start of the file always looks similar to {{{
# bzr-builder format 0.2 deb-version 1.0+{time}
}}}

The `# bzr-builder format 0.2` is the same for each, and just specifies the version of the format in use. The current format is `0.2`, and is increased as the format changes.

`deb-version 1.0+{time}` specifies the version to use for the generated package. `{time}` here is a substitution variable, more information on those will be given later.

The next line specifies the branch to base the package on: {{{
Recipes have a simple format.

'''Note:''' you can find a typical example in the [[Packaging/SourceBuilds/GettingStarted|getting started]] guide.

They always start with a line similar to this:

{{{
# bzr-builder format 0.3 deb-version 1.0+{revno}
}}}

Let's take a look at this in more detail:

 * ```# bzr-builder format 0.3``` specifies which recipe format we're using. The current format is 0.4.
 * If you're using git, this will be ```# git-build-recipe format 0.4``` instead.
 * ```deb-version 1.0+{revno}``` specifies the version to give the package we're building. ```{revno}``` is a substitution variable; more on which later.
 
== Specifying the branches ==
 
The next line of a recipe specifies which branch to base the package on:

{{{
Line 26: Line 45:
This says that we will use the trunk of the `bzr` project.

Then there is any number of other lines to specify other branches to include. The usual way to do this is to use the `merge` keyword to specify a simple merge of the branch. {{{
This says that we will use the trunk of the ```bzr``` project in Launchpad. This could just as easily be any other branch in Launchpad, using the short format that you can find on any branch overview page.

If you're using git, then the format is similar, but you should normally provide a branch name as a revision specifier (if you don't, then the recipe builder will assume HEAD):

{{{
lp:germinate master
}}}

Note that if you've converted a project from bzr to git, then the "lp:PROJECT" alias for the project's default branch may still be configured for bzr, and will currently take precedence over the git default repository. You can always disambiguate like this:

{{{
https://git.launchpad.net/germinate master
}}}

Next, you can specify any number of other branches to include. There are two ways to include those branches additional branches:

 * merge: this specifies a simple ```bzr merge``` of the two branches.
 * nest: inserts the content of the second branch into a specific location within the main branch.

=== Merging ===

|| `merge SHORT-NAME URL [REVISION]` ||

Most often you'll use the "merge" command:

{{{
Line 32: Line 74:
`fix-build` is the id of the branch within this file, it doesn't have to be the same as any part of the branch URL, but it has to be unique within the file. `lp:~bzr/bzr/fix-build` is again the location of the branch. What we are doing here is merging in a branch that we know we need in order to fix the build, but that hasn't landed
in `lp:bzr` yet.

The other way to include code is to use the `nest` keyword. {{{
Here ```fix-build``` is a unique short name that we'll use to refer to this branch in substitution variables. The short name can be anything you like, so long as it is unique to this branch within this recipe.

```lp:~bzr/bzr/fix-build``` is the location of the branch.

In this example, the branch ''fix-build'' fixes a problem in the trunk that prevents it from building. This branch could be anything: stand-alone packaging information, some other modification to the branch that's not yet present in the trunk and so on.

If you're using git, then the format is similar, but you should normally provide a branch name as a revision specifier:

{{{
merge fix-build lp:~contributor/germinate fix-build
}}}

The second `fix-build` here is something that identifies a commit, usually a ref (branch or tag) name. Be careful not to confuse this with the short name used in substitution variables; you could equally well use the following and only have to adjust some variable references:

{{{
merge some-nonsense lp:~contributor/germinate fix-build
}}}

=== Nesting ===

|| `nest SHORT-NAME URL TARGET-DIRECTORY [REVISION]` ||

Nesting works in a similar way but has more scope:

{{{
Line 39: Line 102:
The `nest` keyword will get the contents of one branch put inside another branch, instead of merging it. In this case we are nesting `lp:pyfoo` in to `lp:bzr`. We are using `pyfoo` to refer to it in the file, and we want it nested in the `foo` directory, so we specify that last.

It is possible to act on the nested branch in the same way as with the main branch. Any lines that are indented by two spaces under a `nest` line will act on the nested branch, e.g. {{{
The ```nest``` keyword puts the contents of one branch into a specific location in another branch, instead of merging it.

In this case, we are nesting the contents of ```lp:pyfoo``` in a new ```foo``` directory in the ```lp:bzr``` branch. Again, we've given the branch a short name, ```pyfoo```, that we can use to refer to it in substitution variables.

You can also act on the nested branch in the same way as you can the main branch: you can merge and nest other branches in your nested branch.

Here's how:

{{{
nest pyfoo lp:pyfoo foo
Line 45: Line 115:
would merge the `ubuntu-branding` branch in to `foo`.

The resulting branch needs to already have the `debian` directory in place with the packaging in it, as it can't be auto-generated. Therefore you will often need to merge one or more packaging branches: {{{
merge packaging lp:~bzr/bzr/packaging
}}}

In total this recipe would look like {{{
# bzr-builder format 0.2 deb-version 1.0+{time}
If you're using git, then the format is similar, but you should normally provide a branch name as a revision specifier:

{{{
nest pyfoo lp:pyfoo foo master
  merge branding lp:~bob/pyfoo ubuntu-branding
}}}

Be careful not to confuse the short name used for substitution variables (here, `branding`) with the git branch name (here, `ubuntu-branding`).

Any lines that are indented by two spaces, and are directly below your ```nest``` line, will act on the nested branch. In this example, the ```ubuntu-branding``` branch will be merged into ```pyfoo``` before it is nested in your primary branch.

==== nest-part ====

|| `nest-part SHORT-NAME URL SOURCE-DIRECTORY [TARGET-DIRECTORY [REVISION]]` ||

If you want to nest only one directory from another branch, you can use ```nest-part```. It works in the same way as ```nest```, except that you specify which directory you're taking from the nested branch.

For example:

{{{
nest-part packaging lp:~some-person/some-project/trunk-with-packaging debian debian
}}}

If you're using git, then the format is similar, but you should normally provide a branch name as a revision specifier:

{{{
nest-part packaging lp:~some-person/some-project debian debian master-with-packaging
}}}

Again, we've given the branch a short name, `packaging`, that we can use to refer to it in substitution variables. Be careful not to confuse this with the git branch name (here, `master-with-packaging`).

=== Packaging information ===

One of the branches that you include in your recipe must include packaging information in the ```debian``` directory. If it doesn't appear in one of the other branches you specify, you must specifically pull in a ```debian``` directory from elsewhere.

In our example recipe we'll use the ```nest-part``` above.


== What our recipe looks like ==

Adding up all the lines above, our full recipe would look like this:

{{{
# bzr-builder format 0.3 deb-version 1.0+{revno}
Line 57: Line 163:
merge packaging lp:~bzr/bzr/packaging
}}}

=== Specifying revisions ===

Sometimes you want to specify a specific revision of a branch to use, rather than the tip. You can do this by including a revision specifier at the end of any branch line, e.g. {{{
nest-part packaging lp:~some-person/some-project/trunk-with-packaging debian debian
}}}

Or for git:

{{{
# git-build-recipe format 0.4 deb-version 1.0+{revtime}
lp:germinate
merge fix-build lp:~contributor/germinate fix-build
nest pyfoo lp:pyfoo foo master
  merge branding lp:~bob/pyfoo ubuntu-branding
nest-part packaging lp:~some-person/some-project debian debian master-with-packaging
}}}

== Specifying revisions ==

Sometimes you want to specify a specific revision of a branch to use, rather than the tip (or the HEAD symbolic reference in the case of git).

You can do this by including a revision specifier at the end of any branch line. For example:

{{{
Line 66: Line 187:
or, for the first branch line {{{
lp:bzr tag:1.0
}}}

would mean that every time the recipe was used it would use the "1.0" tag from that branch.

=== Version numbers ===

The `deb-version` specifier allow you to specify a version number for the resulting package, but it's not very useful if it is only ever a single version number, as you need it to increase.

The example above (`1.0+{time}`), use a substitution variable. This will be replaced when the version number is needed with the current date and time (UTC). This will ensure that later packages get higher version numbers.

There is a second variable that you can use as well: `{revno}`. This will be replaced with the revision number of the revision that was used from the primary branch. You can use the revision number of any branch by using `{revno:<branch id>}`.

You can use as many substitution variables as you like, e.g. {{{
Similarly for the main branch:

{{{
lp:bzr revno:1234
}}}

Bazaar allows you to tag a certain revision with an easily memorable name. You can request a specific tagged revision like this:

{{{
lp:bzr tag:2.0
}}}

Here, the recipe would use the revision that has the tag "2.0".

For git, a revision specifier may be anything that you could pass to ```git rev-parse``` in a clone of the given repository.

== Version numbers and substitution variables ==

In the first line of the recipe, we specified a version number for the package we want to build, using:

{{{
deb-version 1.0+{revno}
}}}

Rather than specify a fixed version number, we need it to increase every time the package is built. To allow for this, you can use multiple substitution variables.

||<tablestyle="border: none;" rowstyle="background-color: #2a2929; font-weight: bold; color: #f6bc05; padding: 2px 5px 2px 5px;">Variable||Purpose||Introduced in (recipe format version)||Bazaar||Git||
||time||Replaced by the date and time (UTC) when the package was built.||0.1||Yes||Yes||
||revno||Replaced by the revision number.||0.1||Yes||Yes (see note below)||
||latest-tag||Replaced by the name of the most recent tag||0.4||Yes||Yes||
||revdate||Replaced by the date of the revision that was built||0.4||Yes||Yes||
||revtime||Replaced by the time of the revision that was built||0.4||Yes||Yes||
||svn-revno||Replaced with the svn revision number of the revision that was built||0.4||Yes||No||
||git-commit||Replaced with the first 7 characters of the git commit that was built||0.4||Yes||Yes||
||debversion||Replaced with the version in ```debian/changelog```||0.3||Yes||Yes||
||debupstream||Replaced by the upstream portion of the version number taken from ```debian/changelog```. For example: if the version is ```1.0-1```, this would evaluate to ```1.0```.||0.1||Yes||Yes||
||debupstream-base||Similar to {debupstream}, but with any VCS identifiers (e.g. "bzr42", "svn200") stripped, and updated to always end with a "+" or "~")||0.3||Yes||Yes||

All variables other than ```time``` are derived from a particular branch. By default they use the base branch (eg. ```{revno}```), but they can also use a named branch (eg. ```{revno:packaging}```).

```debversion```, ```debupstream``` and ```debupstream-base``` require ```debian/changelog``` to exist in the given branch. For recipe versions 0.4 and newer, you '''must''' specify the name of the branch (e.g. ```{debupstream-base:packaging}```). In recipe versions 0.3 and earlier, the changelog will be read from the final tree if the branch name is omitted.

The advantages of using {revno} and/or {time}:

 * '''{revno}:''' if you want to ensure there's a new package version number whenever the contents of the branch has changed. This is particularly useful if you specify a {revno} for each branch in your recipe.
 * '''{time}:''' if you want the package version to increase whether or not the contents of one of more of the branches has changed.

You can use as many substitution variables as you like. For example:

{{{
Line 84: Line 240:
''Please note that the `packaging` in `revno:packaging` refers to a branch id.''

Which would expand to something like {{{
  deb-version 1.0+4264-0ubuntu0+2145+200907201627
}}}

 * `{time}` will be substituted with the current date and time, such as 200908191512.
 * `{revno}` will be the revno of the base branch (the first specified).
 * `{revno:<branch name>}` will be substituted with the revno for the branch named <branch name> in the recipe.
 * `{debupstream}` will be replaced by the upstream portion of the versionnumber taken from `debian/changelog` in the final tree. If when the tree is built the top of `debian/changelog` has a version number of "1.0-1" then this would evaluate to "1.0".
Here the ```packaging``` in ```revno:packaging``` refers to the short name we gave the branch we're using for packaging.

This example would generate a package version something like:

{{{
1.0+4264-0ubuntu0+2145+200907201627
}}}

Most substitution variables work the same way for git-based recipes. ```{revno}``` is a little peculiar: it is replaced by the length of the left-hand history (i.e. following only the first parent at each step) of the commit in question, to emulate Bazaar revision numbers. This is not very git-like, and most users should normally use ```{revtime}``` or ```{revdate}``` instead, which are monotonically increasing. However, this can result in very long version strings, especially if there are multiple branches involved; so this may for instance be useful as ```{revno:packaging}``` to encode the length of the packaging branch.

```{git-commit}``` should be used with care. Commit hashes do not increase monotonically, so are not normally suitable for use in version strings. At best, this may be useful for information.

= Next steps =

Let's look at [[Packaging/SourceBuilds/BzrBuilder|bzr-builder in more detail]].

||<tablestyle="width: 100%;"> ~-[[Packaging/SourceBuilds/GettingStarted|< Daily builds getting started]] -~ ||<style="text-align: right;"> ~- [[Packaging/SourceBuilds/BzrBuilder|bzr-builder in detail >]] -~ ||

Launchpad Help > Packaging > Daily builds > Recipes

lp-diamond-16.png Source package recipes in Launchpad

Home

Getting Started

Knowledge Base

List of Daily Builds

Overview

A "recipe" is a description of the steps needed to construct a package from a set of Bazaar or Git branches. Its format specifies:

  • Which branch to use for the source code: trunk branch, beta branch, etc.
  • Which branch to use for the packaging information: separate branch, Ubuntu branch, etc.
  • The correct package version (so people can still upgrade to the new stable version when it's released.
  • What to modify to make the source build properly.

The same recipe format is used for bzr via bzr-builder and git via git-build-recipe.

Writing a recipe

Recipes have a simple format.

Note: you can find a typical example in the getting started guide.

They always start with a line similar to this:

# bzr-builder format 0.3 deb-version 1.0+{revno}

Let's take a look at this in more detail:

  • # bzr-builder format 0.3 specifies which recipe format we're using. The current format is 0.4.

  • If you're using git, this will be # git-build-recipe format 0.4 instead.

  • deb-version 1.0+{revno} specifies the version to give the package we're building. {revno} is a substitution variable; more on which later.

Specifying the branches

The next line of a recipe specifies which branch to base the package on:

lp:bzr

This says that we will use the trunk of the bzr project in Launchpad. This could just as easily be any other branch in Launchpad, using the short format that you can find on any branch overview page.

If you're using git, then the format is similar, but you should normally provide a branch name as a revision specifier (if you don't, then the recipe builder will assume HEAD):

lp:germinate master

Note that if you've converted a project from bzr to git, then the "lp:PROJECT" alias for the project's default branch may still be configured for bzr, and will currently take precedence over the git default repository. You can always disambiguate like this:

https://git.launchpad.net/germinate master

Next, you can specify any number of other branches to include. There are two ways to include those branches additional branches:

  • merge: this specifies a simple bzr merge of the two branches.

  • nest: inserts the content of the second branch into a specific location within the main branch.

Merging

merge SHORT-NAME URL [REVISION]

Most often you'll use the "merge" command:

merge fix-build lp:~bzr/bzr/fix-build

Here fix-build is a unique short name that we'll use to refer to this branch in substitution variables. The short name can be anything you like, so long as it is unique to this branch within this recipe.

lp:~bzr/bzr/fix-build is the location of the branch.

In this example, the branch fix-build fixes a problem in the trunk that prevents it from building. This branch could be anything: stand-alone packaging information, some other modification to the branch that's not yet present in the trunk and so on.

If you're using git, then the format is similar, but you should normally provide a branch name as a revision specifier:

merge fix-build lp:~contributor/germinate fix-build

The second fix-build here is something that identifies a commit, usually a ref (branch or tag) name. Be careful not to confuse this with the short name used in substitution variables; you could equally well use the following and only have to adjust some variable references:

merge some-nonsense lp:~contributor/germinate fix-build

Nesting

nest SHORT-NAME URL TARGET-DIRECTORY [REVISION]

Nesting works in a similar way but has more scope:

nest pyfoo lp:pyfoo foo

The nest keyword puts the contents of one branch into a specific location in another branch, instead of merging it.

In this case, we are nesting the contents of lp:pyfoo in a new foo directory in the lp:bzr branch. Again, we've given the branch a short name, pyfoo, that we can use to refer to it in substitution variables.

You can also act on the nested branch in the same way as you can the main branch: you can merge and nest other branches in your nested branch.

Here's how:

nest pyfoo lp:pyfoo foo
  merge branding lp:~bob/pyfoo/ubuntu-branding

If you're using git, then the format is similar, but you should normally provide a branch name as a revision specifier:

nest pyfoo lp:pyfoo foo master
  merge branding lp:~bob/pyfoo ubuntu-branding

Be careful not to confuse the short name used for substitution variables (here, branding) with the git branch name (here, ubuntu-branding).

Any lines that are indented by two spaces, and are directly below your nest line, will act on the nested branch. In this example, the ubuntu-branding branch will be merged into pyfoo before it is nested in your primary branch.

nest-part

nest-part SHORT-NAME URL SOURCE-DIRECTORY [TARGET-DIRECTORY [REVISION]]

If you want to nest only one directory from another branch, you can use nest-part. It works in the same way as nest, except that you specify which directory you're taking from the nested branch.

For example:

nest-part packaging lp:~some-person/some-project/trunk-with-packaging debian debian

If you're using git, then the format is similar, but you should normally provide a branch name as a revision specifier:

nest-part packaging lp:~some-person/some-project debian debian master-with-packaging

Again, we've given the branch a short name, packaging, that we can use to refer to it in substitution variables. Be careful not to confuse this with the git branch name (here, master-with-packaging).

Packaging information

One of the branches that you include in your recipe must include packaging information in the debian directory. If it doesn't appear in one of the other branches you specify, you must specifically pull in a debian directory from elsewhere.

In our example recipe we'll use the nest-part above.

What our recipe looks like

Adding up all the lines above, our full recipe would look like this:

# bzr-builder format 0.3 deb-version 1.0+{revno}
lp:bzr
merge fix-build lp:~bzr/bzr/fix-build
nest pyfoo lp:pyfoo foo
  merge branding lp:~bob/pyfoo/ubuntu-branding
nest-part packaging lp:~some-person/some-project/trunk-with-packaging debian debian

Or for git:

# git-build-recipe format 0.4 deb-version 1.0+{revtime}
lp:germinate
merge fix-build lp:~contributor/germinate fix-build
nest pyfoo lp:pyfoo foo master
  merge branding lp:~bob/pyfoo ubuntu-branding
nest-part packaging lp:~some-person/some-project debian debian master-with-packaging

Specifying revisions

Sometimes you want to specify a specific revision of a branch to use, rather than the tip (or the HEAD symbolic reference in the case of git).

You can do this by including a revision specifier at the end of any branch line. For example:

merge packaging lp:~bzr/bzr/packaging revno:2355

Similarly for the main branch:

lp:bzr revno:1234

Bazaar allows you to tag a certain revision with an easily memorable name. You can request a specific tagged revision like this:

lp:bzr tag:2.0

Here, the recipe would use the revision that has the tag "2.0".

For git, a revision specifier may be anything that you could pass to git rev-parse in a clone of the given repository.

Version numbers and substitution variables

In the first line of the recipe, we specified a version number for the package we want to build, using:

deb-version 1.0+{revno}

Rather than specify a fixed version number, we need it to increase every time the package is built. To allow for this, you can use multiple substitution variables.

Variable

Purpose

Introduced in (recipe format version)

Bazaar

Git

time

Replaced by the date and time (UTC) when the package was built.

0.1

Yes

Yes

revno

Replaced by the revision number.

0.1

Yes

Yes (see note below)

latest-tag

Replaced by the name of the most recent tag

0.4

Yes

Yes

revdate

Replaced by the date of the revision that was built

0.4

Yes

Yes

revtime

Replaced by the time of the revision that was built

0.4

Yes

Yes

svn-revno

Replaced with the svn revision number of the revision that was built

0.4

Yes

No

git-commit

Replaced with the first 7 characters of the git commit that was built

0.4

Yes

Yes

debversion

Replaced with the version in debian/changelog

0.3

Yes

Yes

debupstream

Replaced by the upstream portion of the version number taken from debian/changelog. For example: if the version is 1.0-1, this would evaluate to 1.0.

0.1

Yes

Yes

debupstream-base

Similar to {debupstream}, but with any VCS identifiers (e.g. "bzr42", "svn200") stripped, and updated to always end with a "+" or "~")

0.3

Yes

Yes

All variables other than time are derived from a particular branch. By default they use the base branch (eg. {revno}), but they can also use a named branch (eg. {revno:packaging}).

debversion, debupstream and debupstream-base require debian/changelog to exist in the given branch. For recipe versions 0.4 and newer, you must specify the name of the branch (e.g. {debupstream-base:packaging}). In recipe versions 0.3 and earlier, the changelog will be read from the final tree if the branch name is omitted.

The advantages of using {revno} and/or {time}:

  • {revno}: if you want to ensure there's a new package version number whenever the contents of the branch has changed. This is particularly useful if you specify a {revno} for each branch in your recipe.

  • {time}: if you want the package version to increase whether or not the contents of one of more of the branches has changed.

You can use as many substitution variables as you like. For example:

  deb-version 1.0+{revno}-0ubuntu0+{revno:packaging}+{time}

Here the packaging in revno:packaging refers to the short name we gave the branch we're using for packaging.

This example would generate a package version something like:

1.0+4264-0ubuntu0+2145+200907201627

Most substitution variables work the same way for git-based recipes. {revno} is a little peculiar: it is replaced by the length of the left-hand history (i.e. following only the first parent at each step) of the commit in question, to emulate Bazaar revision numbers. This is not very git-like, and most users should normally use {revtime} or {revdate} instead, which are monotonically increasing. However, this can result in very long version strings, especially if there are multiple branches involved; so this may for instance be useful as {revno:packaging} to encode the length of the packaging branch.

{git-commit} should be used with care. Commit hashes do not increase monotonically, so are not normally suitable for use in version strings. At best, this may be useful for information.

Next steps

Let's look at bzr-builder in more detail.

< Daily builds getting started

bzr-builder in detail >

Packaging/SourceBuilds/Recipes (last edited 2021-06-07 13:41:17 by cjwatson)