Diff for "VcsImportRequests"

Not logged in - Log In / Register
Differences between revisions 11 and 135 (spanning 124 versions)
⇤ ← Revision 11 as of 2006-06-12 13:56:11 →
Size: 2877
Editor: nor75-18-82-241-238-155
Comment:
← Revision 135 as of 2007-01-26 14:59:46 → ⇥
Size: 19592
Editor: nor75-18-82-241-238-155
Comment: merge entries for dojo-trunk
Deletions are marked like this. Additions are marked like this.
Line 13: Line 13:
 * [https://launchpad.net/people/<launchpad-user> <launchpad-user>]: https://launchpad.net/products/<launchpad-product>/<series> [[BR]]
Line 15: Line 14:
   [@`DATE`@]  * [https://launchpad.net/people/<launchpad-user> <launchpad-user>]: [https://launchpad.net/products/<launchpad-product>/<series> <product>/<series>] [[BR]] <location> [[BR]] @`DATE`@
Line 18: Line 17:

[[TableOfContents()]]
----

= Incoming =
Line 25: Line 26:
 * [https://launchpad.net/people/kamion kamion]: [https://launchpad.net/products/installation-guide/trunk installation-guide/trunk] [[BR]] svn://svn.debian.org/svn/d-i/trunk/manual [[BR]] [[Date(2007-01-25T09:44:08Z)]]

== In progress ==

Imports being processed.

= Blocked =

Imports that are currently failing, and for which no fix is currently being worked on.

== [svn] svn_subst_translate_stream3 ==

Imports that failed with the following error:

{{{
python2.4: /build/buildd/subversion-1.3.1/subversion/libsvn_subr/subst.c:809: svn_subst_translate_stream3: Assertion `eol_str || keywords' failed.
process killed by signal 6
}}}

It looks like a problem in the svn code for keyword expansion.

 * [https://launchpad.net/people/kamion kamion]: [https://launchpad.net/debian-installer/main debian-installer/main] [[BR]] svn://svn.debian.org/d-i/trunk/installer [[BR]] [[Date(2007-01-23T13:52:03Z)]] [[BR]] Note: the import is currently set up to import from a local copy of the repository: `file:///home/importd/d-i.repo/trunk/installer`

 * nobody: [https://launchpad.net/mcs/trunk mcs/trunk] [[BR]] svn://svn.myrealbox.com/source/trunk/mcs [[BR]] [[Date(2007-01-23T13:56:36Z)]]

== [svn] wrong repository root ==

The current svn import code determines the root of the repository by walking up the URL until svn ls fails. This was necessary because older versions of libsvn_client did not expose the repository root information. Newer libsvn_client allow to determine this information directly. In cases where there are nested svn repositories, the old logic is wrong and that cause the change filtering (ignoring change outside of the branch being imported) to ignore legitimate changes.

 * nobody: [https://launchpad.net/products/libsyncml/trunk libsyncml/trunk] [[BR]] http://svn.opensync.org/libsyncml/trunk [[BR]] [[Date(2006-11-28T11:56:17Z)]]

== [svn] unreliable server ==

The current svn import code makes many connections and does not retry when one fails. If the server is in the least unreliable (one failure every few thousand attempts to create a session) that will cause the import to fail.

It could be that some of these failures are caused by another specific bug. From a discussion with Jelmer:

{{{
> > > * Don't rely on listdir() kind results. Fixes imports of several
> > > repositories. (#56647)
> > I'm curious about this bug. Can you tell me more?
> It turned out that the .kind member of dirents returned by
> svn.ra.get_dir2() is (semi-?)random when used against older Subversion
> servers. It would occasionally be 1 (svn.core.svn_node_file) even though
> the entry in question was a directory. This took me a while to track
> down. Instead, I've switched to always trying to open something as a
> directory and catching the appropriate exception that indicates it's a
> file.

That's very interesting. A number of Launchpad imports fail in a not-deterministic way with "failed REPORT". Currently I just classify those failures as "unreliable server", but I was puzzled by a pattern I noticed: the failures did not happen in a really random way.

It might be the same bug.
}}}

 * nobody: [https://launchpad.net/products/deluge/trunk deluge/trunk] [[BR]] http://deluge-torrent.googlecode.com/svn/trunk [[BR]] [[Date(2006-11-28T11:51:08Z)]]

 * nobody: [https://launchpad.net/products/python2.4-matplotlib/trunk matplotlib/trunk] [[BR]] https://svn.sourceforge.net/svnroot/matplotlib/trunk/matplotlib [[BR]] [[Date(2006-11-29T09:58:17Z)]]

 * nobody: [https://launchpad.net/products/brltty/main brltty/main] [[BR]] svn://mielke.cc/main/brltty [[BR]] [[Date(2006-11-29T10:32:43Z)]]

 * nobody: [https://launchpad.net/products/codeblocks/trunk codeblocks/trunk] [[BR]] svn://svn.berlios.de/codeblocks/trunk [[BR]] [[Date(2006-12-04T14:03:14Z)]]

 * nobody: [https://launchpad.net/products/xorg-driver-synaptics/main xorg-driver-synaptics-main] [[BR]] https://oops.kerneljanitors.org/repos/synaptic/trunk [[BR]] [[Date(2006-12-06T10:40:09Z)]]

 * [https://launchpad.net/people/dsnopek dsnopek]: [https://launchpad.net/products/dojo/trunk dojo/trunk] [[BR]] http://svn.dojotoolkit.org/dojo/trunk [[BR]] [[[Date(2006-08-31T20:10:43Z)]]]

 * nobody: [https://launchpad.net/products/wesnoth/trunk wesnoth/trunk] [[BR]] http://svn.gna.org/svn/wesnoth/trunk [[BR]] [[Date(2006-12-08T13:26:17Z)]]

 * nobody: [https://launchpad.net/products/libgcrypt/trunk libgcrypt/trunk] [[BR]] svn://cvs.gnupg.org/libgcrypt/trunk [[BR]] [[Date(2006-12-08T13:29:30Z)]]

 * nobody: [https://launchpad.net/products/libcaca/trunk libcaca/trunk] [[BR]] svn://svn.zoy.org/libcaca/trunk [[BR]] [[Date(2006-12-11T14:46:59Z)]]

 * nobody: [https://code.launchpad.net/gwenview/trunk gwenview/trunk] [[BR]] svn://anonsvn.kde.org/home/kde/trunk/extragear/graphics/gwenview [[BR]] [[Date(2007-01-23T14:03:32Z)]]

== [svn] missing uri-encoding ==

cscvs uses the svn_client API to libsvn to retrieve data (on [[Date(2006-11-09T14:31:29Z)]]), but does not correctly uri-encode locations. So file names that contain characters that must be escaped cause import failures.

 * nobody: [https://launchpad.net/products/dcsharp/trunk dcsharp/trunk] [[BR]] [[Date(2006-11-09T14:31:29Z)]]

 * nobody: [https://launchpad.net/products/wxmsn/trunk wxmsn/trunk] [[BR]] [[Date(2006-11-09T14:34:42Z)]]

 * nobody: [https://launchpad.net/products/tmw/trunk tmw/trunk] [[BR]] https://svn.sourceforge.net/svnroot/themanaworld/tmw/trunk [[BR]] [[[Date(2006-09-29T15:57:33Z)]]]

 * nobody: [https://launchpad.net/products/pcm/trunk pcm/trunk] [[BR]] http://clanmanager.googlecode.com/svn/trunk [[BR]] [[Date(2006-11-29T09:11:13Z)]]

== [cvs] failed to open the modules file ==

 * nobody: [https://launchpad.net/products/greasemonkey/trunk greasemonkey/trunk] [[BR]] :pserver:guest:guest``@mozdev.org:/cvs greasemonkey [[BR]] [[Date(2006-11-09T14:24:47Z)]]

cscvs needs to read the modules file to ignore configs (aka externals, nested trees).

{{{
$ cvs -d :pserver:guest:guest@mozdev.org:/cvs co -c
cvs [checkout aborted]: failed to open the modules file
}}}

== [svn] svn:externals ==

[https://launchpad.net/products/launchpad-bazaar/+bug/69513 Bug 69513]

 * nobody: python-musicbrainz2/trunk [[BR]] [[Date(2006-11-09T14:19:15Z)]]

 * nobody: [https://launchpad.net/products/hyperic-hq/trunk hyperic-hq/trunk] [[BR]] http://svn.hyperic.org/projects/hq/trunk [[BR]] [[Date(2006-11-09T14:49:55Z)]]

 * nobody: [https://launchpad.net/products/closetgnome/trunk closetgnome/trunk] [[BR]] http://svn.wowace.com/wowace/trunk/ClosetGnome [[BR]] [[Date(2006-11-27T14:54:44Z)]]

 * nobody: [https://launchpad.net/products/openvpn/trunk openvpn/trunk] [[BR]] http://svn.openvpn.net/projects/openvpn/trunk/openvpn [[BR]] [[Date(2006-11-27T18:18:14Z)]]

 * nobody: [https://launchpad.net/products/shotserver/trunk shotserver/trunk] [[BR]] http://svn.browsershots.org/trunk/shotserver [[BR]] [[Date(2006-11-29T09:52:44Z)]]

 * nobody: [https://launchpad.net/products/taglib/trunk taglib/trunk] [[BR]] svn://anonsvn.kde.org/home/kde/trunk/kdesupport/taglib [[BR]] [[Date(2006-12-01T09:33:05Z)]]

 * [https://launchpad.net/people/radix radix]: [https://launchpad.net/products/pypy/trunk pypy/trunk] [[BR]] http://codespeak.net/svn/pypy/dist [[BR]] [[Date(2006-12-04T13:56:20Z)]]

 * nobody: [https://launchpad.net/products/kdebindings/trunk kdebindings/trunk] [[BR]] svn://anonsvn.kde.org/home/kde/trunk/KDE/kdebindings [[BR]] [[Date(2006-12-08T13:43:31Z)]]

 * nobody: [https://launchpad.net/products/clientcookie/trunk clientcookie/trunk] [[BR]] http://codespeak.net/svn/wwwsearch/ClientCookie/trunk [[BR]] [[Date(2006-12-11T14:44:59Z)]]

 * [https://launchpad.net/people/spiv spiv]: [https://launchpad.net/products/sqlobject/trunk sqlobject/trunk] [[BR]] http://svn.colorstudy.com/SQLObject/trunk [[BR]] [[Date(2006-12-20T04:09:02Z)]]

== [cvs] file created by merge ==

 * [[Anchor(epiphany-browser)]][https://launchpad.net/people/seb128 seb128]: https://launchpad.net/products/epiphany-browser/main [[BR]] :pserver:anonymous``@anoncvs.gnome.org:/cvs/gnome epiphany [[BR]] [[[Date(2006-10-16T14:54:17Z)]]]
   * On changeset MAIN.67 [[BR]] ValueError: attempt to patch non extant file : data/ui/epiphany-toolbar.xml.in
   * The cscvs log (with changesets) first mentions epiphany-toolbar at changeset MAIN.67, it refers to revision 1.2 of this file and records it as a 'M' (modification), which fails to apply since the file is not present in the imported tree at this point.

Relevant fragment of the cvs log:

{{{
$ cvs rlog epiphany/data\/ui/epiphany-toolbar.xml.in

RCS file: /cvs/gnome/epiphany/data/ui/Attic/epiphany-toolbar.xml.in,v
head: 1.17
branch:
locks: strict
access list:
symbolic names:
[...]
        eog-menu-api: 1.1.0.2
keyword substitution: kv
total revisions: 19; selected revisions: 19
description:
----------------------------
[...]
----------------------------
revision 1.2
date: 2003/01/20 18:57:15; author: mpeseng; state: Exp; lines: +15 -0
2003-01-20 Marco Pesenti Gritti <marco@it.gnome.org>

        * Merge eog-menu-api branch
----------------------------
revision 1.1
date: 2003/01/20 17:53:25; author: mpeseng; state: dead;
branches: 1.1.2;
file epiphany-toolbar.xml.in was initially added on branch eog-menu-api.
----------------------------
revision 1.1.2.1
date: 2003/01/20 17:53:25; author: mpeseng; state: Exp; lines: +15 -0
implement context menus
----------------------------
[...]
}}}

 * [[Anchor(grub2)]][https://launchpad.net/people/jbailey jbailey]: https://launchpad.net/products/grub/grub2 [[BR]] :pserver:anonymous``@cvs.savannah.gnu.org:/sources/grub grub2 [[BR]] [[[Date(2006-10-20T16:10:46Z)]]]
   * On changeset MAIN.340 [[BR]] ValueError: attempt to patch non extant file : disk/lvm.c

== [svn] Cannot import whole repository ==

The svn import code fails at processing imports at the root of a svn repository. Some repository are set up that way, with the actual root containing the source code. That's broken, but people actually do this.

 * nobody: https://launchpad.net/products/gtimelog/devel [[BR]] http://mg.pov.lt/gtimelog/svn [[BR]] [[[Date(2006-10-23T16:31:04Z)]]]

 * nobody: https://launchpad.net/products/josm/trunk [[BR]] http://www.eigenheimstrasse.de/svn/josm [[BR]] [[[Date(2006-10-23T16:31:04Z)]]]

 * nobody: https://launchpad.net/products/comix/trunk [[BR]] https://svn.sourceforge.net/svnroot/comix [[BR]] [[[Date(2006-10-23T16:31:04Z)]]]

[[Anchor(squid)]]
== [cvs] None branch in findLastFileRevision ==

While creating changesets: assertion error, branchName is None in findLastFileRevision CVS/CacheGenerator.py:160

It looks like it means something is really busted with filler revisions and default branches.

 * [https://launchpad.net/people/lifeless lifeless]: https://launchpad.net/products/squid/2.6 [[BR]] :pserver:anoncvs:anoncvs`@`cvs.squid-cache.org:/squid squid [[BR]] [[[Date(2006-10-31T16:02:19Z)]]]

 * [https://launchpad.net/people/lifeless lifeless]: https://launchpad.net/products/squid/3.0 [[BR]] :pserver:anoncvs:anoncvs`@`cvs.squid-cache.org:/squid squid3 [[BR]] [[[Date(2006-10-31T16:02:19Z)]]]

== File name not supported by bzr ==

 * nobody: https://launchpad.net/products/unixodbc/main [[BR]] :pserver:anonymous`@`unixodbc.cvs.sourceforge.net:/cvsroot/unixodbc unixODBC [[BR]] [[[Date(2006-08-29T11:27:45Z)]]]
   * on changeset MAIN.1 [[BR]] The branch contains a file named `odbctest/C:\IB6TRACE.LOG` (yes, starting with `C:\`!) [[BR]] in bzrlib.workingtree.WorkingTree.add [[BR]] bzrlib.errors.NotVersionedError: ['odbctest', 'C:'] is not versioned [[BR]]

== [cvs] Absurd filler revision ==

[[Anchor(xrms)]]
 * nobody: [https://launchpad.net/products/xrms/trunk xrms/trunk] [[BR]] :pserver:anonymous``@xrms.cvs.sourceforge.net:/cvsroot/xrms/ xrms [[BR]] [[[Date(2006-09-29T21:20:41Z)]]]
   * on changeset MAIN.4167: [[BR]] CRITICAL: requested checkout of reporoot '/cvsroot/xrms', file 'xrms/include/adodb/pear/Attic/readme.Auth.txt', revision '1.1.1.2' [[BR]] CRITICAL: log of checkout responses follow. [[BR]] CRITICAL: "E cvs checkout: cannot find module [[BR]] `xrms/include/adodb/pear/Attic/readme.Auth.txt' - ignored\n" [[BR]] CRITICAL: 'error \n'
   * cscvs creates a filler changeset 1.1->1.1.1.2 for include/adodb/pear/readme.Auth.txt, but there is no revision 1.1.1.2 for that file. The cvs rlog shows revision 1.1 (branches: 1.1.1) used in changeset MAIN.915, revision 1.1.1.1, not used by cscvs. The cvs log header for that file reads "head: 1.1" and "branch: 1.1.1".

lifeless suggests:
{{{
(15:28:25) lifeless: I'd do a search in the file revision table for that file, all revisions, see what cscvs is thinking happened to it
(15:29:45) ddaa: you suggest it might be a problem in the log parser and checking the catalog contents would help narrowing the cause of the problem?
(15:33:11) lifeless: something is inventing the .2
(15:33:22) lifeless: if its in the catalog then the catalog generator can be checked
(15:33:42) lifeless: if its not in the catalog then the application-of-changesets can be checked
(15:33:45) lifeless: it will narrow it down
(15:34:03) lifeless: As its a very small rlog I'd make a test case from it
}}}

 * [https://launchpad.net/people/jordi jordi]: [https://launchpad.net/products/gtetrinet/trunk gtetrinet/trunk] [[BR]] :pserver:anonymous``@anoncvs.gnome.org:/cvs/gnome gtetrinet [[BR]] [[Date(2006-11-30T17:27:49Z)]]
   * on changeset MAIN.394 [[BR]] CRITICAL: requested checkout of reporoot '/cvs/gnome', file 'gtetrinet/src/Attic/tetris.h', revision '1.15.2.2' [[BR]] CRITICAL: log of checkout responses follow. [[BR]] CRITICAL: "E cvs server: cannot find module `gtetrinet/src/Attic/tetris.h' - ignored\n" [[BR]] CRITICAL: 'error \n'
   * cscvs creates a filler changeset 1.1->1.15.2.2 for gtetrinet/src/tetris.h but the log of that file only has revisions 1.1, 1.1.1.1, 1.1.1.1.2.2, 1.1.1.1.2.1.

== [cvs] Checkout gave no file data ==

[[Anchor(tomboy)]]
 * nobody: [https://launchpad.net/products/tomboy/head tomboy/head] [[BR]] :pserver:anonymous``@anoncvs.gnome.org:/cvs/gnome tomboy [[BR]] [[Date(2006-12-04T14:23:12Z)]]

{{{
WARNING: N changeset MAIN.745
CRITICAL: checkout did not give file data and did not indicate file was removed.
CRITICAL: requested checkout of reporoot '/cvs/gnome', file 'tomboy/www/img/intro.gif', revision '1.1'
CRITICAL: log of checkout responses follow.
CRITICAL: 'ok\n'
}}}

The cscvs log for this changeset:

{{{
ChangeSet: 745
Author: orph
Date: 2004/09/18 08:52:21
Branch: MAIN
CVSDefaultFiller
Revisions:
        F www/img/intro.gif [1.1->1.1] +0-0

Filler changeset 1.1->1.1
}}}

== [svn] repository password protected ==

 * nobody: [https://launchpad.net/products/yast/trunk yast/trunk] [[BR]] https://forgesvn1.novell.com/svn/yast/trunk [[BR]] [[Date(2006-12-08T13:34:33Z)]]

 * nobody: [https://launchpad.net/products/bauble/trunk bauble/trunk] [[BR]] https://forgesvn1.novell.com/svn/bauble/trunk [[BR]] [[Date(2006-12-08T13:46:06Z)]]
Line 29: Line 276:
Imports that have failed, but whose cause of failure has not yet been ascertained.

 * [https://launchpad.net/people/mvo mvo]: https://launchpad.net/products/smart/trunk [[BR]]
   [[[Date(2006-06-12T13:15:00Z)]]] [[BR]]
   failed on revision 29: [[BR]]
   pysvn._pysvn.ClientError: REPORT request failed on '/!svn/bc/28/trunk/epm/backends' '/!svn/bc/28/trunk/epm/backends' path not found

 * [https://launchpad.net/people/jani janimo]: https://launchpad.net/products/xfdesktop/trunk [[BR]]
   [[[Date(2006-06-12T13:15:00Z)]]] [[BR]]
   failed on revision 1: [[BR]]
   pysvn._pysvn.ClientError: REPORT request failed on '/svn/xfce/!svn/bc/22073/modules/trunk/xfdesktop' '/svn/xfce/!svn/bc/22073/modules/trunk/xfdesktop' path not found

 * [https://launchpad.net/people/jani janimo]: https://launchpad.net/products/thunar/trunk [[BR]]
   [[[Date(2006-06-12T13:15:00Z)]]] [[BR]]
   failed on revsion 18919: [[BR]]
   pysvn._pysvn.ClientError: REPORT request failed on '/svn/xfce/!svn/bc/18918/thunar/trunk/examples/tex-open-terminal' '/svn/xfce/!svn/bc/18918/thunar/trunk/examples/tex-open-terminal' path not found

== Blocked on bzr-native ==

Imports that have failed and that will be tried again when the vcs-import system will do native bzr imports instead of using a baz branch as an intermediate step.

 * [https://launchpad.net/people/siretart siretart]: https://launchpad.net/products/xine-lib/head [[BR]]
   [[[Date(2006-06-12T13:52:22Z)]]] [[BR]]
   failed catalog rebuilding: AssertionError: no log found for changeset 5317 [[BR]]
   [https://launchpad.net/products/launchpad-bazaar/+bug/49398 bug 49398]



 * [https://launchpad.net/people/siretart siretart]: https://launchpad.net/products/mplayer/trunk [[BR]]
   [[[Date(2006-06-12T13:15:00Z)]]] [[BR]]
   failed with OSError: [Errno 12] Cannot allocate memory [[BR]]		  * nobody: [https://launchpad.net/products/dtv/trunk dtv/trunk] [[BR]] https://svn.participatoryculture.org/svn/dtv/trunk [[BR]] [[[Date(2006-10-23T16:19:12Z)]]]
   * On checkout (yeah, tested with actually "svn co" on command line) [[BR]] `REPORT request failed on '/svn/dtv/!svn/vcc/default'` [[BR]] `REPORT of '/svn/dtv/!svn/vcc/default': Could not read chunk delimiter: Secure connection truncated (https://svn.participatoryculture.org)`

 * nobody: [https://launchpad.net/products/cups/main cups/main] [[BR]] http://svn.easysw.com/public/cups/trunk [[BR]] [[Date(2006-11-29T10:08:15Z)]]
   * On full-tree import of initial revision

{{{
WARNING: change 161
WARNING: change 166
Traceback (most recent call last):
  File "/home/importd/dists/launchpad/lib/importd/CommandLineRunner.py", line 67, in ?
    main()
  File "/home/importd/dists/launchpad/lib/importd/CommandLineRunner.py", line 64, in main
    doer.runMethod(method, dirname)
  File "/home/importd/dists/launchpad/lib/importd/CommandLineRunner.py", line 57, in runMethod
    getattr(self._job, method)(logger=self.logger, dir=dirname)
  File "/srv/importd.ubuntu.com/dists/launchpad/lib/importd/Job.py", line 188, in runJob
    strategy(self, dir, logger)
  File "/srv/importd.ubuntu.com/dists/launchpad/lib/importd/JobStrategy.py", line 100, in Import
    self.runtobaz("-SI", "%s.1" % aJob.branchfrom, target_path, logger)
  File "/srv/importd.ubuntu.com/dists/launchpad/lib/importd/JobStrategy.py", line 84, in runtobaz
    totla.totla(config, logger, config.args, self.sourceTree())
  File "/srv/importd.ubuntu.com/dists/launchpad/lib/cscvs/cmds/totla.py", line 262, in totla
    apply_changeset(source_tree, logger, apply, revision, target_tree, revisions)
  File "/srv/importd.ubuntu.com/dists/launchpad/lib/cscvs/cmds/totla.py", line 130, in apply_changeset
    fileChange.apply(logger, target_tree, prune)
  File "/srv/importd.ubuntu.com/dists/launchpad/lib/svn_oo/PresentFile.py", line 46, in apply
    self.url(), self.revision)
  File "/srv/importd.ubuntu.com/dists/launchpad/lib/svn_oo/util.py", line 116, in writeFile
    revision=pysvn_revision, peg_revision=pysvn_revision)
pysvn._pysvn.ClientError: Code: 00020f58
}}}

== Failed to fork, memory error ==

Some large imports fail to os.fork() (when spawning gpg to sign a revision) with "OSError: [Errno 12] Cannot allocate memory". It's not clear why that happen, since Linux should overcommit memory up to 50x (the value of /proc/sys/vm/overcommit_ratio) and allow large processes like cscvs to spawn small processes like gpg.

 * [https://launchpad.net/people/siretart siretart]: https://launchpad.net/products/mplayer/trunk [[BR]] svn://svn.mplayerhq.hu/mplayer/trunk [[BR]] [[[Date(2006-06-12T13:15:00Z)]]]
   * On changeset changeset 5343

 * [https://launchpad.net/people/jbailey jbailey]: https://launchpad.net/products/glibc/head [[BR]] :pserver:anoncvs`@`sources.redhat.com:/cvs/glibc glibc [[BR]] [[[Date(2006-08-29T10:36:24Z)]]]
   * On changeset MAIN.15819

 * [https://launchpad.net/people/carlos]: https://launchpad.net/products/elisa/trunk [[BR]] https://core.fluendo.com/elisa/svn/trunk [[BR]] [[[Date(2006-09-06T09:43:12Z)]]]
   * On changeset 7

= Fix in progress =

Failures for which a fix is currently being worked on.

This page tracks requests for actions from the vcs-imports operator.

To request an import, please:

  • Create a product in Launchpad and set the source details to the CVS or Subversion repository you wish to have imported. In particular
  • Add a request on this page, with the name of your launchpad account so I can find your email.

Note that we only do imports of MAIN branches on CVS (the form will have to be updated to remove the "branch" field) and trunk branches on Subversion.

We will handle your request and keep you informed of progress or issues. You can contact the vcs-imports operator: ddaa in #launchpad on irc.freenode.net.

Template to request an import:

TableOfContents()

Incoming

Open requests

Imports that need intervention from the vcs-imports operator.

In progress

Imports being processed.

Blocked

Imports that are currently failing, and for which no fix is currently being worked on.

[svn] svn_subst_translate_stream3

Imports that failed with the following error: 

python2.4: /build/buildd/subversion-1.3.1/subversion/libsvn_subr/subst.c:809: svn_subst_translate_stream3: Assertion `eol_str || keywords' failed.
process killed by signal 6

It looks like a problem in the svn code for keyword expansion.

[svn] wrong repository root

The current svn import code determines the root of the repository by walking up the URL until svn ls fails. This was necessary because older versions of libsvn_client did not expose the repository root information. Newer libsvn_client allow to determine this information directly. In cases where there are nested svn repositories, the old logic is wrong and that cause the change filtering (ignoring change outside of the branch being imported) to ignore legitimate changes.

[svn] unreliable server

The current svn import code makes many connections and does not retry when one fails. If the server is in the least unreliable (one failure every few thousand attempts to create a session) that will cause the import to fail.

It could be that some of these failures are caused by another specific bug. From a discussion with Jelmer: 

> > >    * Don't rely on listdir() kind results. Fixes imports of several 
> > >      repositories. (#56647)
> > I'm curious about this bug. Can you tell me more?
> It turned out that the .kind member of dirents returned by
> svn.ra.get_dir2() is (semi-?)random when used against older Subversion
> servers. It would occasionally be 1 (svn.core.svn_node_file) even though
> the entry in question was a directory. This took me a while to track
> down. Instead, I've switched to always trying to open something as a
> directory and catching the appropriate exception that indicates it's a
> file.

That's very interesting. A number of Launchpad imports fail in a not-deterministic way with "failed REPORT". Currently I just classify those failures as "unreliable server", but I was puzzled by a pattern I noticed: the failures did not happen in a really random way.

It might be the same bug.

[svn] missing uri-encoding

cscvs uses the svn_client API to libsvn to retrieve data (on Date(2006-11-09T14:31:29Z)), but does not correctly uri-encode locations. So file names that contain characters that must be escaped cause import failures.

[cvs] failed to open the modules file

cscvs needs to read the modules file to ignore configs (aka externals, nested trees). 

$ cvs -d :pserver:guest:guest@mozdev.org:/cvs co -c
cvs [checkout aborted]: failed to open the modules file

[svn] svn:externals

[https://launchpad.net/products/launchpad-bazaar/+bug/69513 Bug 69513]

[cvs] file created by merge

Relevant fragment of the cvs log: 

$ cvs rlog epiphany/data\/ui/epiphany-toolbar.xml.in

RCS file: /cvs/gnome/epiphany/data/ui/Attic/epiphany-toolbar.xml.in,v
head: 1.17
branch:
locks: strict
access list:
symbolic names:
[...]
        eog-menu-api: 1.1.0.2
keyword substitution: kv
total revisions: 19;    selected revisions: 19
description:
----------------------------
[...]
----------------------------
revision 1.2
date: 2003/01/20 18:57:15;  author: mpeseng;  state: Exp;  lines: +15 -0
2003-01-20  Marco Pesenti Gritti  <marco@it.gnome.org>

        * Merge eog-menu-api branch
----------------------------
revision 1.1
date: 2003/01/20 17:53:25;  author: mpeseng;  state: dead;
branches:  1.1.2;
file epiphany-toolbar.xml.in was initially added on branch eog-menu-api.
----------------------------
revision 1.1.2.1
date: 2003/01/20 17:53:25;  author: mpeseng;  state: Exp;  lines: +15 -0
implement context menus
----------------------------
[...]

[svn] Cannot import whole repository

The svn import code fails at processing imports at the root of a svn repository. Some repository are set up that way, with the actual root containing the source code. That's broken, but people actually do this.

Anchor(squid)

[cvs] None branch in findLastFileRevision

While creating changesets: assertion error, branchName is None in findLastFileRevision CVS/CacheGenerator.py:160

It looks like it means something is really busted with filler revisions and default branches.

File name not supported by bzr

[cvs] Absurd filler revision

Anchor(xrms)

  • nobody: [https://launchpad.net/products/xrms/trunk xrms/trunk] BR :pserver:anonymous@xrms.cvs.sourceforge.net:/cvsroot/xrms/ xrms BR [Date(2006-09-29T21:20:41Z)]

    • on changeset MAIN.4167: BR CRITICAL: requested checkout of reporoot '/cvsroot/xrms', file 'xrms/include/adodb/pear/Attic/readme.Auth.txt', revision '1.1.1.2' BR CRITICAL: log of checkout responses follow. BR CRITICAL: "E cvs checkout: cannot find module BR `xrms/include/adodb/pear/Attic/readme.Auth.txt' - ignored\n" BR CRITICAL: 'error \n'

    • cscvs creates a filler changeset 1.1->1.1.1.2 for include/adodb/pear/readme.Auth.txt, but there is no revision 1.1.1.2 for that file. The cvs rlog shows revision 1.1 (branches: 1.1.1) used in changeset MAIN.915, revision 1.1.1.1, not used by cscvs. The cvs log header for that file reads "head: 1.1" and "branch: 1.1.1".

lifeless suggests: 

(15:28:25) lifeless: I'd do a search in the file revision table for that file, all revisions, see what cscvs is thinking happened to it
(15:29:45) ddaa: you suggest it might be a problem in the log parser and checking the catalog contents would help narrowing the cause of the problem?
(15:33:11) lifeless: something is inventing the .2
(15:33:22) lifeless: if its in the catalog then the catalog generator can be checked
(15:33:42) lifeless: if its not in the catalog then the application-of-changesets can be checked
(15:33:45) lifeless: it will narrow it down
(15:34:03) lifeless: As its a very small rlog I'd make a test case from it

  • [https://launchpad.net/people/jordi jordi]: [https://launchpad.net/products/gtetrinet/trunk gtetrinet/trunk] BR :pserver:anonymous@anoncvs.gnome.org:/cvs/gnome gtetrinet BR Date(2006-11-30T17:27:49Z)

    • on changeset MAIN.394 BR CRITICAL: requested checkout of reporoot '/cvs/gnome', file 'gtetrinet/src/Attic/tetris.h', revision '1.15.2.2' BR CRITICAL: log of checkout responses follow. BR CRITICAL: "E cvs server: cannot find module `gtetrinet/src/Attic/tetris.h' - ignored\n" BR CRITICAL: 'error \n'

    • cscvs creates a filler changeset 1.1->1.15.2.2 for gtetrinet/src/tetris.h but the log of that file only has revisions 1.1, 1.1.1.1, 1.1.1.1.2.2, 1.1.1.1.2.1.

[cvs] Checkout gave no file data

Anchor(tomboy) 

WARNING: N changeset MAIN.745
CRITICAL: checkout did not give file data and did not indicate file was removed.
CRITICAL: requested checkout of reporoot '/cvs/gnome', file 'tomboy/www/img/intro.gif', revision '1.1'
CRITICAL: log of checkout responses follow.
CRITICAL: 'ok\n'

The cscvs log for this changeset: 

ChangeSet: 745
Author: orph
Date: 2004/09/18 08:52:21
Branch: MAIN
CVSDefaultFiller
Revisions:
        F www/img/intro.gif [1.1->1.1] +0-0

Filler changeset 1.1->1.1

[svn] repository password protected

Unknown failures

WARNING: change 161
WARNING: change 166
Traceback (most recent call last):
  File "/home/importd/dists/launchpad/lib/importd/CommandLineRunner.py", line 67, in ?
    main()
  File "/home/importd/dists/launchpad/lib/importd/CommandLineRunner.py", line 64, in main
    doer.runMethod(method, dirname)
  File "/home/importd/dists/launchpad/lib/importd/CommandLineRunner.py", line 57, in runMethod
    getattr(self._job, method)(logger=self.logger, dir=dirname)
  File "/srv/importd.ubuntu.com/dists/launchpad/lib/importd/Job.py", line 188, in runJob
    strategy(self, dir, logger)
  File "/srv/importd.ubuntu.com/dists/launchpad/lib/importd/JobStrategy.py", line 100, in Import
    self.runtobaz("-SI", "%s.1" % aJob.branchfrom, target_path, logger)
  File "/srv/importd.ubuntu.com/dists/launchpad/lib/importd/JobStrategy.py", line 84, in runtobaz
    totla.totla(config, logger, config.args, self.sourceTree())
  File "/srv/importd.ubuntu.com/dists/launchpad/lib/cscvs/cmds/totla.py", line 262, in totla
    apply_changeset(source_tree, logger, apply, revision, target_tree, revisions)
  File "/srv/importd.ubuntu.com/dists/launchpad/lib/cscvs/cmds/totla.py", line 130, in apply_changeset
    fileChange.apply(logger, target_tree, prune)
  File "/srv/importd.ubuntu.com/dists/launchpad/lib/svn_oo/PresentFile.py", line 46, in apply
    self.url(), self.revision)
  File "/srv/importd.ubuntu.com/dists/launchpad/lib/svn_oo/util.py", line 116, in writeFile
    revision=pysvn_revision, peg_revision=pysvn_revision)
pysvn._pysvn.ClientError: Code: 00020f58

Failed to fork, memory error

Some large imports fail to os.fork() (when spawning gpg to sign a revision) with "OSError: [Errno 12] Cannot allocate memory". It's not clear why that happen, since Linux should overcommit memory up to 50x (the value of /proc/sys/vm/overcommit_ratio) and allow large processes like cscvs to spawn small processes like gpg.

Fix in progress

Failures for which a fix is currently being worked on.

VcsImportRequests (last edited 2021-05-27 14:19:59 by cjwatson)