kompozer import (strange one)
filed bug for bogus repository root detection
|Deletions are marked like this.||Additions are marked like this.|
|Line 53:||Line 53:|
|== [svn] wrong repository root ==
The current svn import code determines the root of the repository by walking up the URL until svn ls fails. This was necessary because older versions of libsvn_client did not expose the repository root information. Newer libsvn_client allow to determine this information directly. In cases where there are nested svn repositories, the old logic is wrong and that cause the change filtering (ignoring change outside of the branch being imported) to ignore legitimate changes.
|== [svn] bogus repository root detection ==
[https://bugs.launchpad.net/bugs/156744 Bug 156744].
This page tracks requests for actions from the vcs-imports operator.
To request an import, please see the instructions on VcsImports.
Note that we only do imports of MAIN branches on CVS (the form will have to be updated to remove the "branch" field) and trunk branches on Subversion.
We will handle your request and keep you informed of progress or issues. You can contact the vcs-imports operator: ddaa in #launchpad on irc.freenode.net.
Template to request an import:
[https://launchpad.net/people/<launchpad-user> <launchpad-user>]: [https://launchpad.net/products/<launchpad-product>/<series> <product>/<series>] BR <location> BR @DATE@
Imports that need intervention from the vcs-imports operator.
Imports being processed.
Imports that are currently failing, and for which no fix is currently being worked on.
Imports that failed with the following error:
python2.4: /build/buildd/subversion-1.3.1/subversion/libsvn_subr/subst.c:809: svn_subst_translate_stream3: Assertion `eol_str || keywords' failed. process killed by signal 6
It looks like a problem in the svn code for keyword expansion.
[https://launchpad.net/people/kamion kamion]: [https://launchpad.net/debian-installer/main debian-installer/main] BR svn://svn.debian.org/d-i/trunk/installer BR Date(2007-01-23T13:52:03Z) BR Note: the import is currently set up to import from a local copy of the repository: file:///home/importd/d-i.repo/trunk/installer
[svn] bogus repository root detection
[https://bugs.launchpad.net/bugs/156744 Bug 156744].
[svn] unreliable server
The current svn import code makes many connections and does not retry when one fails. If the server is in the least unreliable (one failure every few thousand attempts to create a session) that will cause the import to fail.
See [https://launchpad.net/bugs/120992 bug 120992] (use persistent svn_ra sessions and reconnect on failure).
It could be that some of these failures are caused by another specific bug. From a discussion with Jelmer:
> > > * Don't rely on listdir() kind results. Fixes imports of several > > > repositories. (#56647) > > I'm curious about this bug. Can you tell me more? > It turned out that the .kind member of dirents returned by > svn.ra.get_dir2() is (semi-?)random when used against older Subversion > servers. It would occasionally be 1 (svn.core.svn_node_file) even though > the entry in question was a directory. This took me a while to track > down. Instead, I've switched to always trying to open something as a > directory and catching the appropriate exception that indicates it's a > file. That's very interesting. A number of Launchpad imports fail in a not-deterministic way with "failed REPORT". Currently I just classify those failures as "unreliable server", but I was puzzled by a pattern I noticed: the failures did not happen in a really random way. It might be the same bug.
nobody: [https://launchpad.net/products/xorg-driver-synaptics/main xorg-driver-synaptics-main] BR https://oops.kerneljanitors.org/repos/synaptic/trunk BR Date(2006-12-06T10:40:09Z)
[svn] missing uri-encoding
cscvs uses the svn_client API to libsvn to retrieve data (on Date(2006-11-09T14:31:29Z)), but does not correctly uri-encode locations. So file names that contain characters that must be escaped cause import failures.
- A file called .#Makefile.in.1.4
[https://answers.edge.launchpad.net/~frederik-carlier Frederik Carlier]: [https://launchpad.net/products/opendental/trunk opendental/trunk] BR https://188.8.131.52:23793/svn/opendental/head BR Date(2007-10-16T22:02:30Z)
- A directory called "Forms Reports"
[svn] import stalls, or is too slow
The import system will kill any job that does produce output for "a long time" (4 hours, at time of writing, on Date(2007-02-15T17:32:46Z)). So if, for some reason, an import does not report progress for a long time, either because it is stuck or because it is really slow, it will fail.
[cvs] failed to open the modules file
cscvs needs to read the modules file to ignore configs (aka externals, nested trees).
$ cvs -d :pserver:guest:firstname.lastname@example.org:/cvs co -c cvs [checkout aborted]: failed to open the modules file
[cvs] file created by merge
Anchor(epiphany-browser)[https://launchpad.net/people/seb128 seb128]: https://launchpad.net/products/epiphany-browser/main BR :pserver:email@example.com:/cvs/gnome epiphany BR [Date(2006-10-16T14:54:17Z)]
- The cscvs log (with changesets) first mentions epiphany-toolbar at changeset MAIN.67, it refers to revision 1.2 of this file and records it as a 'M' (modification), which fails to apply since the file is not present in the imported tree at this point.
Relevant fragment of the cvs log:
$ cvs rlog epiphany/data\/ui/epiphany-toolbar.xml.in RCS file: /cvs/gnome/epiphany/data/ui/Attic/epiphany-toolbar.xml.in,v head: 1.17 branch: locks: strict access list: symbolic names: [...] eog-menu-api: 184.108.40.206 keyword substitution: kv total revisions: 19; selected revisions: 19 description: ---------------------------- [...] ---------------------------- revision 1.2 date: 2003/01/20 18:57:15; author: mpeseng; state: Exp; lines: +15 -0 2003-01-20 Marco Pesenti Gritti <firstname.lastname@example.org> * Merge eog-menu-api branch ---------------------------- revision 1.1 date: 2003/01/20 17:53:25; author: mpeseng; state: dead; branches: 1.1.2; file epiphany-toolbar.xml.in was initially added on branch eog-menu-api. ---------------------------- revision 220.127.116.11 date: 2003/01/20 17:53:25; author: mpeseng; state: Exp; lines: +15 -0 implement context menus ---------------------------- [...]
On changeset MAIN.30541 ValueError: attempt to patch non extant file : lib/VMCore/ConstantFold.cpp
On changeset MAIN.177 ValueError: attempt to patch non extant file : VMS/CONFIGURE.COM
On changeset MAIN.693 ValueError: attempt to patch non extant file : goo/gmem.cc
[svn] Cannot import whole repository
The svn import code fails at processing imports at the root of a svn repository. Some repository are set up that way, with the actual root containing the source code. That's broken, but people actually do this.
See bug [https://bugs.edge.launchpad.net/launchpad-cscvs/+bug/145240 145240].
[cvs] None branch in findLastFileRevision
While creating changesets: assertion error, branchName is None in findLastFileRevision CVS/CacheGenerator.py:160
It looks like it means something is really busted with filler revisions and default branches.
File name not supported by bzr
This is [https://bugs.launchpad.net/bzr/+bug/58180 bzr bug #58180]. When/if that gets fixed, all these could be retried.
dan_p on #launchpad: https://launchpad.net/moodle/trunk BR :pserver:email@example.com:/cvsroot/moodle moodle BR [Date(2007-05-25T09:35:05Z)] BR There are a few revisions with a file called \ in them: http://moodle.cvs.sourceforge.net/moodle/moodle/mod/data/field/text/%5C?hideattic=0&view=log#rev1.2
[https://edge.launchpad.net/~s-devrieze s-devrieze]: [https://launchpad.net/products/coccinella/trunk coccinella/trunk] BR :pserver:firstname.lastname@example.org:/cvsroot/coccinella coccinella BR Date(2007-09-18T20:25:38Z) BR Contains a file name with a newline. See [https://launchpad.net/bugs/3918 bug 3918].
[cvs] Absurd filler revision
cscvs creates a filler changeset 1.1->18.104.22.168 for include/adodb/pear/readme.Auth.txt, but there is no revision 22.214.171.124 for that file. The cvs rlog shows revision 1.1 (branches: 1.1.1) used in changeset MAIN.915, revision 126.96.36.199, not used by cscvs. The cvs log header for that file reads "head: 1.1" and "branch: 1.1.1".
(15:28:25) lifeless: I'd do a search in the file revision table for that file, all revisions, see what cscvs is thinking happened to it (15:29:45) ddaa: you suggest it might be a problem in the log parser and checking the catalog contents would help narrowing the cause of the problem? (15:33:11) lifeless: something is inventing the .2 (15:33:22) lifeless: if its in the catalog then the catalog generator can be checked (15:33:42) lifeless: if its not in the catalog then the application-of-changesets can be checked (15:33:45) lifeless: it will narrow it down (15:34:03) lifeless: As its a very small rlog I'd make a test case from it
cscvs creates a filler changeset 1.1->188.8.131.52 for gtetrinet/src/tetris.h but the log of that file only has revisions 1.1, 184.108.40.206, 220.127.116.11.2.2, 18.104.22.168.2.1.
[cvs] Checkout gave no file data
WARNING: N changeset MAIN.745 CRITICAL: checkout did not give file data and did not indicate file was removed. CRITICAL: requested checkout of reporoot '/cvs/gnome', file 'tomboy/www/img/intro.gif', revision '1.1' CRITICAL: log of checkout responses follow. CRITICAL: 'ok\n'
The cscvs log for this changeset:
ChangeSet: 745 Author: orph Date: 2004/09/18 08:52:21 Branch: MAIN CVSDefaultFiller Revisions: F www/img/intro.gif [1.1->1.1] +0-0 Filler changeset 1.1->1.1
[svn] repository password protected
[svn] svn:special with bad file content
http://ascendsvn.cheme.cmu.edu/ascend/code/trunk/pygtk/interface/config.guess@416 has svn:special set but contains real source, instead of special content.
[svn] ClientError: Code: 00020f58
Cryptic and mysterious error from libsvn. Completely reproducible.
Failure reproduced on Date(2007-04-05T17:54:02Z).
WARNING: change 166 Traceback (most recent call last): File "/home/importd/dists/launchpad/lib/importd/CommandLineRunner.py", line 67, in ? main() File "/home/importd/dists/launchpad/lib/importd/CommandLineRunner.py", line 64, in main doer.runMethod(method, dirname) File "/home/importd/dists/launchpad/lib/importd/CommandLineRunner.py", line 57, in runMethod getattr(self._job, method)(logger=self.logger, dir=dirname) File "/home/importd/dists/launchpad/lib/importd/Job.py", line 188, in runJob strategy(self, dir, logger) File "/home/importd/dists/launchpad/lib/importd/JobStrategy.py", line 102, in Import self.runtobaz("-SI", "%s.1" % aJob.branchfrom, target_path, logger) File "/home/importd/dists/launchpad/lib/importd/JobStrategy.py", line 86, in runtobaz totla.totla(config, logger, config.args, self.sourceTree()) File "/home/importd/dists/launchpad/lib/cscvs/cmds/totla.py", line 261, in totla apply_changeset(source_tree, logger, apply, revision, target_tree, revisions) File "/home/importd/dists/launchpad/lib/cscvs/cmds/totla.py", line 129, in apply_changeset fileChange.apply(logger, target_tree, prune) File "/home/importd/dists/launchpad/lib/svn_oo/PresentFile.py", line 27, in apply self.url(), self.revision) File "/home/importd/dists/launchpad/lib/svn_oo/util.py", line 90, in writeFile client.cat(url, revision=revision.rev), pysvn._pysvn.ClientError: Code: 00020f58 program finished with exit code 1
- On full-tree import of initial revision
- On changeset 1469.
- On changeset 1919.
- On changeset 43.
nobody: [https://launchpad.net/products/matchbox-window-manager/trunk matchbox-window-manager/trunk] BR http://svn.o-hand.com/repos/matchbox/trunk/matchbox-window-manager BR Date(2007-08-24T20:31:07Z)
- On first changeset (653).
nobody: [https://launchpad.net/products/geotools-mirror/trunk geotools-mirror/trunk] BR http://svn.geotools.org/geotools/trunk/gt BR Date(2007-10-09T08:13:18Z)
[cvs] unsupported connection method 'ext'
Regression in cscvs for the support of the :ext: connection method. Example traceback:
WARNING: CVS checkout: ['cvs', '-q', '-d', ':ext:email@example.com:/cvs', 'co', '-Pd', 'cvsworking', 'openssh'] WARNING: checking out: 32 kB WARNING: checking out: 1380 kB WARNING: checking out: 2892 kB CRITICAL: Repository base not available -- catalog created with an older cscvs? Rebuilding ERROR: protocol initialisation failed: Error: unsupported connection method 'ext' Traceback (most recent call last): File "/home/importd/dists/launchpad/lib/importd/CommandLineRunner.py", line 67, in ? main() [...] File "/srv/importd.ubuntu.com/dists/launchpad/lib/CVS/__init__.py", line 620, in newCatalog if self.repository().protocol() is not None: File "/srv/importd.ubuntu.com/dists/launchpad/lib/CVS/__init__.py", line 335, in protocol self._initProtocol() File "/srv/importd.ubuntu.com/dists/launchpad/lib/CVS/__init__.py", line 278, in _initProtocol self._protocol.initialiseServer() File "/srv/importd.ubuntu.com/dists/launchpad/lib/CVS/__init__.py", line 1114, in initialiseServer self._sendList([ "Root %s" % self._repository, File "/srv/importd.ubuntu.com/dists/launchpad/lib/CVS/__init__.py", line 1045, in _sendList self._connection.writeline(line); File "/srv/importd.ubuntu.com/dists/launchpad/lib/CVS/protocol.py", line 393, in writeline self._get_write_pipe().write("%s\n" % line) File "/srv/importd.ubuntu.com/dists/launchpad/lib/CVS/protocol.py", line 320, in _get_write_pipe return self._get_pipes() File "/srv/importd.ubuntu.com/dists/launchpad/lib/CVS/protocol.py", line 310, in _get_pipes raise Connection.Error( CVS.protocol.Error: unsupported connection method 'ext'
Problems with unicode filename
- Changeset 2616
File "/home/importd/dists/launchpad/lib/svn_oo/util.py", line 80, in addFile writeFile(target_tree, relpath, url, revision) File "/home/importd/dists/launchpad/lib/svn_oo/util.py", line 91, in writeFile filemode(url, revision)) File "/home/importd/dists/launchpad/lib/SCM/__init__.py", line 203, in write_file_breaking_links os.unlink(newname) UnicodeEncodeError: 'ascii' codec can't encode character u'\xa7' in position 96: ordinal not in range(128)
[svn] failed to find repository root
Failed to fork, memory error
Some large imports fail to os.fork() (when spawning gpg to sign a revision) with "OSError: [Errno 12] Cannot allocate memory". It's not clear why that happen, since Linux should overcommit memory up to 50x (the value of /proc/sys/vm/overcommit_ratio) and allow large processes like cscvs to spawn small processes like gpg.
- On changeset changeset 5343
- On changeset MAIN.15819
- On changeset 7
[svn] mystery 403
It may be that the server has DOS protection, and thinks we're attacking it.
WARNING: N changeset 531 Traceback (most recent call last): File "/home/importd/dists/launchpad/lib/importd/CommandLineRunner.py", line 67, in ? main() File "/home/importd/dists/launchpad/lib/importd/CommandLineRunner.py", line 64, in main doer.runMethod(method, dirname) File "/home/importd/dists/launchpad/lib/importd/CommandLineRunner.py", line 57, in runMethod getattr(self._job, method)(logger=self.logger, dir=dirname) File "/srv/importd.ubuntu.com/dists/launchpad/lib/importd/Job.py", line 188, in runJob strategy(self, dir, logger) File "/srv/importd.ubuntu.com/dists/launchpad/lib/importd/JobStrategy.py", line 108, in Import self._importIncrementally(target_path) File "/srv/importd.ubuntu.com/dists/launchpad/lib/importd/JobStrategy.py", line 153, in _importIncrementally self.runtobaz("-SC", "%s::" % lastCommit, target_path, self.logger) File "/srv/importd.ubuntu.com/dists/launchpad/lib/importd/JobStrategy.py", line 86, in runtobaz totla.totla(config, logger, config.args, self.sourceTree()) File "/srv/importd.ubuntu.com/dists/launchpad/lib/cscvs/cmds/totla.py", line 252, in totla apply_changeset(source_tree, logger, apply, revision, target_tree, revisions) File "/srv/importd.ubuntu.com/dists/launchpad/lib/cscvs/cmds/totla.py", line 122, in apply_changeset fileChange.apply(logger, target_tree, prune) File "/srv/importd.ubuntu.com/dists/launchpad/lib/svn_oo/PatchedFile.py", line 31, in apply svn_oo.util.writeFile( File "/srv/importd.ubuntu.com/dists/launchpad/lib/svn_oo/util.py", line 91, in writeFile filemode(url, revision)) File "/srv/importd.ubuntu.com/dists/launchpad/lib/svn_oo/util.py", line 42, in filemode proplist = client.propget('svn:executable', url, revision=revision.rev) pysvn._pysvn.ClientError: PROPFIND request failed on '/parrot/trunk/t/op/time.t' PROPFIND of '/parrot/trunk/t/op/time.t': 403 Forbidden (http://svn.perl.org)
Problems with filename containing ..
The programmatic API for adding files to a tree has different expectations than the user friendly API the command line uses ([https://bugs.launchpad.net/bzr/+bug/132639 bug 132639]). Depending on how this bug gets resolved, the following import could be retried after it is fixed.
[svn] cannot easily import from a url that does not contain 'trunk'
(see [https://bugs.launchpad.net/launchpad-bazaar/+bug/130883 bug 130883], which should be fixed very soon).
Fix in progress
Failures for which a fix is currently being worked on.