Conflict in custom Twisted branches/dav-acl-1608-4 prevents CalendarServer from running

[s.marechal@…]

CalendarServer refuses to run due to problems with the branches/dav-acl-1608-4 branch of Twisted that CalendarServer uses. That branch uses twisted/python/reflect.py which in turn tries to import modules from the twisted that is installed system-wide instead of the twisted/branches/dav-acl-1608-4 branch that was donwloaded by the run -s script.

See the attached files for the errors. In short, branches/dav-acl-1608-4/python/reflect.py tries to import /usr/lib/python2.5/site-packages/twisted/plugins/cred_anonymous.py instead of branches/dav-acl-1608-4/twisted/plugins/cred_anonymous.py. This causes problems because the system-wide Twisted is 8.1.0. The cred plugins of 8.1.0 depend on twisted.cred.strcred but the two year old dav-acl-1608-4 doesn't have those. Result: Failure.

[s.marechal@…]

output of ./run showing the python twisted exceptions.

[s.marechal@…]

Cc Me!

[wsanchez@…]

OS/version?

[s.marechal@…]

Debian Lenny (testing), using the official calendarserver.org package instead of the one that comes from Debian.

[wsanchez@…]

We have an official Debian package? Where?

[s.marechal@…]

You do. It's right here: http://packages.debian.org/lenny/calendarserver

It suffers the same problem as the one I reported here, but Debian worked around it by simply conflicting with Twisted as a whole. That means I can't run CalendarServer and another Twisted app on the same server. And that's why I installed the original upstream version to begin with.

Any chance of running CalendarServer on a recent, standard Twisted instead of that two year old branch? If not, any chance of getting the required changes into Twisted trunk?

[wsanchez@…]

OK, we don't. debian.org != calendarserver.org. We don't have official packages for any platform, actually.

What we do on OS X is install a separate Twisted and put that in PYTHONPATH first. What works for us; not sure why it wouldn't work on Debian.

As to getting back into alignment with Twisted, I'd love to, but it's going to be a lot of work, and other things are occupying all of our time right now. I have been putting some thought into how to get there, but it's going to be a while, I think.

[s.marechal@…]

As I said, I am not using the debian package. I am using the CalendarServer I downloaded from CalendarServer.org

As you can see from the paths in the attached logfile it does use a separate Twisted in /usr/local/CalendarServer/Twisted/. The problem is that your twisted/python/reflect.py tries to import from the global Twisted instead of that separate Twisted. So either CalendarServer does not set PYTHONPATH correctly or there is a bug in your custom reflect.py that makes it ignore PYTHONPATH.

[s.marechal@…]

PS: The relevant snipper from the logfile is lines 19-23:

19 File "/usr/local/CalendarServer/Twisted/twisted/python/reflect.py", line 361, in namedAny 20 topLevelPackage = import(trialname) 21 File "/usr/lib/python2.5/site-packages/twisted/plugins/cred_anonymous.py", line 14, in <module> 22 from twisted.cred.strcred import ICheckerFactory 23 exceptions.ImportError: No module named strcred

[agx@…]

Replying to s.marechal@…:

You do. It's right here: http://packages.debian.org/lenny/calendarserver It suffers the same problem as the one I reported here, but Debian worked around it by simply conflicting with Twisted as a whole. That means I can't run CalendarServer and another Twisted app on the same server. And that's why I installed the original upstream version to begin with.

We're not conflicting on the whole twisted, merely on python-twisted-web2 and python-twisted-runner and we only ship these parts taking the rest from the twisted Debian package. The main reason is that if we ship the whole twisted we've got twice the security work and unexpectes side effects like the one you encountered. So nothing should really be stopping you from running another twisted app.

I'm not saying it's an ideal solution but probably the best we get until calendarserver catched up with upstream.

[s.marechal@…]

Replying to agx@…:

We're not conflicting on the whole twisted, merely on python-twisted-web2 and python-twisted-runner

I know. And I have my own thoughts about just how that was done. But I filed a bug about that in the Debian bug tracking system.

This bug report is about this upstream CalendarServer from calendarserver.org. It's not about the Debian package. There is either a bug in upstream calendarserver or in the dav-acl-1608-4 twisted branch that the "run" downloads. See comment 7.