OpenLDAP directory service

[jusiskin@…]

To implement OpenLDAP as a directory service.

[wlynch@…]

I've been looking at implementing this. One of the big issues that I've been hitting, is digest authentication. Generally, I've seen the LDAP server store the password as a hash. So unless someone has an idea, I think that either this will probably be limited to basic authentication in most instances (unless you actually do store the password in plain text on the server).

[wsanchez@…]

In the OpenDirectory case, we do digest auth via some API in OpenDirectory, but I don't know how to do that with just LDAP. (Not an LDAP guy.)

But I agree that getting it working with basic auth is a good first step; one can at least use SSL (or kerberos).

[register@…]

I might be interested in contributing some effort to this. Simple authentication over SSL/TLS would be acceptable to me (although OpenLDAP will support anything that Cyrus SASL supports, which does include DIGEST.)

What other work needs to be done in order to support OpenLDAP? Would it be possible to generalize the schema requirements at all, say by allowing mapped attribute types and user-defined search filters?

[wsanchez@…]

The module needs to implement the IDirectoryService API (see source:CalendarServer/trunk/twistedcaldav/directory/idirectory.py).

The XML directory service is a pretty simple example: (source:CalendarServer/trunk/twistedcaldav/directory/xmlfile.py)

The OpenDirectory service is considerably more complicates, in large part due to a complicated schema that we implemented and now hope to get rid of: (source:CalendarServer/trunk/twistedcaldav/directory/appleopendirectory.py)

I haven't seen any work started on this yet, so I don't know of a module that starts this work from which you can pick up, so I think you have to start from scratch.

[rahul@…]

Hi, I have looked at the XML DirectoryService and NSS DirectoryService code. Having considerable experience with the python ldap library, I feel this is very much doable. And as LDAP Directory Service is needed in our company, I am thinking of implementing this. As it has been one month since the last post in this thread, I would like to know if there has been any progress on this front. Or whether I can go ahead with the implementation?

[brandon.evans@…]

I am very close to finishing up the LDAP modules I have been working on for the past couple months. The modules I have written drop into the CalendarServer-1.2 code base. The code is basically a copy of the appleopendirectory.py with a few changes that allow LDAP to be used rather than OpenDirectory. I was able to workout features such as Attendee & location auto complete, calendar delegation and auto schedule. I was planing on releasing it sometime in the next couple weeks. If you would like a copy of what I have done, or to discuss the LDAP directory modules, contact me.

[brandon.evans@…]

I am attaching the LDAP directory service modules I have created. These modules are functional, but may take some tuning to get to work with your setup. As is they work with the included schema. The Attendee auto complete, calendar delegation, and groups are some of the features that work. I plan on rewriting the ldapwrap.py and adding true ldap authentication in the future.

[brandon.evans@…]

Pervious upload was corrupt.

[ggm@…]

I know this project is not a democracy (joke!) but please PLEASE can you prioritize an adoption of this LDAP change? Requiring Apple Directory and the bizarre schema is making it very hard to integrate a singleton iCal server in an otherwise openldap world.

Since this change also provides PAM backed pmauth validation of the plaintext password, insecure though it is, I think it will make other people happy too: In principle, it removes the need for Kerberos doesn't it?

[wsanchez@…]

Note: "bizarre schema" is gone in trunk, though it still relies on OpenDirectory.

[ggm@…]

Replying to wsanchez@…:

Note: "bizarre schema" is gone in trunk, though it still relies on OpenDirectory.

many thanks. I will try to avoid pejoratives in comments in future.

-G

[wsanchez@…]

Meh, it was pretty bizarre.

[aymeric.augustin@…]

I've been developping a solution to plug calendarserver into a standard OpenLDAP installation under Debian for a few days. My goal is to have a module that works on *your* setup without any tuning apart from the configuration in caldavd.plist. I'm doing it because I couldn't figure out how Brandon's solution is working or what his schema is.

The main problem I've been tackling is: how to provide a flexible way to interpret the LDAP data as records of different types? For example, there are several ways to implement groups. I've designed a solution to specify all this data in caldavd.plist, assuming the most standard schemas are used. Basically, for each record type, I give the base DN and the name of the attribute for the RDN. Then there's quite a lot of glue code to implement the interface.

I'm attaching a diff file with my current implementation. It's a work in progress and I'm aware that several things need fixing, especially replace hardcoded stuff by configuration, implement the "proxy" stuff, and avoid synchonous operations (I do not really need caching at this time). I would really appreciate comments on my approach: - is the configuration understandable (see the changes in config.py)? - would that work with your LDAP setup?

If there is positive feedback, I'm willing to improve the code so that it can be integrated in calendarserver. Otherwise, maybe that will help with the "design issues" mentionned here:  http://lists.macosforge.org/pipermail/calendarserver-users/2007-May/000304.html

[aymeric.augustin@…]

Cc Me!

[soren@…]

Replying to jusiskin@…:

To implement OpenLDAP as a directory service.

Is it at all possible to use the ldap directory service along with Kerberos authentication, using the ldap plugins v.0.7?I ask because I'd really hate to lose single sign-on capabilities by having to rely on cleartext passwords instead of Kerberos tickets for auth processing ...

[rahul@…]

Replying to soren@…:

Replying to jusiskin@…:

To implement OpenLDAP as a directory service.

Yes it is possible. There is no special configuration for this. Also I have a small suggestion. Instead of using pwauth for authentication, PAM could be used. Below is a sample implementation of the verifyCredentials function.

import PAM

def verifyCredentials(self,credentials):
    """
    Verify that the given credentials can authenticate the principal
    represented by this record.
    @param credentials: the credentials to authenticate with.
    @return: C{True} if the given credentials match this record,
        C{False} otherwise.
    """

    def pam_conv(auth, query_list, userData):
        return [(credentials.password, 0)]

    auth = PAM.pam()
    auth.start("caldav") # PAM service name is "caldav"
    auth.set_item(PAM.PAM_USER, credentials.username)
    auth.set_item(PAM.PAM_CONV, pam_conv)
    try:
        auth.authenticate()
    except PAM.error, resp:
        return False
    else:
        return True

[aymeric.augustin@…]

patch-add-ldap-directory-service-to-twistedcaldav-1.2.diff

[aymeric.augustin@…]

Following up on my previous post... I just attached a new patch, with the code I am now running in production at several medium-sized companies.

All necessary settings are now configurable (for example email addresses generation for groups) and caching is implemented (the web interface is 6x faster on my setup, which is fairly standard).

Is it likely that this patch will be integrated into calendarserver, and would any additionnal work be necessary for this ? (more doc ?)

[wiget@…]

Cc Me!

[oxullo@…]

I'd like to add some lines about my personal experience trying to make OpenLDAP work with DCS. First of all, thanks to Aymeric and Brandon for their proposals. I tried both suggestions, with these objectives:

• To integrate iCal and TB+Lightning 1.0pre in a calendar system, using an OpenLDAP directory to hold user's data
• To have attendees' lookups work with both iCal and TB+L
• To reduce the complexity of users' schema (quite naive implication, but relevant to the fact that there are no tools, at least known to me, to easily maintain directory's data)

As reported in many forums ( 1,  2,  3,  4) iCal lookup troubles are caused by the lack of some information on which iCal rely, in order to set a proper scoping relating calendar server with server's users.

1. Brandon Evans' package

Finely documented installation process (wiki page here:  5). This suggestion seems to be the most integrated solution for iCal's requirements. However, I could not make it work: no explicit errors, but calendars placeholders are not automatically generated on document root's structure, so I end up with principals with no default calendar. I can provide more details to whom is interested. Moreover, my concern is: whenever this solution works, how am I supposed to maintain it platform independently? Resource info, service locators et cetera..

2. Aymeric Augustin's patch

Completely undocumented, but quite straightforward and at least simple solution for RFC2307 integration. I could make it work (at least with no groups, nor delegates) but clearly no lookup support for iCal. In this specific situation I peeked into iCal<>LDAP communication. Lookup request is performed with the correct filter and data are returned, but iCal simply discard them. TB+L lookups are damned easy and they work out of the box.

Relatively to iCal, it seems to me that the easiest way to fix this problem is to break the need of iCal to filter out contacts with high level metadata. Isn't possible just to configure this behavior?

Another thing that shakes me is that lookup for local groups works  4. I don't really want to end up to maintain a fake group list on each company's Mac!

1.  http://discussions.apple.com/thread.jspa?messageID=9201124
2.  http://www.zimbra.com/forums/isync-caldav/14812-ical-attendee-lookup-using-zimbra-directory.html
3.  http://forums.kerio.com/index.php?t=msg&goto=51672
4.  http://discussions.apple.com/message.jspa?messageID=9469770
5.  http://wiki.expertmx.com/doku.php?id=applecalendarserver

[aymeric.augustin@…]

You're right about the lack of documentation of my patch. Non-obvious configuration options should be documented, at least.

Groups are supported "flexibly". Here is how.

• If you don't use NIS groups, group records have member fields giving the dn of each user belonging to the group. This is the default configuration :

        <key>groupSchema</key>
        <!-- Base LDAP groups -->
        <dict>
          <key>membersAttr</key>
          <string>member</string>
          <key>memberIdAttr</key>
          <string></string>
        </dict>
  

• If you use NIS groups, group records have memberUid fields giving the uid of each user belonging the group. The dn of the user can be rebuilt with the information in the rdnSchema section of the configuration.

        <key>groupSchema</key>
        <!-- NIS groups -->
        <dict>
          <key>membersAttr</key>
          <string>memberUid</string>
          <key>memberIdAttr</key>
          <string>uid</string>
        </dict>
  

• Other representations could probably be mapped to one of these two.

Delegates are not supported, I'm not even sure what they are.

The configuration may be too generic and too flexible. However the defaults work out of the box. Do you have an opinion on this matter?

[oxullo@…]

Dear Aymeric,
thanks to the suggestion, I've successfully implemented groups, based on groupOfNames schema.

I think that the configuration is fine, at least with these assumptions:

1. the 'attr' key of every rdnSchema entries has to be the rdn of the element (wasn't so implicit to me!)
2. locations and resources don't work unless few lines are added to the method that create LDAP->principal mappings (patch follows)

This patch is a lame way to have a quick working support for resources and locations. If there is no need at all for code redundancy (dunno if check against cn is enough), the branch can be integrated with the group's one.

--- ldapdirectory.py.orig	2009-06-26 19:54:12.000000000 +0200
+++ ldapdirectory.py	2009-06-26 19:54:44.000000000 +0200
@@ -165,6 +165,12 @@
             fullName = self._getUniqueLdapAttribute(attrs, 'cn')
             calendarUserAddresses   = emailAddresses
             enabledForCalendaring   = True
+        elif (recordType == DirectoryService.recordType_resources or
+                recordType == DirectoryService.recordType_locations):
+            shortName = self._getUniqueLdapAttribute(attrs, 'cn')
+            fullName = self._getUniqueLdapAttribute(attrs, 'cn')
+            calendarUserAddresses   = emailAddresses
+            enabledForCalendaring   = True
 
         return LdapDirectoryRecord(
             service     = self,

Can you kindly suggest an example LDIF so we can refer ourselves to a specific and suggested schema? Right now I'm using:

• context containers (groups, locations, resources, users): organizationalUnit
• users: inetOrgPerson + simpleSecurityObject
• groups: groupOfNames
• resources: device
• locations: (suggestions? why room 's cn is "commonName"?)

Then, I gave a quick look to attendees support, which seems to be broken. Probably due to a missing email'S 'mailto:' specification for a fully qualified url. I will give a look to that tomorrow.

Finally, about delegates: this is the mechanism which allows one principal to have access to another principal. Right now, for instance and on a XMLDirectory instance, I have two locations and one resource configured as delegates to a group. Every person on this group can add each location and resource's calendar at ease and with no other credential set.

I wish that is worthy for someone to get this snippet published here as a template (avoiding pydict to plist boring mapping):

<key>DirectoryService</key>
<dict>
  <key>type</key>
  <string>twistedcaldav.directory.ldapdirectory.LdapDirectoryService</string>
  <key>params</key>
  <dict>
      <key>credentials</key>
      <dict>
         <key>dn</key>
         <string>cn=admin,dc=example,dc=com</string>
         <key>password</key>
         <string>secretWord</string>
      </dict>

      <key>groupSchema</key>
      <dict>
         <key>membersAttr</key>
         <string>member</string>
         <key>memberIdAttr</key>
         <string></string>
      </dict>

      <key>rdnSchema</key>
      <dict>
         <key>base</key>
         <string>dc=example,dc=com</string>
         
         <key>users</key>
         <dict>
             <key>rdn</key>
             <string>ou=users</string>
             <key>attr</key>
             <string>cn</string>
             <key>emailSuffix</key>
             <string></string>
         </dict>
         
         <key>groups</key>
         <dict>
             <key>rdn</key>
             <string>ou=groups</string>
             <key>attr</key>
             <string>cn</string>
             <key>emailSuffix</key>
             <string></string>
         </dict>
         
         <key>resources</key>
         <dict>
             <key>rdn</key>
             <string>ou=resources</string>
             <key>attr</key>
             <string>cn</string>
             <key>emailSuffix</key>
             <string></string>
         </dict>
         
         <key>locations</key>
         <dict>
             <key>rdn</key>
             <string>ou=locations</string>
             <key>attr</key>
             <string>cn</string>
             <key>emailSuffix</key>
             <string></string>
         </dict>
      </dict>
  </dict>
</dict>

[oxullo@…]

Just as a quick suggestion for whoever may be interested to use delegation with Aymeric's patch.

As delegation works with proxies and due to the fact that they can be specified by client side, there is a simple way to add them using CalDAVClientLibrary, as described briefly here  1.

Opendirectory interface, as well XMLDirectory one, allows to create proxies via configuration among resources/locations and users or groups. It would be nice to have the same behavior for LDAPDirectory, but I'm struggling with schemas (how to specify such linkage?). Any suggestion is welcomed!

1.  http://www.mail-archive.com/calendarserver-users@lists.macosforge.org/msg00798.html

[wsanchez@…]

Replying to oxullo@…:

Opendirectory interface, as well XMLDirectory one, allows to create proxies via configuration among resources/locations and users or groups.

Note that in trunk, we are moving proxy management out of the directory.

[oxullo@…]

Ok, therefore is more sane to get focused on the proxies' client-side definition.

The reason why this mechanism does not work with iCal and its convenient "Manage access to my account" interface is still due to failing lookups of the user on the directory, is that right? And with no opendirectory serving the lookups there is no way to add proxies with iCal, still right?

thanks for the suggestion

[wsanchez@…]

Leopard iCal required OpenDirectory for attendee lookups and proxies ("delegates").

Snow Leopard iCal will do this via DAV to the calendar server, which means it should work with all the directory plugins.

[rahul@…]

Hi Aymeric, Firstly, thanks for the patch you have written. This is extremely standard based and should be able to integrate with any LDAP implementation using RFC2307bis schema. I highly recommend that this patch be merged with the main calendarserver trunk. Anyway, I have two queries reg. this patch.

1. Is it possible to do free-busy lookups? I have used OpenLDAP with TB 3.0beta2 + Lightning 1.0pre2. While inviting attendees, and adding them, the free-busy schedule is not shown? Is this a bug? Also is free-busy lookup the same as attendee lookup?

2. Does your code take into consideration primary group membership? Because from what I know, the gidNumber of the user object class represents the primary group of that user and all the groups which have the user's dn listed in their uniqueMember attribute are secondary groups.

Looking forward to a response.

Regards, Rahul.

Replying to aymeric.augustin@…:

Following up on my previous post... I just attached a new patch, with the code I am now running in production at several medium-sized companies. All necessary settings are now configurable (for example email addresses generation for groups) and caching is implemented (the web interface is 6x faster on my setup, which is fairly standard). Is it likely that this patch will be integrated into calendarserver, and would any additionnal work be necessary for this ? (more doc ?)

[oxullo@…]

Rahul, availability's lookup doesn't work due to a small bug: every mail should be stored in the directory record object (after being read from LDAP) with RFC2368 scheme. Please note that I had an issue with TB3b2+L1p2 (due to a Lightning's bug, filed as 435854  1): whenever an user has a calendar with an empty inbox, Lightning starts to burst the server with polls. Can you tell me if you experience the same?

I attach a patch which:

• fixes availability lookup
• fixes the support for resources and locations
• adds the possibility to create delegation of resources and location via CalDAV compliant clients (like CalDAVClientLibrary)

1.  https://bugzilla.mozilla.org/show_bug.cgi?id=435854

[oxullo@…]

Aymeric's patch fixes

[rahul@…]

Replying to oxullo@…:

Rahul, availability's lookup doesn't work due to a small bug: every mail should be stored in the directory record object (after being read from LDAP) with RFC2368 scheme. Please note that I had an issue with TB3b2+L1p2 (due to a Lightning's bug, filed as 435854  1): whenever an user has a calendar with an empty inbox, Lightning starts to burst the server with polls. Can you tell me if you experience the same? I attach a patch which: * fixes availability lookup * fixes the support for resources and locations * adds the possibility to create delegation of resources and location via CalDAV compliant clients (like CalDAVClientLibrary) 1.  https://bugzilla.mozilla.org/show_bug.cgi?id=435854

Thanks for the patch Oxullo. It worked perfectly :). I tested for the bug which you have mentioned. I did not experience anything as such. But I am not sure if I followed the procedure for testing it properly. We shall be using this anyway in a production environment. So I should be able to give you feedback once it is deployed.

[rahul@…]

I have modified the patch given to me by Oxullo to include LDAP TLS support as well as filters. Also authentication is done using PAM rather than LDAP. I have only commented out the LDAP authentication code just in case you intend to revert to using LDAP server for authentication. The configuration options are as below now (I have included a sample filter option as well). Also the tlsCACertDir option does not seem to be working (no idea as to why this option is not working).

  <key>DirectoryService</key>
  <dict>
    <key>type</key>
    <string>twistedcaldav.directory.ldapdirectory.LdapDirectoryService</string>
  
    <key>params</key>
    <dict>
      <key>realmName</key>
      <string>EXAMPLE.COM</string>
      <key>uri</key>
      <string>ldap://dbs.example.com:389/</string>
      <key>tls</key>
      <true/>
      <key>tlsCACertFile</key>
      <string>/etc/ssl/certs/dbs.example.com.pem</string>
      <key>tlsCACertDir</key>
      <string></string>
      <key>tlsRequireCert</key>
      <string>demand</string>
      <key>credentials</key>
      <dict>
        <key>dn</key>
        <string>cn=admin,dc=example,dc=com</string>
        <key>password</key>
        <string>admin123</string>
      </dict>
      <key>rdnSchema</key>
      <dict>
        <key>base</key>
        <string>dc=example,dc=com</string>
        <key>users</key>
        <dict>
          <key>rdn</key>
          <string>ou=People</string>
          <key>attr</key>
          <string>uid</string>
          <key>emailSuffix</key>
          <string></string>
          <key>filter</key>
          <string>(&amp;(objectClass=x-scs-Person)(x-scs-PrivilegeName=caldav)(!(x-scs-AccountInactive=TRUE)))</string>
        </dict>
        <key>groups</key>
        <dict>
          <key>rdn</key>
          <string>ou=Group</string>
          <key>attr</key>
          <string>cn</string>
          <key>emailSuffix</key>
          <string></string>
          <key>filter</key>
          <string></string>
        </dict>
        <key>locations</key>
        <dict>
          <key>rdn</key>
          <string>ou=Locations</string>
          <key>attr</key>
          <string>cn</string>
          <key>emailSuffix</key>
          <string></string>
          <key>filter</key>
          <string></string>
        </dict>
        <key>resources</key>
        <dict>
          <key>rdn</key>
          <string>ou=Resources</string>
          <key>attr</key>
          <string>cn</string>
          <key>emailSuffix</key>
          <string></string>
          <key>filter</key>
          <string></string>
        </dict>
      </dict>
      <key>groupSchema</key>
      <dict>
        <key>membersAttr</key>
        <string>uniqueMember</string>
        <key>memberIdAttr</key>
        <string></string>
      </dict>
    </dict>
  </dict>

[rlalkaka@…]

Cc Me!

[rlalkaka@…]

Replying to rahul@…:

I have modified the patch given to me by Oxullo to include LDAP TLS support as well as filters. Also authentication is done using PAM rather than LDAP. I have only commented out the LDAP authentication code just in case you intend to revert to using LDAP server for authentication. The configuration options are as below now (I have included a sample filter option as well). Also the tlsCACertDir option does not seem to be working (no idea as to why this option is not working).

I'm having some trouble getting this working. Leaving the credentials field blank (which the comments in twistedcaldav/directory/ldapdirectory.py imply will anonymously bind) prompts ./run to spew errors about the blank credentials DN (log and trace below). Is there an easy way to change the patched LDAP to bind to uid=<username>,ou=people,dc=example,dc=com and then proceed with PAM auth if the record exists, instead of searching as in the current implementation? Apologies if this is a trivial change -- I can't make head or tail of LDAP right now.

[startup] Configuring directory service of type: twistedcaldav.directory.ldapdirectory.LdapDirectoryService
[LdapDirectoryService] Calling ldap.initialize('ldap://ldap.server-removed.com:389/').
[-] Traceback (most recent call last):
[-]   File "../Twisted/bin/twistd", line 21, in ?
[-]     run()
[-]   File "/tmp/src/Twisted/twisted/scripts/twistd.py", line 27, in run
[-]     app.run(runApp, ServerOptions)
[-]   File "/tmp/src/Twisted/twisted/application/app.py", line 379, in run
[-]     runApp(config)
[-]   File "/tmp/src/Twisted/twisted/scripts/twistd.py", line 23, in runApp
[-]     _SomeApplicationRunner(config).run()
[-]   File "/tmp/src/Twisted/twisted/application/app.py", line 157, in run
[-]     self.application = self.createOrGetApplication()
[-]   File "/tmp/src/Twisted/twisted/application/app.py", line 202, in createOrGetApplication
[-]     ser = plg.makeService(self.config.subOptions)
[-]   File "/tmp/src/CalendarServer-1.2/twistedcaldav/tap.py", line 749, in makeService
[-]     service = serviceMethod(options)
[-]   File "/tmp/src/CalendarServer-1.2/twistedcaldav/tap.py", line 471, in makeService_Slave
[-]     baseDirectory = directoryClass(**config.DirectoryService["params"])
[-]   File "/tmp/src/CalendarServer-1.2/twistedcaldav/directory/ldapdirectory.py", line 144, in __init__
[-]     self._updateStorage(recordType)
[-]   File "/tmp/src/CalendarServer-1.2/twistedcaldav/directory/ldapdirectory.py", line 238, in _updateStorage
[-]     logging.info("Retrieving subtree of %s." % ldap.dn.dn2str(base), system="LdapDirectoryService")
[-] AttributeError: 'module' object has no attribute 'dn'

[naoh123@…]

Sorry if this isn't the place, but is the LDAP patch at all applicable to CalendarServer 2.x?

[rahul@…]

The patch should work exactly this way. I have just tested it. The dn and password entries should be present in the configuration file but their values should be empty. As per the code, if the user is not found in the ldap database, authentication does not proceed.

Replying to rlalkaka@…:

Replying to rahul@…:

I have modified the patch given to me by Oxullo to include LDAP TLS support as well as filters. Also authentication is done using PAM rather than LDAP. I have only commented out the LDAP authentication code just in case you intend to revert to using LDAP server for authentication. The configuration options are as below now (I have included a sample filter option as well). Also the tlsCACertDir option does not seem to be working (no idea as to why this option is not working).

I'm having some trouble getting this working. Leaving the credentials field blank (which the comments in twistedcaldav/directory/ldapdirectory.py imply will anonymously bind) prompts ./run to spew errors about the blank credentials DN (log and trace below). Is there an easy way to change the patched LDAP to bind to uid=<username>,ou=people,dc=example,dc=com and then proceed with PAM auth if the record exists, instead of searching as in the current implementation? Apologies if this is a trivial change -- I can't make head or tail of LDAP right now. {{{ [startup] Configuring directory service of type: twistedcaldav.directory.ldapdirectory.LdapDirectoryService [LdapDirectoryService] Calling ldap.initialize(' ldap://ldap.server-removed.com:389/'). [-] Traceback (most recent call last): [-] File "../Twisted/bin/twistd", line 21, in ? [-] run() [-] File "/tmp/src/Twisted/twisted/scripts/twistd.py", line 27, in run [-] app.run(runApp, ServerOptions) [-] File "/tmp/src/Twisted/twisted/application/app.py", line 379, in run [-] runApp(config) [-] File "/tmp/src/Twisted/twisted/scripts/twistd.py", line 23, in runApp [-] _SomeApplicationRunner(config).run() [-] File "/tmp/src/Twisted/twisted/application/app.py", line 157, in run [-] self.application = self.createOrGetApplication() [-] File "/tmp/src/Twisted/twisted/application/app.py", line 202, in createOrGetApplication [-] ser = plg.makeService(self.config.subOptions) [-] File "/tmp/src/CalendarServer-1.2/twistedcaldav/tap.py", line 749, in makeService [-] service = serviceMethod(options) [-] File "/tmp/src/CalendarServer-1.2/twistedcaldav/tap.py", line 471, in makeService_Slave [-] baseDirectory = directoryClass(**config.DirectoryServiceparams?) [-] File "/tmp/src/CalendarServer-1.2/twistedcaldav/directory/ldapdirectory.py", line 144, in init [-] self._updateStorage(recordType) [-] File "/tmp/src/CalendarServer-1.2/twistedcaldav/directory/ldapdirectory.py", line 238, in _updateStorage [-] logging.info("Retrieving subtree of %s." % ldap.dn.dn2str(base), system="LdapDirectoryService") [-] AttributeError: 'module' object has no attribute 'dn' }}}

[rahul@…]

I would like to know whether anyone is using this patch in a production environment. I am using this patch with calendarserver in a company with about 250 users simultaneously connected with Lightning. The error log shows that authentication has failed for about 4-5 users and it keeps on retrying continuously. The calendar server process initially starts normally but suddenly its cpu usage goes to above 90%.

The server is a Intel Core 2 1.86 Ghz CPU with more than 3 GB RAM.

I am trying to figure out the possible reason for the high CPU usage.

1. The server configuration is not sufficient for 250 simultaneous users. 2. The ldap patch is coded in a way which is leading to the high CPU usage. 3. The repeated authentication attempts is leading to the high CPU usage.

Any help in this regard would be highly appreciated.

[rahul@…]

I have figured out the problem. I could probably say the problem is with the client (Thunderbird beta+Lightning). When connection to the calendar server fails because the username does not exist or if the authentication fails because the wrong password was saved in Thunderbird, the client should stop connecting (or try after some time). But instead it kept on trying to reconnect which affected the calendar server in two ways:

1. The major affect was when clients which were configured with usernames not existing in the ldap directory kept trying to connect. As the username was not found in the cache, the entire cache for users was being refreshed by the ldap patch in this page. And as the client kept on trying to reconnect upon failure, the cache was refreshed continuously which increased the cpu usage tremendously.

2. Also upon authentication failure, the client kept retrying which also lead to an increase in CPU usage. But this overload was not as high as in the previous case.

The obvious fix is to fix the code with client (Thunderbird and/or Lightning). When connection fails 2 or 3 times for whatever reason (username does not exist, wrong password entered, etc.), connection attempts should be stopped.

Also some code modification could be done on the server side to limit connections from such misconfigured clients so that other users do not face any inconvenience because of these small number of badly configured clients

Replying to rahul@…:

I would like to know whether anyone is using this patch in a production environment. I am using this patch with calendarserver in a company with about 250 users simultaneously connected with Lightning. The error log shows that authentication has failed for about 4-5 users and it keeps on retrying continuously. The calendar server process initially starts normally but suddenly its cpu usage goes to above 90%. The server is a Intel Core 2 1.86 Ghz CPU with more than 3 GB RAM. I am trying to figure out the possible reason for the high CPU usage. 1. The server configuration is not sufficient for 250 simultaneous users. 2. The ldap patch is coded in a way which is leading to the high CPU usage. 3. The repeated authentication attempts is leading to the high CPU usage. Any help in this regard would be highly appreciated.

[rahul@…]

Updated patch to address AD ldap connectivity issues and other minor fixes

[elesueur@…]

I'm looking at using this patch as well - Am I correct in assuming that it won't apply to the current trunk? I need to checkout version 1.2 and apply it to that tree?

[rahul@…]

I have uploaded the patch (ldapdirectory.patch) which works against calendarserver 2.3. This might work against trunk as well. Some points to note:

1. To use this patch, use urls of the from  http://server.example.com:8008/calendars/users/admin/... and not  http://server.example.com:8008/principals/__uids__/85cba509-fdaa-5cae-b4ea-fbeb4612dd50/... because support for querying the directory using guid is not added currently.

2. For authentication, PAM is used (the pam service name is "caldav"). If you want to use ldap server for authentication, you can just uncomment the LDAP authentication code and comment out the PAM authentication code in verifyCredentials function.

This patch has not been extensively tested. So you might find some bugs in which case kindly post in this thread.

[elesueur@…]

thanks for that - much appreciated.

How do I configure CalendarServer to use this ldapdirectory authentication method?

[rahul@…]

I have modified the patch to add two new options - authMethod and guidAttr. If guidAttr is defined, then the directory can be queried on guid as well i.e. urls of the form  http://server.example.com:8008/principals/__uids__/85cba509-fdaa-5cae-b4ea-fbeb4612dd50/ could be used without any worry.

Also I am attaching a sample caldav.plist to list the complete set of options (listed within ldap directory service).

[rahul@…]

A quick note. The possible values for "authMethod" currently are "PAM" and "LDAP".

Replying to rahul@…:

I have modified the patch to add two new options - authMethod and guidAttr. If guidAttr is defined, then the directory can be queried on guid as well i.e. urls of the form  http://server.example.com:8008/principals/__uids__/85cba509-fdaa-5cae-b4ea-fbeb4612dd50/ could be used without any worry. Also I am attaching a sample caldav.plist to list the complete set of options (listed within ldap directory service).

[rahul@…]

Sample caldav.plist file listing the ldap directory service configuration options

[elesueur@…]

thanks, I was getting confused about the authentication section in the caldavd.plist config file. I eventually figured it out!

[rahul@…]

Patch for using ldap directory (such as Openldap) with PAM (for authentication)

[rahul@…]

Cc Me!