Re: possible LDAP over SSl bug in OS 10, 10.4, 10.5, 10.6

From: Schley Andrew Kutz (_at_)
Date: 11/16/04 

Date: Tue, 16 Nov 2004 07:13:40 -0600

I should point out that I mean 10.3, 10.3.4, 10.3.5, and 10.3.6

On 2004-11-16 07:11:01 -0600, Schley Andrew Kutz <s a NO kutz @ SPAM g
OR mail dOt ELSE c0m> said:

> This was submitted to Apple Bug Reporter on Oct. 23rd 2004.
>
> ---
>
> 23-Oct-2004 02:57 PM Schley Kutz:
> Summary:
> --------
> As I have been reminded, for all intensive purposes I represent the
> greater community of Mac users here at the University of Texas at
> Austin who want to get Entourage to be as feature-full as it is
> advertised to be. UT@Austin is the largest public university in the
> nation, and at one time it was the largest concentration of Macs in the
> nation as well. Hence, this is something that *really* needs to work.
>
> Microsoft released Entourage 2004 into the ready hands of users
> clamoring for a product that was the equivalent of Outlook 2003 and
> would give Mac users a portal to access Exchange. Entourage 2004 had
> some problems however. Particularly it did not let users securely
> access the GAL (Directory Access in Entourage) or securely acknowledge
> themselves as delegates. These were both LDAP over SSL issues. These
> problems were thought to be on Entourage's part.
>
> Then Service Pack 1 for Office 2004 came out. I applied it with great
> hopes that it would fix the LDAP over SSL issue that plagued so many
> users. It did not. I then composed a *very* long email to our
> Microsoft Technical Account Manager (TAM). The email detailed all the
> steps I had taken with Ethereal and ssltap to determine that Entourage
> could decidedly still not properly talk to LDAP over SSL. Towards the
> end of composing the email I decided to reconfirm that native OS X apps
> would access a LDAP server over SSL. Much to my surprise (since this
> had been tested before) Address Book could not talk to
> austin.utexas.edu via SSL.
>
> So the problem is deeper than Entourage 2004. The problem that I found
> seems to be an issue with the way that OS X's Directory Services
> framework interacts with OpenLDAP.
>
>
> Steps to Reproduce:
> -------------------
> Thus began my journey on a different path. I compiled ssldump for OS X
> and began examining different configurations of LDAP queries. I
> realized that I had to test these configurations on more than just our
> Active Directory so Eric Irrgang and Robert Kennedy of the ITS Unix
> group graciously assisted me with this. Eric set me up with an account
> on the Unix group's internal LDAP server, and Robert showed me some
> basic openldap command line commands such as ldapsearch. In fact, I
> used several programs to test LDAP over SSL functionality on OS X.
> They are:
>
> a) Address Book.app
> b) JXplorer
> c) ldapsearch
> d) Entourage 2004 - Directory Service
> e) Entourage 2004 - Delegates
>
> d and e may seem surprising but it is the case while Directory Service
> (the GAL) and Delegates in Entourage 2004 both issue the same type of
> LDAP requests (I watch unencrypted traffic in Ethereal to prove this),
> they produce different errors in ssldump when they do not work as
> advertised.
>
> There were three distinctly different results when trying to access
> LDAP over SSL:
>
> 1) LDAP over SSL works successfully and much giddiness ensued
>
> 2) LDAP over SSL works and ssldump reports (in several fashions) an
> Unknown Certificate Authority (ca|CA) error. This permutation will in
> some cases report an error to the user but in fact all the traffic is
> still encrypted and as far as Entourage's Directory Service is
> concerned the data is still retrieved.
>
> 3) LDAP over SSL fails and ssldump reports that an Unknown CA was the cause.
>
> 4) LDAP over SSL fails completely. This is what sparked the OS X
> investigation. What is odd about this error was the ssldump output.
> It showed that although the commands were being issued over port 636
> (ldaps port), the client was trying to initiate a NON SSL connection to
> the server. At this point the server basically says "Um, yes, I am
> terribly sorry sir but it would be entirely improper of me if I were to
> allow you to continue this obvious assault on my person. Please make
> haste in your efforts to retreat from my doorstep lest I be forced to
> commit a barbaric act of v101ence."
>
> I exported two Certification Authority certificates to Base64 format.
> They were the Secure Server Certification Authority certificate (SSCA)
> (a standard Verisign CA), which is used by our Active Directory, and
> the ITS Unix CA certificate, used by the Unix group's LDAP server. I
> imported these into Keychain Access's X509Anchors keychain (I removed
> the original SSCA CA from the keychain prior to adding mine) which
> should be used by Address Book.app and Entourage 2004. Next, I
> imported these into Jxplorer's Key Store. Finally, I used the
> TLS_CACERT option in my .ldaprc file to specify the CA to use when
> executing the OpenLDAP command ldapsearch. I then removed ALL of the
> certificates and ran the tests again in order to get a full suite of
> debugging information.
>
> Here is a simple chart of the different configurations I tested. I
> have attached text files of the logs for these configurations.
> Although I tested this on 5 different systems (see Isolation for OS
> versions), I only generated log files for my primary test system.
>
> For interpreting the chart please refer to the alpha and numeric scheme
> I used for lableling the applications I tested and the results I found.
>
> yca = with Certificate Authority certificate installed for the ldap
> server I was accessing
>
> nca = without Certificate Authority certificate installed for the ldap
> server I was accessing
>
>
> a | b | c | d | e |
> |----------|----------|----------|----------|----------|
> yca | 4 | 1 | 1 | 2 | 4 |
> |----------|----------|----------|----------|----------|
> nca | 4 | 3 | 3 | 2 | 4 |
> |----------|----------|----------|----------|----------|
>
>
> yca/a - Address Book.app fails completely nca/a - Address Book.app
> fails completely
>
> yca/b - JXplorer works
> nca/b - Jxplorer fails because of unknown CA
>
> yca/c - ldapsearch works
> nca/c - ldapsearch fails because of unknown CA
>
> yca/d - Entourage Directory Service throws an error but continues to
> return good data that IS encrypted nca/d - Entourage Directory Service
> throws an error but continues to return good data that IS encrypted
>
> yca/e - Entourage Delegates fails completely nca/e - Entourage
> Delegates fails completely
>
> So what do these results tell me? Well, what is MOST interesting is
> that 2 different parts of the application that started this all,
> Entourage, must use ENTIRELY DIFFERENT methods of doing a directory
> lookup. I know this because they throw completely different errors.
>
> What is telling about the Delegates error is that it is exactly the
> same as the Address Book.app's error. This got me thinking. How can I
> duplicate all of Entourage's errors (Address Book.app's error is the
> same as Entourage's Delegates error so if I can dupe the Delegates
> error I will have duped the Address Book.app error) errors with
> OpenLDAP's command line tool, ldapsearch?
>
> First, the not so shocking discovery. The error that the Entourage
> Directory Service gives is that it cannot find the CA. Well that is
> easy enough to reproduce. I just removed the TLS_CACERT definition
> that pointed to the SSCA cert that our AD uses from my .ldaprc file. I
> then defined TLS_CERT as 'request' and did an ldapsearch against our
> AD. This produced the exact same error that Entourage gives when using
> its Directory Service. Essentially, when Entourage issues a directory
> request from its Directory Service it gets to OpenLDAP as a call that
> does not use a CA but requests one. This causes the client to say
> "Hey, I know you have a CA to give me, but I don't have one to give
> you. Eh, don't worry about it. I'll take a warning sure, but we can
> still talk in that crazy encrypted lingo that those cats like to call
> SSL." The logs show a warning but all traffic still occurs and is
> still encrypted.
>
> When I removed the TLS_CACERT definition and executed the same command:
>
> ldapsearch -H "ldaps://server.name.utexas.edu:636" -W -D "CN=not
> real,OU=yeah right,OU=not on your
> life,DC=server,DC=name,DC=utexas,DC=edu"
>
> I received this error:
>
> TLS certificate verification: Error, unable to get local issuer certificate
>
> This just proves that it is at a different location than OpenLDAP that
> the request of the CA from KeyChain happens. I could not find a
> reference to X509 or Keychain (grep -ri) in any of the DirectoryAccess
> sources either. I could not find a header for OpenSSL on a clean 10.3
> install either. Hmmm. Duh! I finally found ssl.h in the DevSDK and
> openssl header includes in the DirectoryAccess headers. OpenSSL
> sources do not exist, that I found, in a clean install of OS X, but
> rather are compiled into /usr/bin/openssl and the OS X Directory
> Service.
>
> This still begs the question, why is the "Cannot find root certificate"
> error occurring? 1 or 2 of 2 things is happening. Either something is
> improperly using /usr/bin/openssl to generate a temporary root ca from
> KeyChain or something is improperly hooking into the OpenSSL libraries.
>
> Second, the discovery that left me saying, "Huh?" Error #4 was just
> odd. All the other ssldumps showed the client's first contact with the
> server as "C->S SSLv2 Compatible Hello (636)." Basically the client
> was talking to the server over 636 in SSL lingo, as it should. But the
> Address Book.app and Entourage's Delegates generated this log, "C->S
> (636)." It was as if the client was trying to issue LDAP commands with
> a normal ldap uri "ldap://" over 636. Well this won't fly cat. Nope
> man, you've got to use "ldaps://" if you want to dig on 636. At least
> with OpenLDAP you do. Anyway, so I imitated this with OpenLDAP. I
> executed this command to both LDAP servers:
>
> ldapsearch -H "ldap://server.name.utexas.edu:636" -W -D "CN=not
> real,OU=yeah right,OU=not on your
> life,DC=server,DC=name,DC=utexas,DC=edu"
>
> Voilla! Same error as Address Book.app and Entourage's Delegates. I
> know it sounds crazy, but it looks like those two programs are issuing
> LDAP commands with a non secure form of the ldap uri over port 636.
> This is just silly. Of course it won't work.
>
> How, you might ask, could this have happened? Well, really, it is not
> that hard to imagine. LDAP is a protocol that is used to access what
> has been, until recently, philosophically something that is "public"
> information. Directories, large ones especially, are by their nature
> meant to be open and read by the public. If a directory is small and
> needs to be secure there are better solutions than a LDAP enabled
> directory.
>
> Anyway.
>
> So I was able to deduce quite a bit, but I was still kind of ticked
> off. I may have possibly sort of perhaps figured out *what* was going
> on, but *where* is the on going? That is the real zinger of the month
> ... Well, as I mentioned, Entourage seems to hook into OS X's
> networking API at different places, depending on where it is calling it
> from. Also, I realized that if OpenLDAP is not grabbing goodies from
> X509Anchors it must be Directory Access doing the dirty deed.
>
> This is where my good friend and hated enemy Nicholas "St. Who?"
> Lauland came in. I absolutely despise him (tongue in cheek there) for
> informing me that "You can get the source code for Darwin Directory
> Service off of ADC. Well crap. I already knew that I could get the
> source for OpenLDAP (duh! ->OPEN<-LDAP), but this new revelation just
> meant that I was going to be compelled to see this through much further
> along than I was mentally capable of.
>
> Do you know what happens after you press "Search" on the Address
> Book.app and expect it to search a configured LDAP server?
>
> Neither do I. But I do know where things go from when
> CLDAPv3Plugin::ProcessRequest invokes CLDAPv3Plugin::HandleRequest.
> And that is really the point where the story can start anyway since
> CLDAPv3Plugin inherits the CServerPlugin class. Any LDAP-centric
> configuration is going to take place once a CLDAPv3Plugin object is
> instantiated.
>
> I have attached the OpenLDAP headers and sources that contain the
> functions I am going to reference. I also attached the
> DirectoryService sources and headers. The Adc membership it takes to
> request them is free, so whomever could actually go online an get them.
> Here is a full list of the headers and sources that contain the
> functions that I am going to reference. In fact, they are listed in
> the order in which they are invoked, so I will only list them here:
>
> CServerPlugin::ProcessRequest -
> DirectoryService-257.1/Server/CServerPlugin.h .cpp
>
> CServerPlugin::HandleRequest -
> DirectoryService-257.1/Server/CServerPlugin.h .cpp
>
> CLDAPv3Plugin::OpenDirNode -
> DirectoryService-257.1/Plugins/LDAPv3/CLDAPv3Plugin.h .cpp
>
> CLDAPNode::SafeOpen - DirectoryService-257.1/Plugins/LDAPv3/CLDAPNode.h .cpp
>
> CLDAPNode::AuthOpen - DirectoryService-257.1/Plugins/LDAPv3/CLDAPNode.h .cpp
>
> CLDAPNode::BindProc - DirectoryService-257.1/Plugins/LDAPv3/CLDAPNode.h .cpp
>
> ldap_bind - OpenLDAP-37.2.1/include/ldap.h /libraries/libldap/bind.c
>
> ldap_simple_bind - OpenLDAP-37.2.1/include/ldap.h /libraries/libldap/sbind.c
>
> ldap_sasl_bind - OpenLDAP-37.2.1/include/ldap.h /libraries/libldap/sasl.c
>
> ldap_send_initial_request - OpenLDAP-37.2.1/include/ldap.h
> /libraries/libldap/request.c
>
> ldap_send_server_request - OpenLDAP-37.2.1/include/ldap.h
> /libraries/libldap/request.c
>
> The error occurs in the last function. The code that throws the error
> is as follows:
>
> if ( srv == NULL ) {
> if ( !use_ldsb ) {
> ber_sockbuf_free( lc->lconn_sb );
> }
> LDAP_FREE( (char *)lc );
> ld->ld_errno = LDAP_SERVER_DOWN;
> return( NULL );
> }
>
> The text of the error reads, "Can't contact LDAP Server." The error is
> defined in OpenLDAP-37.2.1/include/ldap.h and looks like this:
>
> #define LDAP_SERVER_DOWN 0x51
>
>
> For laughs I decided to set Address Book.app and Entourage Delegates to
> use SSL over port 389. Guess what? Instead of acting in error like
> they do when you specify port 636/3239 (Global Catalog SSL, GCS) and
> trying to talk to the server without first initiating a compatible
> SSLv2 Hello, they issue a compatible SSLv2 Hello! However, I used
> ethereal and verified that the traffic is in fact traveling in the
> clear when you do this. BUT, it does cause Address Book.app and
> Entourage Delegates to issue the correct Hello.
>
> It seems to me that the combination of specifying SSL and talking over
> port 636 causes some type of confusion in the Directory Access API. I
> do not know where since I do not have the sources or headers for
> Address Book.app and/or Entourage.
>
> I confirmed the errors by duplicating the theorized process with
> ldapsearch. This was the command I used:
>
> ldapsearch -H "ldaps://server.name.utexas.edu:389" -W -D "CN=not
> real,OU=yeah right,OU=not on your
> life,DC=server,DC=name,DC=utexas,DC=edu"
>
> I also tested all of the above by adding a LDAPv3 entry into OS X's
> Directory Access preferences for austin.utexas.edu. I checked it by
> searching against OS X's Directory Services in Address Book.app and by
> doing a "lookupd -d <return> userWithName: akutz." Both failed.
>
>
> Expected Results:
> -----------------
>
> Well, that table I drew earlier should look like this:
>
> a) Address Book.app
> b) JXplorer
> c) ldapsearch
> d) Entourage 2004 - Directory Service
> e) Entourage 2004 - Delegates
>
> 1) LDAP over SSL works successfully and much giddiness ensued
>
> 2) LDAP over SSL works and ssldump reports (in several fashions) an
> Unknown Certificate Authority (ca|CA) error. This permutation will in
> some cases report an error to the user but in fact all the traffic is
> still encrypted and as far as Entourage's Directory Service is
> concerned the data is still retrieved.
>
> 3) LDAP over SSL fails and ssldump reports that an Unknown CA was the cause.
>
> 4) LDAP over SSL fails completely. This is what sparked the OS X
> investigation. What is odd about this error was the ssldump output.
> It showed that although the commands were being issued over port 636
> (ldaps port), the client was trying to initiate a NON SSL connection to
> the server. At this point the server basically says "Um, yes, I am
> terribly sorry sir but it would be entirely improper of me if I were to
> allow you to continue this obvious assault on my person. Please make
> haste in your efforts to retreat from my doorstep lest I be forced to
> commit a barbaric act of v101ence."
>
> yca = with Certificate Authority certificate installed for the ldap
> server I was accessing
>
> nca = without Certificate Authority certificate installed for the ldap
> server I was accessing
>
>
> a | b | c | d | e |
> |----------|----------|----------|----------|----------|
> yca | 1 | 1 | 1 | 1 | 1 |
> |----------|----------|----------|----------|----------|
> nca | 2/3 | 2/3 | 2/3 | 2/3 | 2/3 |
> |----------|----------|----------|----------|----------|
>
>
> yca/a - Address Book.app works
> nca/a - Address Book.app throws an error but continues to return good
> data that IS encrypted OR fails because of unknown CA
>
> yca/b - JXplorer works
> nca/b - Jxplorer throws an error but continues to return good data that
> IS encrypted OR fails because of unknown CA
>
> yca/c - ldapsearch works
> nca/c - ldapsearch throws an error but continues to return good data
> that IS encrypted OR fails because of unknown CA
>
> yca/d - Entourage Directory Service works nca/d - Entourage Directory
> Service throws an error but continues to return good data that IS
> encrypted OR fails because of unknown CA
>
> yca/e - Entourage Delegates works
> nca/e - Entourage Delegates throws an error but continues to return
> good data that IS encrypted OR fails because of unknown CA
>
> Why 2 OR 3 without the certificate authority? Because I OS X 's
> Directory Service should, and from what I can tell from my testing it
> apparently is not, respect any .ldaprc file in my homedir or any
> ldap.conf file in /etc/openldap/ldap.conf. The ldap.conf can be in 1
> or 2 more places, but the lid to my powerbook is closed, and I enjoy
> watching the button light throb too much to open the lid back up to
> bring up ldap.conf 's man page. Meaning Directory Service should
> respect whether or not I tell OpenLDAP to ignore TLS_REQCERT and
> TLS_CHECKPEER. The ldapsearch command does (of course) and anything
> that implements OpenLDAP should too.
>
> Actual Results:
> ---------------
>
> a) Address Book.app
> b) JXplorer
> c) ldapsearch
> d) Entourage 2004 - Directory Service
> e) Entourage 2004 - Delegates
>
> 1) LDAP over SSL works successfully and much giddiness ensued
>
> 2) LDAP over SSL works and ssldump reports (in several fashions) an
> Unknown Certificate Authority (ca|CA) error. This permutation will in
> some cases report an error to the user but in fact all the traffic is
> still encrypted and as far as Entourage's Directory Service is
> concerned the data is still retrieved.
>
> 3) LDAP over SSL fails and ssldump reports that an Unknown CA was the cause.
>
> 4) LDAP over SSL fails completely. This is what sparked the OS X
> investigation. What is odd about this error was the ssldump output.
> It showed that although the commands were being issued over port 636
> (ldaps port), the client was trying to initiate a NON SSL connection to
> the server. At this point the server basically says "Um, yes, I am
> terribly sorry sir but it would be entirely improper of me if I were to
> allow you to continue this obvious assault on my person. Please make
> haste in your efforts to retreat from my doorstep lest I be forced to
> commit a barbaric act of v101ence."
>
> yca = with Certificate Authority certificate installed for the ldap
> server I was accessing
>
> nca = without Certificate Authority certificate installed for the ldap
> server I was accessing
>
>
> a | b | c | d | e |
> |----------|----------|----------|----------|----------|
> yca | 4 | 1 | 1 | 2 | 4 |
> |----------|----------|----------|----------|----------|
> nca | 4 | 3 | 3 | 2 | 4 |
> |----------|----------|----------|----------|----------|
>
>
> yca/a - Address Book.app fails completely nca/a - Address Book.app
> fails completely
>
> yca/b - JXplorer works
> nca/b - Jxplorer fails because of unknown CA
>
> yca/c - ldapsearch works
> nca/c - ldapsearch fails because of unknown CA
>
> yca/d - Entourage Directory Service throws an error but continues to
> return good data that IS encrypted nca/d - Entourage Directory Service
> throws an error but continues to return good data that IS encrypted
>
> yca/e - Entourage Delegates fails completely nca/e - Entourage
> Delegates fails completely
>
>
> Workaround:
> -----------
>
> I have not found one. Nick suggested that I recompile OpenLDAP and
> throw in some extra debugging statements. I even thought of checking
> for the presence of port 636 and forcing the uri to start with "ldaps."
> Quite frankly, that's what the folks at Apple get paid to do : ) For
> now I know the limitations of OS X and subsequently Entourage 2004 /
> SP1.
>
> All we Entourage users here at the University can do is wait.
>
> Isolation:
> ----------
>
> The same errors have occurred on 5 different Macs. Here are they
> operating systems:
>
> 10.3.5
> 10.3.5
> 10.3.5
> 10.3.4
> 10.3
>
>
>
> 'one file to bind them.zip' and 'System Profile.spx'were successfully uploaded 

-- 
-a