Client certificate authentication fails with SVN and OpenSSL 1.0.1

[gadamiak]

Hi,

My Subversion repositories running under Apache 2.2.22 on Ubuntu Server 12.04 amd64 can't be accessed any more with TortoiseSVN above 1.7.6 (and CLI SVN) when client certificate authentication is required. The connection fails leaving almost no trace in Apache logs:

~~~
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 read client hello A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write server hello A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write certificate A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write key exchange A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write certificate request A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 flush data
[debug] ssl_engine_io.c(1908): OpenSSL: I/O error, 5 bytes expected to read on BIO#7f8df0001340 [mem: 7f8df8006c73]
[debug] ssl_engine_kernel.c(1903): OpenSSL: Exit: error in SSLv3 read client certificate A
[debug] ssl_engine_kernel.c(1903): OpenSSL: Exit: error in SSLv3 read client certificate A
~~~

Strangely the same setup works fine when client certificate authentication is switched off. The server and client certificates are issued by a self-signed CA. The CA chain is:

Primary CA
→ Intermediate server CA → server certificates
→ Intermediate user CAs → user certificates

The same setup has been running for almost 4 years without any problems till TSVN 1.7.6 using OpenSSL 1.0.0g. Later TSVN versions use OpenSSL 1.0.1x with 1.0.1e being the latest. The same happens despite HTTP library being used (errors from CLI svn client shipped with latest TSVN 1.7.13):

* Neon 0.29.6 returns error:

~~~
svn: E175002: Unable to connect to a repository at URL 'https://...'
svn: E175002: OPTIONS of 'https://...': SSL handshake failed: SSL error: unsupported algorithm nid (https://...)
~~~

* Serf (don't know the version) returns error:

~~~
svn: E120171: Unable to connect to a repository at URL 'https://...'
svn: E120171: Error running context: APR does not understand this error code
~~~

Where is the source of the problem?

I know there have been changes in OpenSSL 1.0.1 handling of self-signed certificates. Is this related? If so, what has to be changed/added to user certificates to make them work? Or are Neon and Serf libraries failing?

Definitely need help with that. Thanks.

[gadamiak]

I tested the same configuration with `mod_gnutls` instead of `mod_ssl` but to no avail. However mod_gnutls returns different error in Apache log (repeated a few times for a connection):

~~~
[info] [client ...] GnuTLS: Handshake Failed (-9) 'A TLS packet with unexpected length was received.'
~~~

Client errors are exactly the same.

Here are versions of Apache mods (namely packages in Ubuntu 12.04):

* libssl1.0.0: 1.0.1-4ubuntu5.9 (http://packages.ubuntu.com/precise/libssl1.0.0)
* libapache2-mod-gnutls: 0.5.10-1 (http://packages.ubuntu.com/precise/libapache2-mod-gnutls)

[gadamiak]

Did more testing with latest SVN 1.8.0 rc3 (commandline) downloaded from https://www.visualsvn.com/files/beta/Apache-Subversion-1.8.0-rc3.zip. SVN 1.8 uses only Serf library.

The effect is the same: no connection. However different errors are reported by both the SVN CLI:

~~~
svn: E120171: Unable to connect to a repository at URL 'https://...'
svn: E120171: Error running context: An error occurred during SSL communication
~~~

and Apache server:

~~~
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 read client hello A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write server hello A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write certificate A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write key exchange A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write certificate request A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 flush data
[debug] ssl_engine_io.c(1897): OpenSSL: read 5/5 bytes from BIO#7f3b3c001340 [mem: 7f3b3c006c73] (BIO dump follows)
[debug] ssl_engine_io.c(1830): +-------------------------------------------------------------------------+
[debug] ssl_engine_io.c(1869): | 0000: 16 03 03 00 07 ..... |
[debug] ssl_engine_io.c(1875): +-------------------------------------------------------------------------+
[debug] ssl_engine_io.c(1897): OpenSSL: read 7/7 bytes from BIO#7f3b3c001340 [mem: 7f3b3c006c78] (BIO dump follows)
[debug] ssl_engine_io.c(1830): +-------------------------------------------------------------------------+
[debug] ssl_engine_io.c(1869): | 0000: 0b 00 00 03 .... |
[debug] ssl_engine_io.c(1873): | 0007 - <SPACES/NULS>
[debug] ssl_engine_io.c(1875) +-------------------------------------------------------------------------+
[debug] ssl_engine_kernel.c(1884): OpenSSL: Write: SSLv3 read client certificate B
[debug] ssl_engine_kernel.c(1903): OpenSSL: Exit: error in SSLv3 read client certificate B
[debug] ssl_engine_kernel.c(1903): OpenSSL: Exit: error in SSLv3 read client certificate B
~~~

Any clues? Maybe thats something wrong for OpenSSL 1.0.1 with either of certificates?

As previously, all works fine unless client certificate authentication is required.

[c***@shelton-family.net]

Post by gadamiak
Hi,
~~~
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 read client hello A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write server hello A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write certificate A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write key exchange A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write certificate request A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 flush data
[debug] ssl_engine_io.c(1908): OpenSSL: I/O error, 5 bytes expected to read on BIO#7f8df0001340 [mem: 7f8df8006c73]
[debug] ssl_engine_kernel.c(1903): OpenSSL: Exit: error in SSLv3 read client certificate A
[debug] ssl_engine_kernel.c(1903): OpenSSL: Exit: error in SSLv3 read client certificate A
~~~
Primary CA
→ Intermediate server CA → server certificates
→ Intermediate user CAs → user certificates
~~~
svn: E175002: Unable to connect to a repository at URL 'https://...'
svn: E175002: OPTIONS of 'https://...': SSL handshake failed: SSL error: unsupported algorithm nid (https://...)
~~~
~~~
svn: E120171: Unable to connect to a repository at URL 'https://...'
svn: E120171: Error running context: APR does not understand this error code
~~~
Where is the source of the problem?
I know there have been changes in OpenSSL 1.0.1 handling of self-signed certificates. Is this related? If so, what has to be changed/added to user certificates to make them work? Or are Neon and Serf libraries failing?
Definitely need help with that. Thanks.

Is it possible that the changes with SSL/TLS negotiation that were added in openssl 1.0.1 are causing these issues? There is a good discussion of the scope of this issue, and a couple of possible workarounds here:
https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/965371

The result of this issue is that by default, clients using openssl 1.0.1 cannot negotiate a secure connection with any servers running openssl older than 1.0.1 unless there are additional ssl/tls related options passed to the client programs.

In your case, the client certificate request is likely causing your server to act like a client when making a request for the certificate.

You can see if this is the issue by using this kind of a basic ssl connectivity test:
openssl s_client -connect www.evernote.com:443
If you can connect, there will be some details about the certificates provided by www.evernote.com (or whatever server you are connecting to). If you just get an error message, the issue is with the openssl layer (lower than neon/serf).

[gadamiak]

Post by c***@shelton-family.net
https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/965371

I've read the discussion but it seems to apply to running a client on Ubuntu, while my issue is on Ubuntu Server with Windows clients.

Post by c***@shelton-family.net
The result of this issue is that by default, clients using openssl 1.0.1 cannot negotiate a secure connection with any servers running openssl older than 1.0.1 unless there are additional ssl/tls related options passed to the client programs.
In your case, the client certificate request is likely causing your server to act like a client when making a request for the certificate.

I've run a few connectivity test as you suggested and all succeeded! I checked:

* OpenSSL 1.0.1e 11 Feb 2013 on Windows XP SP3 (virtual)
* OpenSSL 1.0.1 14 Mar 2012 on Ubuntu Server 12.04.2 LTS amd64
* OpenSSL 1.0.1 14 Mar 2012 on Ubuntu Desktop 12.04.2 LTS amd64

I used command `openssl s_client -connect my.host.tld:443 -cert path/to/pem`. In each one openssl connected and waited for a request. Protocols and ciphers used were:

* Protocol: TLSv1.1 Cipher: DHE-RSA-AES256-SHA on Ubuntus
* Protocol: TLSv1.2 Cipher: DHE-RSA-AES256-GCM-SHA384 on Windows

All outputs showed some errors, however:

~~~
verify error:num=19:self signed certificate in certificate chain
verify return:0
~~~

Post by c***@shelton-family.net
If you just get an error message, the issue is with the openssl layer (lower than neon/serf).

If OpenSSL can connect, does it mean the problem lies in Neon/Serf?

[Pradeep Behera]

I am also seeing same issue, need help !!

Post by gadamiak
Hi,
~~~
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 read client hello A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write server hello A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write certificate A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write key exchange A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 write certificate request A
[debug] ssl_engine_kernel.c(1874): OpenSSL: Loop: SSLv3 flush data
[debug] ssl_engine_io.c(1908): OpenSSL: I/O error, 5 bytes expected to read on BIO#7f8df0001340 [mem: 7f8df8006c73]
[debug] ssl_engine_kernel.c(1903): OpenSSL: Exit: error in SSLv3 read client certificate A
[debug] ssl_engine_kernel.c(1903): OpenSSL: Exit: error in SSLv3 read client certificate A
~~~
Primary CA
→ Intermediate server CA → server certificates
→ Intermediate user CAs → user certificates
~~~
svn: E175002: Unable to connect to a repository at URL 'https://...'
svn: E175002: OPTIONS of 'https://...': SSL handshake failed: SSL error: unsupported algorithm nid (https://...)
~~~
~~~
svn: E120171: Unable to connect to a repository at URL 'https://...'
svn: E120171: Error running context: APR does not understand this error code
~~~
Where is the source of the problem?
I know there have been changes in OpenSSL 1.0.1 handling of self-signed certificates. Is this related? If so, what has to be changed/added to user certificates to make them work? Or are Neon and Serf libraries failing?
Definitely need help with that. Thanks.

[Robert Rodriguez]

LGC Decorators can offer a convenient https://www.lgcdecorators.co.uk, affordable way to get the professional painting and decorating services you need to make your home or office look its best. We’re here to help you create a beautiful home or office that you can enjoy for years to come.