We wrote tests for our third-party security libraries, and you won’t believe what happened next! (CVE-2017-8028)

On the importance of thorough testing

Much of modern software development revolves around the concept of “quality”. As with all abstract concepts, “quality” is somewhat difficult to pin down, but for this article we can define it as “how well software conforms to its requirements”. Less formally, we can say that “high-quality software does what it’s supposed to”.

One way to ensure that you consistently deliver high-quality software is to have automated tests. I’m not a Test-Driven Development fundamentalist insisting on 100% test coverage; but if some part of your software is important to your users, there should be at least one automated test demonstrating that it works. This is obviously true for your software’s functional requirements, but it applies equally to its non-functional requirements such as auditing and performance.

So what non-functional requirements are important to the users of Basefarm’s internally-developed applications?
One thing that immediately springs to (at least my) mind is security. So we need some tests for that, right?

Think like a hacker

When developing secure applications, it is always useful to try to think like an attacker. If I were to try to get unauthorized access to this application, how would I go about it?

Obviously, you’d first need to log in. For Basefarm’s internal applications we use username/password logins authenticating against our internal LDAP server; this means that an attacker must obtain a set of working credentials somehow.

This suggests three tests:

  1. An existing user can log in with the correct password (this is what we call the happy-path test, which demonstrates that in fair weather conditions the software works as intended);
  2. A non-existent user is not allowed access;
  3. An existing user logging in with an incorrect password is not allowed access

As part of these tests, it is also useful to think about things like auditing and information leakage; login attempts — successful or not — should be logged, and the error message(s) presented on failure should not give an attacker information she does not already have. In particular, authentication failures should not say “no such user” or “incorrect password”; “Invalid credentials” is a safe, neutral way to put it.

So, in pseudo-code, we have the following tests:

authenticate("existinguser", "correctpassword") must succeed

authenticate("nonexistentuser", "anypassword") must fail

authenticate("existinguser", "wrongpassword") must fail

Don’t reinvent the wheel

The developers at Basefarm, as in most other places, don’t have the time (or, to be honest, the expertise) to write everything from scratch. It is hard enough to implement our specific functionality; implementing and maintaining code to talk to databases and LDAP, handle authentication and transactions and all the other things a large-scale (dare I say Enterprise?) application needs would be impossible. So like many other teams on the JVM platform we lean heavily on the Spring framework and its attendant ecosystem of libraries; in particular, we use Spring Security for authentication and authorization.

Setting the scene

Since our LDAP server (as all good LDAP servers should) requires authentication and encrypted communication using STARTTLS, our Spring Security configuration looks something like this (ignoring some details on how, exactly, things are wired together):

<bean id="tlsAuthStrategy" 
      class="" />

  username="cn=Manager,dc=springframework,dc=org" password="secret" 
  authentication-strategy-ref="tlsAuthStrategy" />

    user-search-filter="(uid={0})" />

The idea is that the security framework, authenticating with the manager credentials, searches for a user to really authenticate as.

We believe this to be a fairly typical LDAP-auth setup; in fact, the actual configuration is more or less copy-pasted from the Spring Security documentation.

We originally implemented this in 2011; and the configuration worked well, all the tests passed, and we were happy.

Let’s dance!

Fast-forward six years, to 2017. As part of our regular maintenance cycle, I went through all of our application’s dependencies and updated them to the latest version, to get the benefit of whatever new features and fixes are available.

This is usually a cushy job: Update some version numbers; recompile everything; drink coffee while the tests pass; and then ship it.

So I upgrade Spring Security to 4.2.1, and start the test run.

Imagine my surprise when I returned from my coffee binge to see

Tests failed: 1

Waltz Tango Foxtrot?

And the test that failed?

authenticate("existinguser", "wrongpassword") must fail

Whisky Tequila Fernet? In short, WTF???

Finding the problem

I’ll spare you the details of how we debugged the issue; it’s not really interesting, involving as it did copious amounts of coffee, creative swearing, scratching of heads, wailing and gnashing of teeth, and poring over code, logs and packet captures.

It turned out that the problem was not in Spring Security per se, but in Spring LDAP; a change in Spring Security’s use of that library exposed a weakness in DefaultTlsDirContextAuthenticationStrategy, which never actually performs an LDAP “bind” operation using the provided credentials.

The desired sequence of events when authenticating in our setup is something like this:

  1. Open a new LDAP connection
  2. Secure it using STARTTLS
  3. Bind with the search credentials (username/password from the context-source)
  4. Perform an LDAP search for the DN to bind as for the real authentication step
  5. Open another new LDAP connection
  6. Secure it using STARTTLS
  7. Bind using the DN from step 4 and the provided password

The logs from the LDAP server show what really happens (I’ve removed some entries from the log; the full log can be seen here):

-- step 1
59ceb606 conn=1000 fd=15 ACCEPT from IP= (IP=
-- step 2
59ceb606 conn=1000 op=0 STARTTLS
-- step 3
59ceb606 conn=1000 op=1 BIND dn="cn=admin,dc=example,dc=org" mech=SIMPLE ssf=0
-- step 4
59ceb606 conn=1000 op=2 SRCH base="ou=People,dc=example,dc=org" scope=2 deref=3 filter="(cn=user)"
59ceb606 conn=1000 fd=15 closed
-- step 5
59ceb606 conn=1001 fd=15 ACCEPT from IP= (IP=
-- step 6
59ceb606 conn=1001 op=0 STARTTLS
59ceb606 conn=1001 fd=15 closed (connection lost)

Step 7 never happens; as long as you have a username, all passwords are accepted.

Further investigation showed that this had been noticed as far back as 2013 by one mwebb and reported as a bug in November 2016, but not recognized as a security issue.

We had also noticed it before, earlier in 2017, but due to time pressures we did not properly investigate the issue at that time.

Houston, you may want to look at this

The Spring Framework is currently maintained by Pivotal Software, Inc., so we notified their security team of the problem. They, in turn, notified the maintainers of Spring LDAP, who initially concluded that this is not a bug because anonymous LDAP access was involved, as it was in the original example project exhibiting the bug.

However, a slight modification of the example project showed that the erroneous behaviour persists even when anonymous bind is not used; it is clear from the log that there is no bind attempt with the user credentials at all.

After a little back and forth, the Spring LDAP team released a fixed version of Spring LDAP on October 6, 2017, nearly a year after the initial bug report.


2016-11-11 (approximately)
Tobias Schneider discovers the issue
Schneider reports the bug to Spring LDAP, with a pull request
We initially notice the issue, but due to time pressures do not investigate and roll back to an earlier version
We attempt to upgrade again; again, we roll back to the earlier, working version, but this time (probably due to higher blood caffeine levels) a full investigation is scheduled
We isolate the problem and discover the earlier bug report
We notify Pivotal’s security team that we consider this a security vulnerability
The Spring LDAP team determines that this is not a vulnerability
We demonstrate that the issue exists without involving anonymous search
The Spring LDAP team commits a fix and releases a working version
We request a CVE via Dell EMC and Pivotal
Pivotal publishes CVE-2017-8028

The moral of the story (a.k.a TL;DR)

  • Security is an important part of your application; write tests for it
  • All non-trivial software has bugs, and some of these will be security-related
  • Report security issues to the security contact for the involved software

POODLE returns

The POODLE has returned, with a vengeance! This time it’s affecting vendors such as F5, even though SSLv3 is disabled. This means that TLSv1.0, TLSv1.1 and TLSv1.2 can be affected if the SSL termination is being done on a vulnerable server. Those with F5 are advised to update to the latest version as soon as possible, and you can check on SSLLabs if your site is affected by this (in which case it will automatically be graded F-).

More information:


Drupal 7 includes a database abstraction API to ensure that queries executed against the database are sanitized to prevent SQL injection attacks.

A vulnerability in this API allows an attacker to send specially crafted requests resulting in arbitrary SQL execution. Depending on the content of the requests this can lead to privilege escalation, arbitrary PHP execution, or other attacks.

This vulnerability can be exploited by anonymous users.

More information:

Possible SSLv3 Vulnerability

“The Register has learned that news of yet another security vulnerability – this time in SSL 3.0 – is probably imminent.”
While this is currently unverified, it’s still good to take a look at if any services of yours are using SSLv3. SSLv3 came out in 1996, and have since been superseded by TLSv1.0, TLSv1.1 and eventually TLSv1.2.

The recommendation is to disable SSLv3, although this does mean that those using Windows XP (which has been End of Life for a while now) with Internet Explorer 6 (but Internet Explorer 7 would work) are unable to access the service.

More information:

WordPress and Drupal patched for DDoS vulnerability

WordPress and Drupal have been patched for, amongst other things, a vulnerability that allows an attacker to take down a WordPress or Drupal site.

The PHP XML parser used by both projects has a XML-RPC endpoint is vulnerable to an XML entity expansion attack and other related XML payload attacks which can cause CPU and memory exhaustion and the site’s database to reach the maximum number of open connections. Any of these may lead to the site becoming unavailable or unresponsive (denial of service).

Users of WordPress should upgrade to 3.9.2 as soon as possible:

More information:

Vulnerability in Cisco ASA

Cisco Adaptive Security Appliance (ASA) Software is affected by the following vulnerabilities:
Cisco ASA ASDM Privilege Escalation Vulnerability
Cisco ASA SSL VPN Privilege Escalation Vulnerability
Cisco ASA SSL VPN Authentication Bypass Vulnerability
Cisco ASA SIP Denial of Service Vulnerability
These vulnerabilities are independent of one another; a release that is affected by one of the vulnerabilities may not be affected by the others.

Successful exploitation of the Cisco ASA ASDM Privilege Escalation Vulnerability and the Cisco ASA SSL VPN Privilege Escalation Vulnerability may allow an attacker or an unprivileged user to elevate privileges and gain administrative access to the affected system.

Successful exploitation of the Cisco ASA SSL VPN Authentication Bypass Vulnerability may allow an attacker to obtain unauthorized access to the internal network via SSL VPN.

It is recommended to upgrade as soon as possible in order to avoid breaches.

More information and upgrade information:

Critical OpenSSL vulnerability

A security vulnerability in OpenSSL was published on April 7. With this vulnerability, an attacker is able to remotely dump the memory contents of a process using OpenSSL.
This exposes both the content of SSL/TLS encrypted communications, and the associated private keys. This is a major issue as OpenSSL is a critical component of most encrypted Internet services.

Basefarm’s Security Incident Response Team, together with other Basefarm personnel, investigated which of the servers hosted by us were affected, and to what extent. Those services which were managed by Basefarm were then patched and affected customers were notified. This was completed Tuesday afternoon.

There is unfortunately no way of knowing for certain which information has been stolen during the attack window, so we recommend anyone being affected by this vulnerability to assume that your SSL/TLS private keys have been stolen, even if we have no concrete indication of this.
This means that you will need a new key pair and certificate for any exposed SSL/TLS keys and certificates. Basefarm will help you with this if we manage the keys for you. Your old certificates will also need to be revoked.

Any other information passed over a vulnerable SSL/TLS connection may also have been captured, including usernames, passwords, credit card numbers and other personally identifiable information.
We recommend that you initiate a password change for any account where the password has been passed over av vulnerable SSL/TLS connection over the last day or so.

Please note that if Personally Identifiable Information, credit card or cardholder data, or other sensitive data may have been compromised, you probably have an obligation to alert the proper authorities.
Unfortunately, there is no way of knowing exactly which data has been compromised.

Here are the mitigation steps in detail
1. Emergency fix the vulnerability itself by patching, reconfiguring, or both, on all exposed servers.
2. Generate a new key pair following best practice guidelines.
3. Purchase a new certificate using the new key pair. Without a new key pair, the old and possibly stolen key pair could be abused, e.g. to eavesdrop or to impersonate the service.
4. Switch to the new certificate on all relevant servers.
5. Revoke the old certificate. This is important, as otherwise a stolen private key could still be used to impersonate the web server until the old certificate expires.
6. Consider initiating a change of all passwords etc. that have been sent over SSL/TLS using the old key pair, at least those sent over the last day or so. If someone has an old “recording” of an encrypted conversation, and also gets hold of the old keys, they can now decrypt that conversation. There’s nothing to be done about that in itself, but any reusable credentials could be stolen this way and abused if they are not changed.

Step 6 really is up to you and/or your end users. High-profile and/or high-value sites would be well advised to at least recommend that their users change their passwords, and could use this as an opportunity to convey a strong message that they care about the security of their users.
We have previously written a note in our security tips section of this newsletter about passwords:

So what are the lessons learned?
Things can always be done faster and automated better and that’s something we’re always working towards, and in this case we should have focused a bit more on communication – actually been better at informing the customers that we were working with patching their services.

Apple security updates

Apple have released multiple critical security updates for iOS, OS X, Safari and Quicktime. These updates fixes critical issues with SSL traffic, so make sure you update as soon as possible.
The updates will push your iOS devices to 7.0.6, your OS X to 10.9.2, your Quicktime to 7.7.5 and Safari to 7.0.2 (included in the 10.9.2 version of OS X).

More information:

Adobe Flash Vulnerability

Adobe has released security updates for Adobe Flash Player and earlier versions for Windows and Macintosh and Adobe Flash Player and earlier versions for Linux. These updates address vulnerabilities that could potentially allow an attacker to take control of the affected system. Adobe is aware of reports that an exploit for CVE-2014-0502 exists in the wild, and recommends users update their product installations to the latest versions.

You can find some tips regarding Flash here;

More information:

Solr <4.6 vulnerable

Several vulnerabilities were fixed in recent versions of Solr:
– directory traversal when using XSLT or Velocity templates
(CVE-2013-6397 / SOLR-4882)
– XXE in UpdateRequestHandler (CVE-2013-6407 / SOLR-3895)
– XXE in DocumentAnalysisRequestHandler (CVE-2013-6408 / SOLR-4881)

These vulnerabilities were confirmed to be exploitable also on old
versions like 3.6.2. Gaining remote code execution is easy by combining
the directory traversal and XXE vulnerabilities.

If you wonder how these vulnerabilities could be exploited in real life
setups when Solr isn’t reachable directly from the Internet, you may be
interested in the following blog post: