The topic of this blog post has been long on my mind, but I did not have a good example to use. Finally I found one.
Software security is a very complex field and a asymmetric problem space. Arguments whether defense or offense is harder have been fought for a long time and they will likely never stop. I think most of us can agree on those two statements:
- offense needs to find only a handful of problems and work hard to turn them into a compromise
- defense needs to architect software to be resilient and work hard to ideally (though not practically) not introduce any problems
Each side requires unique skills and is extremely rare for people to be really good at both. What really irks me is that lots of people in the security industry tend to bash the other side. It is easy, one understands their problem space very very well and knows how hard it is to be an expert. Also, it feels that the other side is not that hard. I mean, how hard can it be, right? Wrong!
In this post I will pick the side of the defender, since this is where I spend most of my time. The example I will use is the recent events with the Aviator browser, because it is near and dear to my heart. One thing I want to make clear from the get go - I totally respect their efforts and applaud them for trying. Forking Chromium is not a small feat and not for the faint of heart. The goals for Aviator are admirable and we definitely need people to experiment with bold and breaking changes. It is through trial and error that we learn, even in proper engineering disciplines :). What can we use more of the security industry?
Humbleness!
It is no surprise people on the offensive side bash software developers for “stupid” mistakes, since the grass is always greener on the other side. The problem is that many trivialize the work required to fix those mistakes. Some are indeed easy. Fixing a stack-based buffer overflow is not too hard. In other cases, it is harder due to code complexity or just fundamental architecture of the code.
What humbles me personally is having tried the attack side. It is not too bad if you want to exploit a simple example problem. Once you try to exploit a modern browser, it is a completely different game. I admire all the exploit writers for it and am constantly amazed by their work. Same goes for a lot of the offensive research going on.
I have secretly wished in the past for some of the offensive folks to try and develop and ship a product. When WhiteHat Security released the Aviator browser, I was very much intrigued how it will develop. It is not a secret that Jeremiah Grossman and Robert Hansen have given lots of talks on how the web is broken and how browser vendors do not want to fix certain classes of issues. They have never been kind in their remarks to browser vendors, but now they have become one. I watched with interest to see how they have mitigated the issues they have been discussing. Heck, I wanted to see clickjacking protection implemented in Chromium, since it is the authors of Aviator that found this attack vector and I have personally thought about that problem space in the past.
Chris Palmer and I have played around with the idea of “Paranoid Mode” in Chromium and as a proof of concept we have written Stannum (source) to see how far we can push it through the extensions APIs. It is much safer to add features to Chromium using extensions than writing C++ code in the browser itself1. So when Aviator was announced and released initially, I reached out to WhiteHat Security to discuss whether the features they have implemented in C++ could be implemented through the extensions API. My interest was primarily motivated by learning what they have done and what are the limitations of the extensions subsystem. Unfortunately, the discussion did not go far :(.
Where do I believe they could have done better? You might have guessed it right - being humble. The marketing for Aviator is very bold - “the most secure and private Web browser available”. This is a very daring claim to make, hard promise to uphold and anyone who has been in security should know better. Securing a complex piece of software, such as a browser, is a fairly hard task and requires lots of diligence. It takes quite a bit of effort just to stay on top of all the bugs being discovered and features committed, let alone develop defenses and mitigations.
Releasing the source for Aviator was a great step by WhiteHat. It gives us a great example to learn from. Looking at the changes made, it is clear that most the code was written by developers who are new to C++. When making such bold statments, I would have expected more mature code. Skilled C++ developers that understand browsers are rare, but it is a problem that can be solved. It takes a lot of time, effort and desire for someone to learn to use the language and most importantly understand the architecture of the browser. Unfortunately, I did not see any evidence that whoever wrote the Aviator specific code did any studying of the source code or attempted to understand how Chromium is written and integrate the changes well.
What really matters at the end of the day, though, is not the current state of a codebase. After all, every piece of software has bugs. I believe there is one key factor which can determine long term success or failure:
Attitude!
Security vulnerabilities are a fact of life in every large enough codebase. Even in the project I work on we have introduced code that allowed geohot to pull off his total ChromeOS pwnage! We owned up to it, the bug was fixed and we looked around to ensure we did not miss other similar instances.
However, what I was most disappointed by was the reaction from WhiteHat when a critical vulnerability was found in the Aviator specific code:
“Yup, Patches welcome, it’s open source.” 2
Our industry would go further if we follow a few simple steps:
- Do not trivialize the work of the opposite side, it is more complex than it appears on the surface.
- When working on a complex software or problem, study it first
- Share ideas and collaborate
- Own up to your mistakes
- Be humble
1. Even Blink is starting to implement rendering engine features in JavaScript. 2. Nevermind that there is no explanation on how to build Aviator, so one can actually verify that the fix works.