Assurance vs Penetration Testing
22 December, 2012
What's all the fuss about? Is there really any difference between these types of tests?

Appsecure performs two different styles of testings for our clients. In most cases our new clients contact us to request for a "penetration test" to be performed against their systems. In some cases though, there's alot of myths out there about penetration testing. The purpose of this post is to clear these up, and really understand what are the positives and negatives of both styles. We're not trying to say in the following blog one is better then the other, but explaining the process and difference between them and which one to choose based on your required outcome.

Penetration Testing
This is the most common format of penetration testing out there. It's always been popular and will continue to be popular as everyone does it. This type of testing puts alot of "magic" in the hands of the company testing however, and removes the guarantee of a good test. In 99.9% of cases Penetration testing is completed within a time-limit, however it's sets to a "methodology" or process of how the test is conducted. The largest problem with testing is caused by human nature. During a penetration test if a tester identifies a potenial vulnerability they could spend large amounts of the test time just trying to exploit that one particular vulnerability. If successful, you've found a risk you need to fix, but what about the rest of the system?

This is a common problem, at Appsecure we mention Penetration testing being a "depth" approach to testing, you may not get full coverage (as in finding vulnerabilities across the system), but you will find serious vulnerabilities and how far you can go with them, but you're not going to fully understand whether your system is completely secure. The other challenge facing penetration testing is skill level of the tester and automation. When running a test program, there should be a limit placed exclusively on automated testing, normally not more then 15% of the entire project time.

Automating testing is good, the key element this service adds, is to identify quick and easy vulnerabilities that would be found by a script kiddie or someone with very limitied skills. Although they may not present high exploits they are easy to find by nearly anyone or a quick tool and should be addressed.

Assurance Testing
So we've just discussed in breif Penetration testing, and the weakenesses such as skill level, automation and depth. Assurance testing is designed to minimise those weakeneses and provide "coverage" across the system, with a little less depth. In most cases when we talk with clients, the phrase "Can you test this for vulnerabilities" is what they are after.

So if we conduct a penetration assessment, you get depth, but we are more then likely going to miss vulnerabilities due to the time limitations to conduct the test. Assurance testing to a detailed methodology removes this problem and ensures that coverage is applied across the entire system. So really what you are getting here is more coverage across the application and less depth. (this is good and we will explain why.)

More coverage ensures that we don't miss things, it also means the entire platform was looked at, therefore we didn't spend large amounts of time on one particular area doing research and forgot to look at another. We've seen testers during an penetration assessment use 3 days to try and prove he could break into 1 vulnerability (which he didn't) and then took another 1 day to complete the assessment, missing alot of weakensses. This is the largest shortfall of Penetration testing out there.

Penetration testing these days is very easy to be done by automation tools. It's alot harder to test for business logic, and exploits that are covered in assurance testing, therefore if you can't automate it typically organisations will avoid doing it, as it is time consuming and finding and keeping good penetration testers is hard and costly. Next time you complete a penetration test program, ask your testing provider to provide a detailed test plan (with time allocation) on what activities they are going to undertake. This gives you some assurance that the test being conducted is not just scanning tools!

So what do i choose? I'm now totally confused?
Really it's actually got alot to do with what you are wanting to achieve? If you want a test that "Find me really bad vulnabilities that are easy to find quickly" then you want a penetration testing, but if you want to understand what vulnerabilities are found within your entire system to test then really you want an assurance test. We've included so pro's and con's for the types of testing below.