Boxes, yet more boxes!

This article is the second in a series of blog posts for internal auditors and C-suite to understand the types of VAPT (security assessment) testing for web and mobile applications.


First post: One Vulnerability Assessment Penetration Testing (VAPT) question asked by Internal auditors and CISOs
Second post: Boxes, yet more boxes!
Last post: Where do blackbox and grey box tests play in securing your cyber assets?


Last week, we stopped at the types of testing and the four subtypes of security assessments. This week, we will go into further detail about these four subtypes of security assessments and their assurance levels.


Blackbox, Greybox, Whitebox

"Unknown unknowns" Secretary of Defense Donald H. Rumsfeld
Photo: “Unknown unknowns” Secretary of Defense Donald H. Rumsfeld

As the names black box and grey box suggests, there are unknowns which the security assessor will not be able to see or access, thus not able to test. The security assessor can attempt bruteforce methods to try to elicit vulnerable responses, unfortunately it is not foolproof. Therefore 100% testing cannot be asserted with either black box or grey box testing methodologies.

Source code review, a type of static testing, refers to checking source code for potential vulnerabilities. These checks are usually automated to cut down on human effort, however, these (static analysis tools) do produce false positives, which programmers find them more of a nuisance.

Whitebox testing with source code review, combines both static and dynamic testing, to provide a comprehensive assessment on the web or mobile application. White box testing with source code review is the optimum method to detect vulnerabilities for three reasons:

  1. Developers mainly focus on the business functional requirements. After all, the business functional requirements have higher priority compared to other requirements.
  2. Security assessors focus on the security aspects and tend to pick up the non-obvious weaknesses.
  3. Working together, security assessors and developers can address vulnerabilities and improve coding practices from each of their perspectives.


Assurance levels

Next we come to the assurance level accorded by each of the four subtypes of security assessments.

Assurance level vs Assessment time and effort
Assurance level vs Assessment time and effort

The level of assurance vs assurance time and effort can be summarised above, where black box testing has the lowest assurance whereas whitebox with source code review has the highest assurance.

As the required assurance level increase, experienced security assessors will be required to perform the security testing including source code review. The security assessors will work together with the software developers to understand the code flow and the security/convenience tradeoffs from a security perspective.


Characteristics of various dynamic testing security assessments

Black box vs Grey box vs White box with source code review
Black box vs Grey box vs White box with source code review

As you can see, both black box and grey box testing are unable to assert test completeness. Only whitebox testing with source code review would be able to address the completeness requirements.

We will stop here and continue next week on how can black box and grey box testing can play in securing your cyber assets.


Again, stay Cyber Strong!

Say Hong TAN

CISA, CRT, CA (Singapore)
Founder and Director
Dynafense Cybersecurity

Leave a Reply

Your email address will not be published. Required fields are marked *