“We need a pen test” is one of the most common requests we receive, and one of the most commonly mis-scoped. Often what the buyer actually needs is a vulnerability assessment. Sometimes they need both. Occasionally they need neither yet, because the system has no threat model to test against. Knowing the difference saves money and produces findings you can act on.
The short answer
A vulnerability assessment asks: which known weaknesses exist in this environment? It is broad, largely automated, repeatable and relatively cheap. It produces a list.
A penetration test asks: what can an attacker actually achieve here? It is focused, manual, creative and goal-driven. It produces a story: how an attacker got in, what they reached, and which weaknesses made it possible.
Both are useful. They answer different questions.
What a vulnerability assessment does well
A vulnerability assessment uses scanners and configuration checks to compare your systems against databases of known issues: missing patches, outdated libraries, weak TLS settings, default credentials and common misconfigurations. A good assessment adds human triage on top, removing false positives and ranking what remains.
It is the right tool when you need:
- Coverage across many hosts, containers or applications.
- Frequency, for example monthly or after every release, so regressions are caught quickly.
- Hygiene evidence for customers, auditors or insurers.
Its limitation is structural. A scanner can tell you that a library is outdated. It cannot tell you that your order API lets one customer read another customer’s invoices because the authorisation check uses an identifier from the request. That class of flaw, broken object-level authorisation, sits at the top of the OWASP API Security Top 10 precisely because automated tools struggle to see it.
What a penetration test does well
A penetration test is performed by a person who thinks like an attacker. Working within an agreed scope, the tester maps the attack surface, forms hypotheses and tries to turn weaknesses into impact: reading data they should not see, escalating privileges, moving between systems or abusing business logic.
The value is in three things scanners cannot provide:
- Business-logic flaws. Coupon stacking, negative quantities, skipped workflow steps and race conditions in payments look like valid requests to a scanner.
- Chaining. Real breaches rarely rely on one critical bug. They chain a verbose error message, a predictable identifier and a missing rate limit into account takeover. A tester shows the chain, which is what makes a medium-severity finding urgent.
- Proof of impact. “An attacker could read all customer records” moves a fix up the backlog far faster than a CVSS score.
How to choose
| Situation | Start with |
|---|---|
| You have never assessed this environment | A vulnerability assessment, then fix the basics |
| Basic hygiene is in place and the system handles sensitive data or money | A penetration test |
| You are launching a new application or API | A penetration test before go-live |
| A customer, regulator or standard requires it | Both, on the cadence the requirement sets |
| You release continuously | Automated assessment in the pipeline, plus periodic penetration tests |
For organisations handling payment card data, PCI DSS 4.0 makes the cadence explicit: internal and external vulnerability scans at least once every three months, and internal and external penetration testing at least once every twelve months and after any significant change.
Black box, grey box or white box?
- Black box: the tester starts with no knowledge, like an outside attacker. Realistic, but time is spent on discovery rather than depth.
- Grey box: the tester has user accounts and basic documentation. This is usually the best value, because it mirrors a malicious customer or a compromised account and spends the budget on the parts that matter.
- White box: the tester has source code and architecture documents. It is the most thorough, and it pairs naturally with a secure code review.
What a good report contains
Whichever you buy, insist on a report that your engineers and your leadership can both use:
- An executive summary in plain language: what is at risk and what to do first.
- Findings ranked by exploitability and business impact, not raw CVSS alone.
- Reproduction steps for each finding, so engineers can verify it.
- Specific remediation, ideally at code or configuration level.
- A retest of fixed findings, so you can show the risk is closed.
Recognised methodologies such as the OWASP Web Security Testing Guide, PTES and NIST SP 800-115 are a good sign. Equally important is whether the tester understands how your system is built.
Our approach
Our penetration tests are led by a Certified Ethical Hacker who also designs production systems and builds security products. That combination matters. Someone who has designed APIs, event-driven systems and cloud platforms knows where the seams are, and where defects tend to appear between components rather than inside them.
If you are unsure which assessment you need, that is the right place to start a conversation. Read more about our penetration testing and vulnerability assessment service, our API security work, or get in touch.