Three vendors, three proposals, three completely different price points, and you have no real way to tell which one is going to give you a credible report versus a glorified scanner output with a logo on the cover. This is the exact moment most SaaS founders get stuck, and it is the moment that determines whether your pentest actually moves your enterprise deal forward or sits in a folder doing nothing.
A real pentest vendor evaluation is not about comparing line-item prices. It is about figuring out which of these firms is going to put a skilled human in front of your application versus which one is going to run a scanner and write up the output. Here is exactly how to tell the difference before you sign anything.
Why Pentest Vendor Evaluation Matters More Than Price
The cost of a bad pentest is not the invoice. It is what happens afterward. A bad pentest costs you twice, once when you pay for it, and again when an auditor flags it as insufficient evidence, an enterprise security team rejects it, or a real vulnerability that nobody tested for ends up in production.
For a seed or Series A SaaS company, the consequences are specific and predictable. Your auditor reviews the report for your SOC 2 evidence package and finds it thin. Your enterprise prospect’s security team asks a follow-up question your report cannot answer. Or worse, six months later an actual attacker finds the access control bug that a real manual tester would have caught and a scanner never could.
None of these outcomes are visible at the moment you sign the contract. That is exactly why the evaluation has to happen before you sign, not after the report arrives.
The Questions That Actually Separate Vendors
How many hours of manual testing are allocated to this engagement, and who specifically will be doing it?
This is the single most diagnostic question you can ask. A vendor with a repeatable, well-defined testing process will give you a specific number and name the tester’s credentials, such as OSCP, without hesitation. A vendor that gives a vague answer, or quotes a number that seems too low for your scope, is telling you something important before the engagement has even started.
What methodology do you follow, and which version?
A credible answer names a specific, recognized standard — OWASP Web Security Testing Guide, OWASP API Security Top 10, PTES, or NIST SP 800-115 by name and version. A methodology section that only lists tool names like Burp Suite, Nmap, or Metasploit without mapping to any of these standards is a marketing artifact, not an actual methodology. Tools find things. Standards tell you what should have been tested and how thoroughly.
Can I see a sanitized sample report before we sign?
Every credible vendor has one ready to share. If a vendor refuses or stalls on this request, you are being asked to buy a deliverable you have never seen. The sample report tells you more about quality than any sales conversation will, look specifically at whether findings include proof-of-concept evidence, clear severity ratings, and remediation guidance that your engineering team could actually act on.
How is this priced — by scope and hours, or by asset count?
Pricing structure reveals testing approach. Per-IP or per-page pricing usually signals automated scanning dressed up as a pentest, because real manual testing does not scale linearly with the number of targets. A workload-based quote, priced by tester time or engagement scope, is the structure a genuine manual engagement uses.
What does your retest process look like, and is it included?
A pentest without a retest leaves you with an unverified list of findings. Ask whether retesting is included in the base price or billed separately, and what exactly gets re-checked, only the original findings, or also anything introduced during remediation.
What happens after the report is delivered?
A firm that drops a PDF and disappears leaves your team to interpret findings alone. Ask whether they offer a debrief call, a window for follow-up questions, or ongoing support if your auditor or an enterprise prospect has questions about the report later.
Red Flags That Should End the Conversation
A handful of warning signs are reliable enough that they should disqualify a vendor on the first call, regardless of how good the rest of the pitch sounds.
Timeline promises that do not match scope are the most common. If a vendor promises a comprehensive engagement in under five business days for anything beyond a narrow, focused scope, real manual testing simply does not fit in that window. Either the scope is too small to matter or the testing is too shallow to be credible.
A refusal to share a sample report on request is disqualifying on its own. There is no good reason a legitimate firm cannot show you sanitized output from a past engagement.
Pricing that focuses only on asset count, IPs, pages, or domains, without any reference to testing hours or tester time is a strong signal you are buying scanner output, not human expertise.
Reluctance to discuss methodology in specific terms, or a methodology explanation that only names tools rather than testing standards, suggests the vendor does not have a repeatable, documented process, they are improvising per engagement.
No professional indemnity insurance, or unwillingness to provide proof of coverage, is a real operational risk. Penetration testing involves deliberate attempts to access systems. If something goes wrong, data loss, service disruption, unintended access, you want to know the firm carries insurance, and you should ask for a certificate before signing.
What Good Looks Like
The flip side of every red flag above is a green flag, and together they form a clear picture of what a credible vendor actually looks like.
A good vendor gives you a specific testing hour estimate and names the tester’s credentials, ideally OSCP or an equivalent recognized certification, without you having to push for it. They name a specific testing methodology and version. They share a sample report proactively, often before you even ask. They price based on scope and effort rather than a simple asset count. They build retesting into the engagement by default rather than treating it as a costly add-on. And they stay available after delivery, for a debrief, for follow-up questions, for help interpreting a finding your engineering team is not sure how to prioritize.
None of this requires a large firm or an enterprise budget. A boutique vendor with a small, credentialed team can check every one of these boxes just as well as a large firm, sometimes better, because the relationship is more direct and the communication is faster.
The SaaS-Specific Layer
Beyond the general evaluation criteria, SaaS companies have a few additional things worth checking specifically.
Ask whether the vendor has tested multi-tenant architectures before, and how they approach tenant isolation testing specifically, this is the category of vulnerability that automated tools almost never catch and that matters most for a SaaS platform handling multiple customers’ data on shared infrastructure.
Ask how they handle API testing as part of the engagement, not as an afterthought to the main web application test. Most SaaS products expose significant functionality through APIs, and a vendor who treats API testing as a checkbox rather than a core part of the engagement is missing where a meaningful share of real-world SaaS vulnerabilities live.
If you are pursuing SOC 2, ISO 27001, or HIPAA, ask whether the vendor can map findings to the relevant control framework in the final report. A report that connects each finding to a specific compliance control saves you significant time when you hand it to your auditor.
The Bottom Line
A genuine pentest vendor evaluation comes down to verifying the people and the process behind the engagement, not the logo on the proposal. The questions above take less than fifteen minutes on a scoping call and reliably separate vendors who will find what an actual attacker would exploit from vendors who will run a scanner and email you a polished PDF.
The cost of getting this wrong is rarely visible until it is too late, a rejected security questionnaire, an auditor’s follow-up question you cannot answer, or a real vulnerability that sat untested in production. Spending fifteen minutes on the right questions before you sign is the cheapest insurance available against that outcome.
Packet33 conducts web application, API, and external network penetration testing for SaaS and HealthTech startups, with OSCP-certified testers, transparent scope-based pricing, and reports mapped to SOC 2, ISO 27001, and HIPAA where relevant.
Talk to us about what a transparent pentest engagement looks like for your environment.
Packet33 is a penetration testing and compliance advisory firm serving SaaS and HealthTech startups in the US, Canada, and UK.