A pentest is worth as much as the people who ran it

What is confirmed and what is only claimed

On August 10 MyDr announced it was investigating a serious security incident. That same evening Zaufana Trzecia Strona described contact with people claiming to be the perpetrators, who say they hold 18,814,422 unique PESEL numbers, the Polish national identification number. On August 12 deputy prime minister Krzysztof Gawkowski confirmed the breach and spoke of nearly 19 million people and 12,000 medical facilities. On August 13 the District Prosecutor's Office in Warsaw said it is supervising an investigation run by CBZC, the Polish cybercrime bureau, and the president of UODO, the Polish data protection authority, announced an inspection at the company. On August 14 the e-Health Centre began preemptively replacing the certificates used for communication with the national P1 system, noting that nothing indicates they were taken.

The company itself said the data involved is most likely historical, from 2024 and earlier years. The perpetrators talk about 2.5 TB. Nobody outside the company has verified any of those numbers.

Whether MyDr commissioned penetration tests, I do not know. Who ran them and what the scope covered, I do not know either, because nobody has stated it publicly. Anyone writing today that the company evidently let security slide is guessing. Usually guessing with relief that this time it is someone else. So I will spend the rest of this text on the question worth asking at your own company after a week like this.

The only technical detail we have comes from the attackers

They described their entry path to Zaufana Trzecia Strona: an XXE vulnerability in the handling of PKCS#12 certificates, remote code execution from there, then a GitHub API key, the source code of the service, and finally the infrastructure in AWS. The outlet states plainly that it was not able to verify this, and the company has not confirmed the scenario. In cases like this the final findings can differ from the first version, so treat it as an exercise, not as fact.

The exercise is instructive regardless of whether it happened in that particular place.

A vulnerability scanner sees a function that accepts a file with a certificate and a 200 response. A human asks what that container gets taken apart by along the way, whether an XML parser sits in the path and whether anyone remembered to disable external entities. Then they ask what that process keeps in its environment variables. Then they check whether the token they found actually opens the repository, and if it does, whether more secrets are lying in the commit history. None of those steps follows from a checklist. Each one is somebody's decision, made on the third day of the test, when the easy things have already been checked.

Why “did we have a pentest” is the wrong question

A penetration test is not a product off the shelf, you are buying specific people for a specific number of days. Two proposals can carry the same title, the same methodology written into the PDF and a price difference of half the rate, while underneath being an entirely different undertaking. In one, somebody spends four days digging through file handling and electronic signatures. In the other, somebody runs a scanner, moves the output into a template and sends the report on Friday afternoon.

No test comes with a guarantee. It checks a fragment of the environment in one specific week, and two weeks later a release lands that nobody has looked at yet. Where that test does look, though, is not decided by chance. It is decided by who runs it and how many days they have for it. One pentester will stop at the login form, because that is where their routine ends. Another will open the certificate import and start wondering what that file gets taken apart by along the way.

A good share of our projects arrive on applications and networks that somebody has already tested. They regularly turn up things that are not in the previous report. Sometimes small details, sometimes findings that land on the first page of the summary. I would rather write that this is the exception. Instead, at closing meetings I hear one question more often than any other: why did the others not find this?

Six questions before you order the next test

• Who by name will carry out the test and what that person has been doing for the past year. If the answer is “a team of experienced experts”, keep asking, because names usually appear only on the second attempt.
• How many mandays the proposal covers and how they split across reconnaissance, manual work and the report. The total on its own says nothing, because the same ten days can hold a solid test or a week of writing a document.
• What stays out of scope and whose decision that was. What drops out most often is file handling, electronic signatures, integrations with third parties, repositories and CI/CD, usually without a single sentence of explanation in the proposal.
• How many findings in a sample report come from a scanner and how many from manual work. Ask for that report before signing the contract, because afterwards you have nothing left to negotiate with.
• Whether a retest after fixes is included in the price and until when you can use it.
• Who reads the report before you do. A second pentester catches technical errors, but also findings written up in a way that means nobody will ever fix them.

The board has simple arithmetic at this point. The controller of a patient's medical records remains the clinic, so it is the clinic that answers to the data protection authority and the clinic that receives claims under Article 82 GDPR, even though it neither wrote nor maintained the system. In this case there are more than ten thousand such facilities, and today all of them are waiting for the vendor to establish whose data leaked. The difference between two penetration testing proposals is usually a few tens of thousands of zloty. Proceedings after a breach are counted on a different scale and drag on for years.

None of those six questions requires a budget or board approval. One email to the vendor and fifteen minutes to read the answer is enough. Send it to the firm that tests you today, then count how long it took them to answer the first question on the list. A test is worth as much as the people who ran it, so one question is left for the end: can you name today the person who checked your most important application?