If you run ZAP multiple times against a target then you may well find that the results are subtly different even though the target has not changed.
This is not unusual, and we do not consider this a significant problem.
In our experience it is usually all down to how the application is explored - the traditional and ajax spiders seem to be sensitive to small changes, including things like network speed.
Some of the ZAP integration tests perform a baseline scan of https://www.example.com and even though this is just one page we have found that results can vary.
Investigations into this showed that the number of requests made by the traditional spider sometimes differed and the same page could be requested more than once. For most rules and alerts this should not be a problem, but that is not always the case. For example, the Retrieved from Cache rule includes the value of the “Age” header as evidence. If multiple requests are made to the same page then this header value is likely to change, and if it does the alerts are treated as different instances.
If ZAP finds exactly the same URLs and the target has not changed then we would expect the alerts to be the same (although the instances of those alerts may differ).
However if you investigate this issue in more detail and find out there are other problems then please let us know.
Significantly different results are another matter.
Running a scan via the Desktop, via the Automation Framework, or via one of the Packaged Scans should make no difference, as long as ZAP is doing the same things in each case.
If you are finding significant differences when running ZAP in different places then check:
- That you are running the same versions of ZAP
- You have the same add-ons and rules installed and enabled
- All instances are up to date
- The configurations are exactly the same
- That you are doing exactly the same things
It is also well worth checking that:
- The target really has not changed
- You are testing the same target (different environments may have different routing)
- There are no other security services in the way, such as WAFs
- That the target is not changing as a result of being scanned by ZAP (e.g. storing attack payloads)