ZAP – ZAP vs Security Crawl Maze

Google Security Crawl Maze is a comprehensive testbed for web security crawlers.

It is available online at https://security-crawl-maze.app/ and the GitHub repo is https://github.com/google/security-crawl-maze It does appear to be being actively maintained and has merged a fix that we submitted.

As long as one of the ZAP spiders finds the relevant page we count that as a pass, but ideally both spiders will find as many of the URLs as possible.

Section			Score
All URLs			85%
Individual Tests	Standard	Ajax	Client
Top Level: https://security-crawl-maze.app	61	29	42
/javascript/frameworks/angular/event-handler.found	❌ FAIL	❌ FAIL	✓ Pass
/javascript/frameworks/angular/router-outlet.found	❌ FAIL	❌ FAIL	✓ Pass
/javascript/frameworks/angularjs/index.html#!/ng-href.found	❌ FAIL	✓ Pass	✓ Pass
/javascript/frameworks/polymer/event-handler.found	❌ FAIL	❌ FAIL	❌ FAIL
/javascript/frameworks/polymer/polymer-router.found	❌ FAIL	❌ FAIL	❌ FAIL
/javascript/frameworks/react/search.found	❌ FAIL	❌ FAIL	✓ Pass
/javascript/frameworks/react/route-path.found	❌ FAIL	❌ FAIL	✓ Pass
/test/css/font-face.found	✓ Pass	❌ FAIL	❌ FAIL
/test/headers/content-location.found	✓ Pass	❌ FAIL	❌ FAIL
/test/headers/link.found	✓ Pass	❌ FAIL	❌ FAIL
/test/headers/location.found	✓ Pass	✓ Pass	✓ Pass
/test/headers/refresh.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/body/a/href.found	✓ Pass	❌ FAIL	✓ Pass
/test/html/body/a/ping.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/body/applet/archive.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/body/applet/codebase.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/body/audio/source/src.found	❌ FAIL	✓ Pass	✓ Pass
/test/html/body/audio/source/srcset1x.found	❌ FAIL	❌ FAIL	❌ FAIL
/test/html/body/audio/source/srcset2x.found	❌ FAIL	❌ FAIL	❌ FAIL
/test/html/body/audio/src.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/background.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/blockquote/cite.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/body/embed/src.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/form/action-get.found	✓ Pass	❌ FAIL	✓ Pass
/test/html/body/form/action-post.found	✓ Pass	❌ FAIL	✓ Pass
/test/html/body/form/button/formaction.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/frameset/frame/src.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/iframe/src.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/iframe/srcdoc.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/img/dynsrc.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/body/img/longdesc.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/body/img/lowsrc.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/body/img/src-data.found	❌ FAIL	❌ FAIL	❌ FAIL
/test/html/body/img/src.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/img/srcset1x.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/img/srcset2x.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/body/input/src.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/isindex/action.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/body/map/area/ping.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/body/object/codebase.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/body/object/data.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/object/param/value.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/body/script/src.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/svg/image/xlink.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/svg/script/xlink.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/table/background.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/table/td/background.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/video/poster.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/video/src.found	✓ Pass	✓ Pass	✓ Pass
/test/html/body/video/track/src.found	❌ FAIL	❌ FAIL	❌ FAIL
/test/html/doctype.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/head/base/href.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/head/comment-conditional.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/head/import/implementation.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/head/link/href.found	✓ Pass	✓ Pass	✓ Pass
/test/html/head/meta/content-csp.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/head/meta/content-pinned-websites.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/head/meta/content-reading-view.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/head/meta/content-redirect.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/head/profile.found	✓ Pass	✓ Pass	✓ Pass
/test/html/manifest.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/misc/string/dot-dot-slash-prefix.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/misc/string/dot-slash-prefix.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/misc/string/string-known-extension.pdf	✓ Pass	❌ FAIL	❌ FAIL
/test/html/misc/string/url-string.found	✓ Pass	❌ FAIL	❌ FAIL
/test/html/misc/url/full-url.found	✓ Pass	❌ FAIL	✓ Pass
/test/html/misc/url/path-relative-url.found	✓ Pass	❌ FAIL	✓ Pass
/test/html/misc/url/protocol-relative-url.found	✓ Pass	❌ FAIL	✓ Pass
/test/html/misc/url/root-relative-url.found	✓ Pass	❌ FAIL	✓ Pass
/test/javascript/interactive/js-delete.found	❌ FAIL	✓ Pass	✓ Pass
/test/javascript/interactive/js-post-event-listener.found	❌ FAIL	✓ Pass	✓ Pass
/test/javascript/interactive/js-post.found	❌ FAIL	❌ FAIL	✓ Pass
/test/javascript/interactive/js-put.found	❌ FAIL	❌ FAIL	✓ Pass
/test/javascript/interactive/listener-and-event-attribute-first.found	❌ FAIL	✓ Pass	✓ Pass
/test/javascript/interactive/listener-and-event-attribute-second.found	❌ FAIL	✓ Pass	✓ Pass
/test/javascript/interactive/multi-step-request-event-attribute.found	❌ FAIL	✓ Pass	❌ FAIL
/test/javascript/interactive/multi-step-request-event-listener-div-dom.found	❌ FAIL	❌ FAIL	❌ FAIL
/test/javascript/interactive/multi-step-request-event-listener-div.found	❌ FAIL	❌ FAIL	❌ FAIL
/test/javascript/interactive/multi-step-request-event-listener-dom.found	❌ FAIL	❌ FAIL	❌ FAIL
/test/javascript/interactive/multi-step-request-event-listener.found	❌ FAIL	❌ FAIL	❌ FAIL
/test/javascript/interactive/multi-step-request-redefine-event-attribute.found	❌ FAIL	✓ Pass	❌ FAIL
/test/javascript/interactive/multi-step-request-remove-button.found	❌ FAIL	❌ FAIL	❌ FAIL
/test/javascript/interactive/multi-step-request-remove-event-listener.found	❌ FAIL	❌ FAIL	❌ FAIL
/test/javascript/interactive/two-listeners-first.found	❌ FAIL	❌ FAIL	✓ Pass
/test/javascript/interactive/two-listeners-second.found	❌ FAIL	❌ FAIL	✓ Pass
/test/javascript/misc/automatic-post.found	❌ FAIL	❌ FAIL	❌ FAIL
/test/javascript/misc/comment.found	✓ Pass	❌ FAIL	❌ FAIL
/test/javascript/misc/string-concat-variable.found	❌ FAIL	❌ FAIL	❌ FAIL
/test/javascript/misc/string-variable.found	✓ Pass	❌ FAIL	❌ FAIL
/test/misc/known-files/robots.txt.found	✓ Pass	❌ FAIL	❌ FAIL
/test/misc/known-files/sitemap.xml.found	✓ Pass	❌ FAIL	❌ FAIL

Configuration

Config	Details
Frequency	Daily
Scripts	https://github.com/zapbot/zap-mgmt-scripts/blob/master/scans/crawlmaze/
Action	https://github.com/zapbot/zap-mgmt-scripts/actions/workflows/zap-vs-crawlmaze.yml

Settings

The latest Nightly ZAP Docker image is run with the default settings against this app with the following exceptions:

The traditional Spider “maxDepth” is set to 10 to find the deeper links.
The number of browsers launched by the Ajax Spider is set to 10 to speed up the crawling.

ZAP vs Security Crawl Maze

All URLs

85%

Configuration

Settings