New crawler enables smarter scans and more relevant findings

The crawler is the heart of the Detectify service. During the crawling phase, Detectify is trying to collect all possible pages of your website. That might seem like a simple task, and it used to be – back when the web was static. Now, more and more pages are generated dynamically, on the fly, by your customers as they visit your site.

Detectify has been working on an better and more thorough crawler for a long time and many of our awesome users tried out the new crawler when it was in beta and provided us with helpful feedback.

With this week’s release, we have now implemented the new crawler into our service, enabling you to run smarter scans and getting improved reports.

This is what the new crawler does:

• To be able to reach better results, our crawler covers pages rendered by javascript.
• The new crawler detect and collects dynamic pages much better than a regular crawler.
• We have focused on consistency and more precise and in-depth scans, which can result in slightly longer scan times.  However, with better filtering, you most likely reduce the scan time.

Please note that due to improved crawling, your new findings might differ from old reports.

If you have any feedback or suggestions, go ahead and drop us a line at hello[at]detectify.com. We will continuously work on improving Detecitfy as a service, to make sure you can produce safer code.

Thank you for helping us improve our service!

Happy scanning!