Documentation for crwlr / crawler (v1.7)

Attention: You're currently viewing the documentation for v1.7 of the crawler package.
This is not the latest version of the package.
If you didn't navigate to this version intentionally, you can click here to switch to the latest version.


When your crawler finally collected a lot of data, you'll probably want to store it. A convenient way to do this are stores. Just make your own store class extending the abstract Store class and add it to your crawler. The crawler will then automatically call the store() method of your store class with every Result object.

use Crwlr\Crawler\Result;
use Crwlr\Crawler\Stores\Store;

class MyStore extends Store
    public function store(Result $result): void
        // Store the Result however you prefer.

And where you're defining your crawling procedure:

$myCrawler->setStore(new MyStore());

The package ships with a very, very simple store class, the SimpleCsvFileStore. It will store your results in a CSV file.

You can also access your crawler's logger via $this->logger from within your store class after adding it to the crawler.