Act

petr_cermak/crawler-results-deduplicate

  • Builds
  • latest 0.0.4 / 2017-10-25
  • Created 2017-10-06
  • Last modified 2017-10-26
  • grade 3

Description

This acts takes crawler execution results and deduplicates them.


API

To run the act, send a HTTP POST request to:

https://api.apify.com/v2/acts/petr_cermak~crawler-results-deduplicate/runs?token=<YOUR_API_TOKEN>

The POST payload will be passed as input for the act. For more information, read the docs.


Example input

Content type: application/json

{
    "_id": "EXECUTION_ID",
    "data": "{
        \"compareKey\": \"YOUR_COMPARE_KEY\"
    }"
}

Readme

act-crawler-results-deduplicate

This acts takes crawler execution results and deduplicates them.

Example input:

{
    "_id": "EXECUTION_ID",
    "data": "{
        \"compareKey\": \"YOUR_COMPARE_KEY\"
    }"
}

The optional "data" attribute must be a stringified JSON and can contain a "compareKey" attribute. This key is used to select an object attribute for comparison. If no key is provided, whole object will be used for comparison. It is expected to be called from a crawler webhook.