Act

drobnikj/check-crawler-results

  • Builds
  • latest 0.1.23 / 2018-04-27
  • beta 0.1.20 / 2018-04-27
  • Created 2017-06-05
  • Last modified 2018-04-27
  • grade 14

Description

Act checks results from crawler execution depends on input. It sends mail if one or more errors was found in crawler results. It is designed to run from crawler finish webhook.


API

To run the act, send a HTTP POST request to:

https://api.apify.com/v2/acts/drobnikj~check-crawler-results/runs?token=<YOUR_API_TOKEN>

The POST payload will be passed as input for the act. For more information, read the docs.


Readme

apify-act-check-crawler-results

This act checks crawler results or default act dataset items and send notification if finds some errors. It is designed to run from crawler finish webhook.

Usage Crawler

For a specific crawler set the following parameters:

Finish webhook URL (finishWebhookUrl)

https://api.apify.com/v2/acts/drobnikj~check-crawler-results/runs?token=APIFY_API_TOKEN

Finish webhook data

sampleCount
  • Number
  • Number of results that act checks
  • Default is 1000
minOutputtedPages
  • Number
  • Indicates minimum outputted pages of crawler to checks if attribute is set.
jsonSchema
  • Object
  • If jsonSchema is set act check all sample results against schema.
compareWithPreviousExecution
  • Boolean
  • If compareWithPreviousExecution is set to true act compare results with previous execution.
  • If tag for execution is set compare act result from previous results with same tag.

notifyTo

  • String
  • Mail where act send notification if found error
runActOnSuccess
  • Object
  • If act found errors runs this act.
  • Example:
    {
      "id": "apify/send-mail",
      "input": {
          "to": "[email protected]",
          "subject": "test on success",
          "text": "No errors in crawler Amazon"
      }
    }
    NOTE: If you didn't set input, it set from input of main act and errors output.
runActOnError
  • Object
  • If didn't find any errors runs this act.
  • Same format as runActOnSuccess

Usage Act

You can call it from other Act, for example:

await Apify.call('drobnikj/check-crawler-results', {
    actId: 's7Jj8ik07gfV',
    runId: 'sd86hGfHk0Uh6gF',
    options: {
        minOutputtedPages: 1000,
    }
});

actId

  • String
  • Act ID you want to check

runId

  • String
  • Run ID of act you want to check

options

  • Object
  • Options for checking
  • There are same params as in Finish webhook data except compareWithPreviousExecution
  • sampleCount, jsonSchema, notifyTo, runActOnSuccess, runActOnError