Act

petr_cermak/run-crawler-url-list-batch

  • Builds
  • latest 0.0.22 / 2017-11-22
  • Created 2017-10-04
  • Last modified 2017-11-22
  • grade 2

Description

This act starts crawler runs for a specified list of URLs.


API

To run the act, send a HTTP POST request to:

https://api.apify.com/v2/acts/petr_cermak~run-crawler-url-list-batch/runs?token=<YOUR_API_TOKEN>

The POST payload will be passed as input for the act. For more information, read the docs.


Example input

Content type: application/json; charset=utf-8

{
    "urls": ["URL_1", "URL_2", "URL_3"],
    "label": "START_URL_LABEL",
    "batch": "N_OF_URLS_IN_BATCH"
}

Readme

act-run-crawler-url-list-batch

This act starts crawler runs for a specified list of URLs.

It starts the runs in batches of "N_OF_URLS_IN_BATCH", always waits for one batch to finish before starting the next. After all runs finish, the act returns a list of all ExecutionIDs.

Example input:

{
    "urls": ["URL_1", "URL_2", "URL_3", ...],
    "label": "START_URL_LABEL",
    "batch": "N_OF_URLS_IN_BATCH"
}

Example output:

{
    "executionIds": [
        "EXECUTION_ID_1",
        "EXECUTION_ID_2",
        "EXECUTION_ID_3",
        ...
    ]
}

It is useful to chain output of this act into "petr_cermak/executions_merge" act.