Actor

petr_cermak/merge-act-crawler-results

  • Builds
  • latest 0.0.9 / 2017-12-22
  • Created 2017-12-22
  • Last modified 2017-12-22
  • grade 3

Description

Act for merging act-crawler results into one OUTPUT.


API

To run the actor, send a HTTP POST request to:

https://api.apify.com/v2/acts/petr_cermak~merge-act-crawler-results/runs?token=<YOUR_API_TOKEN>

The POST payload will be passed as input for the actor. For more information, read the docs.


Example input

Content type: application/json; charset=utf-8

{ "storeId": "KEY_VALUE_STORE_ID" }

Source code

Based on the apify/actor-node-basic Docker image (see docs).

const Apify = require('apify');
const request = require('request-promise');

Apify.main(async () => {
    const input = await Apify.getValue('INPUT');
    
    const storeId = input.storeId;
    const simpl = input.simplified === false || input.simplified === 0 ? '' : 'SIMPLIFIED-';
    
    let data = [];
    for(let i = 1; i > 0; i++){
        try{
            const result = await Apify.client.keyValueStores.getRecord({
                storeId: storeId, 
                key: 'RESULTS-' + simpl + i + '.json'
            });
            //console.dir(result);
            if(!result){break;}
            data = data.concat(result.body);
        }
        catch(e){
            console.log(e);
            break;
        }
    }
    
    await Apify.setValue('OUTPUT', data);
});