No credit card required

Resurrect run on Out of memory

lukaskrivka/resurrect-on-out-of-memory

No credit card required

Simple helper actor to resurrect your runs when they run out of memory.

.editorconfig

1root = true
2
3[*]
4indent_style = space
5indent_size = 4
6charset = utf-8
7trim_trailing_whitespace = true
8insert_final_newline = true
9end_of_line = lf

.eslintrc

1{
2    "extends": "@apify"
3}

.gitignore

1# This file tells Git which files shouldn't be added to source control
2
3.idea
4node_modules
5
6apify_storage

Dockerfile

1# First, specify the base Docker image. You can read more about
2# the available images at https://sdk.apify.com/docs/guides/docker-images
3# You can also use any other image from Docker Hub.
4FROM apify/actor-node:16
5
6# Second, copy just package.json and package-lock.json since it should be
7# the only file that affects "npm install" in the next step, to speed up the build
8COPY package*.json ./
9
10# Install NPM packages, skip optional and development dependencies to
11# keep the image small. Avoid logging too much and print the dependency
12# tree for debugging
13RUN npm --quiet set progress=false \
14 && npm install --only=prod --no-optional \
15 && echo "Installed NPM packages:" \
16 && (npm list --only=prod --no-optional --all || true) \
17 && echo "Node.js version:" \
18 && node --version \
19 && echo "NPM version:" \
20 && npm --version
21
22# Next, copy the remaining files and directories with the source code.
23# Since we do this after NPM install, quick build will be really fast
24# for most source file changes.
25COPY . ./
26
27# Optionally, specify how to launch the source code of your actor.
28# By default, Apify's base Docker images define the CMD instruction
29# that runs the Node.js source code using the command specified
30# in the "scripts.start" section of the package.json file.
31# In short, the instruction looks something like this:
32#
33# CMD npm start

apify.json

1{
2	"name": "resurrect-on-out-of-memory",
3	"version": "0.0",
4	"buildTag": "latest",
5	"env": null,
6	"template": "project_empty"
7}

package.json

1{
2	"name": "resurrect-on-out-of-memory",
3	"version": "0.0.1",
4	"description": "This is a boilerplate of an Apify actor.",
5	"dependencies": {
6		"apify": "^2.2.2"
7	},
8	"devDependencies": {
9		"@apify/eslint-config": "^0.1.3",
10		"eslint": "^7.0.0"
11	},
12	"scripts": {
13		"start": "node src/main.js",
14		"lint": "./node_modules/.bin/eslint ./src --ext .js,.jsx",
15		"lint:fix": "./node_modules/.bin/eslint ./src --ext .js,.jsx --fix",
16		"test": "echo \"Error: oops, the actor has no tests yet, sad!\" && exit 1"
17	},
18	"author": "It's not you it's me",
19	"license": "ISC"
20}

src/main.js

1const Apify = require('apify');
2
3const { log } = Apify.utils;
4
5Apify.main(async () => {
6    // We should only be called via webhook
7    const input = await Apify.getInput();
8
9    const { email } = input;
10
11    const client = Apify.newClient();
12
13    const { resource } = input;
14    const { id } = resource;
15
16    // We check if the run was out of memory (137 status)
17    const { exitCode, status } = await client.run(id).get();
18
19    if (status !== 'FAILED') {
20        log.warning(`Run ${id} finished with status ${status}, not FAILED. We have nothing to resurrect. `
21            + `Did you configure the webhook correctly only for FAILED status?`);
22    }
23    const isOutOfMemory = exitCode === 137;
24
25    if (isOutOfMemory) {
26        log.info(`Run ${id} was out of memory (status 137), we will resurrect it.`);
27        await client.run(id).resurrect();
28        log.info(`Run resurrected.`);
29        if (email) {
30            log.info(`Sending email report and finishing`);
31            const emailInput = {
32                to: email,
33                subject: `Apify - Run ${id} was out of memory, resurrected`,
34                text: `Apify - Run ${id} was out of memory, resurrected`,
35            };
36            await client.actor('apify/send-mail').start(emailInput);
37        }
38    } else {
39        log.info(`Run ${id} was not out of memory, it failed with exit code ${exitCode}. We will not resurrect it.`);
40    }
41});

Developer

Lukáš Křivka

Actor metrics

1 monthly users
100.0% runs succeeded
days response time
Created in Apr 2022
Modified about 2 years ago

Categories

Developer tools

Web Scraper

apify/web-scraper

Crawls arbitrary websites using the Chrome browser and extracts data from pages using a provided JavaScript code. The actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.

Apify

63.4k

Website Content Crawler

apify/website-content-crawler

Automatically crawl and extract text content from websites with documentation, knowledge bases, help centers, or blogs. This Actor is designed to provide data to feed, fine-tune, or train large language models such as ChatGPT or LLaMA.

Apify

13.3k

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

4.3k

Puppeteer Scraper

apify/puppeteer-scraper

Crawls websites with the headless Chrome and Puppeteer library using a provided server-side Node.js code. This crawler is an alternative to apify/web-scraper that gives you finer control over the process. Supports both recursive crawling and list of URLs. Supports login to website.

Apify

3.3k

Merge, Dedup & Transform Datasets

lukaskrivka/dedup-datasets

The ultimate dataset processor. Extremely fast merging, deduplications & transformations all in a single run.

Lukáš Křivka

1.7k

Actor fail manager

lukaskrivka/actor-fail-manager

Automatically triggered on a failed run to analyze if the run should be resurrected and to create an error report for the author.

Lukáš Křivka

2.7k

BeautifulSoup Scraper

apify/beautifulsoup-scraper

Crawls websites using raw HTTP requests. It parses the HTML with the BeautifulSoup library and extracts data from the pages using Python code. Supports both recursive crawling and lists of URLs. This Actor is a Python alternative to Cheerio Scraper.

Apify

609

Website Screenshot Generator

apify/screenshot-url

Create a screenshot of a website based on a specified URL. The screenshot is stored as the output in a key-value store. It can be used to monitor web changes regularly after setting up the scheduler.

Apify

2.4k

Anti Captcha Recaptcha

petr_cermak/anti-captcha-recaptcha

🧰 Actor for solving Google reCAPTCHA using the anti-captcha.com service. You need to have an anti-captcha subscription.

Petr Cermak

1.3k

Page Scraping Analyzer

apify/page-analyzer

Performs analysis of a webpage to figure out the best way how to scrape its data. Provide a URL and data points to find and get back a detailed dashboard showing how the data can be scraped. Works with initial and rendered HTML, JavaScript variables and dynamically loaded data.

Apify

1.1k

Apify ❤️ Python: Releasing a Python SDK for Actors

Apify Crawler product to be replaced by an Apify Actor

September updates from Apify

Build new tools

Are you a developer? Build your own Actors and run them on Apify.

Learn more

Get a custom solution

Get a custom web scraping or RPA solution.

Book a demo

Resurrect run on Out of memory

.editorconfig

.eslintrc

.gitignore

Dockerfile

apify.json

package.json

src/main.js

Web Scraper

Website Content Crawler

Cheerio Scraper

Puppeteer Scraper

Merge, Dedup & Transform Datasets

Actor fail manager

BeautifulSoup Scraper

Website Screenshot Generator

Anti Captcha Recaptcha

Page Scraping Analyzer

Related articles

Where next?

Build new tools

Get a custom solution

.editorconfig

.eslintrc

.gitignore

Dockerfile

apify.json

package.json

src/main.js

.editorconfig

.eslintrc

.gitignore

Dockerfile

apify.json

package.json

src/main.js

You might also like these Actors

Web Scraper

Website Content Crawler

Cheerio Scraper

Puppeteer Scraper

Merge, Dedup & Transform Datasets

Actor fail manager

BeautifulSoup Scraper

Website Screenshot Generator

Anti Captcha Recaptcha

Page Scraping Analyzer

Related articles

Where next?

Build new tools

Get a custom solution

.editorconfig

.eslintrc

.gitignore

Dockerfile

apify.json

package.json

src/main.js