7 days trial then $40.00/month - No credit card required now
Work Scraper
7 days trial then $40.00/month - No credit card required now
Extract data from the top freelancing websites. Search by URL or search terms, filter by categories, English level, and hourly rate. Get info about freelancers and agencies without login. Download your data as an HTML table, JSON, CSV, Excel, or XML.
Scraper Skipping Results & Incorrect Upload Time
Closed
I ran the scraper a few times this morning as it was giving me the 100 results I asked for, capping out at 74 and a lesser number another time when I requested 100 results. I increased the time-out limit, which helped, but it appears less reliable than I remember.
It appears on this run, it gave me 100 results, but to do so, it skipped the results on some pages and went to further pages. There are 10 results per page, it skipped some, I think, on page 6 and another, only picking up 4 results, and then continued through to Page 13 to get the 100 results. I would like (and expect it) to give me 10 results per page, and up to page 10 only - not skip over results - any idea what could be causing this?
I also think the 'createdAt', which I guess was when uploaded to Upwork, is incorrect. Some of the results, for instance, say they were uploaded on the 5th of July, but when I check Upwork it was actually uploaded on the 4th, this is happening even when I account for timezone differences.
Looking to purchase this scraper soon for my monthly workflow, but need to know it's picking up all the results as expected.
Thank you for sharing your run. I am taking a look to see what is happening.
I don't see any error in the createdAt value. The value is stored in UTC so you need to covert it to your current timezone.
I have added some improvements to the scraper. Please let me know how it is performing for you know. I am closing the ticket but you can still reply back,
Gustavo Rudiger- 7 monthly users
- 100.0% runs succeeded
- 0.25 days response time
- Created in Jun 2020
- Modified 23 days ago
