I run a ton of Puppeteer jobs (300k in the last month), currently on EC2 and Digital Ocean VM's, mostly due to the subtle difficulties of running Puppeteer on Lambda.
Will certainly have a look at this project and contribute where possible.
My main concerns are not so much cold start time, as for my use case this is not really a huge issue, but mainly the performance of Chrome on AWS Lambda boxes. The rendering, navigation etc. needs to be snappy.
Here's another alternative lambda layer containing headless chrome with and puppeteer example: https://github.com/RafalWilinski/serverless-puppeteer-layers
This is fantastic.
- I'm just getting started with Lambda so pardon if this is ignorant, but what's the cold start time of Chromium? Or can you warm start it somehow?
- Since scraping often depends on state, wouldn't you hit a timeout doing longer scraping joba?
What is the advantage of running automated tests as lambda ? Typically when automation tests are run, they are long running processes and lambda execution may not be suitable. The cold start times of lambda is another challenge. What is a good practice/model for running suites ? One test per lambda or, one spec per lambda ? Still inclined to have ec2 instances created and destroyed via devops tools like terraform to run automation. Thoughts please.