Solving wait for responses problem in puppeteer
Picture by Aron Visuals
Waiting correctly for requests in puppeteer could be tricky, this package ensures waiting for all outgoing requests.
Introduction
If you previously tried to wait for API’s responses to be received so do something like manipulating the DOM, there is a high chance that you have struggled to understand the behavior of using built-in puppeteer functions or other packages. Usually, when you need to track many requests at once or some requests are lately received like E2E testing or scraping an infinite scroll page and mostly you do not know which requests to track or to wait for.
I have built a simple but powerful package, it may be used even to wait for hundreds of requests at once and guarantees to wait for all responses.
Quick Tutorial
Using the package is fairly simple, you can install it with npm or yarn easily:
npm i puppeteer-response-waiter
To install it with yarn
yarn add puppeteer-response-waiter
Usage
These examples from the documentation demonstrate the basic usage of the package.
const puppeteer = require('puppeteer');
const {ResponseWaiter} = require('puppeteer-response-waiter');
let browser = await puppeteer.launch({ headless: false });
let page = await browser.newPage();
let responseWaiter = new ResponseWaiter(page);
await page.goto('http://somesampleurl.com');
// start listening
responseWaiter.listen();
// do something here to trigger requests
await responseWaiter.wait();
// all requests are finished and responses are all returned back
// remove listeners
responseWaiter.stopListening();
await browser.close();
You can even filter the requests to wait for and can control the flow of requests and responses as you want to check correct responses, download responses, files, images…etc.
const puppeteer = require('puppeteer');
const {ResponseWaiter} = require('puppeteer-response-waiter');
let browser = await puppeteer.launch({ headless: false });
let page = await browser.newPage();
let responseWaiter = new ResponseWaiter(page, {
waitFor: (req) => req.resourceType() == 'fetch',
// get you response here and do something with it
onResponse: async (response)=> console.log(await response.json())// do something with response
});
await page.goto('http://somesampleurl.com');
// start listening
responseWaiter.listen();
// do something here to trigger requests
await responseWaiter.wait();
// all requests are finished and responses are all returned back
// remove listeners
responseWaiter.stopListening();
await browser.close();
Conclusion
We showed an overview of the uses cases and some usage examples, the package has some other configurations to work with, for more information check the documentation and repository on GitHub or on Npm.
