Puppeteer Page.Evaluate Queryselectorall Return Empty Objects

Puppeteer page.evaluate querySelectorAll return empty objects

The values returned from evaluate function should be json serializeable.
https://github.com/GoogleChrome/puppeteer/issues/303#issuecomment-322919968

the solution is to extract the href values from the elements and return it.

 await this.page.evaluate((sel) => {
        let elements = Array.from(document.querySelectorAll(sel));
        let links = elements.map(element => {
            return element.href
        })
        return links;
    }, sel);

puppeteer page.evaluate() returns empty object

Looks like the problem is really with waiting, you are looking for elements even if full dom content isnt fully loaded.

  const scrapeNinja = async () => {
  const browser = await puppeteer.launch({headless: false})

  const page = await browser.newPage()

  await page.goto(`https://poe.ninja/challenge/builds?time-machine=day-6`, {
    waitUntil: 'networkidle2',
  })
  

  const getArray = await page.$$eval('#openSidebar > div > section:nth-child(3) > div > div > div > ul li .css-1h2ruwl',
    el => el.map(item => item.textContent))

  console.log(getArray)
}

scrapeNinja()

This code works perfectly for me, even you dont have to initialize array. In the future use networkidle2 in waitUntil option

puppeteer page.evaluate returns an empty object

You are trying to pass an element handle from the browser to your puppeteer application. To transfer data from the browser puppeteer uses JSON.stringify and the element handle gets transformed into an empty object.

You could try to pass something else back to your application like the text content of the element.

const browser = await pptr.launch()
const page = await browser.newPage()
await page.goto("url")
const price = await page.evaluate(() => {
    return document.querySelector("selector").textContent
})
console.log(price)

Puppeteer page.evaluate querySelectorAll is always empty

I found a solution for this, I am putting this out there for whoever needs it. when you get array of elements, you need to spicify the attribute of the element that you want

let test= await page.evaluate((sele) =>{

    const elements = Array.from(document.querySelectorAll(sele))
    let links = elements.map(element=>{
        return element.getAttribute('fields');
    })
    return links;
    },'area[templateid="ucChart_pnlip"]')

console.log(test)

Cannot get querySelectorAll to work with puppeteer (returns undefined)

The callback passed to page.evaluate runs in the emulated page context, not in the standard scope of the Node script. Expressions can't be passed between the page and the Node script without careful considerations: most importantly, if something isn't serializable (converted into plain JSON), it can't be transferred.

querySelectorAll returns a NodeList, and NodeLists only exist on the front-end, not the backend. Similarly, NodeLists contain HTMLElements, which also only exist on the front-end.

Put all the logic that requires using the data that exists only on the front-end inside the .evaluate callback, for example:

const numberOfDivs = await page.evaluate(_ => {
  return document.querySelectorAll('div').length;
});

or

const firstDivText = await page.evaluate(_ => {
  return document.querySelector('div').textContent;
});

Puppeteer returns empty objects

Unfortunately, page.evaluate() can only transfer serializable values (roughly, the values JSON can handle). As document.querySelectorAll() returns collection of DOM elements that are not serializable (they contain methods and circular references), each element in the collection is replaced with an empty object. You need to return either serializable value (for example, an array of hrefs) or use something like page.$$(selector) and ElementHandle API.

---

Related Topics