Chrome Extension - Get Dom Content

Chrome Extension - Get DOM content

The terms "background page", "popup", "content script" are still confusing you; I strongly suggest a more in-depth look at the Google Chrome Extensions Documentation.

Regarding your question if content scripts or background pages are the way to go:

Content scripts: Definitely

Content scripts are the only component of an extension that has access to the web-page's DOM.

Background page / Popup: Maybe (probably max. 1 of the two)

You may need to have the content script pass the DOM content to either a background page or the popup for further processing.

Let me repeat that I strongly recommend a more careful study of the available documentation!

That said, here is a sample extension that retrieves the DOM content on StackOverflow pages and sends it to the background page, which in turn prints it in the console:

background.js:

// Regex-pattern to check URLs against. 
// It matches URLs like: http[s]://[...]stackoverflow.com[...]
var urlRegex = /^https?:\/\/(?:[^./?#]+\.)?stackoverflow\.com/;

// A function to use as callback
function doStuffWithDom(domContent) {
    console.log('I received the following DOM content:\n' + domContent);
}

// When the browser-action button is clicked...
chrome.browserAction.onClicked.addListener(function (tab) {
    // ...check the URL of the active tab against our pattern and...
    if (urlRegex.test(tab.url)) {
        // ...if it matches, send a message specifying a callback too
        chrome.tabs.sendMessage(tab.id, {text: 'report_back'}, doStuffWithDom);
    }
});

content.js:

// Listen for messages
chrome.runtime.onMessage.addListener(function (msg, sender, sendResponse) {
    // If the received message has the expected format...
    if (msg.text === 'report_back') {
        // Call the specified callback, passing
        // the web-page's DOM content as argument
        sendResponse(document.all[0].outerHTML);
    }
});

manifest.json:

{
  "manifest_version": 2,
  "name": "Test Extension",
  "version": "0.0",
  ...

  "background": {
    "persistent": false,
    "scripts": ["background.js"]
  },
  "content_scripts": [{
    "matches": ["*://*.stackoverflow.com/*"],
    "js": ["content.js"]
  }],
  "browser_action": {
    "default_title": "Test Extension"
  },

  "permissions": ["activeTab"]
}

Chrome Extension - Obtaining Active Tab's DOM information through background script

Problem 1

Per the documentation this API method returns a Promise when there's no callback parameter.

To get the value of the Promise, add await and mark the function as async:

chrome.tabs.onActivated.addListener(async info => {

  let domRes = await chrome.scripting.executeScript({
    target: {tabId: info.tabId},
    func: getDOM,
  }).catch(console.error);
  if (!domRes) return;

  console.log(domRes);
});

There's a catch because some tabs don't support injection e.g. the start tab or chrome:// tabs.

Problem 2

In JavaScript += doesn't work with arrays, but only with numbers/strings. Use htmlarr.push() instead. There's also no need to convert the array to an object via Object.assign.

Actually, let's rewrite getDOM using Array.from:

function getDOM() {
  return Array.from(
    document.getElementsByTagName('div'),
    el => el.innerHTML
  );
}

How to get HTML element in javascript in Chrome Extension

Your extension scripts runs in a separate window, so it cannot access DOM of other pages via its own document object. You need to specify tabs permissions in your extension manifest, and inject asome javascript into the page itself

chrome.browserAction.onClicked.addListener(function(tab) {
  chrome.tabs.executeScript(tab.id,{code:   `document.getElementById("productTitle").innerText`},sendCurrentTitle);
});

Please not that you need to use title[0] inside sendCurrentTitle, as the callback receives an array

Access DOM elements through chrome extension

The solution:
You need a manifest file, a background script and a content script. This is not really clear in the documentation that you have to use it and also, how to use it. For alerting the full dom, see here. Because I have a hard time finding a complete solution that actually works and not just snippets that are useless for newbies, like me, I included a specific solution:

manifest.json

{
    "manifest_version": 2,
    "name":    "Test Extension",
    "version": "0.0",

    "background": {
        "persistent": false,
        "scripts": ["background.js"]
    },
    "content_scripts": [{
        "matches": ["file:///*"],
        "js":      ["content.js"]
    }],
    "browser_action": {
        "default_title": "Test Extension"
    },

    "permissions": ["activeTab"]
}

content.js

/* Listen for messages */
chrome.runtime.onMessage.addListener(function(msg, sender, sendResponse) {
    /* If the received message has the expected format... */
    if (msg.text && (msg.text == "report_back")) {
        /* Call the specified callback, passing 
           the web-pages DOM content as argument */
    sendResponse(document.getElementById("mybutton").innerHTML);
    }
});

background.js

/* Regex-pattern to check URLs against. 
   It matches URLs like: http[s]://[...]stackoverflow.com[...] */
var urlRegex = /^file:\/\/\/:?/;

/* A function creator for callbacks */
function doStuffWithDOM(element) {
    alert("I received the following DOM content:\n" + element);
}

/* When the browser-action button is clicked... */
chrome.browserAction.onClicked.addListener(function(tab) {
    /*...check the URL of the active tab against our pattern and... */
    if (urlRegex.test(tab.url)) {
        /* ...if it matches, send a message specifying a callback too */
        chrome.tabs.sendMessage(tab.id, { text: "report_back" },
                                doStuffWithDOM);
    }
});

index.html

<html>
  <button id="mybutton">click me</button>
</html>

Just save the index.html somewhere and load in the folder as an extension, containing the three other files. Open the index.html and push the extension button. It should show "click me".

Inserting DOM elements using content script in chrome extension

For Twitter, MutationObserver is the way to go, and for how to use it, I prefer this Mozilla guide.

To get is working correctly, note the syntax: here in registering the MutationObserver, it says watch the entire body and descendants, and call function appendCustomNode when change occurs.

// Step 1. Create an observer instance linked to the callback function
// const observer = new MutationObserver(callback);
// Step 2. Start observing the target node for configured mutations
// observer.observe(targetNode, config);

const observer = new MutationObserver(appendCustomNode); 
observer.observe(document.body, {subtree: true, childList: true});

Next step is to define this function appendCustomNode.

In planning how to append nodes to the cards, it is necessary to take into consideration that you will see the same node possibly multiple times, which requires having a strategy to append the custom element only once. Picking some sufficiently random value, then tagging all previously seen nodes with that value, will help accomplish this. Here is an example:

const customTag = `tag-${Date.now()}` // your choice of a valid & unique value

function appendCustomNode(){

  // find all timeline cards, their tag name is article
  const cards = document.getElementsByTagName('article');

  // check each card
  for (let n = 0; n < cards.length; n++) {

     const card = cards[n];

     // check that cards is new / has not been seen before
     if (!card.hasAttribute(customTag)) { 

       addButton(card);

       // mark the card as "processed" by adding the custom tag
       card.setAttribute(customTag, true);
     }
  }
}

Lastly implement the logic to append the button you want to add to the card. Here card attribute represents the DOM node of one timeline item.

function addButton(card){

   const myButton = document.createElement('div');
   myButton.innerText = 'hello world';

   // select the div of action buttons
   // or wherever you want to insert your custom node 
   // SEE NOTE BELOW
   const target = card.querySelector('....?....')

   // append/prepend/insertAdjacentHTML the button here
   target.append(myButton);
}

I will leave this last part open for a few reasons: when deciding how to find the buttons row in the card, if using classes as a selector, the implementation breaks if the classes change, which is out of your control. This is always a challenge when extending 3rd party web apps. I suggest trying to find a more reliable selector (which is why earlier I used tag name article to select cards). Secondly the buttons/card structure changes periodically, so that is another point to consider: where do you want to insert this custom node. Lastly this approach does not require jQuery.

How to access the webpage DOM/HTML from an extension popup or background script?

ManifestV3 service worker doesn't have any DOM/document/window.

ManifestV3/V2 extension pages (and the scripts inside) have their own DOM, document, window, and a chrome-extension:// URL (use devtools for that part of the extension to inspect it).

You need a content script to access DOM of web pages and interact with a tab's contents. Content scripts will execute in the tab as a part of that page, not as a part of the extension, so don't load your content script(s) in the extension page, use the following methods:

Method 1. Declarative

manifest.json:

"content_scripts": [{
  "matches": ["*://*.example.com/*"],
  "js": ["contentScript.js"]
}],

It will run once when the page loads. After that happens, use messaging but note, it can't send DOM elements, Map, Set, ArrayBuffer, classes, functions, and so on - it can only send JSON-compatible simple objects and types so you'll need to manually extract the required data and pass it as a simple array or object.

Method 2. Programmatic

ManifestV2:
Use chrome.tabs.executeScript in the extension script (like the popup or background) to inject a content script into a tab on demand.
The callback of this method receives results of the last expression in the content script so it can be used to extract data which must be JSON-compatible, see method 1 note above.
Required permissions in manifest.json:
- Best case: "activeTab", suitable for a response to a user action (usually a click on the extension icon in the toolbar). Doesn't show a permission warning when installing the extension.
- Usual: "*://*.example.com/" plus any other sites you want.
- Worst case: "<all_urls>" or "*://*/", "http://*/", "https://*/" - when submitting into Chrome Web Store all of these put your extension in a super slow review queue because of broad host permissions.
ManifestV3 differences to the above:
Use chrome.scripting.executeScript.
Required permissions in manifest.json:
- "scripting" - mandatory
- "activeTab" - ideal scenario, see notes for ManifestV2 above.
If ideal scenario is impossible add the allowed sites to host_permissions in manifest.json.

Struggling to get DOM data in Chrome extension

Remove default_popup from browser_action:

The problem is that you are defining a default_popup for the browser_action in your manifest.json. Change the browser_action to:

"browser_action": {
  "default_title": "Test Extension",
  "default_icon": "icon.png"
},

If you define a default_popup, then Chrome tries to display the popup and does not send a click event to your background script. You should have seen a popup displayed saying that your file was not found.

Only injecting into https scheme pages:
Given that you are only injecting your content script into https:// pages, make sure you are testing on a page that is using the https scheme.

You may find it easier to test by injecting your content script into all URLs. You can do so by changing the content_script key in your manifest.json to:

"content_scripts": [{
  "matches": ["<all_urls>"],
  "js": ["content.js"]
}],

It is also not injected into pages in the chrome scheme (e.g. chrome://extensions/) (even when using <all_urls>). Thus, your extension will not work on those pages.

You must load content page after loading the extension:
You will also need to make sure you have reloaded the page, opened a new tab, or navigated to a new page after loading, or reloading, the extension. Your content script is not injected into already loaded pages.

The Output is in the console for your background page:
In addition, you will need to be looking at the console for your background page. There are multiple consoles which you might need to be looking at depending on in which context/scope you are calling console.log(). The answer linked in the first sentence of this section describes how to view them.

manifest.json with all mentioned changes:

{
  "manifest_version": 2,
  "name": "Test Extension",
  "version": "0.0",
  "background": {
    "persistent": false,
    "scripts": ["background.js"]
  },
  "content_scripts": [{
    "matches": ["<all_urls>"],
    "js": ["content.js"]
  }],
  "browser_action": {
    "default_title": "Test Extension",
    "default_icon": "icon.png"
  },

  "permissions": ["activeTab"]
}

Chrome Extension - Get Dom Content