How to Remove Script Tags With Beautifulsoup

Can I remove script tags with BeautifulSoup?

from bs4 import BeautifulSoup
soup = BeautifulSoup('<script>a</script>baba<script>b</script>', 'html.parser')
for s in soup.select('script'):
    s.extract()
print(soup)
baba

Remove all style, scripts, and html tags from an html page

It looks like you almost have it. You need to also remove the html tags and css styling code. Here is my solution (I updated the function):

def cleanMe(html):
    soup = BeautifulSoup(html, "html.parser") # create a new bs4 object from the html data loaded
    for script in soup(["script", "style"]): # remove all javascript and stylesheet code
        script.extract()
    # get text
    text = soup.get_text()
    # break into lines and remove leading and trailing space on each
    lines = (line.strip() for line in text.splitlines())
    # break multi-headlines into a line each
    chunks = (phrase.strip() for line in lines for phrase in line.split("  "))
    # drop blank lines
    text = '\n'.join(chunk for chunk in chunks if chunk)
    return text

How can I remove all different script tags in BeautifulSoup?

You are asking about get_text():

If you only want the text part of a document or tag, you can use the
get_text() method. It returns all the text in a document or beneath a
tag, as a single Unicode string

td = soup.find("td")
td.get_text()

Note that .string would return you None in this case since td has multiple children:

If a tag contains more than one thing, then it’s not clear what
.string should refer to, so .string is defined to be None

Demo:

>>> from bs4 import BeautifulSoup
>>> 
>>> soup = BeautifulSoup(u"""
... <td><a href="http://www.irit.fr/SC">Signal et Communication</a>
... <br/><a href="http://www.irit.fr/IRT">Ingénierie Réseaux et Télécommunications</a>
... </td>
... """)
>>> 
>>> td = soup.td
>>> print td.string
None
>>> print td.get_text()
Signal et Communication
Ingénierie Réseaux et Télécommunications

Beautiful Soup Fails to Remove ALL Script Tags

This is happening because some of the <script> tags are within HTML comments (<!-- ... -->).

You can extract these HTML comments checking if the tags are of the type Comment:

from bs4 import BeautifulSoup, Comment

soup = BeautifulSoup(html, "html.parser")

# Find all comments on the website and remove them, most of them contain `script` tags
[
    comment.extract()
    for comment in soup.findAll(text=lambda text: isinstance(text, Comment))
]

# Find all other `script` tags and remove them
[tag.extract() for tag in soup.findAll("script")]

print(soup.prettify())

Remove script tags inside p tags using beautifulsoup

First, remove all script tags and then get the text:

soup = BeautifulSoup(open('MUFC.html'))

for script in soup.find_all('script'):
    script.extract()

paragraphs = soup.find_all('p')
for paragraph in paragraphs:
    print(paragraph.get_text(strip=True))

Remove Script tag and on attributes from HTML

As for removing all the attributes that start with on, you can try this

It uses the regex:

\s?on\w+="[^"]+"\s?

And substitutes with the empty string (deletion). So in Python it should be:

pattern = re.compile(ur'\s?on\w+="[^"]+"\s?')
subst = u""
result = re.sub(pattern, subst, file) 

If you are trying to match anything between the script tags try:

<script[\s\S]+?/script>

DEMO

The problem with your regex is that that dot (.) doesn't match newline character. Using a complemented set will match every single character possible. And make sure use the ? in [\s\S]+? so that it is lazy instead of greedy.

---

Related Topics