How to Set Headers Using Python's Urllib

How do I set headers using python's urllib?

adding HTTP headers using urllib2:

from the docs:

import urllib2
req = urllib2.Request('http://www.example.com/')
req.add_header('Referer', 'http://www.python.org/')
resp = urllib2.urlopen(req)
content = resp.read()

Making a POST request using urllib with multiple headers gives 400 Bad Request error

The server is failing in request.get_json(). It's only happening when the client sends both headers because that's when it reaches this line.

To fix it, change the client to send the data as JSON:

import json            # <-- Import json
import urllib.request
import urllib.parse

d = {"spam": 1, "eggs": 2, "bacon": 0}
data = json.dumps(d)   # <-- Dump the dictionary as JSON
data = data.encode()
req = urllib.request.Request("http://localhost:5000/random", data)
req.add_header('Content-Type', 'application/json')
req.add_header('Authorization', 12345)
with urllib.request.urlopen(req) as f:
    print(f.read().decode('utf-8'))

I hope this helps

urllib.urlretrieve with custom header

I found a way where you only have to add a few extra lines of code...

import urllib.request

opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
urllib.request.install_opener(opener)
urllib.request.urlretrieve("type URL here", "path/file_name")

Should you wish to learn about the details you can refer to the python documentation: https://docs.python.org/3/library/urllib.request.html

How can I add a header to urllib.request.urlretrieve keeping my variables?

Check the below link:
https://stackoverflow.com/a/7244263/5903276

The most correct way to do this would be to use the urllib.request.urlopen function to return a file-like object that represents an HTTP response and copy it to a real file using shutil.copyfileobj.

---

Related Topics