Php: Get HTML Source Code with Curl

php: Get html source code with cURL

Try the following:

$ch = curl_init("http://www.example-webpage.com/file.html");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
$content = curl_exec($ch);
curl_close($ch);

I would only recommend this for small files. Big files are read as a whole and are likely to produce a memory error.

---

EDIT: after some discussion in the comments we found out that the problem was that the server couldn't resolve the host name and the page was in addition a HTTPS resource so here comes your temporary solution (until your server admin fixes the name resolving).

what i did is just pinging graph.facebook.com to see the IP address, replace the host name with the IP address and instead specify the header manually. This however renders the SSL certificate invalid so we have to suppress peer verification.

//$url = "https://graph.facebook.com/19165649929?fields=name";
$url = "https://66.220.146.224/19165649929?fields=name";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Host: graph.facebook.com'));
$output = curl_exec($ch);
curl_close($ch); 

Keep in mind that the IP address might change and this is an error source. you should also do some error handling using curl_error();.

can't get html source code with cUrl or file_get_contents

Just add User-Agent header:

<?php

$curl = curl_init();

curl_setopt_array($curl, array(
  CURLOPT_URL => "http://picbear.com/tag/ok",
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_ENCODING => "",
  CURLOPT_MAXREDIRS => 10,
  CURLOPT_TIMEOUT => 30,
  CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
  CURLOPT_CUSTOMREQUEST => "GET",
  CURLOPT_HTTPHEADER => array(
    "cache-control: no-cache",
    "user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" // Here we add the header
  ),

));

$response = curl_exec($curl);
$err = curl_error($curl);

curl_close($curl);

if ($err) {
  echo "cURL Error #:" . $err;
} else {
  echo $response;
}

Get HTML Code of HTML File using PHP CURL

Please check this

$ch= curl_init();
curl_setopt ($ch1, CURLOPT_URL, 'http://192.168.1.1/userRpm/SystemStatisticRpm.htm?Num_per_page=100' );
curl_setopt($ch1, CURLOPT_HEADER, 0);
curl_setopt($ch1,CURLOPT_VERBOSE,1);
curl_setopt($ch1, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0)');
curl_setopt ($ch1, CURLOPT_REFERER,'http://www.google.com');  //just a fake referer
curl_setopt($ch1, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch1,CURLOPT_POST,0);
curl_setopt($ch1, CURLOPT_FOLLOWLOCATION, 20);

$htmlContent= curl_exec($ch1);

How do I get the HTML code of a web page in PHP?

If your PHP server allows url fopen wrappers then the simplest way is:

$html = file_get_contents('https://stackoverflow.com/questions/ask');

If you need more control then you should look at the cURL functions:

$c = curl_init('https://stackoverflow.com/questions/ask');
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
//curl_setopt(... other options you want...)

$html = curl_exec($c);

if (curl_error($c))
    die(curl_error($c));

// Get the status code
$status = curl_getinfo($c, CURLINFO_HTTP_CODE);

curl_close($c);

Curl, not getting page source code

I have modified your code..
Try this.

Remove

header('content-type:text/plain');

and replace your code with mine.

$ch = curl_init();
$url = "https://www.amazon.com/gp/css/shiptrack/view.html/ref=TE_SIMP_typ?ie=UTF8&addressID=olqnkqjstjp&latestArrivalDate=1483848000&orderID=103-0753412-5410653&shipmentDate=1483706902&orderingShipmentId=4160817392100&packageId=1";
curl_setopt($ch, CURLOPT_URL, $url); 
curl_setopt($ch, CURLOPT_HEADER, false);

curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 1);
$output = curl_exec($ch);
curl_close($ch);
echo $output;

EDIT

Since you want ctrl+u

Here is the PHP code to fetch the html source code of any website specified. fopen function is used to open the website URL. stream_get_contents is used to read the opened URL. The fetched code is displayed inside a textarea.

$domain = 'https://www.amazon.com/gp/css/shiptrack/view.html/ref=TE_SIMP_typ?ie=UTF8&addressID=olqnkqjstjp&latestArrivalDate=1483848000&orderID=103-0753412-5410653&shipmentDate=1483706902&orderingShipmentId=4160817392100&packageId=1';
$handle = fopen($domain, 'r');
$content = stream_get_contents($handle);
/**
// alternative to stream_get_contents
$contents = '';
while (!feof($handle)) {
    $content .= fread($handle, 8192);
}
**/ 
fclose($handle);
/** 
 * print html content on textarea
 */
echo "<textarea rows='20' cols='80'>$content</textarea>";

output will be like this:

OR

Also if you want to manipulate the retrieved page somehow, you might
want to try some php DOM parser. I find PHP Simple HTML DOM
Parser very easy to use.

php using curl to get the html and save it on a txt

You need to use:

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1 );

Otherwise the result is just printed out.

How to get HTML source of any website with PHP?

That is because facebook runs on HTTPS (SSL Protocol) . Add this to your existing cURL parameters

curl_setopt($curl_handle, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl_handle, CURLOPT_SSL_VERIFYHOST, false);

Get Html with CURL

$ch = curl_init("http://www.example.com/");
$fp = fopen("example_homepage.txt", "w");

curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HEADER, 0);

curl_exec($ch);
if(curl_error($ch)) {
    fwrite($fp, curl_error($ch));
}
curl_close($ch);
fclose($fp);

source

---

Related Topics