Using PHP & Curl to Login to My Websites Form

Using PHP & Curl to login to my websites form

You should send via POST all data that the orignal form is sending. So you are missing autologin=1&userlogin=Login in your $postdata.

$postdata = "username=$username&userpass=$password&autologin=1&userlogin=Login";

how to login this web site by php curl?

Try this snippet I wrote, replace username and password with your corresponding credentials, and replace the URL with the page you want to grab data from (a page that requires a login).

Check the post data that is sent in browser inspector. Replace $postinfo with the correct post data to ensure you are sending the right data. "email" and "Password" as the $postdata might not work for the site you are trying to login into. If you have trouble, post the inspector's data that is reported in the network tab and I'll try and help.

    $username = "email@example.com";
    $password = "MyPassword123";

    $dir = $_SERVER['DOCUMENT_ROOT']."/ctemp";

    $path = tempnam(sys_get_temp_dir(), 'MyTempCookie');

    $url = "https://www.example.com/dashboard/";
    $postinfo = "email=".$username."&Password=".$password;

    $cookie_file_path = $path."/cookie.txt";

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_HEADER, false);
    curl_setopt($ch, CURLOPT_NOBODY, false);
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);

    curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
    //for certain features, set the cookie the site has - this is optional
    curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
    curl_setopt($ch, CURLOPT_USERAGENT,
        "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);

    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
    curl_exec($ch);

    curl_setopt($ch, CURLOPT_URL, $url);
    $html_data = curl_exec($ch);
    curl_close($ch);
    echo $html_data;

Here at the end I'm simply just echoing the return html data. But you can do whatever you want with it and search for the data inside the response that you need.

Edit:
Just saw that the site has a recaptcha for their login process. Logging in via cURL is going to be far more complex now since they present the recaptcha code to you in an image. Might want to look into tutorials on spoofing recaptcha data. Good luck.

login into webpage with php with cURL

The book you're using is old, and Packt Publishing have changed their website. It now includes a CSRF token, and without passing this you will never be able to log in.

I've developed a working solution. It uses pQuery for parsing the HTML. You can install this using Composer, or download the package and include it into your application. If you do this, remove the require __DIR__ . '/vendor/autoload.php'; and replace with the location to the pquery package on your system.

To test via the command line simply run: php packt_example.php.

You will also notice that many headers are not even required, such as the useragent. I have left these out.

<?php

require __DIR__ . '/vendor/autoload.php';

$email = 'myemail@gmail.com';
$password = 'mypassword';

# Initialize a cURL session.
$ch = curl_init('https://www.packtpub.com/register');

# Set the cURL options.
$options = [
    CURLOPT_COOKIEFILE      => 'cookies.txt',
    CURLOPT_COOKIEJAR       => 'cookies.txt',
    CURLOPT_RETURNTRANSFER  => 1
];

# Set the options
curl_setopt_array($ch, $options);

# Execute
$html = curl_exec($ch);

# Grab the CSRF token from the HTML source
$dom = pQuery::parseStr($html);
$csrfToken = $dom->query('[name="form_build_id"]')->val();

# Now we have the form_build_id (aka the CSRF token) we can
# proceed with making the POST request to login. First,
# lets create an array of post data to send with the POST
# request.
$postData = [
    'email'         => $email,
    'password'      => $password,
    'op'            => 'Login',
    'form_build_id' => $csrfToken,
    'form_id'       => 'packt_user_login_form'
];

# Convert the post data array to URL encoded string
$postDataStr = http_build_query($postData);

# Append some fields to the CURL options array to make a POST request.
$options[CURLOPT_POST] = 1;
$options[CURLOPT_POSTFIELDS] = $postDataStr;
$options[CURLOPT_HEADER] = 1;

curl_setopt_array($ch, $options);

# Execute
$response = curl_exec($ch);

# Extract the headers from the response
$headerSize = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
$headers = substr($response, 0, $headerSize);

# Close cURL handle
curl_close($ch);

# If login is successful, the headers will contain a location header
# to the url http://www.packtpub.com/index
if(!strpos($headers, 'packtpub.com/index'))
{
    print 'Login Failed';
    exit;
}

print 'Logged In';

PHP cURL Log into website and perform action on different page

So, a couple of things jump out at me:

  • You're assuming that the magic values for __VIEWSTATE and __EVENTVALIDATION won't change. This is unlikely to be the case. You should pull those values again after fetching the data page.

  • You're passing the $viewstate variable as the value for __VIEWSTATE on the initial login, but you leave that blank on the subsequent post and instead pass $viewstate as __VIEWSTATE_GUID. Not sure if this is intentional or not.

  • You're using an array for CURL_POSTFIELDS which may cause problems. The documentation says:

    Passing an array to CURLOPT_POSTFIELDS will encode the data as multipart/form-data, while passing a URL-encoded string will encode the data as application/x-www-form-urlencoded.

  • And, very important, do NOT disable certificate validation, fix your server setup instead.

A few other suggestions, perhaps more a matter of style than substance though.

  • Passing an empty string as CURLOPT_COOKIEFILE will enable session handling without the need to save to a file.

  • You don't need to do curl_close() and curl_init() multiple times in a script; just reuse the existing handle. This saves having to redefine the options and reuse the session cookies.

  • Use curl_setopt_array() for cleaner code.

  • curl_exec() returns false on error, you should check for it explicitly.

Here's how I'd clean up the code:

<?php

$url = "https://someurl/login.aspx";
$ckfile = tempnam("/tmp", "CURLCOOKIE");
$useragent = $_SERVER['HTTP_USER_AGENT'];

$username = "someuser@hotmail.co.uk";
$password = "somepassword";

$viewstate_pattern = '~<input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="(.*?)" />~';
$eventval_pattern = '~<input type="hidden" name="__EVENTVALIDATION" id="__EVENTVALIDATION" value="(.*?)" />~';

$ch = curl_init();
curl_setopt_array($ch, [
    CURLOPT_URL            => $url,
    CURLOPT_COOKIEFILE     => "",
    CURLOPT_FOLLOWLOCATION => true,
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_USERAGENT      => $useragent,
]);

// Getting the login form
$html = curl_exec($ch);

if ($html !== false) {
    preg_match($viewstate_pattern, $html, $viewstate);
    preg_match($evenval_pattern, $html, $eventValidation);
    $viewstate = $viewstate[1];
    $eventValidation = $eventValidation[1];

    $postfields = http_build_query([
        "__EVENTTARGET"=>"",
        "__EVENTARGUMENT"=>"",
        "__VIEWSTATE"=>$viewstate,
        "__EVENTVALIDATION"=>$eventValidation,
        "btnLogin"=>"Login",
        "txtPassword"=>$password,
        "txtUserName"=>$username,
    ]);

    curl_setopt_array($ch, [
        CURLOPT_REFERER=>$url,
        CURLOPT_POST=>true,
        CURLOPT_POSTFIELDS=>$postfields,
    ]);
    // Submitting the login form
    $html = curl_exec($ch);

    if ($html !== false) {
        curl_setopt_array($ch, [
            CURLOPT_URL=>'https://someurl.com/financial/quote.aspx?id=12345',
            CURLOPT_POST=>false,
        ]);

        // Getting the data page
        $html = curl_exec($ch);
        if ($html !== false) {
            preg_match($viewstate_pattern, $html, $viewstate);
            preg_match($evenval_pattern, $html, $eventValidation);
            $viewstate = $viewstate[1];
            $eventValidation = $eventValidation[1];

            $postfieldsInner = http_build_query([
                "__EVENTTARGET"=>"",
                "__EVENTARGUMENT"=>"",
        // Should this be empty?
                "__VIEWSTATE"=>"",
                "__VIEWSTATE_GUID"=>$viewstate,
                "__EVENTVALIDATION"=>$eventValidation,
                'ctl00$PageContent$btn2'=>"Accepted",
            ]);

            curl_setopt_array($ch, [
                CURLOPT_POST=>true,
                CURLOPT_POSTFIELDS=>$postfieldsInner,
            ]);
            // Posting the data page
            $html = curl_exec($ch);

            if ($html === false) {
                echo 'An error has occurred: ' . curl_error($ch);
            } else {
                var_dump($html);
            }
        } else {
            // Error getting the data page
        }
    } else {
        // Error submitting the login page
    }
} else {
    // Error getting the login page
}
curl_close();

Using PHP CURL to login to a remote web site

EDIT: The URL you specified is wrong, it should be:

https://vp1-voiceportal.megapath.com/servlet/com.broadsoft.clients.oam.servlets.Login

And not:

https://vp1-voiceportal.megapath.com/Login/servlet/com.broadsoft.clients.oam.servlets.Login

It looks like you need to follow redirects and specify the cookie file (for reading), try:

$ch = curl_init(); 
curl_setopt ($ch, CURLOPT_URL, $url); 
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE); 
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"); 
curl_setopt ($ch, CURLOPT_TIMEOUT, 60); 
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1); 
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt ($ch, CURLOPT_COOKIEJAR, $cookie); 
curl_setopt ($ch, CURLOPT_COOKIEFILE, $cookie); 
curl_setopt ($ch, CURLOPT_REFERER, $url); 

curl_setopt ($ch, CURLOPT_POSTFIELDS, $postdata); 
curl_setopt ($ch, CURLOPT_POST, 1); 
$result = curl_exec ($ch); 

echo $result;  

curl_close($ch);

It's also a good practice so specify an absolute (and writable) path to the cookie file.

how to login through php curl which is submited by javascript i.e no submit button in form

$EMAIL            = '';
$PASSWORD         = '!';

$cookie_file_path = "/tmp/cookies.txt";
$LOGINURL         = "/login.jsf"; 
$agent            = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1";

// begin script
$ch = curl_init(); 

// extra headers
$headers[] = "Accept: */*";
$headers[] = "Connection: Keep-Alive";

// basic curl options for all requests
curl_setopt($ch, CURLOPT_HTTPHEADER,  $headers);
curl_setopt($ch, CURLOPT_HEADER,  0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);         
curl_setopt($ch, CURLOPT_USERAGENT, $agent); 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); 
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path); 
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path); 

// set first URL
curl_setopt($ch, CURLOPT_URL, $LOGINURL);

// execute session to get cookies and required form inputs
$content = curl_exec($ch); 

 // grab the hidden inputs from the form required to login
$fields = getFormFields($content);
$fields['loginForm:userName'] = $EMAIL;
$fields['loginForm:password'] = $PASSWORD;

// get x value that is used in the login url
$x = '';
if (preg_match('/login\.jsf/i', $content, $match)) {
echo $x = $match[1];
}

//$LOGINURL   = "https://cart2.barnesandnoble.com/mobileacct/op.asp?stage=signIn";
   $LOGINURL   = "/pages/login.jsf";

// set postfields using what we extracted from the form
   $POSTFIELDS = http_build_query($fields); 

// change URL to login URL
curl_setopt($ch, CURLOPT_URL, $LOGINURL); 

// set post options
curl_setopt($ch, CURLOPT_POST, 1); 
curl_setopt($ch, CURLOPT_POSTFIELDS, $POSTFIELDS); 

// perform login
 $result = curl_exec($ch);  

 //print $result; 
$url2='';
curl_setopt($ch, CURLOPT_URL, $url2); 

$result2 = curl_exec($ch);  

function getFormFields($data)
{
if (preg_match('/(<form id="loginForm.*?<\/form>)/is', $data, $matches)) {
    $inputs = getInputs($matches[1]);

    return $inputs;
} else {
    die('didnt find login form');
}
}

Using Curl to login to a website, go to a link, again submit a form and get the output

You need to attach the cookie(s) to CURL. There are two options:

  1. You can do this all manually by reading the cookie, and attaching manually using

    $ch = curl_init();
    $sCookie = 'JSESSIONID=' . $sessionIDSavedEarlier . '; path=/'; 
    curl_setopt ( $ch , CURLOPT_COOKIE, sCookie );
    ... rest of POST

  2. You can use a "cookie jar" (probably easiest)

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
    curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
    ... rest of POST

(Making sure you can read/write to a file called "cookie.txt" - if you have multiple users using hte script, then make that file unique for each user through a PHP session!)

Use Browser cookies in PHP Curl to log-in to a website

To verify the format of a cookie file: You can set 

curl_setopt($ch, CURLOPT_COOKIEJAR, "path_to_the_cookie_file")

next run a CURL request to any website that sets cookies, then check the format of the set cookie file.

To send your cookies: Then build your cookie file with the desired cookies and use the setting

curl_setopt($ch, CURLOPT_COOKIEFILE, "path_to_the_cookie_file")

to make your CURL request send the cookies.

Edit #1: From the official PHP documentation (http://php.net/manual/en/function.curl-setopt.php):

The cookie file can be in Netscape format, or just plain HTTP-style
headers dumped into a file.

Edit #2: this is an example of a cookie file generated by CURL (source):

# Netscape HTTP Cookie File
# This file was generated by libcurl! Edit at your own risk.
.auto.com   TRUE    /   FALSE   1452087781  ___suid 2ecfe4287cbeacd8399eaf98bec9ce0b.59089b9d033bc7c6dce8ea2fca139920
.auto.com   TRUE    /   FALSE   1452865380  all7_user_region_confirmed  1
.auto.com   TRUE    /   FALSE   1452865380  geo_location    a%3A3%3A%7Bs%3A7%3A%22city_id%22%3Ba%3A0%3A%7B%7Ds%3A9%3A%22region_id%22%3Ba%3A1%3A%7Bi%3A0%3Bi%3A89%3B%7Ds%3A10%3A%22country_id%22%3Ba%3A0%3A%7B%7D%7D
.auto.com   TRUE    /   FALSE   1423921380  autoru_sid  ee094d60fa32eada_daf2da69dc79a59b7c8702a29554abbc
.auth.auto.com  TRUE    /   FALSE   1421329026  autoru_sid  
.auth.auto.com  TRUE    /   FALSE   1421329026  autoru_sid_key  
.auto.com   TRUE    /   FALSE   1421329026  cc6882cb6b6f0c912cf9589734fcc1e6    
.auto.com   TRUE    /   FALSE   1452865027  user_name   igor.savinkin5%40gmail.com
.auto.com   TRUE    /   FALSE   1452865027  username   igor.savinkin5%40gmail.com

Edit #3: What are all those entries in my cookies.txt file? (http://www.cookiecentral.com/faq/#3.5)

From left-to-right, here is what each field represents:

domain - The domain that created AND that can read the variable.

flag - A TRUE/FALSE value indicating if all machines within a given
domain can access the variable. This value is set automatically by the
browser, depending on the value you set for domain.

path - The path within the domain that the variable is valid for.

secure - A TRUE/FALSE value indicating if a secure connection with the
domain is needed to access the variable.

expiration - The UNIX time that the variable will expire on. UNIX time
is defined as the number of seconds since Jan 1, 1970 00:00:00 GMT.

name - The name of the variable.

value - The value of the variable.


Related Topics

How to Run PHP File Using Cron Jobs

MySQL and Query to Satisfy on Same Column

Download the Contents of a Url in PHP Even If It Returns a 404

Reverse Letters in Each Word of a String Without Using Native Splitting or Reversing Functions

When Should I Close a Database Connection in PHP

Post Ajax Data to PHP and Return Data

How to Make a Chat Room Script with PHP

How to Add Item to the JSON File Formatted Array

PHP Regex for Human Names

How to Force Ssl in Codeigniter

Get First Element in PHP Stdobject

Converting Nsarray -> JSON -> Nsdata -> PHP Server ->JSON Representation

How to Display HTML to the Browser Incrementally Over a Long Period of Time

Png Transparency Resize with Simpleimage.PHP Class

Getting Hour and Minute in PHP

Parsing Xml Data Using PHP to Put into MySQL Database

Composer:How to Add a Dependency Without Network Connection

When Do/Should I Use _Construct(), _Get(), _Set(), and _Call() in PHP


Leave a reply



Submit