How to Programmatically Log in to a Website to Screenscape

How to programmatically log in to a website to screenscape?

You'd make the request as though you'd just filled out the form. Assuming it's POST for example, you make a POST request with the correct data. Now if you can't login directly to the same page you want to scrape, you will have to track whatever cookies are set after your login request, and include them in your scraping request to allow you to stay logged in.

It might look like:

HttpWebRequest http = WebRequest.Create(url) as HttpWebRequest;
http.KeepAlive = true;
http.Method = "POST";
http.ContentType = "application/x-www-form-urlencoded";
string postData="FormNameForUserId=" + strUserId + "&FormNameForPassword=" + strPassword;
byte[] dataBytes = UTF8Encoding.UTF8.GetBytes(postData);
http.ContentLength = dataBytes.Length;
using (Stream postStream = http.GetRequestStream())
{
    postStream.Write(dataBytes, 0, dataBytes.Length);
}
HttpWebResponse httpResponse = http.GetResponse() as HttpWebResponse;
// Probably want to inspect the http.Headers here first
http = WebRequest.Create(url2) as HttpWebRequest;
http.CookieContainer = new CookieContainer();
http.CookieContainer.Add(httpResponse.Cookies);
HttpWebResponse httpResponse2 = http.GetResponse() as HttpWebResponse;

Maybe.

Auto login to a website programmably?

Well, it really depends on what authentication scheme the sites use. It is possible to pass a Credentials object with an HttpWebRequest which can be used to authenticate against sites that use basic authentication, windows authentication and similar. But I can't think of a reliable way that would work for any and all sites.

Is it possible to login from a winforms application to a website which uses spring security?

Authentication can be setup in many ways with Spring Security. For example:

<http pattern="/webservice/**" create-session="stateless" realm="TestRealm">
  <http-basic/>
  <intercept-url pattern="/**" access="ROLE_USER"/>
</http>

Then it is as simple as this to authenticate from C#:

var request = (HttpWebRequest)WebRequest.Create(url);
request.Credentials = new NetworkCredential(username, pwd);

Login to website from app

When I tried getting an XML file from a website which required credentials I used the following:

 public static void GetFileWithCredentials(string userName, string password, string url)
{

    using (WebClient wc = new WebClient())
    {
        wc.Credentials = new NetworkCredential(userName, password);
        string xml = wc.DownloadString(url);

        XmlDocument tournamentsXML = new XmlDocument();
        tournamentsXML.LoadXml(xml);
    }

}

Hopefully it is of some use for you. If it's not would it be possible to post an example of what the data you are trying to get looks like? I would've commented first but my reputation doesn't seem to allow me!

How to WebView.postUrl() in WebBrowser

In Windows Forms' WebBrowser you post data by calling the form's InvokeMember("submit"):

If you know the ID of the form you would like to submit:

HtmlElement form = webBrowser1.Document.GetElementById("FormID");
if (form != null)
    form.InvokeMember("submit");

---

Related Topics