Problems Submitting a Login Form With Jsoup

Problems submitting a login form with Jsoup

Besides the username, password and the cookies, the site requeires two additional values for the login - VIEWSTATE and EVENTVALIDATION.

You can get them from the response of the first Get request, like this -

Document doc = loginForm.parse();
Element e = doc.select("input[id=__VIEWSTATE]").first();
String viewState = e.attr("value");
e = doc.select("input[id=__EVENTVALIDATION]").first();
String eventValidation = e.attr("value");

And add it after the password (the order doesn't really matter) -

org.jsoup.nodes.Document document = (org.jsoup.nodes.Document) Jsoup.connect("https://www.capitaliq.com/CIQDotNet/Login.aspx/authentication.php").userAgent("Mozilla/5.0")               
            .data("myLogin$myUsername", "MyUsername")
            .data("myLogin$myPassword, "MyPassword")
            .data("myLogin$myLoginButton.x", "22")                   
            .data("myLogin$myLoginButton.y", "8")
            .data("__VIEWSTATE", viewState)
            .data("__EVENTVALIDATION", eventValidation)
            .cookies(loginForm.cookies())
            .post();

I would also add the userAgent field to both requests - some sites test it and send different pages to different clients, so if you would like to get the same response as you get with your browser, add to the requests .userAgent("Mozilla/5.0") (or whatever browser you're using).

Edit

The userName's field name is myLogin$myUsername, the password is myLogin$myPassword and the Post request also contains data about the login button. Ican't test it, because I don't have user at that site, but I believe it will work. Hope this solves your problem.

EDIT 2

To enable the remember me field during login, add this line to the post request:

.data("myLogin$myEnableAutoLogin", "on")

Login using Jsoup

Few remarks:

• You're getting HTTP status 302. It may be a good idea to use Jsoup.connect(...).followRedirects(true)
• Some servers check the site you're coming from so it's recommened to set referrer header: Jsoup.connect(...).referrer(URL + "/login")
• Are you sure the hidden value is always 69? Maybe it's different for each request. You can get it dynamically like this: formData.put("hidden",html.select("co_js").first().attr("value"));

• I don't like your way of getting the token. Let's use Jsoup to extract it:
  String authToken = html.select("input#token").first().attr("value");

  Edit:

• I tried to do this from scratch and submitted login form was missing Content-Type definition. Try using:
  Jsoup.connect(...).header("Content-Type","application/x-www-form-urlencoded")
  I got the idea by analyzing headers in Chrome's dev tools. Now I can successfully login.
• You can also check if you're logged in using this code:
  Document doc = homePage.parse();
System.out.println("Logged in as: " + doc.select(".dropdown-toggle").text());

• By letting server know you can handle compressed pages you can decrease downloaded page size. For every request use: Jsoup.connect(...).header("accept-encoding", "gzip, deflate") It's transparent and you don't have to do anything special to handle it, but it works internally.

  Edit 2:

Providing final solution based on previous advices:

    import java.util.HashMap;
    import java.util.Map;

    import org.jsoup.Connection;
    import org.jsoup.Connection.Response;
    import org.jsoup.Jsoup;
    import org.jsoup.nodes.Document;

    public class Stackoverflow51734840 {

        private static final String URL = "https://time2watch.in";
        private static final String URL_LOGIN = URL + "/login/";
        static String userAgent = "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36";

        public static void main(final String[] args) throws Exception {

            Map<String, String> headers = new HashMap<String, String>();
            headers.put("accept-encoding", "gzip, deflate");

            Connection.Response loginForm = Jsoup.connect(URL + "/login").userAgent(userAgent).headers(headers).execute();
            Map<String, String> cookies = loginForm.cookies();
            Document html = loginForm.parse();

            String authToken = html.select("input#token").first().attr("value");
            System.out.println("Found authToken:" + authToken);

            Map<String, String> formData = new HashMap<String, String>();
            formData.put("username", "!!!!!!!!!!!!!!!!!!");
            formData.put("pwd", "!!!!!!!!!!!!!!!!!!");
            formData.put("hidden", "69");
            formData.put("token", authToken);
            headers.put("Content-Type", "application/x-www-form-urlencoded");

            System.out.println("cookies before login:");
            System.out.println(cookies);
            System.out.println(" Logged in cookie present? " + cookies.containsKey("s4icookuser"));

            Connection.Response afterLoginPage = Jsoup.connect(URL_LOGIN).cookies(cookies).headers(headers)
                    .userAgent(userAgent).data(formData).method(Connection.Method.POST).referrer(URL_LOGIN).execute();
            // update cookies
            cookies = afterLoginPage.cookies();

            System.out.println("cookies after login:");
            System.out.println(cookies);
            System.out.println(" Logged in cookie present? " + cookies.containsKey("s4icookuser"));

            Response homePage = Jsoup.connect(URL).cookies(cookies).method(Connection.Method.GET).userAgent(userAgent)
                    .referrer(URL_LOGIN).followRedirects(true).referrer(URL_LOGIN).headers(headers).execute();

            Document doc = homePage.parse();
            System.out.println("Error? " + doc.text().contains("Erreur"));
            System.out.println("OK? " + !doc.text().contains("Se connecter"));
            System.out.println("Logged in as: " + doc.select(".dropdown-toggle").text());
        }

    }

How to post form login using jsoup?

The URL that are you using in order to do the POST request is wrong, simply because when you have to do a specific request to a form you should use the web page that is present in the form tag, in this case "authentication.php".

So the code will be:

    package jsouptest;

    import org.jsoup.Connection;
    import org.jsoup.Jsoup;
    import org.jsoup.nodes.Document;

    public class JsouptTest {
        public static void main(String[] args) throws Exception {

           Connection.Response loginForm = Jsoup.connect("https://www.desco.org.bd/ebill/login.php")
            .method(Connection.Method.GET)
            .execute();

           Document document = Jsoup.connect("https://www.desco.org.bd/ebill/authentication.php")
            .data("cookieexists", "false")
            .data("username", "32007702")
            .data("login", "Login")
            .cookies(loginForm.cookies())
            .post();
           System.out.println(document);

       }

    }

This one correctly retrieves the web page that you want.

---

Related Topics