How do I go about reading and locating where I need to input login information in jsoup in order to access the web on a VPN? I'm interested in both an explanation of the steps/topics involved as well as the programming methodology with java (basically how to code this in java using jsoup). Note: with all the redirects, I have a hard time understanding what is happening and how/when/where to code in a jsoup-login.
Here's my workflow so far:
I have a target page, like the one below
[debug] status code 302 : https://centrale.landingnetwork.com/gp/stores/www.landingnetwork.com/gp/home/
When I navigate to this URL in java/jsoup, I get all kinds of redirects. Here's the trail of my redirects: (in order of which comes next)
[debug] status code 302 : https://centrale.landingnetwork.com/gp/stores/www.landingnetwork.com/gp/signin/gi-signin.html/123-1234567-1234567?ie=UTF8&landat=%2Fgp%2Fstores%2Fwww.landingnetwork.com%2Fgp%2Fhome%2F123-1234567-1234567&ort=1122334455.98765&rrt=1112223334.12121
[debug] status code 200 : https://wa.secureallnetwork.com/login?clienteId=Centrale-prod-wa&nonce=867:5309:867:5309:867:5309:867:5309:867:53099&redirect_uri=https%3A%2F%2Fcentrale.landingnetwork.com%3A443%2Fgp%2Fstores%2Fwww.landingnetwork.com%2Fgp%2Fsignin%2Fgi-landat.html%2F123-1234567-1234567%3Flandat%3D%2Fgp%2Fstores%2Fwww.landingnetwork.com%2Fgp%2Fhome%2F123-1234567-1234567&ort=1122334455.98765&rrt=1112223334.12121
[debug] status code 200 : https://wa.secureallnetwork.com/login?sif_profile=gi_profile_1&clienteId=Centrale-prod-wa&nonce=867:5309:867:5309:867:5309:867:5309:867:53099&redirect_uri=https://centrale.landingnetwork.com:443/gp/stores/www.landingnetwork.com/gp/signin/gi-landat.html/123-1234567-1234567?landat=/gp/stores/www.landingnetwork.com/gp/home/123-1234567-1234567
Now, I don't have a super big background in networking, but I can definitely follow along if explained well/thoroughly.
My issue: when I go through redirects, I don't know why my form posting code of my username / password doesn't work.
Here's my code so far, (two classes);
import java.io.IOException;
import java.net.SocketException;
import java.util.HashMap;
import org.jsoup.Connection;
import org.jsoup.Connection.Response;
import org.jsoup.Jsoup;
import org.jsoup.UncheckedIOException;
import org.jsoup.nodes.Document;
public class App {
public static final String USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0";
public static final String LOGIN_FORM_URL = "https://centrale.landingnetwork.com/gp/stores/www.landingnetwork.com/gp/home/";
public static final String USERNAME = "myusername";
public static final String PASSWORD = "mupassword";
public static void main(String[] args) throws Exception {
WebCrawler wc = new WebCrawler();
// # Go to login page and grab cookies sent by server
Connection.Response loginForm = wc.crawl(LOGIN_FORM_URL);
// this is the document containing response html
Document loginDoc = loginForm.parse();
// save the cookies to be passed on to next request
HashMap<String, String> cookies = new HashMap<>(loginForm.cookies());
// # Prepare login credentials
String authToken = loginDoc.select("form").attr("class", "a-spacing-micro").first().attr("action");
HashMap<String, String> formData = new HashMap<>();
formData.put("usernameInputField", USERNAME);
formData.put("passwordInputField", PASSWORD);
Connection.Response homePage = wc.crawl("https://wa.secureallnetwork.com" + authToken, cookies, formData, Connection.Method.POST, true);
}
}
import java.io.IOException;
import java.util.HashMap;
import org.jsoup.Connection;
import org.jsoup.Jsoup;
import org.jsoup.Connection.Response;
public class WebCrawler {
public static final String USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0";
public Connection.Response crawl(String URL) throws IOException {
Response response = Jsoup.connect(URL).userAgent(USER_AGENT).followRedirects(false).execute();
if (response.hasHeader("location")) {
String redirectUrl = response.header("location");
return crawl(redirectUrl);
} else {
return response;
}
}
public Connection.Response crawl(String URL, HashMap<String, String> cooks, HashMap<String, String> dat, Connection.Method m, boolean follow) throws IOException {
Response response = Jsoup.connect(URL).userAgent(USER_AGENT).cookies(cooks).data(dat).followRedirects(follow).method(m).execute();
if (response.hasHeader("location")) {
String redirectUrl = response.header("location");
return crawl(redirectUrl);
} else {
return response;
}
}
}
When I print out the headers, they seem relatively straight forward, the only things that might stand out to me are, 'X-REQUEST-ID1'
/'X-REQUEST-ID2'
headers, set cookie session id, and a location. But I'm sure that's not where I'm having my main troubles at - I think it's more where I attempt to interact with the data over multiple web pages with jsoup.
To reiterate my question: How can I login pragmatically with java/jsoup to my website? A thorough explanation with details/examples/final code would be a glorious lesson if somebody would be willing to take the time!
Cheers
Aucun commentaire:
Enregistrer un commentaire