Authenticated HTTP proxy with Java
(EDIT: As pointed out by the OP, the using a java.net.Authenticator is required too. I'm updating my answer accordingly for the sake of correctness.)
(EDIT#2: As pointed out in another answer, in JDK 8 it's required to remove basic auth scheme from jdk.http.auth.tunneling.disabledSchemes property)
For authentication, use java.net.Authenticator to set proxy's configuration and set the system properties http.proxyUser and http.proxyPassword.
final String authUser = "user";
final String authPassword = "password";
Authenticator.setDefault(
new Authenticator() {
@Override
public PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(authUser, authPassword.toCharArray());
}
}
);
System.setProperty("http.proxyUser", authUser);
System.setProperty("http.proxyPassword", authPassword);
System.setProperty("jdk.http.auth.tunneling.disabledSchemes", "");
HTTP Basic Authentication in Java with Proxy
Turns out I needed to add a "Proxy-Authorization" header as well.
HttpPost httpPost = new HttpPost("http://host:port/test/login");
String encoding = Base64Encoder.encode ("your_user:your_password");
httpPost.setHeader("Authorization", "Basic " + encoding);
String proxyEncoding = Base64Encoder.encode ("proxy_user:proxy_password");
httpPost.setHeader("Proxy-Authorization", "Basic " + proxyEncoding);
System.out.println("executing request " + httpPost.getRequestLine());
HttpResponse response = httpClient.execute(httpPost);
What Java properties to pass to a Java app to authenticate with a http proxy
Finally I figured out by trial and error. Passing java.net.useSystemProxies=true along with https.proxyPort, https.proxyHost resolved this.
Basically the java vm command line got
-Djava.net.useSystemProxies=true -Dhttps.proxyPort=80 -Dhttps.proxyHost=proxyserver.mycompany.com
I didn't have to pass https.proxyUser, https.proxyPassword. I believe proxy authentication used the same credentials as my login NTLM credentials.
Basic proxy authentication for HTTPS URLs returns HTTP/1.0 407 Proxy Authentication Required
You can extend ProxiedHttpsConnection and handle all the low level related stuff by yourself.
The following steps need to be done to make a connection over a HTTP proxy to a https website:
Note: the communication with the proxy and http server should be in ASCII7.
- Send
CONNECT stackoverflow.com:443 HTTP/1.0\r\nto the proxy - Send your authentication:
Proxy-Authorization: Basic c2F5WW91SGF2ZVNlZW5UaGlzSW5UaGVDb21tZW50cw==\r\n. - End the first request:
\r\n - Read the response from the proxy until you see the combination "\r\n\r\n".
- Parse the first line of the response you got from the proxy and check if it starts with
HTTP/1.0 200. - Start a SSL session in place over the existing connection.
- Send the start of a http request:
GET /questions/3304006/persistent-httpurlconnection-in-java HTTP/1.0\r\n - Set the proper Host header:
Host: stackoverflow.com\r\n - End the request to the http server:
\r\n - Read till
\r\nand parse first line as status message - Read till end of stream for request body
When we want to implement the HttpUrlConnection class, there are a few things we also need to consider:
- At the time the class is constructed, the class should store data for future connections, but NOT make it directly
- Any methods can be called in any order
- The closure of the
OutputStreammeans the data transfer is done, not that the connection must finish - Every api uses the methods in a different order
- HTTP headers are case insensitive, java maps are case sensitive.
Quickly said, there are just many pitfalls
In the class I designed, it uses boolean flags to remember if the connect method and the afterPostClosure methods are called, it also has support if getInputStream() is called before the OutputStream is closed.
This class also uses as little wrapping as possible over the streams returned by the socket, to prevent being really complex.
public class ProxiedHttpsConnection extends HttpURLConnection {
private final String proxyHost;
private final int proxyPort;
private static final byte[] NEWLINE = "\r\n".getBytes();//should be "ASCII7"
private Socket socket;
private final Map<String, List<String>> headers = new TreeMap<>(String.CASE_INSENSITIVE_ORDER);
private final Map<String, List<String>> sendheaders = new TreeMap<>(String.CASE_INSENSITIVE_ORDER);
private final Map<String, List<String>> proxyheaders = new TreeMap<>(String.CASE_INSENSITIVE_ORDER);
private final Map<String, List<String>> proxyreturnheaders = new TreeMap<>(String.CASE_INSENSITIVE_ORDER);
private int statusCode;
private String statusLine;
private boolean isDoneWriting;
public ProxiedHttpsConnection(URL url,
String proxyHost, int proxyPort, String username, String password)
throws IOException {
super(url);
socket = new Socket();
this.proxyHost = proxyHost;
this.proxyPort = proxyPort;
String encoded = Base64.encode((username + ":" + password).getBytes())
.replace("\r\n", "");
proxyheaders.put("Proxy-Authorization", new ArrayList<>(Arrays.asList("Basic " + encoded)));
}
@Override
public OutputStream getOutputStream() throws IOException {
connect();
afterWrite();
return new FilterOutputStream(socket.getOutputStream()) {
@Override
public void write(byte[] b, int off, int len) throws IOException {
out.write(String.valueOf(len).getBytes());
out.write(NEWLINE);
out.write(b, off, len);
out.write(NEWLINE);
}
@Override
public void write(byte[] b) throws IOException {
out.write(String.valueOf(b.length).getBytes());
out.write(NEWLINE);
out.write(b);
out.write(NEWLINE);
}
@Override
public void write(int b) throws IOException {
out.write(String.valueOf(1).getBytes());
out.write(NEWLINE);
out.write(b);
out.write(NEWLINE);
}
@Override
public void close() throws IOException {
afterWrite();
}
};
}
private boolean afterwritten = false;
@Override
public InputStream getInputStream() throws IOException {
connect();
return socket.getInputStream();
}
@Override
public void setRequestMethod(String method) throws ProtocolException {
this.method = method;
}
@Override
public void setRequestProperty(String key, String value) {
sendheaders.put(key, new ArrayList<>(Arrays.asList(value)));
}
@Override
public void addRequestProperty(String key, String value) {
sendheaders.computeIfAbsent(key, l -> new ArrayList<>()).add(value);
}
@Override
public Map<String, List<String>> getHeaderFields() {
return headers;
}
@Override
public void connect() throws IOException {
if (connected) {
return;
}
connected = true;
socket.setSoTimeout(getReadTimeout());
socket.connect(new InetSocketAddress(proxyHost, proxyPort), getConnectTimeout());
StringBuilder msg = new StringBuilder();
msg.append("CONNECT ");
msg.append(url.getHost());
msg.append(':');
msg.append(url.getPort() == -1 ? 443 : url.getPort());
msg.append(" HTTP/1.0\r\n");
for (Map.Entry<String, List<String>> header : proxyheaders.entrySet()) {
for (String l : header.getValue()) {
msg.append(header.getKey()).append(": ").append(l);
msg.append("\r\n");
}
}
msg.append("Connection: close\r\n");
msg.append("\r\n");
byte[] bytes;
try {
bytes = msg.toString().getBytes("ASCII7");
} catch (UnsupportedEncodingException ignored) {
bytes = msg.toString().getBytes();
}
socket.getOutputStream().write(bytes);
socket.getOutputStream().flush();
byte reply[] = new byte[200];
byte header[] = new byte[200];
int replyLen = 0;
int headerLen = 0;
int newlinesSeen = 0;
boolean headerDone = false;
/* Done on first newline */
InputStream in = socket.getInputStream();
while (newlinesSeen < 2) {
int i = in.read();
if (i < 0) {
throw new IOException("Unexpected EOF from remote server");
}
if (i == '\n') {
if (newlinesSeen != 0) {
String h = new String(header, 0, headerLen);
String[] split = h.split(": ");
if (split.length != 1) {
proxyreturnheaders.computeIfAbsent(split[0], l -> new ArrayList<>()).add(split[1]);
}
}
headerDone = true;
++newlinesSeen;
headerLen = 0;
} else if (i != '\r') {
newlinesSeen = 0;
if (!headerDone && replyLen < reply.length) {
reply[replyLen++] = (byte) i;
} else if (headerLen < reply.length) {
header[headerLen++] = (byte) i;
}
}
}
String replyStr;
try {
replyStr = new String(reply, 0, replyLen, "ASCII7");
} catch (UnsupportedEncodingException ignored) {
replyStr = new String(reply, 0, replyLen);
}
// Some proxies return http/1.1, some http/1.0 even we asked for 1.0
if (!replyStr.startsWith("HTTP/1.0 200") && !replyStr.startsWith("HTTP/1.1 200")) {
throw new IOException("Unable to tunnel. Proxy returns \"" + replyStr + "\"");
}
SSLSocket s = (SSLSocket) ((SSLSocketFactory) SSLSocketFactory.getDefault())
.createSocket(socket, url.getHost(), url.getPort(), true);
s.startHandshake();
socket = s;
msg.setLength(0);
msg.append(method);
msg.append(" ");
msg.append(url.toExternalForm().split(String.valueOf(url.getPort()), -2)[1]);
msg.append(" HTTP/1.0\r\n");
for (Map.Entry<String, List<String>> h : sendheaders.entrySet()) {
for (String l : h.getValue()) {
msg.append(h.getKey()).append(": ").append(l);
msg.append("\r\n");
}
}
if (method.equals("POST") || method.equals("PUT")) {
msg.append("Transfer-Encoding: Chunked\r\n");
}
msg.append("Host: ").append(url.getHost()).append("\r\n");
msg.append("Connection: close\r\n");
msg.append("\r\n");
try {
bytes = msg.toString().getBytes("ASCII7");
} catch (UnsupportedEncodingException ignored) {
bytes = msg.toString().getBytes();
}
socket.getOutputStream().write(bytes);
socket.getOutputStream().flush();
}
private void afterWrite() throws IOException {
if (afterwritten) {
return;
}
afterwritten = true;
socket.getOutputStream().write(String.valueOf(0).getBytes());
socket.getOutputStream().write(NEWLINE);
socket.getOutputStream().write(NEWLINE);
byte reply[] = new byte[200];
byte header[] = new byte[200];
int replyLen = 0;
int headerLen = 0;
int newlinesSeen = 0;
boolean headerDone = false;
/* Done on first newline */
InputStream in = socket.getInputStream();
while (newlinesSeen < 2) {
int i = in.read();
if (i < 0) {
throw new IOException("Unexpected EOF from remote server");
}
if (i == '\n') {
if (headerDone) {
String h = new String(header, 0, headerLen);
String[] split = h.split(": ");
if (split.length != 1) {
headers.computeIfAbsent(split[0], l -> new ArrayList<>()).add(split[1]);
}
}
headerDone = true;
++newlinesSeen;
headerLen = 0;
} else if (i != '\r') {
newlinesSeen = 0;
if (!headerDone && replyLen < reply.length) {
reply[replyLen++] = (byte) i;
} else if (headerLen < header.length) {
header[headerLen++] = (byte) i;
}
}
}
String replyStr;
try {
replyStr = new String(reply, 0, replyLen, "ASCII7");
} catch (UnsupportedEncodingException ignored) {
replyStr = new String(reply, 0, replyLen);
}
/* We asked for HTTP/1.0, so we should get that back */
if ((!replyStr.startsWith("HTTP/1.0 200")) && !replyStr.startsWith("HTTP/1.1 200")) {
throw new IOException("Server returns \"" + replyStr + "\"");
}
}
@Override
public void disconnect() {
try {
socket.close();
} catch (IOException ex) {
Logger.getLogger(ProxiedHttpsConnection.class.getName()).log(Level.SEVERE, null, ex);
}
}
@Override
public boolean usingProxy() {
return true;
}
}
Current bugs with the above code:
- Streams are not closed on errors during post
- Streams are not closed during errors with the initial contact with the proxy
- It doesn't support http redirects
- It doesn't support the http 1.1 things like chunked and gzip encoding, but this is no problem as we announce ourselves as a http1.0 client.
The above code can be used like:
ProxiedHttpsConnection n = new ProxiedHttpsConnection(
new URL("https://stackoverflow.com:443/questions/3304006/persistent-httpurlconnection-in-java"),
"proxy.example.com", 8080, "root", "flg83yvem#");
n.setRequestMethod("GET");
n.addRequestProperty("User-Agent", "Java test https://stackoverflow.com/users/1542723/ferrybig");
//try (OutputStream out = n.getOutputStream()) {
// out.write("Hello?".getBytes());
//}
try (InputStream in = n.getInputStream()) {
byte[] buff = new byte[1024];
int length;
while ((length = in.read(buff)) >= 0) {
System.out.write(buff, 0, length);
}
}
If you are going to use this with a kind of proxy selector, you should check the protocol of the url to see if its http or https, if its http, don't use this class, and instead attach the header manually like:
httpURLConnection.setRequestProperty("Proxy-Authorization", "Basic " + encoded);
Why not using httpsUrlConnection.setSSLSocketFactory
While java has this method, attempts to use it will show you why it won't work, java just keeps calling the createSocket(Socket s, String host, int port, boolean autoClose) with an already open connection, making it impossible to do the proxy stuff manually.
Per-Proxy Authentication in Java
Was taken from JSocks project sources:
https://code.google.com/p/jsocks-mirror/source/browse/trunk/src/java/net/sourceforge/jsocks/socks/UserPasswordAuthentication.java
I think it's clean enough to understand full process:
/**
SOCKS5 User Password authentication scheme.
*/
public class UserPasswordAuthentication implements Authentication{
/**SOCKS ID for User/Password authentication method*/
public final static int METHOD_ID = 2;
String userName, password;
byte[] request;
/**
Create an instance of UserPasswordAuthentication.
@param userName User Name to send to SOCKS server.
@param password Password to send to SOCKS server.
*/
public UserPasswordAuthentication(String userName,String password){
this.userName = userName;
this.password = password;
formRequest();
}
/** Get the user name.
@return User name.
*/
public String getUser(){
return userName;
}
/** Get password
@return Password
*/
public String getPassword(){
return password;
}
/**
Does User/Password authentication as defined in rfc1929.
@return An array containnig in, out streams, or null if authentication
fails.
*/
public Object[] doSocksAuthentication(int methodId,
java.net.Socket proxySocket)
throws java.io.IOException{
if(methodId != METHOD_ID) return null;
java.io.InputStream in = proxySocket.getInputStream();
java.io.OutputStream out = proxySocket.getOutputStream();
out.write(request);
int version = in.read();
if(version < 0) return null; //Server closed connection
int status = in.read();
if(status != 0) return null; //Server closed connection, or auth failed.
return new Object[] {in,out};
}
//Private methods
//////////////////
/** Convert UserName password in to binary form, ready to be send to server*/
private void formRequest(){
byte[] user_bytes = userName.getBytes();
byte[] password_bytes = password.getBytes();
request = new byte[3+user_bytes.length+password_bytes.length];
request[0] = (byte) 1;
request[1] = (byte) user_bytes.length;
System.arraycopy(user_bytes,0,request,2,user_bytes.length);
request[2+user_bytes.length] = (byte) password_bytes.length;
System.arraycopy(password_bytes,0,
request,3+user_bytes.length,password_bytes.length);
}
}
Related Topics
Listview Viewholder Checkbox State
Why Can't I Use Resources.Getsystem() Without a Runtime Error
Double Tap: Zoom on Android Mapview
How to Get Selected Xls File Path from Uri for Sdk 17 or Below for Android
Firebase Database Points to Wrong Database Url
Nullpointerexception on Button.Findviewbyid()
Rotate Marker and Move Animation on Map Like Uber Android
What Is the Equivalent to a JavaScript Setinterval/Settimeout in Android/Java
Open Source Face Recognition for Android
Why Would One Declare an Immutable Class Final in Java
How to Use Mockito with Junit5
Why Should Wait() Always Be Called Inside a Loop
Running a .SQL Script Using MySQL with Jdbc
Is a Hashmap Thread-Safe for Different Keys
Serializing and De-Serializing Android.Graphics.Bitmap in Java