Java - Quickest way to check if URL exists
Try sending a "HEAD" request instead of get request. That should be faster since the response body is not downloaded.
huc.setRequestMethod("HEAD");
Again instead of checking if response status is not 400, check if it is 200. That is check for positive instead of negative. 404,403,402.. all 40x statuses are nearly equivalent to invalid non-existant url.
You may make use of multi-threading to make it even faster.
Easy way to check if a link is reachable or not from a java aplication
You can send HEAD
request to the URL and see what response code you are getting. If response code is 404
then you say the URL is not exists. The HEAD
request is much faster than GET
. Also, The HEAD
request will not return response body. This is a standard way to check if URL is exists or not. Please see below code snippet which uses apache HttpClient
version 4.5.5
to check if URL exists or not:
/**
* @param url
* @return
*/
public static boolean isReachable(String url) {
boolean isReachable = true;
try (CloseableHttpClient httpClient = HttpClients.custom()
.setSSLContext(new SSLContextBuilder().loadTrustMaterial(null, TrustAllStrategy.INSTANCE).build())
.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE)
.build())
{
HttpHead request = new HttpHead(url);
CloseableHttpResponse response = httpClient.execute(request);
if (response.getStatusLine().getStatusCode() == 404) {
System.out.println("URL " + url + " Not found");
isReachable = false;
}
} catch (Exception e) {
e.printStackTrace();
isReachable = false;
}
return isReachable;
}
check for validity of URL in java. so as not to crash on 404 error
The URLConnection
throws an error when the status code of the webpage your downloading returns anything other than 2xx (such as 200 or 201 ect...). Instead of passing Jsoup a URL or String to parse your document consider passing it an input stream of data which contains the webpage.
Using the HttpURLConnection
class we can try to download the webpage using getInputStream()
and place that in a try/catch
block and if it fails attempt to download it via getErrorStream()
.
Consider this bit of code which will download your wiki page even if it returns 404
String URL_czech = "https://en.wikipedia.org/wiki/Hudson+Township+%28disambiguation%29";
URL wikidata_page = new URL(URL_czech);
HttpURLConnection wiki_connection = (HttpURLConnection)wikidata_page.openConnection();
InputStream wikiInputStream = null;
try {
// try to connect and use the input stream
wiki_connection.connect();
wikiInputStream = wiki_connection.getInputStream();
} catch(IOException e) {
// failed, try using the error stream
wikiInputStream = wiki_connection.getErrorStream();
}
// parse the input stream using Jsoup
Jsoup.parse(wikiInputStream, null, wikidata_page.getProtocol()+"://"+wikidata_page.getHost()+"/");
Check if URL exists or not on Server
You will get Network On Main Thread Exception
Look at NetworkOnMainThreadException
so your method always returns false because of:
catch (Exception e) {
e.printStackTrace();
return false;
}
quick fix:
public class MainActivity extends Activity {
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
String customURL = "http://www.desicomments.com/dc3/08/273858/273858.jpg";
MyTask task = new MyTask();
task.execute(customURL);
}
private class MyTask extends AsyncTask<String, Void, Boolean> {
@Override
protected void onPreExecute() {
}
@Override
protected Boolean doInBackground(String... params) {
try {
HttpURLConnection.setFollowRedirects(false);
HttpURLConnection con = (HttpURLConnection) new URL(params[0]).openConnection();
con.setRequestMethod("HEAD");
System.out.println(con.getResponseCode());
return (con.getResponseCode() == HttpURLConnection.HTTP_OK);
}
catch (Exception e) {
e.printStackTrace();
return false;
}
}
@Override
protected void onPostExecute(Boolean result) {
boolean bResponse = result;
if (bResponse==true)
{
Toast.makeText(MainActivity.this, "File exists!", Toast.LENGTH_SHORT).show();
}
else
{
Toast.makeText(MainActivity.this, "File does not exist!", Toast.LENGTH_SHORT).show();
}
}
}
}
With a ScheduledThreadPoolExecutor:
but remember to shut down it!!
public class MainActivity extends Activity {
String customURL;
String msg = "";
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
customURL = "http://www.desicomments.com/dc3/08/273858/273858.jpg";
final ScheduledThreadPoolExecutor myTimer = new ScheduledThreadPoolExecutor(1);
myTimer.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
try {
HttpURLConnection.setFollowRedirects(false);
HttpURLConnection con = (HttpURLConnection) new URL(customURL).openConnection();
con.setRequestMethod("HEAD");
System.out.println(con.getResponseCode());
if(con.getResponseCode() == HttpURLConnection.HTTP_OK){
msg = "File exist!";
}else{
msg = "File does not exist!";
}
runOnUiThread(new Runnable() {
@Override
public void run() {
Toast.makeText(MainActivity.this, msg, Toast.LENGTH_SHORT).show();
}
});
}
catch (Exception e) {
e.printStackTrace();
return;
}
}
}, 0,10000, TimeUnit.MILLISECONDS);
}
Related Topics
Hash String via Sha-256 in Java
Is This the Best Way to Rewrite the Content of a File in Java
Registering Multiple Keystores in Jvm
Java Jtree Directory Structure from File Paths
How to Test If My Font Is Rendered Correctly in PDF
Error Message "Unreported Exception Java.Io.Ioexception; Must Be Caught or Declared to Be Thrown"
How to Manually Set an Authenticated User in Spring Security/Springmvc
Plsql Jdbc: How to Get Last Row Id
Kill -3 to Get Java Thread Dump
Stopping a Window from Displaying Till It Is Fully Drawn
How to Execute Windows Commands Using Java - Change Network Settings