Curl Returnin 404 Error

cURL returns 404 while the page is found in browser

I quickly checked the said page with LiveHeaders enabled and I noticed bunch of cookies set. I suspect that, since it's not "normal" url, you need to hand those cookies while being redirected otherwise you end being kicked out with 404. Use CURLOPT_COOKIEJAR with your cURL instance at start. See: http://php.net/manual/pl/function.curl-setopt.php

Curl_exec returns 404 not found error whereas url is found in browser

When you set the CURLOPT_NOBODY option to true, it means cURL will do a HEAD request instead of a GET request.

The URL seems to give a 200 on GET requests, but a 404 on HEAD requests.

Try setting CURLOPT_NOBODY to false.

curl returns 404 on valid page

The problem seems to come from you CURLOPT_NOBODY option.

I've tested your code both with and without this line and the http code returns 404 when CURLOPT_NOBODY is present, and 200 when it's not.

The PHP manual informs us that setting the CURLOPT_NOBODY option will transform your request method to HEAD, my guess is that the server on which bostonglobe.com is hosted doesn't support that method.

cuRL returnin 404 error

I can guess a few things that it can be checked from the server side, to show the error.

1) As it is stated in other answers, be sure to set all the necessary headers, you can check them e.g. by firebug, as it is shown in here,

or you can get the headers by php get_headers function.
to set it use

curl_setopt($ch, CURLOPT_HTTPHEADER, array("HeaderName: HeaderValue"));

2) When you open a page in the browser(excluding form submit with post method) it makes a get request, instead of post, so if in the server side it is checked $_GET, then your post request will not be considered.

3) If you sure that it should be a post request(say, it is a form submit), then the following can be a problem: some forms can have hidden fields, that again are being checked in the server, and if they are not set, error can be returned. So, you should look at the source code of the form and add them(if there are any) to your post parameters.

4) if you are submitting a form, be sure to set the submit button with its name and value as well, because similar to hidden fields, this can be checked as well.

5) Cookies can be a problem as well, because by default browser has it , and curl does not. To to able to set and read cookies use this code

// set cookie 
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file);
    
// use cookie
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file);

here, $cookie_file path to the cookies file. Do not know in linux or mac, but in windows be sure to use absolute path to the cookie file.

6) Also, you can set the referer by

curl_setopt($ch, CURLOPT_REFERER, 'http://www.myaddress.com/mypage.php');

EDIT: In case of ajax request you might want to add a header X-Requested-With with value as XMLHttpRequest

cURL return HTML error 404 instead of JSON when called within bash script requesting php API

With:

set -- "https://example.com/api/"${1}"-api.php"

You replace all the arguments list with only one argument (the URL).
Then argument $2 becomes empty as it calls curl:

curl -S -v -d "$2" "$1"

You don't need to use set -- with bash as it has arrays. It is not needed even for your usage since it does not deal with array data at all. Anyway, here is your code fixed while still using set --

#! /usr/bin/bash
if [ $# = 2 ]; then
    set -- "https://example.com/api/"${1}"-api.php" "$2"
#   t1=`date +%s`
    curl -S -v -d "$2" "$1"
else
    echo "you should specify only two arguments"
fi

Here is equivalent code without using set -- and making arguments immutable, witch is a better coding practice:

#! /usr/bin/bash
if [ $# = 2 ]; then
    url="https://example.com/api/$1-api.php"
#   t1=`date +%s`
    data="$2"
    curl -S -v -d "$data" "$url"
else
    echo "you should specify only two arguments"
fi

Resource Not Found 404 Error when making an API call from cURL

kek just solved this very challenge.

Have you considered that the "Cyber Gang" (context of the challenge) may have changed the endpoint from balances to something else?

Considering that you get a 404 balance from curling get-balances, perhaps you could figure out a way to find other subdomains :)

---

Related Topics