Why Firefox Requires Geckodriver

Why Firefox requires GeckoDriver?

Addressing your questions one by one:

Why does Firefox require GeckoDriver? - For Mozila Firefox till version 47.x it was the legacy browser and we didn't need gecko driver. Mozila Firefox from version 47.x onwards it comes with Marionette, which is an automation driver for Mozilla's Gecko engine. It can remotely control either the UI or the internal JavaScript of a Gecko platform, such as Firefox.
With Chrome and IE works out of the box - Ideally neither Chrome nor IE should have worked. But as you have added the location of the binaries in the Environment Variables knowingly/unknowingly while installation/configuration of Google Chrome & MS Internet Explorer or other dependent softwares, those binaries are easily located & used automatically.
Why only for Firefox do we have to download/configure this driver? - It is not only Firefox but also for Google Chrome & MS Internet Explorer to work with Selenium 3.4.0 you need to mandatory download gecko driver v0.16.0 (or above) from this location or Chrome driver or IEDriverServer and save it in your machine. Upgrade your Mozila Firefox or Google Chrome or MS Internet Explorer to the latest stable version. Use the absolute path of the geckodriver/chromedriver/iedriver in your code while System.setProperty as follows:
```
System.setProperty("webdriver.gecko.driver",  "C:\\Utility\\BrowserDrivers\\geckodriver.exe");
```

geckodriver' executable needs to be in PATH using GeckoDriver and Firefox through Selenium

You can download and store the GeckoDriver executable anywhere with in your system and you need to do pass the absolute path of firefox binary through the attribute binary_location as follows:

from selenium import webdriver
from selenium.webdriver.firefox.options import Options

options = Options()
options.binary_location = r'C:\Program Files\Mozilla Firefox\firefox.exe'
driver = webdriver.Firefox(firefox_options=options, executable_path=r'C:\Users\\ojadi\Downloads\geckodriver-v0.28.0-win64\geckodriver.exe')
driver.get('http://google.com/')

Is geckodriver required for Selenium 3.7 and Firefox ESR 52.4.1?

Does this mean that Selenium.WebDriver is talking directly to the Firefox browser using the Marionette protocol?

As per my understanding, when you set System.setProperty("webdriver.firefox.marionette", "false"); to false or do FirefoxOptions options = new FirefoxOptions() .setLegacy(true); that means it is using the legacy extension (not marionette and gecko) as described in firefox properties here

Marionette can not be used without using gecko (or rather if you want to interact with gecko based browsers, you have to use marionette ). Marionette has a gecko component in it which is the marionette server as mentioned here

geckodriver as it is written on github , provides an API to communicate with Gecko browsers

This program provides the HTTP API described by the WebDriver protocol
to communicate with Gecko browsers

for selenium 3.0 onwards marionette is enabled by default as mentioned here

For more info please refer to this question also

If you are interested in learning more about marionette client-server-gecko interaction , have a look here

EDIT:

the source code of geckodriver states below points about geckodriver at different locations in readme.md

geckodriver is a Proxy for using W3C WebDriver-compatible clients to interact with
Gecko-based browsers.

Selenium client bindings will pick up the geckodriver binary executable
from your [system’s PATH environmental variable][PATH]

3.Since geckodriver is a separate HTTP server that is a complete
remote end
implementation of [WebDriver], it is possible to avoid using the
Selenium remote server

geckodriver translates WebDriver [commands], [responses],
and [errors] to the [Marionette protocol], and acts as a proxy between
[WebDriver] and [Marionette]

By default geckodriver tries to find
and use the system installation of Firefox

So , to answer your questions, this is how it all works

Selenium language bindings reaches to -->geckodriver.exe finds -->system firefox installation(this can be changed though)reaches to inbuilt --> marionette client reaches to --> marionette server reaches to --> gecko engine of the browser which inturn calls out --> element.js,interaction.js,action.js,evaluate.js in gecko engine depending on what is being requested by the bindings or client.

Selenium, Firefox and GeckoDriver

Let me try to address your question one by one:

By using Selenium 3.4.0 what is the latest version of Firefox that doesn't require GeckoDriver to run tests - With Selenium 3.4.0 you can use Mozilla Firefox 47.x without gecko driver. From Mozilla Firefox 47.x on-wards gecko driver is mandatory.

With Selenium 3.4.0, geckodriver v0.16.1 & Mozilla Firefox 53.0 this piece of code works fine:

System.setProperty("webdriver.gecko.driver", "C:\\Utility\\BrowserDrivers\\geckodriver.exe");
DesiredCapabilities dc = DesiredCapabilities.firefox();
dc.setCapability("marionette", true);
WebDriver driver =  new FirefoxDriver(dc);
driver.get("http://google.com/");

Moreover, if you are using any extensions in your default Firefox profile either you need to disable/delete them or create a new Firefox profile for your automation work.

Let me know if this helps you.

Firefox with geckodriver don't start on Ubuntu Server

You need to update your Firefox browser, it can happen because of mismatch between GeckoDriver and the Firefox Browser.

If that also don't work then use below version of gecko driver, try with 2nd and 3rd last updated version of gecko driver.

Not recommanded, still it is possible that you need to downgrade your browser and try it latest or try with 2nd and 3rd last updated version of gecko driver. you should try to update the same with latest version afterwards when binary are available for latest firefox version

It also one of the probability that your geckodriver is not quit and so it is still running behind which suppress new session to open, you can use below article to kill your geckodriver process

https://www.tecmint.com/find-and-kill-running-processes-pid-in-linux/

Selenium using Python - Geckodriver executable needs to be in PATH

selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable needs to be in PATH.

First of all you will need to download latest executable geckodriver from here to run latest Firefox using Selenium

Actually, the Selenium client bindings tries to locate the geckodriver executable from the system PATH. You will need to add the directory containing the executable to the system path.

On Unix systems you can do the following to append it to your system’s search path, if you’re using a Bash-compatible shell:
```
  export PATH=$PATH:/path/to/directory/of/executable/downloaded/in/previous/step
```
On Windows you will need to update the Path system variable to add the full directory path to the executable geckodriver manually or command line** (don't forget to restart your system after adding executable geckodriver into system PATH to take effect)**. The principle is the same as on Unix.

Now you can run your code same as you're doing as below :-

from selenium import webdriver

browser = webdriver.Firefox()

selenium.common.exceptions.WebDriverException: Message: Expected browser binary location, but unable to find binary in default location, no 'moz:firefoxOptions.binary' capability provided, and no binary flag set on the command line

The exception clearly states you have installed Firefox some other location while Selenium is trying to find Firefox and launch from the default location, but it couldn't find it. You need to provide explicitly Firefox installed binary location to launch Firefox as below :-

from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary

binary = FirefoxBinary('path/to/installed firefox binary')
browser = webdriver.Firefox(firefox_binary=binary)

https://github.com/mozilla/geckodriver/releases

For Windows:

Download the file from GitHub, extract it, and paste it in Python file. It worked for me.

https://github.com/mozilla/geckodriver/releases

For me, my path path is:

C:\Users\MYUSERNAME\AppData\Local\Programs\Python\Python39

Why Firefox Requires Geckodriver