Daemon Threads Explanation

Some threads do background tasks, like sending keepalive packets, or performing periodic garbage collection, or whatever. These are only useful when the main program is running, and it's okay to kill them off once the other, non-daemon, threads have exited.

Without daemon threads, you'd have to keep track of them, and tell them to exit, before your program can completely quit. By setting them as daemon threads, you can let them run and forget about them, and when your program quits, any daemon threads are killed automatically.

Daemon threads vs daemon processes in Python

It's just names meaning different things in different contexts.

In case you are not aware, like threading.Thread, multiprocessing.Process also can be flagged as "daemon". Your description of "daemon processes" fits to Unix-daemons, not to Python's daemon-processes.

The docs also have a section about Process.daemon:

... Note that a daemonic process is not allowed to create child processes.
Otherwise a daemonic process would leave its children orphaned if it
gets terminated when its parent process exits. Additionally, these are
not Unix daemons or services, they are normal processes that will be
terminated (and not joined) if non-daemonic processes have exited.

The only thing in common between Python's daemon-processes and Unix-daemons (or Windows "Services") is that you would use them for background-tasks
(for Python: only an option for tasks which don't need proper clean up on shutdown, though).

Python imposes it's own abstraction layer on top of OS-threads and processes. The daemon-attribute for Thread and Process is about this OS-independent, Python-level abstraction.

At the Python-level, a daemon-thread is a thread which doesn't get joined (awaited to exit voluntarily) when the main-thread exits and a daemon-process is a process which gets terminated (not joined) when the parent-process exits. Daemon-threads and processes both experience the same behavior in that their natural exit is not awaited in case the main or parent-process is shutting down. That's all.

Note that Windows doesn't even have the concept of "related processes" like Unix, but Python implements this relationship of "child" and "parent" in a cross-platform manner.

I would think that "daemon" threads would keep running after the main
process has been terminated.

A thread cannot exist outside of a process. A process always hosts and gives context to at least one thread.

Are daemon threads killed when main thread calls sys.exit()?

Referring to the comment posted by zwer,

When a program exits, all of its children threads are killed with it. Threads that are not daemonic will prevent the program from exiting, hence preventing their own destruction. - zwer

In short, yes daemon threads will not stop the program from exiting thus they will be killed on exit.

Meaning of daemon property on Python Threads

Is this saying that this program won't ever finish?

Yes, that program won't finish, just try it out.

I ask because I have a situation where
in my main thread I'm calling
sys.exit(), and the process just hangs
and my other threads are running as I
can see the log. Does this have
anything to do with sys.exit() being
called with threads alive?

Yes, even exit won't stop other threads, it simply raises SystemExit in the main thread. So while the main thread will stop (just like it does on any other unhandled Exception), all other non-daemonic threads will continue to work.

How are daemon threads killed in python (2.7)

When main thread exits all daemon threads exit too. So if your main() exits all daemon threads will exit too.

However, and coming to the second part of your question,according to the official python threading docs for I/O operations this may not happen gracefully.

Daemon threads are abruptly stopped at shutdown. Their resources (such as open files, database transactions, etc.) may not be released properly. If you want your threads to stop gracefully, make them non-daemonic and use a suitable signalling mechanism such as an Event.

So if you want to check whether there are any zombie threads lying around, you can call the isAlive() function.
Now as to how to ensure that cleanup code is executed after the main thread dies; I think it depends on what you mean by dies.

If you mean normal exit of the script (e.g. KeyboardInterrupt exception, sys.exit() is called) then it's really worth taking a look at the atexit module, which registers functions to be executed upon the termination of a process.

If by "dying" you mean that main() is killed by a signal, or a Python fatal internal error occurs, or os._exit() is called then atexit module won't work. In that case, a possible solution that comes to my mind would be to create another watchdog process that is constantly monitoring your main() and will run the cleanup code when appropriate.

Daemon Threads in java and its process space

A "daemon" thread is intended to provide a general service in the background as long as the program is running, but is not part of the essence of the program. Thus, when all of the non-daemon threads complete, the program is terminated "abruptly", killing all daemon threads in the process.

So as soon as all non-daemon exits, the JVM shuts down all the daemons immediately, without any of the formalities you might have come to expect (Not even Finally). Because you cannot shut daemons down in a nice fashion, they are rarely a good idea.

Non-daemon Executors are generally a better approach, since all the tasks controlled by an Executor can be shut down at once.

Now Coming to your particular scenario.

A finishes the job and hence the JVM exits but Thread B is still
running- This Assumption is wrong B is not running

where Thread B execute ? In which process space does it execute?

B Will be terminated "abruptly" as soon as A is finished so it
not in execution now and hence no question of process space.

Joining a Daemon thread

If you want to be able to .join a thread, it's better to not make it a daemon. Daemon threads are for when you want a thread to do its thing and you're not too concerned about when or if it finishes.

The point of making daemon threads is that the program will exit when there are no non-daemon threads left alive.

From the threading docs:

A thread can be flagged as a “daemon thread”. The significance of this
flag is that the entire Python program exits when only daemon threads
are left. The initial value is inherited from the creating thread. The
flag can be set through the daemon property or the daemon constructor
argument.

You can actually call .join on daemon threads, but it's generally considered to be not good practice.

You could get a daemon thread to set an Event just before it finishes, which one or more other threads check, but it's simpler just to use a non-daemon thread and .join it.

An earlier version of this answer claimed that you can't .join a daemon thread. That's incorrect. I was getting mixed up with dummy threads. Sorry about that. :oops:

setDaemon() method of threading.Thread

Here is some basic code using threading:

import Queue
import threading

def basic_worker(queue):
    while True:
        item = queue.get()
        # do_work(item)
        print(item)
        queue.task_done()
def basic():
    # http://docs.python.org/library/queue.html
    queue = Queue.Queue()
    for i in range(3):
         t = threading.Thread(target=basic_worker,args=(queue,))
         t.daemon = True
         t.start()
    for item in range(4):
        queue.put(item)
    queue.join()       # block until all tasks are done
    print('got here')

basic()

When you run it, you get

% test.py
0
1
2
3
got here

Now comment out the line:

         t.daemon = True

Run it again, and you'll see that the script prints the same result, but hangs.
The main thread ends (note that got here was printed), but the second thread never finishes.

In contrast, when t.daemon is set to True, the thread t is terminated when the main thread ends.

Note that "daemon threads" has little to do with daemon processes.

Daemon Threads Explanation