What Happens When a Computer Program Runs

How do instructions from a program get sent to different memory levels? Are they pre-loaded when the chips are manufactured for the first time?

How do instructions from a program get sent to different memory levels?

The processor pulls them into caches using the program counter aka instruction pointer that refers to the next instruction to execute.

Are they pre-loaded when the chips are manufactured for the first time?

Not for general purpose programs. Only special environments burn programs into permanent storage, like ROM, which may happen for booting a computer or for an embedded system. General purpose programs rely on an operating system to associate their code & data with memory.

My question now is, do instructions required for program execution get moved to memory by the compiler the same way data gets pushed?

Recap: The compiler follows the statements & expressions of the (source) code program, translating them into machine code instructions & data; these machine code instructions instruct the processor what to do to carry out the high level language program. This includes machine code instructions to load data into CPU registers, sometimes so that the data is closer to the CPU so quicker to work with, other times because that is the only way that the CPU can operate on the data (by the machine code instruction set of the processor).

There is a difference between having machine code instructions in memory and having them in the processor.

The compiler does not, per se, command instructions to be moved into the processor. The processor is an interpreter of machine code instructions, and all on its own it will pull instructions into itself for execution. The compiler lays out sequences of instructions and the CPU automatically brings them in for execution — that is is the CPU's purpose.

Program code _{(modulo modern caches)} is too large to directly load into the CPU. Thus, programs are stored in main memory, like data that is too large to fit directly in the CPU. Program code is sequences of machine code instructions. Since these instructions are stored in memory, each individual instruction has a unique memory address! The processor refers to instructions by their memory address, and by their memory address has a concept of what instruction is executing now and what will/should execute next.

The mechanism for this goes to the instruction pointer register (in intel-speak) or the program counter register (in virtually all other machine code languages), but the concept is the same: there is a fundamental register inside the processor whose job is to hold the address of machine code instruction that is executing now, and whose job is also to change to indicate what instruction to execute next.

As mentioned, the processor is an interpreter of machine code instructions. It uses its internal program counter in what is called a fetch-execute pattern, where it demands of the memory system the value of the machine code instruction to execute next. That demand, a request to read an some particular memory address, may cause cache misses, or page fault (or both), which ultimately, through conspiracy of both the processor and the operating system, will cause machine code instructions to load into memory, to load in to caches, to load into the processor itself.

Each machine code instruction of the program informs the processor what machine code instruction to execute next. The program can move the program counter backwards to re-execute instructions that have already executed (forming loops) or move it forward to exit a loop or skip a then or else statement. There is a cooperation between the processor and the machine code, where the processor is told, at every step, what machine code to execute next. So, the machine code program is directing the execution of sequences of the machine code program, but it is the processor that is doing the actual execution (though under the direction of the program).

Would we say that the compiler tells the processor to load instruction to execute into the CPU? Sort of, but not quite as explicitly as commands to load data into the CPU.

The compiler creates a sequences in the machine code program used to control execution, so I would say it implicitly tells the processor what instructions to bring into the caches. Otherwise (modulo memory mapping and page faulting) the machine code instructions are already in memory (by the operating system associating the memory with the program code & data), and the processor will manage caches to keep frequently executing instructions in the higher levels of the cache.

what do CPU designers load into the cache/main memory when it comes fresh out of the foundry?

Nothing, the memory is considered blank and can be repurposed. On some systems we cannot even guarantee the value of each memory cell when the processor boots. Where the program (as a file on disc) specifies the proper initial values for memory, then the original values that were in memory, from boot or from another program's execution, are replaced with those specified values. Other memory used by the program is zeroed out before use.

are any instructions preloaded into memory at the time of designing and manufacturing of the chip so that it executes a pre-defined set of sequences when it is powered up for the first time?

Yes, sometime some memories are hard-coded with code and data that helps the computer boot. This is not a general purpose way that programs run.

how does a program runs in memory and the way memory is handled by Operating system

1) What is the stack which this image is referring to?

The stack is for allocating local variables and function call frames (which include things like function parameters, where to return after the function has called, etc.).

2) What is memory mapping segment which is referring to file mappings?

Memory mapping segment holds linked libraries. It also is where mmap calls are allocated. In general, a memory mapped file is simply a region of memory backed by a file.

3) What does the heap have to do with a process. Is the heap only handled in a process or is the heap something maintained by the operating system kernel and then memory space is allocated by malloc (using the heap) when ever a user space application invokes this?

The heap is process specific, and is managed by the process itself, however it must request memory from the OS to begin with (and as needed). You are correct, this is typically where malloc calls are allocated. However, most malloc implementations make use of mmap to request chunks of memory, so there is really less of a distinction between heap and the memory mapping segment. Really, the heap could be considered part of the memory mapped segment.

4) Does this mean that at a time only one program runs in memory occupying entire 4 GB of RAM?

No, that means the amount of addressable memory available to the program is limited to 4 GB of RAM, what is actually contained in memory at any given time is dependent on how the OS allocated physical memory, and is beyond the scope of this question.

5) Is it referring to randomizing the stack within a process or is it referring to something which is left after counting the space of all the processes?

I've never seen anything that suggests 4gb of space "hampers" the effectiveness of memory allocation strategies used by the OS. Additionally, as @Jason notes, the locations of the stack, memory mapped segment, and heap are randomized "to prevent predictable security exploits, or at least make them a lot harder than if every process the OS managed had each portion of the executable in the exact same virtual memory location." To be specific, the OS is randomizing the virtual addresses for the stack, memory mapped region, and heap. On that note, everything the process sees is a virtual address, which is then mapped to a physical address in memory, depending on where the specific page is located. More information about the mapping between virtual and physical addresses can be found here.

This wikipedia article on paging is a good starting point for learning how the OS manages memory between processes, and is a good resource to read up on for answering questions 4 and 5. In short, memory is allocated in pages to processes, and these pages either exist in main memory, or have been "paged out" to the disk. When a memory address is requested by a process, it will move the page from the disk to main memory, replacing another page if needed. There are various page replacement strategies that are used and I refer you to the article to learn more about the advantages and disadvantages of each.

How computer CPU executes a Software Application

As I mentioned in the comments, using a disassembler tool like objdump -d in Linux can help you take a binary/executable file and generate the set of assembly instructions that comprises the entire program.

For instance, if you use objdump -d on notepad.exe (which won't be completely accurate or insightful because objdump is for Linux and Notepad is a Windows program) you will see:

notepad.exe:     file format pei-x86-64

Disassembly of section .text:

0000000140001000 <.text>:
   140001000:   cc                      int3   
   140001001:   cc                      int3   
   140001002:   cc                      int3   
   140001003:   cc                      int3   
   140001004:   cc                      int3   
   140001005:   cc                      int3   
   140001006:   cc                      int3   
   140001007:   cc                      int3   
   140001008:   40 55                   rex push %rbp
   14000100a:   48 8d 6c 24 e1          lea    -0x1f(%rsp),%rbp
   14000100f:   48 81 ec d0 00 00 00    sub    $0xd0,%rsp
   140001016:   48 8b 05 8b 14 03 00    mov    0x3148b(%rip),%rax        # 0x1400324a8
   14000101d:   48 33 c4                xor    %rsp,%rax
   ...

I'm using objdump because I'm on Linux, but as @PeterCordes pointed out in the comments, the assembly instructions should be the same with a Windows disassembler.

The objdump output has more than 43k assembly instructions, so deciphering what each section of assembly does would take forever. This is the entire set of instructions of what Notepad could execute. So if you want to know which assembly instructions are executed and in what order when you do something like type ABC and save it, you would need to use some sort of tracer (e.g. gdb) to step through only those specific executed instructions.

How does program execute? Where does the Operating Systems come into play?

For starters, a modern CPU has (at least) two modes, a mode in which it's running the core of the Operating System itself ("kernel mode") and a mode in which it's running programs ("user mode"). When in user mode, the CPU can't do a whole lot of things.

For instance, a mouse click is typically noticed in the kernel, not user mode. However, the OS dispatches the event to user mode and from there to the correct program. The other way around also requires cooperation: a program can't draw to the screen freely, but needs to go through the OS and kernel mode to draw on its part.

Similarly, the act of starting a program is typically a cooperation. The shell part of the OS is a user-mode program too. It gets your mouse click, and determines that it's a mouse click intended to start a process. The shell then tells the kernel-mode part of the OS to start a new process for that program.

When the kernel mode needs to start a new process, it first allocates memory for bookkeeping, and then proceeds to load the program. This involves retrieving the instructions from the binary, but also hooking up the program to the OS. This usually requires finding the entry point (classically int main(int argc, char** argv)) of the binary, and all points where the program wants to call the OS.

Different Operating Systems use different ways to hook up programs with the OS. As a result, the loading process differs, and the file formats for binaries can differ too. It's not absolute; the ELF format for binaries is used for a number of Operating Systems, and Microsoft uses its PE format on all its current Operating Systems. In both cases, the format does describe the precise format of the binary, so the OS can decide whether the program can be hooked up to the OS. For instance, if it's a Win32 binary, it will be in the PE format, therefore Linux won't load that, Windows 2000 will, as will Windows 7-64. A Win64 binary on the other hand is in PE format too, but Windows 2000 will reject it.

what happens when one says a computer 'Hangs' or freezes?

The computer may freeze under different circumstances. These are the ones I can think of now:

x86 CLI and HLT instructions. CLI disables interrupts, so no asynchronous event (like a timer interrupt or pressing a key) can move the CS:EIP instruction pointer to another instruction and HLT literally halts the processor.
The instruction is seldomly used and only allowed to by the kernel if some initialization routine in boot code fails for example. Although restarting is a better option here.

Note that HLT only halts the core it is run on, not all cores.
A window is not responding (commonly found on Windows). This differs from application to application. More information here.
A resource is attempted to be acquired but is protected by a lock and has already been acquired. The process waits (actually busy-loops or yields another process) until it finally can acquire the resource. This is only a temporary state, though, as opposed to...
A deadlock. Multiple circumstances under which it can occur but a common one is two processes attempting to acquire resources they provide to each other at the same time. None can handle the acquisition request because both are waiting for the other processes, so both processes end up uninterruptible. This is the reason for uninterruptible processes on Linux, which cannot be killed despite being dispatched the signal to.
Multitasking on a slow processor or a processor with few threads. A bad scheduling algorithm makes the situation even worse.

Since one process occupies at least one thread, the number of processes effectively running concurrently is very low. This could be stabilized by a very fast processor, though.

This results in a long response time to events like mouse clicks.

For x86 systems, HLT is the only instruction really hindering the instruction pointer from advancing.

All other cases are just (potentially infinite) loops or program/operating system bugs.