System Programming - Lecture 2, v3 - Kernel, Structure, Process Scheduler

Summary

The lecture explains how processes are represented and managed by the operating system, how they use memory and file descriptors, how processes communicate, and how user programs request services from the kernel.

1. What a process is

• A process is more than a running program’s machine code. It has memory regions for code, global variables, the stack, and dynamic allocations, as well as resources such as file descriptors and signal-related state.
• In Linux, the kernel schedules tasks. A single-threaded process is represented by one task; a multithreaded process has multiple tasks.
• Processes provide virtualization: programs can use an abstract address space and run without needing to manage which physical CPU core or physical memory location is involved.
• Linux processes form a family tree. New processes are created from existing ones, and the initial init process (PID 1) is the common ancestor described in the lecture.

2. Creating and ending processes

• A new process is normally created with fork(). The child initially continues executing the same code as its parent.
  • fork() returns 0 in the child.
  • It returns the child’s PID in the parent.
  • The two processes can use that return value to follow different paths.
• A child commonly starts a different program by calling an exec-family function, which replaces its program image with the requested executable.
• A process ends through exit(), including when a program returns from main().
• The parent can use wait() to collect a child’s termination status. The status value contains information that must be decoded with macros such as WEXITSTATUS; it is not simply the child’s exit code.

3. Copy-on-write, vfork(), and zombies

• Although fork() appears to copy a process, it is generally made efficient with copy-on-write (COW):
  • Parent and child initially share memory pages.
  • If either process tries to modify a shared page, the kernel creates a private copy of that page.
  • Copying therefore happens on demand, page by page, rather than copying all memory immediately.
• vfork() can avoid some of the overhead of setting up COW, especially when the child will immediately call exec(). It is risky if the child modifies or otherwise uses shared state before calling exec() or exit(). The parent is suspended until the child does one of those things.
• If a child exits but its parent does not call wait(), the child remains temporarily as a zombie:
  • Its program memory is released, but the kernel retains a small record containing its exit information and PID.
  • The record is removed when the parent collects the status.
  • Many unreaped zombies can consume process-table and PID resources.
  • If the parent exits, the child is adopted by init, which can reap it.

4. Process memory and its regions

• Each process has a virtual address space. The same virtual address in different processes can refer to different physical memory, or to no valid memory at all.
• The kernel maps virtual regions to physical memory and tracks their permissions and backing. Gaps between mapped regions are invalid; accessing them can cause a segmentation fault.
• Common memory regions include:
  • Text: executable machine instructions.
  • Data: initialized global and static variables.
  • BSS: uninitialized global and static variables, which are zero-initialized when the program starts.
  • Heap: memory used for dynamic allocation, commonly through malloc() or new.
  • Stack: function calls, arguments, and local variables.
  • Arguments and environment: command-line arguments and environment variables supplied at startup.
  • Other mappings may include shared libraries, debug information, and memory mapped from files.
• The lecture emphasizes that the heap is not necessarily one simple region that grows continuously. Allocators may request memory from the kernel in larger blocks and manage smaller allocations themselves. mmap() is one way to obtain mapped memory and is also used for purposes such as loading shared libraries.
• Stack growth is not unlimited: the kernel can extend a stack mapping when a valid stack access reaches its boundary, subject to system limits.
• Stack and heap memory usually use the same physical memory. Stack allocation is often cheaper because it typically adjusts a stack pointer; heap allocation requires more bookkeeping. Accessing already allocated memory is not inherently faster just because it is on the stack.

5. Inspecting memory and program state

• On Linux, /proc/<pid>/maps exposes a process’s mapped memory regions, including address ranges, permissions, and backing files.
• Tools such as objdump can inspect object and executable files, including their sections and machine code.
• Debuggers such as GDB or LLDB can use debug information (typically enabled with -g) to relate machine instructions to source code.
• A core dump records process state after certain failures, such as an unhandled segmentation fault. It can be inspected with a debugger, but full dumps can be very large; more limited dump formats are also used.
• Environment variables are key-value data passed to a process by its parent. The lecture notes that reading them is common, while changing them in multithreaded programs requires care.

6. File descriptors

• A file descriptor is a small integer used by a program to refer to a kernel-managed object. It may refer to a file, terminal, socket, pipe, or another resource—not only a regular file.
• The standard descriptors are:
  • stdin for input.
  • stdout for ordinary output.
  • stderr for error output.
• Library functions such as printf() ultimately write through a descriptor; programs can also use lower-level operations such as write().
• In the kernel, descriptor numbers refer to entries that point to open-file objects. Multiple descriptor numbers can refer to the same underlying open-file object and therefore share state such as the current file position.
• Opening a file normally starts at the beginning. O_APPEND makes each write position the file at the end before writing. Merely seeking to the end first is not safe when multiple processes write concurrently, because another process can write between the seek and the write. A write may also be partial, so applications should not assume an arbitrarily large write is atomic.

7. Duplicating descriptors and using pipes

• dup() creates another descriptor referring to the same open-file object. dup2() lets the caller choose the descriptor number to replace.
• After fork(), the child inherits descriptors that refer to the same underlying open-file objects as the parent’s corresponding descriptors.
• A pipe provides a one-way communication channel:
  • One end is used for writing and the other for reading.
  • Two pipes are needed for two-way communication.
  • After a fork(), both processes may hold references to both ends, so each should close the ends it does not use.
  • Closing unused write ends is particularly important: a reader sees end-of-file only after all write references are closed. Leaving one open can make a reader wait indefinitely.
• A common way to connect a pipe to a program’s standard input or output is to use dup2(). A child can then call an exec-family function to run another program, allowing the parent to communicate with that program through the pipe.

8. Interrupts, exceptions, and system calls

• A task may be interrupted because its time slice ends, a higher-priority task needs to run, a hardware exception occurs, or a signal arrives.
• Some hardware events are handled transparently by the kernel, such as a copy-on-write page fault or valid stack growth. Others can be reported to the process or cause it to terminate; an invalid memory access commonly results in a segmentation fault.
• Processes use system calls to ask the kernel to perform operations they cannot do directly, such as opening files, creating pipes, or communicating over a network.
• System calls are identified by numbers at the kernel boundary. Programs commonly access them through libraries, which provide a more portable interface.
• The lecture describes two ways to enter the kernel on x86 systems:
  • The older software-interrupt approach, such as Linux’s int 0x80.
  • Faster processor instructions such as sysenter or syscall, depending on the system and architecture.
• System-call arguments are passed using registers and, depending on the calling convention, sometimes memory such as the user stack.
• The vDSO is a kernel-provided mapping that lets programs perform certain operations—such as reading time—without a full system-call transition.
• The lecturer contrasts Linux’s system-call interface with Windows, where applications are expected to use documented system libraries rather than rely directly on system-call numbers.

Main practical lessons

• Use wait() to reap child processes and obtain their exit status.
• Understand that fork() usually shares pages initially and copies them only when modified.
• Treat vfork() as a restricted optimization, not a general substitute for fork().
• Do not infer that stack memory is automatically cleared when a function returns.
• Inspect process mappings and executable sections with system tools when learning or debugging.
• When building pipelines, close every unused pipe end and close a writer when no more data will be sent.
• Do not assume that a seek followed by a write is safe against concurrent writers, or that every write completes the entire requested buffer.
• Use system-call wrappers and documented APIs unless low-level, system-specific behavior is explicitly required.

Speakers and sources featured

• Speaker: One lecturer/presenter; no additional speakers are identifiable in the subtitles.
• Sources and systems discussed: Linux kernel internals and Linux/POSIX-style APIs; comparisons with macOS and Windows; tools including ps, /proc/<pid>/maps, objdump, GDB, and LLDB. These are references discussed by the lecturer, not additional speakers.