C Language – Understanding Buffer Overflow Attacks and Weaknesses

1. Buffer Overflow

1.1. Security Vulnerabilities

1.1.1. Causes

  • Buffer Overflow is a security vulnerability where data is written beyond the size of a buffer, overwriting adjacent memory.
  • When data is written past the buffer boundary, adjacent memory gets overwritten.
Buffer Overflow

1.1.2. Impact and Risks

  • The program may crash abnormally or produce errors.
  • Critical data may be corrupted, causing unexpected behavior.
  • An attacker may manipulate memory to execute arbitrary code or take control of the system.

1.2. Stack Buffer Overflow

  • Stack Buffer Overflow is a vulnerability in which data is written beyond the size of a buffer allocated in the Stack area, overwriting adjacent memory regions.
  • The main causes are insufficient input length validation and the use of unsafe functions that do not perform bounds checks.
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

// Demonstrate stack buffer overflow:
//   Stack layout: [buffer 16 bytes][cmd 16 bytes]
static void vulnerable(const char *input)
{
    char cmd[16]  = "date";  // default command to execute
    char buffer[16];         // overflow target (no bounds check)

    // VULNERABLE: strcpy does not check input length.
    // Input longer than 16 bytes overwrites cmd[].
    strcpy(buffer, input);

    // Execute whatever cmd[] contains (may be overwritten).
    system(cmd);
}

int main(int argc, char *argv[]) 
{
    char* input = "";

    if (argc >= 2) {
        input = argv[1];
    }

    vulnerable(input);
    return 0;
}
  • The strcpy() function does not check the input data length, so data larger than the buffer size can be copied.
  • As a result, adjacent variables on the stack are overwritten, and the program's behavior can change.
$ ./test1.out
Wed May 20 15:55:38 UTC 2026

$ ./test1.out "AAAAAAAAAAAAAAAAls -al"
total 80
drwxr-xr-x  2 root root  4096 May 20 15:55 .
drwxr-xr-x 13 root root  4096 May 20 15:46 ..
-rw-r--r--  1 root root   166 May 20 13:50 Makefile
-rw-r--r--  1 root root   853 May 20 15:55 test1.c
-rwxr-xr-x  1 root root 15960 May 20 15:55 test1.out

1.2.1. Normal Operation

input: ""

[Stack]
Low address
┌──────────────────────────────────────────────────────────────────────────────────┐
│ buffer[16]                                                                       │
│ [\0 ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ] │
├──────────────────────────────────────────────────────────────────────────────────┤
│ cmd[16]                                                                          │
│ [ d ][ a ][ t ][ e ][\0 ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ] │
└──────────────────────────────────────────────────────────────────────────────────┘
High address
=> system("date")

1.2.2. Buffer Overflow Behavior

input: "AAAAAAAAAAAAAAAAls -al"
       |<----- 16 ----->|overflows -->|

[Stack]
Low address
┌──────────────────────────────────────────────────────────────────────────────────┐
│ buffer[16]                                                                       │
│ [ A ][ A ][ A ][ A ][ A ][ A ][ A ][ A ][ A ][ A ][ A ][ A ][ A ][ A ][ A ][ A ] │
├──────────────────────────────────────────────────────────────────────────────────┤
│ cmd[16]                                                                          │
│ [ l ][ s ][   ][ - ][ a ][ l ][\0 ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ][ ? ] │
└──────────────────────────────────────────────────────────────────────────────────┘
High address
=> system("ls -al")

1.2.3. How to Fix Safely

{
    ...
    // Copy at most 15 bytes (last 1 byte reserved for \0)
    strncpy(buffer, input, sizeof(buffer) - 1);
    buffer[sizeof(buffer) - 1] = '\0';  // always null-terminate
    ...
}
  • Use safe functions that limit input length (such as strncpy) to avoid exceeding buffer boundaries.
  • Also, because even length-limiting functions do not always guarantee NULL termination, you must explicitly set '\0'.

1.3. Heap Buffer Overflow

  • Buffer Overflow is a vulnerability caused by writing data beyond memory boundaries, and this behavior occurs in the same way not only on the Stack but also in dynamic memory regions such as the Heap.
  • In particular, Heap Overflow can corrupt not only adjacent data but also memory allocation management system metadata, which can lead not only to program crashes but also to arbitrary code execution.

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

// Heap layout (one malloc block for the whole struct):
//   [name: 8 bytes][role: 8 bytes]  <- contiguous inside the struct
typedef struct {
    char name[8];
    char role[8];
} Session;

static void check_access(const char *input) 
{
    Session *s = malloc(sizeof(Session));   // one heap allocation
    if (NULL == s ) {
           perror("malloc");
           exit(EXIT_FAILURE);
    }

    strcpy(s->role, "user");
    memcpy(s->name, input, strlen(input) + 1);

    printf("#'%s' access granted: '%s' privileges.\n", s->name, s->role);

    free(s);
}

int main(int argc, char *argv[]) 
{
    const char *input = (argc >= 2) ? argv[1] : "guest";

    check_access(input);

    return 0;
}
  • Because it copies into name[8] without validating input length, input longer than 8 bytes can overwrite adjacent role[8].
  • memcpy() directly specifies the copy size, but if input length is not validated, it can exceed the buffer size.
$ ./test2.out alice
#'alice' access granted: 'user' privileges.

$ ./test2.out operatoradmin
#'operatoradmin' access granted: 'admin' privileges.

1.3.1. Normal Operation

input: "alice"

[Heap]
┌───────────────────────────────────────────────────────────────────────────────────┐
│ name[8]                                  role[8]                                  │
│ [ a ][ l ][ i ][ c ][ e ][\0 ][ ? ][ ? ] [ u ][ s ][ e ][ r ][\0 ][ ? ][ ? ][ ? ] │
└───────────────────────────────────────────────────────────────────────────────────┘
=> role remains "user"

1.3.2. Buffer Overflow Behavior

input: "operatoradmin"
        |<- 8 ->|overflows -->|

[Heap]
┌───────────────────────────────────────────────────────────────────────────────────┐
│ name[8]                                  role[8]                                  │
│ [ o ][ p ][ e ][ r ][ a ][ t ][ o ][ r ] [ a ][ d ][ m ][ i ][ n ][\0 ][ ? ][ ? ] │
└───────────────────────────────────────────────────────────────────────────────────┘
=> role is overwritten to "admin"

1.3.3. How to Fix Safely

{
    ...
    size_t n = strnlen(input, sizeof(s->name) - 1);
    memcpy(s->name, input, n);
    s->name[n] = '\0';
    ...
}
  • Input validation and length limits must be applied so that buffer boundaries are not exceeded.
  • Also, bounds checks and guaranteed NULL termination are necessary to prevent memory corruption and vulnerabilities.

1.4. Compilation Options for Reproduction

  • Buffer Overflow is a representative security vulnerability in which data is written beyond memory boundaries, and on modern systems it is blocked by default through various security mechanisms.
  • Therefore, to reproduce vulnerability example source code in a practice environment, some security features must be intentionally disabled.
gcc -no-pie -fno-stack-protector -z execstack -o output input.c

1.4.1. ASLR (Address Space Layout Randomization)

  • Memory addresses are placed randomly each time the program runs.
  • If this feature is enabled, the addresses of code, stack, and libraries change each time, making it difficult for attackers to predict exact addresses.
  • The option used to bypass this is -no-pie. This option disables PIE (Position Independent Executable).

1.4.2. Stack Canary

  • Checks whether the stack has been tampered with before a function returns.
  • By default, a Canary value is inserted into the stack frame, and it is verified when the function ends. If the value changes, the program is forcibly terminated.
  • The option that disables this is -fno-stack-protector. With this option, even if a return address is overwritten via a buffer overflow, the program is not forcibly terminated and continues along normal flow.

1.4.3. NX (Non-eXecutable) Stack

  • Sets the stack region to be non-executable.
  • The option to disable this is -z execstack. This option changes the stack region to an executable state.