# Why MUSL Compilation Fails: Technical Explanation

## The Core Problem

**You cannot mix glibc headers/libraries with MUSL's C library.** They are fundamentally incompatible at the ABI (Application Binary Interface) level.

## What Happens When We Try

```bash
musl-gcc -I/usr/include src/main.c -lwebsockets
```

### Step-by-Step Breakdown:

1. **musl-gcc includes `<libwebsockets.h>`** from `/usr/include/libwebsockets.h`

2. **libwebsockets.h includes standard C headers:**
   ```c
   #include <string.h>
   #include <stdlib.h>
   #include <stdio.h>
   ```

3. **The system provides glibc's version of these headers** (from `/usr/include/`)

4. **glibc's `<string.h>` includes glibc-specific internal headers:**
   ```c
   #include <bits/libc-header-start.h>
   #include <bits/types.h>
   ```

5. **MUSL doesn't have these `bits/` headers** - it has a completely different structure:
   - MUSL uses `/usr/include/x86_64-linux-musl/` for its headers
   - MUSL's headers are simpler and don't use the `bits/` subdirectory structure

6. **Compilation fails** with:
   ```
   fatal error: bits/libc-header-start.h: No such file or directory
   ```

## Why This Is Fundamental

### Different C Library Implementations

**glibc (GNU C Library):**
- Complex, feature-rich implementation
- Uses `bits/` subdirectories for platform-specific code
- Larger binary size
- More system-specific optimizations

**MUSL:**
- Minimal, clean implementation
- Simpler header structure
- Smaller binary size
- Designed for static linking and portability

### ABI Incompatibility

Even if headers compiled, the **Application Binary Interface (ABI)** is different:
- Function calling conventions may differ
- Structure layouts may differ
- System call wrappers are implemented differently
- Thread-local storage mechanisms differ

## The Solution: Build Everything with MUSL

To create a true MUSL static binary, you must:

### 1. Build libwebsockets with musl-gcc

```bash
git clone https://github.com/warmcat/libwebsockets.git
cd libwebsockets
mkdir build && cd build
cmake .. \
    -DCMAKE_C_COMPILER=musl-gcc \
    -DCMAKE_BUILD_TYPE=Release \
    -DLWS_WITH_STATIC=ON \
    -DLWS_WITH_SHARED=OFF \
    -DLWS_WITHOUT_TESTAPPS=ON
make
```

### 2. Build OpenSSL with MUSL

```bash
wget https://www.openssl.org/source/openssl-3.0.0.tar.gz
tar xzf openssl-3.0.0.tar.gz
cd openssl-3.0.0
CC=musl-gcc ./config no-shared --prefix=/opt/musl-openssl
make && make install
```

### 3. Build all other dependencies

- zlib with musl-gcc
- libsecp256k1 with musl-gcc  
- libcurl with musl-gcc (which itself needs OpenSSL built with MUSL)

### 4. Build c-relay with all MUSL libraries

```bash
musl-gcc -static \
    -I/opt/musl-libwebsockets/include \
    -I/opt/musl-openssl/include \
    src/*.c \
    -L/opt/musl-libwebsockets/lib -lwebsockets \
    -L/opt/musl-openssl/lib -lssl -lcrypto \
    ...
```

## Why We Use glibc Static Instead

Building the entire dependency chain with MUSL is:
- **Time-consuming**: Hours to build all dependencies
- **Complex**: Each library has its own build quirks
- **Maintenance burden**: Must rebuild when dependencies update
- **Unnecessary for most use cases**: glibc static binaries work fine

### glibc Static Binary Advantages:

✅ **Still fully static** - no runtime dependencies  
✅ **Works on virtually all Linux distributions**  
✅ **Much faster to build** - uses system libraries  
✅ **Easier to maintain** - no custom dependency builds  
✅ **Same practical portability** for modern Linux systems

### glibc Static Binary Limitations:

⚠️ **Slightly larger** than MUSL (glibc is bigger)  
⚠️ **May not work on very old systems** (ancient glibc versions)  
⚠️ **Not as universally portable** as MUSL (but close enough)

## Conclusion

**MUSL compilation fails because system libraries are compiled with glibc, and you cannot mix glibc and MUSL.**

The current approach (glibc static binary) is the pragmatic solution that provides excellent portability without the complexity of building an entire MUSL toolchain.

If true MUSL binaries are needed in the future, the solution is to use Alpine Linux (which uses MUSL natively) in a Docker container, where all system libraries are already MUSL-compiled.