140 lines
4.1 KiB
Markdown
140 lines
4.1 KiB
Markdown
# Why MUSL Compilation Fails: Technical Explanation
|
|
|
|
## The Core Problem
|
|
|
|
**You cannot mix glibc headers/libraries with MUSL's C library.** They are fundamentally incompatible at the ABI (Application Binary Interface) level.
|
|
|
|
## What Happens When We Try
|
|
|
|
```bash
|
|
musl-gcc -I/usr/include src/main.c -lwebsockets
|
|
```
|
|
|
|
### Step-by-Step Breakdown:
|
|
|
|
1. **musl-gcc includes `<libwebsockets.h>`** from `/usr/include/libwebsockets.h`
|
|
|
|
2. **libwebsockets.h includes standard C headers:**
|
|
```c
|
|
#include <string.h>
|
|
#include <stdlib.h>
|
|
#include <stdio.h>
|
|
```
|
|
|
|
3. **The system provides glibc's version of these headers** (from `/usr/include/`)
|
|
|
|
4. **glibc's `<string.h>` includes glibc-specific internal headers:**
|
|
```c
|
|
#include <bits/libc-header-start.h>
|
|
#include <bits/types.h>
|
|
```
|
|
|
|
5. **MUSL doesn't have these `bits/` headers** - it has a completely different structure:
|
|
- MUSL uses `/usr/include/x86_64-linux-musl/` for its headers
|
|
- MUSL's headers are simpler and don't use the `bits/` subdirectory structure
|
|
|
|
6. **Compilation fails** with:
|
|
```
|
|
fatal error: bits/libc-header-start.h: No such file or directory
|
|
```
|
|
|
|
## Why This Is Fundamental
|
|
|
|
### Different C Library Implementations
|
|
|
|
**glibc (GNU C Library):**
|
|
- Complex, feature-rich implementation
|
|
- Uses `bits/` subdirectories for platform-specific code
|
|
- Larger binary size
|
|
- More system-specific optimizations
|
|
|
|
**MUSL:**
|
|
- Minimal, clean implementation
|
|
- Simpler header structure
|
|
- Smaller binary size
|
|
- Designed for static linking and portability
|
|
|
|
### ABI Incompatibility
|
|
|
|
Even if headers compiled, the **Application Binary Interface (ABI)** is different:
|
|
- Function calling conventions may differ
|
|
- Structure layouts may differ
|
|
- System call wrappers are implemented differently
|
|
- Thread-local storage mechanisms differ
|
|
|
|
## The Solution: Build Everything with MUSL
|
|
|
|
To create a true MUSL static binary, you must:
|
|
|
|
### 1. Build libwebsockets with musl-gcc
|
|
|
|
```bash
|
|
git clone https://github.com/warmcat/libwebsockets.git
|
|
cd libwebsockets
|
|
mkdir build && cd build
|
|
cmake .. \
|
|
-DCMAKE_C_COMPILER=musl-gcc \
|
|
-DCMAKE_BUILD_TYPE=Release \
|
|
-DLWS_WITH_STATIC=ON \
|
|
-DLWS_WITH_SHARED=OFF \
|
|
-DLWS_WITHOUT_TESTAPPS=ON
|
|
make
|
|
```
|
|
|
|
### 2. Build OpenSSL with MUSL
|
|
|
|
```bash
|
|
wget https://www.openssl.org/source/openssl-3.0.0.tar.gz
|
|
tar xzf openssl-3.0.0.tar.gz
|
|
cd openssl-3.0.0
|
|
CC=musl-gcc ./config no-shared --prefix=/opt/musl-openssl
|
|
make && make install
|
|
```
|
|
|
|
### 3. Build all other dependencies
|
|
|
|
- zlib with musl-gcc
|
|
- libsecp256k1 with musl-gcc
|
|
- libcurl with musl-gcc (which itself needs OpenSSL built with MUSL)
|
|
|
|
### 4. Build c-relay with all MUSL libraries
|
|
|
|
```bash
|
|
musl-gcc -static \
|
|
-I/opt/musl-libwebsockets/include \
|
|
-I/opt/musl-openssl/include \
|
|
src/*.c \
|
|
-L/opt/musl-libwebsockets/lib -lwebsockets \
|
|
-L/opt/musl-openssl/lib -lssl -lcrypto \
|
|
...
|
|
```
|
|
|
|
## Why We Use glibc Static Instead
|
|
|
|
Building the entire dependency chain with MUSL is:
|
|
- **Time-consuming**: Hours to build all dependencies
|
|
- **Complex**: Each library has its own build quirks
|
|
- **Maintenance burden**: Must rebuild when dependencies update
|
|
- **Unnecessary for most use cases**: glibc static binaries work fine
|
|
|
|
### glibc Static Binary Advantages:
|
|
|
|
✅ **Still fully static** - no runtime dependencies
|
|
✅ **Works on virtually all Linux distributions**
|
|
✅ **Much faster to build** - uses system libraries
|
|
✅ **Easier to maintain** - no custom dependency builds
|
|
✅ **Same practical portability** for modern Linux systems
|
|
|
|
### glibc Static Binary Limitations:
|
|
|
|
⚠️ **Slightly larger** than MUSL (glibc is bigger)
|
|
⚠️ **May not work on very old systems** (ancient glibc versions)
|
|
⚠️ **Not as universally portable** as MUSL (but close enough)
|
|
|
|
## Conclusion
|
|
|
|
**MUSL compilation fails because system libraries are compiled with glibc, and you cannot mix glibc and MUSL.**
|
|
|
|
The current approach (glibc static binary) is the pragmatic solution that provides excellent portability without the complexity of building an entire MUSL toolchain.
|
|
|
|
If true MUSL binaries are needed in the future, the solution is to use Alpine Linux (which uses MUSL natively) in a Docker container, where all system libraries are already MUSL-compiled. |