Linux Platform Support

GPU detection, installation methods, and troubleshooting for Linux

Linux has full support for all GPU vendors with multiple detection methods and fallbacks.

GPU Detection Methods

NVIDIA GPUs

llmfit detects NVIDIA GPUs via two methods:

1. nvidia-smi (Primary)

# Multi-GPU query with addressing mode (unified memory detection)
nvidia-smi --query-gpu=addressing_mode,memory.total,name --format=csv,noheader,nounits

# Fallback query (older nvidia-smi versions)
nvidia-smi --query-gpu=memory.total,name --format=csv,noheader,nounits

Unified Memory Detection:

• NVIDIA Grace/DGX Spark (GB10, GB20): addressing_mode returns “ATS” (Address Translation Services)
• When VRAM is unavailable, falls back to /proc/meminfo for total RAM

Multi-GPU Support:

• Aggregates same-model GPUs (e.g., 2x RTX 4090 → count=2, per-card VRAM=24GB)
• Total VRAM = per-card VRAM × count (for tensor splitting across cards)

2. sysfs Fallback (Containers/Toolbx)

When nvidia-smi is unavailable (common in containers):

# Check vendor ID in /sys/class/drm/card*/device/
cat /sys/class/drm/card0/device/vendor
# 0x10de = NVIDIA

# Read VRAM
cat /sys/class/drm/card0/device/mem_info_vram_total

GPU name resolution:

• lspci -nn queries PCI slot from sysfs uevent file
• Matches GPU model from lspci output (e.g., “GeForce RTX 2060”)
• Fallback to flatpak-spawn --host lspci in Flatpak environments

Backend detection:

• Checks DRIVER= in sysfs uevent file
• nvidia driver → CUDA backend
• Other drivers (nouveau, etc.) → Vulkan backend

AMD GPUs

llmfit detects AMD GPUs via three methods:

1. rocm-smi (ROCm installed)

# VRAM detection
rocm-smi --showmeminfo vram
# Parses "Total Memory (B): 8589934592"

# GPU name
rocm-smi --showproductname
# Looks for "Card Series" or "Card Model" lines

Multi-GPU Support:

• Tracks per-GPU VRAM from multiple “Total” lines
• Backend: ROCm

2. sysfs (No ROCm)

# Check vendor ID
cat /sys/class/drm/card0/device/vendor
# 0x1002 = AMD

# Read VRAM
cat /sys/class/drm/card0/device/mem_info_vram_total

GPU name resolution:

• lspci -nn for AMD/ATI VGA/3D controllers
• Backend: Vulkan (no ROCm)

3. Unified Memory APUs

Ryzen AI series APUs (Strix Halo, Strix Point) share system RAM:

# Detection: CPU name contains "Ryzen AI"
lscpu | grep "Model name"
# AMD Ryzen AI MAX+ 395 w/ Radeon 8060S
# AMD Ryzen AI 9 HX 370 w/ Radeon 890M

Behavior:

• VRAM = total system RAM (unified pool)
• Backend: Vulkan
• unified_memory flag set to true

Intel Arc GPUs

llmfit detects Intel Arc via sysfs and lspci:

Discrete GPUs (Arc A770, A370M)

# Check vendor ID
cat /sys/class/drm/card0/device/vendor
# 0x8086 = Intel

# Read VRAM
cat /sys/class/drm/card0/device/mem_info_vram_total

Integrated GPUs (Arc Graphics in Meteor Lake)

# lspci detection
lspci -nn | grep -i "intel.*arc"
# 00:02.0 VGA compatible controller [0300]: Intel Corporation Arc Graphics [8086:7d55]

Behavior:

• Discrete: Reports exact VRAM
• Integrated: Reports 0 VRAM (shares system memory)
• Backend: SYCL (Intel oneAPI)

Ascend NPUs

llmfit detects Huawei Ascend NPUs via npu-smi:

# List NPU IDs
npu-smi info -l
# NPU ID : 0
# NPU ID : 1

# Per-NPU memory info
npu-smi info -t memory -i 0
# HBM Capacity(MB) : 65536

Behavior:

• Each NPU tracked separately (count=1 per NPU)
• VRAM = HBM capacity in MB → GB
• Backend: Ascend

Installation Methods

Homebrew (Recommended)

brew install llmfit

Quick Install Script

# Install to /usr/local/bin (requires sudo)
curl -fsSL https://llmfit.axjns.dev/install.sh | sh

# Install to ~/.local/bin (no sudo)
curl -fsSL https://llmfit.axjns.dev/install.sh | sh -s -- --local

From Source

git clone https://github.com/AlexsJones/llmfit.git
cd llmfit
cargo build --release
sudo cp target/release/llmfit /usr/local/bin/

Troubleshooting

GPU Not Detected

NVIDIA:

1. Check if nvidia-smi works:

   nvidia-smi
   

2. If nvidia-smi fails, check sysfs:

   ls /sys/class/drm/card*/device/vendor
   cat /sys/class/drm/card0/device/vendor  # Should be 0x10de
   

3. Install nvidia-utils package:

   # Debian/Ubuntu
   sudo apt install nvidia-utils
   
   # Fedora
   sudo dnf install nvidia-driver
   
   # Arch
   sudo pacman -S nvidia-utils
   

4. Manual override:

   llmfit --memory=24G system
   

AMD:

1. Check if rocm-smi works:

   rocm-smi --showmeminfo vram
   

2. Install ROCm:

   # Ubuntu
   wget https://repo.radeon.com/amdgpu-install/latest/ubuntu/jammy/amdgpu-install_*.deb
   sudo dpkg -i amdgpu-install_*.deb
   sudo amdgpu-install --usecase=rocm
   
   # Check /sys/class/drm for AMD cards
   grep -l 0x1002 /sys/class/drm/card*/device/vendor
   

3. Sysfs fallback (no ROCm):

   cat /sys/class/drm/card0/device/mem_info_vram_total
   

Intel Arc:

1. Check sysfs vendor ID:

   grep -l 0x8086 /sys/class/drm/card*/device/vendor
   

2. Check lspci:

   lspci | grep -i "intel.*arc"
   

3. Install Intel compute runtime:

   # Ubuntu
   sudo apt install intel-opencl-icd
   

VRAM Shows 0 or Wrong Value

Broken nvidia-smi:

# Check if driver is loaded
lsmod | grep nvidia

# Reinstall driver
sudo apt reinstall nvidia-driver-535  # or your version

VM/Passthrough setups:

VRAM reporting often fails in VMs. Use manual override:

llmfit --memory=16G

Container/Toolbx:

Sysfs fallback should work. If not, check mount points:

ls /sys/class/drm/card0/device/
# Should see vendor, mem_info_vram_total, etc.

WSL-Specific Issues

llmfit detects WSL via environment variables and /proc/version:

echo $WSL_DISTRO_NAME
cat /proc/version | grep -i microsoft

NVIDIA in WSL:

• Install NVIDIA CUDA on WSL2: https://docs.nvidia.com/cuda/wsl-user-guide/
• nvidia-smi should work once CUDA drivers are installed

AMD in WSL:

• Limited support; ROCm not officially supported in WSL
• Use CPU-only mode or run llmfit natively on Windows

Multi-GPU Detection Issues

Same-model cards not grouped:

Check nvidia-smi output format:

nvidia-smi --query-gpu=memory.total,name --format=csv,noheader,nounits

Expected output:

24564, NVIDIA GeForce RTX 4090
24564, NVIDIA GeForce RTX 4090

llmfit groups by exact name match.

Different-model cards:

llmfit uses the GPU with the most VRAM as primary:

llmfit system
# GPU 1: NVIDIA GeForce RTX 4090 (24.00 GB VRAM, CUDA)
# GPU 2: NVIDIA GeForce RTX 3060 (12.00 GB VRAM, CUDA)

Next Steps

• Platform Overview
• macOS Platform Guide
• Windows Platform Guide