AirLLM: A 70B Model on a 12 GB GPU, and What That Really Means
AirLLM runs models far larger than your VRAM by streaming them layer by layer from disk. It really does put a 70B model on a consumer GPU. A realistic look at how it works, how to …
AirLLM runs models far larger than your VRAM by streaming them layer by layer from disk. It really does put a 70B model on a consumer GPU. A realistic look at how it works, how to …
How to build Model Context Protocol servers that actually work in production. Deep-dive in auto-generation from API routes, permission models, and why 13 small tools beat one smart …
Deep dive into creating a production-ready research agent with LangGraph, streaming support, and OpenWebUI integration featuring collapsed reasoning sections
Prerequisites Windows with WSL2 installed NVIDIA RTX 5070 GPU NVIDIA drivers installed on Windows host Issue Overview RTX 50-series GPUs use CUDA compute capability sm_120, which …
Complete installation guide for Fish Speech (OpenAudio) with RTX 50-series GPU support on Windows WSL2. Prerequisites Windows with WSL2 installed NVIDIA RTX 5070 GPU NVIDIA drivers …
This project showcases a sophisticated approach to making personal fitness data searchable and meaningful using modern AI tools. The system combines Strava activity data, vector …
What began as my technical experiment evolved into something far more meaningful: a system that doesn’t just store and retrieve information, but actively helps me understand …