Flash Attention Gibberish
Problem: Response becomes gibberish with context >8K tokens
Solution
Enable flash attention (-fa) AND fully offload to GPU (-ngl 80)
Source: llama.cpp Discussions
Ollama Context Window Confusion
Problem: Model seems to forget earlier conversation
Solution
Default context is only 2048-4096 tokens. Increase with num_ctx parameter in your Modelfile or runtime.
Source: Ollama Context Documentation
RAG Retrieval Failures
Problem: Retrieved chunks are irrelevant
Solutions
- Use hybrid search (BM25 + dense)
- Add reranking step
- Reduce chunk size (512 tokens)
- Increase chunk overlap (10-20%)
Source: Anthropic Contextual Retrieval
vLLM OOM Errors
Problem: Out of memory during serving
Solutions
- Start at
gpu_memory_utilization=0.98 - Decrease by 1% until stable
- Reduce
max_model_len - Enable chunked prefill (V1 default)
Source: vLLM Optimization Guide
LoRA Overfitting
Problem: Fine-tuned model performs worse on general tasks
Solutions
- Reduce rank (r) - try 8 or 16
- Increase training diversity
- Use smaller learning rate
- Early stopping with validation
Source: Unsloth LoRA Guide
Cursor AI Ignoring Instructions
Problem: AI modifies code you didn't ask to change
Solution
Add explicit constraint: "(Do not change anything I did not ask for)"
Source: Cursor Tips