services.tabbyapi.settings.model.cache_mode
NixOS option
Cache mode for VRAM savings. ExLlamaV2: FP16, Q8, Q6, Q4. ExLlamaV3: specific pair string (e.g., ‘8,8’).
type: stringDefault
declared in: nixos/modules/services/web-apps/tabbyapi.nixView source on NixOS/nixpkgs →"FP16"