• MalReynolds@slrpnk.net
    link
    fedilink
    English
    arrow-up
    9
    ·
    edit-2
    1 day ago

    I love fully local stuff, but the LLM part seems very expensive. Even for such a simple thing as managing our lights and music.

    So, the simple things should just be voice command (not recognition, limited vocabulary), <wakeword> kitchen light on, what’s todays weather (assuming you automatically download it and have it there, needs TTS like whisper). That can run on a recent potato. Somewhat more complex things need full voice recognition e.g. Play me x songname by y artist. Laptop CPU should handle that fine. You’ll only need a full LLM for more general inquiries, e.g.<wakeword> What mess has Trump made today?, which will need web access seeing as small LLMs don’t have a lot of world knowledge (although you can self host Wikipedia and point it at that).

    If you have a desktop computer and a GPU with 16+GB VRAM (or even smaller using RAM and a MoE LLM) that’ll do the job with reasonable smarts e.g. Qwen 3.6 35BA3B, Gemma 4 12B or 26BA4B. You may have to wait a minute or two. Mine pulls around 55W at idle (7800XT) and does double duty as a NAS and gaming rig, or you can have it sleep most of the time and let the laptop wake it up when necessary.

    • smiletolerantly@awful.systems
      link
      fedilink
      English
      arrow-up
      8
      ·
      1 day ago

      I actually have things like “play x by y” functioning really well without a LLM.

      Have a custom service that exports all song/album/artist names from MusicAssistant, does some simple cleanup, and places the list where HomeAssistant expects it for custom voice intents. Then this: https://github.com/charludo/hass-closest-intent is enough that imperfect STT can still easily be matched to those song/artist/… names.

    • Convict45@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      1 day ago

      <wake word> “how much is using local LLMs for trivial things raising world wide energy usage and causing a RAM shortage?”