Not talking about caching (though there would be some decent memory savings due to that on general platforms like ChatGPT and tools like Codex). I am talking about large batch sizes, which are concurrent requests all accessing the same memory at the same time. The model is loaded once onto the GPU(s) and then many simultaneous requests can read that memory at the same time. When those requests are all processing their responses simultaneously, the energy per token drops off a cliff.

And yes, running a smaller model would generally take less power, but thats not really a fair comparison. Small models just wont give you the same results as larger ones. You need to compare it apples to apples. If you want to compare your local Qwen model running on your laptop, you compare those numbers to larger systems supplying that same qwen model to thousands of people. Just because we are comparing cloud services to local doesn’t automatically mean GPT 5.6 vs Qwen 3.6 27B. There are plenty of cloud AI providers running all sorts of models and sizes.

As for one of the articles I learned alot of this from originally, this is one I recommend going through. It really goes deep into the whole topic: https://arxiv.org/html/2601.22076v1

Very serious. Your personal amount of usage means nothing at all in this conversation. It is entirely about tokens per watt. The amount of energy the memory operations involve scale incredibly well when people are accessing the same object in memory simultaneously. Last I looked it was around a 10x difference for the same models efficiency.

If you want me to be your personal search engine you’ll need to wait a bit, im making dinner right now and would rather look for the articles on my desktop.

You have that backwards. The only thing you gain from running local models is privacy. It is not cheaper, it is not more efficient. You are actively hurting the environment MORE by using a local model on your own. LLM efficiency sky rockets the more users there are on a single loaded model.

IMO the only way we get to efficient LLM usage would be by having very efficient non frontier models running only for its local community to use, where you can have assurances on whether its power source is clean or not. That doesn’t help with the plagiarism aspect though

Thankfully I have managed to get everyone to switch from Messenger to Signal, atleast for talking to me haha. Two other family members being privacy oriented too really helped getting the rest of the family moved over.

Must be more popular in your country, in Canada I don’t know a single person on whatsapp

uhm, no? Literally none of that was considered AI. Even chatbots, people weren’t calling them AI until LLMs came around and were stuck in them. Lisp is a language USED for AI research, that doesn’t make it AI itself.

This bot is most definitely not even close to what people consider AI

It wasnt even LLMs until the public took the term and changed it lol. Unless you are calling every algorithim ever made AI these days, this isnt AI.

It isn’t AI, you can take a look at the source code for it from the url it provides. Obviously the detection needs some tweaking, but extra acronyms in the list doesn’t really hurt anything when the other half are relevant.

When did you last check? I believe jellyfin apps made it onto tizen OS a few months ago

My way of doing things is probably built on top of having a not very great memory and focus issues, so its actually difficult for me to operate without visual knowledge of an app being open still. That is also one reason it took me so long to make the switch to linux in the first place, GUIs are just so much easier than memorizing commands for me. More of a pattern learner than memorizing words or commands.

From my experience KDE barely has extensions, it mostly has half baked themes. Gnome extensions are in a whole different league

Yeah that is still a while away though. Gnome is pushing it now

It does certainly help, just not great.

And yes gnome does, because it is built around an extension ecosystem. You can’t multi monitor their default dash, but you certainly can use the dash to panel extension just fine on multiple monitors. Due to extensions, gnome is just way more customizable

Yeah im originally a windows boy lol. I grew up with it, and now I just cannot stand not being able to see everything on all monitors. It’s one of the major reasons I haven’t been able to get used to virtual workspaces / virtual desktops. Something about a window being hidden off into the void until I switch over to it just doesn’t work for me

My issue is when editing a panel there is no way to sync that change to all your other panels. I have five monitors and any new change (pin new app, hide something from tray, appearance change, trying new menu theme) needs to be manually done to every other panel too. It’s alot of effort when you don’t even know if you like the change yet.

That is my gripe haha. I don’t want independent bars, I have five monitors and every little change has to be done four more times and it’s infuriating lol. I want to be able to sync them like I can with dash to panel on gnome, but KDE handles it in such an odd way that it’s even hard to script cloning the task bar changes to other monitors.

The way gnome is built around extensions makes it very likely that if you have a pain point, someone smart enough to make an extension to fix it probably did too. I was able to make a very comfortable experience with it.

I also really miss ArcMenu :( I tried so many KDE menu themes and they are just so janky

Im curious what makes you say this?

I switched to KDE a couple months ago after updating to ubuntu 25.10 and finding out gnome is forcing wayland going forward, but most of my daily used applications dont work on wayland at all. KDE will atleast still support X11 for another couple years.

I really wish I could go back to gnome as I just really don’t like how little KDE has thought about multi monitors with its bottom panels/task managers.

I know how this sounds with where we are currently… but reddit’s /r/usenet has alot of good guides and round ups. That’s what I used back in the day, before the exodus that brought me here a couple years ago

Yes, quite a lot do. It is just a faster, more secure, and more reliable experience compared to torrenting

And that’s not even mentioning usenet… Paying for piracy in many cases can just be overall better