Local AI, Simply

What is a local LLM, in plain words

Sun, 09 Aug 2026 00:00:00 +0000

A “local” AI is one that runs on your own computer, not on someone else’s servers far away.

That one change carries most of the weight of this whole site. When the model lives on your machine, your words stay in the room, it costs nothing to run, and it works with no internet at all: on a train, on a plane, in a blackout.

The big cloud AIs are wonderful, and this is not against them. It is simply another option, one where the model runs on your own machine, and you get to see the quiet wonder of it working right there in front of you.

Do you even need a GPU to start?

Sun, 19 Jul 2026 00:00:00 +0000

No. You can start today on the computer you already have.

This is the question that stops most people before they begin. They read a forum thread about someone’s expensive graphics card and quietly decide local AI is not for them. It is a false wall. You do not need to buy anything to take the first step.

The one thing that actually matters: memory

A model has to fit in memory to run. That is the real limit, not whether you own a fancy graphics card.

The four gates: why a model runs on your machine, or doesn't

Sun, 02 Aug 2026 00:00:00 +0000

Most people run one test on a model: does it fit in memory? If the numbers look okay, they download it and expect it to work.

I used to think that too. Then I actually tried to run a string of big models on my own machine, and watched them fail in four completely different ways. A model does not simply “run” or “not run.” It has to pass four separate gates, and it can die at any one of them. Only the first is easy to check.

Blue light blocking glasses

Sun, 26 Jul 2026 00:00:00 +0000

We are going to spend most of our lives in front of screens. Phone screens, computer screens, TV screens, billboards. Screens, screens, screens.

And we each get one pair of eyes. They are priceless, and we ought to keep them healthy.

Blue light blocking glasses can help, at least a little. They take some of the harsh edge off a long day spent staring at a lit rectangle. I wear them, and honestly, they help. They really do.

Run your first model, step by step

Sun, 05 Jul 2026 00:00:00 +0000

Here is a small promise: on the computer you already own, you will soon be talking to an AI that runs entirely on your own machine. No account. No internet, once it is set up. Nothing you type goes anywhere.

How long it takes is mostly down to one thing: your internet. The setup is a few minutes of clicking. The model itself is about a gigabyte to download, which might be two minutes on fast fibre or twenty on a slow line. The work is quick; the waiting is just your download. So no false “ten minutes” promise here, it depends on your connection.

One machine, two engines, one pool of memory

Sun, 12 Jul 2026 00:00:00 +0000

I run two different programs to run models on the same machine: Ollama and vLLM. They do not know about each other. But they live in the same memory, and that is where the trouble starts.

This post is the honest story of getting two engines to share one box without knocking it over. If you ever run more than one way of loading models on a single computer, this might save you a bad afternoon.

Run a model on your phone

Sun, 14 Jun 2026 00:00:00 +0000

The same small promise as running one on a laptop, but smaller and stranger: the phone in your pocket can run its own AI. No account, no internet once it’s downloaded, nothing sent anywhere. In a few minutes, and a few taps.

If running a model on a computer felt surprising, running one on a phone feels almost silly. Let’s do it.

What you need

A reasonably recent phone (iPhone or Android).
About 2GB of free space for a small model.
Wifi for the one-time download. How long it takes is down to your wifi speed, not the app, so there’s no honest “few minutes” promise here.

Step 1 - Install a local AI app

We’ll use PocketPal AI. It is free, open-source, and does the whole thing on your phone with no account.

Short media content

Sun, 21 Jun 2026 00:00:00 +0000

Let me be honest: this one is hard, and I do not always win it.

You open YouTube to check one thing. You find it. And then, just a second, one more clip. Then another. And the scroll has you. It happens to me too. It is normal. These apps are built so that “just a second” always feels free.

What helps me is not willpower, because willpower runs out. It is a timer. App timers, the ones that tell you when your twenty minutes are gone, do the remembering for you. Some days I still push right past them. That is fine.

The words you keep seeing, in plain English

Sun, 24 May 2026 00:00:00 +0000

You have seen them. AWQ4. Tokens. Quantization. Words thrown around in forums and on model pages as if everyone already knows what they mean. They can be confusing, and honestly, a little intimidating.

So let’s calmly look at them, with the lens of simplicity. That is what this space is for.

None of these are as hard as they look. Here are the ones you will bump into most, grouped so they build on each other. You do not need to memorise anything - read once, come back when a word jumps out, and they slowly stop being scary.

The switcher I built: a seatbelt for one shared box

Sun, 28 Jun 2026 00:00:00 +0000

In a recent post I shared how my whole machine hard-rebooted because two model engines reached into the same memory at once. I ended it by mentioning a small script I wrote the day after - a seatbelt. This is me taking that script apart.

First, what it even is. vLLM - the heavier of my two model engines - only runs one model at a time. So when I want a different model, I run a small script that stops the one that is loaded and starts the next. That script is the switcher. Swapping models sounds harmless, but on a machine where everything shares one pool of memory, it is the most dangerous thing I do routinely - which is exactly why the script grew the shape it did.

The two models I actually run: one to think with, one to work with

Sun, 07 Jun 2026 00:00:00 +0000

For a long time I chased a single “best” model - the one that would do everything. I stopped when I noticed I was really using two, for two completely different jobs. One I talk to. One I hand tasks to. They are not ranked against each other. They are different tools.

If you take one thing from this post: you do not need one model to rule them all. You need to know which job you are doing.

The harness: the program that actually runs the model

Sun, 20 Sep 2026 00:00:00 +0000

People say “the model did it.” But a model, on its own, can’t do much of anything. It takes in some text and gives back the text most likely to follow - once - and then it forgets you were ever there. Whatever actually did the thing (kept the conversation, ran the command, tried again after a mistake) was a program wrapped around the model. That program is the harness, and it does far more of the work than most people give it credit for.

Gateway, MCP, evals: three jobs people keep mixing up

Sun, 30 Aug 2026 00:00:00 +0000

Three tools sit so close together in the agent world that people wire up one and quietly assume they’ve done the work of another. Then the agent still gives bad answers, and it’s not obvious why. The fix is boring: these are three separate jobs, and doing one well says nothing about the other two.

Here they are, plainly:

The gateway decides which model answers.
MCP decides what tools that model can reach.
Evals tell you whether the answer was any good.

Same neighborhood. Three different jobs. Let me take them one at a time.

Exercise and body movements

Sun, 17 May 2026 00:00:00 +0000

We sit to work, sit to rest, sit to scroll. The body was not built for a whole day of stillness, and it lets you know: a stiff neck, a tired back, restless sleep.

You do not need a gym or a plan. You need to move, a little, often.

Stand up once an hour. You will not remember on your own, so let something remind you: a timer on your desk, or a watch with a stopwatch that beeps or vibrates. When it goes off, stand, stretch, look out of a window.
Walk. A short walk with no phone is one of the calmest things you can do for a busy mind.
Move in a way you enjoy. The best exercise is the one you will actually do again tomorrow.

Screens keep the mind busy and the body still. A little movement puts the two back in balance.

One door to all your models: LiteLLM

Sun, 31 May 2026 00:00:00 +0000

Once you run more than one model, a small mess appears. This model lives in Ollama on one port. That one is served by vLLM on another. A third might be a cloud API. Every tool you own has to know where each model lives and how to talk to it. Change anything, and you are editing settings in five places.

LiteLLM fixes this by being a single front door. It puts one address, speaking one common language, in front of all your models - wherever they actually live.

Goose: an honest local agent

Sun, 10 May 2026 00:00:00 +0000

I wanted an agent - something I could hand a task to, that would read files, run commands, search the web, and get it done - but running on my models, with my work staying on my own machine. Goose, an open-source agent from Block, is what I settled on.

Let me be honest up front, because that is the whole point of this site: it is a reliable backup, not a marvel. It is not as smooth as the polished cloud agents. But it runs entirely on my hardware, on my models, and it gets real work done. For private, local, yours - it is more than good enough.

Odysseus: a private AI workspace on my own box

Sun, 03 May 2026 00:00:00 +0000

Everything else on this site has been plumbing - engines, memory, gateways, agents. Useful, but it all lives in a terminal. Odysseus is the friendly face I put on top of it: a self-hosted AI workspace, the local-first version of the ChatGPT or Claude window you already know - running on my own hardware, on my own data.

This is the piece that turns “I run models in a terminal” into “I have a private AI I can actually use, from my phone.”

A healthier relationship with the algorithms around us

Mon, 27 Jul 2026 00:00:00 +0000

We might not notice it, but algorithms surround us. The recommendations on YouTube, on our social apps, on the news feeds - algorithms, all of them. And most of them are trained for one thing: to keep us on the app as long as possible. User retention, maximum engagement. That’s what they’re built for.

So here’s the situation. Nearly every app in your day is quietly trying to take as much of your time and attention as it can. The question is how to live with that without getting sucked in.

Moving a whole stack to a new machine

Sun, 26 Apr 2026 00:00:00 +0000

When my old box had to be replaced, I did not just have files to copy. I had a whole running world to move: models, databases, background jobs, apps my day depends on. All of it living on one machine that was about to be gone.

Moving it taught me more than setting it up did. Here is what I would tell you before you move your own, so your day is calmer than my first one was.

Setting up a DGX Spark for local LLMs: what it's good at, and where it bit me

Mon, 10 Aug 2026 00:00:00 +0000

I run my whole local setup on a DGX Spark now - a small NVIDIA box that sits on a desk and holds models a normal graphics card could never fit. This is the honest account of setting it up: what it is genuinely good at, and the one edge that reached out and bit me.

If you take one thing from this post: the whole box lives on a single idea, unified memory, and that idea is both why it is special and how it will reboot itself if you get greedy.

Local AI vs cloud AI, and should we even be comparing?

Mon, 20 Jul 2026 00:00:00 +0000

There’s a big debate about whether local AI is better than cloud AI. I don’t think it should be a debate at all. They both have their pros and cons, and once you see that clearly, the “which is better” question stops making much sense.

If you take one thing from this: it isn’t local against cloud. It’s knowing which one the job in front of you actually needs.

What each one is good at

Local AI gives you privacy, sovereignty, autonomy, and the ability to experiment with your own personal data. It’s yours, on your machine, offline.

I build restraint open source AI systems: what they are, their limits, and their point

Mon, 03 Aug 2026 00:00:00 +0000

Let’s start with the word “restraint”.

If I was a heavyweight wrestler, and a lightweight challenged me to a fight, and partway through I could see he was already tired and about to lose - I’m not tired, I could choose to keep beating him, show off, humiliate him. But I choose not to. That act of choosing not to, when I could, is restraint.

That is the exact idea I built into the systems I make.

A day off the screen

Tue, 18 Aug 2026 00:00:00 +0000

A day off the screen in a week isn’t a bad idea really.

This might be difficult for many, which is understandable. However, once in a while, a total minimisation of screen time is helpful. We are humans and we need to have some rest. Rest should be cherished whenever the opportunity arrives. Rest from the digital noise, and reconnecting with our fellow humans and nature.

A day of rest is a day I find myself actually being able to cut out most distractions. I sit in silence for some moment, I relax, trying to breathe, and if possible, walk in a calm park.

Does the machine exist to serve humanity, or the other way round?

Sun, 12 Jul 2026 00:00:00 +0000

Does the machine exist to serve humanity, or is it the other way round?

I was watching one of my favourite animations, and one episode has stayed with me. There’s an inventor, and a war is coming - not a local war, an invasion, ships pouring in from different galaxies, armed with guns and cannons, coming to destroy everything. The inventor is the only one who can do anything about it. So he builds machines. Robots, built ready to sacrifice themselves.

About Me

Mon, 01 Jan 0001 00:00:00 +0000

I’m Femi, a data and AI engineer.

I like understanding what happens underneath things - memory, models, serving, the plumbing that makes a system work. I’ve spent years on real systems, and lately most of my curiosity is about bringing AI closer to home.

I run models on my own hardware. Local AI isn’t always better than the cloud, and I use both. What I care about is that people have the choice, and more ownership over the technology they use - their data, the models they run, the freedom to experiment without a powerful machine or a cloud account.

Search

Mon, 01 Jan 0001 00:00:00 +0000