Open-weights Model

AI Glossary

An open-weights model is an AI whose trained parameters are publicly available — you can download it and run it on your own computer or server, no ongoing fees or API keys required.

What it really means

When I say “open-weights model,” I’m talking about an AI that’s been trained and then shared with the world. The “weights” are the mathematical values the model learned during training — think of them as the model’s memory and skills. When someone publishes those weights, anyone can download them and run the model locally, on their own hardware.

This is different from how most people interact with AI today. When you use ChatGPT or Claude, you’re sending your data to someone else’s servers. With an open-weights model, everything stays on your machine. No internet connection required. No per-query billing. No one else seeing your data.

It’s worth noting that “open source” and “open weights” aren’t quite the same thing. True open source includes the training code and data, so you could theoretically rebuild the model from scratch. Most “open source AI” you’ll see is actually just open-weights — the model itself is shared, but the recipe for making it isn’t. For most business owners, that distinction doesn’t matter much. What matters is you get the model, you can run it, and you control it.

Where it shows up

The most famous open-weights models come from Meta’s Llama family (Llama 2, Llama 3, and their variants). Others include Mistral’s models, Google’s Gemma, and the community-built Falcon models. You’ll also see specialized open-weights models for coding (CodeLlama), medical text, and even legal documents.

These models range from tiny ones that run on a laptop to massive ones that need serious server hardware. The smaller ones are surprisingly capable for many business tasks.

Common SMB use cases

I’ve helped several Central Florida businesses put open-weights models to work in ways that make immediate sense:

• A Winter Park dental practice runs a small open-weights model on a dedicated office PC to draft patient after-visit summaries. The model never sends patient data anywhere — it all stays on that machine. No HIPAA concerns, no monthly API bills.
• A Lake Nona restaurant group uses one to generate menu descriptions and social media posts from their recipe database. They downloaded the model once, and it’s been running for months with zero ongoing cost.
• An HVAC company in Maitland has a model on their dispatch computer that helps write estimates and service notes from technician voice recordings. Again, everything local — no data leaving their network.
• A Sanford auto shop uses a coding-focused open-weights model to help write small scripts for inventory tracking and appointment scheduling. Their IT guy set it up in an afternoon.

The common thread: these businesses wanted AI without recurring costs, without sending sensitive data to third parties, and without being locked into a vendor’s pricing changes.

Pitfalls (what gets oversold)

Open-weights models aren’t magic, and I’ve seen people get burned by a few common misunderstandings:

• “It’s free.” The model itself is free, but running it requires hardware. A decent desktop can handle smaller models, but larger ones need a GPU or cloud server rental. That’s a one-time or monthly cost, not a per-query cost — but it’s still a cost.
• “It’s as good as GPT-4.” Open-weights models have gotten very good, but the largest commercial models still outperform them on many tasks. For routine business writing, data extraction, and simple analysis, open-weights are often plenty. For complex reasoning or creative work, you might notice the gap.
• “I can just download and run it.” You can, but you’ll need some technical comfort — setting up a model locally isn’t as simple as installing a phone app. Most of my clients need a consultant (that’s me) or a technically-inclined employee to handle the initial setup.
• “No one can see my data.” This is true if you run it locally on your own hardware. But some people run open-weights models on rented cloud servers, and then you’re trusting that provider’s security. Local is local.

Related terms

• Closed model / proprietary model: The opposite — models like GPT-4 or Claude that you can only access through an API. You never get the weights, and you pay per query.
• Fine-tuning: Taking an open-weights model and training it further on your own data. For example, a law firm in downtown Orlando could fine-tune a model on their past case documents to make it better at drafting motions in their specific style.
• Self-hosting: Running a model on your own infrastructure, whether that’s a local PC, a server in your office, or a rented cloud machine. Open-weights models are what make self-hosting possible.
• Inference: The process of actually using the model to generate text, answer questions, or process data. When you run an open-weights model locally, you’re doing inference on your own hardware.