D&S-05: Cloud, Private Cloud, or Your Own Server?

Part 5 of the Data & Security series. In Part 4 we looked at what regulated firms must prove. Here is the architecture question underneath everything: where should the AI actually live?

Every AI conversation eventually lands on the same fork in the road. Do we use the big cloud services, keep things in a private setup of our own, or run models on a server in our own building?

There is no universal right answer. There is a right answer per pile of data. Let me walk the three homes honestly, including the costs nobody puts in the brochure.

Home one: the public cloud services

This is ChatGPT, Copilot, Claude and Gemini as most companies meet them: you sign in, the model lives in the vendor's data centers, your words travel there and the answers travel back.

The strengths are real. You start today. The capability is the best available anywhere. And whatever you think of large vendors, their security teams are bigger and more practiced than anything a mid-size company will ever hire. Renting their machinery means renting their guards too.

The price of admission is trust, managed by contract. Your data leaves the building, so everything from the earlier parts of this series applies: business tier, no-training terms in writing, retention you control, and a vendor you have actually vetted. For public material, internal drafts and most everyday work, this home is not a compromise. It is the sensible default.

Home two: a private cloud of your own

The middle option rents the same kind of machinery but inside a fence with your name on it: your own cloud tenant, dedicated capacity, data pinned to a region you choose. The model may be the vendor's, but the yard is yours: your access rules, your logs, your retention, your keys in some setups.

This is where many regulated firms land, and for good reason. It answers the auditor's hardest questions (where does the data sit, who else is in the building) while keeping the heavy maintenance on the vendor. The cost is money and some setup discipline. The gain is control you can demonstrate, which in regulated work is half the product.

Home three: your own server room

The third option runs models on hardware you own, inside your walls or your own locked rack. Nothing leaves. For a narrow set of problems (the most sensitive documents, air-gapped environments, work where a regulator or a client demands it), this is the only acceptable answer, and it is a legitimate one.

Now the honest costs. The models you can run well on your own hardware are smaller than the giants in the cloud. For many document jobs (search, summarize, extract, classify) they are genuinely enough, and for open-ended reasoning they are not, yet. Hardware is the visible expense. The hidden one is the human: someone patches it, updates it, watches it, and answers when it is down at 7 in the morning. A local system nobody maintains is not the safe option. It is the dusty option, and dust is its own risk.

The levels in between

Most companies do not need to pick one home for everything, because data is not one thing. A practical ladder I use with clients: public material goes to the public cloud on business accounts. Client data goes to the private setup under a signed agreement. The small pile of truly sensitive material gets redacted before it travels, or stays on the local model. Each step up the ladder costs more and protects more, and the art is matching the step to the pile instead of building a vault for the phone book.

The decision rule

Ask of each pile of information: if this appeared on a stranger's screen tomorrow, what happens? If the answer is “nothing much,” the cloud on a business account is fine. If the answer is “a bad week,” go private and get the paperwork. If the answer is “a regulator, a lawsuit, or the end of a client relationship,” keep it home or redact it until the answer changes.

The building matters less than the locks, the logbook, and whether a named person owns them. Next in the series: the leak that needs no hacker at all, because it is already sitting in your team's browser history.