Short answer. Deploying a model on your servers is not a badge of seriousness: it is the answer to a written constraint, where documents are allowed to go. When it exists, bank, government, health, isolated industrial site, an open model is installed on a machine of yours, behind the same orchestration as for an API: instructions, document retrieval, schemas, human fallback, log. The model is chosen on your evaluation set of sixty real cases, in Arabic, French and darija, not on a leaderboard: size does not tell language. The machine is sized on weight memory, about 2 GB per billion parameters at 16 bits and 0.6 GB at 4 bits, plus context. An inference server batches requests, caches the prefix and serves quantised weights; that is what lets one card serve a team. Cost becomes fixed, a machine and its operation, instead of per request. And when the constraint does not exist, an API in Europe arrives faster and answers better. Ten weeks, and no card bought before the fourth.
This guide is for the IT department of a bank, insurer or government body that was told AI “is in the cloud”, the clinic and the manufacturer whose data does not leave, and the SaaS vendor whose customer requires everything to stay in Algeria. It does not repeat which functions to build, described in the guide on AI in the app, nor where to host the app itself, described in the hosting guide; it describes what is set up between the orchestration and the graphics card, and extends our page on AI functions.
What the figures say
A model’s size does not tell its language. The model card of Llama 3.3 with 70 billion parameters, released by Meta on 6 December 2024 with a 128,000-token context window, lists eight officially supported languages: English, German, French, Italian, Portuguese, Hindi, Spanish and Thai; Arabic is not among them, and the card says developers may fine-tune it for others. Conversely the Jais 3 family, announced on 15 October 2026 by MBZUAI’s Institute of Foundation Models and Inception42, offers six open models from 900 million to 375 billion parameters built for Modern Standard Arabic and dialects “to North African Arabic”; we cite the press announcement, and the licence is read on the repository. What this imposes: the model is chosen on your documents and messages, in Arabic, French and darija, not on parameter count.
One machine serves several people provided it has the right server. The documentation of vLLM, the open inference library we use, lists continuous batching of incoming requests, prefix caching, attention memory management, quantised weights at 8 and 4 bits in several formats, an API compatible with the vendors’, support for NVIDIA and AMD cards, and parallelism across several cards. What this imposes: moving from an API to an on-site model does not rewrite the app, since the orchestration speaks the same API; and the question is not “how many tokens per second” but “how many people at once, with what wait”.
Memory is computed before buying. A weight stored at 16 bits takes two bytes: a model therefore takes about 2 GB per billion parameters, and about 0.6 GB at 4 bits; on top comes context memory, which grows with document length and the number of simultaneous requests. These are orders of magnitude, not a spec sheet: an 8-billion model fits at 4 bits on a 24 GB card with room to spare, a 32-billion model just fits, a 70-billion model needs 48 GB or several cards. What we did not find and do not quote: a public price for cards in Algeria, which depends on import and changes; that is why the model is first chosen on machines rented by the hour, and the machine bought afterwards.
One request, two routes
Private deployment does not replace the API: it adds a route, and the orchestration chooses, request by request.
Inside your walls: the app calls the orchestration, which prepares the request, retrieves only the useful documents and calls the model served on your machine; nothing leaves the network, not the document, not the question, not the answer, not the log. It is the route for anything containing a customer document, a medical file, an ID. Outside your walls: for what contains none, general drafting, a translation, a question on a public text, the orchestration may call a more capable model hosted in Europe, with only the necessary data and a contract in your name. The rule is written in the first week, per request type, and enforced by the orchestration, not left to the user’s judgement.
What it changes: documents go nowhere; cost becomes fixed, a machine and its operation, instead of growing with every request and being paid in foreign currency as the guide on paying for cloud describes; and quality becomes a question to measure, because a mid-size open model does not match the best closed model on every task, but handles extraction, sorting or search in your documents very well. One route alone is almost never the right answer: everything on site deprives non-sensitive work of the most capable models, everything in Europe excludes what must not leave.
A model on your servers, in five layers
Five layers, two of which are exactly those of a function built on an API.
The application does not change: it calls the orchestration and does not know where the model runs. The orchestration is the one described in the guide on AI in the app: versioned instructions, retrieval, output schemas, thresholds, human fallback, log; it gains one rule, the choice of route. The model server is the added layer: an open inference server exposing a compatible API, batching requests that arrive together, keeping the shared prefix cached and serving quantised weights; switching model is a file and a restart, not a project.
The machine carries one or more graphics cards, memory, storage for weights and indexes, a UPS, an internal network and no outbound access; it is sized on the chosen model and the number of simultaneous requests measured in the pilot. Operations reads every week the card load, the queue, the latency and the score on the evaluation set, backs up weights and indexes, and makes every model update go through the evaluation set before production, as the delivery guide does for code.
«On-site deployment is not a badge of seriousness: it is the answer to a written constraint, where documents are allowed to go.»
Which model fits on which card
The question “which machine do we need” is answered the other way round: which model holds the score on your cases, and how much memory it needs.
Small models, around 8 billion parameters, fit at 4 bits in about 5 GB: enough to sort messages into closed categories and extract simple fields, and they answer many people fast. Around 14 billion, about 9 GB at 4 bits: extraction from more varied documents, summaries. Around 32 billion, about 20 GB at 4 bits, so on a 24 GB card with little room for context, or comfortably on 48 GB: search in documents with sourced answers, agents with tools. Around 70 billion, about 42 GB at 4 bits: longer reasoning and drafting, on a 48 GB card or several cards.
Quantisation to 4 bits divides memory by more than three and costs a little quality; how much is measured on your evaluation set, function by function, not assumed. Language comes before size: a 70-billion model that does not list Arabic can do worse on a darija voice note than an Arabic model four times smaller, and voice transcription uses a dedicated recognition model. The rule: the smallest model that holds the score on each function, often two models on the same machine, a small one for sorting and a mid-size one for search, rather than a big one for everything.
Six situations, and what gets set up for each
Six situations we meet, and the set-up chosen in the first week for each.
The bank, the insurer, the government body: customers’ documents do not leave the institution, sometimes not the network; everything on site, open model, index and log on your servers, no outbound access, and a second machine for recovery. The clinic, the laboratory: reading files and prescriptions on site; general drafting to Europe without patient data; mixed set-up. The manufacturer, the isolated site: the connection does not hold; a small model on one card, offline, updates carried in, log synchronised when the link returns, on the principle of offline mode.
The distributor, the B2B portal: catalogue and orders not very sensitive; API in Europe first, with cache and retrieval; moving on site is decided when monthly volume makes the machine cost less than the requests. The founder in France: no server room, a launch to keep; API in Europe, orchestration theirs from the start, and moving on site stays possible without rewriting. The SaaS vendor: customers who accept Europe, others who require Algeria or their own walls; both routes in the same orchestration, chosen per customer, like the isolation described in the multi-tenant SaaS guide, and an on-site model sold as a plan.
«Size does not tell language: the model is chosen on your cases in Arabic, French and darija, then the machine follows from the model.»
Six failures of an on-site model
Six failures we see on the private deployments we take over, and what the architecture does against each.
The saturated card: twenty people ask a question at once and each waits a minute; requests batched by the server, priority queue, token limit, small model for sorting, load read every week. The oversized model: the biggest was chosen “to be safe”, and it does not fit or answers in ten seconds; size chosen per function on the evaluation set, quantisation measured, the smallest that holds the score. Arabic poorly served: the model chosen on an English leaderboard answers badly in Arabic and darija; models compared on your sixty real cases in the three languages, an Arabic model for voice and dialect.
The update that breaks: a new model replaces the old and three functions regress unseen; every change of model or quantisation goes through the evaluation set, release refused if the score drops, one-command rollback. The exposed server: the model API answers on the network without authentication; server reachable by the orchestration only, no outbound access, log per user, keys rotated. The single machine: the card fails, or the power; UPS, backup of weights and indexes, a second machine, or a written fallback to Europe for what is allowed to go there. None is solved by buying a bigger card.
Seen elsewhere: the Llama card, Jais 3, vLLM, and what transposes
Three sources say the essentials. The Llama 3.3 card says a 70-billion-parameter model lists eight languages that do not include Arabic; what transposes is that a large general model must be tested on your Arabic before being installed. The Jais 3 announcement says open models built for Arabic and its dialects exist in several sizes; what transposes is that a mid-size Arabic model is a serious candidate for voice and dialect, to compare on your cases.
The vLLM documentation says an open server batches requests, caches the prefix, serves quantised weights and exposes a compatible API; what transposes is that a function built on an API moves on site without rewriting if the orchestration is yours. What these sources do not say: which documents are allowed to leave at your company, how many people will query the model at once, and which model holds the score on your cases; the project’s first four weeks answer that, with no purchase.
The rules of an on-site model that holds
Six rules, written in the first week. The constraint first: where each type of document is allowed to go, in writing, before any technical choice. The evaluation set chooses the model, in Arabic, French and darija; not a leaderboard. Nothing is bought before the comparison on machines rented by the hour. The smallest model that holds the score, per function; two models rather than a big one. The orchestration is the same as for an API, and it is yours: the route is a setting. Every update goes through the evaluation set, with rollback.
A seventh rule concerns the quote: “sovereign on-site AI” is not a requirement, it is a sentence; the requirement reads “list of document types and their allowed route; open model chosen on sixty real cases in the three languages, score measured; machine sized for N simultaneous requests with wait time measured in the load test; inference server with batching and cache; no outbound access; log per request; backup of weights and indexes; update and rollback procedure; written recovery”, and it is checked as the specification guide recommends. A quote that prices “a GPU server” has priced a machine, not a function.
Six functions, and which route they take
Six functions we often build, and the route chosen when a constraint exists.
- Reading invoices, IDs and files. On site: the document itself is what is sensitive; a reading model then a mid-size model to fill the fields.
- Search in contracts and procedures. On site: index and documents on your servers, sourced answer; the keyword part follows the search guide.
- Sorting messages and mail. On site, small model: closed categories, high volume, fast answer.
- Transcribing darija voice notes. On site when the voice notes come from customers: a multi-dialect Arabic recognition model, tested on your voice notes.
- Agent on WhatsApp. The message travels through Meta whatever the model, as the WhatsApp agent guide describes; the model can be on site, but the channel is not, and the customer is told so.
- General drafting, translation, summarising public texts. Europe: no sensitive data, and the most capable models are there.
The 2027 calendar
The calendar is that of a complete private deployment in ten weeks, two without code and no card bought before the fourth.
Week 0 to 1 writes the constraint and the functions: where each type of document is allowed to go, the two or three functions, what stays on site and what may leave; signed. Week 1 to 2 writes the evaluation set: sixty real cases in Arabic, French and darija, with the right answer. Weeks 2 to 4 choose the model: three or four open models compared on the set, on machines rented by the hour, at 16 and 4 bits; the size and quantisation chosen decide the machine. Weeks 4 to 6 install the machine and the model server, no outbound access, with cache, queue and a load test at the real number of users.
Weeks 6 to 9 wire the orchestration to the on-site model and deliver the functions one by one, then the pilot with human fallback, the share handed to a human and the wait read every day. Week 9 to 10 installs operations: weekly supervision, backup of weights and indexes, update procedure through the evaluation set, and a rehearsal of the machine failure. The running cost, electricity, maintenance, replacement, joins the yearly budget described in the guide on keeping an app alive.
What makes a private deployment fail
The private deployments we take over almost always failed for one of these reasons.
- The machine bought first. A card chosen on a spec sheet, then a model someone tries to fit on it.
- The model chosen on a leaderboard. First in English, mediocre on your documents in Arabic.
- No inference server. A script that loads the model and serves one person at a time.
- Everything on site on principle. Tasks with no sensitive data deprived of the best models.
- No evaluation set. Quality is an impression, and every update a bet.
- The model API open on the network. No authentication, no log; nobody knows who asked what.
- No recovery. A single machine, no backup of the indexes; the outage lasts weeks.
Frequently asked questions
The questions we receive from IT departments, managers in Algeria and SaaS vendors.
- Should we deploy the model on our servers?
- Only if a written constraint requires it: customers’ documents, health files or data that must not leave the institution or the country, or a site without a reliable connection. Without that constraint, an API hosted in Europe, behind an orchestration you own, arrives faster and gives access to the most capable models; and moving on site stays possible later without rewriting.
- Which open model to choose for Arabic and French?
- The one that holds the score on your sixty real cases, in the three languages, darija included. Size does not tell language: the card of a large 70-billion-parameter general model lists eight languages without Arabic, while families built for Arabic and its dialects exist in several sizes. We compare three or four candidates on rented machines before any purchase.
- Which machine do we need?
- It follows from the chosen model: about 2 GB of memory per billion parameters at 16 bits and 0.6 GB at 4 bits, plus context and simultaneous requests. An 8- to 14-billion model fits comfortably on a 24 GB card at 4 bits; 32 billion just fits there or comfortably on 48 GB; 70 billion needs 48 GB or several cards. These are orders of magnitude; the load test with your real number of users decides.
- How many people can one machine serve?
- It depends on the server more than on the card: an inference server that batches requests arriving together and keeps the prefix cached serves a team where a script serves one person. The exact number is measured in the week-five load test, with your documents and your answer length, and the queue is read every week afterwards.
- Is it more costly than an API?
- The cost changes nature: fixed, a machine, its electricity and its operation, instead of per request and in foreign currency. Below a certain volume, the API costs less; above it, the machine pays for itself. The threshold is computed with your volume measured in the pilot, not with a general figure, and the constraint on documents often decides before cost does.
- How long to deploy?
- Ten weeks: one for the constraint and functions, one for the evaluation set, two to choose the model on rented machines, two for the machine and the model server, three for the functions and the pilot, one for operations and recovery. No hardware purchase before week four.