Open-weight model risk, model layer first: the Microsoft and NVIDIA guidance

Teams looking at open-weight models for coding work usually get the same question from security or legal: is an open-weight model riskier than a hosted one? A joint paper from Microsoft and NVIDIA, published on 16 October 2026, answers that the question is framed wrongly. In Microsoft's summary, "weight openness alone does not determine an AI system's risk". Risk depends on the model, the use case, the deployment and the safeguards around it.

The paper, Risk Management for Open-Weight Models in Enterprise, is the first in a planned series. It offers a shared risk taxonomy and a process for applying it, with evaluations mapped to some of the risks. It is useful mainly as a vocabulary and a structure. It is also explicit about what it doesn't cover yet, and for teams running coding agents that is most of what matters.

What the paper contains

A taxonomy. Five risk categories: Cybersecurity, Data Protection, Content, Human-AI Interaction and Bias. Under them sit 30 risk types, and the paper assigns owners, evaluations and mitigations at that level. The paper is clear that the taxonomy is neither a policy nor a checklist. It doesn't set your risk tolerance or decide deployments.

A process. The guidance applies the risk management process in ISO/IEC 23894 and aligns with the Map, Measure and Manage functions of the NIST AI Risk Management Framework. NVIDIA's companion post turns it into five steps: define scope, context and criteria; identify and prioritise risks; evaluate the model and the deployed system; mitigate and evaluate again; and monitor and reassess.

Evaluations. The appendix maps evaluations to selected risk types: 13 entries in its table, eight of them for Content. NVIDIA's post gives slightly different counts, 14 in all with nine for Content. The weighting towards content harms is clear.

The paper and its taxonomy are published in a public repository, where the authors invite pull requests and say the taxonomy will be maintained as a shared, versioned artefact.

Four ideas worth keeping

Provenance before behaviour. The paper says to establish what an artefact is before testing how it behaves. Cryptographic signing, such as OMS Model Signing, checks that the weights you received are the ones the provider published. "Evaluation cannot substitute for it." Where no signature is available, the paper says to log the missing verification as a risk that stays open, however good the evaluation results look.

Modifications affect safety more than capability. Fine-tuning, merging, quantisation and configuration changes "can degrade safeguards while leaving task performance intact". A quantised model that still writes good code isn't evidence that its refusals still work. In the paper's framing, whether a model is safe and whether it is capable have to be checked separately.

Propensity is not likelihood. An evaluation measures how often a model shows a behaviour under test. How often that happens in production depends on exposure, users and adversaries. NVIDIA's worked example makes the same point: if 4 of 100 test summaries contain an unsupported claim, "That result describes this test set; it does not establish how often errors will occur in production."

Controls shift to you. With open weights, provider-run safeguards no longer apply everywhere, and enterprises that operate their own deployments "may take on controls otherwise provided by a managed service", in Microsoft's words. The paper's answer is layered controls you can update quickly, such as input filtering, output moderation, access restrictions, rate limits, monitoring and human oversight, because retraining and redistributing weights can lag behind new jailbreaks.

What it leaves out for now

This first paper is scoped to the model layer. The paper says future work will extend the approach to the harness and application layers. The paper says threats from connected tools, external services and downstream integrations should be assessed at those layers, which later papers will address. Its one worked example is a plain inference endpoint with no tools, retrieval or agentic harness.

Two more exclusions matter for coding teams. The taxonomy deliberately leaves out attack techniques such as jailbreaking and prompt injection, treating them as ways risks arise rather than risks in themselves. And while the Cybersecurity category includes Malicious Code Generation, which covers vulnerable code and malicious or made-up dependencies, and Malicious Tool Call Generation, which covers agents using their tools outside their intended purpose, only the first is mapped to an evaluation in the appendix.

So for a coding agent built on an open-weight model, the paper helps with choosing and checking the model. It doesn't yet help with the harness, the permissions or the tools, which is where agent incidents tend to happen. We looked at one candidate model in our piece on Muse Spark Open, and at the case for small open-weight models as sub-agents. Both are model-layer decisions that this guidance can inform.

What it means for engineering and security leaders

• Use the taxonomy as shared vocabulary. Five categories and 30 named types give engineering, security and legal the same terms when deciding whether a model fits a use case.
• Require provenance checks. Ask where each set of weights came from, whether it is signed, and who changed it. Record unverified weights as an open risk.
• Re-evaluate after every modification, including quantisation and fine-tuning, and test safety behaviour separately from task performance.
• Don't treat a model evaluation as an agent evaluation. Until the later papers arrive, you need your own assessment of what the agent can reach and do.

Practical takeaways

• For each open-weight model in use, record the source, version, any modifications and whether the artefact's signature was verified.
• Pick the risk types that apply to your use and set acceptance criteria before you run any evaluation. NVIDIA's example sets a fabricated financial figure as a blocker in advance.
• Store the model version, prompts, datasets, scoring method and configuration with every result, as both sources recommend, so results can be compared across versions.
• For coding use, include insecure code and invented dependencies in your tests. The paper maps CWEval to Malicious Code Generation.
• Keep "decline the deployment" as a real outcome. NVIDIA lists it alongside changing inputs, filtering outputs, restricting access, human review and changing the model.