What is AI governance?

• AI governance = the set of decisions, policies, and controls determining
  • which AI systems (models, providers, techniques) an organization uses,
  • how and where they are deployed,
  • who is accountable for their outputs, and
  • how they are monitored over time
• It is not (only) a legal matter: most governance decisions are engineering decisions, taken (often implicitly) while designing the system
  • e.g. choosing a provider, a model, a deployment option, or whether a human reviews the outputs
• Making these decisions explicit, motivated, and documented is what distinguishes governance from improvisation

The five dimensions of the trade-off

Every choice about an AI-powered system implies a trade-off among:

1. Quality: how good (accurate, reliable, consistent) are the outputs for the task at hand?
2. Cost: how much do development, operation (per request, per month), and maintenance cost?
3. Control: how much can we inspect, customize, reproduce, and keep running the system, regardless of third parties?
4. Compliance: does the system respect laws (e.g. GDPR, AI Act), contracts, and internal policies?
5. Time-to-market: how quickly can we ship (and change) the system?

Running example: a governance perspective

Recall the running example (PhD admission committee assistant):

• Data: passports, transcripts, and presentation letters of the candidates
  • these are personal data (even identity documents!)
• Task: extract information, summarize, and score candidates
  • outputs contribute to decisions affecting people’s access to education
• Stakeholders: candidates, committee members, the university (as the responsible organization), the AI provider(s)

Where should the model run? The general concept

Three main deployment options, differing in who runs the model, and where data goes:

1. Proprietary cloud API: the model is owned and run by its provider (e.g. OpenAI, Anthropic, Google)
   • you only get an API; data is processed on the provider’s infrastructure
2. Open model, hosted by a third party: an open-weights model run by a cloud provider or aggregator (e.g. via Open Router, Hugging Face Inference Endpoints, cloud GPU rental)
   • same model could be moved elsewhere; data still leaves your premises
3. On-premise (a.k.a. self-hosted, local): an open-weights model run on your own hardware (e.g. via Ollama, vLLM)
   • data never leaves your premises; you are responsible for hardware, operation, and updates

(cf. on-premise vs. on-cloud in the LLM-as-a-Service lecture)

Where should the model run? An intuitive example

Deployment options: comparison

• Energy matters: e.g. generative tasks cost orders of magnitude more energy than task-specific models (Luccioni et al., 2024); GPAI providers must document energy consumption (AI Act Art. 53, Annex XI)
• Hybrid solutions are common: e.g. on-premise for sensitive data, cloud for the rest; or a gateway routing requests according to data sensitivity

Deployment options: a worked cost example

Let’s estimate the yearly cost of the running example, under explicit (and rough) assumptions:

• 500 applications per year; each one = passport picture + transcript + letter ≈ 6k input tokens, plus ~1k tokens of prompts
• 5 requests per application (extraction, validation, scoring, …) $\implies$ ~35k input + ~5k output tokens per application
• hence, per year: 17.5M input + 2.5M output tokens

(prices as of September 2026: they change every few months, always re-check them)

Regulation: the general picture

AI-powered software is subject to (at least) four layers of rules, each with a different object:

1. the GDPR (Reg. (EU) 2016/679) regulates the processing of personal data, whatever the technology
2. the EU AI Act (Reg. (EU) 2024/1689) regulates AI systems (by risk of their use) and general-purpose AI models
3. national laws, e.g. the Italian AI law (Legge 132/2025)
4. institutional policies, e.g. the UniBo GenAI policy

plus contracts (e.g. providers’ terms of service, data processing agreements) and licenses (of models, and of data)

Disclaimer: this is an engineer’s overview, not legal advice: in real projects, involve your organization’s legal office and Data Protection Officer (DPO)

GDPR for LLM-powered applications: key concepts

• Personal data = any information about an identified or identifiable person (e.g. names, passports, grades, letters about someone)
• Controller (who decides why and how data is processed: e.g. the University) vs. processor (who processes data on behalf of the controller: e.g. the LLM provider)
  • a Data Processing Agreement (DPA, Art. 28) with the provider is mandatory
• Principles (Art. 5): lawfulness, purpose limitation, data minimisation (send the LLM only what it needs!), accuracy, storage limitation, security
  • privacy by design and by default (Art. 25): e.g. strip or pseudonymize personal data before sending prompts
  • storage limitation vs. logging: logs kept ≥ 6 months (AI Act Art. 26(6)) contain personal data, so GDPR still applies: define a retention period (e.g. the appeal window + 6 months), then delete or pseudonymise
• International transfers (Chapter V): data leaving the EU need an adequacy decision (e.g. the EU–US Data Privacy Framework, upheld by the General Court in 2025, appeal pending) or other safeguards (e.g. standard contractual clauses)
• Data Protection Impact Assessment (DPIA, Art. 35): required for high-risk processing, e.g. new technologies used to evaluate or score people
• Automated decisions (Art. 22): people have the right not to be subject to decisions based solely on automated processing, with legal or similarly significant effects
  • e.g. an admission rejection decided by an LLM alone
• About the models themselves: EDPB Opinion 28/2024 states that a model trained on personal data is anonymous only if extracting those data is very unlikely, and that an unlawfully developed model may taint its later use

GDPR in practice: providers and enforcement

What providers offer (as of September 2026)

• Free tiers and consumer apps often allow providers to use your data for training: never send personal data there

The Italian Data Protection Authority (Garante) at work

• March 2023: ChatGPT temporarily blocked in Italy (lawful basis, transparency, age verification), then reopened under conditions
• December 2024: €15M fine to OpenAI, later annulled by the Court of Rome (March 2026) since the competent authority was the Irish one (one-stop-shop mechanism)
• January 2025: urgent limitation of processing for DeepSeek

EU AI Act: a risk-based approach

• Unacceptable risk $\rightarrow$ prohibited (Art. 5)
  • e.g. social scoring, manipulative techniques, emotion recognition in workplaces and education, untargeted scraping of facial images
• High risk $\rightarrow$ strict obligations (Art. 6 and Annex III)
  • e.g. CV screening, credit scoring, and education: systems to determine access or admission to educational institutions (Annex III, point 3(a)), to evaluate learning outcomes (3(b)), or to proctor exams (3(d))
• Transparency risk $\rightarrow$ disclosure obligations (Art. 50)
  • e.g. chatbots must disclose they are AI; synthetic content must be marked; deepfakes must be labelled
• Minimal risk $\rightarrow$ no specific obligations
  • e.g. spam filters, AI in video games
• Orthogonally: general-purpose AI (GPAI) models, with obligations for their providers (Art. 53; systemic risk: Art. 51, Art. 55)

EU AI Act: who must do what?

• Beware: a deployer becomes a provider if it substantially modifies a system, or puts its own name on it (Art. 25)
  • e.g. a University building its own admission assistant on top of an LLM API is the provider and the deployer of that system
• GPAI model providers (e.g. OpenAI, Mistral) must provide technical documentation to downstream providers, a copyright policy, and a public summary of training data (Art. 53); models with systemic risk (presumed above 10²⁵ FLOPs of training, Art. 51(2)) also need evaluations, adversarial testing, incident reporting (Art. 55)
  • open-source GPAI models are exempted from the documentation duties only, and only if they do not pose systemic risk (Art. 53(2), details later)
  • a voluntary GPAI Code of Practice (July 2025) details how to comply

EU AI Act: timeline, literacy, penalties

Timeline (as amended by the Digital Omnibus on AI, Reg. (EU) 2026/1744)

EU AI Act: literacy, penalties, standards

• AI literacy (Art. 4): providers and deployers must take measures to support the AI literacy of their staff (this course is part of it!)
  • as amended by the Omnibus: previously, they had to ensure a sufficient level of literacy
• Penalties (Art. 99): up to €35M or 7% of worldwide turnover for prohibited practices; up to €15M or 3% for most other violations
• Transparency: a Code of Practice on AI-generated content details marking and labelling
• Standards: conformity in practice goes through harmonised standards (CEN-CENELEC JTC 21, still being drafted: one reason for the Omnibus delay); AI management systems: ISO/IEC 42001:2023; outside the EU: NIST AI RMF 1.0 (2023), plus its GenAI profile (NIST AI 600-1, 2024)

National law and institutional policies

Italy: Legge 23 settembre 2025, n. 132 (in force since 10 Oct 2025)

• complements the AI Act: AgID and ACN are the national AI authorities (Art. 20)
• public administration: AI is only instrumental, the human official remains solely responsible for decisions (Art. 14)
• intellectual professions: AI only for supporting activities, and clients must be informed (Art. 13)
• minors: under 14s need parental consent to use AI (Art. 4)
• copyright: only human works are protected (AI-assisted ones are, if they result from the author’s intellectual effort); rules on text and data mining (Art. 25)
• new crime: unlawful dissemination of AI-generated deepfakes (Art. 26)
• health: AI supports, the physician decides (Art. 7); work: employers must inform workers about AI use (Art. 11)
• delegated decrees (Art. 24): approved in final reading on 4 Aug 2026; the one on police use and liability is D.Lgs. 160/2026 (GU 15 Sep 2026), the one on authorities, training, and work was expected in Oct 2026

University of Bologna: GenAI policy for teaching and research

• GenAI use is allowed: principles of human centrality, transparency, accountability, accuracy, respect for rights
• substantial uses (e.g. AI-written papers, theses, or assessed work) must be declared; GenAI cannot be an author
• no substantial use where it affects others: e.g. peer review, evaluating research projects, grading students
• no personal data of third parties, confidential or unpublished material in online GenAI tools, without a legal basis and guarantees
• a separate policy for administrative staff exists

What makes a model “open”? The general concept

A model release is a bundle of separately openable artifacts: downloading one of them (usually the weights) says little about the others

Openness is graded, not binary

• Open-washing: presenting a model as “open” or “open source” based on one disclosed component (typically the weights), while the artifacts or legal freedoms needed to study, reproduce, modify, or freely use it are missing
  • e.g. Llama 4 (downloadable weights, restrictive license), Gemma 3 (custom terms with use restrictions), Mistral 3 (called “open-source” by its vendor: Apache 2.0 weights, undisclosed training data)
  • by contrast, OpenAI calls gpt-oss “open-weight”: the more accurate term

The OSI Open Source AI Definition (OSAID 1.0, Oct. 2024)

• Users must be able, without asking permission, to:
  1. use the system for any purpose
  2. study how it works
  3. modify it for any purpose
  4. share it, with or without modifications
• This requires the preferred form for making modifications:
  • data information: enough detail on training data for a skilled person to build a substantially equivalent system
  • code: the complete training and inference pipeline
  • parameters: weights and configuration, under open terms
• Controversy: OSAID does not require releasing the training data itself (cf. OSI’s FAQ)
  • OSI: data may be personal, copyrighted, or confidential, so requiring it would exclude too many systems
  • critics: without the data, exact reproduction and data-level auditing are impossible
• OSI’s initial validation found that Pythia, OLMo, Amber, CrystalCoder, and T5 passed, while Llama 2, Grok, Phi-2, and Mixtral did not
  • a snapshot of 2024, not a certification registry for later versions

The licensing landscape (as of September 2026)

The licensing landscape (as of September 2026)

Licenses change across versions: always check the exact checkpoint

What do restrictive licenses mean in practice?

Derivatives: fine-tunes, distillation, merges, data, outputs

• Fine-tunes remain subject to the base model’s license: easy for Apache/MIT, restrictions propagate for Gemma 3 and Llama
• Distillation is license-specific:
  • Gemma 3 counts models trained on Gemma’s outputs (synthetic data) as Model Derivatives
  • DeepSeek-R1 (MIT) allows distillation, but its published distills are fine-tunes of other models, and inherit their licenses (Qwen → Apache 2.0, Llama → Llama license)
• Merges: a merge of models A and B must (conservatively) satisfy both licenses; a restrictive component is not “washed away”
  • whether weights are copyrightable at all is still legally unsettled
• Training data has its own license: e.g. OLMo 3 is Apache 2.0, but its dataset Dolma 3 is ODC-By (attribution required), and its sources may carry further terms
• Outputs: “the vendor claims no rights on outputs” $\neq$ “outputs are free of restrictions”
  • e.g. using Llama outputs to train a distributed model triggers Llama’s naming requirement
  • proprietary vendors offer copyright indemnities for outputs (e.g. Microsoft’s Copilot Copyright Commitment, Google’s generative AI indemnification, OpenAI’s Copyright Shield, Anthropic’s commercial terms), usually for paid or enterprise tiers only, and with conditions; open-weights licenses give none
  • in Italy, only human works are protected (L. 132/2025, Art. 25: cf. below)

Open models under the EU AI Act

• To qualify, the model must be released under a license allowing access, use, modification, and distribution, with weights, architecture, and usage information public
  • this is much weaker than OSAID: no training data or training code needed
• Non-EU providers of open-source GPAI are also exempted from appointing an EU authorized representative (Art. 54(6)), unless the model poses systemic risk
• cf. the Commission’s AI Act Service Desk FAQ on open-source GPAI models

The AI Act gives open-source GPAI a documentation discount: not a copyright exemption, not a training-data-summary exemption, and not a systemic-risk exemption

“Sovereign AI”: a framing, and its limits

• Several European and Italian actors frame open, locally-developed models as a matter of digital sovereignty
  • reduce dependency on US and Chinese providers; better support for EU languages; compliance with EU law
• The framing mixes (at least) three distinct goals, which should not be conflated:
  1. where a model is developed (nationality of the developer)
  2. where it is executed, and data are processed (operational control)
  3. whether its making can be inspected and reproduced (auditability)
• Goals 2 and 3 do not require goal 1:
  • any open-weights model (e.g. Qwen3, gpt-oss, OLMo 3) can run in an EU data centre
  • the most reproducible models are not necessarily European (e.g. OLMo 3, from the US), and a European vendor may disclose little (e.g. Mistral)

The gap, in numbers

Data from the Stanford AI Index 2026 (2025 figures):

• The gap is not closing: it is widening, at least for private investment
• Caveats: private investment understates state funding (notably China’s, and EU/national programmes such as EuroHPC AI Factories); counts by affiliation may double-count multinational teams

Italian initiatives

• A “national” model says nothing about its openness: compare Minerva (Apache, open data) with Italia 10B (restrictive license)
• Italian models trained from scratch range from 0.35B (Minerva) to 14B (Velvet); larger ones (e.g. ANITA-NEXT 24B) are fine-tunes of foreign models: one or two orders of magnitude below the frontier

European initiatives

Infrastructure and policy

• EuroHPC AI Factories: supercomputing centres opened to AI training and use
  • IT4LIA, at CINECA in Bologna (with Austria and Slovenia), was among the first seven selected in Dec. 2024 (€1.5bn in total, EU + national funds)
  • beware: IT4LIA is an AI Factory, not an AI Gigafactory: the Gigafactories call (for frontier-scale training) opened only in July 2026, with a Nov. 2026 deadline
• Data: ALT-EDIC’s LLMs4EU (March 2025, 3 years, 20+ countries) collects language resources for all EU languages, and targets sector-specific models
• Italian policy: the Italian AI Strategy 2024–2026 and Law 132/2025 call for technology “tailored to the Italian context” and open research artifacts

How good are Italian models?

• Measuring it is hard: translated English benchmarks are not enough
  • ITA-Bench (CLiC-it 2024) mixes translated tasks with native Italian datasets
  • Evalita-LLM (2025) uses native Italian tasks, and multiple prompts per task, since rankings are sensitive to prompt wording (leaderboard)
• Published numbers show that small Italian models can be useful, e.g. Velvet-2B reports 39.6 on Italian MMLU and 61.9–86.6 on ITA-Bench tasks…
• … but there is no robust evidence that Italian models outperform large multilingual open models (e.g. Qwen3, Gemma 4), let alone frontier ones, even on Italian

Recurring issues of European open models

1. Data: too little high-quality text in less-resourced languages (the very reason for LLMs4EU and for synthetic data)
2. Compute: every project depends on a public supercomputer allocation (Leonardo, MareNostrum 5, JUWELS, Alps); an “open” model is not reproducible by anyone who lacks one
3. Scale and timing: models trained from scratch up to 14B parameters, released months to years after comparable global open models; flagships “planned” or “pending”
4. Maintenance: projects have finite funding (e.g. OpenGPT-X ended in 2025): who fixes, updates, and re-evaluates the models afterwards?
5. Adoption: public-administration use is mostly aspirational in the sources, not a demonstrated large-scale deployment

Engineering take-aways: pre-adoption checklist

The compliance unit is checkpoint + license version + code licenses + data licenses + deployment + intended use, not a brand name

Mnemonic: weights → license → recipe → data → regulatory role: a model is only as open as the layer that matters for your use

Engineering take-aways: three scenarios

Which technique? Recall

• Recall the technique selection from the RAG lecture: prompting, tools and agents, RAG, skills and workflows, fine-tuning
  • each option adds effort to the previous ones, and complexity to the system
  • choose by asking what is missing (knowledge, actions, a reliable procedure, a style), cf. the intuitive example
• From the governance viewpoint, what matters are the trade-offs (next slide): cost, control, compliance, and time-to-market

Which technique? Trade-offs

• These options combine: e.g. a workflow whose steps use prompting, RAG, and tools
• Empirically, RAG tends to outperform fine-tuning at injecting new knowledge (Ovadia et al., 2023), while fine-tuning is better at shaping form

Recall the rule of thumb: start with prompting, measure, and add complexity only when evaluations show that it is needed

Which model? The general concept

• Choosing a model is a governance decision, since it affects all five dimensions at once:
  • quality (capabilities), cost (price per token, or hardware), control (open weights? version pinning? deprecation?), compliance (license, provider’s jurisdiction and data policies), time-to-market (availability, ecosystem)
• It requires information about models, which comes from:
  1. their documentation (model cards, system cards, licenses, terms of service)
  2. public evaluations (benchmarks, leaderboards)
  3. your own evaluations on the task at hand
• It is not a one-off decision: models are updated, deprecated, and surpassed every few months, so the choice must be re-assessed periodically (and made reproducible)

(cf. model zoos in the LLM-as-a-Service lecture)

Model documentation: model cards, system cards, datasheets

• Model cards: recall them from the model zoos of the LLM-as-a-Service lecture (Mitchell et al., 2019; Hugging Face’s guide); here, what matters is what they say about governance (below)
• Datasheets for datasets (Gebru et al., 2021): the same idea, for training data (motivation, composition, collection, recommended uses)
• System cards: longer reports published by frontier providers for each model release, focusing on safety evaluations, red-teaming, and mitigations
  • e.g. OpenAI’s and Anthropic’s system cards

• What to look for, as an engineer: license, languages, context window, knowledge cut-off, intended uses (does yours fit?), known limitations, and evaluations relevant to your task

Measuring general capabilities: frontier benchmarks

As models saturate classic benchmarks, new ones try to measure what models cannot (yet) do:

• ARC-AGI (Chollet, 2019): intelligence as skill-acquisition efficiency, measured on visual puzzles that are easy for humans, and require abstracting rules from a few examples
  • ARC-AGI-2 (2025) raised the difficulty; the ARC Prize 2025 grand prize went unclaimed
  • ARC-AGI-3 (March 2026) is interactive: cf. the agentic benchmarks of the agents lecture, and the leaderboard below
• Humanity’s Last Exam (Phan et al., 2025): 2,500 expert-written questions across disciplines, designed to be un-googleable
  • best models went from single digits (early 2025) to ~60% (cf. Artificial Analysis, September 2026)
• METR’s time horizon (Kwa et al., 2025): the length of tasks (in human-expert time) that an agent completes with 50% success
  • doubling every ~7 months over 2019–2025 (212 days, in the original paper), and faster since 2023: ~4 months (129 days, METR’s Time Horizon 1.1 fit, cf. live measurements)
• Aggregated views: the Stanford AI Index 2026 describes a jagged frontier: models winning maths olympiad gold medals, yet failing at reading analog clocks

Source: Stanford AI Index 2026, ch. 2, Figure 2.1.1 (CC BY-ND 4.0)

Engineering take-away: frontier benchmarks tell you where the state of the art is, not whether a model is good for your task

Frontier benchmarks: METR’s time horizons

• E.g. the 50% time horizon of Claude Mythos Preview (April 2026) is ~17 hours, at the limit of what METR’s task suite can measure
• Caveats: software tasks only, 50% success (the 80% horizon is much shorter), wide confidence intervals

Source: METR, time horizons, retrieved October 2026

Frontier benchmarks: METR’s time horizons (cont.)

• The original fit: doubling every 212 days (2019–2025)

• How a horizon is computed: success rate vs. human task length, then a logistic fit, read at 50%

Source: METR, time horizons, retrieved October 2026

Frontier benchmarks: the ARC-AGI-3 leaderboard

• From < 1% at launch (March 2026) to 62.7% with the standard harness, and ~99% with a general-purpose harness (keeping reasoning state, compacting the conversation), in 6 months
  • at a cost of thousands to tens of thousands of dollars per full run; humans solve all environments

Source: ARC Prize leaderboard, retrieved October 2026

How to choose a model for a given task?

How to evaluate models?

1. Public benchmarks and leaderboards (generic, cheap to consult)

• benchmarks: fixed datasets + metrics for some capability
  • e.g. knowledge (MMLU), expert reasoning (GPQA), coding (SWE-bench), tool use, long context…
  • holistic frameworks like HELM evaluate many models on many scenarios and metrics (not just accuracy)
• leaderboards: rankings aggregating benchmark results or human preferences
  • e.g. Arena (formerly Chatbot Arena / LMArena: humans blindly compare pairs of answers, cf. Chiang et al., 2024; but see also the critique by Singh et al., 2025), Artificial Analysis (quality vs. price vs. speed)
• caveats:
  • data contamination: benchmark items may have leaked into training data (cf. Xu et al., 2024)
  • saturation: top models all score near 100%, so benchmarks stop discriminating (e.g. Hugging Face retired its Open LLM Leaderboard in 2025)
  • Goodhart’s law: once a benchmark becomes a target, it stops being a good measure
  • benchmarks measure generic skills, possibly not the ones your task requires

2. Task-specific evaluation (the one that really matters)

• run each shortlisted model on your test dataset, with your scorers, as in the validating lecture (deterministic scorers first, LLM-as-a-Judge where code can’t help)
• measure latency and cost too (both reported in the API responses’ metadata, e.g. usage)

Exercise 1: Comparing Models on the Running Example (pt. 1)

Goal: the admission committee wants to pick the model to be used for letter scoring (cf. Example 1 of the Prompt Engineering lecture), based on evidence rather than on hype

Code: put your solution in snippets/lecture_governance/exercise1/ of lab-snippets, and run it via poetry run python -m snippets -l governance -x 1

TO-DO List

1. Shortlist 3 models, e.g.:
   • one small free model on Open Router,
   • one larger (possibly free) model on Open Router,
   • one local model via Ollama (if your hardware allows it)
2. Run the letter-scoring system on all 3 letters of the running example, 5 times per letter per model
   • the script is already parametric w.r.t. the model: just change OPENAI_MODEL (and OPENAI_BASE_URL=http://localhost:11434/v1 for Ollama)
3. For each model, collect:
   • the scores (average, and variance across the 5 runs: is the model consistent?)
   • the agreement among models (do they rank candidates in the same way?)
   • the number of failures (e.g. invalid JSON, validation errors)
   • latency (wall-clock time per request) and tokens used (from response.usage), hence cost
4. Summarize the results in a table, and motivate your choice

Exercise 1: Comparing Models on the Running Example (pt. 2)

Decision points and hints

• Same idea as step 5 of Exercise 2 of the validating lecture, now with costs and latency; a plain script, as DeepEval or MLflow score one system at a time (MLflow would work too: one run per model)
• How to automate the runs?
  • e.g. a script looping over models × letters × repetitions, writing one row per run in a CSV file
  • reuse the caching mechanism of Exercise 1 of the LLM-as-a-Service lecture? (careful: caching would hide the variance!)
• How to measure time?
  • e.g. time.perf_counter() before and after the request
• How to compute costs?
  • score_letter(...) only returns the parsed LetterInfo: adapt it to also return the whole response (or its usage)
  • multiply usage.prompt_tokens and usage.completion_tokens by the prices listed in the zoo (Open Router also reports usage.cost directly)
• How to deal with rate limits of free models?
  • reuse the retry mechanism of Exercise 2 of the LLM-as-a-Service lecture
• What if models disagree?
  • which one is right? you need a reference: e.g. score the letters yourself, and compare each model against your scores

How to test it?

• the table should contain 3 models × 3 letters × 5 runs = 45 rows (minus failures, which should be counted, not hidden)
• re-running the script with a different temperature (e.g. 0 vs. 1) should visibly affect the variance

Solution: a walkthrough follows in the next (vertical) column — try on your own first!

Exercise 1: Comparing Models on the Running Example — Solution

⚠️ The walkthrough of a possible solution is about to start

• do not proceed until you have attempted the exercise on your own
• press → to skip it, ↓ to see it
• the code is on the master branch of lab-snippets (while you cloned the exercises branch)

Exercise 1 — Solution: Shortlist, Reference, and Clients

Step 1: shortlist 3 models (and fix a reference to compare them against)

# step 1: the shortlist (free models are rate-limited: retries below)
DEFAULT_MODELS = ["google/gemma-4-26b-a4b-it:free", "nvidia/nemotron-3-super-120b-a12b:free", "llama3.2@http://localhost:11434/v1"]
REPETITIONS = int(os.environ.get("REPETITIONS", 5))
TEMPERATURE = os.environ.get("TEMPERATURE")
# the reference: the committee's OWN scores, by reading the letters (without it, one can tell whether models agree, not who is right)
REFERENCE = {"mario-rossi": 5, "jean-dupont": 3, "mohammed-ali": 2}
OUTPUT = Path(os.environ.get("COMMITTEE_OUTPUT", "output")) / "model-comparison.csv"


def client_for(model: str) -> tuple[OpenAI, str]:
    """e.g. 'llama3.2@http://localhost:11434/v1' -> a client for Ollama, and 'llama3.2'."""
    name, _, url = model.partition("@")
    return OpenAI(base_url=url or base_url, api_key=api_key if not url else "local", max_retries=0), name

Exercise 1 — Solution: Scoring a Letter, Keeping the Usage

Step 2: adapt score_letter(...) to return the whole response, not just the parsed LetterInfo

def score_letter(client: OpenAI, model: str, letter_text: str):
    """As in Example 1, but returns the WHOLE response (i.e. also its usage), not just the parsed LetterInfo."""
    options = dict(temperature=float(TEMPERATURE)) if TEMPERATURE else {}
    return with_retries(lambda: client.chat.completions.parse(  # retries upon rate limits (cf. Exercise 2 of the LLM-as-a-Service lecture)
        model=model,
        messages=[
            dict(role="system", content=general_instructions),
            dict(role="user", content=f"Score the following letter:\n\n<letter>\n{letter_text}\n</letter>"),
        ],
        response_format=LetterInfo,
        **options,
    ), retries=5, initial_delay=5)

Exercise 1 — Solution: One Run = One Row

Steps 2–3: run once, and collect score, failures, latency, tokens, and cost

def run_once(model: str, candidate: str, repetition: int) -> dict:
    """Step 2-3: one run, as a row of the CSV file. Failures are rows too: they must be counted, not hidden."""
    client, name = client_for(model)
    row = dict(model=model, candidate=candidate, repetition=repetition, score=None, error=None, latency=None, prompt_tokens=None, completion_tokens=None, cost=None)
    start = time.perf_counter()
    try:
        response = score_letter(client, name, data.letter(candidate).read_text())
        row["latency"] = round(time.perf_counter() - start, 2)  # wall-clock time, retries included
        row["score"] = response.choices[0].message.parsed.score  # may fail, e.g. upon refusals or invalid JSON
        row["prompt_tokens"], row["completion_tokens"] = response.usage.prompt_tokens, response.usage.completion_tokens
        row["cost"] = getattr(response.usage, "cost", None)  # in USD, reported by Open Router (not part of OpenAI's API)
    except Exception as e:
        row["error"] = f"{type(e).__name__}: {e}"[:200]
    return row

Exercise 1 — Solution: Summarising per Model

Step 4: one line per model, comparing mean ± sd against the reference

def summarise(rows: list[dict]) -> list[dict]:
    """Step 4: one line per model."""
    summary = []
    for model in dict.fromkeys(row["model"] for row in rows):  # models, in order of appearance
        ok = [row for row in rows if row["model"] == model and row["error"] is None]
        scores = {c: [row["score"] for row in ok if row["candidate"] == c] for c in REFERENCE}
        means = {c: statistics.mean(s) for c, s in scores.items() if s}
        summary.append(dict(
            model=model,
            **{c: f"{means[c]:.1f} ± {statistics.pstdev(s):.1f}" if s else "n/a" for c, s in scores.items()},  # consistency
            error=round(statistics.mean(abs(row["score"] - REFERENCE[row["candidate"]]) for row in ok), 2) if ok else None,  # w.r.t. the reference
            ranking=" > ".join(sorted(means, key=means.get, reverse=True)),  # agreement among models: compare these
            failures=sum(row["model"] == model for row in rows) - len(ok),
            latency=round(statistics.mean(row["latency"] for row in ok), 1) if ok else None,
            tokens=sum((row["prompt_tokens"] or 0) + (row["completion_tokens"] or 0) for row in ok),
            cost=round(sum(row["cost"] or 0 for row in ok), 4),
        ))
    return summary

Exercise 1 — Solution: The Main Loop

Automate the runs: models × letters × repetitions, raw data to CSV, summary as a table

if __name__ == "__main__":
    models = sys.argv[1:] or DEFAULT_MODELS
    rows = []
    for model in models:
        for candidate in REFERENCE:
            for repetition in range(REPETITIONS):
                rows.append(row := run_once(model, candidate, repetition))
                outcome = row["error"] or f"score {row['score']} in {row['latency']}s"
                print(f"# {model} | {candidate} #{repetition + 1}: {outcome}")

    OUTPUT.parent.mkdir(parents=True, exist_ok=True)
    with OUTPUT.open("w", newline="") as f:  # raw data: one row per run, for further analysis
        writer = csv.DictWriter(f, fieldnames=list(rows[0]))
        writer.writeheader()
        writer.writerows(rows)
    print(f"\nRuns written to {OUTPUT}, reference scores: {REFERENCE}\n")
    print(markdown(summarise(rows)))

Exercise 1 — Solution: Project Structure

Files of this solution, in the lab-snippets repository (master branch):

lab-snippets/
├── data/
│   ├── letter-jean-dupont.txt                # inputs (running example)
│   ├── letter-mario-rossi.txt
│   ├── letter-mohammed-ali.txt
│   └── __init__.py                           # helpers to locate data files
├── snippets/
│   ├── lecture_llmaas/
│   │   └── exercise2/
│   │       └── repl_chat_retry.py            # with_retries (reused)
│   ├── lecture_prompting/
│   │   └── example1/
│   │       └── letter_scoring_openai.py      # prompt and schema (reused)
│   └── lecture_governance/
│       └── exercise1/
│           └── compare_models.py             # the solution
└── pyproject.toml                            # dependencies of all snippets

poetry run python -m snippets -l governance -x 1                       # default shortlist
poetry run python -m snippets -l governance -x 1 MODEL1 MODEL2@URL ...  # custom shortlist

• env vars: OPENAI_API_KEY (and OPENAI_BASE_URL), REPETITIONS (default 5), TEMPERATURE, COMMITTEE_OUTPUT (default output/)

Enveloping: the world adapting to AI

• Machines work well only in environments shaped around them
  • a dishwasher does not wash dishes as we do: we build a box where a simple machine can succeed
  • industrial robots work in cages, robot lawnmowers need perimeter wires, self-driving cars need mapped and well-marked roads
• Enveloping = adapting the environment (and our own behaviour) so that AI can operate in it (cf. Floridi, 2011)
  • digital data, sensors, standard formats, and APIs progressively turn the world into a place where machines can act
• LLMs are agency without intelligence (Floridi, 2023): they succeed because much of our world is already made of text and data
  • recall: intelligence, autonomy, and agency are separable dimensions (genai lecture)

Enveloping in the age of LLMs: examples

• Most of these are reasonable engineering choices…
• … but they change how people write, work, and are evaluated, often without anyone deciding it

Bad vs. good enveloping

Rule of thumb: if the world must adapt to the AI, make the adaptation visible and contestable: an artefact, not a habit

Enveloping in the running example

Recall the running example (PhD admission committee assistant):

The last item is no longer about the world adapting to AI, but about the people using it: see next slides

Responsibility remains human: the general concept

• An AI system cannot be held accountable: it cannot be sanctioned, has no duties, cannot answer for its outputs
  • accountability stays with people and organisations: providers, deployers, the humans in the loop
  • e.g. in the Italian public administration, the official remains solely responsible (L. 132/2025, Art. 14)
• Yet learning, autonomous systems open a responsibility gap: nobody fully controls, nor can fully predict, their behaviour (Matthias, 2004)
• The usual remedy is a human in the loop… who risks becoming a moral crumple zone (Elish, 2019)
  • the human with little actual control absorbs the blame when the system fails, protecting the system itself
  • e.g. the safety driver of a self-driving car, the official signing an AI-drafted decision
• Human oversight works only if the human can actually understand, question, and override the output

Biases in LLM-agent use

• Expertise and practice do not eliminate automation bias (Parasuraman & Manzey, 2010)
• Telling users to “be careful” does not fix overreliance, and some mitigations (e.g. explanations) can even increase it (Passi & Vorvoreanu, 2022)

Automation bias in the law

• The EU AI Act names automation bias explicitly: high-risk systems must enable their overseers (Art. 14(4)):
  • (a) to understand the system’s capacities and limitations, and to monitor it
  • (b) “to remain aware of the possible tendency of automatically relying or over-relying on the output […] (automation bias)”, in particular for systems providing recommendations for decisions taken by people
  • (c) to correctly interpret the output
  • (d) to decide not to use it, or to disregard, override, or reverse its output
  • (e) to interrupt it (a “stop” button)
• Deployers must assign oversight to people with the necessary competence, training, and authority (Art. 26(2)), and support their staff’s AI literacy (Art. 4)
• GDPR (Art. 22): rubber-stamping does not turn an automated decision into a human one
  • human involvement must be meaningful, by someone with the authority and competence to change the decision (WP29 guidelines on automated decision-making, endorsed by the EDPB)

Countermeasures: designing for real oversight

Oversight is a feature to engineer, with its own requirements and tests, not a checkbox

Human oversight in the running example

These choices are part of the human oversight decision of Exercise 2

Exercise 2: A Governance Decision Record for the Running Example (pt. 1)

Goal: the University asks the committee to justify the adoption of the admission assistant, before using it on real applications

Deliverable: a document (e.g. decision-record.md), in snippets/lecture_governance/exercise2/ of lab-snippets

TO-DO List

Write a short (2–4 pages) decision record, answering:

1. Regulatory classification: is the system high-risk under the EU AI Act? Which obligations follow for the University (as provider and/or deployer? cf. Art. 25)? Which GDPR obligations apply to candidates’ documents?
2. Deployment: cloud API, hosted open model, or on-premise? Support the choice with a cost estimate (number of applications per year × tokens per application × price, vs. hardware) and privacy considerations
3. Model: which model? Support the choice with the results of Exercise 1, and with the model’s documentation
   • apply the pre-adoption checklist to the exact checkpoint: license, multimodal restrictions, derivatives, AI Act role
4. Technique: which parts rely on prompting / structured output / tools / RAG? Why not fine-tuning?
5. Human oversight: what do humans review, with which countermeasures to automation bias? Who is accountable?
6. Monitoring: what is logged, and when is the decision re-assessed (e.g. model deprecation, new regulations)?

Exercise 2: A Governance Decision Record for the Running Example (pt. 2)

Decision points and hints

• Take inspiration from Architecture Decision Records (ADRs, cf. https://adr.github.io/): context, options considered, decision, consequences
• Use the five dimensions (quality, cost, control, compliance, time-to-market) as the evaluation criteria for each option, possibly in a table
• For regulation, cite the article of the law supporting each claim
• For costs, state your assumptions explicitly (e.g. tokens per document), and compute a break-even point
• Be explicit about what the system does not decide: e.g. scores as suggestions to the committee, never automatic rejections
• Be explicit about what the process adapts to the assistant (enveloping): e.g. what candidates are told about how their documents are evaluated

How to assess it?

• Could a colleague reproduce your choice from the record alone?
• Would the record survive a question from the University’s Data Protection Officer?
• Does the record say when it should be revised?

Solution: a walkthrough follows in the next (vertical) column — try on your own first!

Exercise 2: A Governance Decision Record for the Running Example — Solution

⚠️ The walkthrough of a possible solution is about to start

• do not proceed until you have attempted the exercise on your own
• press → to skip it, ↓ to see it
• the sample record is on the master branch of lab-snippets (while you cloned the exercises branch): decision-record.md, an ADR-like document (status, context, sections 1–6, consequences)
• below, a summary of its key sections: read the full record for the details (and the article of the law behind each claim)

Exercise 2 — Solution: Regulatory Classification

Decision 1: the system is high-risk, and the University is both its provider and its deployer

Exercise 2 — Solution: Deployment Options and Decision

Decision 2: the five dimensions as criteria; 500 applications/year ≈ 17.5M input + 2.5M output tokens/year

• break-even of C vs. B: centuries (even vs. A: ≈ 80 years) → at this scale, cost is not the deciding factor: compliance and control are
• Decision: B for the pilot, with the same model runnable on-premise (C) should the DPO object; A discarded: pinned snapshots (cf. gpt-4o-2024-08-06 in LLM-as-a-Service) are retired on the vendor’s schedule, forcing re-validation; and data would leave the University
• Decision 3 (model): a decision rule applied to Exercise 1’s table (no failures, lowest error, sd ≤ 0.5, then latency and cost), whose cells are left as [Ex. 1] placeholders, to be filled from your run

Exercise 2 — Solution: Human Oversight

Decision 5: what is reviewed by whom, who is accountable, and how automation bias is countered

Exercise 2 — Solution: Monitoring

Decision 6: what is logged, and when the decision is re-assessed

• logged: model and checkpoint, prompts’ version, tool calls, extractions, scores, human overrides
• override rate monitored in both directions: too many overrides $\rightarrow$ unreliable assistant; too few (or no disagreement in the blind sample) $\rightarrow$ rubber-stamping
• test suites re-run before each admission round, and at every change of prompt, model, or provider
• re-assess when: the model is deprecated or changed, tests fail, overrides exceed 20% or fall below 2% in a round, regulations or guidelines change, the DPO asks, or by the revise-by date

Consequence: the committee’s workload moves from reading to reviewing, and reviews must be real, not rubber stamps

Exercise 2 — Solution: Project Structure

Files of this solution, in the lab-snippets repository (master branch):

lab-snippets/
└── snippets/
    └── lecture_governance/
        ├── exercise1/
        │   └── compare_models.py   # produces the results table (Exercise 1)
        └── exercise2/
            └── decision-record.md  # the decision record (this solution)

• nothing to run: the deliverable is a document
• to fill in its [Ex. 1] placeholders, run poetry run python -m snippets -l governance -x 1, and paste the printed table

Check your understanding

Check your understanding (1/5)

Making and documenting choices

Check your understanding (2/5)

Rules and responsibilities

Check your understanding (3/5)

Openness, licensing, and sovereignty

Check your understanding (4/5)

Choosing techniques and models

Check your understanding (5/5)

Human responsibility and oversight