Peer review decides how competitive research funding is allocated, yet the judgments it produces are rarely observable at scale. Large language models have meanwhile become capable enough that funding bodies and journals have begun to consider them as reviewing aids, while evidence on how a model’s assessment of a full research proposal compares with the assessments human experts actually produce remains thin — not on stylised excerpts, but on complete applications judged under a real instrument with real consequences for the applicants. This project addresses that gap. Under a signed confidentiality statement with the Swedish Foundation for Strategic Research (SSF), we have been given access to approximately 5,800 research funding applications together with the reviewer reports they received. Each application is a complete submission of some 40 pages: cover metadata, budget, the principal investigator’s CV and publication list, CVs and publications of co-applicants, a detailed research plan with references, a statement of strategic relevance, and letters of intent and recommendation. Every application was assessed independently by four to five reviewers, giving on the order of 25,000 human reviews. The reviews follow one structured instrument: free-text assessments of the applicant’s previous accomplishments and network, of the originality and feasibility of the research plan, and of strategic relevance; a mark for each of those three dimensions and an overall mark; a judgment of whether the application corresponds to the announcement; a ranking; and the reviewer’s own statement of expertise. Complete applications, several independent expert reviews of each, and a fixed scoring instrument together make a controlled comparison possible in a setting where it usually is not. We will have an open-weights large language model review each application under exactly the instrument the human reviewers used, with constrained decoding so that the model returns the same marks, the same conformity judgment and the same ranking in the same format. Three questions follow. First, how closely do the model’s assessments agree with the reviewers’, dimension by dimension and overall? Second — the benchmark that makes the first question interpretable — how does that agreement compare with the agreement among the human reviewers themselves, who are known to diverge considerably on the same proposal? Third, how stable is the model’s own judgment: by generating several independent reviews per application we measure its self-consistency, the direct counterpart of human inter-rater reliability. Where the model’s written reasoning departs from the reviewers’, we characterise how it departs. A single pass over 5,800 applications is roughly 145 million tokens of prompt and some 17 million tokens of generated review; repeated draws per application, and comparison across model sizes, multiply that several times. Inference at this scale needs GPU nodes, and it must run on infrastructure we control: the confidentiality statement prohibits disclosing the documents or any part of them, which rules out every commercial model service. SSF has specified that the models must be downloaded and run within firewalls.