AI metascientist: a method for structured, guided analysis of research systems

A method that uses AI to decompose, route, and structure the analysis of research and knowledge production.

What it is

AI metascientist is a method for analysing and evaluating research systems through AI-mediated, structured reasoning. It operates at the level of knowledge production rather than individual outputs alone, treating research as a system of artefacts, practices, and relationships.

The method does not replace researchers or analysts. Its role is to formalise and carry methodological and technical knowledge within the analytical process. AI is used to structure reasoning, guide analytical choices, and make intermediate steps explicit, allowing complex analyses to be conducted in a more accessible and inspectable way.

How it works

The method is organised into a set of coordinated stages:

• decomposition of research questions or outputs into analysable units
• routing to appropriate analytical workflows based on context and intent
• application of structured evaluation steps aligned with domain-specific norms
• aggregation of intermediate results into interpretable outputs
• encoding of the analytical process as a reproducible workflow

These stages are supported by contextual information, such as institutional data, prior analyses, or defined peer groups, which can persist across analyses. The outcome is not a single answer but a structured analytical process that can be inspected, repeated, and adapted.

Full implementation details, including workflows and internal logic, are not public.

What it operates on

The method operates on knowledge artefacts and systems, including:

• publications and other scholarly outputs
• datasets and derived indicators
• institutional or strategic documents
• AI-generated content and analytical results

It is not limited to text. It applies to structured and semi-structured representations of research activity and evaluation.

Why it is needed

Traditional bibliometric approaches are constrained by fixed indicators and often require substantial methodological expertise to apply appropriately. At the same time, unstructured use of AI tends to produce outputs that are difficult to reproduce, inspect, or validate.

AI metascientist addresses this gap by combining structured evaluation with AI-mediated reasoning. It enables analyses that are methodologically grounded while making their underlying steps more visible and repeatable.

Example

A set of institutional research outputs is analysed to assess international reach.

The method:

• identifies relevant outputs and constructs a corpus
• applies field-appropriate norms to account for disciplinary differences
• evaluates patterns of collaboration and dissemination
• produces a structured summary of international engagement across units

The output is a multi-part result showing differences across fields and units, together with the steps used to derive them.

This example is simplified and does not represent full implementation.

Interpretation

Outputs are structured and process-based rather than definitive conclusions. They support reasoning by making analytical steps, assumptions, and methodological choices visible. Results are intended to be inspected, compared, and reused, not treated as final or authoritative on their own.

Scope and limits

The method depends on the quality and completeness of input data, as well as on the validity of the methodological assumptions embedded within it. AI-mediated reasoning introduces risks of misinterpretation or inappropriate method selection when context is incomplete or ambiguous.

While the method structures and guides analysis, it does not remove the need for expert judgement. Outputs may be over-interpreted if they are treated as definitive rather than conditional on their inputs and assumptions.

Related work

The method connects to frameworks such as GRID+, which decompose evaluation into explicit dimensions, and to tools such as AVA, which support structured validation of outputs. It also aligns with work that treats analytical workflows as reproducible processes rather than one-off results.

Depth

• overview: available
• methodological detail: partial
• full protocol and implementation: not public