Using Synthetic Data without Overclaiming

1. Research problem

When do synthetic datasets support legitimate method development but fail substantive inference? The project would examine this question within methodology & evidence, where institutional arrangements, measurable outcomes and the costs of implementation may point in different directions. It is designed as a falsifiable research programme rather than a claim that the proposed mechanism has already been demonstrated.

2. Working proposition

Utility tests and disclosure of generation assumptions can clarify valid and invalid uses. This is a testable proposition, not an established result. Competing explanations and contexts in which the mechanism fails should be specified before data collection begins.

3. Research theses for examination

1. Mechanism: Utility tests and disclosure of generation assumptions can clarify valid and invalid uses.
2. Boundary conditions: Assess whether the proposition remains plausible when jurisdiction, organisational maturity, resource constraints or stakeholder interests change.
3. Disconfirmation: Record which observations would favour an alternative explanation or show that the proposed approach adds no meaningful value.

4. Research design and methods

Benchmark synthetic and real-data properties under approved access protocols. The initial protocol should establish sampling criteria, the unit of analysis, definitions of variables or legal concepts, a transparent source register, and access and ethics approvals where necessary.

5. Evidence horizon and relevant developments

NIST’s Generative AI Profile provides a voluntary risk-management reference. It addresses trustworthiness considerations in developing, deploying and evaluating generative systems rather than certifying specific products.

The OECD’s Digital Government Outlook 2026 finds that digital infrastructure and public AI adoption are progressing but end-to-end service delivery, procurement and algorithmic transparency remain uneven. Availability should not be confused with effective implementation.

The cited publications provide current institutional context and candidate sources for the literature review. They do not certify the proposed hypothesis; original empirical work, careful comparison and jurisdiction-specific validation would still be required.

6. Planned work packages

1. Conceptual groundwork: Define the research object, terminology, competing frameworks and decision context.
2. Evidence mapping: Create an auditable inventory of relevant primary evidence, with dates, definitions, strengths and limitations.
3. Evaluation: Benchmark synthetic and real-data properties under approved access protocols; contrast results with alternative specifications, control cases or rival interpretations.
4. Critical review: Report missing observations, conflicting evidence, implementation costs and circumstances that limit transferability.
5. Translation into practice: Prepare the intended practical output and an independent review checklist.

7. Anticipated research product

Synthetic-data research reporting checklist. The deliverable will be evaluated against the research question and the evidence collected.

8. Limitations, ethics and scholarly integrity

At this stage no original sample, statistical test, clinical cohort or experimental result is claimed. Any future research involving human participants, proprietary data, medical outcomes or regulated activities must obtain the applicable permissions. Publicly accessible official publications are cited for context only; their findings are not represented as findings of BBARTI.

9. Selected institutional reading

• NIST · Generative AI Profile, AI RMF 600-1 ↗26 July 2024
• OECD · Digital Government Outlook 2026 ↗15 June 2026