← Back to project catalogue
GP-DA-1GYWHUIData and AIOpen for request

Indic document intelligence benchmark

A reproducible benchmark for comparing OCR, information retrieval, and question-answering systems on multilingual Indian administrative documents.

  • Nuxt
  • Vue
  • FastAPI
  • PostgreSQL
  • Python
  • PyTorch
  • Docker

Project definition

Problem statement

Indian administrative documents contain multiple languages, varied scripts, scanned pages, tables, stamps, and inconsistent layouts. A model that works on clean English text may perform poorly on these documents.

The engineering problem is to build a controlled benchmark that separates OCR, retrieval, and question-answering performance so that systems can be compared using the same documents, questions, ground truth, and metrics.

Project objectives

  • Prepare a versioned corpus of permitted multilingual administrative documents.
  • Create ground truth for page text, document fields, retrieval passages, and question answers.
  • Compare selected OCR, retrieval, and question-answering approaches under identical conditions.
  • Measure results by language, script, document type, scan quality, and pipeline stage.
  • Make every experiment reproducible from a saved configuration and dataset version.

System design

System modules

01

Corpus manager

Imports permitted documents, records language and layout metadata, and maintains fixed training, validation, and test splits.

02

Annotation workspace

Supports page transcription, field marking, relevant-passage selection, answer entry, and reviewer agreement checks.

03

OCR runner

Runs selected OCR engines through a common adapter and stores text, confidence, layout, latency, and failure information.

04

Retrieval runner

Indexes extracted text, executes prepared queries, and records ranked passages for lexical and semantic retrieval methods.

05

Question-answering runner

Produces answers from retrieved evidence and preserves the passages used for later scoring and review.

06

Evaluation dashboard

Calculates metrics, compares systems, filters errors, and exports tables and experiment records.

Methodology

System workflow

  1. 01
    Prepare corpus

    Documents are checked for permitted use, classified, versioned, and assigned to fixed dataset splits.

  2. 02
    Create ground truth

    Annotators transcribe selected pages and prepare fields, questions, answers, and relevant passages using written guidelines.

  3. 03
    Run pipelines

    Selected OCR, retrieval, and question-answering configurations process the same frozen test set.

  4. 04
    Calculate metrics

    The system scores each stage and groups results by language, document type, scan quality, and model.

  5. 05
    Analyse errors

    Users inspect substitutions, missed fields, failed retrievals, unsupported answers, and pipeline-level error propagation.

Demonstration scenario

A prepared set of English, Hindi, and Kannada administrative documents is processed by two OCR configurations. The extracted text is indexed by two retrieval methods and used to answer a fixed question set. The dashboard compares stage-level metrics and traces one failed answer back to an OCR or retrieval error.

Engineering

Technical architecture

Web application
Nuxt and Vue for corpus review, annotation, experiment setup, result comparison, and error inspection.
Evaluation API
FastAPI for dataset access, model jobs, metric calculation, experiment state, and export endpoints.
Data layer
PostgreSQL for document metadata, annotations, dataset versions, experiment configurations, predictions, and scores.
Model adapters
Python interfaces that allow OCR, retrieval, and question-answering components to be replaced without changing the benchmark.
Evaluation engine
Stage-specific metrics with language-level grouping, confidence intervals, saved seeds, and fixed test splits.
Reproducibility
Docker-based setup with dataset manifests, configuration files, model version records, and exportable experiment results.

Testing

Evaluation

Evaluation measures

  • Character and word error rates for OCR output
  • Field extraction precision, recall, and F1 score
  • Recall at k and mean reciprocal rank for retrieval
  • Exact match, token F1, and evidence correctness for answers
  • Performance differences across selected languages and document conditions
  • Runtime, failure rate, and reproducibility across repeated experiments

System boundaries

  • The corpus uses public, licensed, synthetic, or explicitly permitted documents only.
  • The benchmark does not contain private student, citizen, or employee records.
  • Results apply only to the documented dataset, languages, and model versions.
  • The system evaluates document intelligence methods and does not make administrative decisions.

Included

  1. 01Document corpus and ground-truth management interface
  2. 02Replaceable OCR, retrieval, and question-answering pipelines
  3. 03Metric calculation and model-comparison dashboard
  4. 04Prepared benchmark dataset, experiments, and error analysis
  5. 05Complete source code in a private GitHub repository
  6. 06Complete project documentation with synopsis, abstract, methodology, processing diagrams, experiment results, screenshots, and conclusion
  7. 07Setup and usage guide

Project record

No buyer information is collected on this page.

Permanent project ID
GP-DA-1GYWHUI
Catalogued
21 Aug 2026
Completed
Pending
Verified
Pending
Demonstration
Added when ready

Handover

After purchase

  1. 01
    Payment is confirmed

    The project is marked unavailable and cannot be purchased again.

  2. 02
    Repository access is granted

    The buyer's submitted GitHub account receives access to the private repository.

  3. 03
    The purchase record is delivered

    The certification sheet is prepared from the reviewed buyer details and sent privately by email.