دیوان · Divan

An open-source site for reading and searching classical Urdu poetry and prose, in Urdu script (ur-PK). Content comes from divan-data: public-domain texts from Urdu Wikisource, with short poet introductions from Urdu Wikipedia.

Divan follows the features of GanjoorService (the software behind ganjoor.net), rebuilt in Node.js, TypeScript and PostgreSQL with its own UI. The first version was a fork of GanjoorService (.NET + SQL Server); that code is kept at the tag dotnet-final.

Layout

Path	What
db/schema.sql	PostgreSQL schema: poets, categories, poems, verses; trigram index for Urdu substring search
api/	Node.js API (Fastify + pg). TypeScript runs natively on Node 24+, no build step
web/	The site (Astro, server-rendered). Naskh by default with a Nastaliq option, light/dark, RTL, mobile-first
reader/index.html	A single-file static reader over the divan-data CDN (no server)

Run locally

# PostgreSQL 17 (any Postgres 14+ works; port 5433 avoids clashing with a local one)
docker run -d --name divan-pg -e POSTGRES_USER=divan -e POSTGRES_PASSWORD=divan_local -e POSTGRES_DB=divan \
  -p 5433:5432 -v divan-pg:/var/lib/postgresql/data postgres:17-alpine

cd api && npm install
npm run import -- ../../divan-data      # a local divan-data checkout, or the CDN:
# npm run import -- https://cdn.jsdelivr.net/gh/anas-rashid/divan-data@main/
npm start                               # API on http://127.0.0.1:4100
npm test                                # Urdu normaliser tests

cd ../web && npm install && npm run build && npm start   # site on http://127.0.0.1:4200

Settings: DATABASE_URL (API, default postgres://divan:divan_local@localhost:5433/divan), PORT/HOST; API_URL (web, default http://127.0.0.1:4100); SITE_HOSTS (web, at build time: the site's hostnames, comma-separated, default 127.0.0.1,localhost; form posts from other origins are refused).

The import upserts, so re-running it after a divan-data sync applies the changes.

Accounts and admin. Readers sign up with an email address and password (no email is sent). The first admin is made on the server: sign up on the site, then npm run make-admin -- you@example.com in api/. Admins manage users at /admin (search, password reset on a reader's request, disable, roles, delete) and see every admin action at /admin/audit. Moderators (L2 junior, L1 senior) get scoped permissions from admins: a scope (all poets, a poet, a book with everything in it, or one work), content types (poets, books, works, dictionary) and actions (create, edit, delete, arrange); can() in api/src/permissions.ts is the one check for moderation.

Moderation. Moderators open ترمیم کریں on a work they may edit, change its Divan text (see docs/content-model.md) with an edit summary and submit it. An L1 moderator covering that work approves, returns (with a reason) or rejects it; an admin publishes. L1 drafts go straight to the admin and an admin's own edits publish directly. Every step is recorded (/mod, the activity log, each work's history at /mod/work/<id>). Admins are super moderators: they edit, review and publish any work without grants. Publishing numbers the version, writes it to divan-data's divan/ folder (DIVAN_DATA_DIR, default ../divan-data), commits it there (the moderator as author by public name, with Reviewed-by: and Approved-by: trailers; DIVAN_GIT_PUSH=1 also pushes), and shows it on the site at once with a link to the commit (DIVAN_DATA_COMMIT_URL, {sha} replaced). Public content never carries email addresses. A draft started before a newer version was published cannot be published. The daily sync rebases on pull so these commits are kept.

Daily content sync (server)

deploy/sync.sh keeps a server current: it updates a divan-data checkout, fetches new and edited works from Wikisource (incremental, about a minute), rebuilds the export and upserts it into PostgreSQL. The site shows new content immediately. Runs are locked so they never overlap.

# crontab -e   (daily at 03:15; DATABASE_URL as for the API)
15 3 * * * DATABASE_URL=postgres://divan:...@localhost:5432/divan /opt/divan/deploy/sync.sh >> /var/log/divan-sync.log 2>&1

The same script keeps the word dictionary current (npm run dict-sync in api/): the full Wiktionary data for Urdu, Persian and Arabic (English Wiktionary via kaikki.org, and the Urdu, Persian and Arabic Wiktionary dumps) is re-imported when upstream publishes new files, and each day's Wiktionary edits are applied from recent changes. The first run downloads about 700 MB.

Settings: DIVAN_DATA_DIR (default /opt/divan-data, cloned on first run), DIVAN_APP_DIR (default: this repo), DIVAN_DATA_PUSH=1 to also commit and push data changes (needs git push access). Needs git, python3 and Node 24+.

API

Endpoint	Returns
GET /api/poets	all poets
GET /api/page?url=/p238/...	the poet, category or poem at a site URL (breadcrumbs, children, verses, prev/next)
GET /api/search?q=&poet=1,2&page=	poems containing all words (or a "quoted phrase"), Urdu-normalised; exact phrase first; each with the best-matching couplet or paragraph (snippet); optionally only some poets/writers; plus authors (who the results come from, with counts), and on page 1 poets (by name) and books (books/chapters by title)
GET /api/word?w=	one word's meanings and pronunciation from the local Wiktionary data (Urdu, Persian, Arabic in that order; English meanings; Urdu equivalents via English when Urdu Wiktionary has none)
/api/auth/*	accounts: sign-up, sign-in (returns a Bearer token), profile, password, delete
/api/mod/*	moderation: what a moderator may do, queue, drafts, save/submit/approve/return/reject/publish, a work's history, activity log
/api/library/*	the signed-in reader's library: toggle poets, works, couplets and words; list with full paths; notes
GET /health	database check

Search normalises both stored text and queries: Arabic ي/ك/ه → Urdu ی/ک/ہ, ۂ/ۓ, diacritics and the Urdu full stop removed; do-chashmi ھ stays distinct.

Roadmap

v2 is reaching parity with Ganjoor's features in phases: reading, accounts, community (comments, bookmarks), editorial tools, recitations, then operations and deployment.

License

GPL-3.0 (see LICENSE). Texts are public domain; the divan-data compilation is CC BY-SA 4.0 (Urdu Wikisource and Wikipedia contributors).