anas/divan
1
0
Fork 0
Code Issues 12 Pull Requests Actions Packages Projects Releases Wiki Activity

Research: content model and moderation (no wiki engine embedded; Divan text format) #49

Merged
anas merged 3 commits from research/content-model into main 2026-10-08 22:16:18 +00:00
Conversation 0 Commits 3 Files Changed 5 +404
anas commented 2026-10-08 22:07:55 +00:00
Owner
Copy Link

Closes #38. Step 5 of the plan. A research PR: please read docs/content-model.md; the code is a tested prototype that later steps build on.

Question: moderation like a wiki, with tags for what a couplet, stanza, chapter and word are, and whether an open-source wiki content-management module can be integrated.

Finding (checked October 2026): no wiki engine is a good fit to embed.

  • MediaWiki:
    • a second PHP stack with its own users and permissions;
    • FlaggedRevs is "not recommended for production use" (its own documentation);
    • Approved Revs has one approval level only, not L2 → L1 → admin;
    • content is unstructured wikitext.
  • Wiki.js (v3 still beta), BookStack, Outline (not open source), Docmost and DokuWiki: no multi-level review, no git publishing, no verse structure.
  • Useful parts:
    • wikiparser-node (GPL-3.0) for Wikisource;
    • Tiptap 3 core / ProseMirror (MIT) for a structured Urdu editor;
    • CodeMirror 6 for a source view;
    • TEI P5 as an export;
    • Ganjoor's correction records as the reference for review data.

Recommendation

  1. Divan text: a small, Wikisource-compatible markup as the published source in divan-data, one file per work.

    • {{دیوان | …}} for data points;
    • <poem>: 2 lines = couplet, 1 = misra, 3+ = stanza;
    • == … == for chapters;
    • [[word]] for dictionary links, <ref> for footnotes, {{نسخہ|…}} for variant readings.

    Git diffs stay readable: a one-word fix in a misra is a one-line diff (shown in the doc against the JSON export).

  2. The moderation workflow (drafts, L2 → L1 → admin, comments) lives in PostgreSQL. Publishing commits the .dtx file with Reviewed-by/Approved-by trailers.

  3. Editors: a Tiptap structured Urdu editor and a CodeMirror source view.

  4. Wikisource sync through wikiparser-node; Divan-published works are never overwritten.

  5. TEI export (a couplet is <lg type="sher"> with two <l>).

Prototype (api/src/divantext.ts, tests in divantext.test.ts, samples in docs/content-model/): parses Divan text, converts it to the site's verse JSON, and exports TEI.

  • Same text for readers: on the real Ghalib ghazal, the output is identical to today's export (all 22 lines).
  • Wikisource compatibility: Wikisource markup reads as-is.
  • Prose: Sir Syed's chapter parses into a heading and paragraphs with a footnote.
  • TEI: validates with xmllint.
  • npm test: 32 pass.

Decisions for the owner: accept Divan text as the published format and Tiptap as the editor base. The next steps (#30, #31, #32, #41, #28, #34) follow the doc's "Effect on the plan".

🤖 Generated with Claude Code

Closes #38. Step 5 of the plan. **A research PR: please read `docs/content-model.md`**; the code is a tested prototype that later steps build on. **Question**: moderation like a wiki, with tags for what a couplet, stanza, chapter and word are, and whether an open-source wiki content-management module can be integrated. **Finding** (checked October 2026): no wiki engine is a good fit to embed. - **MediaWiki**: - a second PHP stack with its own users and permissions; - FlaggedRevs is "not recommended for production use" (its own documentation); - Approved Revs has one approval level only, not L2 → L1 → admin; - content is unstructured wikitext. - **Wiki.js** (v3 still beta), **BookStack**, **Outline** (not open source), **Docmost** and **DokuWiki**: no multi-level review, no git publishing, no verse structure. - **Useful parts**: - **wikiparser-node** (GPL-3.0) for Wikisource; - **Tiptap 3 core / ProseMirror** (MIT) for a structured Urdu editor; - **CodeMirror 6** for a source view; - **TEI P5** as an export; - **Ganjoor's correction records** as the reference for review data. **Recommendation** 1. **Divan text**: a small, Wikisource-compatible markup as the published source in divan-data, one file per work. - `{{دیوان | …}}` for data points; - `<poem>`: 2 lines = couplet, 1 = misra, 3+ = stanza; - `== … ==` for chapters; - `[[word]]` for dictionary links, `<ref>` for footnotes, `{{نسخہ|…}}` for variant readings. Git diffs stay readable: a one-word fix in a misra is a one-line diff (shown in the doc against the JSON export). 2. The moderation workflow (drafts, L2 → L1 → admin, comments) lives in PostgreSQL. Publishing commits the `.dtx` file with Reviewed-by/Approved-by trailers. 3. Editors: a Tiptap structured Urdu editor and a CodeMirror source view. 4. Wikisource sync through wikiparser-node; Divan-published works are never overwritten. 5. TEI export (a couplet is `<lg type="sher">` with two `<l>`). **Prototype** (`api/src/divantext.ts`, tests in `divantext.test.ts`, samples in `docs/content-model/`): parses Divan text, converts it to the site's verse JSON, and exports TEI. - **Same text for readers**: on the real Ghalib ghazal, the output is **identical to today's export** (all 22 lines). - **Wikisource compatibility**: Wikisource markup reads as-is. - **Prose**: Sir Syed's chapter parses into a heading and paragraphs with a footnote. - **TEI**: validates with xmllint. - `npm test`: 32 pass. **Decisions for the owner**: accept Divan text as the published format and Tiptap as the editor base. The next steps (#30, #31, #32, #41, #28, #34) follow the doc's "Effect on the plan". 🤖 Generated with [Claude Code](https://claude.com/claude-code)
anas added 1 commit 2026-10-08 22:07:56 +00:00 
docs/content-model.md compares MediaWiki (+FlaggedRevs/Approved Revs, ProofreadPage, SMW, Wikibase),
Wiki.js, BookStack, Outline, Docmost, DokuWiki, wikiparser-node, Tiptap/ProseMirror/CodeMirror, TEI P5
and Ganjoor's correction model, and recommends: Divan text (a Wikisource-compatible markup subset)
as the published source in divan-data, one file per work; moderation workflow in PostgreSQL; Tiptap
editor with a source view; wikiparser-node for Wikisource sync; TEI export.

Prototype api/src/divantext.ts (parse, verse JSON, TEI) with tests on a real Ghalib ghazal and Sir
Syed chapter: identical verses to today's export, Wikisource markup reads as-is, TEI validates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anas added 1 commit 2026-10-08 22:16:01 +00:00
anas added 1 commit 2026-10-08 22:16:15 +00:00
anas merged commit a9626b4d9e into main 2026-10-08 22:16:18 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
area/api

   
area/data

   
area/infra

   
area/site

   
decision

   
deferred

   
i18n/urdu

   
search

   
v2
No Label
area/api
area/data
area/infra
area/site
decision
deferred
i18n/urdu
search
v2
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: anas/divan#49
No description provided.
Delete Branch "research/content-model"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?