anas/divan
1
0
Fork 0
Code Issues 12 Pull Requests Actions Packages Projects Releases Wiki Activity

Make search normalisation Urdu-aware #3

New Issue
Closed
opened 2026-10-04 22:04:31 +00:00 by anas · 2 comments
anas commented 2026-10-04 22:04:31 +00:00
Owner
Copy Link

Scope: LanguageUtils.MakeTextSearchable and query normalisation: Urdu full stop, Arabic/Urdu letter variants, Urdu diacritics; re-index stored searchable text if needed.

Acceptance criteria: Test words with and without diacritics or variant letters return the same results.

Automated validation: Self-check for the normaliser; build.

Manual validation: Search known couplets on the local site.

Dependencies or blockers: local import complete

Tracked in titus-ai projects/divan/TASKS.md.

**Scope:** `LanguageUtils.MakeTextSearchable` and query normalisation: Urdu full stop, Arabic/Urdu letter variants, Urdu diacritics; re-index stored searchable text if needed. **Acceptance criteria:** Test words with and without diacritics or variant letters return the same results. **Automated validation:** Self-check for the normaliser; build. **Manual validation:** Search known couplets on the local site. **Dependencies or blockers:** local import complete _Tracked in titus-ai `projects/divan/TASKS.md`._
anas added this to the Phase 2: Reading experience milestone 2026-10-04 22:04:31 +00:00
anas added the
area/api
i18n/urdu
search
 labels 2026-10-04 22:04:32 +00:00
anas commented 2026-10-04 22:24:59 +00:00
Author
Owner
Copy Link

Progress (9bd44561): search used SQL Server full-text, which the Linux SQL Server image does not ship (search returned 500). Replaced with normalised LIKE patterns; Urdu normaliser. Normaliser self-check passed. Pending: runtime check after API restart + POST /api/divan/regenplaintext/0 for data imported before the change.

Progress (9bd44561): search used SQL Server full-text, which the Linux SQL Server image does not ship (search returned 500). Replaced with normalised LIKE patterns; Urdu normaliser. Normaliser self-check passed. Pending: runtime check after API restart + `POST /api/divan/regenplaintext/0` for data imported before the change.
anas commented 2026-10-08 17:50:24 +00:00
Author
Owner
Copy Link

Verified at runtime on a fresh import: API search HTTP 200 with results for دل and دلِ (diacritic normalised); site search page HTTP 200. Closing.

Verified at runtime on a fresh import: API search HTTP 200 with results for دل and دلِ (diacritic normalised); site search page HTTP 200. Closing.
anas closed this issue 2026-10-08 17:50:24 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
area/api

   
area/data

   
area/infra

   
area/site

   
decision

   
deferred

   
i18n/urdu

   
search

   
v2
No Label
area/api
area/data
area/infra
area/site
decision
deferred
i18n/urdu
search
v2
Milestone
Clear milestone
No items
No Milestone
Phase 2: Reading experience
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: anas/divan#3
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?