SilverNine and Claude Fable 5
0621687276
fix(search): port the CJK keyword fallback to the Postgres engine (PGLite parity)
...
websearch_to_tsquery('english') can't tokenize CJK, so the keyword arm
of hybrid search degrades badly for Korean/Chinese/Japanese queries on
the Postgres engine — particle-suffixed Korean often matches nothing.
PGLite has had the hasCJK() -> ILIKE + occurrence-count-ranking fallback
since v0.32.7; the Postgres engine never got it.
Port _searchKeywordCJK to postgres-engine.ts at both FTS sites
(searchKeyword dedup path + searchKeywordChunks), keeping the host
engine's conventions (sql.begin + SET LOCAL statement_timeout, named
limit params, its extra-filter set including types/exclude_slugs).
scoreExpr, ILIKE ESCAPE discipline, empty-query guard, and the
dedup/chunk-grain split are copied from PGLite exactly.
Measured on a private production Korean corpus (74k chunks):
particle-suffixed queries returned 7 rows via english FTS vs 152 via
the fallback; a common two-word query 21 vs 847 — a 20-40x recall gap
this closes.
Tests: 6 source-guardrail tests in test/postgres-engine.test.ts +
3 Postgres-vs-PGLite parity e2e tests with synthetic Korean fixtures.
bun run verify 31/31.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-05 12:51:40 +09:00
..
2026-04-18 16:57:38 +08:00
2026-04-09 10:17:13 -07:00
2026-05-07 21:52:58 -07:00
2026-05-06 21:14:34 -07:00
2026-05-09 22:58:19 -07:00
2026-06-21 06:36:43 -07:00
2026-05-24 11:37:03 -07:00
2026-05-07 21:52:58 -07:00
2026-05-22 11:53:11 -07:00
2026-05-27 08:50:53 -07:00
2026-05-23 08:49:28 -07:00
2026-05-06 21:14:34 -07:00
2026-05-11 22:54:35 -07:00
2026-04-29 23:46:36 -07:00
2026-05-14 19:50:50 -04:00
2026-05-24 09:30:57 -07:00
2026-06-03 06:43:11 -07:00
2026-05-23 08:49:28 -07:00
2026-06-03 15:24:00 -07:00
2026-05-23 08:49:28 -07:00
2026-05-26 17:31:48 -07:00
2026-05-10 20:06:31 -07:00
2026-05-09 16:57:47 -07:00
2026-05-07 21:52:58 -07:00
2026-06-01 22:01:14 -07:00
2026-06-03 08:59:39 -07:00
2026-05-09 16:57:47 -07:00
2026-04-30 01:23:29 -07:00
2026-06-03 15:24:00 -07:00
2026-05-09 16:57:47 -07:00
2026-05-08 17:07:51 -07:00
2026-05-25 10:39:09 -07:00
2026-05-30 10:30:56 -07:00
2026-05-14 21:22:10 -07:00
2026-05-18 21:26:12 -07:00
2026-05-18 21:26:12 -07:00
2026-05-07 21:52:58 -07:00
2026-07-05 12:51:40 +09:00
2026-06-01 22:13:19 -07:00
2026-05-01 09:02:43 -07:00
2026-05-17 08:32:03 -07:00
2026-05-24 21:50:08 -07:00
2026-05-18 21:26:12 -07:00
2026-05-10 06:34:40 -07:00
2026-05-25 15:10:23 -07:00
2026-05-09 16:57:47 -07:00
2026-05-09 16:57:47 -07:00
2026-05-11 19:25:48 -07:00
2026-05-10 12:49:14 -07:00
2026-05-09 16:57:47 -07:00
2026-05-09 16:57:47 -07:00
2026-05-24 11:37:03 -07:00
2026-04-26 20:45:05 -07:00
2026-06-02 16:33:06 -07:00
2026-05-14 19:13:14 -07:00
2026-05-23 10:01:08 -07:00
2026-05-23 10:01:08 -07:00
2026-06-14 09:32:58 -07:00
2026-05-09 16:57:47 -07:00
2026-05-25 14:43:12 -07:00
2026-06-03 15:24:00 -07:00
2026-05-21 18:57:05 -07:00
2026-05-11 23:02:03 -07:00
2026-05-24 11:37:03 -07:00
2026-06-10 22:00:36 -07:00
2026-05-09 16:57:47 -07:00
2026-05-22 21:04:30 -07:00
2026-05-07 21:52:58 -07:00
2026-04-08 23:26:11 -10:00
2026-05-25 12:48:55 -07:00
2026-05-09 16:57:47 -07:00
2026-05-09 20:46:34 -07:00
2026-05-09 16:57:47 -07:00
2026-05-10 12:49:14 -07:00
2026-05-10 12:49:14 -07:00
2026-05-24 11:37:03 -07:00
2026-05-09 16:57:47 -07:00
2026-05-24 11:37:03 -07:00
2026-05-24 11:37:03 -07:00
2026-05-24 11:37:03 -07:00
2026-06-21 06:36:43 -07:00
2026-05-24 11:37:03 -07:00
2026-05-19 13:12:40 -07:00
2026-05-09 16:57:47 -07:00
2026-05-22 21:04:30 -07:00
2026-05-07 21:52:58 -07:00
2026-05-22 13:38:07 -07:00
2026-05-09 16:57:47 -07:00
2026-06-03 07:09:20 -07:00
2026-05-31 08:20:25 -07:00
2026-06-24 06:05:16 -07:00
2026-05-11 21:49:57 -07:00
2026-05-11 21:49:57 -07:00
2026-04-23 19:34:27 -07:00
2026-05-25 14:56:38 -07:00
2026-06-14 09:32:58 -07:00
2026-06-12 07:28:13 -07:00
2026-05-23 10:21:59 -07:00
2026-07-02 08:00:58 -07:00
2026-06-03 08:59:39 -07:00
2026-05-09 16:57:47 -07:00
2026-06-03 08:42:25 -07:00
2026-06-01 23:03:09 -07:00
2026-05-07 21:52:58 -07:00
2026-05-07 21:52:58 -07:00
2026-05-22 13:38:07 -07:00
2026-05-20 09:19:18 -07:00
2026-06-03 06:28:21 -07:00
2026-04-22 01:34:22 -07:00
2026-04-30 23:17:54 -07:00
2026-06-03 06:20:03 -07:00
2026-06-03 06:20:03 -07:00
2026-05-22 21:18:09 -07:00
2026-05-09 16:57:47 -07:00
2026-05-18 20:55:57 -07:00
2026-06-03 15:24:00 -07:00
2026-05-20 13:55:12 -07:00
2026-06-03 05:50:21 -07:00
2026-05-31 08:20:25 -07:00
2026-05-18 12:53:01 -07:00
2026-05-19 18:12:01 -07:00
2026-05-09 16:57:47 -07:00
2026-05-14 20:15:29 -07:00
2026-05-14 19:50:50 -04:00
2026-05-09 16:57:47 -07:00
2026-06-21 06:36:43 -07:00
2026-04-29 22:21:07 -07:00
2026-05-22 09:10:20 -07:00
2026-05-22 09:10:20 -07:00
2026-05-14 19:50:50 -04:00
2026-05-11 22:54:35 -07:00
2026-05-25 14:43:12 -07:00
2026-06-03 15:24:00 -07:00
2026-05-26 16:58:18 -07:00
2026-05-30 10:30:56 -07:00
2026-06-07 19:22:33 -07:00
2026-05-11 19:25:48 -07:00
2026-05-07 23:07:00 -07:00
2026-05-07 23:07:00 -07:00
2026-05-10 06:34:40 -07:00
2026-05-22 13:38:07 -07:00
2026-05-22 22:37:15 -07:00
2026-05-27 07:01:28 -07:00
2026-04-09 12:25:04 -10:00
2026-05-21 22:11:00 -07:00
2026-05-07 21:52:58 -07:00
2026-05-09 20:46:34 -07:00
2026-05-19 17:14:53 -07:00
2026-06-14 09:32:58 -07:00
2026-05-24 11:37:03 -07:00
2026-05-12 14:33:29 -07:00
2026-04-26 15:49:48 -07:00
2026-05-17 08:00:29 -07:00
2026-05-24 11:37:03 -07:00