8 Commits

Author	SHA1	Message	Date
Kishore Nallan	1d1712f391	Refactor tokenizer to use index, skip and separate logic.	2021-04-16 17:55:52 +05:30
kishorenc	3a92685967	Integrate with Kakasi.	2021-04-05 12:25:50 +05:30
kishorenc	dd72e2a78c	Introduce field level locale.	2021-04-02 21:28:49 +05:30
kishorenc	c2eec85277	Fix highlighting of strings with special characters.	2021-03-20 12:58:30 +05:30
kishorenc	f501b137b7	Tokenize on special characters.	2021-03-16 11:39:53 +05:30
kishorenc	a912a250ff	Fix bad unicode characters in highlight snippet.	2020-12-28 19:19:59 +05:30
kishorenc	9533b73609	Fixed a few higlighting/splitting edge cases.	2020-11-17 20:10:34 +05:30
kishorenc	6997e35f72	Combine various token operations in a single flow. ... Splitting, normalizing etc. are now done in a single loop.	2020-11-17 20:10:34 +05:30