Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

If anything, the word STRASSE is a great example why case-folding followed by string comparison is not the same as a case-independent string comparison

Or if you really want to do it, your case-folding needs to map certain lowercase characters to to other lowercase characters (lowercase ß to ss, lowercase ı to i, etc), losing some of the meaning



> If anything, the word STRASSE is a great example why case-folding followed by string comparison is not the same as a case-independent string comparison

s/case-folding/lowercasing/

Proper Unicode case-folding absolutely does map ß to ss, ς to σ, etc. Moreover, there are some scripts (IIRC Georgian) where for historical reasons case-folding yields uppercase letters, not lowercase ones. The case-folding mapping is specifically designed in concert with the comparison rules to yield the same result, that’s why it’s a separate operation from lowercasing.

(I believe the thing described in TFA is supposed to be proper case-folding in that sense, but given TFA is AI-written I wouldn’t trust its descriptions either way.)

That said, if you want to match the sort order customary in a specific language, you need to use language-specific rules for producing collation keys rather than generic case-folding. There’s no way out of this because different users of e.g. the Latin alphabet want contradictory results. And if you think you do want generic casefolding, then you probably actually want NFKC_Casefold instead unless your input is pre-normalized.


In some cases your're also lost ng all of the meaning. For example: aß/Ass, Maß/Mass, Buße/Busse, Floß/floss




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: