"ASCII-fying" DMRId database

[SA0ASM]

Hi!

I've run some experiments trying to find a better way of converting the data in the DMRId.net user database to the limited (ASCII alphanumeric?) set of characters used in the "compressed" OpenGD77 in-radio format. The current "RemoveDiactritics" function isn't great at converting non-latin characters to ASCII (e.g. the greek alphabet), and I thought I would contribute a substitute that does a better job.

The attached program, when run against the user.csv database file, successfully ASCII-fies all but ~500 entries. Those entries are mostly asian scripts that cannot easily be alphabetized, with some instances of incorrect UTF-8 (e.g. double-encoded german characters). Those characters are ignored/removed since they cannot be displayed by the radio anyway.

I hope that this could be included in a future CPS release, to better reflect european languages in the limited available character set.

73 de SA0ASM

[VK3KYY]

Hi!

I've run some experiments trying to find a better way of converting the data in the DMRId.net user database to the limited (ASCII alphanumeric?) set of characters used in the "compressed" OpenGD77 in-radio format. The current "RemoveDiactritics" function isn't great at converting non-latin characters to ASCII (e.g. the greek alphabet), and I thought I would contribute a substitute that does a better job.

The attached program, when run against the user.csv database file, successfully ASCII-fies all but ~500 entries. Those entries are mostly asian scripts that cannot easily be alphabetized, with some instances of incorrect UTF-8 (e.g. double-encoded german characters). Those characters are ignored/removed since they cannot be displayed by the radio anyway.

I hope that this could be included in a future CPS release, to better reflect european languages in the limited available character set.

73 de SA0ASM

Thanks

I will look at your conversion method

BTW. We know that Radio is.net does not do any validation or sanity checks on the data. I noticed a lot of mistakes in the data people entered for their details etc.
I doubt if any DMR radio is capable of displaying all data from RadioID.Net correctly, unless perhaps it's one of those Android phone based devices

Thanks again

VK3KYY

[VK3KYY]

FYI

I'm swamped with other work in the project at the moment, including trying to resolve the serious problem with the 10W version of the UV380 PLUS not working at all with the firmware and people potentially having bricked their radios.
We also also trying to resolve the problem with many of the translations being incomplete

So this improvement will need to wait until I have some spare time