What type should I take for text that can be any charset? (C++)

Question

0.00/5 (No votes)

See more:

I have methods that currently take const char* and an encoding. Like so:

C++

foo(const char* text,text_encoding::utf8)

That's fine for utf8 and latin1, etc
But what if you want utf16 (wchar_t?) or utf32 (no intrinsic type, but i use int32_t)

What if there are others, like some kind of binary businesses that's utf8? My encoders don't care - they take a void*

Should I take a const void* here? I really hate it. But I also hate the idea of limiting the types it can be via a union because I just don't know what types of encodings people may want to use down the road.

I mean, I could do like typedef void* encoded_text_t; (I don't like that name, but I'd take suggestions).

Does anyone have a better idea?

Keep in mind, the STL is off limits to me. I'm on embedded.

What I have tried:

not really applicable here.

I've put everything in the first box.

Posted 16-Aug-24 7:15am

honey the codewitch

Add a Solution

Comments

PIEBALDconsult 16-Aug-24 14:00pm

No matter what you do, someone will still try to pass you a wombat. :D

honey the codewitch 16-Aug-24 14:03pm

Right? If this was LVGL I'd just take void* and call it a day, because that's how they handle every situation like this, but that's straight C, not C++, and also it squicks me out. I just don't like taking void* for what is essentially text. I could live with typedefing it if there are no other good options, but I need a name. text_t? nah. string_t? too confusing (std::string exists) meh.

2 solutions

Add a Solution

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Iids · Answer 1 · 2024-08-17T01:42:00

Solution 1

To store text that can be any charset in C++, you should use std::string.

This data type is designed to handle strings more flexibly compared to the char data type which can only store one character.

Posted 17-Aug-24 1:42am

Iids

Comments

honey the codewitch 17-Aug-24 10:22am

A) Keep in mind, the STL is off limits to me. I'm on embedded.

C) can std::string produce utf32 codepoints? No it cannot

D) Just no

Richard MacCutchan 17-Aug-24 10:24am

OP clearly stated that she cannot use STL.

honey the codewitch 17-Aug-24 11:05am

I've settled on the typedef for now. It's less than ideal. I'm considering a union, but that gets weirder to use than a cast. A void pointer theoretically works, and obviates the need for a cast when passing in the argument - I just hate it.

Richard MacCutchan 17-Aug-24 11:12am

Do you ever get that feeling that just as you reach what you thought was the end of the rabbit hole, there is a bend which leads down into deeper darkness? :(

honey the codewitch 17-Aug-24 11:12am

I'm literally right there now, and I just posted in the lounge about it. LOL

KarstenK 19-Aug-24 3:44am

Be aware that multibyte string are using double space and needing other function. So better stick to my advice or risk a lot of trouble.

honey the codewitch 19-Aug-24 9:10am

I'm not using tchar or wchar_t or C or C++ functions to deal with multibyte characters.

I have encoder classes that take a void* and a length, and they return a series of UTF32 codepoints. Whether it's latin1, utf8, or something else. No wchar needed or wanted.

KarstenK 20-Aug-24 7:24am

stick to my solution because it also handles this problem. Type casting is evil when there isnt any check after it but the machine code. :-/

honey the codewitch 20-Aug-24 7:26am

Absolutely not. Your solution is Microsoft specific. Your solution does not yield UTF32 codepoints. In fact, your solution can barely handle any encodings at all. It's not a good solution.

KarstenK · Answer 2 · 2024-08-17T10:20:00

Solution 2

Depending on your library and compiler you should use some macro for the character identifier. Microsoft is using therefor the

C++

TCHAR

macro which can resolve to char or wchar_t depending on the compiler settings.
That need some change in most string functions like using

C++

_tcscpy

instead of strcpy.

So some string library which can handle it would be very handy.

Posted 17-Aug-24 10:20am

KarstenK

Comments

honey the codewitch 17-Aug-24 16:42pm

oof. I'm trying to keep things C++ wherever possible. I do use the preprocessor, but seldomly. I could use a macro but I don't think that puts me in a better position than a typedef, TBH.

KarstenK 19-Aug-24 3:42am

this is some typedef but it also is some typedef for the API like the strcpy. So it works!!!

What type should I take for text that can be any charset? (C++)

2 solutions

Solution 1

Solution 2

Add your solution here

Preview 0

What type should I take for text that can be any charset? (C++)

2 solutions

Solution 1

Solution 2

Add your solution here

Preview 0

Existing Members

...or Join us