Please come with suggestions for how I should version my registers? - Design and Architecture Discussion Boards

How is that trolling? As I said i made a couple of suggestions which you were free to ignore. I get the distinct impression (reading your other threads above) that you only came here for a fight.

Are you looking for something like protobuf?

Protocol Buffers | Google Developers[^]:
Protocol buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data.

"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer

How extensible are they? As far I can see they are at a "structure level" (I'm not sure if there is proper term for it; by "structure level" I mean extending some language with structures like in C in place of a type (joining few types together), for example (pseudocode): `qux: struct {foo: string, baz: integer}; qux new-variable = struct {foo: "***", baz: 42}`) or are they extensible at deeper level (parsing types, e.g. `new-type: <integer><"-"><integer>; new-type new-variable = 2-3)?

You having trouble extending a text-format?

Whatever uni you represent, I'll come take a piss on them. I'll even pay for it myself.

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

Eddy. You're Dutch, not German, so you have no business paying for scat fetishes. Poke tongue | ;-P

Robust Services Core | Software Techniques for Lemmings | Articles

Living @500 m from Germany, dating German.

Scat fetishes? The hospital threaneted with a transplant.

So wanna talk sh*t? I got one of her books Big Grin | :-D

--edit
I do. If I could, I'd sh*t on ya all day.

Chaoshit: Amazon.co.uk: Shitonya: 9789048400065: Books[^]

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

Dating a German? Say no more! Like going to the doctor: "Open your mouth and say ahhh" Laugh | :laugh:

Robust Services Core | Software Techniques for Lemmings | Articles

I am working with several products that communicate with each other through registers and when a new firmware release is made in any of the products, registers could have been added or discontinued. I was thinking of having something like this:

h-file:

#ifndef DONT_KNOW_IF_REG_IS_STILL_SUPPORTED_YOU_HAVE_TO_ASK
#define DONT_KNOW_IF_REG_IS_STILL_SUPPORTED_YOU_HAVE_TO_ASK    (0xFF)
#endif
#ifndef REG_IS_NO_LONGER_SUPPORTED
#define REG_IS_NO_LONGER_SUPPORTED                             (0)
#endif
#ifndef REG_IS_STILL_SUPPORTED
#define REG_IS_STILL_SUPPORTED                                 (1)
#endif

typedef struct __attribute__((__packed__)) {
    uint8_t versionMajorFunc;
    uint8_t versionMinorFunc;
    uint8_t versionMajorBug;
    uint8_t versionMinorBug;
    uint8_t variant;
} versionAndVariant_s;

typedef struct __attribute__((__packed__)) {
    uint32_t          addr;
    uint8_t           versionMajorFuncRegWasIntroduced;
    uint8_t           versionMinorFuncRegWasIntroduced;
    uint8_t           versionMajorBugRegWasIntroduced;
    uint8_t           versionMinorBugRegWasIntroduced;
    uint32_t          supportedVariantsMask;
    uint8_t           isRegStillSupported;
} regSpec_s;


#ifndef IS_REG_COMPATIBLE
#define IS_REG_COMPATIBLE(__FW_VERSION_AND_VARIANT_PTR__,__REG_SPEC_PTR__)                                                    \
       ((((((uint32_t)(__REG_SPEC_PTR__)->versionMajorFunc)               << 24) |                                            \
          (((uint32_t)(__REG_SPEC_PTR__)->versionMinorFunc)               << 16) |                                            \
          (((uint32_t)(__REG_SPEC_PTR__)->versionMajorBug)                <<  8) |                                            \
          (((uint32_t)(__REG_SPEC_PTR__)->versionMinorBug)                <<  0)) <=                                          \
         ((((uint32_t)(__FW_VERSION_AND_VARIANT_PTR__)->versionMajorFunc) << 24) |                                            \
          (((uint32_t)(__FW_VERSION_AND_VARIANT_PTR__)->versionMinorFunc) << 16) |                                            \
          (((uint32_t)(__FW_VERSION_AND_VARIANT_PTR__)->versionMajorBug)  <<  8) |                                            \
          (((uint32_t)(__FW_VERSION_AND_VARIANT_PTR__)->versionMinorBug)  <<  0))) &&                                         \
        (0 < ((__REG_SPEC_PTR__)->supportedVariantsMask & VARIANT_TO_VARIANT_MASK((__FW_VERSION_AND_VARIANT_PTR__)->variant))) && \
                                                  (!(((__FW_VERSION_AND_VARIANT_PTR__)->versionMajorFunc == 0xFF) &&          \
                                                     ((__FW_VERSION_AND_VARIANT_PTR__)->versionMinorFunc == 0xFF) &&          \
                                                     ((__FW_VERSION_AND_VARIANT_PTR__)->versionMajorBug  == 0xFF) &&          \
                                                     ((__FW_VERSION_AND_VARIANT_PTR__)->versionMinorBug  == 0xFF))))
#endif

extern const regSpec_s MY_REG_1;
extern const regSpec_s MY_REG_2;

C-file:

const regSpec_s MY_REG_1 = {1000, 0, 0, 0, 4, ALL_VARIANTS_MASK, DONT_KNOW_IF_REG_IS_STILL_SUPPORTED_YOU_HAVE_TO_ASK};
const regSpec_s MY_REG_2 = {1002, 0, 0, 0, 4, ALL_VARIANTS_MASK, DONT_KNOW_IF_REG_IS_STILL_SUPPORTED_YOU_HAVE_TO_ASK};

In my code I would then first query the firmware version of the product I want to communicate with and then I can determine whether the register exists or not:

C++

versionAndVariant_s* versionAndVariantOfOtherProduct = getVersionAndVariant(otherProduct);
if (!IS_REG_COMPATIBLE(versionAndVariantOfOtherProduct, &MY_REG_1)) {
    return FALSE; // Error! The other product's fw is too old to have this register
}
if (MY_REG_1.isRegStillSupported == DONT_KNOW_IF_REG_IS_STILL_SUPPORTED_YOU_HAVE_TO_ASK) {
    MY_REG_1.isRegStillSupported = isRegStillSupported(otherProduct, MY_REG_1.addr);
}
if (MY_REG_1.isRegStillSupported == REG_IS_NO_LONGER_SUPPORTED) {
    return FALSE; // Error! At some point the other product supported this register, but not anymore
}

Any suggestions for improvements on this?

modified 13-May-20 6:57am.

I have 4 microcontrollers, A, B, D and E, that can communicate through registers that are exposed in h-files that reside inside Git submodules. A and B reside in the same box and can't be disconnected from each other so from a black box perspective they look like a combination of A and B that we can call C. D and E can be connected/disconnected during runtime from B, but they can also access A through B. E can also be connected/disconnected during runtime to D, which can then see B and A through D. Does anybody have suggestions on how many h-files I should have and what they should contain, in order to have an as clean design as possible?

modified 13-May-20 6:02am.

I would like to make it "impossible" for non-technical people to direcly read a plain text file in e.g. Notepad or Microsoft Word, but I should be able to modify the file in a simple way and then be able to read it. One idea would be to use some kind of 16-bit/character ASCII encoding and then remove the first byte to make it unreadable and put it back when I want to read it. Would this work in practice and what file extension and editor (Wordpad?) should I use to properly read the file? Any other ideas? I am working in standard Windows 10 environment that comes with basic applications such as Notepad and Wordpad and I don't want to write any software or install any new applications.

arnold_w wrote:
One idea would be to use some kind of 16-bit/character ASCII encoding and then remove the first byte to make it unreadable and put it back when I want to read it.

Where would you put these bytes, and how would you replace them when you wanted to see the clear text? But the real question is why do you need to do this? What information are you trying to secure, and why can you not just save it somewhere safe?

Richard MacCutchan wrote:
Where would you put these bytes, and how would you replace them when you wanted to see the clear text?

I would put the byte first in the file, causing the upper and lower bytes being swapped in the 16-bit ASCII format. I would do this using Notepad, which operates on 8-bit ASCII and to view it I would use an editor that supports 16-bit ASCII. Does anybody know if RTF-format would be a good choice?

Richard MacCutchan wrote:
But the real question is why do you need to do this? What information are you trying to secure, and why can you not just save it somewhere safe?

I need to create an index to a bunch of files on a server folder that you are not supposed to access directly, you are supposed to go through a web interface. Unfortunately, the web interface is a bit naive and doesn't reflect the truth what's inside the server folder. Most users don't know the path to the server folder and I would like to keep it that way.

You are correct that this is not the best solution, the best solution would be to link the file index into the .exe-file and then, when needed, alter the .exe-file using the rename-exe-file-when-it-is-use-and-create-a-new-with-the-original-name-technique but in case I can't pull that off I would like to have a backup solution.

arnold_w wrote:
I would do this using Notepad,

I don't know how you would do that in Motepad, since it operates purely on text characters.

arnold_w wrote:
if RTF-format would be a good choice?

RTF merely allows you to display text with various characteristics, such as bold, italic, colours etc.

Why not just make the folder private to an admin type account so users cannot access it?

How does the web interface serve the files to the user? If it's just directing them to a UNC path, then they can already see the path of the folder.

"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer

Way back in the dim dark past we simply added a value to the ascii of each character effectively moving it out of the normal text range, reverse the process to read the content. I think it was 75.

Never underestimate the power of human stupidity -
RAH
I'm old. I know stuff - JSOP

I think I will just zip the file and give it .bin extension, that should make it unreadable.

If it is on a webserver then you should simply pick an extension that the server doesn't "serve"; also, if you were using a database then you could simply have some tables that you don't make accessible over a UI.

Do take into consideration that some non-techies call in the help of a forum to hack into "unreadable" Roll eyes | :rolleyes:

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

If you go to zip, why not use the proper encryption facilities of zip itself?

True encryption seems a bit overkill. But I might add a password, just in case someone figures out that it's a zip-file.

Giving a false idea of security.

online zip password cracker - Google Search[^]

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

The users of the application are my co-workers, not ill-intentioned hackers.

Being one does not automatically mean that you cannot be the other.

Multiple people wrote:
You should treat all user input as potentially malicious.

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

If the filesystem is NTFS you could store the real data in an alternate data stream. A casual user would be unlikely to find that.

What tech/architecture is recommended to handle a high amount of real-time data From Server to the clients?

Please note - Gathering data from Client to server is not difficult. We have ingestion points to collect the data.

But for sending data from Server to Client. Looks like it needs some good study and evaluation.
The cave-man idea that immediately comes to me is using Sockets and keep streaming data back.

Is there any recent frameworks/services available that does this best?
Google Cloud/Firebase has the options, we do use. But anything better out there?

Think of it as even a Share Market Application. Where you have to update all the connected clients every few seconds with Real-Time data This is not domain I'm into but the requirement is technically same. It's all about sending data in real-time.

Client - Web/Android/iOS

modified 26-Apr-20 10:00am.

Just to be sure I understand, you want to send lots of data from a server to the clients? Could there be more than one server doing this?

Robust Services Core | Software Techniques for Lemmings | Articles