Welcome to the Lounge

Mike Hankey22-Dec-23 4:51

Mike Hankey

22-Dec-23 4:51

0x01AA wrote:
entropy

An Entro to Py

As the aircraft designer said, "Simplicate and add lightness".
PartsBin an Electronics Part Organizer - Release Version 1.3.0 JaxCoder.com
Latest Article: SimpleWizardUpdate

Gary Stachelski 202122-Dec-23 5:19

Gary Stachelski 2021

22-Dec-23 5:19

Here is a link to a youtube video on Shannon Information Entropy.
See if this makes more sense.
Information entropy | Journey into information theory | Computer Science | Khan Academy - YouTube[^]

0x01AA22-Dec-23 5:41

22-Dec-23 5:41

Thanks for this. My usual problem is they state 'the probability is this and that' but no explanation why it is Wink | ;)

Gary Stachelski 202122-Dec-23 7:25

Gary Stachelski 2021

22-Dec-23 7:25

In the case of the video, the probabilities were made up to create two different machines. They then went on to illustrate how you could calculate the information entropy for each machine to compare them. The machine with the lower calculated entropy was more predictable Or better organized. The higher the calculated entropy the more disorganized a system is, so it is harder to predict what the output of the next cycle will be.

jochance23-Dec-23 7:24

23-Dec-23 7:24

Mmmm cyclomatic complexity... keep going, we can cut my heating costs.

Gary Stachelski 202123-Dec-23 8:46

Gary Stachelski 2021

23-Dec-23 8:46

You make an excellent observation. Like measurements of Entropy, Lower Cyclomatic complexity programs are more stable (better organization), their behavior is more predictable and you have a higher confidence in predicting their processing results.
In both cases lower is better.
Thanks, did not see the connection.

jochance25-Dec-23 13:55

25-Dec-23 13:55

I uhhh... have to go overboard with how not serious I take myself sometimes.

jochance25-Dec-23 14:03

25-Dec-23 14:03

Reading what you wrote, I was picturing software emulated hardware because I do not 'really' know hardware.

Jo_vb.net22-Dec-23 5:24

Jo_vb.net

22-Dec-23 5:24

Learning this could be a first step
https://machinelearningmastery.com/what-is-information-entropy/[^]

0x01AA22-Dec-23 5:47

22-Dec-23 5:47

Thank you for this. I read a lot, but I'm stumbling again and again.

E.g: from here https://www.physik.uni-wuerzburg.de/fileadmin/11030300/_imported/fileadmin/tp3/ThermoEDynamik/Entropie.pdf[^]

Quote:
Da man mit n Bits bekanntlich 2^n verschiedene Bitmuster bilden kann, ist sofort klar, dass ein System mit 2^n Zuständen mit einer n-Bit-Datei vollständig beschrieben werden kann, so dass in diesem Fall H=n is

For me nothing is clear... Frown | :(

Especally that one can describe e.g. the 2^8 conditions in an 8 bit file. But most probably this is a lack of my understanding

Jo_vb.net22-Dec-23 6:25

Jo_vb.net

22-Dec-23 6:25

I'm sure you understand it (but you do not know that)!

Wortbreite
8 Bit
Bereiche mit Vorzeichen
-128 ~ 127
Bereich ohne Vorzeichen
0 ~ 255

2^8 = 256
and 1 to 256 you can define with 8 bit as shown above with 0 to 255.

Now you should know that you already knew that Big Grin | :-D

0x01AA22-Dec-23 6:33

22-Dec-23 6:33

I think , I'm very save with binary numbers. But again translated by google, I don't get the point on that:

Quote:
Since it is well known that 2^n different bit patterns can be formed with n bits, it is immediately clear that a system with 2^n states can be completely described with an n-bit file, so that in this case H=n

How one can desribe 256 condistions in a file with one byte?

I'm pretty sure I have a problem understanding the article, but I'm also very happy if somebody can explain what I'm missinterpreting Wink | ;)

Jo_vb.net22-Dec-23 6:52

Jo_vb.net

22-Dec-23 6:52

A file with one byte can describe how many states a system has (state value 1 or 2 or ... 256).

If you want describe 256 states you need 256 * 8 bit.

BernardIE531722-Dec-23 12:08

BernardIE5317

22-Dec-23 12:08

my understanding is that a system at any particular moment is in a state . a single state . of course a system can not be in more than one state at any moment unless of course we are discussing Quantum Mechanics which of course we are not i presume . if it is known the system can be in any state of 256 possible states then at any moment only 8 bits are required to specify that state . QED

0x01AA23-Dec-23 0:09

23-Dec-23 0:09

Thank you very much!

jochance23-Dec-23 7:29

23-Dec-23 7:29

A visualization is literally 256 light switches side by side.

Each switch represents logical branching in the code. appStates is an array of bool... if (appStates[34] and appStates[42]) ....

Amarnath S22-Dec-23 5:56

Amarnath S

22-Dec-23 5:56

0x01AA wrote:
Nobody knows what entropy really is

As a Mechanical Engineer by education, I can only say that Second Law of Thermodynamics dictates that the entropy of the Universe is continuously increasing, dS > 0.

0x01AA22-Dec-23 5:59

22-Dec-23 5:59

But there is a limit to this? Something like 2.58###

jschell22-Dec-23 6:16

jschell

22-Dec-23 6:16

As a suggestion attempting to learn one part of a system of study well, probably is always going to require that one learns more about the system that contains it first.

So study information theory first.

0x01AA22-Dec-23 6:22

22-Dec-23 6:22

Quote:
So study information theory first.

I would say 'Shannon's' therorie about Entropy is exactly that very basic theory Wink | ;)

BobbyStrain22-Dec-23 6:44

BobbyStrain

22-Dec-23 6:44

There is a simple equation that defines entropy. Chemical Engineers make use of the term to describe the behavior of substances. We use it, for example, to evaluate the performance of a steam turbine. It has been misused by zealous promoters to obfuscate information.

0x01AA22-Dec-23 6:46

22-Dec-23 6:46

And that simple equation is?

4.6 Entropy - University Physics Volume 2 | OpenStax

BobbyStrain22-Dec-23 8:08

BobbyStrain

22-Dec-23 8:08