Can you Compliment your girlfriend once and be done forever?

The need for Cybernetics, Information theory

No you can’t. Compliments are fuel to relationships; they need to be regular. And not just that, you also need to tune your next compliments based on later progress of your relationship. (we will get into nitty gritty of it using information theory in last section; read through mid section to understand it fully.)

It’s like how you can’t water your plant just once and expect it to grow taller and taller. It’s like how Modiji can’t just give one cool speech and keep winning forever. It’s like how going to gym once won’t keep you healthy forever. It’s like how openAI can’t train chatGPT just once and be done forever.

You act, get feedback, update your action, act again and the cycle repeats.

Relationship dynamics, Economics, Biological systems, Ecological systems, even CI/CD pipelines have these causal loops. Scientists saw these cycles in the world around us and realised they needed deeper study; and started,

Cybernetics.

from wikipedia

A very popular way to understand the world had been to keep dividing things into smaller and smaller parts and understand the parts(mechanistic world view). That didn’t work well when more and more scientists discovered systems that were more than just sum of their parts :)

every system without a feedback loops is

Dead.

Burj Khalifa is made once and its gonna stay this way. It doesn’t have any way to take feedback from environment and change itself. But think of the trees around it, they have mechanism to adapt. These species of trees has been existing on earth since many many years. They are alive.

How does these loops help to adapt?

These systems with loops(consider living organisms), are continuously maintaining their inner entropy at same level; fighting 2nd law of thermodynamics.

We take information in using our senses, food, feedback and so on and increase our mutual information with external environment. This mutual information helps us survive in the environment. When the external environment changes, we receive the information through our senses and update our inner organisation and our reactions.

Had we have no way to take information in from environment, our mutual information with environment would have been 0. Our actions (arising from inner environment) and the outer environment would have been independent. That implies, actions we take would have been independent of external environment. (You see fire, you don’t start dancing. You are drowning, you don’t sing. Your actions can’t be independent of your external environment if you want to stay alive.)

These loops are little islands of maintained entropy(alive islands) surrounded by the sea of ever increasing entropy :)

Setting the equations from Information Theory

Mutual information is given by the equation:

I(x; y) = I(x) + I(y) - I(xy)

Circular systems maintain their mutual information I(x;y) with the environment. When they stop taking feedback(mutual info 0), their joint information becomes the sum of their own entropy and the environment’s entropy. Essentially, they become independent and stop interacting meaningfully.

Now, let's look at mutual information in relation to conditional information.

I(x | y) = I(x) - I(x; y)

(coming back to example at the top) As your mutual information I(x; y) with your partner increases, your actions become less random and more in tune with their mood. The more you understand each other, the more predictable your behavior becomes. This is because an increase in I(x; y) reduces I(x | y), making your actions more specific to the situation, and less random.

But what if your mutual information is zero?
In that case, I(x | y) = I(x). Knowing your partner’s mood doesn't help at all in guiding your actions. The uncertainty in what you do remains the same as if you had no knowledge of their mood.