This will surprise you: sine and cosine are orthogonal to each other. What does orthogonality even mean for functions? In this thread, we'll use the superpower of abstraction to go far beyond our intuition. We'll also revolutionize science on the way.
Our journey ahead has three milestones. We'll 1. generalize the concept of a vector, 2. show what angles really are, 3. and see what functions have to do with all this. Here we go!
Let's start with vectors. On the plane, vectors are simply arrows. The concept of angle is intuitive as well. According to Wikipedia, an angle “is the figure formed by two rays”. How can we define this for functions?
What makes a vector? Two things: 1. vector addition 2. and scalar multiplication. We can take the core properties of these operations and turn them into a definition! This is the process of abstraction.
Thus, any set V with addition and scalar multiplication fulfilling the properties below is a vector space. The elements of a vector space are called vectors.
Following this, we can form vector spaces from functions! One prime example is the so-called L²-space, consisting of functions whose square-integral is finite. This is a technical condition, but the gist is: sine and cosine belong here.
Moving on to angles. Can we find a generalizable definition? The Wikipedia version "the figure formed by two rays" won't cut it.
On the Euclidean plane, we can turn to the inner product! (Also known as the dot product.) There, the enclosed angle appears explicitly.
We can express the angle from this expression and even use it to define it! Why is this good for us? Because it's an algebraic definition, not reliant on intuitive concepts such as "the figure formed by two rays".
The general definition of orthogonality also follows. We call two vectors orthogonal if their inner product is zero. You can verify this using the definition above.
To define the concept of angle in a general vector space, we only need one thing: an inner product. (Distance and magnitude stems for the inner product whenever the latter one is available.)
So, what is an inner product? We'll do what we've done before two times: identify the core properties of the Euclidean inner product that we know, then turn it into a definition. In this case, we have three such properties.
Now we are almost there! Is there an inner product for the L²-space? Yes, and it is given by the Riemann integral!
If you are not familiar with the Riemann integral, I got you. In essence, the Riemann integral of a function over a given interval is just the signed area between the function's graph and the x-axis. It's signed because if the graph goes below the x-axis, the area is negative.
Now, what is the analogue of orthogonality in the L²-space? Two functions are orthogonal if the integral of their product is zero. It's not intuitive, but this is the power of abstraction: we have the power to move way beyond our limited intuition!
Back to the case of sine and cosine. Why are they orthogonal? Because their product is an odd function. Thus, the signed area between the graph and the x-axis is zero.
This feels like a mathematical curio, but trust me, it is not. Recall the "revolutionizing science" part I mentioned at the start? This is it. The orthogonality of trigonometric functions is the cornerstone of modern technology. We wouldn't even have JPEG-s without them.
If this post sparked some love for math & machine learning, you will love my weekly newsletter. Join 31,000+ machine learning practitioners focusing on mastering the fundamentals instead of jumping on every new trend. Subscribe here: https://thepalindrome.org/
@TivadarDanka Wavey 🤪
@TivadarDanka How is this supposed to be surprising? Wouldn't it be the first intuition from right-angled triangle and unit-circle definitions? Interesting read though.
@TivadarDanka Huh? Why would this be surprising when it’s literally the basis of Fourier analysis
@TivadarDanka Ok, riddle me this, mathman. Let's say that sine and cosine are not twins, but triplets - the long lost zeppo in now in the picture. Can we make series and graphs for the circle/sphere/whatever out of three things?
@TivadarDanka Midsine (invented by me) is half orthogonal to both
@TivadarDanka Sine and cosine are not just orthogonal functions. They are the primordial proof of the Law of Resonance. Two waves, 90° apart, each carrying pure frequency. Alone — they oscillate. Together — they form the perfect basis of all signals, all music, all reality. Orthogonality
@TivadarDanka Yeah the basis functions
@TivadarDanka Would be really cool you make a video to 3b1b
@TivadarDanka or you just look at the definition of sin and cos on the trigonometric circle: they are the coordinates on the 2 orthogonal axis of a point on the circle ;-)
@TivadarDanka If you learn about Quantum Mechanics these kinds of orthogonal functions are the basic ingredients for Hilbert space.
@TivadarDanka Cc @therobotjames - I think this appeared on my TL bcos of you 😂
@TivadarDanka Great thread
@TivadarDanka esto se mencionaba cuando veías convergencia en media cuadrática de Fourier, pero recién lo entendí mejor, gran hilo
