What all of these explanations get "wrong" is that they try to explain what a tensor is. Yet tensors themselves are neither fundamental nor important. The fundamental/important thing is the tensor *product*. A tensor is just an object assembled using tensor products... (1/9)
The tensor product ⊗ is, conceptually, the most general (binary) operation that behaves "how a product should behave". In practice, this means that the order of brackets shouldn't matter, i.e. X⊗(Y⊗Z) should be the same as (X⊗Y)⊗Z, for any objects X, Y and Z... (2/9)
...and also that there should be some "do nothing" object I, such that X⊗I and I⊗X should both be the same as just X, for any object X. Any operation obeying these rules (including just ordinary multiplication of numbers) is therefore a kind of "tensor product". (3/9)
Often, when physicists/engineers talk about "tensors" (unqualified), they're referring specifically to the tensor product of *vector spaces*. And vector spaces have a bit of extra structure, namely a notion of "duality": ever vector space has a corresponding "dual space". (4/9)
The dual * is an operation that reverses the order of function composition. If I have a function f : X -> Y, then f* : Y -> X, and if f ∘ g : X -> Z, then (f ∘ g)* = g* ∘ f* : Z -> X. Applying this to linear maps on a vector space, we arrive at the idea of "dual space". (5/9)
A tensor product X⊗I of a vector with the identity is just a vector (vectors are tensors). A tensor product X⊗Y* of a vector with a "dual vector" is just a matrix (matrices are tensors). A tensor product X⊗Y of a vector with a vector is a rank-2 contravariant tensor. Etc (6/9)
In the case of vector spaces, the "transformation laws" like covariance, contravariance, etc. are all downstream of this notion of duality: that the order of composition for linear maps is reversed in the dual space. It's not really a property of tensors, but of *duals*. (7/9)
But the notions of tensor products and duals are far more general than just vector spaces. Tensor products and dual structures can be defined for groups, rings, modules, fields, functions, manifolds, quantum circuits, lambda expressions, etc. "Everything is tensor." (8/9)
[In differential geometry, duals are differential 1-forms. In physics, they're "covectors". But it's all the same idea.] So next time someone says that something is a "tensor", don't ask "What's that?" Ask "What's the tensor product?" Because that's what really matters. (9/9)
@getjonwithit My mental model is that a tensor is any geometric object that acts on n-vectors and k-covectors that is invariant over arbitrary coordinate transformations. Though I see now that this is too limited a conception.
@getjonwithit Strangely, I have written something about tensors today. Shamless plug: https://x.com/compose/articles...
@getjonwithit A tensor is obviously just an n-dimensional array for anyone with eyes to see. Physicists are talking mostly about tensor fields and you know it.
@getjonwithit Is there a « generalized addition »? Standard set addition / direct sum is concatenation which trivially does not collapse to standard addition for ordinary number systems
@getjonwithit Both the mathematician and the physicist in the meme seem to be saying the same thing as this? A tensor is nothing more than the operations you can do with it, i.e. the algebra, or what it “transforms like”
@getjonwithit Tensor product is a bifunctor. It doesn't produce tensors it produces spaces in which tensors live. Tensors are stable points in a dynamic field of permissible interactions. A tensor is a resonant collapse of directional structure across constrained spaces. Most definitions of
@getjonwithit An n dimensional array - nice
@getjonwithit Inm feat that everybody that want to read your posts already know what an algebra is (or at least a vector space and a ring are) so they don't learn anything :/
@getjonwithit Can you shed some light into this from a category theorists point of view?
@getjonwithit Mathematicians ngmi without stepping up their meme game. Physicists know but don’t care, and Comp Sci just wanted a better identifier than nd-array and tensor was good enough
@getjonwithit no, math and physics in this meme use the structure of tensor algebras, while ML is using arrays with Einstein notation for product. The Christoffel symbols are written using this notiation, but they are not tensors. What makes tensor a tensor is being an element of a structure.
@getjonwithit I politely disagree. To put it short: the formal definition is both cause and consequence of the use of tensors in physics and engineering. Also these different perspectives on tensors influence each other. 1/n
@getjonwithit Strange you’d talk about the tensor product like this without mentioning its bilinearity. I was taught that that’s its universal property? (I.e. every product of vector spaces that’s bilinear is a tensor product)
@getjonwithit You are totally cracked.
@getjonwithit tensors are gay 👎
@getjonwithit A tensor is a thing that transforms like a tensor
@getjonwithit most helpful thing I've read about tensors to date; thank you!
