I worked on Figma's sync engine LiveGraph and before that Meteor (the open-source sync engine), some thoughts:
Real-time engines can simplify the application development but building a generalized sync engine that's performant, scalable and provides the right abstractions (and concurrency guarantees) still remains to be a technical challenge (1/)
We blogged about it on the Figma blog: https://www.figma.com/blog/liv...
Lots of people I met in SF tech know of Figma's sync being in the heart of the product, but a few things most of them didn't realize: 1. Figma has 2 custom-built sync engines 2. Both started from different places but they increasingly converge to a DB + event log
Figma has 2 engines: 1. Multiplayer 2. LiveGraph
Multiplayer is the OG sync engine of Figma. It probably looks nothing like the engines of Linear, or InstantDB, or Notion's. It should be thought of the Figma's take on Google Docs tech (diff algo, diff model, but similar purpose)
Multiplayer leans heavily into "enable realtime collaboration": - document-based scope for sync - no ACL, if you have access to the doc, you have access to everything - lots of application-specific tuning in logic, resolving cycles, special fields, etc https://www.figma.com/blog/how...
Multiplayer is a big part of why Figma is Figma: - replaces version control - makes collab simple and seemless
It works incredibly well as long as the document fits into the server memory - the codebase is an efficient Rust process that can pack lots of smartness, usually works on small deltas that have complicated effects. That's Multiplayer. The genius of Figma's infra.
LiveGraph, on the other hand, appeared later. We built it to replace the mixture of Ruby end-points, Redis queues, web-socket based notifications, refetches of large queries. LiveGraph is a spiritual successor to Asana's Luna/LunaDB, Meteor, Apollo GraphQL server, Firebase.
It operates on DB rows, adds ACL checks, business logic in server-side TypeScript, GQL queries, generation of types for application layer.
Things LiveGraph does well: - Given a GQL query, keep it up to date, by sending minimal tree patches to the client - Process a giant object tree for ACL on the backend - But deliver a smaller subset to the client for viewing
LiveGraph relies heavily on querying SQL DB + correlating the WAL of the DB to patch up the tree. The version I worked on never requeried the DB unless absolutely needed. Lots of patches were just inferences from WAL. That's the design of Meteor with MongoDB.
IMO the "key insight" that allowed LiveGraph to follow this approach is a small set of heuristics that very very quickly determines For every update in the WAL in ~O(1) time determine which of ~100k observed queries are affected patch the queries in O(1)
The heuristics are simple but they do not always work well, so we built a small registry of "hints" specific to application data access and write patterns. As application evolves, new processing syncs appear.
But one of the hardest limitations on LiveGraph model is probably the broad scope of its queries. A single GQL query can span a bajillion objects due to the complex ACL model in the modern enterprise apps https://www.figma.com/blog/how...
... and the initial version of LiveGraph did not work well with DB sharding model of Figma. Since I left the team, there were a lot of designs on how to make it scale along-side DB shards https://www.figma.com/blog/liv...
How do Multiplayer and LiveGraph converge? They both approach a DB + Log + in-memory realtime sync layers.
Multiplayer started from document-scope. Documents are persisted to an object storage like S3 after the session closes. But for the most part everything is in memory. Overtime the Figma eng team built an incremental log (WAL) for recovery and possible tailing.
LiveGraph started from being an add-on to the DB. DB already has the persistence, and WAL. LiveGraph added the in-memory processing, slowly adding the tricks from Multiplayer.
Super excited for what the modern sync engine companies will do: @instant_db and @convex_dev come to mind! If done right, they will allow a specific flavor of prof tools and complex applications to focus on building a product, instead of getting nerdsniped by sync correctness.
@instant_db @convex_dev I do not work on this type of systems anymore but if you want to work with me on solving cool problems, I work at @operatordotxyz now
Lots of you DM'd me after this thread asking for architecture advice for your apps and companies. I appreciate the attention! My general advice is: stick to boring tech. Only build custom stuff when it matters to your product. In-depth convo? You can consult hire me
@imslavko also means server app becomes more stateful. how did you guys load balance the workload?
@davidzh 100%, managing state is the price you pay. In an ideal arch you can re-use the caching/computation between users. i.e. all users of the same Figma org subscribe to the same content. All viewers of the same file subscribe to the same design system tokens, comments.
@imslavko what were the biggest challenges scaling livegraph?
@nooriefyi In my experience: deploying new versions without causing thundering herds on your db
@imslavko do you think there is potential for a sync engine for real-time collaborative development? think git branches but each branch has a live sync context.
@jonahseguin My observation is that all bells and whistles of real-time sync (including persistence on the client) need to be making the product significantly better to be worth the pain. Unless the effortless branching in a key part of your product, I don’t know what I would do with it
@imslavko Definitely a fit for some parts of some apps, but aren’t we making things more complicated than need be? Most apps are not figma. Client-side state management isn’t mandatory. Server-side rendering and HTML over the wire works surprising well.
@hunvreus 💯 For a startup I would recommend starting simple unless this is the core part of your product
@imslavko Brillant thread. Fully agree and thanks a lot for sharing! Would love to invite you to the @localfirstfm podcast to talk more about this!