1/ Been getting good questions as people try the hyperliquid API/SDK. A few wondered if it's really possible to do crypto HFT in python. The answer is undoubtedly yes! A thread on how to build a fully automated HFT system, and why python is the ideal language to get started š§µ
2/ A bit of background: our strategy began as a single file python script. We now use an optimized rust system in production, but the journey to get there was crucial. If I had started with rust or C++ from day 1, there would've been no trading. Start where you're comfortable.
3/ Even today, you can write a large class of profitable crypto HFT strategies in python. The fundamental reason is that network latency dominates compute latency. Doesn't matter what language you're using, make sure to measure latencies as early as possible.
4/ So how should the system look? Start with a simple event loop. Find some library that does async websocket IO, maybe even one that directly integrates with the exchange you care about. You'll likely want to listen to binance ws no matter what you're doing, so include that.
5/ Maintain a single state machine through callbacks. Build a system which takes feed messages as input and whose output is a sequence of order and cancel actions. Send those actions asynchronously. Put the book building and strategy on the same single-threaded code path.
6/ pseudocode example. h/t @thiccythot_ for the idea.
7/ Some asked me about the GIL in python and how it'll make things slow. Note that most of the parallelization you want is when reading WS messages and sending order requests (either ws or http). Both are IO bound, so concurrent code running on a single thread is totally fine.
8/ Super important though: The #1 trick to making your python code fast enough for production is pypy. This is the trading analog of using pandas/numpy/torch in machine learning. Make sure you use keep memory layout simple with raw arrays. Then pypy will just work its magic.
9/ Pypy uses just-in-time compilation as you call the same part of your python code. It will detect the hot paths of your logic and optimize those on the fly. Going back to the single threaded concurrency point, note that pypy's JIT compilation is also done in parallel!
10/ For context, "real" HFT systems in tradfi look nothingĀ like this. For example, you might have a feed handler writing data to shared memory, and other processes reading straight from there. Orders might follow the same logic, to completely cut out IO from the strategy.
11/ Take comfort in the fact that you don't need to build and debug complex latency optimized systems in crypto That being said, the moment to evolve to a statically typed, compiled language will be clear I'd punt it as much as possible, but when you do it, revisit everything
12/ You'll have a much better idea of what your system is supposed to do based on the profitable strategies you've written in python. Infra is never going to be general enough to do everything, so come into the rewrite with a concrete spec of what the trading strategy will need.
13/ I hope this thread was helpful to those embarking on the quant journey. It's more tractable than some want you to believe. I learned everything I know about building performant systems by writing and rewriting an automated trading system. Enjoy the rewarding process!
@chameleon_jeff You can use the numba module for JIT compilation/ parallelization as well š¤
@chameleon_jeff @threadreaderapp unroll #HFT