Daniel sent us this one, and it's a topic that's been sitting in the back of my mind for a while. Over eighty percent of equity trades are now executed algorithmically. That's not a prediction, that's the current state of the market. Daniel's asking about the programmatic finance ecosystem that's grown up around that reality — the Python libraries, the quant frameworks, the OSINT-adjacent tooling, and how all of it has matured alongside the AI push. What are the main tools, what's actually interesting, and where is this heading?
By the way, today's episode is powered by Claude Sonnet four point six, which feels appropriate given the subject matter.
A friendly AI writing a script about AIs trading stocks. We're in deep.
But okay — eighty percent. That number deserves a second. Because when people hear "algorithmic trading" they still picture either a Bloomberg terminal in a hedge fund, or some rogue bot melting down a market in 2010. Neither picture captures what's actually happening now, which is that the tooling has become genuinely accessible. The gap between what a two-person quant shop can deploy and what a mid-tier institutional desk is running has collapsed in a way that would have seemed implausible fifteen years ago.
Which is exactly what makes Daniel's question interesting. It's not just "here are some libraries." It's about a whole ecosystem that's been quietly democratizing something that used to require serious infrastructure and serious capital just to get in the door — which raises the question: how did that gap collapse in the first place?
Right, and to understand why that gap collapsed, you have to go back to what "quant" even meant in the early two thousands. The original quant shops — Renaissance, D.Shaw, the early Citadel — were building on proprietary infrastructure. Custom execution systems, in-house data pipelines, research environments that took years to build. The intellectual capital was extraordinary but so was the operational overhead. You needed a small army just to keep the plumbing running.
The question is what changed. Because the math didn't change. Mean reversion is mean reversion.
The math didn't change, but the abstraction layers did. What Python did — and this happened gradually through the twenty-tens — was give you a common language where the data wrangling, the statistical modeling, and the execution logic could all live in the same environment. Before that you might have your research in MATLAB, your data in some proprietary format, your execution in C++, and getting those three things to talk to each other was its own full-time job.
That's what I'd call the real distinction between traditional quant research and what we have now. It wasn't just that the tools got better. It's that the seams between the tools dissolved.
The modern programmatic finance stack is integrated in a way the old one wasn't. You can go from raw price data to a backtested strategy to a live paper-trading environment without ever leaving Python. And then AI started getting layered on top of that integrated stack, which is where things got interesting — because now you're not just automating execution, you're automating parts of the research process itself.
Which raises the obvious uncomfortable question of what "research" even means when the model is generating the hypotheses. And that's exactly what we'll need to unpack next.
That question is going to haunt the second half of this conversation, but let's build the foundation first, because the tooling story is fascinating on its own terms. The reason Python won this space — and it did win, decisively — comes down to a few things that all converged at the right moment. You had Pandas landing in a mature state around 2013, 2014, and suddenly data manipulation that used to require custom C++ or a MATLAB license was just... a pip install away.
Pandas being the thing that made time-series data feel like a first-class citizen rather than something you had to fight constantly.
And you can't talk about Pandas without NumPy underneath it, because NumPy is doing the heavy lifting on array operations. When you're computing rolling correlations across hundreds of instruments, the difference between a pure Python loop and a vectorized NumPy operation can be a factor of a hundred in execution time. That matters enormously when you're iterating on research.
To put that in concrete terms — if your pure Python loop takes ten minutes to run a correlation sweep across a universe of five hundred stocks, the NumPy version is finishing in about six seconds. That's the difference between running twenty iterations in an afternoon and running twenty iterations before lunch.
Research velocity is everything in this space. The faster you can test a hypothesis and kill it or refine it, the more ground you cover. That compounding effect on iteration speed is a big part of why the Python ecosystem pulled away from MATLAB and R for production quant work, even though both of those have perfectly capable statistical libraries. The integrated workflow just moves faster.
You've got your data layer.
Then you need to actually compute signals. That's where TA-Lib comes in — Technical Analysis Library. It's been around for a long time, originally written in C, but the Python wrapper made it accessible to the broader quant community. Hundreds of technical indicators — MACD, Bollinger Bands, RSI — all implemented in optimized C under the hood, callable from Python. The criticism of TA-Lib is fair, which is that classical technical analysis has real limitations, but as a signal-generation layer inside a larger pipeline it's still widely used.
What does "inside a larger pipeline" actually look like in practice though? Because I think people hear "technical indicators" and either dismiss it entirely or treat it as the whole strategy, and you're describing something in between.
Right, so the more sophisticated use case is treating TA-Lib outputs as features rather than signals. You're not saying "RSI crossed below thirty, therefore buy." You're saying "RSI, Bollinger Band width, volume momentum, and three other indicators are all inputs to a model that generates a probability estimate." The individual indicator might be noise; the combination, properly weighted, might carry genuine information. TA-Lib is just the efficient way to compute the raw ingredients. What you do with those ingredients is a separate question entirely.
That brings you to backtesting, which is where things get either exciting or deeply humbling depending on how honest you're being with yourself.
Deeply humbling is the more accurate description for most people. Zipline was for a long time the standard open-source backtesting framework — it came out of Quantopian, which was Robinhood's quant research predecessor in a sense. The architecture was clean: you'd write a strategy as a Python function, feed it historical data, and get out a full performance tearsheet. The problem Zipline ran into was that Quantopian shut down in 2020, and the maintenance burden fell to the community. There's an active fork called Zipline Reloaded that's kept it alive, but the momentum shifted.
Shifted toward what?
Backtrader, primarily, for the practitioner crowd. It handles live trading connections — Interactive Brokers, for example — in addition to backtesting, which Zipline wasn't really designed for. The event-driven architecture in Backtrader is also more explicit about modeling market friction. Slippage, commission structures, partial fills — you can configure those at a fairly granular level. That matters because one of the cardinal sins in backtesting is assuming you can execute at the close price you're using to generate the signal.
The "I'd have gotten rich" fallacy.
And it's worth being specific about why that fallacy is so seductive — your backtest says you buy at the close on the day the signal fires. But in reality you're generating that signal after the close, placing the order overnight, and filling somewhere in the open auction the next morning. That gap between your assumed fill price and your actual fill price, multiplied across hundreds of trades, can turn a backtested Sharpe ratio of one point eight into something that barely clears zero in live trading. Backtrader forces you to confront that gap explicitly rather than letting you paper over it.
The tool is almost doing you a favor by being annoying about it.
The friction in the framework reflects the friction in the market. And then on the more sophisticated end you have Qlib, which is Microsoft Research's framework, and this is where the AI integration becomes structural rather than bolted on. Qlib isn't just a backtesting environment — it's built around the assumption that your alpha generation is going to involve machine learning models. It has a built-in data pipeline, a model zoo with implementations of LightGBM and various neural architectures, and a workflow engine that handles the full lifecycle from feature engineering through portfolio construction. The 2025 update cut backtesting time for complex multi-factor strategies by around forty percent according to the Microsoft Research whitepaper, which is significant when you're running hyperparameter searches across hundreds of configurations.
Citadel apparently took notice. There was reporting that their high-frequency desk moved toward Qlib for strategy development in 2025, which is interesting because Citadel is not a shop that adopts external tools casually.
No, they're not. And the signal there is that even at that level of sophistication, the productivity gains from a well-integrated framework outweigh the NIH instinct. Then there's FinRL, which sits at the more experimental end — it's a deep reinforcement learning library specifically for trading. The idea is that instead of hand-crafting rules or even training a supervised model on historical signals, you let an agent learn a policy directly from market interaction. It's interesting research, but the practical deployment challenges are real. Reinforcement learning agents are notoriously unstable in non-stationary environments, and financial markets are about as non-stationary as it gets.
Which is a polite way of saying the market will find your agent's blind spots for you — repeatedly and expensively.
There's actually a famous example of exactly that failure mode playing out in a related domain. DeepMind's AlphaGo successors learned to play Go at a superhuman level, but when researchers introduced subtle rule perturbations the agents hadn't seen during training, performance collapsed catastrophically — not gracefully degraded, but fell apart entirely. The analogy to financial markets is uncomfortable because markets are constantly introducing rule perturbations. A new Fed communication framework, a regulatory change, a shift in retail participation patterns — any of those can invalidate the environment your agent trained in, and the agent has no mechanism to recognize that the ground has shifted.
The reinforcement learning approach is essentially betting that the future will resemble the past closely enough for the policy to transfer.
Which is a bet every quant strategy makes to some degree, but RL makes it in a particularly brittle way. That expense is actually a useful bridge to the other side of this ecosystem, because the quant signal problem — where do you get an edge that isn't already priced in — is driving a lot of the interest in qualitative and alternative data tooling.
Right, because if everyone has Pandas and Backtrader and access to the same price feeds, the alpha isn't in the infrastructure anymore.
It migrated upstream. The edge is in the data you're feeding the machine, not the machine itself. And that's where you start seeing tools that look more like OSINT infrastructure than anything a traditional quant would recognize. Maltego is the interesting case here — it's primarily known as a cybersecurity and investigative intelligence tool, but the financial community has been adapting it for things like mapping corporate relationship networks, tracking executive movements across filings, identifying supplier dependencies that don't show up cleanly in earnings reports.
You're using a threat-intelligence tool to build a better picture of a company's actual exposure before the market prices it.
That's the pitch. And it's not as exotic as it sounds — alternative data has been a serious institutional category for years. Satellite imagery of retail parking lots, shipping container tracking, credit card transaction aggregates. Maltego is just a particularly flexible graph-based interface for pulling those threads together. The OSINT angle matters because a lot of the most useful alternative signals are semi-public — they exist in regulatory filings, court records, professional network data — but assembling them coherently requires tooling that finance shops weren't traditionally building.
Can you give a concrete example of what that actually looks like? Like, what would a Maltego-based analysis surface that a traditional analyst looking at the same company might miss?
Say you're looking at a mid-cap industrial supplier. The 10-K tells you their top five customers represent sixty percent of revenue, but it doesn't name them. A Maltego graph pulling from SEC filings, LinkedIn employment data, and shipping records might let you reconstruct that customer list with reasonable confidence — and then you can cross-reference those inferred customers' own public guidance to get a leading indicator of your target company's order book before they report. The traditional analyst is reading the same 10-K. You're reading the network around the 10-K.
Then NLP walks in and the earnings call transcript becomes a data source.
Which is where the hybrid model story gets interesting. The setup is straightforward: you have a traditional quant signal — price momentum, factor loadings, whatever — and you want to augment it with a sentiment or tone signal derived from unstructured text. Earnings calls are the obvious starting point because they're structured, they happen on a predictable schedule, and the gap between what management says and what the numbers show is itself informative.
The classic "we're cautiously optimistic about headwinds" tell.
Early NLP approaches to this were fairly crude — bag of words, dictionary-based sentiment scoring using something like the Loughran-McDonald financial wordlist. Those still have value, honestly, but the signal quality improved substantially when you could run the full transcript through a large language model and ask it more nuanced questions. Not just "is the tone positive or negative" but "is management hedging more than they did last quarter" or "are they avoiding specific topics they addressed directly in the prior call.
Which is a qualitatively different kind of analysis. You're not counting words, you're modeling discourse.
There's research backing up how meaningful that distinction is. Academic work on earnings call transcripts has found that the question-and-answer portion — the analyst Q&A at the end — is often more predictive of subsequent stock moves than the prepared remarks at the top. Management controls the prepared remarks. The Q&A is where the hedging and evasion shows up under pressure, and that's exactly the kind of signal a bag-of-words model misses entirely but a language model can actually detect.
Because a language model can recognize when someone answers a different question than the one that was asked.
Which humans do constantly and which is very hard to operationalize without something that actually understands the semantic content. The practical tooling for that has matured fast. You can build a fairly robust earnings call analysis pipeline today using open-source transcription, a retrieval-augmented setup on top of a capable language model, and a vector database for storing and querying the historical record. The infrastructure cost for that workflow has dropped to the point where a solo researcher can run it. Three years ago you needed either a Bloomberg terminal subscription or a Refinitiv deal just to get clean transcript data at scale.
The data access problem is partly solved. What's the challenge that hasn't been solved?
Data quality, first and foremost. The Twitter sentiment angle is instructive here. The theory is clean — aggregate retail sentiment from social data, find divergences from institutional positioning, trade the gap. In practice the signal-to-noise ratio is brutal. You're dealing with bot activity, coordinated pumping, sarcasm that the model misreads, and the fundamental problem that the population of people posting about stocks on social media is not a representative sample of market participants.
It's a sample of people who post about stocks on social media, which is its own very specific thing.
A very specific and often very loud thing. The shops that have made social sentiment work — and some have — tend to be doing heavy preprocessing to filter that noise, and the alpha they're extracting is often thin and regime-dependent. It works until it doesn't, and the "doesn't" tends to be correlated with exactly the high-volatility periods when you most want your signals to be reliable.
The GameStop episode is almost a perfect case study in that failure mode, right? You had social sentiment screaming buy with enormous conviction, and the shops that were using raw Reddit and Twitter data as a signal either got burned if they were short, or got caught in the unwind if they chased the momentum too late.
It's a perfect example because it wasn't that the sentiment signal was wrong about the direction — retail was coordinating a squeeze. It's that the signal had no way to distinguish between "organic bullish sentiment" and "a coordinated short squeeze that will reverse violently once the broker restrictions hit." Those look identical in the raw data and completely different in terms of what you should do with them. That's a judgment call that requires context the model doesn't have.
Then there's the regulatory layer, which has gotten more complicated.
The SEC's algo trading disclosure requirements that came in this year are a real consideration now. The disclosure framework is pushing for more transparency around the decision logic of automated strategies, which creates an interesting tension — the more interpretable you make your model to satisfy a regulator, the more you've potentially revealed about your methodology to competitors who read the filings.
The transparency-alpha tradeoff. You can explain it or you can keep it, but possibly not both.
That's an oversimplification but not a wrong one. And overfitting sits underneath all of this as the foundational challenge that doesn't go away regardless of how sophisticated your tooling gets. The more degrees of freedom you give a model — and a large language model augmenting a multi-factor quant strategy has a lot of degrees of freedom — the easier it is to fit the historical record in ways that have no predictive content whatsoever.
The model found a pattern. The pattern was noise. The model is very confident about it.
Will remain confident right up until it isn't. Walk-forward analysis helps, out-of-sample testing helps, but the fundamental problem is that financial markets are adversarial and non-stationary in a way that most machine learning benchmarks are not. The pattern you found last year may have been real, and it may have been arbitraged away by the time you're trading it, or the regime may have shifted, or both.
Which brings you back to that uncomfortable question from earlier. If AI is generating the hypotheses and AI is evaluating them against historical data, who is actually doing the intellectual work of deciding whether the pattern is real or spurious?
That's the question the industry hasn't fully answered yet. The best practitioners I've read about treat the model outputs as hypothesis generators, not conclusions — they're feeding the AI's suggestions back through human judgment before anything gets near live capital. But the incentive structure pushes toward automation, and automation tends to compress that human review step over time. Which makes me wonder — how should someone even approach this space responsibly?
Given all those pressures, the tooling landscape, the data quality traps, the regulatory layer — what would you actually tell someone who wants to get into this space seriously? Not at the Citadel level, just someone with Python skills and genuine curiosity.
Start with Backtrader. That's my honest answer. Not because it's the most powerful tool in the ecosystem, but because it forces you to think about the right things early. The event-driven architecture makes market friction explicit from day one — you can't just assume perfect execution. You model your commissions, you model your slippage, you watch your beautiful backtested returns degrade into something more honest. That's a valuable education before you've risked anything.
Learn where your assumptions are before the market teaches you.
Once you've built a few strategies in Backtrader and actually understand what the framework is doing, the jump to something like Qlib is much more legible. You're not just running someone else's model zoo — you know what questions to ask about the pipeline.
What about data? Because that's the other obvious barrier. People assume you need expensive vendor feeds to do anything meaningful.
You don't, to start. Yahoo Finance's API and Alpha Vantage both give you clean historical price and fundamental data at no cost, or very low cost for higher rate limits. Alpha Vantage in particular has expanded its coverage considerably — you can get earnings data, economic indicators, some forex and crypto feeds. It's not Bloomberg, but for prototyping a strategy and stress-testing the logic, it's more than sufficient. The expensive vendors become relevant when you need tick data, when you need alternative data sets, when you need sub-second latency. None of that is relevant until you've validated that your strategy concept actually has legs.
There's almost an argument that starting with lower-resolution data is a feature rather than a bug. If your strategy only works with tick-level precision, it's probably a strategy that requires infrastructure you don't have yet. If it works on daily closes, you've found something more robust.
That's a good heuristic. Strategies that require increasingly granular data to show a positive backtest are often strategies that are fitting to microstructure noise rather than to any real underlying phenomenon. The signal should be visible at a coarser resolution if it's real. Which brings up the validation question, because backtesting on the data you used to build the strategy is one of the oldest traps in this space.
Walk-forward analysis is non-negotiable. The basic discipline is: you train on a window, test on the period immediately after, roll forward, repeat. What you're checking is whether the strategy's performance degrades predictably as you move out of sample, or whether it falls off a cliff — which tells you you've overfit. It's not a guarantee of live performance, but a strategy that can't pass walk-forward testing has essentially no argument for live deployment.
The tools we've talked about all support that workflow natively at this point.
Backtrader, Qlib, most of the mature frameworks — yes. The infrastructure isn't the hard part. The hard part is the discipline to actually run the test and then believe the results when they're unflattering.
That discipline to believe unflattering results. That's probably the single most transferable skill in this entire space, honestly.
It really is. And it applies whether you're running a simple momentum strategy in Backtrader or a full reinforcement learning setup in FinRL. The tooling doesn't save you from yourself.
Which leaves me with the question I keep circling back to. If the tools keep getting better, the data keeps getting cheaper, and the models keep getting smarter — does the market eventually run out of inefficiencies to exploit? Does AI-driven trading eat itself?
It's the question, right? My instinct is no, but not for a reassuring reason. Markets are made of people, and people keep generating new forms of irrationality. The inefficiencies shift rather than disappear. The low-hanging ones get arbitraged away fast, but new ones emerge from behavioral patterns, from regulatory changes, from the interactions between the algorithms themselves. Flash crashes are a form of inefficiency the algorithms created.
The algorithms generate the next generation of inefficiencies. There's something almost ecological about that.
Predator-prey dynamics, but everyone's running Python. And there's actually a fun historical footnote here that I think about sometimes — the first documented case of a purely algorithmic trading strategy causing a market disruption wasn't 2010. It was 1987. Program trading — the practice of using computers to automatically execute large basket trades tied to index arbitrage — was widely cited as an accelerant of Black Monday. The Dow dropped twenty-two percent in a single session, and a significant part of the mechanism was automated sell programs triggering other automated sell programs in a feedback loop. The tools were primitive by today's standards, but the dynamic was identical to what we see in modern flash crashes. Algorithms creating the very instability they were designed to exploit.
Which means we've been running this particular experiment for nearly forty years and we keep being surprised by the results.
We keep building more sophisticated versions of the same feedback loop and then acting shocked when it loops. And there's one more disruption sitting on the horizon that I don't think gets enough attention in this conversation — quantum-resistant encryption. The current HFT architecture depends heavily on specific cryptographic protocols for secure, low-latency communication between trading systems and exchanges. The projected rollout of post-quantum cryptographic standards in 2027 is going to force significant re-engineering of that infrastructure. Latency profiles will change. Some of the speed advantages baked into existing HFT setups may not survive the transition intact.
Even if you've built the perfect algorithm, the plumbing underneath it is about to get rebuilt.
Which is very much a finance problem masquerading as a cryptography problem.
It's a reminder that the edge in this space has never been purely about the strategy. It's about the whole stack — the data, the framework, the execution infrastructure, the regulatory environment, the cryptographic substrate that nobody thinks about until it changes. Any one of those layers shifting can redistribute the competitive landscape in ways that have nothing to do with how smart your model is.
That's a unsettling place to leave people. Thanks to Hilbert Flumingtop for producing, and to Modal for keeping our compute bills manageable. If you've got a minute, a review on Spotify goes a long way. This has been My Weird Prompts. We'll see you next time.
