A Statistical FX Model

I’m still thinking about factor models and how to approach them in different ways. This time I’ve let the data decide the relevant factors through PCA, hence why it’s a statistical factor model: A Statistical Factor Model.

But like the fundamental model, it still doesn’t come up with any significant factors. After all, it is still FX data and this is a difficult asset class to build a factor model with. Ultimately, FX doesn’t move enough and I’m missing the most relevant factor - interest rates!

Subscribe now

Fuel Insights

With petrol prices getting rocked by the war in Iran, small talk is increasingly revolving around how much it costs to fill up. By coincidence, the UK government launched its FuelFinder scheme in Feb 2026. This mandates petrol stations to report their petrol prices in near real-time and the government takes this data and makes it available through a public API. Fuel Insights consumes and saves this data to give a historical view of petrol prices across the UK. Turns out petrol on the motorway commands about a 28p premium and supermarkets are about 2p cheaper on average. It’s a slick dashboard with unique data from the government. So whilst the FuelFinder scheme is in its early days, things like this show why making data available leads to good things. I’d be interested to see if the data leads to any real change in pricing behaviour from the public. Looking at my local data I need to drive to my nearest Costco to get the cheapest petrol!

Leave a comment

Dog Treats

There’s nothing I enjoy more than a novel use of statistics. This time it’s the Bradley-Terry model and using it to work out what particular brand of treats this person’s dog enjoys the most.

https://www.wespiser.com/posts/2026-06-19-best-dog-treat.html

The Bradley-Terry model is about pairwise comparison and seeing what “thing” wins when in competition. I’ve used it before to look at red cards (New Applications of Bradley Terry Models]) and this type of model is used frequently in sports modelling and drug trials. I’m up for anything that helps me understand man’s best friend a bit more. Although for my dog, the most expensive treat would be the best predictor.

Zeta Monkey

Mental arithmetic is something that comes up in finance interviews. You are expected to add, subtract, multiply and divide different amounts quickly and without a calculator. ZetaMonkey is a well-presented website that lets you practice those problems in an aesthetically pleasing way. It does one thing and does it well. See if you can beat my personal best of 20.

As always, feel free to reply directly to this email or leave a comment on the post with your thoughts.

Happy modeling and enjoy the last bit of summer!

Subscribe now

I had just attended two virtual conferences: UseR and JuliaCon. I mentioned it in my last look back, but virtual conferences, while necessary at the time, are not something I miss. The microphone I bought especially for recording video still sits gathering dust, and I occasionally threaten my wife that I might start a podcast. The rest is market microstructure, anyone?

For UseR, it’s my State of the Market post and accompanying video. I’d say this is the earliest post of mine that actually got some eyeballs on it—only took me six years of blogging!

For the JuliaCon talks, the high-frequency covariance matrices made a lasting impression. The speaker also wrote a paper that I replicated in Importance Sampling, Reinforcement Learning and Getting More From The Data You Have. Both the paper and my replication are worth checking out.

And finally, CoinbasePro.jl is still alive and kicking. If it ain’t broke, don’t fix it. I might think about letting AI take a look to see if there are any clear bugs I’ve missed, though.

Thanks for reading — enjoy the look back to 2021.

Dean

From the Archives - August 2021

This month’s newsletter will be video-themed. I’ve just come off the back of two conferences (UseR and JuliaCon), so I’ll point you towards a long and a short talk.

First up is JET.jl, a static code analyser for Julia. This talk goes through how to use JET to check your Julia code for any errors without running it. It’s a tool that should be in any Julia programmer’s repertoire, and after running it on my packages I found a number of bugs that I then fixed.

A shorter talk (just 10 minutes) on a practical application of Julia was given by a quant from Credit Suisse on high-frequency covariance matrices. I recommend this for anyone interested in calculating correlation and how it can go wrong when you have lots of data over time.

I’ve also uploaded my virtual poster presentation for UseR as my YouTube debut:

It gives a quick explanation of this blog post.

Finally, I’ve updated CoinbasePro.jl to a new version that can now handle the nanoseconds in the trade data.

Thanks for reading — hopefully you can watch one of the above videos over breakfast, lunch, or dinner depending on how fast you eat!

Leave a comment

Bank Holiday Building: Two Little Cloudflare Projects

Over the bank holiday weekend, I built two little projects using Cloudflare’s suite of developer tools. I’m not sure whether AI has made me more productive or if it’s just given me some motivation to get things done and out there. This newfound inspiration has led me to churn out two little projects: a personal train timetable so I can get home quicker and a mini dashboard for FX volumes.

The Joys of Free Cloudflare

Both of them are built using the free tier of Cloudflare, and I’ve really enjoyed learning these new technologies. My post gives a run-through of how I approached it and what the tools do. The best thing is that you can get going without having to put any payment details in. So avoid any catastrophic bills if something takes off!

Subscribe now

Is AI Writing Too Much Code?

I’m at a crossroads with AI. No doubt it’s the future, but the current iteration of using an agent program feels like the payday loan equivalent of tech debt. I find the code it comes out with is so verbose, and it aims to maximise line count above everything else.

Turns out, you can measure this. In Minimal Editing, the author sets up broken programs and asks various AI models to fix them. Each model is then scored on how different the solution is from the original program. If a model turns out lines and lines of code, then it scores poorly, whereas a model that makes a simple, targeted change does a good job. If you’re missing a semicolon somewhere, the best solution just adds it in, and it doesn’t start null-checking and needlessly catching errors everywhere.

They find Opus 4.6 does the best, with GPT 5.4 doing the most editing. But if you add an explicit instruction to the prompt to minimise the amount of editing, things also improve. This is where using AI still feels a bit rusty. Things like writing a good prompt and knowing the right magic words/incantations are all too opaque and a bit annoying to tune.

There’s a lot more detail in the post, and it goes further with the experiments — worth a read if you find yourself wrestling with bloated AI-generated code.

Leave a comment

What I’m Listening To: Gamecraft

Now, if there is any industry where AI is kicking up a storm, then look no further than game development.

The new series of Gamecraft has started. In this podcast, they talk about the business of video games but focus on the current rather than a historical retrospective, and a recurring theme of this series is where AI is slotting into this industry.

In the first episode of the new season (2026 Trends and Specific Prediction), they have a small section on AI and their experiences with OpenClaw. Like I said, very current rather than historical. It leads into how AI is going to change building game software and how the first AI game might look in 2026. This episode is a good jumping-off point for the podcast. Listen to this one, and go through the catalogue if you like it. If you are a fan of Acquired, you will enjoy Gamecraft.

If you want just the deeper dive into AI, then episode 30 looks at the real future of AI in the games business. Games and AI are an interesting intersection. On the one hand, you have the coders using AI to write better code and be more productive, things where everyone can agree AI is an improvement to how things get done. On the other side of the coin, you have the artists and designers, the creatives. How AI helps them is much more nuanced, and everyone starts to get a bit frosty when creative elements start to lose the human touch. The podcast is a balanced take on both of these angles, but really highlights how we are currently in the complete unknown based on how things are evolving. When I started drafting this newsletter, Opus 4.6 could do no wrong. Now that I’ve sent it, you’ll see more people telling you how Codex is now superior. Personally, I’ll still be crafting these newsletters and posts the old-fashioned way, but perhaps Claude or Gemini will become the editor I’ve always needed.

That’s it for May. As always, reply if anything catches your eye.

Subscribe in case I ever make the FX Volume Dashboard public!

Dean

Football aside, my intro to high-frequency finance series is one of my better-performing blog posts and eventually evolved into a sponsored post for QuestDB. I stumbled across a whole section on post-trade analysis QuestDB - Post Trade Overview in their docs the other day.

Then Kats, the Facebook/Meta time series library. I never got around to using it or writing a blog post about it. It’s not had a release since 2022, so I missed the boat on that one.

And finally, JuliaCon 2021. I presented a poster. Virtually. A strange experience sitting in my room in a virtual conference, waiting for people to look at a 60s video. I don’t think anyone did in the end. If you’ve ever wanted to see my talks from any of my conferences, virtual and in person, I’ve created a YouTube playlist:

Dean's Conference Talks

Hawkes Process, Fantasy Football, RFQ trading. There is something for everyone in that playlist.

Subscribe now

## From the Archive - July 2021

I write this email with two faded England flags painted on my face, and my heart is heavy with another near miss at the football. The loss to Iceland in 2016 was a small price to pay to be in our position now. So onwards to the World Cup.

Onto the regular content. Last month’s post was an introduction to some high-frequency finance concepts. Using my CoinbasePro.jl Julia package, I walk you through some of the main calculations and examples, all with free data so you can play along at home:

Getting Started with High Frequency Finance using Crypto Data and Julia

Next up is a new piece of time series analysis software from Facebook: Kats.

https://facebookresearch.github.io/Kats/

You might have heard of Prophet, another Facebook timeseries library, and this is supposed to be something that builds on top of the lessons from Prophet. It has a slightly different feature set, and I will probably be exploring it in a blog post in the future.

Finally, it has been conference season. I spoke at userR this week and am lined up to present at JuliaCon at the end of July. You can register for free, and I’m sure there will be something you find interesting over the two days:

https://juliacon.org/2021/tickets/

There we go. Hopefully you’ve found something interesting in this month’s email. Until next time!

Subscribe now

My post for March (and first one for 2026) covers two topics that have been on my to-do list. 1. Replicating the DXY calculation and 2. Start thinking about factor models.

Before we start the factor models we need the data and this post looks at a new (free) data source to get currency data. I also run through what the DXY is, how to calculate it and how currencies move with it: Making Sense of the DXY.

This post is step zero in my attempt to build out an FX factor model and I take a lot of the functionality into my next upcoming post. Interestingly, the DXY has been around since 1973 and has only been updated once, 1999 for the euro, so it might be considered a bit legacy and due for an update.

Continuing the theme of systems that might be in need of updating I’ve got two links, one on legacy government IT and another on how to move up the optimisation ladder.

How Big Things Get Done was one of my favourites reads last year and it touches on government IT upgrades and why they are so difficult. This, Why Some DVLA Digital Services Don’t Work at Night, is a nice anecdote from someone on the front line of such projects.

The DVLA is the motoring branch of the British government. It handles things like road tax, driving licenses and car registrations. You can do most of these things online, but with a specific catch, you can only do things during working hours. This turns out to be because of an IT problem. an, an unfinished migration from the legacy systems to something more modern. So now it is the worst of both systems. There is a legacy batch job that updates overnight and has to complete with no issues and can’t be edited in realtime . This leads to a system that can only operate in working hours. Entries stop as the working days end, the batch job can complete overnight and everything opens up the next day.

So in essence it’s a compromise. Public sector IT migrations are hard and the cost of doing things online means we have to restrict it to business hours. Would it be nice to do it 24 hours a day? Yes. Do I want to pay more tax to make it happen? Not really! Would throwing money resolve it? Also unlikely.

Subscribe now

On the subject of programs running slow, The Optimization Ladder, is a great run through of practical steps to speed up Python once you’ve written the core logic.

I think we’ve all been faced with code that we want to run quicker. As the phrase goes premature optimization is the root of all evil, so only once you’ve finished the bulk of the work you start thinking about how to make it run faster. Once you’ve done the simple things, pr-allocate arrays, made sure your types are consistent etc., you are know wondering where the next big speedup comes from. This blog post steps you through each of the steps and how much speed up you can get versus vs how much effort.

So if it’s the DXY, a website to get your driving license or a humble python script there’s always a way to make them better. The real question is whether the juice is worth the squeeze either in time or money.

Until next month.

Dean

It’s the first announcement of my crypto pre-trade tool cryptoliquiditymetrics. This is a simple web app that allows you to assess the current state of the cryptocurrency market and estimate the cost of executing a specific amount. It’s nothing too complicated, and my first foray into JavaScript hosting something on Netlfy. I should have the coding assistants try to enhance it a bit now.

The second link was a blog site similar to mine, and the dinosaur post below still shows how there are interesting data sets out there to practice your modelling on. They are still writing to this day. In fact their latest post [Decaying Bayesian Updating for Non-stationary Time Series Models] they are looking at how to fit Bayesian models on time series data and how to decay the updates to account for non-stationary data.

Tamara Broderick’s website is a gold mine of stats, machine learning info and tutorials. The full machine learning course linked below is well worth a browse, as are the other sections.

Then I ended with my blog post on causal machine learning. Double machine learning is a method of untangling the causal effect of a variable, trying to understand if it actually causes a change and isn’t statistical noise. Double machine learning links both more advanced statistical learning tools and causal inference.

Subscribe now

The Late May Edition - 2021

May’s edition of my newsletter is a bit delayed. It was a bank holiday weekend here in the UK, the sun was out, so I did the right thing and spent too long outside getting burnt. Still, I’ve curated some good links to make the wait worth it.

Firstly, the release of my side-project:

https://cryptoliquiditymetrics.com

This is a little application I’ve cooked up that reflects current market conditions across lots of different cryptocurrencies. You enter how much you would like to trade and I’ll plot some graphs, calculate some statistics and hopefully give you an insight into what’s happening at the minute. It’s built just using JavaScript and was an excuse for me to actually do something on the web rather than the usual R/Julia stuff I build.

As a child, I wanted to be a paleontologist and would wow adults with my dinosaur knowledge picked up from various picture books. So when this post appeared on my Twitter feed, I was overjoyed.

https://austinrochford.com/posts/2021-05-17-tyrannosaurid-pymc3.html

20 big dinosaur fossils are modelled using PyMC3 to build a model relating their age and mass. From then on, it walks you through different types of models that could best represent the data. In fact, it is very similar to the types of posts that I write, and I might take the same data for a Turing tutorial in Julia. Inspiration is just hiding your sources after all.

Next up is a series of machine learning lectures from Tamara Broderick. YouTube links and slides across 14 lectures covering everything from basic regression to reinforcement learning and all the topics inbetween.

http://people.csail.mit.edu/tbroderick/ml.html

I’ve seen her speak a couple of times and would highly recommend her work. So this series of lectures will be of top quality.

Now, finally, my latest blog post on double machine learning.

http://dm13450.github.io/2021/05/28/Double-ML.html

Most of my new subscribers have come after reading this, hence why I’ve left it to the end. Still, maybe you would like to read it again.

Thanks for reading, now go out and enjoy the sun this weekend!

Dean

Subscribe now

First up, a (very) short memoir from Baum of Baum-Welch algorithm fame.

Baum as a trader details his days trading with Jim Simmons of Renaissance Technologies, and it’s a fun read about punting currency and coffee with very little data and reading books to get an idea of previous patterns. Given Jim Simmons is a grandfather of quant investing, it’s interesting how it seems so ad hoc but eventually grows into a model-based approach. The Man for All Markets has been in my Kindle library for a while now; perhaps it’s time to finally crack it open.

Given I’ve been in and out of Heathrow the last few weeks, this post that made its way up Hacker News piqued my interest.

What's going on at Heathrow Airport

Billy creates a lightweight machine learning model to classify which Heathrow runway is used for takeoff and landing. It’s trained on OpenSky Network data, and the model ends up so lightweight that it’s all served from your browser. The result is LHR NET that simply tells you which runway is currently being used. Fun little project.

Two Sigma have a few videos on their YouTube channel that look at machine/deep learning and how it applies to finance. The Deep Learning for Sequences

video is a great introduction to deep learning and how it can be applied to the different aspects of quantitative investing. It also harks back to the COVID days of everyone working from home, if you are nostalgic for that vibe.

To end with, we look back at Feb 2024 and my post on cross-asset skew - Cross Asset Skew - A Trading Strategy. I enjoyed writing this post as it was something a bit different to my usual high-frequency stuff. Instead, I constructed a daily signal of the ‘skew’ of an ETF and then created a sorted portfolio to back test if there are excess returns. I’ve been reading Giuseppe Paleologo’s new(ish) book, Elements of Quantative Investing, and it’s given me some ideas on how to take this skew strategy further.

And that’s it. Let’s hope the next month brings slightly less news.

Share

What better way to look at what can happen this year than an AI-assisted summary of all the year-ahead pieces from various banks and asset managers.

2026 Outlook - views across the street

The ‘year ahead’ is almost mandatory research for the big financial firms to put their stake in the ground over what will and won’t move markets, general themes and geopolitics. It can be as simple as the S&P 500 will be higher/lower/unchanged (delete as appropriate) to guessing (forecasting) how countries will respond to each other’s policies.

This site takes 14 of these research reports and uses GPT (5.1) to give a summary and compare where the analysts agree and disagree with citations to try and minimise hte amount of hallucinations. The author (https://www.charlje.com/) checked the results and found some errors, but minimal and trustworthy enough to release. When I clicked through the links, I found that it seemed to be ok with nothing wildly taken out of context/made up. So rather than reading the 14 reports individually, use the summary to give you a jumping-off point. The next step would be to look at this at the end of the year and see how right/wrong the overall predictions were. Then you have the beginnings of an Alpha Capture strategy.

Subscribe now

It’s also that time of the year when we find out how much we are really worth. The FT has a great piece on how hedge funds pay their portfolio managers.

How top hedge funds can pay traders $100mn

Multi-strategy hedge funds need new ways of obtaining and retaining talent to generate returns, and as such, a PM’s pay packet becomes a game of incentives, clauses and ultimately, the end investor having to foot the bill in a down year. Consequently, it means bonuses are now an expense for the investor rather than something taken from the fund fees. There are some nice graphics in the article, and it shines a light on something that most people will never have to worry about, but like hearing about. Wonder if they still go through PAYE?

Subscribe now

What else is January for? Well, according to all the adverts I’m seeing, it’s booking holidays. Disneyland Paris has a new Frozen section (the film, not everything is now covered in ice) and as part of this new attraction they have built a hyper realistic Olaf. Bare with me, this is relevant if you subscribe to this newsletter. A video of this animatronic in action:

To me, this was incredibly impressive. They used reinforcement learning to train the robot’s actions and movements and had to make sure the robot didn’t overheat its motors. There is also the ‘special’ way in which Olaf walks to make it as close to the animation as possible. They even wrote a paper (told you it was relevant) Olaf: Bringing an Animated Character to Life in the Physical World. For me, to see reinforcement learning used in such a practical way is impressive and shows that there is a wide world of machine learning uses beyond just fancier statistical models.

I’ll leave it there for Jan. Have a good start to the year!

Share

First up was my piece on limit prices and algo executions. Still up on FX Markets and still relevant today.

Then some classic Julia/algorithm work. I accidentally wrote a piece of code that was O(N^2), so as the data grew, it slowed down. A quick Google and rewrite managed to get it down to a more reasonable time.

The post45 piece is a pretty in-depth look at GoodReads data and what people call a ‘classic’. It chucks in some clustering of books and reflects on what drives the proliferation of classics. If your novel gets tagged as a classic, schools will read it, and suddenly, you have a consistent demand to publish and sell the book yearly! It’s my favourite kind of post, unusual data and interesting statistics. I really undercooked my blurb in the original!

For the Paradigm research, I’ve got a draft blog post of the TWAP contract knocking about. I should quickly finish that off and get it out just to finally promote it from drafts to published.

Share

April 2021

The first item on this month’s newsletter is something out of my day job. My latest research has been featured in FX Markets, and the article (for FX-Markets subscribers) can be found here.

Price Limits in FX Algos - Fill Your Boots

In short, I found that using a limit on an algo trade doesn’t affect its performance. So you can freely set a limit on your algo trades and not worry about reducing performance.

Next up is my latest blog post:

Accidentally Quadratic

This document is my process of finding a bug in my code at work because I had written a slow function. It was a surprise as the slowdown only happened once the dataset got large enough. In this post, you’ll see how slow and how my fix drastically reduced the running times.

Now for two interesting links:

The Goodreads “Classics”: A Computational Study of Readers, Amazon, and Crowdsourced Amateur Criticism

It is quite a few words, but it is worth it for the pretty plots. It looks at how we classify what we think a ‘classic’ book is and how the Goodreads data can be explored to answer such questions.

Now for fewer plots and more maths. I’ve been in a bit of a DeFi rabbit hole recently. Looking at all the different platforms (UniSwap, PancakeSwap, SushiSwap) for yield farming, staking and all those things. Like everything else in my life, I like to ruin it with numbers.

https://research.paradigm.xyz/

They post lots of articles on such topics, with maths explaining all the key concepts. They’ve also got whitepapers on other topics. Another resource for math and finance.

There we go. Another month and some more reading.

Subscribe now

We’re all told at some point in time that we can no longer play the children’s game, we just don’t... don’t know when that’s gonna be. Some of us are told at eighteen, some of us are told at forty, but we’re all told.

This is how I feel about having to write some posts in Python rather than Julia. I’ve needed to find some time to sharpen my Python skills, and unfortunately, that means I’ll have to reduce my Julia blog writing time. Expect a few more Python-based posts as I learn Polars, Plotly, Scikit-Learn, and all the other major packages.

Which leads me into my latest post. I’ve taken a look at the data to understand how many points you need to survive in the Premier League. With all the promoted teams doing well and some of the stalwart teams not doing so well, we could be in for a season where things are shaken up.

Premier League Survival – How Many Points Are Enough?

This is my foray into Python and getting my teeth into the new way of doing things, mainly Polars, which is an alternative to Pandas and working with dataframes. All in, I didn’t mind it. Python has its quirks (that Julia mainly fixes), but I need to bring my Python up to my level of Julia. Next up will be building out some models using Scikit-Learn and seeing how it compares to MLJ.jl.

Subscribe now

I also attended the Financial Engineering Workshop last week to listen to a talk by HSBC on Market Making in Foreign Exchange. The slides are here - Market Making in Foreign Exchange. It’s an intro to the FX market for anyone going in blind and then quickly jumps into lots of the maths, but with some good graphs on the practicalities of FX market making.

Finally, there was an interesting story around HashiCorp and how it approached (or rather ignored) its business banking in the early stages of the company. My Startup Business Banking Story. It’s a fun little read. At one point, they had $335mln in a regular business account!

Until the next newsletter!

Dean

Back in March 2021, Julia 1.6 had just come out and was a big release - the largest since Julia 1.0 in 2018. It promised (and delivered) major speedups in package loading, which had been an ongoing criticism. Judging from my brief review below, it worked, and in fact, it's worked so well that I only have a brief memory of importing the packages and waiting multiple minutes for it to finish. Although in my day-to-day with KDB, I now spend an inordinate amount of time waiting for HDBs to load.

I also had a couple of blog posts, Crypto Data using AlphaVantage.jl and Does a Terror Attack Lead to More Terror Attacks? . Plus the first release of Coinbase.jl. Oh, and nice COVID lockdown reference to round it all off.

Overall, more of the same. I'm still using Julia, still writing about crypto occasionally, and I'm even doing a presentation on Hawkes Processes this week at work.

Subscribe now

The March Update - 2021

This month has been one of new releases:

Julia 1.6 has come out. If you've ever tried Julia and been put off by the slow compilations or how sluggish everything feels to start up, well, now is the time to try it again.

There has been much work on making things spin up quicker and trying to reduce the initial wind-up time as everything is being prepared. I've noticed the difference, so I urge anyone thinking about it to try it out again and get stuck into some Julia.

I've written a short blog post on the AlphaVantage.jl library and how it can be used to pull crypto data. This was all done using Julia 1.6, and I felt the compilation speed increased.

Crypto Data using AlphaVantage.jl

I've also released CoinbasePro.jl into the general repo.

This provides methods for getting high-frequency crypto data into Julia. The great thing about crypto data is that it is free. To get the equivalent data in, say, a stock or currency pair, you are probably looking at hundreds of dollars a month. Companies like Coinbase and other exchanges provide this data for free, democratising quant finance.

It's still in the early stages for the package, and a blog post is in the works, but any early adopters can have a play around with the package and let me know what they think.

• Coinbase.jl

In case you missed it, last month's blog post was on Hawkes processes and a look at how they can be used to model terror attacks. Again, in Julia using my package that I wrote throughout my PhD.

• Does a Terror Attack Lead to More Terror Attacks?

That's been about it for this month. Desperately waiting for the lockdown restrictions to loosen so I can get out on the golf course again!

Long story short, time variables run in cycles (e.g., days of the week where Monday follows Sunday), and you can use simple trigonometric functions to encode this cyclical nature. For the simple linear models in the blog post, they don't have a massive effect on the performance, but as a sneak peek for the next blog post, it works better in the neural nets.

Subscribe now

After moving over to my new laptop, I managed to update AlpacaMarkets.jl and CoinbasePro.jl. It was mainly upgrading the dependencies, as they were using older versions of packages that stopped my overall Julia installation from updating (and probably yours if you had the packages installed!). Some other minor tweaks and bugfixes as well.

An interesting paper recently came through Hacker News - Maximizing Battery Storage Profits via High-Frequency Intraday Trading. I enjoyed it as it provides a good overview of the structure of the electricity market. For example, given once electricity is generated it needs to be used or stored quickly and there are contracts expiring every 15 minutes throughout the day to balance the supply and demand across Europe. It's reminiscent of TV Pick Up phenomenon, where UK electricity demand spikes as popular TV shows and football matches end, so the UK grid needs to quickly source more electricity as these pick-ups occur. Overall, it's a different type of trading paper that offers an alternative angle on similar problems. Instead of slippage and transaction costs, you have battery efficiency and degradation. They even have a GitHub with the code - bitepy.

And finally, an interesting mountaineering read about a relatively unknown disaster. Mountaineering Deadliest Avalanche Disaster Lenin Peak. I have a passing interest in mountain climbing, mainly fantasising about having enough time off at work to even think about summiting a peak! I've even written about predicting a successful Mt Everest Climb.

Until the next post!

Share

My dataset was way too small to produce meaningful results but it was more of a post exploring how to fit the models rather than a true data calibration.

Subscribe now

On a similar note a very interesting paper, Generating realistic metaorders from public data, looks at how you can use the public trade data to simulate parent (meta) orders. Given my blog post outlined the two ways you fit price impact models (public trades vs private trades) this new method introduced in the paper gives you a way of coming up with simulated private trades using the available public data. I’ve replicated the results at work and it looks promising. All in all, a simple method using the law of large numbers!

Next up is a good introduction to the practical realities of the FX market and how it is structured - The Brilliant World of FX. This is similar to the equity market structure document from Proof Trading I posted previously (Jan's Newsletter), but this time looking at the FX market.

Finally, a bit of fun—What Really Happens Inside a Dating App. This is a deep dive into the statistics of modern dating apps, how they function, and what that means for men and women separately. There is nothing quite like trying to moneyball your Tinder profile!

Importance Sampling, Reinforcement Learning and Getting More From The Data You Have

When you have something running and working in production you want some evidence that a change will improve things. With importance sampling you can re-weight the production data to provide information on a strategy that hasn’t even been run in production.

Here, we have an xgboost model (blue line) running in production. By re-weighting the observed data (using equal weights, orange line), we can see the performance is worse, so we should not make the change. This blog post was inspired by Choosing Trading Strategies in Electronic Execution using Importance Sampling.

Equity Market Structure Primer

Proof Trading have a nice pdf that offers a basic introduction to the equity market landscape - Proof Market Structure Primer. It lightly covers most of the essential topics in equity trading. Good reading if you're from another asset class (me!) or just getting started in the world of electronic trading. They have a few other interesting reads in the blog section of their website.

Subscribe now

NYC Congestion Pricing Dashboard

NYC has bought in congestion pricing in an attempt to reduce traffic. This is a dashboard that compares the traffic both before and after the introduction of the congestion pricing but also a comparison to Boston and Chicago.

The Congestion Pricing Tracker is a simple dashboard but an interesting way of viewing the results of the experiment. From the data some routes look like they have improved things, but still early days. In a perfect world, we would AB test this type of tax (each day randomises whether you have to pay or not) but that might be a bit politically difficult to sell! London has had congestion pricing for 20 years and still can’t decide whether it had a causal effect on traffic - Re-assessing the Results of the London Congestion Charging Scheme.

And just for fun, turns out you can easily solve Spot the Difference puzzles if you cross your eyes when looking at the two pictures - Spot the Difference Superpower. Now I just need to practice crossing my eyes to see if it’s true.

Happy New Year all!

Dean

First is a piece on Alpha Capture and Acquired. Does the Acquired podcast have Alpha? We want to know when they release a podcast on a company does the stock rise or fall in the days after the release date?

My first time writing about how AI helped collect some data too.

Subscribe now

Next is a piece from the FT - ETFs are eating the bond market. This runs through fixed-income ETFs, their growth mechanics and likely consequences and also touches on the growth of electronic trading in the same space.

At the start of the decade it was a meme that bonds were always teetering on a liquidity crisis and the growth of the ETFs/passive investing was a powder keg waiting to explode when there was true distress in the market. Well, we ended up having the true distress in 2020 and the research suggests that the ETFs helped provide liquidity where it was needed rather than hamper it. It’s a good article that gets you up to speed with the modern landscape of fixed-income trading.

Next up is a new paper from my colleagues - BofA quants propose new model for when to hold, when to sell. It explores a mathematical model of when to take profit or stop loss after a period of holding the risk to give a closed-form solution when you have the market spread and client order arrival rate. Many of these types of problems are solved through optimal control so good to see (in my biased position) an analytical solution.

Speaking of market-making problems and solutions, the recent Unwinding Toxic Flow with Partial Information paper now has accompanying code in a Colab notebook.

As someone who learns better through code than maths, it gives me a chance to fully understand the paper! Also, first time I’ve used a Colab notebook in the wild.

And finally, a fun, if a bit rambly (in a good way!) read about the state of the tech job market from someone’s personal blog - Panic at the Job Market.

Happy Q4 to all those who celebrate and until next time!

Dean

Share

This paper and its concepts are the foundations of optimal execution and it is always good to know the classics. Once you have this under your belt more complicated models await!

Resolve Asset Management have released a white paper on carry - Managed Futures Carry - A Practitioners Guide. This is a good paper that starts with the basics as carry is a fundamental quant strategy but more complicated than momentum. You can use the CHRISS Wiki Continuous Futures dataset to get futures data of different expiry which gives you the basic data needed to replicate the calculations.

Subscribe now

Another link that cropped up on my Twitter feed was CookieCutter Data Science. It’s a template for setting up data science projects and acts as scaffolding for common tasks. It’s about getting organised, setting up the right packages and having opinions on how a data project should be structured. If you find yourself making the same mistakes/bad decisions when doing this type of work you can learn from this setup.

And finally, a nice article on Bank of America’s TWAP and VWAP recent enhancements: BofA Flags Algo Improvement as it Adds VWAP to Bloomberg.

Thanks for reading,

Dean

We enter the last week of Feb so it’s about time I fired off my first newsletter. This will be a collection of things I've read over the month, things I've found interesting, have inspired other ideas and everything in between.

First up, a presentation on data communication. Data Story Telling by Ted Laderas popped up on Twitter and is a good walkthrough of how to build an effective data story.

Everyone, junior or senior, could take something away from this talk.

Next is a new data source, or at least multiple data sources. The Sports Reference website has easily digestible stats for so many different sports; baseball, American football, basketball, ice hockey and of course, proper football (soccer). So take a look and take in all the wonderful numbers.

Finally, here are updates on the projects I maintain/contribute to:

• AlphaVantage.jl released version 0.3. This included a rewrite of the authentication methods and included IPO and earnings calendar functionality.

• HawkesProcesses.jl is slowly building out the documentation to try and help people use it.

Subscribe now

Plus updates on my professional work:

A new metric for quantifying algo execution:

• Measuring an Algo Fill Profile

A look at how Hawkes processes and Dirichlet processes can be used to classify different trading cost regimes.

• Volatility Spikes and Impacts on Liquidity

You've got to be a BestX customer to read those pieces, but the blurbs might be enough to inspire you!

There we go, stay safe out there until the next newsletter.

Overall not a bad first newsletter, the Data Story Telling presentation is timeless and I enjoyed rereading it. I still use Sports Reference when deciding on what to do with the Fantasy Premier League team (more on that in future Archives, Optimising Fantasy Football with JuMP). I've switched from Alphavantage to AlpacaMarkets.jl now for finance data but my HawkesProcesses package is still going strong. I've moved on from BestX anymore but that research is still relevant and useful.

First up is my latest blog post. I take a look at principal component analysis (PCA) and how we can reduce a basket of currencies to a ‘market portfolio’ and use this to hedge the overall exposure to these currencies.

Currency Hedging and Principal Component Analysis

I think it’s a practical example of PCA even if it took a bit of a kicking on the r/quant subreddit.

Stochastic control is one of those things that I frequently read about but struggle to articulate what it means on a practical application. Thankfully this document Stochastic Control Theory and Automated Market Making builds up the problem in an intuitive way, starting from basic calculus to the full market-making solution. My next step would be to write the code that implements the solution, perhaps a blog post for the future!

Subscribe now

As a KDB/q person, I am always on the lookout for better tutorials or help with improving my language comprehension or at least something beyond the official docs! This PDF, Learn K programming, idiom by idiom, came up on Hacker News a few weeks ago and details some common operations that are easy say Python, but not intuitive in q.

For example, if you have an array, x, and want to count all the occurrences of y

q)+/x=y

Compared to pythons

python) sum([z == y for z in x])

although Julia is a bit nicer

julia) sum(x .== y)

And finally, next week equities move to T+1 settlement. Feels like a fairly big change in the financial world and one in which the effects might not be immediately apparent. As an FX guy, I’ll be watching and observing to see if there are any major changes in my market structures.

Thanks for making it this far and until next month.

Dean

It is surprising how much uncertainty is in the parameters and my general conclusion was that you can perhaps have confidence in the sign of the parameters, but actual point estimates can vary wildly.

Subscribe now

If you are looking at getting started in Julia then this guide, Modern Julia Workflow, goes through the full process of getting Julia installed and writing scripts. If you are more of a veteran to Julia it goes into more advanced topics with a nice reference page at the end for all different packages that can help you optimise your Julia code. It’s good to see Julia get these types of tutorials as it matures into a more stable product.

Then on the academic side, there is a new free, online Macroeconomics book for graduate students: https://phdmacrobook.org/

It’s unfinished as of today, but so far almost 800 pages of core macroeconomic theory for free. It’s worth dipping in and out if you have a passing interest. I’ve been using it to top up my AS-Level economics (from over 10 years ago now, god, I’m getting old).

That’s all for this month, see you in 30 days!

Dean

Exploring Causal Regularisation

The term causal regression can be a bit daunting (at least it was for me!) but this type of analysis shows that if you are confident with basic regression/machine learning tasks then you can easily apply them to causal problems and understand the true relationships between your variables.

Crypto's Return

Bitcoin is making all-time highs again so that puts crypto back on the menu which also means I can once again plug my Crypto PreTrade dashboard

Crypto Liquidity Metrics

It's a quick and dirty view of current market conditions built just using JavaScript and hosted with Netlify.

On the investing front, Sparkline Capital wrote an interesting piece on using factors in the crypto markets

Crypto Factor Investing

The proposed crypto equivalent of the 60/40 portfolio (60% equities, 40% bonds) is 60% Bitcoin, 20% large-cap momentum and 20% small-cap intangible value, which passes the common sense test.

It's a well-written post with some nice graphics that also works as a good introduction to factor investing. Given last month I wrote about the Skew Factor () it's timely to take another look at factor-based finance.

Any Other Business

Another thing about moving to Substack, they are big on getting you to recommend other Substacks and get some network effects going. I'm not going to do that, instead, I'm going to highlight a random blog I found (and admired) over the years.

https://sebastiancallh.github.io/

Similar to my posts, they cover a wide range of topics in a beginner-friendly way and give you that practical introduction to lots of different topics. If you like what I write about you will probably enjoy this blog too. Shame it hasn't been updated since August!

I also finally got around to getting Julia 1.10 installed on my machine but this time using juliaup which is a Julia version manager. So now it's just a terminal command to update to the latest version rather than a download and point your shell to a binary. Annoyingly it hasn't been picked up by IJulia, so I need to spend some time working out what has happened there. This meant running my quarterly ```brew upgrade``` which locks down my computer for a few hours. Should probably thinking about replacing my late 2013 MacBook with a shiny M3 one.

And that's it for March, enjoy the end of Q1 and until April!

Share Dean’s Substack