we will stop providing Excel Cloud Share Links within the next 4 weeks.

If you are currently using these links, please consider subscribing to Portfolio123.com so I can add you to the P123 research group of this Substack — or consider cancelling your subscription.

Reasons:

— I cannot automate the Excel lists
— 95% of all paying users of this substack are…

Read more

All in all, our AI Factor Models do well!

Small Caps

Live Trading:

https://www.portfolio123.com/port_summary.jsp?portid=1853536

Live Trading and Holdout Backtest in one chart:

https://www.portfolio123.com/port_summary.jsp?portid=1853549

Large Caps

Live Trading:

https://www.portfolio123.com/port_summary.jsp?portid=1853545

Live Trading and Holdout Backtest in one chart:

https://www.portfolio123.com/port_summary.jsp?portid=1898601

Large Caps Combo AI / Actuals

Live Trading:

https://www.portfolio123.com/port_summary.jsp?portid=1886955

Live Trading and (Holdout) Backtest in one chart:

https://www.portfolio123.com/port_summary.jsp?portid=1920964

Have a great week!

Best regards,

Andreas

Hi, I am still wrestling with the full implications of this stuff, but I am writing it down before the idea escapes me — so if you are into infinity stuff, here is where my head is at ;-) As always the math is from Deepseek and Gemini, my favorite learning tools.

0. The Theoretical Foundation: Information Theory

Information Theory was formulated by mathematician and electrical engineer Claude Shannon in his landmark 1948 paper, “A Mathematical Theory of Communication” (published in the Bell System Technical Journal).

Shannon established the formal mathematical definition of Entropy (H) , measuring the fundamental uncertainty or noise present in a system, and Channel Capacity (C) , defined by the Shannon-Hartley Theorem:

C = B × log₂(1 + S/N)

Where B represents the bandwidth of the system and S/N is the Signal-to-Noise Ratio (SNR).

The Financial Implication

In modern capital markets, the Signal-to-Noise Ratio (S/N) of raw asset returns is infinitesimally close to zero. Under Shannon’s framework, when S/N → 0, the channel capacity to transmit pure alpha collapses unless you drastically expand the system’s informational bandwidth (B).

In quantitative finance, “expanding bandwidth” means abandoning low-dimensional, 10-factor linear screens and moving to high-dimensional feature representations (133–179 features) evaluated across deep historical cross-sections. You cannot extract a faint signal through a narrow pipe; you need maximum data bandwidth to overcome an environment dominated by noise.

1. The Huge Data Space (The Canvas)

Why >100M+ Datapoints Matter: In financial time-series, noise dominates signal by orders of magnitude. A massive dataset (15+ years of point-in-time cross-sectional fundamental and market data across thousands of stocks) acts as the necessary statistical bed.

The Math: According to the Law of Large Numbers, idiosyncratic noise (ε ~ N(0, σ²)) converges to zero only as sample size N → ∞. Without a wide and deep data space, the noise power drowns out the weak alpha signal.

2. The 133–179 Feature Space (The Complexity)***

Why 10 Features Fail: Markets are an adaptive, competitive ecosystem. If an alpha signal were simple (e.g., P/E < 10 + RSI < 30), arbitrageurs would immediately trade it to zero.

The Non-Linear Reality: True persistent alpha exists in multi-variable conditional setups — how a specific value metric interacts with balance sheet quality, earnings momentum acceleration, volatility compression, and sector-relative ranking simultaneously. Mapping that multidimensional surface requires 130+ features.

[***133–179 Features: Empirically Derived, this is my rule of thumb]

3. The Traversal Engine to be able to navigate infinity

The Problem: Once you feed (133–179 features into a non-linear tree framework, the theoretical search space explodes to 10⁵⁰⁰⁰⁰⁺, making brute-force evaluation mathematically impossible.

The Solution: You need a machine learning algorithm (like LightGBM or ExtraTrees) or a heavily regularized structural ensemble (like my clustered 1/N composite models):

The Factor Edge

My Design Rules for Portfolio Strategies with Traditional Ranking Systems

Introduction: The Hidden Math of Ranking Systems…

Read more

2 months ago · 7 likes · 6 comments · Andreas Himmelreich

These algorithms act as continuous mathematical navigators, efficiently carving a path directly to the sub-spaces where structural alpha resides, without getting lost in infinite noise.

Linear Models: The Structural Anchors & Benchmarks

Linear factor models — such as standard cross-sectional z-score ranks or ordinary least squares regressions — assume that relationships between features and forward returns are monotonic, additive, and continuous.

The Baseline Benchmark: Linear models establish the minimum acceptable threshold for any signal. If a complex 130+ feature LightGBM model cannot outperform a simple, transparent linear baseline out-of-sample, the non-linear model is merely overfitting to noise.

Structural Regularization: By enforcing linear constraints — or using hand-crafted 1/N composite nodes across feature clusters — you impose strict mathematical guardrails. These prevent the system from chasing regime-specific chimeras and act as an institutional “sanity check” on portfolio exposure.

The Fundamental Limitation: Linear models treat features in isolation or through rigid, pre-defined interaction terms. They assume that if a factor like high profitability is good, more of it is always strictly better in a straight line. They cannot dynamically capture conditional market realities.

Non-Linear Ensembles: The True Alpha Engines

Financial markets are non-linear, adaptive systems. The relationship between a signal and future return is rarely a straight line — it is bounded, conditional, and heavily regime-dependent.

Linear models assume continuous linearity — a flat, straight-line relationship between factor value and return. Non-linear ensembles capture sweet spots, interactions, and non-linear decay — they adapt to thresholds, diminishing returns, and regime shifts.

Mapping High-Dimensional Topologies: Tree-based ensembles (ExtraTrees, LightGBM) excel at navigating 133–179 feature spaces because decision trees naturally split data on multi-conditional logic — for example, is Value cheap AND Quality high AND Volatility compressing simultaneously?

Automated Interaction Discovery: Rather than manually guessing which non-linear terms matter, gradient boosted trees and randomized ensembles systematically traverse the 10⁵⁰⁰⁰⁰⁺ potential feature combinations, isolating only the conditional subspaces where signal statistically dominates noise.

Noise Mitigation via Randomization: LightGBM uses leaf-wise tree growth guided by gradient loss to quickly home in on dense signal regions across massive cross-sections (>100M+ data points). ExtraTrees (Extremely Randomized Trees) randomizes split thresholds entirely, effectively acting as an intense structural smoother that reduces variance and prevents the model from fitting to historical anomalies.

The Fundamental Takeaway

When the market’s remaining structural inefficiencies are non-linear and high-dimensional, then a Huge Data Space + High Feature Count + Guided Traversal is not just one way to build a quantitative model — it is the only mathematically viable way to extract signal from noise.

Most quants will avoid this path. Not because it is wrong — but because complexity aversion is deeply wired into human cognition. It feels safer to cling to simple models, to pretend that 10 factors are enough, to hope that the old rules still apply.

But reality does not care about our comfort zones. The market is what it is. And if the signal is hidden in high-dimensional, non-linear interactions, then low-dimensional linear models (the 10 Factor cohort on a 50 stock universe) are mathematically guaranteed to miss it.

We can fear infinity. Or we can learn to navigate it.

Best Regards,

Andreas

Link to AI Factor Model: https://www.portfolio123.com/sv/aiFactor/23341/overview

The AI Factor S&P500 model is configured to predict 3-month relative returns for stocks in the S&P 500 universe, using the S&P 500 as its benchmark. The target formula looks forward 65 trading days to calculate each stock’s percentage return relative to the benchmark.

The dataset covers over 21 years of weekly data, from January 2004 to August 2025, giving the model a rich history across multiple market regimes.

For normalization, the model uses Z-Score + Date with a 7.5% trim and an outlier limit of ±5.

Validation

The validation setup follows a basic holdout structure.

The model is trained on 16.4 years of data, ending around mid-2020, followed by a 58-month holdout period from 2020 through 2025. The model has not seen the post-2020 data during training — performance observed during the holdout period represents an out-of-sample validation.

Feature Set Overview

A total of 163 features are used as inputs.

On Redundancy

The AI Factor S&P500 model is built on 163 features spanning fundamental, technical, estimate, and aggregated data. Rather than a curated, minimally redundant set, the feature list is deliberately broad and overlapping. This is intentional — tree-based ensembles like ExtraTrees benefit from redundancy because they sample features randomly at each split. Having multiple-correlated signals gives the model more “good options” for early splits, which increases stability across market regimes.

All features share the same two-step normalization: first Skip (meaning no initial transformation is applied at the feature level — raw values are passed through), then Date normalization, which standardizes cross-sectionally at each point in time.

Earnings Data

Earnings Data includes standard EPS growth metrics across three-year, trailing twelve-month, and prior-year-quarter horizons, plus acceleration measures that compare short-term growth to longer-term growth. Analyst estimates are heavily represented — current quarter and current fiscal year EPS means, plus their 1-week, 4-week, 8-week, and 13-week lags, allowing the model to detect revision momentum. Next-quarter and next-fiscal-year estimates extend the forward-looking view. Estimate variability and standard deviations help the model gauge analyst conviction. Earnings surprises from recent quarters appear both individually and in a weighted composite that gives more importance to the most recent reports. Recommendation changes and long-term growth estimate trends round out the analyst sentiment bucket.

Valuation

Valuation features are diverse: Traditional metrics like trailing P/E, price to book, price to tangible book, price to cash flow, and price to free cash flow sit alongside enterprise value-based measures such as EBITDA to EV, gross profit to EV, and unlevered free cash flow to EV. Forward-looking valuation appears through forward earnings yield and forward revenue yield. The model also includes earnings yield adjusted for R&D spending, effectively treating R&D as a capital investment rather than an expense.

Quality & Profitability

Quality is measured from multiple angles: Operating margin levels (TTM and 5-year average), return on investment, return on assets, and median return on equity provide profitability snapshots. Changes in these metrics — like quarterly changes in gross margin, debt to assets, current ratio, asset turnover, and ROA — capture improving or deteriorating fundamentals. The model also tracks non-recurring items as a percentage of sales to reward clean income statements and uses a consistency measure counting how many of the last six quarters showed year-over-year EPS growth.

Momentum & Price Trends

Price momentum is measured across multiple timeframes: 3-month, 6-month, and 9-month returns (using overlapping periods), plus proximity to 52-week highs and lows. A feature called “velocity” compares a stock’s simple moving average from one month ago to its average from one year ago, normalized by average true range — a smoothed momentum measure that ignores the most recent month to avoid chasing sharp moves. Industry-relative price strength appears for 26-week and 52-week horizons. The model also includes short-term price changes (4-week returns) to capture recent drift.

Volatility & Risk

Risk is captured through beta at 1-year, 3-year, and 5-year horizons. Price volatility appears as standard deviation over 60 days, 12 months, 3 years, and 5 years. Share turnover — both 3-month and 1-year averages — acts as a liquidity proxy and beta predictor. Sortino ratios over 1- and 2-years measure risk-adjusted returns with a focus on downside volatility.

Technical & Flow

Money flow and accumulation signals are well represented. Chaikin Money Flow persistency, Chaikin Accumulation/Distribution, and a Chaikin Trend indicator help the model detect whether institutional money is moving into or out of a stock. Volume trends appear as 1-week versus 1-month volume and 1-month versus 6-month volume. Up-down ratios over 20, 60, and 120 trading days provide additional flow-of-funds perspective. Simple moving average crossovers (20-day EMA divided by 120-day EMA) add a trend-following element.

Industry Context

The model does not look at stocks in isolation. Industry-level aggregates appear throughout: industry RSI, industry EPS growth TTM, industry FCF growth, industry earnings yield, and industry momentum over 1-month, 3-month, 6-month, and 1-year horizons. Subindustry momentum provides an even more granular peer comparison. A feature comparing a company’s gross plant to sales ratio against its industry median helps identify companies with asset structures that look out of line with peers.

Capital Allocation & Growth

Buyback yield, shareholder yield (dividends plus buybacks), and inventory changes indicate how management is deploying capital. Three-year sales compound growth, TTM sales growth, and prior-year-quarter sales growth capture revenue expansion. Book value growth over five years adds a longer-term perspective. Free cash flow growth and free cash flow as a percentage of assets provide additional fundamental momentum signals.

Composite & Engineered Features

Several features combine multiple dimensions into single signals. A triple-factor momentum score integrates price, cash flow, and volatility. A PEG ratio variant compares free cash flow growth to valuation. A feature rewarding PEG ratios near 1 punishes both extremes as unsustainable. Another feature multiplies institutional flow by relative momentum to identify where smart money aligns with price strength.

Hyperparameter Sensitivity

The model was tested across 13 different configurations — including multiple variants of ExtraTrees, LightGBM, XGBoost, and linear models — all trained on the same 163‑feature dataset with the same target.

Despite wide variation in hyperparameters, the predictive accuracy remains stable. This low hyperparameter sensitivity points to robustness.

Predictor

The Predictors are trained from 2004-01-01 to 2020-06-01 (which matches the training period) to preserve a long, untouched holdout period.

The above predictor is used in the ranking system of the following (public!) Portfolio Strategy: https://www.portfolio123.com/port_summary.jsp?portid=1909114

Performance from 2021 through 2025 is based on a simulation using AI Factor predictions. The model has been live since January 20, 2026.

Predictor Lifecycle & Annual Review

The predictor is retrained once per year using the same disciplined methodology: a fixed training window ending June 2020 + 1 Year (or the equivalent representative period), the same 163-feature set, and identical hyperparameters.

But retraining does not mean automatic replacement. Each year, the newly trained predictor is evaluated against the existing one. The bar is not “different” – it is better.

Performance is compared across multiple dimensions:

- Out-of-sample pseudo performance

- Stability across regimes

- Alignment with the existing equity curve

Only if the new predictor demonstrates clear and measurable improvement over the full 5‑year holdout period does it replace the existing version. If performance is comparable or worse, the current predictor remains in place.

This ensures that the model evolves only when it genuinely benefits from additional data – not for the sake of change. The result is a predictor that improves over time without introducing unnecessary instability or chasing short-term noise.

Discussion of Design Choices

Obviously, there are other successful approaches to building AI Factor predictors — different validation methods, predictor lifecycle, different feature combinations, training on more recent data, and so on. The model shown here is just one example.

Paper on Design Choices

If you want to explore alternative design choices in more depth, this paper from Hanauer et al is a good starting point:

“Design Choices, Machine Learning, and the Cross‑Section of Stock Returns”
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5031755

Hanauer’s key insights for P123 users:

• Target → Use market‑relative returns for example 3MRel (Rel%Chg_D(65, #Bench)) for non‑linear models (like ExtraTrees, LightGBM, XGB)

• Target transformation → Keep returns continuous (High Max Return).

• Training window → Training on long, uninterrupted data (expanding windows) significantly outperforms short rolling windows that discard old data.

• Features → Add novel signals (actuals, revisions, volume). Published factors alone underperform.

• Feature selection → Do not obsess over pruning. Trees handle redundancy.

• Non‑linear edge → ExtraTrees / LightGBM / XGB beat linear models only with the right target + continuous returns + long training.

• Validation → Keep holdout truly unseen (basic holdout or multiple cutoffs).

Share

AI Factor S&P1500

We implemented the above discussed AI Factor Model 1:1 to the SP1500 Universe: https://www.portfolio123.com/sv/aiFactor/24355/overview

Corresponding Portfolio Strategy: It trades live since 01/30/2026

https://www.portfolio123.com/port_summary.jsp?portid=1913454

Best regards,

Andreas

P.S.: A big Thank you to Dan and Filip, we worked together on the above models, great discussions, I learned a ton!

When building systematic trading strategies with machine learning models like LightGBM, it is easy to fall into traditional academic traps. Many practitioners get nervous when a small change in an ML model dramatically alters the output, immediately labeling it as “unstable.”

Subscribe now

But in complex, non-linear search spaces, sensitivity does not equal non-robustness!

If you are deploying advanced machine learning predictors in your portfolio strategies, here are four structural realities you need to account for to ensure your model is actually capturing true alpha.

1. The Danger of Training Through Black Swans (The COVID Distortion)

In many of my core models, I deliberately concentrate the training period from 2003-01-01 to 2020-06-01, avoiding the heavy inclusion of the 2020 COVID-19 regime.

Why? Training through unprecedented exogenous shocks can distort a model significantly. The extreme variance and structural breaks of that period are simply not representative of standard market history. While it is possible to train through it, you need to use high regularization to compensate so the model doesn’t overfit to an anomaly.

BUT —> this is me, I have Quants in my network that train through 2021 and 2022 with a ton of success (small and large caps!). There is no perfect way, just cups of tea you prefer ;-)

2. Path Dependency & Test Start Dates

Backtesters often overlook how sensitive an equity or capital curve is to its inception date. Shifting the start date of a rolling test by just a few months can completely alter the compounding path.

Share

Example:

https://www.portfolio123.com/port_summary.jsp?portid=1844442 (free in my group https://www.portfolio123.com/app/group/317/home) —>

https://www.portfolio123.com/port_summary.jsp?portid=1953664

Same strategy, just the last 5 years (e.g. one rolling test), if you copy the strategy this (not the above) is your benchmark.

3. Embyonic Randomness in Gradient Boosting

LightGBM (for example!) has inherent randomness baked into its architecture through stochastic features like subsampling and column sampling. Because of this, even when using the exact same hyperparameters, different training runs can — and will — produce slightly variance-driven outputs.

This is entirely normal. It does not mean your model is broken; it means the algorithm is navigating a vastly complex mathematical landscape.

4. The Vast Search Space: Rethinking “Robustness”

In quantitative machine learning, a single feature modification can completely shift a model’s direction. Because financial data has an incredibly low signal-to-noise ratio, the model isn’t looking for one solitary “correct” solution. Instead, it is discovering one of many possible mathematical pathways that all approximate the same underlying financial signal.

So, how do you actually test for true robustness?

The real test is Low Hyperparameter Sensitivity (within the AI Factor Model, the next step is to train the predictor and test it in a Portfolio Strategy).

If you run 20 to 25 (or more!) different machine learning configurations with different hyperparameters — across different ML families —a nd they all successfully extract signal, you have strong evidence that the underlying relationship is real, persistent, and tradeable.

This is key: Start focusing on whether your core signal remains powerfully predictive across an entire ensemble of settings.

This is how low HP-Sensitivity looks like —> it basically means the MLs do not have a hard time finding signal, the distribution of results is smooth and explainable:

Lift Charts:

Plus: in my production small-cap portfolio strategies (large-caps are an entirely different beast), I prioritize AI Factor Models where all (!!!) machine learning variants demonstrate a clean, robust lift chart.

5. Training Horizon: Why Dynamic Regime Learning Takes Time

When dealing with machine learning models in quant finance, shorter is rarely better. A longer (and critically, uninterrupted) training duration is crucial—especially when modeling relative return targets.

A protracted training timeline doesn’t just teach the model which individual factors predict performance; it teaches the model the broader market context. It forces the algorithm to learn which specific market environments favor which factors, allowing it to dynamically rotate factor exposures as regimes shift over decades.

Also:

Keep in mind that extending a training horizon by even two years is not a minor parameter tweak; it completely reshapes the model’s baseline understanding of market dynamics and can radically alter your performance outcomes.

This is my favorite config:

This picture sums it up really well: Deploying AI Factor Models (Machine Learning running on precise Point-in-Time factors) is exactly like engineering a Formula 1 car.

You are operating an incredibly high-performance machine built for speed and alpha extraction. Because the environment is so competitive and the margins are razor-thin, a single, tiny setup change can mean a massive shift in performance:

• The F1 Reality: Adjusting a rear wing angle by a mere fraction of a degree can be the difference between maximum downforce in the corners or blistering top speed on the main straight. Get the setup slightly wrong for the specific track conditions, and your lap times collapse.

• The ML Reality: Adding a single new feature, adjusting your training time horizon, or altering your buy and sell rules—these small structural changes have a huge impact. They completely rewrite how the algorithm cuts its decision trees and navigates the data, resulting in a radically different equity curve.

Some Quants look at this sensitivity and assume (understandably!) the car (or the model) is broken. My experience so far: when you build and ML System —> setup sensitivity is entirely normal.

The goal isn’t to build a slow, clunky car that handles every track the exact same way.

The goal is to master the setup, understand the track context, and use rigorous ensemble validation to ensure your machine is tuned to win across the entire season.

Best regards,

Andreas

Share

Actuals Data

The models are based on actuals data from FactSet. FactSet captures data from press releases and feeds it to P123. Very potent. Super strong data.

But the data is too sparse before 2020 to test properly.

I analysed it and was fine with a 5‑year backtest. The 2003–2020 backtest looks good — but the models really exploded after 2020, thanks to FactSet’s data innovation on actuals.

Subscribe now

As long as out‑of‑sample live trading matches my 5‑year backtest — and I am monitoring that — I am fine with publishing and using the model.

That is not typical quanting. But I am not a typical quant. I look at data innovation — what factors are new and strong from FactSet — before I build models.

What Are Actuals?

From Portfolio123.com:

“Actuals are companies’ historical financial data, such as earnings, sales, and net income. Unlike fundamentals that come from official filings submitted to government agencies, Actuals are collected primarily from company press releases and/or from brokers.

Actuals are primarily used to calculate surprises vs the analyst estimates. However, integrating Actuals in fundamental analysis can offer a unique perspective compared to only using data from 10‑K and 10‑Q filings. This is particularly relevant when Actuals reflect different GAAP treatments, including one‑off items or other specific financial occurrences.

Many semiannual companies release earning releases every quarter, and analysts follow suit with quarterly estimates. The main downside is that Actual data can be spotty and inconsistent through time, especially in the past. For example sometimes more complete actuals are the semiannual ones, and sometimes the quarterly. We use some heuristics to determine which interim items to use for a particular point in time.”

Key Points About Actual Data

• Never restated or overwritten

• Can be more frequent (e.g., quarterly for semiannual companies)

• More timely. Filings can have long delays, especially for small caps.

• Can be quite different from GAAP numbers from filings.

• Actuals are widely used as actionable information.

Why Actuals + Estimates Is So Powerful

Actuals alone tell you what happened. Estimates alone tell you what was expected. But when you combine them, you get the surprise — the gap between expectation and reality. That gap is where the market moves. That is the fundamental momentum engine.

This is why the post‑earnings announcement drift (PEAD) exists. The market reacts to surprises, not to absolute numbers. And surprises are only observable when you have both the actual number and the consensus expectation.

Best regards,

Andreas

Share

https://www.portfolio123.com/app/r2g/summary?id=1811955

Subs chasing concentrated 10 Position High Beta Portfolio Strategies.
Carful now, Standard deviation is 51% versus 13% on the SPY!

I will publish a version (in 52 Days) that trims winners and does not weight to market cap.

I personally would not put more than 25% of my capital in models like this!

Subscribe now

https://www.portfolio123.com/app/r2g/summary?id=1861584 ➡️⬇️

https://www.portfolio123.com/app/r2g/summary?id=1861586➡️⬇️

https://www.portfolio123.com/app/r2g/summary?id=1893733 ➡️⬇️

https://www.portfolio123.com/app/r2g/summary?id=1873113 ➡️⬇️

https://www.portfolio123.com/app/r2g/summary?id=1890053 ➡️⬇️

https://www.portfolio123.com/app/r2g/summary?id=1765750 ➡️⬇️

Subs chasing concentrated 10 Position High Beta Portfolio Strategies.
Carful now! Rebalance!
I personally would not put more than 25% of my capital in models like this!

https://www.portfolio123.com/app/r2g/summary?id=1757039 ➡️⬇️

Subs chasing concentrated 10 Position High Beta Portfolio Strategies.
Carful now! Rebalance!
I personally would not put more than 25% of my capital in models like this!

https://www.portfolio123.com/app/r2g/summary?id=1812296 ➡️⬇️

Models of this Substack➡️

Share

https://www.portfolio123.com/port_summary.jsp?portid=1886955 ➡️⬇️

https://www.portfolio123.com/port_summary.jsp?portid=1853545 ➡️⬇️

https://www.portfolio123.com/port_summary.jsp?portid=1853536 ➡️⬇️

https://www.portfolio123.com/port_summary.jsp?portid=1829380 (I put this on hold in between, some subscribers continued trading it ;-)) ➡️⬇️

Kudos and thank you!!!

Kudos and thank you to our subscribers — some of you have been here since 2020.

We have had our rough patches. But replication rates — OOS performance matching or even exceeding backtests — are now exactly where they should be.

Thank you for sticking with me!

Best regards,

Andreas

Best regards,

Andreas

Quick update:

All good. Small and large cap AI Factor models are humming along.

About 13 months ago, I started publishing AI Factor models. The results are staggering.

Most importantly, live trading replicates OOS pseudo (the holdout test) in >99% of all portfolio strategies I have built. That is a quant dream.

Subscribe now

You build it. You test it. And your test replicates in real‑time trading — always relative to market impact, of course.

Important for me: my predictor is always trained the same way — with long, uninterrupted training (see the dark green and red sections above). That means I am not flying blind.

I lose the ability to train the predictor up to “yesterday.” I give up some recent data.

But here is what I gain: I can compare live trading directly with the holdout test. As long as live trading replicates the holdout, I know the model is in good shape.

That is the trade‑off: I choose validation length over recency.

Large Caps

Live Trading:

https://www.portfolio123.com/port_summary.jsp?portid=1853545

https://www.portfolio123.com/port_summary.jsp?portid=1898601

Large Caps Long Short

Live Trading:

https://www.portfolio123.com/port_summary.jsp?portid=1906462

https://www.portfolio123.com/port_summary.jsp?portid=1891839

Some customers leverage up the long‑short large cap strategy book.
That is not my cup of tea, but I get what they are doing.

https://www.portfolio123.com/port_summary.jsp?portid=1891839

Small Caps

Live Trading:

https://www.portfolio123.com/port_summary.jsp?portid=1853536

https://www.portfolio123.com/port_summary.jsp?portid=1853549

Large Cap Combo Strategy Book: AI Factor, Traditional Sentiment, Actuals & Sentiment

Live Trading:

https://www.portfolio123.com/port_summary.jsp?portid=1886955

Share

https://www.portfolio123.com/port_summary.jsp?portid=1920964

Have a great weekend!

Best regards,

Andreas

AI Large Caps

https://www.portfolio123.com/port_summary.jsp?portid=1898601

This is Out Of Sample Live Trading —>

https://www.portfolio123.com/port_summary.jsp?portid=1853545

AI Small Caps

https://www.portfolio123.com/port_summary.jsp?portid=1853549

This is Out Of Sample Live Trading —>

https://www.portfolio123.com/port_summary.jsp?portid=1853536

Mix AI Large Caps, EPS Estimates + Actuals

https://www.portfolio123.com/port_summary.jsp?portid=1920964

This is Out Of Sample Live Trading —>

https://www.portfolio123.com/port_summary.jsp?portid=1886955

🔐 Access to the live models above — for paying subscribers only

Paying subscribers of this Subsgtack get access to the above models via a Portfolio123 research group.

Simply create a Portfolio123 account, connect it to Interactive Brokers, add the live strategies, and replicate the trades — all in a few clicks every Monday.

Here's my P123 account — real money, but a small slice of my stake. (My main account is still on TradeStation, old habits 😉) —>

Read more

AI Large Caps

https://www.portfolio123.com/port_summary.jsp?portid=1898601

https://www.portfolio123.com/port_summary.jsp?portid=1853545

This is Out Of Sample Live Trading!

AI Small Caps

https://www.portfolio123.com/port_summary.jsp?portid=1853549

https://www.portfolio123.com/port_summary.jsp?portid=1853536

This is Out Of Sample Live Trading!

Mix AI Large Caps, EPS Estimates + Actuals

https://www.portfolio123.com/port_summary.jsp?portid=1920964

What a monster!

https://www.portfolio123.com/port_summary.jsp?portid=1886955

This is Out Of Sample Live Trading. What a monster!

By the way: I stay bullish here!

The media is doing a terrible job of seeing the situation clearly.

I have always been a long term bull on the United States. But I have never been more bullish than I am right now!

I am staying long!

The Factor Edge

Why I stick to my #SecularBullMarketUSA Thesis

The Seven Points of the Energy Moat Landlord Thesis…

Read more

5 months ago · 10 likes · Andreas Himmelreich

The Factor Edge

📈The Pain Trade is Up! #SecularBullMarketUSA

Skin in the Game…

Read more

5 months ago · 10 likes · 6 comments · Andreas Himmelreich

Have a great week!

Best Regards

Andreas

Subscribe now

AI Large Caps

https://www.portfolio123.com/port_summary.jsp?portid=1898601

https://www.portfolio123.com/port_summary.jsp?portid=1853545

AI Small Caps

https://www.portfolio123.com/port_summary.jsp?portid=1853549

https://www.portfolio123.com/port_summary.jsp?portid=1853536

Mix AI Large Caps, EPS Estimates + Actuals

https://www.portfolio123.com/port_summary.jsp?portid=1920964

https://www.portfolio123.com/port_summary.jsp?portid=1886955

Family Project on Investing / Systematic Investing

We started a Family Project :-)

Subscribe now

Share

No sure yet, we might start a separate substack where we write / video about the subjects. Let’s see.

Have a great week!

Best regards,

Andreas

Stop hiding your left-tail risk!

$1M+ invested and years in the dirt, I’ve realized the only realistic way to design a long only Portfolio Strategy is to accept the trade-offs.

Subscribe now

Survival over optimization

Nassim Nicholas Taleb spent his career warning that the Black Swan isn't a risk you model — it's the event that breaks every model.

2008. 2020. Whatever comes next:

The funds that blow up aren't the ones with the wrong beta — they're the ones that pretended correlation would stay stable, that hedges would hold, that the machine would keep humming.

Taleb's critique of the financial establishment is simple: they mistake "not having seen a black swan" for "black swans don't exist."

By accepting β = 1 in crashes, you stop making that mistake. You acknowledge that in a true liquidity event, everything gets sold.

Your job isn't to avoid the wave — it's to be standing when the water recedes.

Survival isn't a byproduct of good design. It is the design.

The market doesn't reward smoothness. It rewards the ability to stay in the game when everyone else is forced out.

Our design doesn't pretend we can avoid the storm. It ensures we're still standing when it passes:

- β = 1 in crashes
- Fast recovery
- No hidden left-tail risk.

That's how you turn volatility from a threat into a paycheck!

Share

My (Beta) Design Goals

Up Markets: β > 1. Capture the surge. You’re here to make money, not track the index.

Down Markets: β < 1. Relative strength. Protect capital when the tide goes out.

Crashes: β = 1. Accept market reality. Don’t choke your alpha with fake protection that fails anyway.

Recovery: Maximum speed. Good Investors don’t quit on hiccups — they quit on years of pain. Recover fast or lose them!

1. The “Beta = 1 during Crash” Realism

This is the most sophisticated part of the design.

The Trap: Most people try to engineer β < 0.5 during a crash. To do that, you usually have to bake in so much “defensiveness” that you kill your alpha during the 80% of the time the market is moving up.

The Reality: In a true liquidity event — 2008, March 2020, or a flash crash — correlations go to 1. Everything is sold to cover margin. There is no safe corner.

By accepting β = 1 in crashes, you aren’t hiding the risk. You’re acknowledging that the strategy is a long-only vehicle, and its job is to be the best up-market engine possible.

You stop pretending you can build a car that goes 200 mph and never crashes.

Instead, you build one that accelerates like hell, brakes decently, and doesn’t kill you when the road ends.

2. The Separation of Powers

Treat your alpha model like a high-performance engine and your strategy book like the brakes and suspension.

The Engine (for example an AI Factor Model): Its only job is to find stocks that outperform. It should be pure. If you force the engine to also be the brakes — via complex internal hedging, volatility targets, or embedded defensive factors — you end up with a Franken-strategy that underperforms on both ends. It’s too slow to capture upside, too compromised to survive drawdowns.

The Suspension (The Strategy Book): Handling drawdowns via Gold, index shorts, cash, or volatility overlays at the portfolio level is institutional-grade thinking. It keeps the alpha signal clean. You can swap out the engine without rebuilding the chassis. You can test different hedges without contaminating the core model.

This separation is what makes a strategy scalable, auditable, and — most importantly — survivable.

3. The Strategy Book Handles the Rest

Negative covariance diversifiers:
- Gold. It’s not perfect. But when everything else breaks, it often doesn’t. It’s one of the few assets that has a long-term history of going up when equities go down — not always, but enough to matter.

Index shorts:
- Hedge the market without trying to time it. Keeps you net long but reduces drawdowns. You don’t have to call the top. You just reduce the pain when the tide goes out.

https://www.portfolio123.com/port_summary.jsp?portid=1906462

Outsourced timing
- Focus on stock selection. Let someone else — or a systematic model — handle the market exposure. Most quants are great at selection but terrible at sequencing. Accepting that you will be wrong at the turns is the only way to stay in the game long enough to win.

Outsourced tail risk protection
- Again: Focus on stock selection. Tail risk via options isn't trivial — get someone who specializes in it. Don't let your alpha engine double as your insurance desk. That's how protection fails when you need it most.

4. Pick your Pain!

The game gets easier once you accept that it won’t be easy.

Hedging pain:
- You’ll hate bull runs. Every time the market rips, your hedges will drag on performance. You’ll watch unhedged friends post bigger numbers. Your clients will ask why they’re paying for protection they didn’t “need.”

Long-only pain:
- Drawdowns test your soul. You’ll sit through -20% corrections knowing there’s no insurance. You’ll second-guess every position. You’ll watch your hard-earned gains evaporate in weeks.

Leverage pain:
- Feels like a genius move until the blow-up day. And there’s always a blow-up day ;-)

Best Regards

Andreas

How this Substack works!

Subscribe now

We are doing o.k.!

Combo AI Factor & Traditional Sentiment & Earnings Actuals

https://www.portfolio123.com/port_summary.jsp?portid=1920964

Out of Sample (OOS) Live Trading:
https://www.portfolio123.com/port_summary.jsp?portid=1886955

Large Caps

https://www.portfolio123.com/port_summary.jsp?portid=1898601

Out of Sample (OOS) Live Trading:
https://www.portfolio123.com/port_summary.jsp?portid=1853545

Large Caps Long Short

https://www.portfolio123.com/port_summary.jsp?portid=1891839

Out of Sample (OOS) Live Trading:
https://www.portfolio123.com/port_summary.jsp?portid=1906462

Small Caps

https://www.portfolio123.com/port_summary.jsp?portid=1853549

Out of Sample (OOS) Live Trading:
https://www.portfolio123.com/port_summary.jsp?portid=1853536

So all in all, we are doing fine.

Most important —> OOS Live replicates OOS Pseudo!

OOS Pseudo —> Let us say the AI Factor Predictor has been trained from 2003 to 2020.06. We then put the predictor into a Portfolio Strategy and test the last 5 years.
This is called OOS pseudo. Yes, the predictor has not seen this data, but we have tested > 15 ML Algos before to see if our AI Factor model has low Hyperparameter Sensitivity.

Then we put the Portfolio Strategy to Live Trading —> THIS is pure OOS Live!

Now what I am seeing here, and this is very important: I have built > 300 Portfolio Strategies with > 150 AI Factor Models. And I cannot find any strategies that do not (!) replicate their OOS Pseudo capital curve. There might be strategies that are a bit messy OOS Live, but only when OOS pseudo was also a bit messy. What I am seeing is: Great OOS Pseudo gets transformed into great OOS Live. That is pure Gold!

Share

Have a great weekend.

All the best and best regards,

Andreas

Most important posts are here:

Subscribe now

Markets are complex (and not complicated!)

The highest-IQ people in finance fail every day.

Not because they’re not smart enough.

Because they confuse understanding the map with navigating the terrain.

Intelligence tests measure something. Pattern recognition. Logical deduction. The ability to solve puzzles with clear rules and stable cause-and-effect.

Markets have neither.

In complex systems, high IQ can actually hurt you.

Smart people build elaborate models. They overfit. They fall in love with their own logic. They mistake comprehension for control.

They try to understand the rainforest by counting every leaf.

Meanwhile, the rainforest does what it always does: shift, adapt, surprise.

The market doesn’t reward understanding. It rewards robustness.

It doesn’t care how many books you’ve read. It cares whether your system survives drawdowns and captures fat tails.

It doesn’t reward prediction. It rewards adaptation.

Share

Probe → Sense → Respond: The AI Factor Way

https://www.open.edu/openlearn/money-business/hybrid-working-planning-the-future/content-section-4.2

Here is how we apply that to systematic investing.

🧪 PHASE ONE: PROBE

We train once. One window only.

2003 to June 2020.

Not 2021. Not 2022. Not the latest data. Just the clean window. Pre-anomaly. Pre-meme-stock insanity.

Why? Because representative data beats recent data every time.

We don’t chase the last three years. We capture the regimes that repeat.

🔬 PHASE TWO: SENSE

We test across 15+ hyperparameter sets.

Not one. Not the best one we found after hours of tuning. All of them.

And here is the rule: All must perform.

Not “most.” Not “the top three.” All!

If the signal is real, hyperparameters don’t matter. If the signal is fragile, we don’t want it.

This is not overfitting. This is robustness by design.

⚙️ PHASE THREE: RESPOND (Part One)

We retrain the predictor.

Exactly the same way. Same window. Same logic. Same conservative assumptions.

No cheating. No peeking at the future. No “now that we know what works, we train on everything.”

The setup that worked in validation is the setup we deploy.

Why change what isn’t broken?

📊 PHASE FOUR: VERIFY

We embed the predictor in a Portfolio Strategy.

Then we test 5 years of pseudo out-of-sample data.

Last 5 years. Realistic slippage. Realistic fills. Realistic trading constraints.

We watch the equity curve. Does it hold? Does it track? Does it behave the way the validation suggested?

If yes, we move forward. If no, we kill it and start over.

No attachment. No ego. Just data.

🚀 PHASE FIVE: DEPLOY

We put the system live.

Real money. Real fills. Real market impact.

Then we do the only thing that matters:

We watch.

👁️ PHASE SIX: CONFIRM

Does OOS Live replicate OOS pseudo?

Not exactly. Not to the decimal. But relative to market impact? Relative to expected behavior?

If yes, we’re done. The system works. We leave it alone.

If no, we kill it. No hesitation. No “let’s wait and see.”

The market decides. Not us.

🌲 On Not Understanding Everything

Here is the truth that most quants cannot accept:

We will never fully understand what (complex: e.g. LightGBM, XGB, ExtraTrees) ML is doing.

My best models contain up to 65,000 decision rules on up to ONE BILLION DATA POINTS.

Each tree is a “mini ranking system” (this is not exactly true, but points in the right direction). Each split captures a conditional relationship. Each leaf represents a pattern that worked historically.

No human can grasp that. No committee can approve it. No allocator can nod along while you explain it.

And that is completely fine.

We just need to validate it.

OOS Live is your compass. It tells you what works. Not your intuition. Not your backtest. Not your confidence in the code.

The market decides!

🌊 Why This Works in Complex Systems

Markets are not complicated. They are complex.

That is all we need.

Complicated is a watch. You can take it apart, understand every gear, put it back together. Cause and effect are stable. Linear models work.

Complex is a rainforest. Cause and effect shift. Relationships change. You cannot understand it by disassembly. You cannot model it linearly. You can only adapt.

In complex systems, no model “understands” reality. And usually, the most intelligent people do not get this ;-)

[The Bias is called “Complexity Aversion”, but that is another post!]

📉 Fat Tails Are Not Your Enemy

Most investors fear fat tails. They build models that clip outliers at 2.5 sigma. They hide from the very thing that drives returns.

We do the opposite.

We set outlier limits to 5 sigma by default. And we test up to 10 sigma to prove that 5 is optimal.

Why? Because signal lives in the tails!

A 10-sigma momentum breakout is not a data error. It is information. A 2008-style volatility spike is not a glitch. It is regime data.

Clip too hard, and you collapse the signal gradient. Everything above 2.5 sigma looks the same. The model can no longer tell “strong” from “extreme.”

That is why performance collapses at 2.5 sigma.

And that is why we don’t clip there.

🧭 Your Compass Is Not Your Model

Your compass is OOS Live!

Not your confidence. Not your track record. Not your belief in the code.

Every day, the market gives you feedback.

Important: you need to build models that “track the market” to be able to do so, my AI Factor models do track well!

• Market up → does your model go up harder?

• Market down → does your model go down less?

• Does the fingerprint hold?

If yes, we stay.
If no, we kill it.

🔁 The AI Factor Way

Probe. Sense. Respond.

Not because we are smarter. Not because we see the future.

Because we built systems that can survive not knowing.

Because we trust OOS Live more than we trust ourselves.

Because we know that in complex systems, adaptation beats prediction every time.

Fat tails are not your enemy. They are why this works!

Share

🧭Example

So how long will I trade this model?

Easy.

Until OOS Live stops tracking OOS Pseudo.

That’s it. No calendar. No committee. No “feeling.”

The model runs until the market proves it’s broken.

Leave a comment

Best regards,

Andreas

Sometimes it makes sense to revisit your original feature set and test yourself on targets you haven’t explored yet—for example, the 3M Relative (Target 3MRel).

However, the original feature set contained 133 features. One of them couldn’t be tested from 2003 due to a lack of data, so I killed it. This allowed me to extend the training window. (By the way, I plan to do the same with other feature sets—gaining an extra year or two of training data can make a significant difference!)

So far, the predictor has only been trained on data from January 2003 to June 2020—meaning everything you're seeing here is pseudo out-of-sample. Let's put it live and see what happens.

Best regards,

Andreas

Subscribe now

Why “Train Wide, Filter Smart” beats pre-filtering—and why traditional ranking systems fracture under the same logic...

The Mathematical Foundations

Traditional Ranking Systems: Linear Combinations with Fixed Weights

A traditional ranking system typically takes the form:

Score = w₁f₁ + w₂f₂ + ... + wₙfₙ

Where:

• fᵢ are factors (e.g., value, momentum, quality)

• wᵢ are fixed weights (often equal-weighted or optimized once)

The Mathematical Fragility:

1. Linear Superposition: The score is a simple sum. If one factor breaks down (e.g., momentum crashes), that component directly corrupts the final score with no mechanism to down-weight it dynamically.

2. Fixed Decision Boundaries: The relationship between factors is static. If the market regime shifts, the model cannot recalibrate. It’s like having a map that never updates.

3. Multicollinearity Amplification: When factors are correlated (they almost always are), the linear combination creates mathematical instability. Small changes in inputs can produce large swings in rankings because the correlated factors effectively double-count the same signal.

4. Polynomial Overfitting: When practitioners try to add complexity (interaction terms, squared terms), they get:
   Score = w₁f₁ + w₂f₂ + w₃f₁f₂ + w₄f₁² + ...
   This becomes a high-degree polynomial that fits historical noise beautifully but shatters out-of-sample. The coefficients are still fixed, so you’re just adding more ways to overfit.

ML Predictors (Ensemble Trees): Adaptive, Non-Linear, and Democratic

A tree ensemble like LightGBM builds:

Score = (1/T) Σₜ₌₁ᵀ Treeₜ(x)

Where each Treeₜ(x) is a series of conditional splits:

Treeₜ(x) = Σⱼ₌₁ᴸ leaf_valueⱼ * I(x ∈ regionⱼ)

The Mathematical Robustness:

1. Adaptive Partitioning: Instead of fixed linear weights, trees create adaptive regions in feature space. The model learns that “value matters more when momentum is strong” or “quality is crucial in volatile markets” as conditional rules, not fixed interactions.

2. Ensemble Averaging = Variance Reduction: The prediction is an average of T independent estimates. The variance of the average is σ²/T (if trees were independent). Mathematically:
   Var(Score_ML) ≈ (1/T) * Var(Tree)
   Traditional ranking: Var(Score_Trad) = w’Σw (which grows with correlation)

3. Implicit Regularization Through Stochasticity: Each tree sees a random subset of data and features. This creates a natural regularization effect—no single spurious relationship can dominate because it must survive across many random subsamples.

The Math of Why Buy Rules Work with ML But Break Traditional Systems

Case 1: ML Predictor + Buy Rule

Let’s model what happens mathematically:

Final Score_ML = I(quality) * (1/T) Σ Treeₜ(x)

The buy rule is a multiplicative gate, not an additive component.

Why this preserves robustness:

• The expectation E[Score_ML | quality=1] is still the average of many trees

• The variance reduction property (1/T) remains intact

• The conditional rules inside trees still capture nuanced relationships within the quality cohort

• Mathematically: The buy rule simply shifts the domain of the function, but the function itself (the ensemble) remains unchanged and fully expressive

Case 2: Traditional Ranking + Buy Rule

Final Score_Trad = I(quality) * (w₁f₁ + w₂f₂ + ...)

The mathematical failure modes:

1. Multiplicative Discontinuity: The buy rule creates a sharp discontinuity. Stocks with quality=0 get score 0, quality=1 get the linear combination. This amplifies any noise at the boundary.

2. Fixed Weights + Filtered Universe: Within the quality cohort, the factor weights (wᵢ) were likely optimized on the full universe. They are now suboptimal for the filtered subset. Mathematically, you’re solving:
   min_w E[(y - w’f)² | quality=1] but your w came from min_w E[(y - w’f)²]
   These are different optimization problems unless factors are perfectly uncorrelated with quality.

3. Correlation Structure Changes: The covariance matrix Σ within the quality cohort is different from Σ in the full universe. Traditional ranking has no way to adapt. Its variance becomes:
   Var(Score_Trad | quality=1) = w’Σ_quality w
   Since w was optimized for Σ_full, this can be substantially larger than intended.

The Fundamental Mathematical Insight

The difference reduces to function approximation vs. conditional expectation:

Traditional Ranking: Learns a global linear function f(x) = β’x. Adding a filter means you’re evaluating this fixed function on a subset. If the true relationship is different in that subset, you’re using the wrong function.

ML Ensemble: Learns a conditional expectation E[y|x] through adaptive partitioning. The buy rule simply restricts the domain, but within that domain, the model still approximates E[y|x, quality=1] because its training exposed it to enough examples from that region to learn the local structure.

Mathematically, the ML model minimizes:
E[(y - ĝ(x))²]

And achieves:
ĝ(x) ≈ E[y|x]

For x in the quality region, ĝ(x) ≈ E[y|x, quality=1] because the model saw those x’s during training.

The traditional model minimizes:
E[(y - β’x)²]

And gets:
β’x ≈ E[y|x] only if the true relationship is linear and global

When you restrict to quality=1, β’x is no longer guaranteed to approximate anything meaningful.

This is why this is save…

Summary

The Core Thesis: “Train Wide, Filter Smart” is mathematically sound, practically validated, and represents a genuine advancement in quant strategy design. I fully agree that:

1. Decoupling pattern recognition from portfolio construction is the right architectural choice

2. ML ensembles (like LightGBM) have fundamental mathematical properties—variance reduction through averaging, adaptive partitioning, implicit regularization—that make them inherently more robust than linear ranking systems

3. The buy rule as a focusing lens (not a model input) preserves the model’s intelligence while ensuring output quality

4. The harmonization of filters with normalization methods (simple absolute rules for Z-score, complex percentiles for rank-based) is a critical, often-missed insight

Best regards,

Andreas

P.S. This article was developed in collaboration with DeepSeek and other LLMs. The math would have taken me weeks to articulate clearly — but I wanted these ideas out now. Yes, I used AI to help write about using AI in quant finance. Train Wide, Filter Smart applies to writing too ;-)

Universe selection is not the first step. It is the strategy.

The factors matter. The ML settings matter. The training window matters. But if your universe is wrong, nothing else matters.

Here is what I learned by failing, succeeding, and failing again.

The Two Paths to AI Success

After years of testing, I have found that AI Factor models only work consistently in two types of universes:

Path 1: The Homogeneous Universe

Clean. Focused. Predictable.

Examples:

• S&P 500

• S&P 1500

• EU Small Caps (filtered by liquidity, not market cap)

• United States Small Caps (filtered by liquidity, not market cap)

What happens here:
The AI learns the specific “dialect” of that group. US large caps behave like US large caps. EU small caps behave like EU small caps. The patterns are consistent because the companies are comparable.

Think of this like training a mechanic on only Toyotas. They become experts on Toyotas. Give them a Toyota, they will diagnose it instantly.

Path 2: The Massive Universe

So large that dominant signals emerge.

Examples:

• North Atlantic (all stocks)

• Global Developed Markets

What happens here:
The AI doesn’t try to model everything equally. It gravitates toward the strongest, most consistent signal cluster. In my testing, when I use “North Atlantic (all stocks),” the AI overwhelmingly picks EU stocks based on value and momentum. The EU signal is so dominant that the model essentially ignores the US.

Think of this like training a mechanic on every car ever made. They won’t know every model, but they will understand fundamental principles—engines, transmissions, brakes—that apply everywhere.

The “Death Zone”: Where AI Factors Go to Die

Here is what you must avoid at all costs:

Universes that are:

• Too big to be homogeneous

• Too small to have dominant cross-regime signals

• Rules in the Universe (The problem isn't your rules. The problem is when your rules leave you with a universe that's too small to be massive and too messy to be homogeneous.)

My Personal Working Settings

After extensive testing, here are the universes that consistently work for me:

United States (Filtered)

Starting Universe: United States (Incl. Foreign Primary)
Currency: USD

Rules:
! InSet(ExchangeCode, 13, 19)        // No OTC
AvgDailyTot(20) < 10000000            // Min liquidity
! GICS(FINANCIAL)                     // Optional: exclude financials

I vary the liquidity filter between 10M, 20M, and 30M depending on how many names I want. I also tested 200M with good results, especially if you train wide and then filter smart!

S&P Indices

• S&P 500

• S&P 1500

These are quality-filtered by construction. The companies in them are profitable, large, and likely to survive. The AI doesn’t have to learn how to avoid bankruptcy; it can focus entirely on finding alpha.

EU Small Caps

A beautiful homogeneous universe. European small caps behave differently than US small caps, but they behave consistently among themselves.

North Atlantic (All Stocks)

This is my “Train Wide” universe. I let the AI scan everything, and it consistently finds dominant Value/Momentum signals in EU stocks. The key is understanding why it picks what it picks — and not interfering.

Why This Works: Degrees of Freedom

Here is the theory behind all of this:

Machine learning models consume degrees of freedom like an engine consumes fuel. Every unique stock, every weird outlier, every OTC penny stock that behaves differently—they all consume model capacity trying to explain them.

In a clean, homogeneous universe, the model’s capacity is focused entirely on finding real patterns among comparable stocks.

In a massive universe, the model has enough degrees of freedom to identify causal patterns that transcend individual markets—like why Value works differently in Europe than in the US.

In the death zone? The model runs out of capacity before finding truth, and settles for spurious correlations.

Why Your Biases are Your Biggest Enemy in AI Factor Design!

“Remove REITs.” “No Chinese stocks.” “Price above $5.” “Financials are bad.” “Biotech is too volatile.”

Rule after rule. Until their universe looks like their personal prejudices, not a coherent market segment.

The AI doesn’t need your opinions. It needs data. Clean, abundant, representative data.

Your job is not to curate the stocks. Your job is to curate the character of the universe—liquidity profile, region, size band—and then get out of the way.

Every rule you add beyond that is you competing with the AI. And you will lose.

I know because I’ve tested it. The universes where I added the fewest opinions are the ones that perform best.

Your biases are loud. Let the data be louder.

Here are the specific biases I’ve seen destroy AI Factor universes:

Control Illusion
What It Sounds Like: “I need to remove REITs, and biotech, and Chinese stocks...”
What It Does: Universe shrinks to 300 random leftovers
Why You Avoid It: You let the universe define character, not your opinions

Anchoring
What It Sounds Like: “I always filter out sub-$5 stocks”
What It Does: Arbitrary cutoff with no statistical basis
Why You Avoid It: You test, don’t assume

Availability
What It Sounds Like: “That one Chinese stock crashed my portfolio once”
What It Does: Overweighting vivid memories
Why You Avoid It: You trust the AI’s pattern recognition, not your trauma

Complexity Bias
What It Sounds Like: “More rules = more sophisticated”
What It Does: 15 exclusion rules, 3 inclusion rules
Why You Avoid It: You know simplicity beats complexity

Loss Aversion
What It Sounds Like: “I need to remove anything risky”
What It Does: No volatility left to generate returns
Why You Avoid It: You let the AI find patterns in risk

Confirmation
What It Sounds Like: “Everyone says financials are bad in crises”
What It Does: Excluding sectors without testing
Why You Avoid It: You test financials in/out and decide

A Note on Sector Exclusions: Intent Matters

You might notice that I sometimes exclude financials (! GICS(FINANCIAL)) in certain models.

This is not because I believe “financials are bad.”

It’s because my features don’t fit.

Financial companies have fundamentally different economics:

• Debt is capital, not liability

• Book value matters differently

• Earnings are calculated differently

• Momentum signals mean different things

If my feature set was designed for industrial companies (P/E, EBITDA, etc.), applying it to banks is like using a meat thermometer on ice cream.

[again, I use both —> excluding Financials and keeping them in the universe]

The Two-Layer Secret: Why I Never Put Trading Constraints in My Universe

Most P123 users cram everything into the universe: liquidity floors, price minimums, sector excludes. They end up with 300 stocks and wonder why their AI is unstable.

I do the opposite.

My universe defines character, not tradability. If I want small caps, I use < 10M volume — which includes zombies. Why? Because the AI needs to see the bad ones to understand what “good” looks like.

Then, at the portfolio strategy level, I add the mechanical filters: > 50k volume, whatever ensures I can execute.

The result:

• AI trained on the full ecosystem of my target group

• No arbitrary cutoffs creating statistical artifacts

• Clean, tradable output

Zombies stay in school. They just don’t graduate ;-)

Why the S&P 500 Is a Great Universe

Conventional wisdom says: “The S&P 500 is efficient. Alpha comes only from small caps.”

My OOS-Live Results say otherwise. Here is why the S&P 500 consistently delivers:

1. Information Flow Creates Opportunity

More analyst coverage, more earnings calls, more institutional trading, more news flow. Conventional thinking says this means everything is priced in. Reality says more coverage means more disagreement, more misinterpretation, and more short-term dislocations.

When fifty analysts update models after earnings, they don’t agree. When institutions rebalance billions, they create temporary price pressure. Information doesn’t create efficiency. It creates chaos that looks like efficiency at daily closes but reveals itself at higher frequencies.

2. Liquidity Enables High Turnover

At $50M AUM, a $2.5M position in an S&P 500 stock represents less than 0.5% of a day’s volume. You are invisible. Slippage is negligible. In small caps, that same position might be 10-20% of daily volume—you become the market.

The S&P 500 lets you trade as fast as your signals decay without paying a liquidity premium.

3. Quality as a Free Base Layer

The S&P 500 is not random. It is a designed index with implicit quality filters: major exchanges, positive earnings, sufficient float, survival to become large.

When you start here, you inherit decades of survivorship and quality bias automatically. The AI doesn’t need to learn how to avoid bankruptcy or OTC manipulation. It focuses entirely on finding alpha.

4. Homogeneity Without Artificial Constraints

The S&P 500 is homogeneous enough for pattern recognition but large enough for statistical power. All US large caps, all USD, similar regulatory environments, comparable liquidity, institutional coverage.

Yet five hundred names is enough for robust patterns without overfitting. This is the sweet spot most users chase with custom filters and fail to find.

5. Institutional Constraints Create Inefficiency

A $10B fund cannot turn over 200% annually. They cannot hold nineteen stocks. They cannot react to short-term signals. Their size forces them to hold hundreds of names, trade slowly, think in quarters.

You have no such constraints. At $50M, you harvest the short-term alpha institutions leave on the table because they are too big to pick it up —>

The Institution Problem

Think about how a $100B fund actually operates:

• They cannot hold fewer than 50-100 stocks (diversification mandates)

• They cannot turn over more than 50-80% annually (impact constraints)

• They cannot react to day-to-day signals (too slow)

• They cannot concentrate positions (risk management)

They are forced to leave money on the table by design.

Every short-term dislocation, every post-earnings drift, every momentum spike that lasts three days instead of three months—they cannot touch it.

You can!

The Bottom Line

After building and testing dozens of AI Factor models, here is my conclusion:

AI Factor success is 80% Universe Design and 20% Factor Selection.

Most users spend 80% of their time tweaking factors and 20% on the universe. They have it backwards.

The universe is the strategy. The AI just fills in the details.

Choose your universe wisely:

• Go homogeneous for precision

• Go massive for dominant signals

Best regards,

Andreas

If you’re a quantitative researcher or data scientist in finance, you know the dirty secret of our industry: we spend 80% of our time on data problems, and 20% on actual research.

I’ve been there. The late nights debugging data pipelines. The horror of discovering look-ahead bias in your backtest. The months spent just getting to square one before you can even think about running a single ML model.

But what if I told you there’s a better way?

The 4,500+ Factor Universe

Most quants work with the usual suspects — maybe 50, maybe 100 factors if they’re ambitious. The brave ones might push to 200. But what if you could access 4,500+ meticulously engineered factors spanning more than two decades?

That’s billions of point-in-time data points. Clean. Structured. Ready to feed directly into your models.

This isn’t theoretical. It’s available right now through Porfolio123.com, and it’s changing how quantitative research gets done.

Download (for example) 97,604,400 Data Points in one run?

And you can work with those factors within Portfolio123.com or you can download the data!

The Three Data Traps (And How to Avoid Them)

1. The Look-Ahead Bias + Survivorship Bias Trap

Every quant’s nightmare: building a beautiful strategy that works perfectly in backtests but collapses in live trading because your data wasn’t truly point-in-time. P123 factors are point-in-time — what you see for January 2010 is exactly what was available in January 2010, not revised data from 2015.

✅ Splits, dividends, universe constituents, preliminary earnings — everything is handled for you!

2. The Data Engineering Black Hole

How many brilliant researchers are stuck playing data engineer instead of doing research? P123 delivers factors in ready-to-use formats — CSV or API!

Use those documents for example with Claude.ai —>

https://portfolio123.customerly.help/en/dataminer-api/introduction-to-the-api-and-tools

https://portfolio123.customerly.help/en/dataminer-api —> Data Miner API
https://github.com/portfolio-123 —> Data Miner Code

Data Miner Operations
https://portfolio123.customerly.help/en/dataminer-api/dataminer-operations

The API Wrapper - p123api
https://portfolio123.customerly.help/en/dataminer-api/the-api-wrapper-p123api

P123 Rest API
https://api.portfolio123.com/docs/index.html

3. The Signal Discovery Ceiling

Most quant shops hit a wall because they’re limited by their factor library size. With 4,500+ factors, you’re not just tweaking momentum and value — you’re exploring corners of the factor space that most funds never reach.

What This Actually Enables

Ensemble Models That Actually Work

If your ensemble model only uses 10–20 signals, you’re leaving predictive power on the table. Real strength comes from scale—which is why I run AI/ML models with over 65,000 rules, engineered through frameworks like LightGBM and ExtraTrees.

I’ve seen researchers build ML models that ingest over 300 factors simultaneously —a feat once impossible without a massive data engineering team. (My own sweet spot lies between 90–190 factors for optimal signal purity.)

True Factor Research

Ever read a paper about an interesting new factor and thought “I’d love to test that”? With P123, you’re not limited to replicating academic papers — you’re testing, combining, and discovering your own.

Production-Ready, From Day One

The biggest gap in quant research isn’t idea generation — it’s getting from backtest to production. P123’s daily updates mean the model you train today can be trading tomorrow, not six months from now after you’ve built the data infrastructure.

Best regards,

Andreas

Market

Watching some pretty wild divergences right now.

Software and Bitcoin are selling off hard, while regional banks, energy, and small caps are showing real strength. Precious metals? Volatile!

The good news: our AI factor models are handling this environment exactly as designed.

Large caps: minimal software exposure — risk sidestepped!
Small caps: positioned for relative strength where capital is actually flowing.

Access to the AI Factor Models

Paying subscribers of this blog have access to this research group in P123 which contains the following strategy books (+ more!) + I share all trades via Excel Cloud Files (DM me if you need the links to them) if you want to implement the trades manually. PLEASE use the same email here and in Portfolio123.com.

https://www.portfolio123.com/app/group/348/home

Small Caps —> ATHs!

https://www.portfolio123.com/port_summary.jsp?portid=1853536

Large Caps —> ATHs!

https://www.portfolio123.com/port_summary.jsp?portid=1898601

https://www.portfolio123.com/port_summary.jsp?portid=1853545

Large Caps Long Short

https://www.portfolio123.com/port_summary.jsp?portid=1891839

https://www.portfolio123.com/port_summary.jsp?portid=1906462

Combo Large Caps

https://www.portfolio123.com/port_summary.jsp?portid=1886955

Mid to Large Caps (S&P 1500) —> ATHs!

https://www.portfolio123.com/port_summary.jsp?portid=1851540

Best regards,

Andreas