After another fantastic turnout at our previous events, we’re bringing the KDB/Q community together once again for an evening of great conversations, new connections, and plenty of laughs. Whether you’re a seasoned developer, just starting your journey with KDB/Q, or simply curious about the ecosystem, this is the perfect opportunity to meet like-minded people in a relaxed, informal setting.

This time, we’re delighted to be hosted by First Derivative (FD) of EPAM Financial Services, who have kindly opened the doors of their London office for the community. A huge thank you to the EPAM team for supporting the event and helping us continue to grow the KDB/Q community.

Event Details

📅 Date: Thursday, 17th September 2026
🕕 Time: 18:00 onwards
📍 Venue: EPAM London Office, 9 Devonshire Square, London
✏️ Sign up: DefconQ x First Derivative Happy Hour Eventbride

Please note that registration is mandatory for security reasons. Unfortunately, entry cannot be granted without prior registration.

More Than Just a Meetup

DefconQ Happy Hour isn’t about presentations or slide decks.

It’s about conversations.

It’s about finally putting faces to the names you’ve seen on LinkedIn or Discord. It’s about sharing ideas, discussing architecture, debating language features, swapping war stories from production systems, and maybe even discovering your next career opportunity.

Whether you’re looking for technical discussions, career advice, collaborators for your next project, or simply a good chat over a drink, you’ll find all of that—and more.

Come for the Community, Stay for the Conversations

The KDB/Q ecosystem has always thrived because of its people. Every Happy Hour brings together developers, quants, architects, hiring managers, students, and industry veterans under one roof.

So if you’ve been meaning to get more involved in the community, this is your chance.

Spaces are limited, so make sure to register early.

I can’t wait to see familiar faces, meet plenty of new ones, and raise a glass to another fantastic evening with the community.

See you on 17th September!

About First Derivative & EPAM: Strategic Synergy

First Derivative, established in 1996 in Newry, Co. Down, Ireland, is a global provider of technology products and consulting services to leading institutions in finance, technology and energy. With deep expertise in trading technology, risk management, data analytics and regulatory compliance for capital markets, First Derivative delivers capital markets intelligence to clients worldwide. Today, the company employs over 1,300 professionals across 15 offices globally.

As part of EPAM Systems, First Derivative benefits from the scale and innovation of a leading global provider of digital platform engineering and software development services. Founded in 1993 in Princeton, New Jersey, EPAM specializes in bespoke software development, digital transformation, cloud solutions, data engineering, and experience design across industries including finance, healthcare, retail, and more. EPAM’s global presence includes over 60,000 professionals in more than 50 countries across Europe, Asia, North America and beyond.

Together, First Derivative and EPAM combine capital markets expertise with world-class digital engineering, offering clients strategic solutions and transformative technology to meet the demands of today’s dynamic markets.

Open Roles at First Derivative

KDB Developer - HYBRID IN THE UNITED KINGDOM: LONDON

KDB Developer - HYBRID IN THE UNITED KINGDOM: NEWRY, THE UNITED KINGDOM: BELFAST

It seems the weather isn’t the only thing refusing to cool down this summer. Only a few days ago, I wrote about L, Jacob Loveless’ new runtime for k and q, and why I believe it’s one of the projects worth watching in the evolving k ecosystem.

Well... here’s another one.

This time, it’s PeachQ, an open-source implementation of the q language spearheaded by none other than Ryan Hamilton.

If you’re sensing a pattern, you’re right. Innovation is happening. And, perhaps more importantly, it’s happening in the open.

Subscribe now

Meet Ryan Hamilton

Ryan Hamilton is hardly a new face in the community.

If you’ve spent any amount of time working with KDB/Q over the last decade, chances are you’ve used one of his tools, read one of his articles, or benefitted from one of his countless contributions.

As the founder of TimeStored, Ryan has quietly become one of the ecosystem’s most prolific builders.

In many ways, you could say he’s the Pulse of the community.

Yes, that’s a deliberate pun.

Not only because one of his flagship products is called Pulse, but because Ryan has consistently kept the community moving forward through new ideas, thoughtful discussions, and practical tooling. Over the years he has built:

• QStudio: one of the best-known IDEs for KDB/Q development.

• Pulse: powerful platform for building real-time data applications.

• Numerous open-source tools, integrations and community resources spanning far beyond traditional KDB/Q development.

I’ve had the pleasure of working with Ryan on several feature implementations for Pulse, and one thing quickly becomes apparent: if there’s a useful feature to build, Ryan usually finds a way to ship it.

A Long-Time Advocate for Improving q

One thing I’ve always appreciated about Ryan is that he doesn’t simply point out problems: he proposes solutions.

Over the years, his blog has become a collection of thoughtful ideas about where the ecosystem could evolve. He’s written about making q more approachable, improving interoperability with modern data tooling, embracing open-source technologies, expanding SQL support, introducing richer language features such as type hints, simplifying deployment, broadening compatibility, and generally making the language easier to adopt beyond its traditional financial audience.

Some of these articles were deliberately speculative. Others explored “what if?” scenarios.

With PeachQ, many of those ideas are no longer just blog posts.

Ryan now has the opportunity to implement them himself.

That might be one of the most exciting aspects of the project.

So, What Is PeachQ?!?

At its core, PeachQ is an MIT-licensed, open-source implementation of the q language. It aims to preserve the expressive array programming model and familiar qSQL semantics while providing a modern, community-driven implementation that anyone can use, inspect, extend, and improve. Today it runs on the Rayforce in-memory engine, with future plans including DuckDB-backed storage and broader interoperability with modern data ecosystems.

That last point is particularly important.

Unlike proprietary runtimes, PeachQ is being developed in public.

The project openly tracks compatibility, encourages contributions, and invites developers to help shape everything from language support and tooling to libraries, drivers, IDE integration, and package ecosystems. For a language that has traditionally lived behind commercial licensing, that’s a significant shift.

Compatibility Is Already Moving Fast

At the time of writing, PeachQ reports approximately 66% language compatibility with q.

Now, depending on how you look at it, that number can either sound encouraging or daunting.

Personally?

I think it’s incredibly impressive.

Building a language is hard.

Building a language with decades of existing behaviour, subtle semantics, and production expectations is even harder.

Having worked with Ryan before, I’m probably a little less surprised than others. He’s someone who iterates quickly, listens carefully to feedback, and ships improvements at an impressive pace. If history is anything to go by, I wouldn’t expect that compatibility percentage to stand still for very long.

More Than Just Another Runtime

It’s tempting to view PeachQ purely as another implementation of q.

I think that would be underselling it.

Open implementations change ecosystems.

They lower barriers to entry.

They encourage experimentation.

They enable universities, startups, independent developers and hobbyists to learn, prototype and build without worrying about licensing constraints.

Perhaps most importantly, they encourage tooling.

Open languages tend to attract IDEs, language servers, testing frameworks, package managers, notebooks, AI integrations, linters, debuggers and countless community projects that become difficult to justify in closed ecosystems.

That’s how ecosystems grow.

Why This Matters

If you’ve been following DefconQ recently, you’ll notice a recurring theme.

First L.

Now PeachQ.

And trust me—they’re not the only projects currently being worked on.

For many years, innovation around q largely revolved around applications built on top of the language.

Now we’re beginning to see innovation happening within the language ecosystem itself.

That’s healthy.

Competition drives innovation.

Different implementations explore different ideas.

Developers gain more choice.

And the entire community benefits.

Whether you ultimately choose commercial q, L, PeachQ, or something else entirely almost becomes secondary.

The important part is that people are building.

People are experimenting.

People are investing their time into making the ecosystem stronger.

Looking Ahead

PeachQ is still in its early days, but it’s already one of the most interesting open-source projects in the k and q world.

Knowing Ryan, I expect it to evolve rapidly. I also wouldn’t be surprised to see many of the ideas he’s championed over the years gradually finding their way into the language itself.

One thing is certain: I’ll be following the project closely.

And if this summer has taught us anything, it’s that the k and q ecosystem is entering one of its most exciting periods in years.

I’ll be here covering every step of that journey.

Good Luck Ryan!

Stay tuned.

Not only have the temperatures been rising this summer, but so has the market for exciting new projects in the k and q ecosystem. While I’m aware of several groups quietly working on some very interesting ideas behind the scenes, L-Labs a new entity founded by K/Q veteran and former HFT engineer Jacob Loveless, is the first to bring something new to market.

Meet L.

A brand-new runtime for k and q that has just become publicly available, with the ambitious goal of rethinking how k and q applications are executed.

As someone who spends a large amount of time looking at everything happening around the KDB/Q ecosystem, it’s always exciting to see innovation emerge, especially when it comes from someone who has been working with the language for years.

Why Build Another Runtime?

It’s a fair question.

After all, q has been around for decades, has powered some of the world’s largest financial institutions, and remains one of the fastest technologies available for time-series analytics. So why build another runtime? Jacob’s motivation is refreshingly straightforward.

As he explains in his Why L? article, the goal wasn’t to replace existing tools for the sake of replacing them. It was to build a modern runtime that explores different implementation choices, removes historical constraints where possible, and provides a fresh foundation for future development. Rather than simply recreating the past, L is an experiment in how a modern k and q runtime could look today.

Projects like this are healthy for an ecosystem.

They challenge assumptions.

They encourage experimentation.

And they often introduce ideas that eventually influence everyone.

Performance Looks… Interesting

One of the first things that caught my eye was the benchmark section. L has been benchmarked across a variety of workloads and, at least in the published results, performs extremely well, often outperforming many other language runtimes in those tests. It’s an impressive showing and certainly suggests there’s some serious engineering under the hood.

Of course, benchmarks always deserve a healthy dose of context.

Different benchmarks measure different things, and synthetic workloads don’t necessarily translate directly into production systems. The workloads you care about might look very different from those used in published comparisons. For those of us working with production KDB/Q systems, the natural question is how L behaves on the kinds of real-world problems we care about: large in-memory databases, tickerplants, gateways, IPC-heavy applications, historical queries, and high-throughput market data processing.

That’s still something the community will have to explore over time. But regardless of where those comparisons ultimately land, it’s encouraging to see a new runtime pushing performance and introducing fresh implementation ideas.

More Choice Is Good for Everyone

Perhaps the most exciting aspect of L isn’t a benchmark chart.

It’s the fact that somebody decided to build something new.

For a long time, the k and q ecosystem has been relatively quiet when it comes to alternative runtimes and implementations. Seeing fresh ideas emerge lowers the barrier for experimentation and opens up entirely new opportunities for developers, researchers, and companies interested in the language. Competition drives innovation.

New implementations often lead to new tooling. Different implementation strategies expose different strengths. Three decades have seen several implementations of k, none as fast as k4. L has supported commercial systems for a decade. And it claims to be significantly faster than k4. And even if they never become the dominant runtime, they frequently inspire improvements across the wider ecosystem. That’s good news for everyone who enjoys working with vector programming languages.

What Else Is Brewing?

L certainly isn’t the only exciting project currently in the works. Behind the scenes, several teams are quietly building new languages, tools, runtimes, libraries, and ideas that have the potential to shape the future of the k and q ecosystem. Some are still under wraps, others are getting closer to seeing the light of day. As always, DefconQ aims to stay at the forefront of everything happening in the community. I’ll be following these developments closely and sharing them here as soon as they’re ready to go public. So if you’d like to stay ahead of the curve, make sure to follow DefconQ. The easiest way is to subscribe to the free newsletter and have every new article delivered straight to your inbox, or follow me on LinkedIn where I regularly share the latest news, tutorials, and developments from across the ecosystem.

Because if this summer has shown us anything, it’s that the k and q world is far from standing still.

I’ll Be Keeping a Close Eye on This One

It’s still very early days, and only time will tell how L evolves and where it ultimately fits within the ecosystem. But one thing is certain: it’s one of the most interesting developments in the k and q world this year. I’ll definitely be keeping a close eye on the project, experimenting with it, and following its progress as it matures. If you’re curious yourself, I’d encourage you to take a look, play around with it, and form your own opinion. As always, I’ll keep you posted as I learn more.

Stay tuned.

The more I thought about it, however, the more I realised that extended trading hours are not really about trading hours at all.

They are about a much bigger shift taking place across financial markets.

Markets are becoming more continuous. More fragmented. More data-intensive. More demanding.

And most importantly, they are beginning to expose assumptions that have been baked into market data systems for decades.

That’s exactly why I’m excited to be joining KX for an upcoming webinar discussing the infrastructure implications of always-on markets. Together we’ll explore what extended trading hours, odd lots, sub-penny pricing, AI-driven workflows, and increasingly continuous markets mean for firms operating in today’s landscape.

The End of “Business Hours”

For most of modern market history, technology teams have enjoyed a luxury they rarely appreciated: downtime. Markets opened. Markets closed. Systems had time to breathe. The overnight window was often used to recover from failures, perform maintenance, replay data, rebuild databases, execute batch processes, and generally prepare for the next day. If something went wrong at 6 p.m., there was usually time to fix it before the opening bell the next morning.

Those assumptions are rapidly disappearing.

Whether it’s extended trading sessions, overnight equity markets, crypto’s 24/7 trading model, or increasing globalisation of liquidity, markets are steadily moving toward continuous operation. For market participants, this creates new opportunities. For infrastructure teams, it creates new headaches. Because an always-on market eventually requires always-on infrastructure.

And that changes everything.

More Than Just More Data

When people hear “extended trading hours,” the first reaction is often: “So we’ll have more data?”

Yes. But that’s probably the least interesting part of the story. The real challenge comes from the combination of trends happening simultaneously.

Odd lots are becoming increasingly important. Sub-penny pricing continues to evolve. Market fragmentation continues to increase. AI-driven workflows are creating entirely new categories of data consumers. Real-time analytics are expected everywhere. Historical data is no longer simply stored; it’s continuously queried, replayed, modelled, and fed into machine learning systems. Each of these developments adds pressure individually. Together, they fundamentally change how firms need to think about infrastructure.

What used to be a relatively clean separation between real-time systems, historical databases, research environments, and analytics platforms is becoming increasingly blurred.

And that’s where things start getting interesting for KDB/Q developers.

The Recovery Window Is Disappearing

One of the topics I’m particularly looking forward to discussing during the webinar is the disappearing recovery window. Historically, many systems relied on periods of reduced activity to catch up. Data could be reconciled. Processes could be restarted. Infrastructure could be upgraded. Large batch jobs could run. Many firms still have critical workflows that quietly assume these windows will always exist.

But what happens when they don’t?

What happens when there is no convenient downtime?

What happens when trading activity never truly stops?

The challenge is no longer simply building systems that are fast. The challenge becomes building systems that remain continuously available, continuously observable, and continuously resilient. That is a very different engineering problem.

When Architectures Start to Crack

Most firms didn’t intentionally design fragmented architectures. In fact, many of the decisions that created today’s technology stacks were entirely reasonable at the time. One system handled real-time data. Another stored historical information. A separate platform supported research. Analytics ran somewhere else. AI workloads are added nowadays.

Over time, however, every additional component introduces another handoff. Another integration, another potential point of failure, another place where latency, inconsistency, operational risk, and complexity can accumulate.

When markets operate within defined sessions, those compromises can often be managed. When markets become continuous however, the cracks become much harder to hide. The cost of moving data between systems increases.The operational burden increases. The complexity increases.

And eventually firms start asking a difficult question: “Would we build it this way if we were starting today?”

Building for the Next Decade

This naturally leads to the final topic we’ll discuss during the webinar: what principles should guide the next generation of market infrastructure? If markets are becoming more continuous, more granular, and more data-intensive, then infrastructure needs to evolve alongside them.

Firms increasingly need platforms that can handle real-time analytics, historical data, research workloads, and AI-driven applications without requiring a maze of disconnected systems. They need interoperability. They need resilience. They need simplicity. And they need technology capable of operating continuously rather than merely surviving until the next maintenance window.

This is where KDB-X enters the conversation.

One of the reasons KDB/Q has remained so dominant in capital markets is its ability to unify problems that traditionally required multiple technologies. KDB-X takes that philosophy further, providing a platform designed for modern workloads where real-time, historical, analytics, and AI increasingly coexist.

We’ll explore exactly what that means during the discussion.

Join Us

Whether you’re a KDB/Q developer, quant engineer, market data architect, researcher, or simply someone interested in where market infrastructure is heading, this webinar should be a fascinating discussion. Because the real question isn’t whether markets will continue to evolve.

They will.

The real question is whether our infrastructure is evolving with them.

I hope you’ll join us as we explore what happens when markets never sleep.

SIGNUP HERE

You can build the cleanest tickerplant in the world, design elegant schemas, optimise queries until they scream, and architect the perfect stack, but without data flowing into it, it’s all just a very expensive and slightly nerdy sculpture.

Market data is the lifeblood of every serious KDB/Q system. It’s where the story begins. And yet, for something so fundamental, getting access to market data as an individual developer has historically been ... painful.

That’s exactly why I’m excited to announce that DefconQ and Databento are teaming up for an upcoming Meetup in London, a community event for quants, financial engineers, developers, and anyone working with market data who wants to see what happens when market data stops being a headache and starts becoming something you can actually work with.

Who Is Databento?

If you haven’t come across Databento yet, chances are you will. Databento recently celebrated its third anniversary, and in a remarkably short time has established itself as one of the most exciting and fastest-growing players in the market data space. The company was built around one core idea: creating modern market data infrastructure capable of serving some of the world’s leading trading firms while still remaining accessible to individual developers, researchers, startups, and smaller teams.

And clearly, that vision resonated.

Today, Databento serves more than 40,000 users and powers firms representing hundreds of billions in AUM, alongside products spanning fintech, AI, quantitative research, and trading infrastructure. What makes Databento particularly refreshing is that it genuinely feels like it was built by engineers who have personally suffered through the pain of traditional market data providers before.

Because they have.

Historically, market data has often felt like it was built for procurement teams, lawyers, and infrastructure departments first ... and developers somewhere near the bottom of the priority list. Complicated onboarding. Opaque pricing. Questionable APIs. Documentation that feels like it was written during the Cold War. Strange licensing restrictions. Integration projects that somehow become multi-week engineering exercises.

Databento flipped that script.

Their platform makes it possible to access historical and real-time market data through a clean, well-documented API, without the usual layers of friction many developers have simply accepted as “part of the process.” Fast access. Transparent pricing. Modern tooling. Excellent documentation.

It sounds simple, but anyone who has integrated market data providers before knows this is absolutely not a given.

The Founders Behind It

Databento was founded by people who clearly understand both the technical and commercial realities of market data because they lived through them themselves while working inside the industry.

At the top is Christina Qi, alongside Luca Lin and Renan Gemignani. Together with the broader core team, they bring experience from prestigious hedge funds, quantitative trading firms, and high-performance financial infrastructure environments where market data is not just important, it is mission critical.

And honestly, I think that experience is exactly why the product feels different.

This wasn’t built by people theorising about market data problems from the outside. It was built by engineers and quants who were tired of dealing with bad market data vendors themselves. The frustrations that many of us have simply learned to tolerate, painful integrations, poor tooling, strange workflows, slow onboarding, became the motivation for building something better.

And perhaps that’s what stood out to me most when I met Christina and the Databento team in London: they genuinely understand the problems developers and engineers face because they’ve been on the receiving end of them.

That combination of technical depth, practical thinking, and building from real-world frustration tends to produce very good products.

How We Got Here

This meetup actually started with a DefconQ tutorial. A while ago, I wrote my real-time stock market tutorial, walking readers through how to stream live data into a KDB/Q stack. Initially, I used Yahoo Finance to pull market data because, well ... it was free and it worked. Until it didn’t.

Anyone who has ever used Yahoo Finance for anything even remotely serious knows what happens next: you request a little too much data and suddenly the door gets slammed shut. That was a problem. I wanted to create something realistic for DefconQ readers, something that actually resembled how a production-style system works. But I also wasn’t exactly sitting on a Bloomberg budget to pay for a full real-time market data feed just to support a tutorial.

That’s when I came across Databento. The free credits they provide were exactly what I needed to make the tutorial happen.

Now, I’ll be honest: my expectations were cautiously optimistic.

Because integrating market data providers can be painful. Documentation gaps, weird APIs, unexpected licensing complexity, strange authentication workflows, if you’ve done it before, you know.

So I mentally blocked off some time for “integration pain.” And then something strange happened: There wasn’t any.

Thanks to the excellent documentation and a genuinely easy-to-use API, I managed to integrate Databento into the tutorial in a single evening, while my wife was at a Beyoncé concert.

That might sound like a silly benchmark, but trust me: in market data integration time, that’s lightning fast.

From Tutorial to Meetup

The tutorial went on to become one of DefconQ’s more popular hands-on walkthroughs, and to my surprise, Christina herself featured it. That eventually led to me meeting Christina and the Databento team at a London meetup, and if you’ve ever wondered whether technical people can also be genuinely fun people, the answer is yes.

Now they’re back in town.

And this time, we’re hosting something together.

Why You Should Attend

This meetup is not just another fintech networking event where everyone pretends to enjoy warm white wine and vague conversations about “innovation.” This is a real community event for people who actually work with data.

If you are a:

• Quant

• Financial Engineer

• Market Data Engineer

• KDB/Q Developer

• or anyone building systems that depend on fast, reliable market data ...

... this is genuinely worth attending.

You’ll get the opportunity to learn directly from the Databento team, understand how their platform works, ask technical questions, and see why accessing market data has become dramatically easier than it used to be.

And perhaps just as importantly, you’ll meet others in the DefconQ community who care about the same problems you do, architecture, infrastructure, data engineering, market data, production systems, and all the weird little things that only people in this niche get excited about.

There will be conversations, technical discussions, networking, and plenty of opportunities to ask the questions you don’t usually get to ask.

Because at the end of the day, market data is where many of our systems begin.

And if someone makes that easier for developers, it’s worth paying attention.

See you there.

The Details:

When: Wednesday, July 1st 2026
Where: Hedosophia, 2 Soho Place, London W1D 3BG
Sign Up Link: here

IMPORTANT: This event is expected to reach capacity, so sign up as soon as possible. If you can no longer attend, please cancel your registration to free up your spot for someone else.

Subscribe now

That’s exactly why we’re launching the DefconQ Weekly KDB/Q Challenge.

The idea is simple: have a bit of fun, keep your Q brain sharp, and improve your KDB/Q skills on the side, without it feeling like homework. Every week, a new KDB/Q challenge will be posted on the DefconQ Community Discord Server, and everyone is invited to participate.

What Is the Weekly KDB/Q Challenge?

Think of it as your weekly Q workout. Each week, we will post a brand-new challenge designed to test your KDB/Q skills, your vector thinking, your creativity, and occasionally your patience.

Some challenges will be quick and fun. Others might make you stare at your screen for an hour questioning your life choices. That’s part of the experience. The difficulty will vary from week to week, meaning there will be something for everyone, from complete Qbies taking their first steps in Q, to seasoned KDB/Q developers looking for a challenge worthy of their ego.

And the best part?

There’s usually more than one way to solve a problem in Q.

That means the real fun often starts after the challenge, when the community begins comparing solutions, debating elegance versus performance, and collectively admiring (or questioning) the weird genius of some of the answers.

Why Are We Doing This

DefconQ has always been about making KDB/Q easier to learn and more enjoyable to work with. But the truth is, reading tutorials and watching walkthroughs will only take you so far. At some point, you need to get your hands dirty. You need to write code. Break things. Try ideas. Compare solutions. Learn from others. That’s where the Weekly Challenge comes in. It creates a fun, low-pressure way to keep practicing regularly while exposing yourself to different types of problems and different styles of thinking.

Because one of the best ways to improve in Q is to see how someone else solved the same problem in a completely different, and often much more elegant, way. And let’s be honest: Q developers are a competitive bunch. So why not turn learning into a bit of a friendly battle?

Hosted on the DefconQ Community Discord Server

All challenges will be hosted on the DefconQ Community Discord Server, where participants can submit solutions, discuss approaches, compare ideas, and occasionally argue over whether one-liners are elegant or just deeply irresponsible. However, this isn’t about winning in the traditinal sense. It’s about

• Improving your skills, while

• learning from others, while

• seing new techniques

• and having some fun with the community along the way.

Of course, a little bragging rights never hurt either.

Ready To Rumble?

Whether you are a beginner trying to build confidence, an intermediate developer looking to sharpen your skills, or an experienced Q wizard who thinks nobody can out-vector you, the DefconQ Weekly KDB/Q Challenge is for you.

A new challenge. Every week. Different levels of difficulty. Community discussion. Multiple solutions. Plenty of learning. And perhaps the occasional reminder that Q can still humble all of us.

Come join the DefconQ Community Discord Server, take on the challenge, and let’s see what you’ve got.

DefconQ Community Discord Server invite link here

Happy coding.

Subscribe now

The goal is simple. We want to create a place where people can:

• exchange ideas

• share best practices

• ask questions without feeling intimidated

• discuss architecture and production systems

• collaborate on projects

• and ultimately learn from each other

And importantly: this is not intended to become just another forum about KDB/Q.

The DefconQ server is designed to be a genuine community space: independent, objective, and driven by the people participating in it. No consultancy agenda. No hidden sales pitch. No “our framework solves everything” marketing. Just developers helping developers and people passionate about vector programming sharing knowledge.

Whether you are:

• completely new to q

• building your first tickerplant

• debugging IPC at 2AM

• designing distributed architectures

• or debating whether everything should really be a list

...there will be a place for you.

The server will include dedicated channels for:

• KDB/Q help

• architecture discussions

• performance tuning

• KDB-X modules

• code reviews

• learning resources

• and community projects

Over time, the goal is to grow this into one of the strongest independent KDB/Q communities out there, a place where knowledge is shared openly and where people can genuinely level up together. So if that sounds like your kind of place, come join us.

Join the DefconQ Community Discord Server here

Happy chatting and Happy coding.

Subscribe now

While previous meetings spanned a range of Iversonian programming languages, this year’s meeting will be entirely on KDB/Q. That said, the principles and concepts explored remain broadly applicable across the wider family of vector languages, with future meetings set to expand again.

Iverson College is half conference, half house party. There’s no rigid agenda, no back-to-back slide decks, and no passive audience. The participants are the programme. It’s a collaborative, immersive environment where ideas are shared, challenged, and built upon in real time. Inspired by the ethos outlined in the college’s foundations, the focus is on deep learning through interaction, not presentation. Even though there will be no pre-set agenda, there will be a handful of presentations, knowledge-sharing sessions, and story times from some of the most experienced voices in the industry. We shall bring together some of the smartest, most experienced kdb developers.

With the recent release of KDB-X and the introduction of modules, this year’s meeting of the College will centre around building the modules the community actually needs. And we won’t just talk about them, we’ll start implementing them. The goal is to lay the groundwork for a shared, community-driven codebase that can be reused and extended in real production environments.

The setting for all of this is Milden Hall, a Tudor barn in Suffolk that has hosted some of the college’s most memorable meetings. It’s a unique workspace that fosters the kind of close collaboration and focused work the event is known for. The ticket price of £1300 includes meals (we bring our own chefs) and hostel-level accommodation, ensuring everyone can fully immerse themselves in the experience without distraction. Places are strictly limited to just 20 attendees, keeping the environment intimate, focused, and highly interactive.

When: 6 - 10 September, 2026
Where: Milden Hall, Suffolk
Price: 1300£
Signup here: You can secure your ticket by sending an email to dean@iversoncollege.com and we will send you the payment instructions

Some impressions of the 2024 edition of the Iverson College in Cambridge, captured by Miki Yamanouchi, a London based photographer and film maker.

You can learn more about Iverson College and explore previous editions here.

KDB-X modules are designed to change that.

What are KDB-X modules

At a high level, modules introduce a structured way of packaging logic into self-contained, reusable components. Instead of throwing functions into a shared global namespace and hoping nothing collides, each module lives in its own isolated space. It decides what to expose, what to keep private, and how it should be consumed. This might sound like a small shift, but in practice, it’s a fundamental upgrade in how you design and reason about your systems.

What makes this particularly powerful is the consistency the module framework brings. There is now a standardised way to define, load, and interact with components. You no longer have to guess where something is defined or how it should be used, the structure enforces clarity. Over time, this reduces cognitive load, improves maintainability, and makes collaboration far less painful.

The real value, however, shows up when your codebase starts to grow.

Why you should be using KDB-X modules

Modularity is not just a “nice to have”, it’s what separates scripts from systems. When code is modular, it becomes easier to isolate functionality, reason about behaviour, and make changes without unintended side effects. You’re no longer editing a monolithic block of logic; you’re working with well-defined building blocks that can evolve independently.

This naturally leads to reusability. Instead of rewriting the same utilities across projects, logging, data ingestion, query handling, connectivity, you encapsulate them once as modules and reuse them wherever needed. The same component can be loaded into multiple processes, shared across teams, or even reused across entirely different systems. Over time, this compounds into a significant productivity gain. You spend less time rewriting and more time building.

And then comes scalability, not just in terms of performance, but in terms of architecture.

Subscribe now

As systems grow, complexity doesn’t increase linearly. It explodes. Without structure, even small changes can have unpredictable consequences. Modules provide a way to contain that complexity. New functionality can be introduced as new modules, existing ones can be extended or replaced, and the overall system remains stable because the interfaces between components are clearly defined. You’re effectively building a system that is designed to evolve.

A useful way to think about this is through a slightly nostalgic analogy.

Remember building Lego sets as a kid. At first, you followed the instructions. Step by step, piece by piece, until you had exactly what was on the box. It was satisfying, but also a bit limiting. The real fun began when you stopped following the instructions. When you took pieces from different sets, combined them, and started building something entirely your own.

KDB-X modules are those Lego pieces.

Each module represents a building block, a piece of functionality that does one thing well. On their own, they’re useful. But the real power comes from how you combine them. You can stack them, extend them, swap them out, or integrate them into larger systems. You’re no longer constrained by a predefined structure; you’re designing your own.

And that’s the key shift.

You move from asking, “How do I write this function?” to “How do I design this component?” From “Where do I put this code?” to “How does this fit into the system?”. It’s a transition from writing code that works today to building architecture that will still make sense tomorrow.

KDB-X modules don’t just give you a cleaner way to organise your code. They push you toward better engineering practices. They encourage you to think in terms of boundaries, interfaces, and responsibilities. They make reuse the default rather than an afterthought. And most importantly, they give you the tools to scale, not just your systems, but the way you build them.

Because in the end, the difference between a fragile codebase and a robust system isn’t how clever the code is. It’s how well it’s put together.

How to create a module

Now that we’ve covered the what and the why, let’s look at something more practical, the how. How do you actually create a KDB-X module? The requirements are refreshingly simple. To define a module, all you need is a variable called export. This variable contains a dictionary that represents the public interface of your module, in other words, the functions (or values) you explicitly want to expose to the outside world.

Let’s pause for a moment and walk through a concrete example.

// /Users/alexanderunterrainer/.kx/mod/foo/init.q
export:([f:{x+1};g:{x*2}])

This defines a module with two functions, f and g. That’s it. No boilerplate, no ceremony, just a clean declaration of what your module provides.

To make this module usable, you place it on the search path (more details on how to use an appropriate file structure can be found on the official documentation here). Once it’s in place, you can load it using the use function:

alexanderunterrainer@Mac:~/.kx/mod/foo|⇒  pwd
/Users/alexanderunterrainer/.kx/mod/foo
alexanderunterrainer@Mac:~/.kx/mod/foo|⇒  ls
init.q
alexanderunterrainer@Mac:~/.kx/mod/foo|⇒  q
KDB-X 5.0 2026.01.22 Copyright (C) 1993-2026 Kx Systems
q)foo:use`foo
q)foo
f| {x+1}
g| {x*2}
q)foo.f
{x+1}
q)foo.g
{x*2}
q)foo.f 3
4
q)foo.g 5
10

When you call use, it loads the module and returns its export dictionary. This allows you to assign the module to a variable, in this case foo, and access its functions in a structured and predictable way.

Simple as it looks, this pattern is incredibly powerful. You’ve just defined a reusable, self-contained component with a clear interface, a building block that can now be plugged into larger systems, shared across projects, or extended over time.

The export variable

As we have already seen, the export variable is essentially a dictionary that defines the public interface of a module, the functions and values you explicitly expose to the user:

export:([f:{x+1};g:{x*2}])

You can keep variables and functions private, meaning they are only accessible within the module itself, simply by not including them in the export dictionary. Let’s see how this works in practice.

// /Users/alexanderunterrainer/.kx/mod/foo/init.q
privateVar:100;
publicVar:`HelloWorld
privateFunc:{show publicVar}
publicFunc:{show privateVar}

export:([f:{x+1};g:{x*2};publicVar;publicFunc])

When loading the module, we can observe the following:

q)myNameSpace:use`foo
q)myNameSpace
f         | {x+1}
g         | {x*2}
publicVar | `HelloWorld
publicFunc| {show privateVar}
q)myNameSpace.publicFunc
{show privateVar}
q)myNameSpace.publicFunc[]
100

However, there is an important nuance when it comes to variables. While you can include variables in the export dictionary, they are exposed as copies, not references. This means any modification on the user side won’t affect the internal state of the module. To properly expose and control access to internal variables, the recommended approach is to provide getter and setter functions.

This becomes evident when you try to modify the private variable privateVar by overwriting it.

q)myNameSpace.privateVar:1000000
q)myNameSpace
f         | {x+1}
g         | {x*2}
publicVar | `HelloWorld
publicFunc| {show privateVar}
privateVar| 1000000
q)myNameSpace.publicFunc[]
100

As shown, this doesn’t actually change the original variable inside the module, instead, it creates a new variable in the user’s namespace. Meanwhile, the original value defined within the module remains intact and continues to be used by the public function publicFunc.

To properly update the value of privateVar, we need to expose controlled access. Let’s add a public setter function that allows us to modify the internal state of the module.

// /Users/alexanderunterrainer/.kx/mod/foo/init.q
privateVar:100;
publicVar:`HelloWorld
privateFunc:{show publicVar}
publicFunc:{show privateVar}
setPrivateVar:{privateVar:x}
export:([f:{x+1};g:{x*2};publicVar;publicFunc;setPrivateVar])

We can now restart our KDB/Q process and reload the module.

q)myNameSpace:use`foo
q)myNameSpace
f            | {x+1}
g            | {x*2}
publicVar    | `HelloWorld
publicFunc   | {show privateVar}
setPrivateVar| {privateVar:x}
q)myNameSpace.setPrivateVar[1000]
1000
q)myNameSpace.publicFunc
{show privateVar}
q)myNameSpace.publicFunc[]
100

Somewhat unexpectedly, the private variable wasn’t updated by our setter function when using the assign operator :. Let’s try again, this time using the set operator and referencing the variable name directly.

// /Users/alexanderunterrainer/.kx/mod/foo/init.q
privateVar:100;
publicVar:`HelloWorld
privateFunc:{show publicVar}
publicFunc:{show privateVar}
setPrivateVar:{`privateVar set x}
export:([f:{x+1};g:{x*2};publicVar;publicFunc;setPrivateVar])

Once again we restart our KDB/Q process and load the module

q)myNameSpace:use`foo
q)myNameSpace
f            | {x+1}
g            | {x*2}
publicVar    | `HelloWorld
publicFunc   | {show privateVar}
setPrivateVar| {`privateVar set x}
q)myNameSpace.setPrivateVar[1000]
`privateVar
q)myNameSpace.publicFunc[]
100
q)privateVar
1000

To our surprise, the result still isn’t what we expected. Rather than updating the private variable inside the module, the set operator ends up creating a new global variable, definitely not the outcome we were aiming for. To understand why, we need to take a closer look at the official KDB-X module documentation. There, we find an important detail: when assigning to globals from within functions, the standard assignment operator : must be replaced with the double-colon :: operator to ensure the variable is treated as global rather than local.

Once we make that change, the behaviour aligns with our expectations, the private variable inside the module is correctly updated.

// /Users/alexanderunterrainer/.kx/mod/foo/init.q
privateVar:100;
publicVar:`HelloWorld
privateFunc:{show publicVar}
publicFunc:{show privateVar}
setPrivateVar:{privateVar::x}
export:([f:{x+1};g:{x*2};publicVar;publicFunc;setPrivateVar])

After restarting our KDB/Q session and loading the module one final time

q)myNameSpace:use`foo
q)myNameSpace
f            | {x+1}
g            | {x*2}
publicVar    | `HelloWorld
publicFunc   | {show privateVar}
setPrivateVar| {privateVar::x}
q)myNameSpace.publicFunc[]
100
q)myNameSpace.setPrivateVar[1000]
q)myNameSpace.publicFunc[]
1000

That’s it, we made it. You should now have a solid foundation in KDB-X modules and be ready to build and assemble your own KDB/Q Lego pieces... and maybe even your battleship.

Tips and Tricks

As you’ve probably noticed, we had to restart and reload our KDB/Q process quite frequently while building the final module, not exactly the most enjoyable workflow. Fortunately, KX anticipated this and introduced a reuse function that allows you to reload a module without restarting the entire process.

While this functionality isn’t officially documented yet, you can find it in the .Q.m namespace, the dedicated namespace for everything module-related.

q).Q.m
     | ::
M    | (+`p`s!(("";"/Users/alexanderunterrainer/.kx/mod/foo");``))!+(,`m)!,``.m.foo
dp   | k){"/"\:x}
pd   | k){"/"/:x}
pe   | k){y,/:x}[(".k";".q";".k_";".q_";".ma64.so")]
p1   | k){x@*&0<#:'!:'`$":",'x}
pl   | k){,/{pd'(,y),/:,/((-1_x),/:,:'pe[:/x];x,/:,:'pe["init"])}[$y]'x}
ld   | k){S:(,`)!,(::);g:{.[`/:x,y;();:;z]}d:."\\d";$[|/x like/:-1 0_\:"*.?_";[."\\l ",x;$[99h=@e:$[`export in !d;d`export;'"no export: ",x];$[@. e;g[`export]e:S,e;e];g[`export]S,e y]]
 [x:`$|(1+x?".")_x:|x;g[`export]S,r:(@[x 2:;(`kexport;..
ef   | k){e0:."\\e";r:@[{."\\e 1";(0b;x y)}x;y;{(1b;x)}];@[{."\\e ",$x};e0;::];r}
ch   | k){[m;p;s;z;a];d0:."\\d";m0:-200!0;p0:-201!0;-200!m1:`$1_$m;-201!p;."\\d .",$m1;M[(p;s)]:([m:`:]);r:ef[z;a];M::([],p;,s)_ M;."\\d ",$d0;-200!m0;-201!p0;r}
dl   | k){$[(:/d:dp x)like"init.*";pd (-1_d);x]}
up   | k){[m;x;a]([p;s:`]):M?([m]);if[~^u:M[(p;x)]`m;$[u~`:;'`cycle;:u]];r:ch[m;p;x;m[`provide] .;(x;a)];:[r;(0b;r);[M[(p;x)]:([m:r]);r];(1b;r);'r]}
mn   | k){a:1_(),x;x:x0:*x;if[1<#b:":"\:$x;:up[mn`$*b;(`$":"/:1_b);a]];n:$[j:+/&\^x:`\:x;[x:j_x;{`/:`,$[0<l:1+x-(#y:`\:y);`m,`$($l);(-x)_y]}[j-1;-200!0]];`.m]
 if[~#l:p1 ll:pl[$[j;,{$[~#x;,".";x]}pd {y:((,".")~*y)_y;$[0<l:x-#y;l#,"..";(-x)..
mp   | k){([p]):M?([m:mn x]);p}
reuse| k){r:ch[m:mn x;p;`;ld[p1 $[(:/dp p:mp x)like"*.*";,p;p,/:pe["/init"]]];1_(),x];:[r;(0b;r);[M[(p;`)]:([m]);m];(1b;r);'r];use x}
fp   | k){$[$[99h=@y;11h=@!y;0];(`/:'x,/:!y).z.s'y;(100h=t)|((103h<t)&(t<112h))|$[112h=t:@y;"code"~$y;0b];x . ();y]}
use  | k){. `/:mn[x],`export}
SP   | ,"/Users/alexanderunterrainer/.kx/mod"

Final Thoughts

KDB-X modules are more than just a new feature, they’re a powerful shift in how you build, structure, and scale kdb+/q systems. By embracing modularity, you move from writing scattered scripts to designing clean, reusable building blocks that grow with your architecture. The real value comes when you start creating your own modules, combining them, and shaping systems that are truly yours. So don’t just read about it, start building, experiment, and turn your ideas into components you can stack, scale, and evolve.

Happy Coding!

Resources

Offical KX Modules Overview

I am excited to share that DefconQ x Squarepoint are joining forces to invite all KDB/Q developers, engineers, and enthusiasts to a community get together and networking event in the heart of London.

And this one is going to be special, for two big reasons:

First: With the backing of Squarepoint, one of the largest KDB/Q users on the street and a firm operating at the forefront of quantitative trading and large-scale data engineering, you will get the chance to connect with some of the strongest KDB/Q minds in the industry.

Second (and even better): Squarepoint’s recruiters will be there in person, ready to chat about open roles across engineering, quantitative research, and infrastructure. If you’ve ever wondered what it really takes to build, scale, and operate KDB/Q systems in a top-tier quant environment, this is your chance to ask the people doing it.

Opportunities like this don’t come around every day, we hope you won’t miss it. Light refreshments will be provided.

What to Expect

• Welcoming talk by Squarepoint

• Informal networking with the DefconQ community

• Deep technical conversations about KDB/Q in production

• Insight into how leading hedge funds leverage time-series architecture

• Direct access to recruiters hiring for real, impactful roles

See you at the Globe.

When: Wednesday, 18th March, from 18:00
Where: The Globe, Moorgate
Signup here: Event Signup

About Squarepoint

Squarepoint is a privately held firm led by its four founding partners across seventeen cities around the world. The business dates back to 2000 and has evolved through Lehman Brothers and Barclays before becoming independent in December 2014.

We are committed to developing a world-class quantitative investment platform through our systematic, diversified approach. At Squarepoint, collaboration occurs across all teams, sharing investment ideas and promoting a culture of ownership and accountability. We pride ourselves on our deep expertise across Quantitative Investment Management, Technology and Trading Operations.

You can read more about Squarepoint here...

Open Roles at Squarepoint

Squarepoint’s success comes down to its people. Open Roles at Squarepoint

From Blank Slate to Gateway: What We’re About to Build

In the following sections we will build the core pieces of a scalable query routing framework inspired by the Query Routing from KX white paper, written by Kevin Holsgrove. The goal is to illustrate a practical, production-ready pattern where multiple processes coordinate to handle user queries efficiently in a distributed KDB/Q system.

But before we dive into the technical implementation of the framework, let’s take a brief step back and look at the system from a higher-level perspective, focusing on the processes we’re about to build and how they fit together. At the center of our design are three key players:

• Gateway: This process sits at the edge of your system, accepts client requests, annotates them with a unique sequence identifier, and hands them off for routing. Each incoming query is captured in an internal table, tagged with a unique sequence number, and linked back to the client’s process handle. The gateway then asks the load balancer for an available service and forwards the query to whatever worker gets allocated. When the result comes back, the gateway uses the sequence number to match it to the original request and sends the response straight back to the right user.

• Load Balancer / Connection Manager: Think of this as the brains of query distribution. It knows what services are available (e.g., multiple HDBs, RDBs, or custom compute services), which ones are busy, and which are free to take work. Rather than simple round-robin, this component tags incoming queries and only allocates them to idle backends, minimizing queueing and improving latency.

• Service Instances: These are the actual workers: database processes, analytics engines, or custom report builders that evaluate user queries. Each one registers itself with the load balancer so it can be included in allocation decisions. Once a service finishes a request, it reports back to the load balancer so it can be assigned another job. By running multiple instances of the same service, for example, HDBs backed by identical data sets or RDBs subscribing to the same tickerplant, we create a shared pool of resources that can be spread across different servers. This opens the door to a hot-hot setup, where the framework can not only distribute load efficiently across available instances, but also seamlessly reroute user queries if a service or host goes down, without clients needing to be aware of the failure.

The Life of a Query

From a user’s perspective, interaction is straightforward. Queries are sent to the gateway handle via a single entry point, the userQuery function, which takes a two-element list: the target service and the query to execute. Communication with the gateway uses deferred synchronous messaging, giving users a familiar request–response model while allowing the system to remain non-blocking under the hood.

Services backing the different database types can be distributed across multiple servers, with the number of instances driven by available hardware and user demand, and configured independently per service type. To keep the focus where it matters, we intentionally keep the gateway’s routing logic simple here and put the spotlight on the load balancer’s role in the system. The flow itself is equally clean. Once a query enters the system, the gateway coordinates with the load balancer to allocate an appropriate service, forwards the request, and waits for the result. Responses are then routed back to the user through the gateway. If something goes wrong, an invalid service request or an error during query evaluation, the gateway propagates a clear error back to the client, keeping failure handling centralized and predictable.

The load balancer, instead of relying on a simple round-robin strategy, takes a more deliberate approach. It actively tracks which service instances are free and only allocates queries to resources that are actually available. Once a service finishes executing a query, it explicitly notifies the load balancer that it’s ready to take on more work, allowing the system to maintain an accurate view of capacity at all times. The only exception to this flow occurs when a service has been allocated but the user disconnects before the query completes. In that case, the gateway steps in and informs the load balancer that the service is no longer needed, ensuring resources are released promptly and not left unnecessarily occupied.

The diagram below shows the full lifecycle of a query as it moves through the system.

Where the Rubber Meets the Road: The Technical Walkthrough

In this section, we’ll roll up our sleeves and walk through the technical implementation of each moving part in the system: the gateway, the load balancer, the backend services, and a client process sending queries through the gateway. We will first outline the key responsibilities and core functionality of each process, so it’s clear what role they play and how they fit together. Once the concepts are in place, we’ll look at how they’re implemented in practice. The full codebase for this walkthrough is available in the DefconQ GitHub repository, for anyone who wants to explore, run, or extend it themselves.

Gateway

At its core, the gateway maintains two tables, one for tracking available backend services and another for tracking incoming user queries, along with a small set of functions that coordinate communication between the client, the gateway, and the load balancer. In the following sections, we will break these pieces down and explain how they work together.

The services Table

The services table keeps track of all backend services currently available to the gateway. It is implemented as a keyed table, with the backend service address as the key. For each entry, the table stores the service name as well as the corresponding connection handle. Note that this handle represents the IPC connection established from the gateway to the backend service, not the client connection.

services:([address:()] serviceName:();sh:());

The userQueries Table

The userQueries table records every query that enters the gateway. This is also a keyed table, using a unique sequence number as the key. The sequence number is a simple counter that increments with each new incoming query. Alongside it, the table captures the user handle that submitted the query, the timestamp when the query was recorded, the timestamp when it was forwarded to a backend service, the timestamp when the result was returned, the user name, the service handle and service name associated with the execution, and finally the query itself. Together, this table provides full visibility into the lifecycle of each request as it flows through the gateway.

userQueries:([sq:`int$()] uh:`int$(); rec:`timestamp$(); snt:`timestamp$(); ret:`timestamp$(); user:`$(); sh:`int$(); serv:`$(); query:());

The connectToLoadbalancer Function

The connectToLoadbalancer function is responsible for establishing the IPC connection from the gateway to the load balancer. It opens the connection and stores both the original handle, a positive handle used for synchronous communication, and its negative counterpart for asynchronous messaging, which is how most interactions will be handled. If the connection attempt fails, the function simply reports an error.

connectToLoadbalancer:{@[{NLB::neg LB::hopen x};`:localhost:1234;{show x}]};

The registerGatewayWithLoadbalancer Function

The registerGatewayWithLoadbalancer function then performs the initial handshake with the load balancer. It sends a synchronous message invoking the registerGW function on the load balancer. While we will explain the inner workings of the load balancer later, at a high level this call registers the gateway and returns the full set of backend services currently known to the load balancer. The gateway uses this response to populate its internal services table via the addService function. This exchange is deliberately synchronous: just like a real-time subscriber registering with a tickerplant, the gateway must wait for this response before continuing its startup sequence.

If you’d like to go deeper into IPC in KDB/Q or the mechanics of the tickerplant, check out the dedicated posts linked in the resources section at the end of this article.

registerGatewayWithLoadbalancer:{addService LB(`registerGW;`)};

The addService Function

The addService function updates the gateway’s internal service table with the details of all available backend services. As part of this process, it establishes a direct IPC connection from the gateway to each backend service and then upserts the corresponding entries into the services table with the relevant connection and service metadata.

addService:{ `services upsert `address xkey update sh:{hopen first x}each address from x };

The getData Function

The getData function is the main entry point into our KDB/Q system, the front door to the data warehouse, the “open sesame” to whatever insights are hiding behind your data points. It’s also the first place where the gateway logic starts to get a little more interesting, so it’s worth walking through it line by line.

We begin with a simple conditional check that splits execution into two paths. First, we verify whether the service the user is trying to query actually exists. If it does, the gateway logs the query along with all its metadata, increments the global sequence number, and then asks the load balancer to allocate the next available backend service capable of handling the request. From there, the query continues its journey through the system. In the (hopefully rare) case where the requested service does not exist, the gateway immediately returns a clear error message to the client, and that’s where this particular query’s very short life comes to an end.

Remember, the getData function expects a tuple, a two-element list, where the first element specifies the name of the service to query, and the second element contains the actual query the user wants to execute.

getData:{
  $[(serv:x 0) in exec distinct serviceName from services;
    [userQueries,:(SEQ+:1;.z.w;.z.p;0Np;0Np;.z.u;0N;serv;x 1);
      NLB(`requestService;SEQ;serv)];
    (neg .z.w)(`$"Service Unavailable")] };

The serviceAlloc Function

The next function we look at is serviceAlloc. This function is called by the load balancer once it has identified an available backend service and is returning its address to the gateway. At this point, the gateway once again uses a conditional check to determine whether the user who originally submitted the query is still around and waiting for a response, after all, client processes can disappear for any number of reasons.

To make this determination, the gateway looks up the query in the userQueries table, which tracks all active requests. This works because the sequence number assigned to the query was passed to the load balancer when the gateway initially requested a free service, and is now returned alongside the allocation. That sequence number lets the gateway reliably match the allocation back to the correct query.

If the user is no longer waiting, there’s nothing left to do. The gateway simply releases the backend service by calling releaseService on the load balancer, freeing it up for other work. If the user is still waiting, the gateway sends an asynchronous message to the allocated backend service, invoking the queryService function to kick off execution. Finally, the gateway updates the query’s metadata in the userQueries table and continues with the normal execution flow.

serviceAlloc:{[sq;addr]
  $[null userQueries[sq;`uh];
    NLB(`releaseService;sq);
    [(neg sh:services[addr;`sh]) (`queryService;(sq;userQueries[sq;`query]));
    userQueries[sq;`snt`sh]:(.z.p;sh)]]};

The returnRes Function

Next up is the function that closes the loop and delivers the result back to the client. The returnRes function is called by the backend service once it has finished executing the user’s query. It receives a tuple, a two element list, where the first element is the sequence number of the query and the second element is the actual result containing the requested data.

Using the sequence number, the gateway looks up the corresponding entry in the userQueries table and retrieves the handle of the waiting client. If the client is still connected, the gateway sends the result back to the user. As a final step, it updates the timestamp marking when the response was returned. With that, the lifecycle of this query comes to a close, and the gateway is ready to handle the next request.

returnRes:{[res]
  uh:first exec uh from userQueries where sq=(res 0);
  if[not null uh;(neg uh)(res 1)];
  userQueries[(res 0);`ret]:.z.p };

The .z.pc Function

One important edge case to cover is what happens when a connection to the gateway is dropped. We handle this by overriding the default behaviour of the internal system function .z.pc with gateway-specific logic. This allows us to clean up properly and ensure no KDB/Q process is left hanging indefinitely.

The cleanup happens in stages. First, we null out the user handle for all entries in the userQueries table where the handle matches the disconnected client. Next, we remove any backend services whose service handle corresponds to the dropped connection. Rather than branching with conditional checks, we simply delete matching rows, if there’s no match, nothing happens.

If the disconnected handle belongs to a backend service, we then identify all affected, still-waiting users and return an appropriate “Service Disconnected” error message. Finally, we check whether the dropped connection was the load balancer itself. If so, the gateway broadcasts an error to all clients with outstanding queries, closes all backend service connections (since no further allocations are possible), clears the services table, and updates the sent and return timestamps for all in-flight queries.

As a last step, the gateway resets its load balancer state and arms a timer with a 10-second interval, attempting to re-establish the connection to the load balancer every ten seconds until it’s back online.

.z.pc:{[handle]
  update uh:0N from `userQueries where uh=handle;
  delete from `services where sh=handle;
  if[count sq:exec distinct sq from userQueries where sh=handle,null ret;
    returnRes'[sq cross `$"Service Disconnect"]];
  if[handle~LB;
    (neg exec uh from userQueries where not null uh,null snt)@\:`$"Service Unavailable";
    hclose each (0!services)`sh;
    delete from `services;
    update snt:.z.p,ret:.z.p from `userQueries where not null uh,null snt;
    LB::0; NLB::0; value"\\t 10000"] };

The .z.ts Function

Last but definitely not least, we wire up the function that should fire whenever the internal timer ticks by assigning it to the system hook .z.ts. On each trigger, the gateway attempts to open a connection to the load balancer and, if successful, proceeds to register itself by calling registerGatewayWithLoadbalancer.

To keep things robust, the whole sequence is wrapped in a trap, f anything goes wrong, we catch it and return a meaningful error instead of letting the gateway stumble forward in an undefined state. Better safe than sorry. Once the connection is established and the gateway is successfully registered, we shut the timer back down by resetting the interval to zero. At that point, the gateway, the load balancer, and the backend services are all in sync and fully operational.

z.ts:{
  connectToLoadbalancer[];
  if[0<LB;@[registerGatewayWithLoadbalancer;`;{show x}];value"\\t 0"] };

Loadbalancer

Now that the gateway has done its job of funnelling user queries, it’s time to meet the load balancer, the quiet workhorse that decides who actually does the heavy lifting. Think of it as mission control: it keeps a live view of available backend services, queues incoming requests from the gateway, and tracks every gateway currently plugged into the system. Backed by a small set of purpose-built tables and a handful of core functions, the load balancer is responsible for making sure work is assigned efficiently, safely, and at the right time. In the sections that follow, we’ll break down these building blocks and walk through the logic that keeps everything moving.

The services Table

This table tracks all available service instances and the gateways currently consuming them. It is keyed by the backend service handle and records the service address, service name, the gateway handle currently assigned to the service, the sequence number of the query being processed, and the corresponding timestamp.

services:([handle:`int$()] address:`$(); serviceName:`$(); gwHandle:`int$(); sq:`int$(); udt:`timestamp$());

The serviceQueue Table

The serviceQueue table keeps track of all requests waiting for an available resource. It is keyed by the gateway handle the request originated from and the corresponding service request sequence number. In addition, it records the target service name and the timestamp marking when the request reached the load balancer.

serviceQueue:([gwHandle:`int$();sq:`int$()] serviceName:`$(); time:`timestamp$());

The gateways List

The load balancer also keeps a lightweight list of gateway handles, aptly named gateways, which tracks all gateways currently connected to it.

gateways:()

The registerGW Function

To track which gateways are connected to the load balancer, we maintain a simple list of gateway handles. New entries are added via the registerGW unction. When a gateway connects, this function records its handle and immediately responds with the full set of currently available backend services. Armed with this information, the gateway can populate and synchronise its internal service table with the service names and addresses known to the load balancer.

registerGW:{gateways,:.z.w ; select serviceName, address from services};

The registerService Function

Gateways matter, but without backend services there’s nothing to route to. To onboard a newly connected backend, we define the registerService function. It takes the service name and address and upserts the entry into the load balancer’s services table. Since this service is now part of the ecosystem, its details must be broadcast to all connected gateways. We do this using the each-left iterator, invoking addService on every registered gateway handle with the relevant metadata. Once that propagation is complete, the service is marked as available and ready to take work.

registerService:{[name;addr]
  `services upsert (.z.w;addr;name;0N;0N;.z.p);
  (neg gateways)@\:(`addService;enlist`serviceName`address!(name;addr));
  serviceAvailable[.z.w;name] };

The sendService Function

When a gateway asks the load balancer for an available backend, we need a clean way to hand that information back. That’s the job of the sendService function. It selects a free backend service, returns its handle and address to the gateway, and immediately invokes the gateway’s serviceAlloc function so the user query can be forwarded to its final destination.

If you look closely at the implementation, you’ll notice the use of raze when constructing the message sent back to the gateway. At first glance this might seem a little odd, but it’s entirely intentional. Indexing into the services table by handle yields the sequence number and address, which we then combine with the symbolic name of serviceAlloc.

q)services
handle| address   serviceName       gwHandle sq udt
------| ---------------------------------------------------------------------
7     | :mac:2222 EQUITY_MARKET_RDB             2026.02.10D22:13:43.941150000
q)services[7;`sq`address]
0Ni
`:mac:2222
q)(`serviceAlloc;services[7;`sq`address])
`serviceAlloc
(0Ni;`:mac:2222)
q)0N!(`serviceAlloc;services[7;`sq`address])
(`serviceAlloc;(0Ni;`:mac:2222))
`serviceAlloc
(0Ni;`:mac:2222)

Without raze, this produces a two-element list: the function name and a tuple containing the arguments. That would cause serviceAlloc to be called with a single list argument, which is not what we want. Since serviceAlloc expects two atomic parameters, we flatten the structure with raze, ensuring the function is invoked with the correct argument shape.

q)raze (`serviceAlloc;services[7;`sq`address])
`serviceAlloc
0Ni
`:mac:2222
q)0N!raze (`serviceAlloc;services[7;`sq`address])
(`serviceAlloc;0Ni;`:mac:2222)
`serviceAlloc
0Ni
`:mac:2222

Full implementation:

sendService:{[gw;h]neg[gw]raze(`serviceAlloc;services[h;`sq`address])};

The requestService Function

Before we can dispatch a free backend to a gateway, we first need to receive and process the request. That responsibility sits with the requestService function. It takes two parameters: the sequence number assigned by the gateway and the symbolic name of the backend service that should handle the user’s query.

The first step inside requestService is resource discovery. We query the services table for a row where the service name matches the requested service and the gateway handle column is null. In other words, a service instance that is currently idle and not allocated to another gateway. Availability is not assumed; it is explicitly verified.

If a suitable service is found, we immediately log the request in the serviceQueue table via addQueryToQueue. This gives us traceability and ordering. We then update the services table itself, assigning the backend instance to the requesting gateway, recording the query’s sequence number, and stamping the allocation time. At this point, the resource is officially reserved.

With the bookkeeping done, we send the handle of the allocated backend service to the gateway. From there, the gateway takes over and forwards the client’s query to its final destination, the backend service that will actually execute it.Clean allocation. Clear state.

requestService:{[seq;serv]
  res:exec first handle from services where serviceName=serv,null gwHandle;
  $[null res;
    addQueryToQueue[seq;serv;.z.w];
    [services[res;`gwHandle`sq`udt]:(.z.w;seq;.z.p);
      sendService[.z.w;res]]] };

The addQueryToQueue Function

If an incoming request cannot be processed immediately, it needs to be parked in the serviceQueue table. That’s precisely the role of the addQueryToQueue function. It takes the query’s sequence number, the target service name, and the gateway handle from which the request originated, and inserts this information into the queue table. As part of the process, it also stamps the entry with the current timestamp, ensuring we retain a clear record of when the request entered the load balancer.

addQueryToQueue:{[seq;serv;gw]`serviceQueue upsert (gw;seq;serv;.z.p)};

The releaseService Function

Once a backend service has either completed a client request or the client has disconnected and no longer requires a response, the service instance must be freed and marked as available again. That responsibility falls to the releaseService function.

If releaseService is triggered by a backend service, meaning the query was successfully processed, we simply call serviceAvailable with the relevant service details, marking the instance as idle and ready for new work. If, on the other hand, releaseService is invoked by a gateway, this signals that the client disconnected before the result was returned. In that case, we identify the backend service that was allocated to that query using the gateway handle and the query’s service name, and then call serviceAvailable with those details. Either way, the outcome is the same: the resource is returned to the pool and can be reassigned.

releaseService:{
  serviceAvailable . $[.z.w in (0!services)`handle;
    (.z.w;x);
    value first select handle,serviceName from services
      where gwHandle=.z.w,sq=x ] };

The serviceAvailable Function

The releaseService function delegates some of the workload to the serviceAvailable function. This function determines whether the freed resource should simply be marked as idle or immediately reassigned to the next waiting request. In doing so, it updates both the services and serviceQueue tables to reflect the new state.

The first step is to retrieve all gateway handle and sequence number combinations from the serviceQueue where the service name matches the backend service being released. Thanks to KDB/Q’s left-of-right evaluation efficiency, we can directly extract the first gateway/sequence pair from that result set, effectively identifying the next request in line.

We then remove that entry from the serviceQueue and update the services table with the allocation details. If a queued request exists, the backend service is immediately reassigned, and its handle is sent to the corresponding gateway so processing can continue without delay. If no matching requests remain, the execution flow stops there. This means the service is simply marked as available and waits for the next allocation.

serviceAvailable:{[zw;serv]
  nxt:first n:select gwHandle,sq from serviceQueue where serviceName=serv;
  serviceQueue::(1#n)_ serviceQueue;
  services[zw;`gwHandle`sq`udt]:(nxt`gwHandle;nxt`sq;.z.p);
  if[count n; sendService[nxt`gwHandle;zw]] };

The .z.pc Function

The final piece on the load balancer side mirrors the gateway’s approach: we override the system function .z.pc to handle disconnections cleanly and deterministically.

When a process disconnects, the first step is to remove its handle from the services table. If the disconnected handle does not belong to a backend service, the table remains unchanged, no branching logic required. We then apply the same logic to the list of registered gateways, removing the handle if it belongs to a gateway; otherwise, the list stays as is.

Next, we clean up the serviceQueue by deleting any outstanding requests submitted by the disconnected gateway. If the originating gateway is no longer alive, there is no client to receive the result, making those queued queries obsolete. Finally, we check whether any backend service is currently allocated to the gateway that just dropped. If so, we release that resource by resetting the occupying gateway handle to null. A disconnected gateway cannot hold onto a backend service, and the system must reflect that immediately.

And with that, the load balancer is fully implemented, its responsibilities defined, its edge cases handled. The routing layer is now operational, and we can shift our focus to building a simple backend service to complete the architecture.

Subscribe now

Backend Service

In the next section, we’ll walk through the implementation of a simple backend service. For the purpose of this tutorial, we’ll assume the backend represents an equity real-time database (RDB) exposing two dummy tables: trade and quote. Alongside the data itself, we’ll store the service name and the connection details required to register with the load balancer.

At the core of the backend sits the execQuery function, responsible for executing incoming client queries. However, to prevent a malformed or poorly written query from destabilising the service, we don’t expose execQuery directly. Instead, it is wrapped inside queryService, where we use a trap to safely execute the request and contain potential errors.

Let’s break down each of these components in detail.

The connectToLoadbalancer Function

The first function we’ll examine is connectToLoadbalancer. Its purpose is straightforward: establish a connection between the backend service and the load balancer. The load balancer’s host and port are hardcoded within the function and used to initiate the connection.

Once connected, the backend stores both the positive handle (for synchronous communication) and the negative handle (for asynchronous calls). To ensure robustness, the entire connection attempt is wrapped in a trap, so any failure results in a controlled error rather than an unhandled exception.

connectToLoadbalancer:{@[{NLB::neg LB::hopen x}; `:localhost:1234; {show "Can't connect to Load Balancer-> ",x}]};

The serviceDetails List

The serviceDetails list acts as a parse tree that is sent to the load balancer immediately after a connection has been established. Its first element is the symbolic name of the registerService function, followed by the service name and the backend service’s address as arguments. When transmitted, this structure invokes registerService on the load balancer side, effectively registering the backend service and making it known to the system.

serviceDetails:(`registerService; serviceName; `$":" sv string (();.z.h;system"p"));

The execQuery Function

The function that ultimately executes the client request on the backend is execQuery. It takes two parameters. The first is the (negative) handle of the gateway that submitted the query, this handle is used to send the result back once processing is complete. The second parameter is a tuple, a two-element list: the sequence number of the query and the actual client query to be executed. The sequence number allows the gateway to correctly reconcile the response with the original request when it is returned.

The client query itself is executed inside an error trap on the backend, ensuring that any runtime issues are safely contained. Once execution finishes, whether successfully or with an error, the result is sent back to the gateway by invoking returnRes, which will then forward the response to the client.

execQuery:{[nh;rq] nh(`returnRes;(rq 0;@[value;rq 1;{x}]));nh[]};

The queryService Function

Before reaching its final destination in execQuery, he request is first routed through queryService. This is the function invoked by the gateway, and it receives a tuple containing the query’s sequence number and the actual client query.

Rather than calling execQuery directly, we first construct a projection that wraps the execution logic with proper error handling. This ensures that any exception raised during execution is safely captured and controlled. We then invokeexecQuery and allow the query lifecycle to continue.

Regardless of whether the query completes successfully or fails with an error, the final step inside queryService is to release the backend service. Once execution finishes, the service is no longer blocked and can be returned to the pool for the next request.

queryService:{
  errProj:{[nh;sq;er]nh(sq;`$er);nh[]};
  @[execQuery[neg .z.w];x;errProj[neg .z.w;x 0]];
  NLB(`releaseService;serviceName) };

Backend Bootstrapping and Bureaucracy: Wiring the Service to the Grid

With the core logic in place, all that remains is a bit of framework housekeeping.

We start by defining the .z.ts function: the routine executed on each timer tick. Inside it, the backend attempts to establish a connection to the load balancer via connectToLoadBalancer. If the connection succeeds, the service registers itself by sending the serviceDetails parse tree to the load balancer. Once registration is complete and the backend is officially online, we reset the timer to zero, no further reconnection attempts are needed.

For resilience, we also override .z.pc, the handler triggered whenever a process disconnects from the backend. If the disconnected process happens to be the load balancer, we react by resetting the timer to a ten-second interval and repeatedly attempt to reconnect. This ensures the backend can recover automatically from infrastructure hiccups.

Finally, we initialise some dummy trade and quote data to simulate our equity RDB and invoke .z.ts to kick everything off. With that, the backend service is live, registered, and ready to handle incoming client queries.

.z.ts:{connectToLoadbalancer[];if[0<LB;@[NLB;serviceDetails;{show x}];value"\\t 0"]};

z.pc:{[handle]if[handle~LB;LB::0;value"\\t 10000"]};

quote:([] date:10#.z.D-1; sym:10#`APPL; time:09:30t+00:30t*til 10; bid:100.+0.01*til 10; ask:101.+0.01*til 10)
trade:([] date:10#.z.D-1; sym:10#`AAPL; time:09:30t+00:30t*til 10; price:100.+0.01*til 10; size:10#100)

.z.ts[]

Client Onboarding: Injecting Queries into the Machine

With the gateway, load balancer, and backend service all up and running, it’s time to simulate a client and push some queries through the framework.

We spin up a fresh KDB/Q process and define a helper function that allows us to submit queries to the gateway. More specifically, we create a projection that accepts two inputs: the target service name and the query to be executed. The projection establishes a connection to the gateway using the supplied connection details, then constructs a lambda that invokes the getData function asynchronously. After sending the request, it flushes the output queue to ensure the message is dispatched immediately.

This implements a deferred synchronous pattern: the client blocks while waiting for the result, but the gateway does not. That distinction is critical. The gateway must remain free to handle other incoming requests, because a gateway that blocks on the first client query isn’t much of a gateway at all.

q)gw:{h:hopen x;{(neg x)(`getData;y);x[]}[h]}[`:localhost:5555]
q)gw(`EQUITY_MARKET_RDB;"select from trade")
date       sym  time         price  size
----------------------------------------
2026.02.14 AAPL 09:30:00.000 100    100
2026.02.14 AAPL 10:00:00.000 100.01 100
2026.02.14 AAPL 10:30:00.000 100.02 100
2026.02.14 AAPL 11:00:00.000 100.03 100
2026.02.14 AAPL 11:30:00.000 100.04 100
2026.02.14 AAPL 12:00:00.000 100.05 100
2026.02.14 AAPL 12:30:00.000 100.06 100
2026.02.14 AAPL 13:00:00.000 100.07 100
2026.02.14 AAPL 13:30:00.000 100.08 100
2026.02.14 AAPL 14:00:00.000 100.09 100

Lights On: Watching the Framework Come Alive

With the line-by-line walkthrough behind us, it’s time to see the architecture in motion. Let’s run a short demonstration of the system we’ve just built and watch the components interact in real time. The illustration below highlights the full setup and how the pieces fit together. If you’d like to recreate the framework yourself, you can find the complete codebase on DefconQ’s GitHub repository.

Where Do We Go From Here? Scaling the Blueprint

This tutorial walked through a lightweight gateway-load balancer framework, breaking down the functionality and code step by step. By now, you should have a solid set of foundational building blocks, enough to evolve this simple architecture into something far more powerful.

A natural next step would be to replace the deferred synchronous pattern with the deferred response pattern introduced in KDB/Q v3.6. From there, you could expand the ecosystem: add additional backend services, refine the load balancer’s routing logic, introduce user-level permissions, or deploy multiple gateways segmented by region or asset class. The framework is intentionally minimal, the ceiling is defined only by your imagination.

As always, happy coding.

Resources

• Gateways: The Bouncer at the Front Door of Your KDB/Q Stack

• Advanced KDB/Q Architecture

• Fundamentals of Interprocess Communication (IPC)

• Beyond the Fundamentals: Next Level Interprocess Communications

• KDB Tick Explained: A Walkthrough

• KX Whitepaper: Query Routing: A kdb+ framework for a scalable, load balanced system by Kevin Holsgrove

“Hey, where do I find the options data?”
“Why did my query lock up the HDB?”
“I swear I didn’t break anything...”

If you have ever been on the receiving end of questions like these, or if your users have occasionally accidentally brought your system to its knees, you are living proof that a missing (or poorly implemented) gateway is a costly architectural gap.

In this post, we will answer the big question:

What is a Gateway, and why is it such a critical component in modern KDB/Q architecture?

We will break down best practices, real design options, and even extend the classic ideas with modern approaches like async routing, caching, security, and observability.

What Is a Gateway?

The way I like to think about a gateway is as the single point of entry for all clients interacting with your KDB/Q system. But it’s not just an access point, it’s much more than that. A gateway acts as a protective layer for your stack: it shields your system from poorly formed or expensive queries, verifies who is allowed to query what, blocks requests when necessary, and routes valid queries to the correct endpoint.

Just as importantly, a gateway removes a whole set of responsibilities from your users. They no longer need to

• Know where the data lives (RDB vs HDB vs other services)

• Understand how to combine results from multiple sources, or

• Send several requests and stitch the responses together themselves.

In other words, the gateway is your bouncer at the front door of the KDB/Q stack, checking IDs, enforcing the rules, and making sure only the right requests make it inside.

Clients make one call to the gateway, and magic happens behind the scenes:

• Data is retrieved from the correct services

• Requests are load balanced if needed

• Security and permissions are enforced

• A clean, unified result is returned

This separation is powerful. It decouples client logic from system internals, which means you can evolve your architecture without constantly breaking downstream users. You can move data, split databases, add replicas, or introduce new services, and the client never needs to know. From their perspective, the gateway is the system.

Gateway Roles and Responsibilities

Thinking of a gateway purely as an access point is selling it short. Yes, it is the single entry point for all clients interacting with your KDB/Q stack, but more importantly, it’s the place where intent is translated into action. Clients express what they want, not how or where to get it. The gateway takes responsibility for figuring out the rest.

1. Access Interface

The access interface defines how clients interact with your KDB/Q system, and predictability here is everything. A gateway should expose a small, well-defined set of entry points that clients can rely on, whether that’s through KDB/Q function calls, stored procedures, or endpoints for non-q consumers. By enforcing a consistent request structure, the gateway reduces ambiguity, simplifies client implementations, and prevents users from crafting ad-hoc queries that bypass system rules. A clear access interface also makes it easier to validate inputs, apply permissions, version APIs, and evolve the backend without breaking consumers. In short, the gateway turns an otherwise flexible and potentially dangerous system into something stable, consumable, and safe to use.

2. Authentication, Authorization, and Entitlements

Security in KDB/Q systems often starts, and ideally ends, at the gateway. Before talking to any service, it verifies:

• User identity

• What they are allowed to request

• Whether the operation is permitted under current policy

This is often done with functions like .z.pw, .z.pg, or .z.ps to gate specific tables, symbols, or services without exposing internals. Rather than spreading entitlement logic across multiple processes, the gateway centralizes access control in one place. This includes verifying who the user is, what they are allowed to see, and which operations they are permitted to perform. From a design perspective, this dramatically simplifies the rest of the stack. Downstream services can assume that incoming requests are already authorized and well-formed. From an operational perspective, it gives you a single point to audit access, apply changes, and reason about who touched what data and when.

In larger systems, however, avoiding bottlenecks and single points of failure becomes critical. Following the one process, one taskprinciple, it can make sense to introduce a dedicated, centralized entitlements service that gateways can query as needed. This keeps authorization logic decoupled, easier to maintain, and simpler to evolve over time. As with most architectural decisions, there’s no one-size-fits-all solution, the right approach depends on scale, complexity, and operational requirements.

3. Routing Queries to the Right Endpoint

In a non-trivial KDB/Q system, data rarely lives in just one place. Recent data may sit in an RDB, historical data in one or more HDBs, and derived analytics in separate services altogether. Expecting clients to understand this layout is both unrealistic and dangerous. The gateway abstracts this complexity away. Based on query parameters, such as date ranges, instruments or asset classes, it decides which backend service (or combination of services) should handle the request. The routing logic can be as simple or as sophisticated as your system requires, but the key point remains the same: clients should never need to care.

4. Load Balancing and Backpressure Management

As usage grows, the gateway also becomes a strategic control point for managing load. Whether you’re serving more users, running heavier analytics, or supporting extended trading hours, a single process handling all requests will eventually become a bottleneck. Load balancing is how you scale horizontally without rewriting your entire system. In a KDB/Q context, load balancing typically means running multiple instances of the same service, most commonly HDBs, analytics services, or even gateways themselves, and distributing incoming queries across them. Because KDB/Q processes are lightweight and fast to start, spinning up additional instances is often far easier than vertically scaling a single, overworked process. A load balancer can track process health, query latency, or simple round-robin allocation to ensure work is spread evenly and no single process is overwhelmed.

A well-designed gateway architecture doesn’t just route queries; it distributes them intelligently. Load balancing ensures that requests are spread evenly across available resources, be that multiple RDBs, HDBs, or downstream analytic services, so no single process becomes a bottleneck. There are a few approaches you can take:

• Round-robin: The simplest method, each request goes to the next available process in line. Great for evenly distributed workloads.

• Least-loaded: The gateway keeps an eye on real-time processes and routes queries to the node with the most capacity.

• Data-locality aware: Particularly in mult-database environments, the gateway can send queries to the node closest to the data, minimizing network latency and improving throughput.

But load balancing isn’t just about performance, it’s also about resilience. If one node becomes unresponsive, the gateway can detect the failure, reroute queries, and keep the system alive without users even noticing. This combination of performance optimization and graceful degradation turns your gateway from a mere traffic cop into a full-fledged air traffic controller, keeping everything flowing safely and efficiently.

Much like entitlements, load balancing can be handled in two ways: either as a dedicated, standalone service, or by embedding the load-balancing logic directly within the gateway itself.

And remember: a gateway without load balancing is like a London pub with only one bartender on a Friday night, it might technically work, but no one’s going to have a good time.

5. Data Aggregation

Sometimes a single request needs data from more than one place, for example, combining historical data from the HDB with live updates from the RDB. In these cases, the gateway acts as the orchestrator: it fans out the request to the relevant backends, aggregates and reconciles the results, and composes a single, consistent response before returning it to the client. This keeps the client interface simple while hiding the underlying complexity of the data landscape.

Gateway Design Options

As usage increases, the gateway evolves beyond a simple entry point into your KDB/Q stack and starts acting as a coordinator, opening up a range of different architectural and design choices. It is worth keeping in mind that there’s no single “right” way to design a gateway. Different problems call for different solutions, so the shape of your gateway should always be driven by the actual needs of your system. Before settling on a design, it’s useful to step back and ask a few fundamental questions:

• What query volume does the gateway need to sustain?

• How many clients connect, and are they humans, automated services, or a mix of both?

• How many downstream processes does the gateway orchestrate (RDBs, HDBs, CEP engines, etc.)?

• Are those processes distributed across multiple servers or even multiple regions?

• Will there be a single central gateway, or several gateways working in parallel?

• What are the requirements around resilience and business continuity?

• How are queries expressed and executed?

• How are results returned? Does the gateway need to merge or aggregate data before responding?

Answering these upfront will naturally guide you toward a gateway design that fits your use case, rather than forcing your use case to fit the design. In the next section, we will look at two different approaches outlined in KX’s “Common Design Principles for kdb+ Gateways, and walk through the benefits and trade-offs of each.

Pass-Through Gateway

This is the simplest pattern. When the gateway gets a request it passes it straight to a dedicated load balancer. That balancer then picks the least busy backend service and returns the result, back through itself, to the gateway. Think of it like a traffic cop taking the request, deciding the best lane, and handing you back the response.

Pros

• Simple logic.

• Backend choice is based on load.

Cons

• Extra hop adds overhead.

• Every response flows back through the balancer (extra IPC).

When this works

• Very simple systems

• Use of External load balancers (L4/L7)

• You don’t care about extra IPC hops

Why it often hurts

• Extra latency (double IPC)

• Load balancer becomes hot very quickly

• Harder to introspect query lifecycle from the gateway

In KDB/Q terms: it works, but it’s rarely elegant.

Connection-Manager Load Balancer (Recommended)

This approach splits responsibility a bit more smartly. The gateway asks the load balancer: “Give me the address of a service that can run this query.” The Loadbalancer responds with a host:port, and then the gateway talks directly to that service for both request and result. You can implement this two ways:

• Round-Robin: simple rotation through services.

• Availability-Aware: only return a service if it’s currently free, otherwise queue the request until one becomes free.

The latter is more work, but it’s way better at keeping services busy and minimizing unnecessary queueing. These more advanced gateways go one step further by actively maintaining a service registry and continuously tracking the state of the underlying services. Instead of blindly rotating requests, the gateway makes informed routing decisions based on factors such as service availability, current system load, and the expected complexity of the incoming query. A lightweight lookup determines which process is free, which one is already under pressure, and which is best suited to handle the work at hand. This moves the system well beyond simple round-robin scheduling and results in lower tail latencies, better overall throughput, and significantly improved resource utilisation across the entire KDB/Q stack.

Key Takeaways

PatternIPC OverheadResource UtilizationComplexityPass-ThroughHighModerateLowConnection Manager (Smart)LowHighMedium-High

For smaller deployments or proof-of-concepts, simple round-robin might suffice. But at real scale — where queries run concurrently and durations vary, having the load balancer actually track service state and manage queues is worth the engineering investment.

Beyond the Basics: Key Considerations for Advanced Gateway Design

As your platform scales and the architecture becomes more sophisticated, the gateway naturally evolves with it. What started as a simple entry point now takes on more responsibility and nuance. In this section, we will look beyond the basics and explore a set of additional considerations that come into play when designing more advanced gateways, and how to approach them in practice.

Synchronous vs. Asynchronous Communication in Gateway Design

When you start building a gateway, one of the first architectural decisions you will run into is how processes talk to each other, and this isn’t just academic. The way components communicate directly shapes throughput, latency, and whether your gateway will become a scalable coordinator or a bottleneck.

Synchronous Communication: Simple but Blocking

In a synchronous world, every call waits for a reply. The gateway asks a service for data, and blocks until it gets a response. It’s straightforward: send a query, wait, get data, repeat. That simplicity has its charm, it makes reasoning about behavior easy, and for very small systems or simple tooling it “just works” without much code overhead.

But a gateway that blocks on every request doesn’t really make sense, or does it? Imagine hundreds of users firing off thousands of queries throughout the day. If the gateway had to wait for each request to fully complete before moving on to the next, everything would grind to a halt. Throughput would collapse, queues would explode, and response times would stretch into the absurd. In that kind of environment, a blocking gateway isn’t just inefficient, it’s simply not an option.

Asynchronous Communication: Parallel by Defaut

Enter asynchronous communication, the game-changer for gateways at scale. With async, the gateway sends a request to a downstream service and doesn’t wait around. Instead of blocking, it continues processing, accepting other client queries, sending more requests, and tracking outstanding work. When results come back, the gateway collects them and finalises the response. This pattern dramatically increases throughput: the gateway isn’t idle while waiting on I/O, and you can overlap slow operations with useful work. Want to query both RDB and HDB at the same time and then merge the results? With asynchronous messaging, that’s the norm rather than the exception.

Deferred Responses: Best of Both Worlds

DefconQ readers should note that many real deployments use a hybrid approach: clients make synchronous requests to the gateway (since they expect a result), but the gateway uses asynchronous calls internally to backends. This lets the gateway juggle work efficiently while preserving a request–response model at the API boundary, synchronous from the client’s perspective but asynchronous under the hood.

The Bigger Picture

Choosing between synchronous and asynchronous isn’t just about RPC (Remote Procedure Call) semantics, it’s about keeping the gateway alive and responsive under load. If your gateway is blocked waiting for every downstream service, you have a serialization point that kills parallelism; if you embrace deferred/asynchronous messaging internally, you unlock far more scalability with relatively modest added complexity.

TIP: If you would like to read more about Interprocess Communication (IPC) in KDB/Q, I have written two dedicated blog posts that explore the topic in more detail.
Fundamentals of Interprocess Communication (IPC)
Beyond the Fundamentals: Next Level Interprocess Communications

Resilience and Fault Tolerance

A production-grade gateway shouldn’t fall apart just because a single downstream component has a bad day. In real systems, processes crash, networks hiccup, and services get overloaded, and the gateway is often the first line of defence against all of that. Instead of blindly forwarding requests, a resilient gateway actively monitors the health of its dependencies and adapts its behaviour when things go wrong. If a primary HDB becomes unavailable, the gateway can transparently route requests to a redundant replica. If an RDB is under heavy load, it may choose to return a degraded but still useful response rather than letting queries pile up and time out. And when no viable option exists, the gateway should fail gracefully, returning clear, informative errors instead of leaving users guessing. This is why robust gateways are typically paired with health checks, heartbeats, and well-defined fallback strategies, not just to keep the system running, but to keep it predictable and user-friendly under failure conditions.

Caching in Gateways

One of the most underappreciated, and often overlooked, enhancements in gateway design is smart caching. A well-placed cache allows the gateway to access or store the results of frequently requested queries and serve them directly without hitting downstream services every single time. When combined with sensible invalidation strategies, such as time-to-live (TTL) expiries or event-driven invalidation on data updates, caching can dramatically reduce unnecessary load on RDBs and HDBs. This is especially powerful for predictable or repetitive queries, dashboards, and reference-style lookups where freshness requirements are well understood. Importantly, caching is not about replacing your databases or hiding bad query patterns, it’s about complementing them. Used correctly, gateway-level caching can transform overall system performance, smooth out traffic spikes, and deliver consistently fast response times to end users.

Observability: Because You Can’t Improve What You Don’t Measure

A gateway should never be a black box. As the central coordination point of your KDB/Q stack, it has a unique view of how the system is actually being used, and that insight is far too valuable to ignore. A well-designed gateway exposes clear, actionable metrics such as request volume, latency distributions, error rates, service-level timeouts, and cache hit or miss ratios. These signals allow you to understand not just whether the system is up, but how it behaves under real load. When fed into dashboards and alerting systems, this data becomes the foundation for capacity planning, performance tuning, and rapid incident response. Without observability, you’re flying blind; with it, the gateway becomes one of your most powerful tools for continuously improving reliability and performance.

Aggregating and Normalizing Results

Even when data is distributed across multiple backends, users almost always expect a single, coherent answer. The gateway is the natural place to bring those pieces together. Rather than pushing complexity onto clients, the gateway can aggregate partial responses from different services, normalize schemas, and present a consistent, well-defined output format. This keeps client code simple and decoupled from the underlying data topology. As systems evolve, this aggregation layer becomes increasingly important: new data sources can be introduced, storage formats can change, and legacy services can be retired without breaking consumers. By absorbing that change and enforcing a stable contract at the boundary, the gateway provides continuity over time, and that stability often becomes one of the most valuable and underappreciated features of a well-designed gateway.

Final Thoughts: A Gateway Is More Than a Middleman

A well-designed gateway is not just a thin layer sitting between clients and data, it is the backbone of a responsive, secure, scalable, and maintainable KDB/Q ecosystem. It brings together data access, service health, security, performance, observability, and high-concurrency handling into a single, cohesive control plane. Whether your system serves ten users or ten thousand, the gateway ensures that clients can express what they need in a clean, consistent way, while backend services focus on doing their job efficiently and safely. If you’re building or scaling a real production system, putting a thoughtfully designed gateway in front of it isn’t merely a best practice, it’s a prerequisite for long-term success.

In our next blog post, we will look into a practical, hands-on implementation of a gateway. Until then, stay tuned, and happy coding.

BONUS: Resources

• KX Whitepaper: Common design principles for kdb+ gateways by Michael McClintock

• Mastering kdb+ Architecture: A Panel Discussion

After more than two years of creating content to help developers get started, level up, deepen their understanding or grow their skills in KDB/Q, and after receiving positive feedback and requests for something beyond blog-based, more structured support, it feels like the right time to take DefconQ one step further. This next chapter will introduce a set of new offerings designed for those who want to go deeper, move faster, and build with confidence, including services for ...

... Students

I would like to support a small number of motivated students each year by mentoring them through a project, dissertation, or other academic coursework. This would be open globally to two selected students (free of charge) who are genuinely curious, hardworking, and eager to get a head start by learning KDB/Q before graduation.

The project would need to be part of your academic requirements and aligned with your university’s guidelines. Depending on the setup, a discussion with a supervising professor or academic staff member may be required to make sure everything fits within the formal framework, but those details can be explored together.

... Universities

I am very happy to collaborate with universities in a variety of ways. This could range from guest lectures and industry talks, to hands-on masterclasses or workshops in KDB/Q, and depending on location and scope, even a full course offering.

The audience can come from disciplines such as Computer Science, Finance, Engineering, Physics, Mathematics, or Quantitative Finance, though it’s by no means limited to these. If this sounds interesting, feel free to reach out for an initial, informal conversation.

... Professionals

For professionals, I’m offering two structured, paid mentorship programs, depending on where you are in your journey.

Career Mentorship

This program is focused purely on career progression. It includes our one-hour one-to-one sessions where we will talk through your goals, the role you’re aiming for, and how to position yourself to get there. This could involve an internal move, a transition from sell-side to buy-side, a shift toward a more business-facing role, or any other direction you’d like to explore. This track does not include technical coaching, it’s all about clarity, strategy, and next steps.

Technical Mentorship

This program is designed for anyone looking to deepen their KDB/Q skills or transition into a KDB/Q role. Over the past months, I’ve been piloting this format and have helped developers with no prior KDB/Q experience ramp up and successfully move into KDB/Q positions. The program includes:

• An onboarding call to understand your background and goals

• A personalised weekly study plan

• Ten one-hour one-to-one sessions with me (one hour per week)

Everything is tailored to your level and pace.

... Conferences, Meetups, and Podcasts

Over the past years, I have had the opportunity to speak at conferences, meetups, and podcasts such as Quant Strats/Future Alpha, KX events and livestreams, Fancy Quant podcast with Dimitri Bianco, and Future of Finance & AI Conference hosted by Alexander Fleiss of Rebellion Research. I am keen to continue these engagements throughout the year.

If you’re organising a conference, meetup, or podcast in fintech, finance, quantitative finance, or a related space, feel free to reach out and let’s see if there’s a good fit.

... Sponsores

I’m opening up a small number of sponsorship opportunities, including:

DefconQ Happy Hours

In 2025, I hosted DefconQ Happy Hours in London, Belfast, and New York, all of which turned out to be great community events. I am planning to continue these on a quarterly basis in 2026 and am looking for sponsors who’d like to support one of the events, typically by settling the tab.

Sponsored Content

DefconQ’s reach has grown beyond the core KDB/Q community and now spans a broader finance audience. If you’d like to promote your business in a subtle, thoughtful, and occasionally witty way, sponsoring a blog post could be a good option. There’s even a non-zero chance Adam Sandler might make a cameo.

... Everything Else

If you’re interested in engaging with DefconQ but don’t see something that directly matches your situation above, don’t worry. Just drop me a message. I’m always happy to chat and explore ideas together, sometimes the best things come from conversations that weren’t planned in advance.

Why Now

Now is an especially good time to take advantage of the DefconQ Pro offerings. Over the past year, demand for KDB/Q developers has not only increased but reached its highest level in several years. As data volumes continue to grow, the need for engineers who can work efficiently with high-performance time-series systems grows with it. While opportunities exist across all experience levels, there has been a clear preference for mid-to-senior-level developers, those who can design, build, and reason about real systems.

DefconQ Pro is designed to help you close that gap and position yourself exactly where the market demand is strongest.

Why DefconQ

Choosing DefconQ means learning KDB/Q from a practical, real-world perspective. Everything is grounded in hands-on experience, from building full tick stacks and analytics engines to designing scalable architectures used in production environments. The focus isn’t on isolated syntax or academic examples, but on understanding whysystems are built the way they are and how to apply that knowledge with confidence. Based on years of experience not only building systems but also mentoring, upskilling, and teaching developers KDB/Q, DefconQ offers a level of guidance that’s hard to find elsewhere. Ultimately, the results speak for themselves.

Who This is For

DefconQ Pro is designed for highly motivated individuals who are serious about progressing their careers and deepening their KDB/Q expertise. It’s for developers who are willing to put in the work required to succeed in a competitive space and who want to leverage focused guidance to reach their goals faster and more efficiently. If you’re proactive, curious, and committed to improving, DefconQ Pro is built to support you on that journey.

What DefconQ Pro Is Not

DefconQ Pro is not a passive, “attend-only” or click-through program with guaranteed outcomes. There are no shortcuts, you have to do the work and take ownership of your progress. It’s also not a recruiting service or a guaranteed path to a KDB/Q role. While the program can significantly improve your skills, confidence, and interview readiness, success ultimately depends on you showing up, performing, and earning the role yourself.

How To Apply

If you’re ready to commit to this journey and are interested in any of the DefconQ Pro offerings, feel free to get in touch via email defconqpro@gmail.com or LinkedIn message here. As availability is limited, applications are carefully reviewed to ensure a strong fit on both sides. To keep the process efficient, please include your LinkedIn profile or CV when reaching out.

What Changed: More Time, More Data, Same Expectations

Historically, markets had well-defined trading sessions. Systems were designed around them:

• A clear start of day

• A clear end of day

• Overnight batch processes

• Predictable data volumes

• Fixed intraday windows for analytics

Extended trading hours fundamentally challenge these assumptions. From a systems perspective, this means:

• Longer continuous data streams

• Higher total daily data volume

• More intraday state to manage

• Fewer clean reset points

• Increased overlap across regions and venues

And crucially: Latency, throughput, and availability expectations do not relax just because the day is longer.

The Hidden Cost: Volume Isn’t Linear

A common mistake is to assume that extending trading hours simply increases volume linearly. In practice, it often doesn’t. Extended hours introduce:

• Higher market fragmentation

• More algorithmic and automated participation

• Increased event-driven bursts (news, macro, cross-venue reactions)

• Longer periods where real-time analytics must stay hot

That means more ticks, more state, more joins, more aggregations, and more pressure on memory and compute.

Systems that were “just about fast enough” during regular hours may suddenly fall over.

Why Many Databases Will Struggle

Most modern data stacks weren’t designed for continuous, high-frequency, time-series workloads.

1. Row Based Storage

Row-oriented systems are inefficient when queries scan time ranges across a small number of columns, which is exactly what market data workloads do.

2. Disk-First Architectures

Systems that rely heavily on disk I/O, even with caching, struggle when real-time queries and ingestion happen simultaneously over extended periods.

3. Complex Streaming Pipelines

Many stacks glue together:

• A streaming layer

• A storage layer

• A compute layer

• A query layer

Each boundary adds latency, failure modes, and operational complexity.

4. Overnight Assumptions

Batch jobs, rollovers, save-downs, and re-indexing jobs often assume a quiet overnight window. Extended trading hours shrink, or eliminate, that window entirely.

Why KDB Handles This Naturally

This is where KDB/Q stands apart, not because it’s “fast”, but because of how it’s designed.

1. Built for Continuous Time-Series

KDB/Q is not a database that supports time-series. It is a time-series system by design.

• Data is stored in vectors/arrays

• Time is a first-class concept

• Vector operations are second nature

• Queries scale with time ranges, not table size

Longer trading hours simply mean more rows. The access pattern doesn’t change.

2. In-Memory First, Disk as an Optimization

KDB/Q’s architecture cleanly separates:

• Real-time data (RDB), in memory

• Historical data (HDB), memory-mapped on disk

Extended trading hours don’t require redesigning the system. They just change how long the RDB stays hot and how often data is persisted.

Memory-mapped storage ensures:

• No deserialization overhead

• No ORM layers

• No impedance mismatch between memory and disk

3. Streaming Is Native, Not Bolted On

In KDB/Q, streaming isn’t an add-on, it’s the default.

• IPC is lightweight and event-driven

• Subscribers process updates as they arrive

• Backpressure patterns (e.g. chained tickerplants) are well understood

Whether the market trades for 6 hours or 20 hours is largely irrelevant. The system is already designed to run (almost) continuously.

4. Analytics Live Where the Data Lives

This becomes even more important with longer trading days. Instead of:

“Pull all the data out, then analyze it somewhere else”

KDB/Q allows:

“Run the analytics where the data already is”

That means:

• Smaller result sets over the network

• Lower bandwidth pressure

• Faster response times

• More predictable performance

Extended trading hours amplify the cost of moving data. KDB avoids that cost entirely.

Operational Reality: Fewer Moving Parts Win

When markets run longer, operational simplicity matters more than ever. KDB/Q based systems typically replace:

• Streaming frameworks

• Online Analytical Processing (OLAP) engines

• Time-series databases

• Analytics runtimes

... with ONE COHERENT SYSTEM

That matters when:

• Monitoring windows are longer

• Failure recovery must be faster

• Engineers are on-call for extended periods

• Systems must degrade gracefully, not catastrophically

This Is a Trend, Not a One-Off

Nasdaq extending trading hours isn’t an isolated event. Already towards the end of last year, we saw the New York Stock Exchange extend its trading hours (Read my blog post here). Most cryptocurrency exchanges already operate 24/7, and it’s clear that more traditional exchanges will follow this trend. We’re seeing:

• More global overlap

• More electronic liquidity

• More automation

• More demand for always-on systems

Markets are moving closer to continuous operation. Architectures that depend on:

• Quiet periods

• Batch windows

• Heavy ETL

• Complex pipelines ... will increasingly struggle to keep up.

Final Thoughts

Extended trading hours are a stress test for market data architectures. They expose:

• Hidden assumptions

• Scalability limits

• Operational fragility

Systems built with KDB/Q (KDB-X) don’t need to be reinvented for this shift. They were already designed for it. If anything, changes like this highlight why KDB/Q has remained dominant in high-performance market data systems for decades, not because it’s fashionable, but because it solves the right problems at the right level of abstraction. And as markets keep pushing toward longer, faster, and more continuous trading, those design choices matter more than ever.

If you need help designing or architecting your KDB/Q system, feel free to reach out! Happy to chat. Until then, happy coding.

Reference: Reuters: Nasdaq seeks to extend trading hours, as Wall Street gears up for 24/7 move

Here’s what the final dashboard will look like.

Before We Dive In: Gearing Up for the Strategy Build

Since we’ll be reusing parts of the code introduced in earlier tutorials, it’s helpful to familiarise yourself with those foundations, they’re explained in detail and will make this tutorial much easier to follow. Don’t worry, though: all new additions will be fully explained as we go. To visualise our trading strategy and build the dashboard, you’ll also need to install Pulse. Installation instructions are provided below.

Prerequisites:

1. Review the previous tutorials to understand the core building blocks we’ll build on

   • KDB Tick Explained: A Walkthrough [PART 1]

   • KDB Tick Explained: A Walkthrough [PART 2]

   • Back to the Future: Building Your Research HDB for Stock Market Insights

   • The Plain Vanilla Tick Setup

2. Install Pulse: Follow the instructions here

Blueprints Before Code: Designing Your Trading Engine the Right Way

Before we jump into writing any code, it’s worth pausing to look at the most critical part of building any solid application: the architecture. Even a small, simple system benefits enormously from a clean, extensible design. With that mindset, here’s how we’ll structure our trading engine:

• Data Ingestion: A dedicated process responsible for feeding market data into our Tickerplant.

• Tickerplant: The heartbeat of every KDB-X application. It streams incoming data to all real-time subscribers and writes each update to a Tickerplant Log file, ensuring the system can recover seamlessly if needed.

• Complex Event Processing Engine (CEP): This component serves as our trading engine, executing the strategy logic and generating signals based on the incoming data.

Feeding the Beast: Bringing Data Into Your Trading Engine

For this tutorial, we’ll use end-of-day files from Yahoo Finance, the same dataset we worked with in an earlier tutorial. You can reuse those files or download fresh ones if you prefer. We’ll write a function that loads these files into memory and publishes the data to our Tickerplant. And since we also want to build a backtesting engine, we’ll stream the historical data on a timer to simulate a “time-lapse” playback of market history.

Let’s get started.

First, we define the table schema for the data we want to load and gather the filenames of all historical data files into a variable. When following along, remember to update the file path to match the location of your own data. Also make sure that the folder contains only data files, as the key operator will list everything inside that directory.

// Define table schema for data we are going to load
data:([] time:`timestamp$(); sym:`symbol$(); feedHandlerTime: `timestamp$(); open: `float$(); high: `float$(); low: `float$(); close: `float$(); volume: `long$());

// List of files we want to load (Change this path to your file location. Note:folder should only contain your data as everything in the folder will be loaded)
fileList:key `$”:/Users/alexanderunterrainer/repos/defconQ/projects/tradingStrategy/stock_data”;

loadData:{[file]
    // Extract the name of the stock from the file name
    name:`$first “_” vs string file;
    // Load the data for the specific file, update the sym column with the stock name and a feedHandlerTime
    `data insert update sym:name,`timestamp$`date$time,feedHandlerTime:time from `time`open`high`low`close`volume xcol (”PFFFFJ”;enlist csv) 0:hsym file;
 };

After defining our function, we can run it

q)loadData each fileList;

Before we can start streaming any data, we first need to bring up the Tickerplant. For this tutorial, we’re using an even more stripped-down version of the plain vanilla Tickerplant. If you’d like a detailed, line-by-line explanation of how it works, check out my walkthrough linked here.

For now, let’s go ahead and start the Tickerplant.

alexanderunterrainer@Mac:~/repos/defconQ/projects/tradingStrategy|master⚡ ⇒  qx tick.q sym . -p 5010
KDB-X 0.1.1 2025.09.29 Copyright (C) 1993-2025 Kx Systems
q)stocks
time sym feedHandlerTime open high low close volume
---------------------------------------------------
q)

Once the Tickerplant is up and running, we can connect to it using hopen, which creates a connection handle between our feed handler and the Tickerplant. From there, we’re able to publish our data. Since we haven’t started any real-time subscribers yet, the Tickerplant will simply write the updates to its log file without broadcasting them to any subscribers.

q)h:hopen 5010
q)h(`.u.upd;`stocks;data)

Now that our data pipeline is in place, it’s time to build the Complex Event Processing Engine that will execute our trading strategy.

Complex Event Processing Engine (CEP): Effortless Business Logic in Motion

For our trading strategy, we’ll implement a straightforward moving average crossover model. It uses two averages: a short-term 50-day moving average and a long-term 200-day one. The signal rules are simple: when the short average crosses above the long average, we generate a buy signal; when it crosses below, we trigger a sell signal. This captures the classic idea that prices tend to revert toward their longer-term trend over time.

Calculating the moving averages couldn’t be easier, KDB-X gives us the mavg keyword out of the box. This is another perfect example of why it excels at analytical workloads. Using mavg, we can directly update our table with both moving averages. The only detail to watch out for is that our dataset contains multiple symbols, so we need to compute the averages for each symbol. That’s easily achieved using a by clause as you can see in the line below

q)update sma50:mavg[50;close],sma200:mavg[200;close] by sym from `stocks
`stocks

So far so good, we’ve made solid progress. But now it’s time to bring our data to life. Let’s start building our dashboard. As a reminder, this is what the final result will look like:

Pulse: The Catwalk for Your Data Model

For those who know me, I’m much more of a backend person, I usually steer clear of UI and front-end work. But Pulse is so simple and intuitive that even I found myself almost enjoying the UI side of things. Once you’ve installed Pulse (installation guide linked here), you can launch it directly from your terminal. This opens a new tab in your browser where you can begin designing your dashboard. The first step is to set up a connection to the KDB-X process that holds our data.

// Start Pulse from your terminal. This will launch Pulse and open the landing page
alexanderunterrainer@Mac:~|⇒  java -jar pulse.jar

Now let’s create a new connection by specifying the host and port. In our example, the process is running on localhost using port 5001.

With the connection configured, we can move on to creating our dashboard. Go ahead and add a new one to get started.

A new dashboard will open in design mode, and we can start adding components right away. I’m a big believer in the “low-hanging fruit principle”, tackle the easiest wins first. So let’s begin by charting the volume. Before that, though, let’s quickly save and rename our dashboard.

Once that’s done, click the Time Series button in the top panel to add a new time-series chart.

Pulse will generate a placeholder chart with simulated data, which we can now replace by updating the query to display the volume we’re interested in.

// Query for volume
`time xasc select time,volume from stocks

Hold on, something looks off. There are far too many data points, and the volume fluctuations don’t make much sense. That’s because our stocks table contains data for multiple symbols, so when we chart the volume directly, we’re plotting everything at once. The fix is simple: we add a dropdown selector so we can choose the specific stock we want to display.

To do this, add a User Form. Click the User Form button in the top panel and place it above your volume chart. Then, in the Form Editor on the right, add a Dropdown component. We can populate the dropdown with our stock symbols by running a simple KDB-X query. Details below.

asc select distinct sym from stocks

All that’s left is to connect the dropdown to our volume chart so that selecting a symbol updates the chart accordingly. We can do this by referencing the dropdown’s assigned key directly in the chart’s query. The updated query looks like this:

`time xasc select time,volume from stocks where sym=`${{key1}}

Not too bad, right? Since our historical dataset spans several decades, it’s useful to control how much of it we display at once. Let’s add another dropdown to let us choose the time range. Later, when we switch to real-time backtesting, this dropdown won’t be needed anymore and can be removed.

We start by adding another dropdown to choose the date range we want to display. This time, instead of populating the options with a Q-SQL query, we simply hard-code a list of day counts that correspond to 1, 2, 3, 5, 7, 10, and 15 years. We also include a large value to allow selecting the full time series.

Next, we update the volume query to incorporate the dropdown’s key so the chart adjusts based on the selected range. For a small visual upgrade, we switch the chart from a line plot to a bar chart, and just like that, we have a clean, intuitive volume visualization for our stocks.

-{{key2}}#select time,volume from stocks where sym=`${{key1}}

Alright, with the volume chart in place, it’s time to move on to the price chart and start implementing our trading strategy. As before, we’ll tackle the easy part first, plotting the price series along with the moving averages.

Just like we did for the volume chart, we’ll add a new time-series component and define a Q-SQL query that selects the time, close, sma50 (50-day simple moving average), and sma200 (200-day simple moving average). As before, we’ll use the values from our drop-downs to limit the chart to a specific stock and chosen time range.

And just like that, we now have a clean visual of the price data alongside both moving averages. But how do we turn this into trading signals? Our rules are simple:

• buy when the short-term moving average (sma50) crosses above the long-term average (sma200).

• sell when the short-term moving average (sma50) crosses below the long-term average (sma200).

Instead of jumping straight into solving this on our full dataset, let’s first strip the problem down and build the logic step by step. We’ll start with two simple lists: one for the short moving average and one for the long moving average:

• s: 3 7 8 12 10 11 9 7 (short MA)

• l: 5 9 10 11 8 7 12 13 (long MA)

First, we subtract the long moving average l from the short moving average s. This tells us whether the short MA is above or below the long MA at each point.

Next, we apply the built-in signum function to that result. signum returns -1 for negative values, 1 for positive values, and 0when the input is zero. We then compare this list against 0 to get a boolean mask indicating where the short MA is below the long MA. This effectively captures when the relationship between the two averages flips from negative to positive (or the other way around).

Finally, we use deltas on this boolean sequence to generate our trading signals:

• A delta of +1 → we buy

• A delta of -1 → we sell

• A delta of 0 → we do nothing

q)s:3 7 8 12 10 11 9 7
q)l:5 9 10 11 8 7 12 13
q)s-l
-2 -2 -2 1 2 4 -3 -6
q)signum s-l
-1 -1 -1 1 1 1 -1 -1i
q)0<signum s-l
00011100b
q)deltas 0<signum s-l
0 0 0 1 0 0 -1 0i
q)1+deltas 0<signum s-l
1 1 1 2 1 1 0 1
q)`sell`doNothing`buy 1+deltas 0<signum s-l
`doNothing`doNothing`doNothing`buy`doNothing`doNothing`sell`doNothing

The final step is to use the deltas output as an index into a list of symbols, allowing us to map the numeric values directly to their corresponding trading signals.

Now let’s integrate the signal-generation logic into our dashboard so we can produce a chart like the one shown below. It’s not particularly complicated, but it does require a few lines of code. We’ll also take advantage of some built-in Pulse features to display those nice visual marker, bubbles that highlight each buy or sell signal right on the chart.

Pulse is extremely flexible and even lets you write logic directly inside the chart’s code editor. However, to keep things clean and maintainable, we’ll follow good coding practices and wrap our logic in a function that Pulse can call, with parameters controlled by the UI.

We’ll create a function called priceChart that takes two arguments: the stock symbol we want to plot and the time range (in days). We’ll build it step by step. First, we reuse the original query that returns the price and the two moving averages.

select time,sym,sma50:mavg[50;close],sma200:mavg[200;close],price:close from stocks where sym=stock

Then we extend that result with our signal-generation logic, keeping the numeric encoding for now: -1 for sell, 1 for buy, and 0for no action.

update signal:deltas 0<signum sma50-sma200 from
select time,sym,sma50:mavg[50;close],sma200:mavg[200;close],price:close from stocks where sym=stock

Next, we derive a trade action and the corresponding buy/sell price, which for simplicity we set to the closing price at the signal time (ignoring bid/ask). After that, we create the visual “bubbles” that mark buys and sells using Pulse’s built-in conventions: if you add a column named <field>_SD_CIRCLE and define <field>_SD_SIZE, Pulse will draw a bubble of that size at the corresponding point. By adding four columns, buy_SD_CIRCLE, buy_SD_SIZE, sell_SD_CIRCLE, and sell_SD_SIZE, we get clear visual markers for our buy and sell signals.

update buy_SD_CIRCLE:buy*signal,buy_SD_SIZE:10,sell_SD_CIRCLE:abs sell*signal,sell_SD_SIZE:10 from
update action:`sell`doNothing`buy 1+signal,buy:?[signal=1;price;0N],sell:?[-1=signal;price;0n] from
update signal:deltas 0<signum sma50-sma200 from
select time,sym,sma50:mavg[50;close],sma200:mavg[200;close],price:close from stocks where sym=stock

In the final step, we narrow things down to just the columns we care about and apply sublist to restrict the output to the selected date range. Here’s the full function in one piece.

priceChart:{[stock;range]
        sublist[range;] select time,price,sma50,sma200,buy_SD_CIRCLE,buy_SD_SIZE,sell_SD_CIRCLE,sell_SD_SIZE from
        update buy_SD_CIRCLE:buy*signal,buy_SD_SIZE:10,sell_SD_CIRCLE:abs sell*signal,sell_SD_SIZE:10 from
        update action:`sell`doNothing`buy 1+signal,buy:?[signal=1;price;0N],sell:?[-1=signal;price;0n] from
        update signal:deltas 0<signum sma50-sma200 from
        select time,sym,sma50:mavg[50;close],sma200:mavg[200;close],price:close from stocks where sym=stock}

Following the same good coding practices, let’s also create a dedicated function for the volume chart.

volumeChart:{[stock;range] select time, avgVolume:avg volume, volume from sublist[range;] select time,volume from stocks where sym=stock }

As you’ve probably noticed, we also added the average volume to our volume chart. However, because KDB-X evaluates expressions from right to left, we can’t just compute the average volume first and then apply sublist, that would calculate the average over the entire dataset, giving us the wrong value for the selected time window.

To get the correct result, we must first extract the sublist for the chosen date range and then compute the average volume on that subset.

If you’re wondering why we use sublist instead of the take # operator, it comes down to their different behaviors. When you use take # on a list of 5 elements and request 7, take will start cycling through the list again from the beginning to reach the desired length. In contrast, sublist returns only min(n; count list) elements.

This matters when a stock’s price history is shorter than the time range the user selects, sublist prevents accidental cycling and ensures we only return the data that actually exists.

q)7#til 5
0 1 2 3 4 0 1
q)7 sublist til 5
0 1 2 3 4

Now that our functions are in place, the final step is simply adding them into our Pulse dashboard. We just add the function calls to the TimeSeries code editor and pass in the selected stock and time-range values from our dropdown inputs. That’s all it takes, let’s hook everything up.

The last element to complete our dashboard is a summary table positioned at the top. This table presents key information at a glance, buy or sell price, the generated signal, volume, SMA50, SMA200, and the current price. It adds a numerical layer of context to complement the visual cues from our charts, rounding out the trading interface with clear, actionable insight.

If we look closely at our priceChart function, we’ll notice that nearly all the data required for our summary table is already being produced. The only real difference is the set of columns we want to return. So instead of duplicating logic, we’ll refactor the function, rename it to analytics, and introduce a third parameter that specifies which column set we want, whether for the price chart or for the summary table.

By using a functional select, we can make the underlying query far more flexible and dynamic. It’s yet another great example of how expressive and powerful KDB-X can be. You’ll find the refactored version of the code below.

NOTE: You can read everything about functional selects on my dedicated blog post here

analytics:{[stock;range;columns]
         chartColumns:`time`price`sma50`sma200`buy_SD_CIRCLE`buy_SD_SIZE`sell_SD_CIRCLE`sell_SD_SIZE;
         tableColumns:`time`sym`action`buy`sell`price`volume`sma50`sma200;
         c:();
         if[columns=`tab;c:tableColumns];
         if[columns=`chart;c:chartColumns];
         ?[;();0b;{x!x} c]
         sublist[range;] `time xasc update buy_SD_CIRCLE:buy*signal,buy_SD_SIZE:10,sell_SD_CIRCLE:abs sell*signal,sell_SD_SIZE:10 from
         update action:`sell`doNothing`buy 1+signal,buy:?[signal=1;price;0N],sell:?[-1=signal;price;0n] from
         update signal:deltas 0<signum sma50-sma200 from
         select time,sym,sma50:mavg[50;close],sma200:mavg[200;close],price:close,volume from stocks where sym=stock}

Now all that remains is a bit of clean-up to make the dashboard look polished and professional. Rename each component to something meaningful, add some spacing to the left and right of the charts, and tidy up the layout. And just like that, voilà, you’ve got a sleek, production-ready dashboard that supports your traders’ decision-making with clarity and style.

Subscribe now

Backtesting for Alpha: Turning Strategy Into Performance

At the start of this tutorial, I promised we would build a backtesting engine for our trading strategy, and I always deliver. In this final section, we’ll put everything together by creating a simple yet powerful backtester, powered by KDB-X’s built-in timer functionality: .z.ts.

Normally, .z.ts is undefined, but once you assign a function to it, that function is executed automatically every n milliseconds. You control this interval via a system command \t, either when starting your process with -t n or dyhnamically at runtime with \t n.

Let’s look at a small example to understand how it works. We define .z.ts:{show x}, which prints the argument x every time the timer fires. By default, x is the current timestamp. So, when we set the timer to fire every 1000 milliseconds using \t 1000, we get the current time printed once per second. Simple, elegant, and exactly what we’ll use to “time-lapse” our historical data for backtesting. You can reseet the timer by setting it to zero \t 0

q).z.ts:{show x}
q)\t 1000
q)2025.12.07D15:23:23.316105000
2025.12.07D15:23:24.316105000
2025.12.07D15:23:25.316105000
2025.12.07D15:23:26.316105000
2025.12.07D15:23:27.316105000
2025.12.07D15:23:28.316105000
2025.12.07D15:23:29.316105000
2025.12.07D15:23:30.316105000
\t 0

Remember, our original feed handler holds the entire price history for our stocks, about 75,000 records spanning from 1980 all the way to 2025. For the backtesting engine, we want to replay this history as a time-lapse, streaming one day’s worth of data every 100 milliseconds.

q)count data
75863
q)exec `startDate`EndDate!(min;max)@\:time by sym from data
       | startDate                     EndDate
------ | -----------------------------------------------------------
7203.T | 1999.05.06D00:00:00.000000000 2025.05.26D00:00:00.000000000
9988.HK| 2019.11.26D00:00:00.000000000 2025.05.26D00:00:00.000000000
AAPL   | 1980.12.12D00:00:00.000000000 2025.05.23D00:00:00.000000000
AMZN   | 1997.05.15D00:00:00.000000000 2025.05.23D00:00:00.000000000
GOOG   | 2004.08.19D00:00:00.000000000 2025.05.23D00:00:00.000000000
META   | 2012.05.18D00:00:00.000000000 2025.05.23D00:00:00.000000000
MSFT   | 1986.03.13D00:00:00.000000000 2025.05.23D00:00:00.000000000
QQQ    | 1999.03.10D00:00:00.000000000 2025.05.23D00:00:00.000000000
SPY    | 1993.01.29D00:00:00.000000000 2025.05.23D00:00:00.000000000
TSLA   | 2010.06.29D00:00:00.000000000 2025.05.23D00:00:00.000000000
VOW3.DE| 1998.07.22D00:00:00.000000000 2025.05.26D00:00:00.000000000
VTI    | 2001.06.15D00:00:00.000000000 2025.05.23D00:00:00.000000000

The logic is straightforward:

1. Extract the list of all distinct trading dates from our dataset.

2. Initialize a counter to track our current position within that date list.

3. On each timer tick (every 100 ms), publish the data for the next date to the Tickerplant.

4. Increment the counter so the following tick streams the next day in sequence.

Alright, time to bring it all together!

q)dateList:exec distinct `date$time from data
q)dateList
1999.05.06 1999.05.07 1999.05.10 1999.05.11 1999.05.12 1999.05.13 1999.05.14 1999.05.17 1999.05.18 1999.05.19 1999.05.20 1999.05.21 1999.05.24 1999.05.25 1999.05.26 1999.05.27 1999.05.28 1999.05.31 1999.06.01 1999.06.02 1999.06.03 1999.06...
q)(min;max)@\:dateList
1980.12.12 2025.05.26
q)cntr:0
q)select from data where $[`date;time]=dateList 0
time                          sym     feedHandlerTime               open      high      low       close     volume
---------------------------------------------------------------------------------------------------------------------
1999.05.06D00:00:00.000000000 7203.T  1999.05.06D00:00:00.000000000 396.8346  410.4793  392.2863  410.4793  15575000
1999.05.06D00:00:00.000000000 AAPL    1999.05.06D00:00:00.000000000 0.3498498 0.3521977 0.3305959 0.3343524 433148800
1999.05.06D00:00:00.000000000 AMZN    1999.05.06D00:00:00.000000000 3.7       3.775     3.39375   3.434375  365688000
1999.05.06D00:00:00.000000000 MSFT    1999.05.06D00:00:00.000000000 24.73093  24.86503  23.75395  23.88805  74021000
1999.05.06D00:00:00.000000000 QQQ     1999.05.06D00:00:00.000000000 45.50791  45.90524  43.83911  44.26293  15173400
1999.05.06D00:00:00.000000000 SPY     1999.05.06D00:00:00.000000000 84.72099  85.15424  83.42122  84.43543  13507200
1999.05.06D00:00:00.000000000 VOW3.DE 1999.05.06D00:00:00.000000000 12.69323  12.72496  12.17915  12.50283  107812
q)stream:{-1”Streaming data for date:”,string dateList cntr;select from data where $[`date;time]=dateList cntr;cntr::cntr+1}
q)stream[]
Streaming data for date:1999.05.10
q)stream[]
Streaming data for date:1999.05.11
q)stream[]
// Add the actual publishing
stream:{-1”Streaming data for date:”,string dateList cntr;h(`.u.upd;`stocks;select from data where $[`date;time]=dateList cntr);cntr::cntr+1}

Before we begin streaming, let’s reset the CEP by clearing out any previously stored data and create a copy of our Pulse dashboard to preserve the historical version. In the new Backtesting Dashboard, we’ll remove the date-range dropdown and adjust our functions accordingly. With these two changes in place, we’re ready to fire up the backtesting engine. It’s showtime!!!

Video Tutorial

Code repo: Pulse Powered Alpha: Building your First Trading Strategy with KDB-X and Pulse

In this blog post, I’ll break down the ten most common errors you’ll run into, explain what they mean, show examples of how they arise, and walk you through how to avoid them. I will also share some helpful tips and tricks for debugging your code so you can track down issues faster and with confidence.

Strap in, Euston, prepare for lift-off.

type

Let’s start with one of the most intuitive errors you’ll encounter: the type error. As the name implies, this happens when the data type of one or more arguments isn’t what KDB/Q expects. A classic example is attempting to add a number to a symbol, an operation that simply doesn’t make sense.

Because KDB/Q is a dynamic array language where both variables and functions can change at runtime, mismatched types can appear more easily than you expect. When they do, KDB/Q politely (well… tersely) lets you know with a type error.

Some practical examples are:

q)4+`a
‘type
  [0]  4+`a
        ^
q)sum `hello`world
‘type
  [0]  sum `hello`world
       ^

As shown in the examples above, a type error is raised in situations where incompatible data types are used. In the first case, adding a number to a symbol triggers the error; in the second, attempting to sum two symbols results in the same issue, an operation that simply has no meaning in KDB/Q. Even when using other operators, such as amending a list of integers with a symbol via the amend‐at, KDB/Q will rightfully complain with a type error.

q)@[1 2 3;0 1;:;`hello]
‘type
  [0]  @[1 2 3;0 1;:;`hello]
       ^

However, there are cases where you might expect a type error, yet KDB/Q happily carries on without any complaints. A classic example is adding an integer to a float. Since this operation is mathematically valid, KDB/Q simply performs it, no error needed.

q)3+4.0
7f

More surprisingly, concatenating an integer or even a symbol to a string doesn’t trigger a type error either. While this might be unexpected at first, at a second glance it works because in KDB/Q, a string is simply a list of characters, and KDB/Q fully supports mixed lists. By appending a value of a different type to a string, KDB/Q implicitly creates a mixed list. You can easily verify this behaviour with the example below.

q)”Hello World”,4
“H”
“e”
“l”
“l”
“o”
“ “
“W”
“o”
“r”
“l”
“d”
4
q)”Hello World”, `Errors
“H”
“e”
“l”
“l”
“o”
“ “
“W”
“o”
“r”
“l”
“d”
`Errors

By applying the type operator to the results of these concatenations, we can clearly confirm that both outputs are treated as mixed lists (type 0h). In other words, KDB/Q doesn’t reject the operation, it simply adapts the data structure.

length

Next up is the length error, another fairly intuitive one. This error is raised whenever you attempt to perform an operation on two parameters that don’t share the same length. There is one important caveat, though: sometimes KDB/Q can perform an implicit atomic extension to make the operation valid. But before we get ahead of ourselves, let’s walk through a few examples.

// Operation on parameters of same length, no error is thrown
q)1+2
3
q)1 2 3+4 5 6
5 7 9
// Operation on two parameters of different length, a length error is raised
q)1 2+3 4 5
‘length
  [0]  1 2+3 4 5
          ^
q)

So far so good, right? Now let’s talk about the atomic extensions mentioned earlier. When an operation involves one list and one atom, KDB/Q doesn’t immediately complain. Instead, it will automatically extend the atom to match the length of the list and then apply the operation element-wise. This clever behaviour prevents a length error and makes working with arrays both powerful and concise.

q)1 2 3+10
11 12 13

But we’re not done yet, there’s another key detail worth highlighting. Let’s take a step back and remind ourselves what KDB/Q fundamentally is: an array programming language. (f you’d like to explore this in more detail, take a look here). In KDB/Q, tables aren’t magic objects; they are simply flipped column dictionaries (Want to nerd out a bit more? You can read the full explanation here), meaning each column is just a vector. And that leads to an important rule: whenever you operate on a table column, the result must produce vectors of the same length as the table itself. If it doesn’t, you’ll immediately run into a length error. The example below illustrates exactly what happens when this rule is broken.

q)t:([] a:1 2 3;b:`One`Two`Three)
q)show t:([] a:1 2 3;b:`One`Two`Three)
a b
-------
1 One
2 Two
3 Three
q)update c:10 20 from t
‘length
  [0]  update c:10 20 from t
       ^
q)update c:10 20 30 from t
a b     c
----------
1 One   10
2 Two   20
3 Three 30

Hopefully this makes length errors a lot less mysterious and helps you spot them quickly whenever they show up.

rank

The rank error is somewhat like a cousin of the length error, at least that’s how I like to think about it. In KDB/Q the rank of a function refers to the number of arguments it expects. For example, the addition operator + takes exactly two arguments. If you try to pass more (or fewer), Q throws a rank error. In short, your “argument count” doesn’t match what the function requires. Luckily, this one is easy to fix: just adjust the number of parameters you’re passing.

q)+[1;2;3]
‘rank
  [0]  +[1;2;3]
       ^
q)f:{[one;two;three] one+two+three}
q)f[1;2;3]
6
q)f[1;2;3;4]
‘rank
  [0]  f[1;2;3;4]
       ^

domain

Before diving into the next error, it’s worth revisiting what a function actually is: “In mathematics, a function assigns each input value to exactly one output value. The set of valid inputs is called the domain, and the set from which outputs are taken is called the codomain.” (Q for Mortals, Overview). If we provide a value that doesn’t belong to the domain, the function simply can’t operate on it, resulting in a domain error. A simple example is the til operator. Given a positive integer x, it generates all natural numbers from 0 to x-1. But its domain excludes negative integers, so passing one instantly triggers a domain error.

// The argument is in the domain of the function
q)til 5
0 1 2 3 4
// The argument is outside of the domain of the function
q)til -1
‘domain
  [0]  til -1
       ^

mismatch

If you try to join two tables with a simple comma join but their schemas don’t match, KDB/Q will throw a mismatch error. And that’s exactly how I think of it: the schema between the two tables simply doesn’t match.

q)([]a:til 4; b:`one`two`three`four),([]b:til 4;c:`one`two`three`four)
‘mismatch
  [0]  ([]a:til 4; b:`one`two`three`four),([]b:til 4;c:`one`two`three`four)
                                         ^
q)([]a:til 4; b:`one`two`three`four),([]a:til 4;b:`one`two`three`four)
a b
-------
0 one
1 two
2 three
3 four
0 one
1 two
2 three
3 four

from

Sticking with table-related issues for a moment: if you write a poorly structured Q-SQL query, you’ll run into a from error. This isn’t limited to forgetting the from keyword; it appears whenever the query itself is malformed.

q)select t where a=1
‘from
  [0]  select t where a=1
                        ^
q)selec from t where a=1
‘from
  [0]  selec from t where a=1
             ^

A simple way to avoid poorly structured Q-SQL queries is to brush up on the basics. You can start by reading my blog post on Q-SQL here.

upd

The upd error is a classic one you’ll often encounter during Tickerplant (TP) log replays. Although it’s specific to the replay workflow, it highlights a much broader and more fundamental error pattern. So let’s make sure we really understand what’s going on. To understand why this error occurs, we first need to remind ourselves what’s inside a TP log file. As shown below, a TP log is essentially a list of lists.

`upd `trade (0D14:56:01.113310000;`AUDUSD;”S”;1000;96.96)
`upd `trade (0D14:56:01.115310000;`SGDUSD;”S”;5000;95.45)
`upd `trade (0D14:56:01.119310000;`AUDUSD;”B”;1000;95.08)
`upd `trade (0D14:56:01.121310000;`AUDUSD;”B”;1000;95.65)
`upd `trade (0D14:56:01.122310000;`SGDUSD;”B”;5000;98.14)

Each line contains the function to execute (upd), the target table name, and the data being written. Once you understand that structure, the reason behind the upd error becomes much clearer. The upd error isn’t random, it only occurs when the updfunction has not been defined. We can confirm this with a simple example. First, we mimic a typical TP log entry by creating a list where the first element is a function and the remaining elements are its arguments. For simplicity, we’ll use the + operator with two integers. Evaluating this list with value correctly applies the function and returns the result.

q)value (+;2;3)
5

Next, we try the same idea using a TP-style entry. We define a small table with user names and ages, then attempt to insert a record by evaluating a list that calls upd.

q)show t:([] name:`$(); age: `int$())
name age
--------
q)upd[`t;(`alex;38i)]
‘upd
  [0]  upd[`t;(`alex;38i)]
       ^

Since upd is not defined, evaluating the expression triggers the upd error.

This leads us to a more general rule: whenever KDB/Q references a function or variable that doesn’t exist in the current namespace, it throws an error using that missing name. In other words, the error message is simply the name of what’s undefined. Let’s see a practical example of this behaviour.

q)thisFunctionDoesntExist[4;5]
‘thisFunctionDoesntExist
  [0]  thisFunctionDoesntExist[4;5]
       ^
q)2+thisVariableDoesntExist
‘thisVariableDoesntExist
  [0]  2+thisVariableDoesntExist
         ^

Understanding this pattern makes it much easier to track down where an error comes from and resolve it quickly.Understanding this pattern makes it much easier to track down where an error comes from and resolve it quickly. The key takeaway: if you’re going to fail, fail fast!

wsfull

The next error is one you never want to see. It doesn’t just signal a (potential) flaw in your code, it means you’ve hit a hardware limit: you’ve run out of memory. At this point, the process will crash, you’ll stop intake of real-time data, and you may lose data entirely. The example below illustrates how this can happen, but do not run it unless you want to intentionally crash your machine, and never, ever in production.

q)999999999999#0

As you can see below, once my MacBook began running out of memory, it slowly started suspending processes one by one.

It put up a brave fight, but eventually tapped out, shut down, and restarted. For me, it was just an unexpected coffee break before getting back to blogging. But in production? A crash won’t just wake you up like caffeine, it will shake you harder than an espresso shot to the system.

-w abort

Instead of letting your KDB/Q process eat up all available RAM like it’s at an all-you-can-eat buffet, you can put it on a strict diet. Setting a memory limit ensures that the process dies when it overeats, not your entire server. Once this limit is hit, the process terminates immediately, but at least the machine stays alive and the rest of your system keeps running. Use the -w argument to specify how many MB it’s allowed to consume. For example, -w 10 restricts it to 10MB. If you leave -w at its default value of 0, the process gets unlimited access to your machine’s memory... and we all know how that story ends.

alexanderunterrainer@Mac:~|⇒  qx -w 10
KDB-X 0.1.1 2025.09.29 Copyright (C) 1993-2025 Kx Systems
m64/ 8()core 24576MB alexanderunterrainer mac 192.168.1.177 EXPIRE 2025.12.31 COMMUNITY #5029523

q)999999999#0
‘-w abort

u-fail

The u-fail error is hands-down my favourite. Not because I bump into it often, in fact, I rarely do, but because of its name. It’s hard not to smile at an error that casually tells you: “you fail.” I like to think it’s KDB/Q’s subtle tribute to every qbie discovering the language and making their first glorious mistakes.

You’ll run into u-fail when trying to apply the unique attribute to a list that... well... isn’t unique. Likewise, you’ll trigger it by applying the parted attribute to a list that isn’t actually parted. It’s KDB/Q’s way of gently whispering, “nice try, but your attributes need some work.”

A close cousin to this is the s-fail error, thrown when you attempt to mark a list as sorted when it’s not actually sorted in ascending order. If you’ve been blessed with both u-fail and s-fail, it may be time for a quick refresher on attributes.

Luckily, DefconQ comes to rescue. Check out my deep dive on attributes here. Happy debugging!

q)`u#2 2 3 4
‘u-fail
  [0]  `u#2 2 3 4
         ^
q)`u#2 2 3 3 2 2 4 3 3 3
‘u-fail
  [0]  `u#2 2 3 3 2 2 4 3 3 3
         ^
q)`u#2 3 4
`u#2 3 4
q)`p#2 2 2 3 3 3 4 2 2 3 3
‘u-fail
  [0]  `p#2 2 2 3 3 3 4 2 2 3 3
         ^
q)`p#2 2 2 3 3 3 4
`p#2 2 2 3 3 3 4
q)`s#5 4 3 2 1
‘s-fail
  [0]  `s#5 4 3 2 1
         ^
q)`s#1 2 3 4
`s#1 2 3 4

BONUS: nyi

The nyi error is another special one, it stands for “Not Yet Implemented.” You’ll come across it whenever you attempt an operation that could make sense logically, but simply hasn’t been built into the language (at least not yet). In other words, KDB/Q is telling you: “Interesting idea... maybe one day, but not today.” It’s a little reminder that the language continues to evolve, and some ideas still need time to mature.

Catch Me If You Can: Error Traps that Secure Your KDB/Q Apps

Up to this point, we’ve explored the most common KDB/Q errors, what triggers them, and how to prevent them. But no matter how careful you are, unexpected scenarios still happen, and when they do, you don’t want your entire application grinding to a halt.

To protect your system from crashing in the face of an error, you should wrap any risky or unpredictable code in an error trap, using the (heavily overloaded) trap . or trap at @ operator. The table below shows the syntax and how to use it effectively:

Function nameFunction syntaxFunction semanticsTrap.[g;gx;e]Try g . gx ; catch with eTrap at@[f;fx;e]Try f@fx ; catch with e

Examples:

q)neg `a
‘type
  [0]  neg `a
       ^
q)@[neg;`a;`error]
`error
q).[+;”ab”;`ouch]
`ouch

For a deeper exploration of trap, you can read the full walkthrough here. The extended trap at functionality is also covered hereand I’ll leave that one for you to experiment with. Enjoy the challenge!

To wrap things up, KDB/Q errors really aren’t as frightening as they first appear. Now that you understand what they mean and how to interpret them, there’s no need to panic the next time one pops up on your screen. Consider them helpful signposts rather than scary roadblocks.

Q: Array Programming, Supercharged

If you’re into sports cars like I am, think of Q as the American muscle car of programming, supercharged, powerful, and leaving everything else in the dust on the quarter mile. But before we get Too Fast and Too Furious, let’s hit the brakes and talk code. Q (or KDB/Q) isn’t just another database, it is first and foremost an array programming language. To put it precisely, as Nick Psaris, author of Q Tips: Fast, Scalable, and Maintainable KDB, defines it:

“Q is an interpreted, dynamic, event-driven, functional array programming language.”

So what does that actually mean? Let’s take a closer look. We start with the most important point: Q is an array programming language. In simple terms, it’s designed to seamlessly extend operations on single values (scalars, or Atoms, how we call them in Q) to entire vectors, matrices, and higher-dimensional arrays. Other examples of array programming languages include APL, K, J, Fortran, MATLAB, Analytica, Octave, R, and of course, our all-time favorite Q.

// Addition of Atoms
q)2+2
4
// Addition of two vectors
q)1 2 3+4 5 6
5 7 9
// Addition of two matrices
q)(1 2 3;4 5 6;7 8 9)+(10 20 30;40 50 60;70 80 90)
11 22 33
44 55 66
77 88 99

INFO

If you’re curious about how K, KDB, and Q differ, check out my blog post on the history of KDB/Q here

When Functions Behave… Mostly

The next defining feature of Q is that it’s a (semi-)functional programming language. In practice, this means functions are treated as first-class citizens, you can pass them as arguments, return them as results, assign them to variables, and even store them inside data structures.

// Passing a function as a variable
q)f:{x+y}
q)g:{[f;x;y] f[x;y] }
q)g[f;5;6]
11
// storing functions in a dictionary
q)show dict:`add`sub`mult!({x+y};{x-y};{x*y})
add | {x+y}
sub | {x-y}
mult| {x*y}
q)dict[`add;3;4]
7
q)dict[`sub;10;4]
6
q)dict[`mult;10;4]
40

Q is considered semi-functional because its functions can produce side effects, they’re able to modify the state of the system. In contrast, pure functional programming languages do not allow any side effects, ensuring that functions are entirely self-contained and deterministic.

q)changeTheWorld:{[param] `varA set param}
q)changeTheWorld “New World Order”
`varA
q)varA
“New World Order”

Ain’t Nobody Got Time for Compilation

Q is an interpreted language, which means there’s no separate compilation step, everything is compiled into bytecode at runtime. Function definitions are compiled when the process starts and resolved lazily. The only catch: they must be syntactically valid at startup, but their order doesn’t matter.

alexanderunterrainer@Mac:~|⇒  cat def.q
g:{f[x;y]};
f:{x+y};

g[3;4]

alexanderunterrainer@Mac:~|⇒  qx def.q
KDB-X 0.1.1 2025.09.29 Copyright (C) 1993-2025 Kx Systems
m64/ 8()core 24576MB alexanderunterrainer mac 192.168.1.177 EXPIRE 2025.12.31 COMMUNITY #5029523
7

Dynamic by Design

Q is dynamic in every sense, both functions and variable types can change at runtime. This gives you tremendous flexibility, but it also means things can go sideways if those definitions shift unexpectedly. As Uncle Ben wisely put it: with great power comes great responsibility.

q)f:{x+y}
q)a:3
q)b:5
q)f[a;b]
8
q)f:{show x}
q)a:”With great power comes great responsibility”
q)f a
“With great power comes great responsibility”

Turning Events into Action, the Q-Style

The ease of interprocess communication in Q makes it an ideal fit for event-driven architectures, so much so that it’s often the default and recommended approach. With built-in support for both synchronous and asynchronous messaging, IPC in Q is refreshingly simple. That said, if you prefer timer-based designs, Q won’t stand in your way either.

The Toolkit Behind the Fastest Systems You Know: What You Can Build with KDB/Q

Now that we’ve covered what KDB/Q actually is, the next question is: what can you build with it? In short, almost anything. Well, anything on the backend, to be precise. Born on Wall Street, KDB/Q has long been the weapon of choice for financial systems, powering some of the largest and longest-running platforms since the late ’80s.

But its strengths go far beyond finance. Any domain that generates massive volumes of data and requires rapid processing is a perfect match for KDB/Q. Think smart devices: your Fitbit, Apple Watch, Garmin, whatever you use is capturing millions of data points every single day. What better technology to stream, store, analyse, and make sense of that data than KDB/Q? So the next time you’re faced with a challenge involving mountains of data that need to be processed at lightning speed, give KDB/Q a serious try. And if you need a place to start learning, check out my study plan here.

To wrap things up, I want to leave you with a quote from Alan Perlis. With that, happy coding.

Over the years, I’ve received countless messages about:

• 💡 How to start learning KDB/Q

• 🚀 Career paths and progression in the KDB/Q ecosystem

• 📚 Best learning resources and study plans

• ⚙️ Real-world applications and architecture insights

So instead of replying to everyone one by one, let’s bring it all together. The AMA will run for 24 hours, and you will be able to ask your questions completely anonymously.

Looking forward to hearing from you all!

You can find the link here: Reddit DefconQ AMA

Quant Strats took place right in the heart of the City of London, the historical financial district, just a short walk from Bank Station. As you stroll down Threadneedle Street, passing the historic Bank of England on your left and the Royal Exchange, one of the world’s oldest trading hubs, first opened in 1571, you can almost feel the spirit of global finance in the air. Voltaire captured this essence perfectly in Letters on the English (1733), writing:

“Take a view of the Royal Exchange in London, a place more venerable than many courts of justice, where the representatives of all nations meet for the benefit of mankind. There the Jew, the Mahometan, and the Christian transact together, as though they all professed the same religion, and give the name of infidel to none but bankrupts.”

A fitting reflection on the city that still serves as one of the world’s great crossroads of commerce and ideas.

But on this particular Tuesday morning, the usual buzz of the City of London was matched, no surpassed, by the energy inside the halls of Quant Strats Europe 2025. With over 500 attendees representing some of the world’s leading buy- and sell-side institutions, the atmosphere was electric, a room full of some of the brightest minds in quantitative finance, all eager to connect, learn, and push the boundaries of the field.

Setting the Stage & Big Themes

After the welcoming remarks of the host, the conference opened with a macroeconomic outlook from Mike Bell, setting the tone for the conversations around risk, regime shifts, and policy tailwinds. From there, participants dived into panels and sessions on topics such as:

• Geopolitical shocks & real-time market signalling

• Alpha warfare and how AI is reshaping quant strategy

• Machine learning interpretability in finance

• Architecting where models live (cloud vs on-prem vs hybrid)

• Extracting latent signals from unstructured data (LLMs, NLPs)

• Portfolio construction innovations (multi-asset, private & public)

• Execution optimization, alternative data, and data pipelines

The agenda was ambitious, bridging theory and practice, and bringing deep technical sessions alongside broader strategy talks.

A few standout features:

• Quickfire “Insight Unlocked” sessions: five-minute lightning talks delivering tight, actionable ideas

• Masterclass / workshop tracks on specialized topics

• Expo demo drives and networking breaks built in to let practitioners explore tools and connect

• An app that truly elevates the conference experience: Guiding you seamlessly through the event with a host of powerful features. I already raved about the Quant Strats Event App last year (you can read my full review here), and it continues to impress.

• A spotlight on data infrastructure, deployment architecture, and real-time systems rather than just pure model research

Overall, the focus shifted from signal generation to signal execution, model robustness, deployment trade-offs, and system architecture, exactly where the rubber meets the road in quant finance.

My Highlight: Masterclass Data to Deployment, End-to-End Coding for Quant Analysts

For me, the highlight of Quant Strats was leading the 90-minute masterclass titled Data to Deployment: End-to-End Coding for Quant Analysts. In that session, I guided attendees through a full pipeline:

1. Introduction to KDB/Q: History, evolution and important concepts of Wall Street’s most powerful array programming language

2. Ingest & Stream Data - live integration from historical end-of-day (EOD) files as well as real time data from Databento, a market data provider, giving you 125$ credit when signing up.

3. Data Handling & Cleaning - parsing, transformations, and schema logic

4. Signal Generation & Strategy Logic - crafting a simple rading signal, using simple moving averages (SMA), in KDB/Q

5. Optimization, Deployment & Execution - from prototype to a deployable system

6. Leveraging existing libraries such as ExeQution Analytics, to focus on what’s important: turning data into alpha

7. Visualization & Dashboarding - showing results using Pulse, the framework that even makes me enjoy UI work

What made the Masterclass special is the balance between hands-on coding and strategic architecture. Many participants told me afterward that seeing how everything connected: data ➡️ model ➡️ execution ➡️ monitoring, helped solidify how to build a real quant system, not just isolated research code.

One recurring theme during the masterclass (and across the event) was the significance of performance, interpretability, and model flexibility. Building for speed is important, but so is building for maintainability and resilience over time.

Memorable Panels & Moments

Some highlights that stuck with me:

• The Geopolitical Shock Panel asked: How do you separate noise vs signal in global events? and What data sets help decode policy shocks in real time? It’s a reminder that markets don’t operate in a vacuum and quant models must ingest events.

• Alpha Warfare: Winning the AI Arms Race explored where quant teams get edge when everyone has access to advanced models and compute. Not just model selection, but infrastructure, data, pipelines, and latency matter.

• Model interpretability & explainable AI sessions tackled guardrails: how to deploy ML in finance when regulators or stakeholders demand transparency.

• The discussion on where models should live, cloud vs on-prem, tackled real tradeoffs: latency, compliance, cost, and hybrid flexibility.

• And the closing keynote From Pit Wall to Portfolio, by Neil Martin (ex-Formula 1 strategist) drew powerful parallels between motorsport analytics and quant finance: split-second decisions, feedback loops, and optimizing under pressure.

Reflections & Takeaways

• Systems matter as much as models: Many talks framed quant work not just as statistical or ML problems, but as engineering and software problems.

• Deployment & infrastructure tradeoffs shine: It’s one thing to produce a signal in Jupyter or research code; it’s another to embed it into a production pipeline.

• Data quality, latency & pipeline robustness are differentiators: Several speakers emphasized that alpha can be lost more in poor data ingestion or messy pipelines than in suboptimal modeling.

• Hybrid architectures are the future: The middle ground, balancing on-prem control with cloud scalability, is where many quant firms will land.

• AI & LLMs are maturing but need guardrails: The promise is high, but interpretability, drift, and risk control cannot be ignored.

Conclusion

From a personal perspective, seeing the breadth of talent in one room, researchers, engineers, portfolio experts, platform builders, reaffirms that the future of quant is multidisciplinary. Model authors must understand pipelines; engineers must think in signal space. And KDB/Q continues to prove why it’s the gold standard for quantitative finance. Its unmatched ability to process, analyse, and store massive volumes of real-time and historical data makes it the ultimate choice for building modern, scalable, and ultra-low-latency systems. Whether you’re streaming tick data, engineering features for machine learning, or running complex backtests, KDB/Q empowers quants to focus on insights rather than infrastructure. Simply put, when it comes to speed, scalability, and precision, nothing comes close.

I walked away energized, with new connections, fresh ideas, and a clearer view of where quant finance is heading.

If you’re working with KDB/Q, or planning to, and need expert guidance on architecture reviews, system design, or simply want to deepen your understanding of the language, don’t hesitate to reach out. I’m always happy to help fellow quants and developers get the most out of KDB/Q and bring their data infrastructure to the next level.

Quant Strats Europe Photo Dump

What began as a small personal hobby, mainly to improve my own KDB/Q skills, has since evolved into a full-fledged side project and growing community. I never expected the blog to gain such traction, nor could I have imagined the incredible feedback and connections it would bring.

I’m truly grateful to everyone who has supported DefconQ along the way, whether you’ve subscribed to the newsletter, liked or reshared a LinkedIn post, attended a community meet-up, or simply read an article. Your encouragement and engagement mean the world to me.

As is now tradition, I’d love to share some of the highlights from the past year. Let’s take a look back and celebrate what we’ve built together. And if you’re wondering who won the FREE PREMIUM ticketto Quant Strats Europe 2025, where I’ll be hosting a KDB/Q Masterclass, keep reading. The winner is revealed in this blog post!

DefconQ's Year in Review

TL;DR

The summary below highlights all DefconQ activities from September 4th, 2024 through September 4th, 2025.

q)show TLDR:(`$("Posts";"Reach";"Followers";"Meetups & Conferences";"Livestreams & Podcasts";"Collaborations";"Awards";"Newsletters";"Swag Store";"Bench Press PR";))!31 349686 655 1 3 0 1 2 1 120
Posts                 | 31
Reach                 | 349686
Followers             | 655
Meetups & Conferences | 1
Livestreams & Podcasts| 3
Collaborations        | 0
Awards                | 1
Newsletters	      | 2
Swag Store	      | 1
Bench Press PR	      | 120	

Blog Posts

Just like last year, I want to start my recap with the metric that makes me proudest: the number of blog posts published. At its heart, DefconQ has always been about sharing knowledge and helping others learn KDB/Q. Over the past 12 months, I published 31 blog posts, a steady increase from the 26 articles in year one. From deep dives into key KDB/Q concepts and hands-on tutorials, to curated resources, podcasts, community meet-ups, industry insights and news, the range of content has been both fun and rewarding to create. On average, this equates to one post every two weeks, a rhythm that shows consistency, but also reflects sustainable growth. Thank you to everyone who has read, shared, or joined along the way. Below are some highlights of the traffic and reach DefconQ achieved this year!

While DefconQ typically sees a steady flow of a few hundred visitors per day, the chart shows two significant spikes toward the end. Both are easy to trace back: the first coincides with my blog post on KDB-X, the newly launched KDB/Q community edition, while the second follows shortly after my third appearance on the ArrayCast as a guest panelist. The first spike from my blog post on KDB-X (link here), drew nearly 2000 views on its own and pushed total daily traffic past 3000. The ArrayCast recap (link here) generated almost the same number of views, though overall traffic for that day leveled out just above 2000.

Subscribe now

Even more impressive is the breakdown of user activity: over the past year, total visits grew from 30k to 69k, more than a 100% increase. Active users also doubled, climbing from just over 5k to more than 10k. A clear sign that developers see real value in the content I’m sharing.

Reach

Another key metric I track is reach, because what’s the point of creating content if it doesn’t reach people, right? Since my mission is to share KDB/Q knowledge and make this powerful array programming language more accessible, it’s important to connect with as many readers as possible. As with last year, the statistics below reflect a combination of my blog readership, my personal LinkedIn presence, and, new this year, my Substack newsletter.

One notable takeaway from the user-by-country breakdown is that the United States has now overtaken the United Kingdom, with the majority of users coming from the US. Another shift worth highlighting is Canada moving into fourth place, surpassing both Hong Kong and Ireland. While Ireland dropped out of the top ranks, Germany and Singapore made their way into the leaderboard.

On the bright side, London held onto its spot as the city with the most active users.

Much like last year, most of my blog post announcements went out via my personal LinkedIn profile, so it makes sense to look at those numbers too. And honestly, they blew me away. Impressions nearly doubled, climbing from just under 200k to 350k.

My top-performing posts reached 17.5k and 15k impressions each, and overall, more than 60k people engaged with my content. A milestone I’m truly proud of.

Followers

Our DefconQ LinkedIn community experienced a strong growth over the past year, reaching 655 followers, a 50% increase compared to the first year. This is a fantastic milestone considering most of the conversations still happen through my personal profile. On top of that, my DefconQ Substack newsletter, which I only launched a year ago, already has 173 subscribers! If you haven’t joined yet, head over and subscribe, it’s free, and it guarantees you’ll never miss a blog post (they’ll come straight to your inbox!).

Meetup and Conferences

While I didn’t speak at any major conferences this year, I had the pleasure of hosting something even more special: the very first KDB/Q Community Happy Hour in London! Nearly 60 developers joined at Flight Club in Shoreditch for an agenda-free evening full of darts, laughter, networking with both new and familiar faces and brilliant conversations. We stayed until the pub finally kicked us out. The energy and turnout were so encouraging that for DefconQ’s second anniversary, we’ll be hosting not one but TWO Happy Hours, one in Belfast and another in New York. You can read more about them here.

September 10th, 2025: Brews and Qs Belfast Edition - KX and DefconQ Happy Hour!

September 17th, 2025: New York, New York - DefconQ Goes International

Following the anniversary, I’ll be back on the conference stage. On September 19th, I’ll be in New York at the Cornell Financial Engineering Manhattan 2025 Future of Finance & Ai Conference, and in October, I’ll present a KDB/Q Masterclass at Quant Strats Europe in London. Tickets for both conferences are still available and you can get a 20% Discount for Quant Strats Europe if you book via my link below.

Tickets:

Cornell Financial Engineering Manhattan 2025 Future of Finance & Ai Conference

Quant Strats Europe 2025

Livestreams and Podcasts

2025 was a podcast-heavy year for me, I had the opportunity to appear on three podcast episodesin total: twice as a guest and once as a (guest) panelist!

Top 10% Podcast

Back in March, I had a great chat with Craig Whiting, MD of RLS Search, on his Top 10% Podcast. We explored the world of Tech in Finance, covering everything from banking graduate schemes and nepotism in investment banks to salary progression, promotions, and more. If you’re considering a career move, navigating the job market, or just starting out, this episode is well worth a listen. Link here

The ArrayCast - Learning q with Alexander Unterrainer

Next up was a real highlight and milestone for me, my first solo appearance on the ArrayCast, the must-listen podcast for array programming enthusiasts. While I had previously joined a group recording during Iverson College 2024, this was the first time I had a full episode to myself, which felt special. Given that The ArrayCast has featured some of the most respected names in the KDB/Q community, it was an honor to add my voice to that list. If you’re passionate about KDB/Q or array programming in general, this one's for you. Link here

The ArrayCast - KDB-X

My second appearance on The ArrayCast this year was as a (guest) panelist, stepping in for Stephen Taylor due to a scheduling clash in Stephen's diary. In episode 112, we welcomed Ashok Reddy, CEO of KX, who shared insights into the newly released KDB-X Community Edition. Together, we explored the motivations behind this release, its current state, and what’s next on the roadmap. Whether you’re a long-time KDB/Q practitioner or just getting started with this powerful array programming language, this episode is well worth a listen. Link here

Collaborations

Not every collaboration I explored this year materialised, but that’s part of the journey. What I did gain were incredible new connections, inspiring conversations, and friendships that I know will spark collaborations down the road. The seeds are planted, and I’m excited for what’s to come!

Certificates and Awards

Quant Strats Advisory Board

Technically, this isn’t an award or certificate, but to me, it feels like one. Earlier this year, I was honored to be nominated to the Advisory Board of Quant Strats, the leading conference for quantitative professionals and finance practitioners. If you’re in finance, this is truly a must-attend event, and I can’t recommend it enough. Curious what to expect? I attended last year and shared a full recap here. Tickets for this year are still available, and with my link below, you can enjoy a 20% discount.

Quant Strats Europe 2025

Reader's Feedback

As I shared last year, the most rewarding achievement for me is the overwhelmingly positive feedback from my readers. There’s nothing more fulfilling than when someone I’ve never met reaches out to thank me, saying my blog posts have been helpful or inspiring. More than once, I’ve had fellow developers recognize me at meetups and strike up a conversation about KDB/Q, which I’m always delighted to do. So if you ever spot me at a meetup, or even just walking around London, don’t hesitate to come and say hi. That kind of feedback is priceless, and it remains one of my strongest motivations to keep writing and sharing.

Failures

Being my own toughest critic, I also acknowledge my failures. Last year, I promised an AMA (Ask Me Anything) session, but unfortunately, it didn’t happen. I’m making it up to you: I’ll be hosting the AMA this October, once all the meet-ups and conferences are wrapped up. During the session, I’ll do my best to answer your questions about KDB/Q, careers, or anything else you’d like to ask. It will take place on Reddit, and questions can be submitted completely anonymously. More details will be shared soon.

Quant Strats Euoper 2025 Giveaway Winner

Now, for the highlight of this wrap-up: it’s time to announce the winner of the DefconQ Giveaway for a free premium ticket to Quant Strats Europe 2025. And of course, how better to pick a winner than with KDB/Q, after all, we are KDB/Q developers! So, without further ado, here we go.

What the Future holds

After such a successful year, the future of DefconQ has never looked brighter. I already have a few projects and ideas in the works, along with some bigger plans I’ve been developing behind the scenes. Everything will be revealed in due time, so for now, stay tuned, subscribe to my newsletter, and keep up with everything happening at DefconQ.

Until then, happy coding!